:W&Bá¾ØjÏ M osy2axttª; 9 /0.29.0+7872fd9d9f86ca5c50617e19cb168122b7f959480.70.0~Ưú‚" 7t0zven2hfnf osy2axttŠÖ osy2axttjuju11-uc-san-diegoSPAtop"š _wandb‚{}  batch_size‚512  dataset_limit‚0.2  dropout‚0.22912906376672712 epochs‚10  hidden_dim‚256 ' learning_rate‚0.0011373348343904125  num_attention_heads‚8  num_branch_encoder_layers‚3  num_embedding_layers‚14  num_encoder_layers‚2Bjjtttt_blocks_osy2axttbs2s2ev0zj*jj-spatop-sweepagent-tttt-pairwise-1-8pwgvŠ мèÔ¸üÿš3 (H¨¸hp"3.12.10*0.29.0j linux-x86_64ì&è{7 osy2axtt&  _runtime‚0  _wandb.runtime‚0Êè»"iÒã -Linux-6.8.0-137-generic-x86_64-with-glibc2.36CPython 3.12.10 ϼèÔˆ«ùp*-of*=/data/spatop/options_files/sweep_tttt_jj/config_osy2axtt.json*-l* /scratch/logs*-n*jjtttt_blocks_osy2axtt2-m spanet.trainRjacksonjulian2014@gmail.comZ /scratch/logsb*jj-spatop-sweepagent-tttt-pairwise-1-8pwgvr/opt/conda/bin/pythonº 94z8yq3tr8vlimvqyuw61o1k7kztxw8oCkc7ÉÒÀ€ˆ€’NVIDIA A100-SXM4-80GB˜¢ /€€€øáo€ ƒ’…]ª€àäêÞÂS NVIDIA A100-SXM4-80GB€€€€À€6"Ampere*(GPU-c1fdb213-5a16-84e3-13b4-cf29bc35dbf8Ê13.2º 94z8yq3tr8vlimvqyuw61o1k7kztxw8oµª½Ç2  requirements.txt|åW osy2axttZF (H¨¸ (H¨¸ (hpð"3.12.10*0.29.0j linux-x86_64—?ÂãY osy2axttZH (H¨¸ (H¨¸ (8hpð"3.12.10*0.29.0j linux-x86_64‰Ï *  osy2axttb trainer/global_step2ÌÊÜV\  osy2axttZK (H¨¸ (H¨¸(8hpð"3.12.10*0.29.0j linux-x86_649}Û/  osy2axttb*"trainer/global_step2K8 §J  osy2axttj9 ѼèÔÐÐù*LOCAL_RANK: 0 - CUDA_VISIBLE_DEVICES: [0] w¨Zp“  osy2axttj  ѼèÔ°éÁÉî â”â”â”â”┳â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”┳â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”┳â”â”â”â”â”â”â”â”┳â”â”â”â”â”â”â”┳â”â”â”â”â”â”â”┓ ┃ ┃ Name ┃ Type ┃ Params ┃ Mode ┃ FLOPs ┃ ┡â”â”â”╇â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”╇â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”╇â”â”â”â”â”â”â”â”╇â”â”â”â”â”â”â”╇â”â”â”â”â”â”â”┩ │ 0 │ embedding │ MultiInputVectorEmbed… │ 3.4 M │ train │ 0 │ │ 1 │ encoder │ JetEncoderWithPairwise │ 2.6 M │ train │ 0 │ │ 2 │ branch_decoders │ ModuleList │ 151 M │ train │ 0 │ │ 3 │ regression_decoder │ RegressionDecoder │ 0 │ train │ 0 │ │ 4 │ classification_decoder │ ClassificationDecoder │ 0 │ train │ 0 │ │ │ other params │ n/a │ 48 │ n/a │ n/a │ └───┴────────────────────────┴────────────────────────┴────────┴───────┴───────┘ rXÂ\ osy2axttjÊ ѼèԨκʷTrainable params: 157 M Non-trainable params: 86 Total params: 157 M Total estimated model params size (MB): 630.533 Modules in train mode: 4316 Modules in eval mode: 0 Total FLOPs: 0 +ÈK)q osy2axttj` ѼèÔ¨÷ÖÊPwandb: WARNING Config item 'hidden_dim' was locked by 'sweep' (ignored update). 9£gÖ{ osy2axttjj ѼèÔàðàÊZwandb: WARNING Config item 'num_embedding_layers' was locked by 'sweep' (ignored update). $nP†y osy2axttjh ѼèÔ°åéÊXwandb: WARNING Config item 'num_encoder_layers' was locked by 'sweep' (ignored update). ‰È¥Ñ€ osy2axttjo ѼèÔØ“òÊ_wandb: WARNING Config item 'num_branch_encoder_layers' was locked by 'sweep' (ignored update). µMvÚz osy2axttji ѼèÔ€ÂúÊYwandb: WARNING Config item 'num_attention_heads' was locked by 'sweep' (ignored update). WHi\t osy2axttjc ѼèÔø‚ËSwandb: WARNING Config item 'dataset_limit' was locked by 'sweep' (ignored update). U ‚3q osy2axttj` ѼèÔè“ŒËPwandb: WARNING Config item 'batch_size' was locked by 'sweep' (ignored update). ç5t osy2axttjc ѼèÔت”ËSwandb: WARNING Config item 'learning_rate' was locked by 'sweep' (ignored update). \Uݽn osy2axttj] ѼèԘќËMwandb: WARNING Config item 'dropout' was locked by 'sweep' (ignored update). âDëYm osy2axttj\ ѼèÔ à¤ËLwandb: WARNING Config item 'epochs' was locked by 'sweep' (ignored update). ÿ¸e÷Ü osy2axtt*Ê  transformer_dim‚256  transformer_dim_scale‚2.0  initial_embedding_dim‚256  position_embedding_dim‚256 ! num_branch_embedding_layers‚3  num_jet_embedding_layers‚0  num_jet_encoder_layers‚2  num_detector_layers‚2  num_regression_layers‚3  num_classification_layers‚3  split_symmetric_attention‚1 ! transformer_activation‚"gelu"  skip_connections‚1 ( "initial_embedding_skip_connections‚1  linear_block_type‚"GRU"  transformer_type‚"Gated"  linear_activation‚"gelu"  normalization‚ "LayerNorm"  masking‚ "Filling" linear_prelu_activation‚true  use_pairwise_interactions‚1  num_pairwise_features‚4  pairwise_embedding_dim‚8  pairwise_input_source‚"all"  pairwise_block_embeddings‚1  pairwise_cross_type‚true  pairwise_feature_set‚""  pairwise_eq2to2‚false  use_moe‚false  num_experts‚8  num_experts_per_tok‚2  moe_loss_scale‚0.01 F event_info_file‚2"/data/spatop/event_files/tttt/tttt_hadronic.yaml" = training_file‚+"/data/spatop/tttt_15M/h5/tttt_training.h5"  validation_file‚""  testing_file‚""  normalize_features‚1  limit_to_num_jets‚0  balance_particles‚0  balance_jets‚0 ! balance_classifications‚false  partial_events‚1  dataset_randomization‚0  train_validation_split‚0.95  limit_val_batches‚1.0  num_dataloader_workers‚0  mask_sequence_vectors‚1  combine_pair_loss‚"min"  optimizer‚"AdamW"  focal_gamma‚0.0  combinatorial_scale‚0.0 # learning_rate_warmup_epochs‚1.0  learning_rate_cycles‚1  assignment_loss_scale‚1.0  detection_loss_scale‚1.0  kl_loss_scale‚0.0  regression_loss_scale‚0.0 ! classification_loss_scale‚0.0 ! use_mass_classification‚false  mass_classes‚[]  balance_losses‚true  l2_penalty‚0.0002  gradient_clip‚10.0 num_gpu‚1  verbose_output‚false  usable_gpus‚""  trial_time‚"" $ trial_output_dir‚"./test_output"FĶë osy2axttjÙ ѼèÔ°Ÿ›ÐÈ/opt/conda/lib/python3.12/site-packages/pytorch_lightning/utilities/_pytree.py:21: `isinstance(treespec, LeafSpec)` is deprecated, use `isinstance(treespec, TreeSpec) and treespec.is_leaf()` instead. Ÿêc®N osy2axttj¼ ѼèÔÀ·­Ð«/opt/conda/lib/python3.12/site-packages/pytorch_lightning/trainer/connectors/data_connector.py:434: The 'val_dataloader' does not have many workers which may be a bottleneck. Consider increasing the value of the `num_workers` argument` to `num_workers=3` in the `DataLoader` to improve performance. ˆ¼†(ë osy2axttjÙ Õ¼èÔÀá×È/opt/conda/lib/python3.12/site-packages/pytorch_lightning/utilities/_pytree.py:21: `isinstance(treespec, LeafSpec)` is deprecated, use `isinstance(treespec, TreeSpec) and treespec.is_leaf()` instead. ŒL‘P osy2axttj¾ Õ¼èÔÀ™™×­/opt/conda/lib/python3.12/site-packages/pytorch_lightning/trainer/connectors/data_connector.py:434: The 'train_dataloader' does not have many workers which may be a bottleneck. Consider increasing the value of the `num_workers` argument` to `num_workers=3` in the `DataLoader` to improve performance. S»“‚(:‰ ß¼èÔ´¯¥° gpu.0.gpu‚73.0 gpu.0.memory‚16+ gpu.0.memoryAllocated‚47.51808166503906* gpu.0.memoryAllocatedBytes‚ 40817721344 gpu.0.temp‚39.0 gpu.0.powerWatts‚119.091' gpu.0.enforcedPowerLimitWatts‚400.0 gpu.0.powerPercent‚29.77275 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0ÒYŒæ‚(:Ü ß¼èÔ‰üˆ¶ disk.nvme0n1.in‚0 disk.dm-0.in‚4 disk.md0p1.out‚ 1.42578125 disk.sdb.out‚ 1.4794921875( proc.memory.availableMB‚ 1301.4765625# proc.memory.rssMB‚ 2967.66015625 disk.sda.in‚0 disk.sda1.out‚0 disk.nvme1n1.out‚ 0.18359375 network.sent‚39680 disk.sda2.in‚0 disk.dm-0.out‚ 137.74609375 disk.sdb1.out‚0 disk.nvme1n1.in‚4 network.recv‚39178 disk.rbd0.in‚0 disk.sdb1.in‚0 disk.sda1.in‚0 disk.md0p1.in‚0 disk.sda2.out‚ 1.4794921875$ memory_percent‚92.05641746520996* proc.memory.percent‚18.113160133361816 disk.sdb2.out‚ 1.4794921875 disk.md0.out‚ 1.42578125 disk.rbd2.out‚0 disk.sdb.in‚0) disk./.usagePercent‚83.24613969935723 disk.sda.out‚ 1.4794921875 disk.nvme0n1.out‚137.5625 proc.cpu.threads‚17% disk./.usageGB‚2977.2851791381836 disk.rbd0.out‚0 disk.sdb2.in‚0 disk.md0.in‚0 disk.rbd2.in‚0 cpu‚25.051313287249442Øëᛜ ‚(:’ î¼èÔ´­® gpu.0.gpu‚93.0 gpu.0.memory‚16+ gpu.0.memoryAllocated‚47.53028869628906* gpu.0.memoryAllocatedBytes‚ 40828207104 gpu.0.temp‚44.0 gpu.0.powerWatts‚237.566' gpu.0.enforcedPowerLimitWatts‚400.0( gpu.0.powerPercent‚59.39149999999999 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0…0‹Ý!‚(:Ó î¼èÔ¯¼þ® cpu‚25.140126056118437 disk.sdb1.in‚0) disk./.usagePercent‚83.23845886111809 disk.sdb.out‚ 3.5234375 disk.dm-0.in‚4 disk.nvme0n1.in‚0 network.sent‚50615 disk.sda1.in‚0 disk.sda1.out‚0 disk.md0.out‚ 3.41015625 disk.rbd2.in‚0 disk.rbd2.out‚0 disk.nvme1n1.in‚4 disk.sdb1.out‚0 disk.md0.in‚0 disk.sdb.in‚0 disk.rbd0.in‚0) proc.memory.availableMB‚ 1299.08203125# proc.memory.rssMB‚ 2967.74609375) proc.memory.percent‚18.11368465423584 disk.sda2.in‚0 disk.nvme1n1.out‚ 0.27734375 disk.sda.in‚0 disk.nvme0n1.out‚ 387.2734375 disk.sdb2.out‚ 3.5234375$ memory_percent‚92.07103252410889 disk.rbd0.out‚0 disk.dm-0.out‚ 387.55078125 network.recv‚57071% disk./.usageGB‚2977.0104751586914 disk.sda2.out‚ 3.5234375 disk.sda.out‚ 3.5234375 disk.md0p1.in‚0 disk.md0p1.out‚ 3.41015625 disk.sdb2.in‚0 proc.cpu.threads‚17mÁ¸'’"‚(:ˆ ý¼èÔ眰 gpu.0.gpu‚31.0 gpu.0.memory‚7+ gpu.0.memoryAllocated‚47.53517150878906* gpu.0.memoryAllocatedBytes‚ 40832401408 gpu.0.temp‚46.0 gpu.0.powerWatts‚315.725' gpu.0.enforcedPowerLimitWatts‚400.0 gpu.0.powerPercent‚78.93125 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0Ï/»Fè#‚(:Þ ý¼èÔ…ŸÒ±% disk./.usageGB‚2977.0102043151855 disk.md0p1.out‚ 4.91015625 disk.sda2.out‚ 5.08740234375 disk.nvme1n1.in‚4 disk.rbd0.out‚0 disk.sda1.in‚0 disk.rbd2.out‚0 disk.sda.in‚0 disk.sdb1.out‚0$ memory_percent‚92.06409454345703 cpu‚25.194096975542273 disk.md0p1.in‚0! disk.nvme0n1.out‚ 491.33203125 disk.sda1.out‚0# proc.memory.rssMB‚ 2967.77734375) proc.memory.percent‚18.11387538909912 proc.cpu.threads‚17 disk.sdb2.in‚0& proc.memory.availableMB‚ 1300.21875( disk./.usagePercent‚83.2384512882203 disk.md0.in‚0 disk.sda2.in‚0 disk.nvme1n1.out‚ 0.37109375 disk.md0.out‚ 4.91015625 disk.sdb.out‚ 5.08740234375 disk.sda.out‚ 5.08740234375 disk.dm-0.in‚4 disk.sdb1.in‚0 disk.sdb.in‚0 disk.sdb2.out‚ 5.08740234375 network.sent‚63941 network.recv‚76216 disk.rbd2.in‚0 disk.rbd0.in‚0 disk.nvme0n1.in‚0 disk.dm-0.out‚ 491.703125n+$‚(:’ Œ½èÔ…Ìý¯ gpu.0.gpu‚98.0 gpu.0.memory‚17+ gpu.0.memoryAllocated‚47.53761291503906* gpu.0.memoryAllocatedBytes‚ 40834498560 gpu.0.temp‚44.0 gpu.0.powerWatts‚224.527' gpu.0.enforcedPowerLimitWatts‚400.0( gpu.0.powerPercent‚56.13174999999999 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0^,öÇó%‚(:é Œ½èÔé¡î° disk.rbd0.out‚0 cpu‚25.22516676751251 disk.sda.out‚ 6.68505859375 disk.dm-0.out‚ 554.55859375 disk.md0.in‚0 disk.sdb.out‚ 6.68505859375 disk.nvme1n1.in‚4$ memory_percent‚92.07601547241211 disk.sdb.in‚0 disk.sda2.in‚0 disk.sda2.out‚ 6.68505859375# disk./.usageGB‚2976.96626663208 disk.sda.in‚0! disk.nvme0n1.out‚ 553.58203125 disk.sdb1.out‚0 disk.sda1.in‚0) disk./.usagePercent‚83.23722277221133 disk.rbd0.in‚0 disk.sda1.out‚0 disk.md0.out‚6.46875 disk.rbd2.out‚0 disk.nvme1n1.out‚ 0.9765625' proc.memory.availableMB‚ 1298.265625# proc.memory.rssMB‚ 2967.79296875 network.recv‚107007 disk.md0p1.in‚0 disk.md0p1.out‚6.46875 disk.sdb2.out‚ 6.68505859375) proc.memory.percent‚18.11397075653076 proc.cpu.threads‚17 disk.nvme0n1.in‚ 0.2421875 disk.dm-0.in‚ 4.2421875 disk.sdb2.in‚0 network.sent‚83844 disk.sdb1.in‚0 disk.rbd2.in‚0â?Ô\œ&‚(:’ ›½èÔÈè•° gpu.0.gpu‚73.0 gpu.0.memory‚14+ gpu.0.memoryAllocated‚47.54249572753906* gpu.0.memoryAllocatedBytes‚ 40838692864 gpu.0.temp‚45.0 gpu.0.powerWatts‚208.776' gpu.0.enforcedPowerLimitWatts‚400.0( gpu.0.powerPercent‚52.19400000000001 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚08'$Þ'‚(:Ô ›½èÔÔα disk.rbd2.in‚0 disk.rbd0.out‚0 disk.sdb1.in‚0 disk.sdb1.out‚0 disk.sdb.out‚ 8.71923828125' proc.memory.availableMB‚ 1298.359375 proc.memory.rssMB‚ 2967.8125* proc.memory.percent‚18.114089965820312 disk.md0p1.out‚ 8.44140625 disk.sdb.in‚0 disk.nvme1n1.out‚73 disk.sdb2.out‚ 8.71923828125$ memory_percent‚92.07544326782227$ disk./.usageGB‚2977.038562774658 disk.sda2.out‚ 8.71923828125 disk.sda.in‚0 disk.sda1.in‚0 disk.dm-0.in‚ 4.2421875 disk.sdb2.in‚0 disk.md0.out‚ 8.44140625 proc.cpu.threads‚17 network.sent‚92644( disk./.usagePercent‚83.2392442026161 disk.md0p1.in‚0 disk.rbd0.in‚0 disk.sda1.out‚0 disk.rbd2.out‚0 disk.sda.out‚ 8.71923828125 disk.nvme0n1.in‚ 0.2421875 disk.md0.in‚0 cpu‚25.248578885531398 disk.sda2.in‚0 disk.nvme1n1.in‚4 disk.nvme0n1.out‚676 disk.dm-0.out‚749 network.recv‚111846üKÑ“(‚(:‰ ª½èÔ©¼«® gpu.0.gpu‚97.0 gpu.0.memory‚17+ gpu.0.memoryAllocated‚47.54493713378906* gpu.0.memoryAllocatedBytes‚ 40840790016 gpu.0.temp‚44.0 gpu.0.powerWatts‚223.691' gpu.0.enforcedPowerLimitWatts‚400.0 gpu.0.powerPercent‚55.92275 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0žó6þ)‚(:ô ª½èÔ»¢À± disk.sdb2.out‚10.25634765625 disk.md0.in‚0 disk.md0.out‚ 9.91796875 disk.sda.in‚0 disk.nvme0n1.in‚ 0.2421875 disk.nvme0n1.out‚ 737.3984375 disk.dm-0.in‚ 4.2421875 disk.sda1.in‚0 disk.sda2.out‚10.25634765625 disk.nvme1n1.out‚ 126.9765625$ memory_percent‚92.07770824432373 disk.nvme1n1.in‚4) disk./.usagePercent‚83.24098927558214 disk.rbd2.out‚0* proc.memory.percent‚18.114161491394043 cpu‚25.255359486236564 disk.md0p1.in‚0 disk.sdb.in‚0 disk.sdb.out‚10.25634765625 disk.sda2.in‚0 disk.sda.out‚10.25634765625# proc.memory.rssMB‚ 2967.82421875 network.recv‚117221 disk.sdb2.in‚0 disk.sda1.out‚0 network.sent‚99531$ disk./.usageGB‚2977.100975036621 disk.sdb1.out‚0 disk.md0p1.out‚ 9.91796875) proc.memory.availableMB‚ 1297.98828125 proc.cpu.threads‚17 disk.rbd2.in‚0 disk.rbd0.in‚0 disk.rbd0.out‚0 disk.dm-0.out‚864.375 disk.sdb1.in‚0Ãìö3*b/ lr-AdamW/pg2"trainer/global_step2:JPhø[3+b/ lr-AdamW/pg1"trainer/global_step2:JPTŸ-,) _wandbruntime‚91  _runtime‚91îtA2À-» % _timestamp‚1.7884852928525774e+09 _runtime‚ 91.954299258 _step‚0 & lr-AdamW/pg1‚9.226722994226856e-05 & lr-AdamW/pg2‚9.226722994226856e-05 trainer/global_step‚49ï^ÌÂ.½ _runtime‚ 91.954299258 _step‚0 & lr-AdamW/pg1‚9.226722994226856e-05 & lr-AdamW/pg2‚9.226722994226856e-05 trainer/global_step‚49 % _timestamp‚1.7884852928525774e+09Ê!že/ba >DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak"trainer/global_step2:JP”aÿl0bh EDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed"trainer/global_step2:JP¹7¦Un1bj GDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed"trainer/global_step2:JPÙQXm2bi FDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak"trainer/global_step2:JP«Ä¢v3br ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current"trainer/global_step2:JPs÷P‘W4bS 0DeviceStatsMonitor.on_train_batch_start/num_ooms"trainer/global_step2:JP'œo5bk HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed"trainer/global_step2:JP¿nùŸg6bc @DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated"trainer/global_step2:JPk–gc7b_ DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak"trainer/global_step2:JP²t½Qp=bl IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated"trainer/global_step2:JP‰¹Ìi>be BDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak"trainer/global_step2:JP|œK4n?bj GDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed"trainer/global_step2:JP.N¦¯t@bp MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed"trainer/global_step2:JPâ5pkAbg DDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated"trainer/global_step2:JP:3-rBbn KDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated"trainer/global_step2:JP&ó§mCbi FDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed"trainer/global_step2:JPä‰8kDbg DDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current"trainer/global_step2:JP(à hEbd ADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak"trainer/global_step2:JP¦Ç®hFbd ADeviceStatsMonitor.on_train_batch_start/oversize_segments.current"trainer/global_step2:JPàÉðUmGbi FDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak"trainer/global_step2:JP#÷QúbHb^ ;DeviceStatsMonitor.on_train_batch_start/segment.all.current"trainer/global_step2:JP³)@^IbZ 7DeviceStatsMonitor.on_train_batch_start/active.all.peak"trainer/global_step2:JPd ÅŽfJbb ?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed"trainer/global_step2:JPÁó…‰gKbc @DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current"trainer/global_step2:JP‚r×NpLbl IDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated"trainer/global_step2:JP…èyÕoMbk HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed"trainer/global_step2:JP!$qÆnNbj GDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed"trainer/global_step2:JPh|1kObg DDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated"trainer/global_step2:JPžh‡_gPbc @DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed"trainer/global_step2:JPÛhQbd ADeviceStatsMonitor.on_train_batch_start/active.large_pool.current"trainer/global_step2:JP•*'TlRbh EDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed"trainer/global_step2:JPFµ¿nSbj GDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak"trainer/global_step2:JP®1œnTbj GDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated"trainer/global_step2:JP5èlUbh EDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated"trainer/global_step2:JPœ{Û™gVbc @DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed"trainer/global_step2:JP›h*ápWbl IDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current"trainer/global_step2:JPŸÙ§wgXbc @DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak"trainer/global_step2:JP+Ê‹jfYbb ?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak"trainer/global_step2:JP/’lZbh EDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current"trainer/global_step2:JPºªŒn[bj GDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak"trainer/global_step2:JP™,j\bf CDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed"trainer/global_step2:JPA¨Õâg]bc @DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak"trainer/global_step2:JP‡Cù a^b] :DeviceStatsMonitor.on_train_batch_start/active.all.current"trainer/global_step2:JP\ô?e_ba >DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed"trainer/global_step2:JP› äo`bk HDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed"trainer/global_step2:JP<·¼Ä_ab[ 8DeviceStatsMonitor.on_train_batch_start/active.all.freed"trainer/global_step2:JPö‹8dbb` =DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak"trainer/global_step2:JPÓ±ˆlcbh EDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated"trainer/global_step2:JPÊ—òUodbk HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed"trainer/global_step2:JP‹µÀxebt QDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated"trainer/global_step2:JPH‹xfbt QDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated"trainer/global_step2:JPvlP„sgbo LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated"trainer/global_step2:JPhÔzrhbn KDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated"trainer/global_step2:JPtÂ8qibm JDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current"trainer/global_step2:JP>«MŒcjb_ DeviceStatsMonitor.on_train_batch_start/allocation.all.current"trainer/global_step2:JPVS3jnbf CDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current"trainer/global_step2:JPU•eoba >DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak"trainer/global_step2:JPçŠÉ»ipbe BDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current"trainer/global_step2:JPUrÓ«_qb[ 8DeviceStatsMonitor.on_train_batch_start/num_device_alloc"trainer/global_step2:JP¢z¬ srbo LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated"trainer/global_step2:JPÂÜ9Êrsbn KDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated"trainer/global_step2:JPi×”Edtb` =DeviceStatsMonitor.on_train_batch_start/segment.all.allocated"trainer/global_step2:JPL€-Þnubj GDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current"trainer/global_step2:JPØ(B6pvbl IDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current"trainer/global_step2:JPhî ¸iwbe BDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current"trainer/global_step2:JPÊKjxbf CDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current"trainer/global_step2:JP¥kÊqybm JDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current"trainer/global_step2:JP­ZÁ‚`zb\ 9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries"trainer/global_step2:JPps&4i{be BDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated"trainer/global_step2:JPBük|bg DDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak"trainer/global_step2:JPàzS(s}bo LDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak"trainer/global_step2:JP¡ÈÙ]~bY 6DeviceStatsMonitor.on_train_batch_start/max_split_size"trainer/global_step2:JPû=`b\ 9DeviceStatsMonitor.on_train_batch_start/segment.all.freed"trainer/global_step2:JPe¿?ˆk€bf CDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated"trainer/global_step2:JPX•‹igbb ?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak"trainer/global_step2:JPG}™j‚be BDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed"trainer/global_step2:JP7à­jƒbe BDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current"trainer/global_step2:JPRãgzl„bg DDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated"trainer/global_step2:JPQ¾Å‚g…bb ?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak"trainer/global_step2:JP- Zq†bl IDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current"trainer/global_step2:JPú­sBt‡bo LDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak"trainer/global_step2:JPo­)^uˆbp MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed"trainer/global_step2:JPÕ øn‰bi FDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated"trainer/global_step2:JP™”“´lŠbg DDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated"trainer/global_step2:JPÖ‡ßËn‹bi FDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak"trainer/global_step2:JP|+ÓciŒbd ADeviceStatsMonitor.on_train_batch_start/active.small_pool.current"trainer/global_step2:JPÙøedb_ I³ƒhšbc @DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed"trainer/global_step2:JPðƒÄ»_›bZ 7DeviceStatsMonitor.on_train_batch_start/num_device_free"trainer/global_step2:JP•€>Jiœbd ADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed"trainer/global_step2:JP§å3\mbh EDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current"trainer/global_step2:JP;Ñkžbf CDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated"trainer/global_step2:JP¹ÕQMoŸbj GDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed"trainer/global_step2:JP Õës bn KDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated"trainer/global_step2:JPÐ^‚Üi¡bd ADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed"trainer/global_step2:JPäâùÒp¢bk HDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current"trainer/global_step2:JP! lQw£br ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current"trainer/global_step2:JP\èAk¤bf CDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated"trainer/global_step2:JPΗö k¥bf CDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed"trainer/global_step2:JPybkíj¦be BDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak"trainer/global_step2:JPßF­Ùn§bi FDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak"trainer/global_step2:JP˾m¨bh EDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak"trainer/global_step2:JP6GUb©b] :DeviceStatsMonitor.on_train_batch_start/num_oom_rejections"trainer/global_step2:JPnî.ª) _wandbruntime‚91  _runtime‚91 eù(\«²O UBDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated‚10876262098432 TDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 35913877504 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 553243648 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 NCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚701855 H?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚1299 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 KBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚1533 MDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚2147 H?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚2147 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚3 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 963120128 ^MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 150337447936 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 TDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 40288387072 VFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 37073‘Ž \"453056 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚18390 BDeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 VEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 116837395968 RGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚341087 NFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚976 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚29097984 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 [HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚10756883573760 L@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚1637338 HDeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚2684 \IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚10756909001728 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚2684 RGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚225772 _MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚ 2427010017792 ODDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚567465 VKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚225775 XFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚ 2577347465728 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 UFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 3214934016 D;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚3680 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚22685 J?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚701849 O@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 2541128704 ZIDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 119353096704 [HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚10340982896400 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 MDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚1533 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚6 XEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚10756883573760 VGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 3142089728 RGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚701855 XEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚10460209762048 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 YIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 37073453056 P@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 38610163200 O?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 40288387072 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚6 WGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 34283657216 NCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚917093 P@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 36985717069 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚18396 Q>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚10873720969728 YHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 116687073864 D8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚1618942 M=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 38610163200 XEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚10876262098432 YHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 116837395968 cQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚ 2427039115776 bQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 150413827072 _LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚10341008324368 VKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚341690 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚25427968 HDeviceStatsMonitor.on_train_batch_start/allocation.all.current‚18396 RCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 2539791784 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚22386 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚606 A8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚3680 ]LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 119201437680 [KDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 37073453056 F=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚3680 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚25427968 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚603 KBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚2147 _step‚1 RCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 2541128704 YJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 2515700736 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚01DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚2684 \IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚10756909001728 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚2684 RGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚225772 _MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚ 2427010017792 ODDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚567465 VKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚225775 XFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚ 2577347465728 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 UFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 3214934016 D;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚3680 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚22685 J?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚701849 O@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 2541128704 ZIDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 119353096704 [HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚10340982896400 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 MDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚1533 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚6 XEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚10756883573760 VGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 3142089728 RGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚701855 XEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚10460209762048 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 YIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 37073453056 P@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 38610163200 O?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 40288387072 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚6 WGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 34283657216 NCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚917093 P@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 36985717069 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚18396 Q>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚10873720969728 YHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 116687073864 D8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚1618942 M=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 38610163200 XEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚10876262098432 YHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 116837395968 cQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚ 2427039115776 bQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 150413827072 _LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚10341008324368 VKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚341690 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚25427968 HDeviceStatsMonitor.on_train_batch_start/allocation.all.current‚18396 RCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 2539791784 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚22386 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚606 A8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚3680 ]LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 119201437680 [KDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 37073453056 F=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚3680 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚25427968 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚603 KBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚2147 _step‚1 RCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 2541128704 YJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 2515700736 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 UBDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated‚10876262098432 TDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 35913877504 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 553243648 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 NCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚701855 H?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚1299 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 KBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚1533 MDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚2147 H?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚2147 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚3 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 963120128 ^MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 150337447936 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 TDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 40288387072 VFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 37073453056 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚18390 BDeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 VEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 116837395968 RGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚341087 NFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚976 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚29097984 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0õub­b] :DeviceStatsMonitor.on_train_batch_end/active.all.allocated"trainer/global_step2:JP¤Õ Ùg®bb ?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current"trainer/global_step2:JP£þ®¾k¯bf CDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed"trainer/global_step2:JPbDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed"trainer/global_step2:JP±zd³b_ DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed"trainer/global_step2:JPÿdÆi½bd ADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated"trainer/global_step2:JPG´Io¾bj GDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated"trainer/global_step2:JPØ´_¿bZ 7DeviceStatsMonitor.on_train_batch_end/segment.all.freed"trainer/global_step2:JPè²(ãnÀbi FDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed"trainer/global_step2:JPc]ÆpcÁb^ ;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak"trainer/global_step2:JPKÁ8mÂbh EDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated"trainer/global_step2:JP[ÿ hqÃbl IDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated"trainer/global_step2:JP°ûW¨rÄbm JDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak"trainer/global_step2:JPv# Ö_ÅbZ 7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries"trainer/global_step2:JPáQÿTlÆbg DDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated"trainer/global_step2:JP=ûP&mÇbh EDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed"trainer/global_step2:JP™ÝNÞlÈbg DDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak"trainer/global_step2:JPé èÊfÉba >DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated"trainer/global_step2:JPÿüàünÊbi FDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current"trainer/global_step2:JPÕE8¾jËbe BDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated"trainer/global_step2:JP‰pÅ'`Ìb[ 8DeviceStatsMonitor.on_train_batch_end/active.all.current"trainer/global_step2:JPÕ%,iÍbd ADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated"trainer/global_step2:JPt˜tiÎbd ADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current"trainer/global_step2:JPCL’–gÏbb ?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed"trainer/global_step2:JPh–‰SmÐbh EDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak"trainer/global_step2:JPP³ý¢^ÑbY 6DeviceStatsMonitor.on_train_batch_end/active.all.freed"trainer/global_step2:JPI{èøfÒba >DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak"trainer/global_step2:JP¹TÕåkÓbf CDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current"trainer/global_step2:JP. )ÉsÔbn KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed"trainer/global_step2:JPñ™¹sÕbn KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed"trainer/global_step2:JP«ž,ÔgÖbb ?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current"trainer/global_step2:JPš,½e×b` =DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed"trainer/global_step2:JP´‚êSpØbk HDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current"trainer/global_step2:JP7Š’JeÙb` =DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed"trainer/global_step2:JPÉ&ËRoÚbj GDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated"trainer/global_step2:JPܬ¸²jÛbe BDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated"trainer/global_step2:JP²qsYlÜbg DDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed"trainer/global_step2:JP?ÇsDmÝbh EDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak"trainer/global_step2:JPôÝ bÞb] :DeviceStatsMonitor.on_train_batch_end/allocation.all.freed"trainer/global_step2:JPVðõ>rßbm JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated"trainer/global_step2:JPÒuàbp MDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current"trainer/global_step2:JPêR Éhábc @DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current"trainer/global_step2:JPßùjâbe BDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak"trainer/global_step2:JP‹m5anãbi FDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed"trainer/global_step2:JPÌ”žWeäb` =DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak"trainer/global_step2:JP'®ôfåba >DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current"trainer/global_step2:JPä.‚ãqæbl IDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated"trainer/global_step2:JP™ð]çbX 5DeviceStatsMonitor.on_train_batch_end/active.all.peak"trainer/global_step2:JPñÆš&jèbe BDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak"trainer/global_step2:JP’ªŠDrébm JDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak"trainer/global_step2:JP*/Š§`êb[ 8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections"trainer/global_step2:JP·ù:9gëbb ?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current"trainer/global_step2:JP”µ‹weìb` =DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak"trainer/global_step2:JP"ª²híbc @DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated"trainer/global_step2:JPZ€{Ydîb_ DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak"trainer/global_step2:JPE@¹ãhöbc @DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current"trainer/global_step2:JP…3•=e÷b` =DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak"trainer/global_step2:JP2>‘Ädøb_ DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed"trainer/global_step2:JP” '\o‹bj GDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current"trainer/global_step2:JP~.‡3]ŒbX 5DeviceStatsMonitor.on_train_batch_end/num_device_free"trainer/global_step2:JPør>ßjbe BDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current"trainer/global_step2:JP7°˜IlŽbg DDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak"trainer/global_step2:JPd—§³wbr ODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated"trainer/global_step2:JPë;žgbb ?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak"trainer/global_step2:JPWh&1p‘bk HDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current"trainer/global_step2:JPÀâœf’ba >DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed"trainer/global_step2:JPÅŒìÝm“bh EDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current"trainer/global_step2:JP|0MÌk”bf CDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current"trainer/global_step2:JPT ý+i•bd ADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed"trainer/global_step2:JPÙyûm–bh EDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed"trainer/global_step2:JP½è<\—bW 4DeviceStatsMonitor.on_train_batch_end/max_split_size"trainer/global_step2:JPLïµm˜bh EDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak"trainer/global_step2:JPÎk|\h™bc @DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current"trainer/global_step2:JP¼ñ¬mšbh EDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated"trainer/global_step2:JPŠ§¸Œl›bg DDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak"trainer/global_step2:JP0¢;™gœbb ?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed"trainer/global_step2:JPŽÜû lbg DDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak"trainer/global_step2:JP¦×ŒSpžbk HDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current"trainer/global_step2:JPQWYiŸbd ADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed"trainer/global_step2:JP—£"…i bd ADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current"trainer/global_step2:JP+©#¨o¡bj GDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current"trainer/global_step2:JPž™n¡d¢b_ DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚577901 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚594 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 LDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚330 `ODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 153362775040 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚25427968 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 trainer/global_step‚49 TEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 2515714048 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚18416 LADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚935435 PEDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚230138 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 UEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 34283657216 I@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚2147 PEDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚953851 LDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚976 R?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚10665538766494 TDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 37073453056 WHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 2515714048 LADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚715739 PADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 2539791992 VGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 3214934016 EDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 _step‚2 ODeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 LADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚953851 ZGDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚10970855072256 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 % _timestamp‚1.7884852944600708e+09 WFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 119193491968 K;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 38610163200 PEDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚715745 TIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚230141 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 553243648 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 SDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 3214934016 J>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚1669596 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 101269504 KBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚1533 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚18422 LADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚715745 PADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 2541142016 R?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚11090023136256 UEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 35913877504 B6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚1651174 N>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 38610163200 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚6 \KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 153290603520 _runtime‚ 93.562133202 ]KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚ 2471140342784 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 WHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 2514364024 H=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚935435 XGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 121709206016 MBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚578498 VDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚ 2624430946304 TEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 3142090752 F:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚1651174 ]JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚10546523984864 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚29097984 P@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 40288387072 QBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 3142090752 YFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚10970829644288 M=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 40288387072 M>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 2541142016 TIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚348357 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚22687 RBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 35913877504 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 963120128 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚18416 F=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚2147 S@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚11092564278272 FDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 36985717077 I@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚1533 F=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚1533 FDeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 LADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚953851 ZGDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚10970855072256 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 % _timestamp‚1.7884852944600708e+09 WFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 119193491968 K;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 38610163200 PEDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚715745 TIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚230141 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 553243648 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 SDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 3214934016 J>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚1669596 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 101269504 KBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚1533 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚18422 LADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚715745 PADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 2541142016 R?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚11090023136256 UEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 35913877504 B6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚1651174 N>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 38610163200 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚6 \KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 153290603520 _runtime‚ 93.562133202 ]KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚ 2471140342784 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 WHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 2514364024 H=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚935435 XGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 121709206016 MBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚578498 VDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚ 2624430946304 TEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 3142090752 F:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚1651174 ]JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚10546523984864 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚29097984 P@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 40288387072 QBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 3142090752 YFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚10970829644288 M=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 40288387072 M>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 2541142016 TIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚348357 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚22687 RBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 35913877504 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 963120128 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚18416 F=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚2147 S@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚11092564278272 FDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 36985717077 I@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚1533 F=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚1533 FDeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚577901 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚594 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 LDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚330 `ODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 153362775040 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚25427968 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 trainer/global_step‚49 TEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 2515714048 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚18416 LADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚935435 PEDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚230138 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 UEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 34283657216 I@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚2147 PEDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚953851 LDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚976 R?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚10665538766494 TDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 37073453056 WHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 2515714048 LADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚715739 PADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 2539791992 VGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 3214934016 EDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 _step‚28òÙ«@«b; loss/FRt4/detection_loss"trainer/global_step2:JPÊ¢A¬b< loss/FRt1/assignment_loss"trainer/global_step2:JPü ¶þA­b< loss/FRt3/assignment_loss"trainer/global_step2:JPf¿ÓA®b< loss/FBt2/assignment_loss"trainer/global_step2:JPò^‘'@¯b; loss/FBt4/detection_loss"trainer/global_step2:JP< ˜A°b< loss/FRt2/assignment_loss"trainer/global_step2:JP~ÁêC±b> loss/SRqqt4/assignment_loss"trainer/global_step2:JP§â\Ò@²b; loss/FBt1/detection_loss"trainer/global_step2:JPÿvÈ@³b; loss/FBt2/detection_loss"trainer/global_step2:JP³2“B´b= loss/SRqqt3/detection_loss"trainer/global_step2:JP ñÁ¯@µb; loss/FRt1/detection_loss"trainer/global_step2:JP¤chbC¶b> loss/SRqqt2/assignment_loss"trainer/global_step2:JP17-A·b< loss/FBt3/assignment_loss"trainer/global_step2:JP 2K@¸b; loss/FRt3/detection_loss"trainer/global_step2:JPßû}dB¹b= loss/SRqqt1/detection_loss"trainer/global_step2:JPãæ•ÿ@ºb; loss/FBt3/detection_loss"trainer/global_step2:JPï2ª¢C»b> loss/SRqqt1/assignment_loss"trainer/global_step2:JP´NãïA¼b< loss/FBt4/assignment_loss"trainer/global_step2:JPðÍÞ@B½b= loss/SRqqt2/detection_loss"trainer/global_step2:JPsÒ£B¾b= loss/SRqqt4/detection_loss"trainer/global_step2:JPº£6ò7¿b2 loss/total_loss"trainer/global_step2:JPGçN-Àb( epoch"trainer/global_step2:JPJdóÃCÁb> loss/SRqqt3/assignment_loss"trainer/global_step2:JP=®½@Âb; loss/FRt2/detection_loss"trainer/global_step2:JP·ä’AÃb< loss/FRt4/assignment_loss"trainer/global_step2:JP½ƒçÙAÄb< loss/FBt1/assignment_loss"trainer/global_step2:JPãðÖ.Å) _wandbruntime‚93  _runtime‚93[£ÐÍMÆÇ /loss/FRt2/assignment_loss‚7.451494838510241 1loss/SRqqt4/assignment_loss‚5.137363762566538 /loss/FBt1/detection_loss‚10.891730813365994 .loss/FBt2/detection_loss‚13.47738629910681 % _timestamp‚1.7884852944649897e+09 0loss/SRqqt3/detection_loss‚4.942802288514726 .loss/FRt1/detection_loss‚6.612070279625746 1loss/SRqqt2/assignment_loss‚5.346506092646351 0loss/FBt3/assignment_loss‚0.4978017330169678 .loss/FRt3/detection_loss‚4.330028463390809 0loss/SRqqt1/detection_loss‚4.794049269755681 /loss/FBt3/detection_loss‚17.244896622002123 1loss/SRqqt1/assignment_loss‚5.211464551289876 0loss/FBt4/assignment_loss‚0.6208953132375705 1loss/SRqqt2/detection_loss‚1.9573348239957267 1loss/SRqqt4/detection_loss‚2.3361244599024453 &loss/total_loss‚126.11746409586777 epoch‚0 trainer/global_step‚49 _step‚3 1loss/SRqqt3/assignment_loss‚5.254746724577511 /loss/FRt2/detection_loss‚2.9085804086976816 /loss/FRt4/assignment_loss‚7.165796926779341 0loss/FBt1/assignment_loss‚0.6454894035828836 /loss/FRt4/detection_loss‚2.3704256844064293 _runtime‚ 93.566454971 /loss/FRt1/assignment_loss‚7.131340613731971 /loss/FRt3/assignment_loss‚7.385008842130251 0loss/FBt2/assignment_loss‚0.5955551030735174 /loss/FBt4/detection_loss‚1.8085707779605888 2Â~QÇË /loss/FRt2/assignment_loss‚7.451494838510241 1loss/SRqqt4/assignment_loss‚5.137363762566538 /loss/FBt1/detection_loss‚10.891730813365994 .loss/FBt2/detection_loss‚13.47738629910681 % _timestamp‚1.7884852944649897e+09 0loss/SRqqt3/detection_loss‚4.942802288514726 .loss/FRt1/detection_loss‚6.612070279625746 1loss/SRqqt2/assignment_loss‚5.346506092646351 0loss/FBt3/assignment_loss‚0.4978017330169678 .loss/FRt3/detection_loss‚4.330028463390809 0loss/SRqqt1/detection_loss‚4.794049269755681 /loss/FBt3/detection_loss‚17.244896622002123 1loss/SRqqt1/assignment_loss‚5.211464551289876 0loss/FBt4/assignment_loss‚0.6208953132375705 1loss/SRqqt2/detection_loss‚1.9573348239957267 1loss/SRqqt4/detection_loss‚2.3361244599024453 &loss/total_loss‚126.11746409586777 epoch‚0 trainer/global_step‚49 _step‚3 1loss/SRqqt3/assignment_loss‚5.254746724577511 /loss/FRt2/detection_loss‚2.9085804086976816 /loss/FRt4/assignment_loss‚7.165796926779341 0loss/FBt1/assignment_loss‚0.6454894035828836 /loss/FRt4/detection_loss‚2.3704256844064293 _runtime‚ 93.566454971 /loss/FRt1/assignment_loss‚7.131340613731971 /loss/FRt3/assignment_loss‚7.385008842130251 0loss/FBt2/assignment_loss‚0.5955551030735174 /loss/FBt4/detection_loss‚1.8085707779605888(A,î“È‚(:ˆ ¹½èÔ˜´ù® gpu.0.gpu‚64.0 gpu.0.memory‚12+ gpu.0.memoryAllocated‚47.54493713378906* gpu.0.memoryAllocatedBytes‚ 40840790016 gpu.0.temp‚45.0 gpu.0.powerWatts‚210.546' gpu.0.enforcedPowerLimitWatts‚400.0 gpu.0.powerPercent‚52.6365 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0˜"½É‚(:ú ¹½èÔŽ²ö· disk.nvme0n1.in‚ 0.2421875 disk.sdb2.out‚11.92529296875 disk.md0.out‚ 11.5234375* proc.memory.percent‚18.114328384399414 proc.cpu.threads‚17 disk.nvme1n1.in‚4 disk.sda.in‚0 disk.dm-0.out‚ 1024.76171875 disk.sda1.in‚0 disk.rbd2.in‚0 disk.sdb.in‚0$ memory_percent‚92.09377765655518) proc.memory.availableMB‚ 1295.35546875" proc.memory.rssMB‚ 2967.8515625! disk.nvme0n1.out‚ 849.54296875 disk.dm-0.in‚ 4.2421875 disk.sda1.out‚0 disk.md0p1.out‚ 11.5234375 disk.rbd2.out‚0 disk.sda2.in‚0 cpu‚25.265612672659458 network.sent‚109986) disk./.usagePercent‚83.24623526719402 disk.md0.in‚0 disk.md0p1.in‚0 disk.sdb.out‚11.92529296875% disk./.usageGB‚2977.2885971069336 disk.sdb1.in‚0 disk.sdb2.in‚0 network.recv‚124014 disk.rbd0.out‚0 disk.sdb1.out‚0 disk.sda2.out‚11.92529296875 disk.nvme1n1.out‚ 175.21875 disk.sda.out‚11.92529296875 disk.rbd0.in‚0ïV…è”Ê‚(:‰ ȽèÔÕ®£® gpu.0.gpu‚98.0 gpu.0.memory‚17+ gpu.0.memoryAllocated‚47.54493713378906* gpu.0.memoryAllocatedBytes‚ 40840790016 gpu.0.temp‚43.0 gpu.0.powerWatts‚117.525' gpu.0.enforcedPowerLimitWatts‚400.0 gpu.0.powerPercent‚29.38125 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0Îé‹æúË‚(:ï ȽèÔîâü® disk.sda.out‚ 13.994140625% disk./.usageGB‚2977.2885971069336 disk.dm-0.in‚ 4.2421875 disk.md0.in‚0 disk.rbd2.out‚0 disk.sda.in‚0 disk.rbd0.out‚0" proc.memory.rssMB‚ 2967.8515625 disk.sdb2.out‚ 13.994140625 disk.md0p1.out‚13.53125 disk.rbd2.in‚0 disk.sdb.in‚0$ memory_percent‚92.09914207458496 disk.sdb1.out‚0 disk.sdb2.in‚0 disk.nvme1n1.out‚ 282.7421875! disk.nvme0n1.out‚ 921.70703125 disk.sda1.out‚0 disk.md0.out‚13.53125 disk.md0p1.in‚0 disk.sda2.in‚0 disk.nvme1n1.in‚4* proc.memory.percent‚18.114328384399414 cpu‚25.275909414348735 network.recv‚129907 disk.sdb1.in‚0 disk.sda1.in‚0 disk.rbd0.in‚0 network.sent‚159623) disk./.usagePercent‚83.24623526719402 disk.sdb.out‚ 13.994140625 disk.sda2.out‚ 13.994140625 disk.nvme0n1.in‚ 0.2421875 disk.dm-0.out‚ 1204.44921875( proc.memory.availableMB‚ 1294.4765625 proc.cpu.threads‚17~( ®Ì‚(:’ ×½èÔž“° gpu.0.gpu‚97.0 gpu.0.memory‚17+ gpu.0.memoryAllocated‚47.54493713378906* gpu.0.memoryAllocatedBytes‚ 40840790016 gpu.0.temp‚45.0 gpu.0.powerWatts‚229.864' gpu.0.enforcedPowerLimitWatts‚400.0( gpu.0.powerPercent‚57.46600000000001 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0©:/çÍ‚(:÷ ×½èÔŠ…ä° disk.dm-0.out‚ 1343.3046875 network.sent‚165900 disk.sdb2.in‚0 disk.rbd2.out‚0 disk.sdb.in‚0! disk.nvme1n1.out‚ 360.34765625 disk.sda.in‚0 disk.dm-0.in‚ 4.2421875% disk./.usageGB‚2977.5385971069336 disk.sda1.in‚0 disk.md0.out‚ 14.98828125 disk.nvme0n1.in‚ 0.2421875 disk.sdb1.in‚0 disk.sdb1.out‚0$ memory_percent‚92.09778308868408 proc.cpu.threads‚17 disk.rbd0.out‚0# proc.memory.rssMB‚ 2967.85546875) disk./.usagePercent‚83.25322537182714 disk.sdb2.out‚ 15.505859375 disk.sda1.out‚0 disk.sdb.out‚ 15.505859375 disk.sda2.in‚0 disk.nvme1n1.in‚4) proc.memory.availableMB‚ 1294.69921875 disk.rbd0.in‚0 disk.md0.in‚0 disk.md0p1.in‚0 disk.sda2.out‚ 15.505859375 network.recv‚133988 disk.rbd2.in‚0* proc.memory.percent‚18.114352226257324 cpu‚25.279143329852964 disk.md0p1.out‚ 14.98828125 disk.sda.out‚ 15.505859375! disk.nvme0n1.out‚ 982.95703125¦K«’΂(:‡ æ½èԞǕ° gpu.0.gpu‚8.0 gpu.0.memory‚4+ gpu.0.memoryAllocated‚47.54493713378906* gpu.0.memoryAllocatedBytes‚ 40840790016 gpu.0.temp‚46.0 gpu.0.powerWatts‚312.611' gpu.0.enforcedPowerLimitWatts‚400.0 gpu.0.powerPercent‚78.15275 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0êSI‚Ï‚(:ü æ½èÔ³ýê±" disk.nvme0n1.out‚ 1044.22265625 disk.sdb1.out‚0 disk.rbd2.in‚0 disk.sdb.in‚0 disk.sda2.in‚0 network.recv‚138707 disk.sdb.out‚17.00439453125 disk.sda2.out‚17.00439453125 disk.rbd0.out‚0 network.sent‚171652) proc.memory.availableMB‚ 1294.32421875 cpu‚25.2849123447488% disk./.usageGB‚2977.5385971069336 disk.sdb1.in‚0 disk.sda.out‚17.00439453125 disk.dm-0.in‚ 4.2421875 disk.md0.out‚ 16.42578125 disk.md0p1.in‚0 disk.rbd0.in‚0 proc.cpu.threads‚17 disk.dm-0.out‚ 1507.98046875 disk.rbd2.out‚0$ memory_percent‚92.10007190704346* proc.memory.percent‚18.114376068115234 disk.nvme0n1.in‚ 0.2421875 disk.sdb2.in‚0 disk.sda1.out‚0 disk.md0p1.out‚ 16.42578125) disk./.usagePercent‚83.25322537182714 disk.sdb2.out‚17.00439453125 disk.sda1.in‚0 disk.md0.in‚0 disk.nvme1n1.out‚ 463.7578125! proc.memory.rssMB‚ 2967.859375 disk.nvme1n1.in‚4 disk.sda.in‚0'öe›Ð‚(: õ½èÔÐÓ£° gpu.0.gpu‚90.0 gpu.0.memory‚15+ gpu.0.memoryAllocated‚47.54493713378906* gpu.0.memoryAllocatedBytes‚ 40840790016 gpu.0.temp‚45.0 gpu.0.powerWatts‚236.1' gpu.0.enforcedPowerLimitWatts‚400.0( gpu.0.powerPercent‚59.02499999999999 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0fÛ·£ìÑ‚(:á õ½èÔøѾ°# memory_percent‚92.1008825302124* proc.memory.percent‚18.114376068115234 disk.sda2.out‚ 18.703125 disk.rbd0.in‚0 disk.md0p1.in‚0 disk.sdb.in‚0! proc.memory.rssMB‚ 2967.859375) disk./.usagePercent‚83.25322547848768! disk.nvme1n1.out‚ 541.34765625 network.sent‚178264 disk.sda2.in‚0 disk.rbd0.out‚0 proc.cpu.threads‚17 disk.nvme0n1.in‚ 0.2421875 disk.dm-0.in‚ 4.2421875 disk.sdb2.in‚0 disk.md0.in‚0 disk.md0p1.out‚ 18.05859375 disk.rbd2.out‚0 network.recv‚143607$ disk./.usageGB‚2977.538600921631 disk.dm-0.out‚ 1646.84765625 disk.sda1.out‚0 disk.rbd2.in‚0 disk.sda.in‚0 disk.sdb1.in‚0 disk.sdb1.out‚0) proc.memory.availableMB‚ 1294.19140625 disk.nvme1n1.in‚4 disk.sdb2.out‚ 18.703125 disk.sda1.in‚0 disk.sdb.out‚ 18.703125 cpu‚25.28779602182997 disk.sda.out‚ 18.703125 disk.nvme0n1.out‚1105.5 disk.md0.out‚ 18.05859375õ!RÂ0Ò+ _wandbruntime‚179  _runtime‚179ç·—:ÂÓ¼ _runtime‚ 179.570795548 _step‚4 & lr-AdamW/pg1‚0.0001864174645772365 & lr-AdamW/pg2‚0.0001864174645772365 trainer/global_step‚99 % _timestamp‚1.7884853804692605e+09Ô_ö/ÆÔÀ & lr-AdamW/pg1‚0.0001864174645772365 & lr-AdamW/pg2‚0.0001864174645772365 trainer/global_step‚99 % _timestamp‚1.7884853804692605e+09 _runtime‚ 179.570795548 _step‚4L0Õ+ _wandbruntime‚179  _runtime‚179±«ûÓ§Ö¼O XEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚21691398253056 BDeviceStatsMonitor.on_train_batch_start/allocation.all.current‚18396 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 963120128 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 YHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 234371032572 OCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚1396355 L@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚3252290 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 WGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 34283657216 XEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚21454187100160 F=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚3680 H?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚1533 D8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚3233894 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚6 K?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚1396349 \JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚ 4931874209280 VFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 37073453056 Q>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚21688857124352 RGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚444072 NFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚976 P@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 36985717077 NFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚330 VKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚677390 VGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 3140732164 O?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 40288387072 H?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚2147 trainer/global_step‚99 % _timestamp‚1.7884853804708257e+09 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚18396 VEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 234670024192 PDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚1121465 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚566 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 RCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 2539791784 _step‚5 O@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 2541128704 TDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 35913877504 SDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 3142090752 YJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 2515700736 XFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚ 4931777120768 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 ]LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 237185724928 OCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚1837545 TDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 40288387072 D;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚3680 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚22388 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚3 KBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚1533 HDeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 RBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 40288387072 KBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚2147 TADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚21688857124352 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚2684 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚67990528 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 TADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚20851136953772 ZKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 3214934016 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚22687 [HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚21454187100160 VKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚444075 bQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 298318843904 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 RCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 2541128704 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚569 H?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚1299 ^MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 298250853376 XEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚20853676745556 XIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 3214934016 VGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 3142090752 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚6 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚18390 _MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚ 4633526267392 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚25427968 G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚2684 TEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 1491343872 cQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚ 4633555365376 ZIDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 237185724928 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚22388 P@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 38610163200 UHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚97088512 UBDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated‚21691398253056@FŸJÆ'×ÀO SGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚1855935 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚18396 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 963120128 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 YHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 234371032572 OCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚1396355 L@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚3252290 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 WGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 34283657216 XEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚21454187100160 F=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚3680 H?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚1533 D8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚3233894 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚6 K?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚1396349 \JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚ 4931874209280 VFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 37073453056 Q>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚21688857124352 RGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚444072 NFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚976 P@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 36985717077 NFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚330 VKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚677390 VGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 3140732164 O?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 40288387072 H?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚2147 trainer/global_step‚99 % _timestamp‚1.7884853804708257e+09 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚18396 VEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 234670024192 PDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚1121465 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚566 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 RCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 2539791784 _step‚5 O@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 2541128704 TDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 35913877504 SDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 3142090752 YJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 2515700736 XFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚ 4931777120768 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 ]LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 237185724928 OCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚1837545 TDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 40288387072 D;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚3680 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚22388 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚3 KBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚1533 HDeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 RBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 40288387072 KBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚2147 TADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚21688857124352 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚2684 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚67990528 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 TADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚20851136953772 ZKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 3214934016 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚22687 [HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚21454187100160 VKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚444075 bQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 298318843904 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 RCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 2541128704 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚569 H?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚1299 ^MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 298250853376 XEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚20853676745556 XIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 3214934016 VGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 3142090752 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚6 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚18390 _MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚ 4633526267392 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚25427968 G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚2684 TEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 1491343872 cQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚ 4633555365376 ZIDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 237185724928 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚22388 P@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 38610163200 UHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚97088512 UBDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated‚21691398253056 XEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚21691398253056 BDeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 2541142016 \KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 301235487744 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 WFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 237026120192 QEDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚1410245 RBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 40288387072 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1299 ZHDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚ 4978989155328 J>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚3284548 MADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚1410239 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 I@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚1533 F=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚2147 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 QBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 3142090752 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚25427968 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 ?6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚3680 I@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚2147 _runtime‚ 181.184172092 LDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚330 N>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 36985717077 UEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 34283657216 YIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 37073453056 aODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚ 4677685690368 TIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚448441 M=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 40288387072 TDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 37073453056 I=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚1855887 MADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚1855887 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 963120128 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 WHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 2514364024 BDeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 F=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚1533 TCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 237026120192 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚29097984 ?6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚3680 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 B9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚3680 B6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚3266126 ODeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 38610163200 F:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚3266126 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚6 EDeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚1131869 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 SFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚97075200 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚67977216 SDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 3214934016 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0KÚAZÜÊM YFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚21668133170688 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚3 PADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 2539791992 MADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚1874303 M>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 2541142016 \KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 301235487744 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 WFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 237026120192 QEDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚1410245 RBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 40288387072 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1299 ZHDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚ 4978989155328 J>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚3284548 MADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚1410239 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_stream˜U‹v!s‚2 I@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚1533 F=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚2147 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 QBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 3142090752 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚25427968 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 ?6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚3680 I@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚2147 _runtime‚ 181.184172092 LDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚330 N>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 36985717077 UEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 34283657216 YIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 37073453056 aODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚ 4677685690368 TIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚448441 M=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 40288387072 TDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 37073453056 I=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚1855887 MADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚1855887 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 963120128 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 WHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 2514364024 BDeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 F=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚1533 TCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 237026120192 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚29097984 ?6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚3680 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 B9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚3680 B6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚3266126 ODeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 38610163200 F:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚3266126 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚6 EDeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚1131869 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 SFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚97075200 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚67977216 SDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 3214934016 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 KBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚1533 ]JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚20822307009664 F:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚3284548 VCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚21907700432896 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚22687 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 RBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 35913877504 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚569 LDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚976 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 TEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 3140732164 [JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 239541834240 PEDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚683431 P@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 40288387072|¹0Ý+ _wandbruntime‚181  _runtime‚181ZÚu!KÞÅ 0loss/SRqqt3/detection_loss‚3.023700128452709 .loss/FBt3/detection_loss‚7.896933895058748 /loss/FBt4/detection_loss‚1.7212474222641876 $ _timestamp‚1.788485382085356e+09 .loss/FRt1/detection_loss‚4.199686677367599 .loss/FRt3/detection_loss‚3.500178941525519 0loss/SRqqt1/detection_loss‚3.942211273415335 .loss/FBt1/detection_loss‚8.365023837983609 _runtime‚ 181.186582053 /loss/FRt4/detection_loss‚2.3017984393024533 .loss/FBt2/detection_loss‚5.241008831188083 /loss/FRt1/assignment_loss‚6.982007097314905 /loss/FRt2/assignment_loss‚7.320585528506508 /loss/FRt4/assignment_loss‚7.224630243850477 1loss/SRqqt1/assignment_loss‚3.273641786356082 2loss/SRqqt2/assignment_loss‚3.6817891221297416 0loss/FBt1/assignment_loss‚0.5263240173359918 0loss/FBt3/assignment_loss‚0.3415216604435932 0loss/SRqqt4/detection_loss‚2.084455829884209 2loss/SRqqt3/assignment_loss‚3.0531807289673734 1loss/FBt2/assignment_loss‚0.24145455971652474 0loss/SRqqt2/detection_loss‚4.283983668998668 %loss/total_loss‚94.56031526170597 epoch‚0 trainer/global_step‚99 _step‚7 .loss/FRt2/detection_loss‚4.280504103514213 /loss/FRt3/assignment_loss‚7.340356613198916 2loss/SRqqt4/assignment_loss‚3.2109172063715317 0loss/FBt4/assignment_loss‚0.5231736485589863²™‹OßÉ /loss/FRt1/assignment_loss‚6.982007097314905 /loss/FRt2/assignment_loss‚7.320585528506508 /loss/FRt4/assignment_loss‚7.224630243850477 1loss/SRqqt1/assignment_loss‚3.273641786356082 2loss/SRqqt2/assignment_loss‚3.6817891221297416 0loss/FBt1/assignment_loss‚0.5263240173359918 0loss/FBt3/assignment_loss‚0.3415216604435932 0loss/SRqqt4/detection_loss‚2.084455829884209 2loss/SRqqt3/assignment_loss‚3.0531807289673734 1loss/FBt2/assignment_loss‚0.24145455971652474 0loss/SRqqt2/detection_loss‚4.283983668998668 %loss/total_loss‚94.56031526170597 epoch‚0 trainer/global_step‚99 _step‚7 .loss/FRt2/detection_loss‚4.280504103514213 /loss/FRt3/assignment_loss‚7.340356613198916 2loss/SRqqt4/assignment_loss‚3.2109172063715317 0loss/FBt4/assignment_loss‚0.5231736485589863 0loss/SRqqt3/detection_loss‚3.023700128452709 .loss/FBt3/detection_loss‚7.896933895058748 /loss/FBt4/detection_loss‚1.7212474222641876 $ _timestamp‚1.788485382085356e+09 .loss/FRt1/detection_loss‚4.199686677367599 .loss/FRt3/detection_loss‚3.500178941525519 0loss/SRqqt1/detection_loss‚3.942211273415335 .loss/FBt1/detection_loss‚8.365023837983609 _runtime‚ 181.186582053 /loss/FRt4/detection_loss‚2.3017984393024533 .loss/FBt2/detection_loss‚5.241008831188083}C1-ýà‚(:ò “¾èÔÙÓݳ disk.sdb1.in‚0 network.sent‚222450) disk./.usagePercent‚83.23722671865102 disk.sda2.in‚0 disk.sda.out‚22.36474609375' proc.memory.availableMB‚ 1293.859375 disk.sda1.in‚0 disk.sdb.out‚22.36474609375 disk.sda.in‚0$ memory_percent‚92.10290908813477 network.recv‚153639 disk.md0.in‚0 disk.dm-0.in‚ 4.2421875 disk.sdb2.in‚0 disk.sda2.out‚22.36474609375 disk.rbd0.in‚0 disk.rbd0.out‚0 cpu‚25.297787539949887$ disk./.usageGB‚2976.966407775879 disk.md0.out‚21.59375 disk.rbd2.out‚0 disk.sdb.in‚0! proc.memory.rssMB‚ 2967.859375 disk.nvme1n1.out‚ 585.734375 proc.cpu.threads‚17 disk.dm-0.out‚ 1875.0859375 disk.sdb2.out‚22.36474609375! disk.nvme0n1.out‚ 1289.3515625 disk.sdb1.out‚0 disk.sda1.out‚0 disk.md0p1.in‚0 disk.md0p1.out‚21.59375 disk.rbd2.in‚0 disk.nvme1n1.in‚4* proc.memory.percent‚18.114376068115234 disk.nvme0n1.in‚ 0.2421875îæB~”á‚(:‰ “¾èÔÄüò³ gpu.0.gpu‚95.0 gpu.0.memory‚17+ gpu.0.memoryAllocated‚47.54493713378906* gpu.0.memoryAllocatedBytes‚ 40840790016 gpu.0.temp‚44.0 gpu.0.powerWatts‚117.193' gpu.0.enforcedPowerLimitWatts‚400.0 gpu.0.powerPercent‚29.29825 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0? ùâ‚(:î ¢¾èÔËÜ„· network.sent‚262555 disk.sdb1.out‚0 disk.sda1.in‚0 disk.nvme0n1.in‚ 0.2421875 cpu‚25.29931480684378 disk.rbd2.out‚0 disk.sdb.in‚0 disk.sda.in‚0 disk.dm-0.in‚ 4.2421875# memory_percent‚92.1022891998291 network.recv‚157987 disk.sdb1.in‚0 disk.sdb.out‚ 23.9892578125 disk.sda2.out‚ 23.9892578125 disk.nvme1n1.in‚4$ disk./.usageGB‚2977.036548614502 disk.sdb2.in‚0 disk.md0p1.in‚0 disk.rbd2.in‚0! disk.nvme1n1.out‚ 657.75390625 disk.rbd0.in‚0 proc.cpu.threads‚17 disk.md0.out‚23.15625 disk.sda2.in‚0 disk.md0p1.out‚23.15625 disk.sdb2.out‚ 23.9892578125 disk.md0.in‚0" disk.nvme0n1.out‚ 1350.71484375( proc.memory.availableMB‚ 1293.9609375! proc.memory.rssMB‚ 2967.859375) disk./.usagePercent‚83.23918788585513 disk.sda1.out‚0 disk.sda.out‚ 23.9892578125 disk.rbd0.out‚0 disk.dm-0.out‚ 2008.46875* proc.memory.percent‚18.114376068115234M¬*žã‚(:“ ¢¾èÔëà»· gpu.0.gpu‚98.0 gpu.0.memory‚17+ gpu.0.memoryAllocated‚47.54493713378906* gpu.0.memoryAllocatedBytes‚ 40840790016 gpu.0.temp‚45.0 gpu.0.powerWatts‚234.448' gpu.0.enforcedPowerLimitWatts‚400.0) gpu.0.powerPercent‚58.611999999999995 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0©ŽГä‚(:ˆ ±¾èÔ¦ŽÈ° gpu.0.gpu‚80.0 gpu.0.memory‚19+ gpu.0.memoryAllocated‚47.54493713378906* gpu.0.memoryAllocatedBytes‚ 40840790016 gpu.0.temp‚44.0 gpu.0.powerWatts‚176.738' gpu.0.enforcedPowerLimitWatts‚400.0 gpu.0.powerPercent‚44.1845 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0B·UÂ÷å‚(:ì ±¾èÔÓÙݱ network.sent‚269150 disk.nvme1n1.out‚ 685.84375 disk.md0.out‚ 25.109375 disk.md0p1.in‚0 disk.rbd2.in‚0 disk.sdb.in‚0 proc.cpu.threads‚17) disk./.usagePercent‚83.24011903230057$ disk./.usageGB‚2977.069850921631 disk.rbd0.in‚0 disk.dm-0.out‚ 2097.8125 disk.md0.in‚0) proc.memory.availableMB‚ 1294.19921875! proc.memory.rssMB‚ 2967.859375* proc.memory.percent‚18.114376068115234 cpu‚25.302487041335247 disk.sda1.out‚0 disk.rbd2.out‚0 network.recv‚163242 disk.sda.in‚0 disk.dm-0.in‚ 4.2421875 disk.sda1.in‚0 disk.sdb.out‚ 26.0087890625 disk.sda2.in‚0 disk.sda.out‚ 26.0087890625 disk.nvme0n1.out‚ 1411.96875 disk.sda2.out‚ 26.0087890625 disk.rbd0.out‚0 disk.sdb1.out‚0 disk.md0p1.out‚ 25.109375 disk.nvme0n1.in‚ 0.2421875 disk.nvme1n1.in‚4 disk.sdb1.in‚0 disk.sdb2.in‚0 disk.sdb2.out‚ 26.0087890625$ memory_percent‚92.10083484649658Ô”æ‚(:‰ À¾èÔ÷° gpu.0.gpu‚97.0 gpu.0.memory‚17+ gpu.0.memoryAllocated‚47.54493713378906* gpu.0.memoryAllocatedBytes‚ 40840790016 gpu.0.temp‚45.0 gpu.0.powerWatts‚226.703' gpu.0.enforcedPowerLimitWatts‚400.0 gpu.0.powerPercent‚56.67575 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0k~Íç‚(:ù À¾èÔÓúϱ disk.rbd2.out‚0 network.recv‚168401 disk.nvme0n1.in‚ 0.2421875 disk.sdb1.out‚0 disk.sda.out‚27.56591796875" memory_percent‚92.085862159729$ disk./.usageGB‚2977.288185119629 disk.rbd0.out‚0 disk.nvme1n1.out‚ 737.84375! proc.memory.rssMB‚ 2967.859375) disk./.usagePercent‚83.24622374785655 disk.dm-0.in‚ 4.2421875 disk.dm-0.out‚ 2272.28515625 disk.sda2.in‚0 disk.rbd0.in‚0 cpu‚25.300475503763376 network.sent‚275740 disk.sda1.out‚0 disk.md0.in‚0 disk.md0p1.in‚0 disk.md0p1.out‚ 26.59765625 disk.sda.in‚0 disk.sdb2.out‚27.56591796875 disk.md0.out‚ 26.59765625 disk.rbd2.in‚0 disk.sdb.out‚27.56591796875 disk.sda2.out‚27.56591796875) proc.memory.availableMB‚ 1296.65234375* proc.memory.percent‚18.114376068115234" disk.nvme0n1.out‚ 1534.44140625 disk.sda1.in‚0 disk.sdb.in‚0 disk.nvme1n1.in‚4 proc.cpu.threads‚17 disk.sdb1.in‚0 disk.sdb2.in‚0"Eå’è‚(:‡ ϾèÔû”د gpu.0.gpu‚93.0 gpu.0.memory‚16+ gpu.0.memoryAllocated‚47.54493713378906* gpu.0.memoryAllocatedBytes‚ 40840790016 gpu.0.temp‚45.0 gpu.0.powerWatts‚246.004' gpu.0.enforcedPowerLimitWatts‚400.0 gpu.0.powerPercent‚61.501 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0çY8é‚(:ü ϾèÔöį° disk.md0p1.in‚0 disk.rbd2.in‚0 disk.sda.out‚29.16455078125 disk.dm-0.out‚ 2411.13671875 disk.nvme1n1.out‚ 815.4453125! proc.memory.rssMB‚ 2967.859375 disk.sdb2.in‚0 disk.sda1.in‚0 disk.sda1.out‚0 disk.md0p1.out‚ 28.12890625 disk.nvme1n1.in‚4 disk.sda.in‚0 disk.rbd0.out‚0" disk.nvme0n1.out‚ 1595.69140625 disk.sdb2.out‚29.16455078125 disk.rbd0.in‚0 disk.sda2.in‚0 disk.sda2.out‚29.16455078125) disk./.usagePercent‚83.24620934868472 disk.sdb1.out‚0 disk.sdb.out‚29.16455078125* proc.memory.percent‚18.114376068115234 proc.cpu.threads‚17$ disk./.usageGB‚2977.287670135498 disk.dm-0.in‚ 4.2421875 disk.sdb1.in‚0 disk.nvme0n1.in‚ 0.2421875 network.recv‚172508 disk.rbd2.out‚0 disk.sdb.in‚0$ memory_percent‚92.08600521087646) proc.memory.availableMB‚ 1296.62890625 cpu‚25.30336761057382 network.sent‚281639 disk.md0.in‚0 disk.md0.out‚ 28.12890625‚0Š0ê+ _wandbruntime‚266  _runtime‚266ÿÁ…—Ãë½ % _timestamp‚1.7884854674173021e+09 _runtime‚ 266.522000539 _step‚8 & lr-AdamW/pg1‚0.0002805676992122044 & lr-AdamW/pg2‚0.0002805676992122044 trainer/global_step‚149|p¥ÇìÁ trainer/global_step‚149 % _timestamp‚1.7884854674173021e+09 _runtime‚ 266.522000539 _step‚8 & lr-AdamW/pg1‚0.0002805676992122044 & lr-AdamW/pg2‚0.0002805676992122044ø O0í+ _wandbruntime‚266  _runtime‚266Íì‰Ä'î¾O OCDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated‚2776387 P@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 38610163200 SGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚2090855 _runtime‚ 266.522129639 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚18396 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 KBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚2147 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚18396 D8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚4848846 \JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚ 7286322140160 A8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚3680 HDeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚32503993278976 [HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚32151490626560 OCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚2757997 WGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 34283657216 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚22687 A8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚3680 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 D>DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 RBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 40288387072 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚22388 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚3 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚70087680 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚2684 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚6 _MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚ 6840042516992 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 TADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚31244603937280 YHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 352054991280 TDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 40288387072 K?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚2757997 H?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚1533 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 O@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 2541128704 WGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 35913877504 HDeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚32503993278976 [HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚32151490626560 OCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚2757997 WGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 34283657216 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚22687 A8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚3680 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 D>DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 RBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 40288387072 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚22388 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚3 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚70087680 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚2684 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚6 _MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚ 6840042516992 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 TADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚31244603937280 YHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 352054991280 TDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 40288387072 K?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚2757997 H?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚1533 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 O@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 2541128704 WGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 35913877504 HDeviceStatsMonitor.on_train_batch_start/allocation.all.current‚18396 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 KBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚2147 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚18396 D8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚4848846 \JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚ 7286322140160 A8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚3680 H8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 PADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 2539791992 YIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 37073453056 J>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚4899500 TIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚666741 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 I@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚2147 M>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 2541142016 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚18416 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 WHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 2514364024 KBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚1533 J>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚1685497 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚3 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚1018759 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 VCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚31455012525502 TDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 37073453056 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚18422 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚6 QEDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚2794755 MADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚2104739 ?6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚3680 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 [JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 356922491038 MADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚2104745 N>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 36985717077 B9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚3680 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚25427968 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚587 SFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚99172352 aODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚ 6884201939968 NBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚1686087 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚1019346 QEDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚2104745 PEDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚666738 LDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚330 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 963120128 UEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 34283657216 TEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 3140732164 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 _runtime‚ 268.062724158 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚6 ZHDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚ 7333414910464 RBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 35913877504 \KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 449142896128 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 R?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚31452472733510 WGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 37073453056 XIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 3214934016 _step‚10 MADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚2794755 TEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 2515714048 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 KBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚2147 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 B6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚4881078 ODeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 F:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚4899500 BDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 38610163200 UEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 35913877504 TEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 3142090752 FDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 36985717077 B9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚3680 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚25427968 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚587 SFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚99172352 aODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚ 6884201939968 NBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚1686087 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚1019346 QEDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚2104745 PEDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚666738 LDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚330 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 963120128 UEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 34283657216 TEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 3140732164 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 _runtime‚ 268.062724158 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚6 ZHDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚ 7333414910464 RBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 35913877504 \KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 449142896128 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 R?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚31452472733510 WGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 37073453056 XIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 3214934016 _step‚10 MADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚2794755 TEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 2515714048 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 KBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚2147 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 B6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚4881078 ODeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 F:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚4899500 BDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 38610163200 UEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 35913877504 TEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 3142090752 F8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 PADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 2539791992 YIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 37073453056 J>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚4899500 TIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚666741 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 I@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚2147 M>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 2541142016 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚18416 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 WHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 2514364024 KBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚1533 J>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚1685497 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚3 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚1018759 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 VCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚31455012525502 TDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 37073453056 Éþ¡L0ó+ _wandbruntime‚268  _runtime‚268õ’­ïRôÌ trainer/global_step‚149 0loss/FRt4/assignment_loss‚6.6805286089579266 0loss/FBt4/assignment_loss‚0.5232121596960092 .loss/FRt1/detection_loss‚2.999508399516344 /loss/FRt2/detection_loss‚2.6999901639202895 1loss/SRqqt1/detection_loss‚3.6327051128564256 1loss/SRqqt2/detection_loss‚2.4170838574950513 1loss/SRqqt4/detection_loss‚1.7030753199982345 .loss/FBt4/detection_loss‚9.234397115977092 %loss/total_loss‚87.33182886334151 /loss/FRt3/assignment_loss‚6.681150136991989 0loss/FBt1/assignment_loss‚0.6251438341724376 /loss/FRt3/detection_loss‚3.7189701691616412 .loss/FBt3/detection_loss‚6.051769474939424 /loss/FRt1/assignment_loss‚6.635958224534988 0loss/SRqqt3/detection_loss‚3.769253912101309 % _timestamp‚1.7884854689630883e+09 _runtime‚ 268.064239435 .loss/FBt1/detection_loss‚4.982070062061151 _step‚11 2loss/SRqqt2/assignment_loss‚1.9103693684706322 1loss/SRqqt3/assignment_loss‚3.331327449844544 epoch‚0 1loss/FBt2/assignment_loss‚0.14113252627250777 .loss/FRt4/detection_loss‚2.165310310282641 /loss/FBt2/detection_loss‚3.6947808389563157 /loss/FRt2/assignment_loss‚6.653978658934771 1loss/SRqqt1/assignment_loss‚3.222163012858187 0loss/FBt3/assignment_loss‚0.6299532650747486 2loss/SRqqt4/assignment_loss‚3.2279968802668466;á£VõÐ trainer/global_step‚149 0loss/FRt4/assignment_loss‚6.6805286089579266 0loss/FBt4/assignment_loss‚0.5232121596960092 .loss/FRt1/detection_loss‚2.999508399516344 /loss/FRt2/detection_loss‚2.6999901639202895 1loss/SRqqt1/detection_loss‚3.6327051128564256 1loss/SRqqt2/detection_loss‚2.4170838574950513 1loss/SRqqt4/detection_loss‚1.7030753199982345 .loss/FBt4/detection_loss‚9.234397115977092 %loss/total_loss‚87.33182886334151 /loss/FRt3/assignment_loss‚6.681150136991989 0loss/FBt1/assignment_loss‚0.6251438341724376 /loss/FRt3/detection_loss‚3.7189701691616412 .loss/FBt3/detection_loss‚6.051769474939424 /loss/FRt1/assignment_loss‚6.635958224534988 0loss/SRqqt3/detection_loss‚3.769253912101309 % _timestamp‚1.7884854689630883e+09 _runtime‚ 268.064239435 .loss/FBt1/detection_loss‚4.982070062061151 _step‚11 2loss/SRqqt2/assignment_loss‚1.9103693684706322 1loss/SRqqt3/assignment_loss‚3.331327449844544 epoch‚0 1loss/FBt2/assignment_loss‚0.14113252627250777 .loss/FRt4/detection_loss‚2.165310310282641 /loss/FBt2/detection_loss‚3.6947808389563157 /loss/FRt2/assignment_loss‚6.653978658934771 1loss/SRqqt1/assignment_loss‚3.222163012858187 0loss/FBt3/assignment_loss‚0.6299532650747486 2loss/SRqqt4/assignment_loss‚3.2279968802668466 $ƒCþö‚(:ó Þ¾èÔÞª² disk.nvme0n1.in‚ 0.2421875" disk.nvme0n1.out‚ 1718.17578125 disk.dm-0.out‚ 2611.2734375 disk.sdb2.out‚ 31.3037109375 disk.md0.in‚0 disk.nvme1n1.in‚4 disk.rbd0.out‚0 disk.md0.out‚ 30.19140625 disk.md0p1.out‚ 30.19140625! disk.nvme1n1.out‚ 893.09765625* proc.memory.percent‚18.114376068115234 disk.dm-0.in‚ 4.2421875 disk.sdb1.out‚0 disk.rbd0.in‚0 proc.cpu.threads‚17 disk.md0p1.in‚0$ memory_percent‚92.08755493164062( disk./.usagePercent‚83.2532318781196 disk.sda2.out‚ 31.3037109375 disk.sda.in‚0! proc.memory.rssMB‚ 2967.859375 disk.sdb.in‚0 cpu‚25.306296887562066 network.recv‚176870 disk.sdb2.in‚0 disk.sda.out‚ 31.3037109375$ proc.memory.availableMB‚1296.375 disk.sda2.in‚0$ disk./.usageGB‚2977.538829803467 disk.sdb1.in‚0 disk.sda1.in‚0 disk.sda1.out‚0 disk.rbd2.in‚0 disk.rbd2.out‚0 disk.sdb.out‚ 31.3037109375 network.sent‚298509Ïmç”÷‚(:‰ Þ¾èԌԗ² gpu.0.gpu‚97.0 gpu.0.memory‚18+ gpu.0.memoryAllocated‚47.54493713378906* gpu.0.memoryAllocatedBytes‚ 40840790016 gpu.0.temp‚43.0 gpu.0.powerWatts‚242.273' gpu.0.enforcedPowerLimitWatts‚400.0 gpu.0.powerPercent‚60.56825 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0†õˆø‚(:’ í¾èÔàÍ»° gpu.0.gpu‚97.0 gpu.0.memory‚17+ gpu.0.memoryAllocated‚47.54493713378906* gpu.0.memoryAllocatedBytes‚ 40840790016 gpu.0.temp‚45.0 gpu.0.powerWatts‚226.141' gpu.0.enforcedPowerLimitWatts‚400.0( gpu.0.powerPercent‚56.53524999999999 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0Ä™ù‚(:õ í¾èÔÜûݱ disk.nvme1n1.in‚4 disk.sda.out‚32.77587890625 disk.sdb1.in‚0 disk.md0.out‚ 31.609375* proc.memory.percent‚18.114376068115234 network.sent‚337133 disk.nvme0n1.in‚ 0.2421875 disk.sdb1.out‚0 disk.sdb2.out‚32.77587890625 disk.rbd2.in‚0! disk.nvme0n1.out‚ 1834.2734375 disk.dm-0.in‚ 4.2421875 disk.sdb.in‚0 disk.sda2.in‚0 disk.sda.in‚0 disk.rbd2.out‚0 cpu‚25.308462628115752 disk.sda2.out‚32.77587890625 disk.dm-0.out‚ 2830.78125 disk.sda1.in‚0 disk.md0p1.in‚0! proc.memory.rssMB‚ 2967.859375 disk.nvme1n1.out‚ 996.5078125 disk.rbd0.in‚0 disk.rbd0.out‚0 disk.sda1.out‚0$ memory_percent‚92.08807945251465 network.recv‚182501) disk./.usagePercent‚83.25320286645487% disk./.usageGB‚2977.5377922058105 disk.sdb.out‚32.77587890625 disk.sdb2.in‚0 disk.md0.in‚0( proc.memory.availableMB‚ 1296.2890625 proc.cpu.threads‚17 disk.md0p1.out‚ 31.609375ZsH“ú‚(:ˆ ü¾èÔª÷Ž± gpu.0.gpu‚15.0 gpu.0.memory‚5+ gpu.0.memoryAllocated‚47.54493713378906* gpu.0.memoryAllocatedBytes‚ 40840790016 gpu.0.temp‚47.0 gpu.0.powerWatts‚307.655' gpu.0.enforcedPowerLimitWatts‚400.0 gpu.0.powerPercent‚76.91375 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0*“½âû‚(:ô ü¾èÔ^Zñ ëúå ²) disk./.usagePercent‚83.25320286645487 disk.nvme1n1.in‚4 disk.sda.in‚0 disk.sdb1.in‚0 disk.rbd2.in‚0! proc.memory.rssMB‚ 2967.859375 proc.cpu.threads‚17 disk.md0p1.out‚ 32.95703125 disk.sdb.in‚0 disk.nvme1n1.out‚ 1099.984375 disk.sda.out‚ 34.189453125 disk.nvme0n1.in‚ 0.2421875 disk.dm-0.in‚ 4.2421875 disk.dm-0.out‚ 2940.6640625 disk.sdb2.out‚ 34.189453125 disk.sda1.in‚0 disk.sda1.out‚0 disk.md0.in‚0 disk.md0.out‚ 32.95703125! disk.nvme0n1.out‚ 1840.6796875 network.sent‚341472% disk./.usageGB‚2977.5377922058105 disk.rbd2.out‚0 disk.sdb.out‚ 34.189453125 disk.sda2.out‚ 34.189453125) proc.memory.availableMB‚ 1296.29296875* proc.memory.percent‚18.114376068115234 disk.sdb2.in‚0 disk.md0p1.in‚0 disk.rbd0.in‚0 cpu‚25.312608117772125 network.recv‚187598$ memory_percent‚92.08805561065674 disk.sdb1.out‚0 disk.sda2.in‚0 disk.rbd0.out‚0é.o´’ü‚(:‡ ‹¿èÔŽ¯¥° gpu.0.gpu‚98.0 gpu.0.memory‚18+ gpu.0.memoryAllocated‚47.54493713378906* gpu.0.memoryAllocatedBytes‚ 40840790016 gpu.0.temp‚45.0 gpu.0.powerWatts‚219.628' gpu.0.enforcedPowerLimitWatts‚400.0 gpu.0.powerPercent‚54.907 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0¤S®oþý‚(:ó ‹¿èÔ¥ñ¶± disk.sda1.out‚0 disk.rbd2.out‚0 network.sent‚349289" disk.nvme0n1.out‚ 1963.13671875 disk.sda.out‚35.84814453125$ memory_percent‚92.08579063415527 disk.sdb2.in‚0 disk.rbd0.out‚0( proc.memory.availableMB‚ 1296.6640625% disk./.usageGB‚2977.0104942321777 disk.sdb1.out‚0 disk.sdb2.out‚35.84814453125 disk.rbd2.in‚0 disk.sda2.in‚0 disk.sda2.out‚35.84814453125 disk.rbd0.in‚0! proc.memory.rssMB‚ 2967.859375 disk.nvme0n1.in‚ 0.2421875 disk.sdb1.in‚0 disk.md0.in‚0 cpu‚25.3129133030593 network.recv‚191564 disk.dm-0.in‚ 4.2421875 disk.md0.out‚ 34.546875 disk.sdb.out‚35.84814453125 disk.nvme1n1.in‚4 disk.sda.in‚0 disk.dm-0.out‚3140.75 disk.md0p1.out‚ 34.546875 disk.sdb.in‚0" disk.nvme1n1.out‚ 1177.61328125* proc.memory.percent‚18.114376068115234) disk./.usagePercent‚83.23845939442074 disk.sda1.in‚0 disk.md0p1.in‚0 proc.cpu.threads‚17}¢­ þ‚(:þ š¿èÔ¥ö¨´ disk.sda.out‚37.77880859375 disk.dm-0.out‚ 3245.67578125 disk.md0.out‚ 36.41796875 disk.md0p1.in‚0 network.recv‚195937) disk./.usagePercent‚83.23845171486244 disk.nvme0n1.in‚ 0.2421875 disk.sda1.in‚0$ memory_percent‚92.08834171295166 disk.dm-0.in‚ 4.2421875 disk.sda1.out‚0 disk.nvme1n1.in‚4 proc.cpu.threads‚17 disk.sdb1.out‚0 disk.sdb2.out‚37.77880859375" disk.nvme1n1.out‚ 1221.23046875 disk.rbd0.in‚0 disk.rbd0.out‚0 disk.sda.in‚0 network.sent‚355364% disk./.usageGB‚2977.0102195739746 disk.sdb2.in‚0 disk.rbd2.in‚0 disk.sda2.in‚0* proc.memory.percent‚18.114376068115234! disk.nvme0n1.out‚ 2024.4453125 disk.rbd2.out‚0 disk.sdb.out‚37.77880859375 disk.sda2.out‚37.77880859375) proc.memory.availableMB‚ 1296.24609375 cpu‚25.31475620269627 disk.md0.in‚0 disk.md0p1.out‚ 36.41796875 disk.sdb.in‚0 disk.sdb1.in‚0! proc.memory.rssMB‚ 2967.859375E븞ÿ‚(:“ š¿èÔª¦»´ gpu.0.gpu‚65.0 gpu.0.memory‚15+ gpu.0.memoryAllocated‚47.54493713378906* gpu.0.memoryAllocatedBytes‚ 40840790016 gpu.0.temp‚44.0 gpu.0.powerWatts‚234.543' gpu.0.enforcedPowerLimitWatts‚400.0) gpu.0.powerPercent‚58.635749999999994 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0¹jV÷”€‚(:‰ ©¿èÔÔ𬮠gpu.0.gpu‚63.0 gpu.0.memory‚14+ gpu.0.memoryAllocated‚47.54493713378906* gpu.0.memoryAllocatedBytes‚ 40840790016 gpu.0.temp‚41.0 gpu.0.powerWatts‚117.193' gpu.0.enforcedPowerLimitWatts‚400.0 gpu.0.powerPercent‚29.29825 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0ž®·#‚(:˜ ©¿èÔ¸Ž² disk.md0.in‚ 0.0078125( proc.memory.availableMB‚ 1296.1328125 disk.sda.out‚ 39.8017578125 network.sent‚361885 network.recv‚200830 disk.dm-0.out‚ 3434.26171875 disk.sda2.in‚ 0.0078125 disk.sdb1.in‚0 disk.sdb1.out‚0 disk.rbd2.out‚0 disk.sdb.out‚ 39.8017578125 disk.rbd0.in‚0 cpu‚25.31650262303907 disk.dm-0.in‚ 4.2421875 disk.sdb2.in‚0 disk.sda1.out‚0 disk.md0p1.in‚ 0.0078125 disk.nvme1n1.in‚4* proc.memory.percent‚18.114376068115234 disk.sdb2.out‚ 39.8017578125 disk.rbd0.out‚0$ memory_percent‚92.08903312683105! proc.memory.rssMB‚ 2967.859375$ disk./.usageGB‚2977.035732269287 disk.md0.out‚ 38.38671875 disk.sdb.in‚0 disk.sda2.out‚ 39.8017578125" disk.nvme1n1.out‚ 1221.33203125 disk.sda.in‚ 0.0078125! disk.nvme0n1.out‚ 2212.9296875 disk.md0p1.out‚ 38.38671875 disk.rbd2.in‚0 disk.nvme0n1.in‚ 0.2421875 proc.cpu.threads‚17) disk./.usagePercent‚83.23916506050126 disk.sda1.in‚0P„Pš0‚+ _wandbruntime‚353  _runtime‚353nÑ+»ÆƒÀ ' lr-AdamW/pg1‚0.00037471793384717233 ' lr-AdamW/pg2‚0.00037471793384717233 trainer/global_step‚199 % _timestamp‚1.7884855545802293e+09 _runtime‚ 353.682916575 _step‚12À°0Ê„Ä % _timestamp‚1.7884855545802293e+09 _runtime‚ 353.682916575 _step‚12 ' lr-AdamW/pg1‚0.00037471793384717233 ' lr-AdamW/pg2‚0.00037471793384717233 trainer/global_step‚199 *²N_0…+ _wandbruntime‚353  _runtime‚353†Æ»Æ'†ÀO SGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚3696839 VKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚880675 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 KBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚1533 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚18396 SGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚1346736 L@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚2227408 YJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 2515700736 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚6 TADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚41638070920788 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 _runtime‚ 353.683346877 XEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚43321670562304 \JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚ 9640748844032 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 O?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 40288387072 NFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚330 P@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 36985717077 TDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 40288387072 D;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚3680 OCDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated‚3696839 Q>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚43319129433600 _LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚42848819580928 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚18390 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 XIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 3214934016 KBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚2147 RCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 2539791784 D8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚6463798 ]LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 472850981376 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚1347315 _MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚ 9046558766592 ]LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 472253313804 MDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚2147 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚18390 ZIDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 472850981376 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚582 XFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚ 9640649658368 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 XEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚41640610712572 HDeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚22388 O@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 2541128704 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚29097984 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 ZKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 3214934016 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 UHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚99185664 YIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 37073453056 F=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚3680 _step‚13 ^MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 594090891776 [HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚42848794152960 YHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 470335280640 H?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚1533 G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚2684 HDeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 BDeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚22388 O@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 2541128704 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚29097984 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 ZKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 3214934016 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 UHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚99185664 YIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 37073453056 F=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚3680 _step‚13 ^MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 594090891776 [HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚42848794152960 YHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 470335280640 H?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚1533 G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚2684 HDeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 BDeviceStatsMonitor.on_train_batch_start/allocation.all.current‚18396 SGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚1346736 L@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚2227408 YJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 2515700736 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚6 TADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚41638070920788 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 _runtime‚ 353.683346877 XEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚43321670562304 \JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚ 9640748844032 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 O?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 40288387072 NFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚330 P@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 36985717077 TDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 40288387072 D;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚3680 OCDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated‚3696839 Q>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚43319129433600 _LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚42848819580928 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚18390 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 XIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 3214934016 KBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚2147 RCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 2539791784 D8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚6463798 ]LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 472850981376 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚1347315 _MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚ 9046558766592 ]LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 472253313804 MDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚2147 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚18390 ZIDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 472850981376 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚582 XFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚ 9640649658368 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 XEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚41640610712572 H8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 WFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 472691376640 ZHDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚ 9687846705664 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 BDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 _runtime‚ 355.288302356 FDeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 2541142016 J>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚2238504 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚29097984 F=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚1533 I=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚3696791 S@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚43537972742144 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1299 ODeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 R?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚43535431600128 VGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 3214934016 VCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚43062740223488 LDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚976 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 trainer/global_step‚199 QBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 3142090752 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 WFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 472092085722 WHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 2515714048 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚22687 J>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚6514452 QEDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚3715207 aODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚ 9090718189568 UEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 34283657216 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 FDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 36985717077 [JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 474606449746 MADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚2799239 NBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚2239080 I=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚2799239 EDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 38610163200 F:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚6496030 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚2684 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚25427968 YIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 37073453056 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚18422 TEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 3142090752 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚1354039 ?6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚3680 D;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚3680 ?6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚3680 % _timestamp‚1.7884855561863484e+09 MADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚3715207 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0ö_/Ô&ŠÎM VCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚43062740223488 LDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚976 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 trainer/global_step‚199 QBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 3142090752 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 WFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 472092085722 WHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 2515714048 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚22687 J>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚6514452 QEDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚3715207 aODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚ 9090718189568 UEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 34283657216 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 FDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 36985717077 [JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 474606449746 MADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚2799239 NBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚2239080 I=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚2799239 EDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 38610163200 F:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚6496030 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚2684 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚25427968 YIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 37073453056 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚18422 TEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 3142090752 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚1354039 ?6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚3680 D;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚3680 ?6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚3680 % _timestamp‚1.7884855561863484e+09 MADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚3715207 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 PADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 2541142016 YFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚43062740223488 SDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 3214934016 B9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚3680 _step‚14 RBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 35913877504 XGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 475207090688 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚3 XIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 3214934016 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚22687 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚6 \KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 597058441728 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚576 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 WFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 472691376640 ZHDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚ 9687846705664 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 BDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 _runtime‚ 355.288302356 FDeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 2541142016 J>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚2238504 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚29097984 F=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚1533 I=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚3696791 S@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚43537972742144 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1299 ODeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 R?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚43535431600128 VGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 3214934016¹v¼z0‹+ _wandbruntime‚355  _runtime‚355œžÎpTŒÎ /loss/FRt3/assignment_loss‚6.691222190856934 1loss/SRqqt4/detection_loss‚1.9774588202229157 -loss/FBt3/detection_loss‚5.03835708910928 .loss/FRt4/assignment_loss‚6.66317605170883 2loss/SRqqt4/assignment_loss‚3.2125604482797474 /loss/FRt1/detection_loss‚4.9815216312805815 .loss/FRt2/detection_loss‚2.581018400463191 /loss/FRt3/detection_loss‚3.7954896038923507 epoch‚0 0loss/FRt1/assignment_loss‚6.7031494299570715 0loss/FRt2/assignment_loss‚6.6594038876620205 /loss/FBt4/assignment_loss‚1.121189095054433 1loss/SRqqt1/detection_loss‚3.8276916522728768 /loss/FBt2/detection_loss‚1.4025015759798538 %loss/total_loss‚85.43281676779549 _step‚15 2loss/SRqqt2/assignment_loss‚2.0082239159429918 1loss/SRqqt3/detection_loss‚3.5181680334260665 /loss/FBt1/detection_loss‚6.8408617831957645 .loss/FBt4/detection_loss‚6.113477182477919 2loss/SRqqt3/assignment_loss‚2.8910615429820785 trainer/global_step‚199 1loss/SRqqt1/assignment_loss‚2.821931053149073 1loss/FBt1/assignment_loss‚0.28919227144946585 0loss/FBt3/assignment_loss‚0.5942507982699146 _runtime‚ 355.289517082 0loss/FBt2/assignment_loss‚0.2629387251298014 /loss/FRt4/detection_loss‚2.4671774578883365 1loss/SRqqt2/detection_loss‚2.9707941271439946 % _timestamp‚1.7884855561882937e+09‚µš XÒ .loss/FRt4/assignment_loss‚6.66317605170883 2loss/SRqqt4/assignment_loss‚3.2125604482797474 /loss/FRt1/detection_loss‚4.9815216312805815 .loss/FRt2/detection_loss‚2.581018400463191 /loss/FRt3/detection_loss‚3.7954896038923507 epoch‚0 0loss/FRt1/assignment_loss‚6.7031494299570715 0loss/FRt2/assignment_loss‚6.6594038876620205 /loss/FBt4/assignment_loss‚1.121189095054433 1loss/SRqqt1/detection_loss‚3.8276916522728768 /loss/FBt2/detection_loss‚1.4025015759798538 %loss/total_loss‚85.43281676779549 _step‚15 2loss/SRqqt2/assignment_loss‚2.0082239159429918 1loss/SRqqt3/detection_loss‚3.5181680334260665 /loss/FBt1/detection_loss‚6.8408617831957645 .loss/FBt4/detection_loss‚6.113477182477919 2loss/SRqqt3/assignment_loss‚2.8910615429820785 trainer/global_step‚199 1loss/SRqqt1/assignment_loss‚2.821931053149073 1loss/FBt1/assignment_loss‚0.28919227144946585 0loss/FBt3/assignment_loss‚0.5942507982699146 _runtime‚ 355.289517082 0loss/FBt2/assignment_loss‚0.2629387251298014 /loss/FRt4/detection_loss‚2.4671774578883365 1loss/SRqqt2/detection_loss‚2.9707941271439946 % _timestamp‚1.7884855561882937e+09 /loss/FRt3/assignment_loss‚6.691222190856934 1loss/SRqqt4/detection_loss‚1.9774588202229157 -loss/FBt3/detection_loss‚5.03835708910928†"<žŽ‚(:“ ¸¿èÔôèΰ gpu.0.gpu‚98.0 gpu.0.memory‚17+ gpu.0.memoryAllocated‚47.54493713378906* gpu.0.memoryAllocatedBytes‚ 40840790016 gpu.0.temp‚45.0 gpu.0.powerWatts‚231.323' gpu.0.enforcedPowerLimitWatts‚400.0) gpu.0.powerPercent‚57.830749999999995 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0ŠöË‚(:’ ¸¿èÔ䜇± disk.nvme1n1.out‚ 1221.59375! disk.nvme0n1.out‚ 2283.0859375 disk.sda1.in‚0 disk.sda1.out‚0 disk.md0.out‚46.6875 disk.rbd2.out‚0 disk.rbd0.in‚0 disk.sdb1.in‚0 cpu‚25.317303406830675 disk.rbd2.in‚0 disk.md0p1.in‚ 0.0078125 disk.dm-0.in‚ 4.24609375! proc.memory.rssMB‚ 2967.859375 proc.cpu.threads‚17% disk./.usageGB‚2977.0379180908203 disk.sdb.in‚0 disk.sda2.in‚ 0.0078125 disk.sda2.out‚48.16357421875 disk.sdb1.out‚0$ memory_percent‚92.08393096923828) disk./.usagePercent‚83.23922617698616 disk.rbd0.out‚0* proc.memory.percent‚18.114376068115234 network.sent‚366266 disk.md0p1.out‚46.6875 disk.nvme0n1.in‚ 0.24609375 disk.sdb2.in‚0& proc.memory.availableMB‚ 1296.96875 network.recv‚205746 disk.sdb.out‚48.16357421875 disk.nvme1n1.in‚4 disk.sda.in‚ 0.0078125 disk.sda.out‚48.16357421875 disk.dm-0.out‚ 3504.6796875 disk.sdb2.out‚48.16357421875 disk.md0.in‚ 0.0078125Gû€¿”‚(:‰ Ç¿èÔîØÈ° gpu.0.gpu‚99.0 gpu.0.memory‚23+ gpu.0.memoryAllocated‚47.54493713378906* gpu.0.memoryAllocatedBytes‚ 40840790016 gpu.0.temp‚44.0 gpu.0.powerWatts‚194.453' gpu.0.enforcedPowerLimitWatts‚400.0 gpu.0.powerPercent‚48.61325 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0ÿì(‘‚(: Ç¿èÔß¾‰± disk.sda1.in‚0 disk.rbd2.in‚0 cpu‚25.320405575580878 disk.dm-0.in‚ 4.25390625 disk.md0p1.in‚ 0.0078125 disk.rbd0.in‚0 disk.rbd0.out‚0! proc.memory.rssMB‚ 2967.859375 network.sent‚417193) disk./.usagePercent‚83.23838206553496" disk.nvme0n1.out‚ 2421.91796875* proc.memory.percent‚18.114376068115234 disk.md0p1.out‚ 47.8203125 disk.sda2.in‚ 0.0078125! disk.nvme1n1.out‚ 1221.7734375 disk.sda1.out‚0 disk.sdb.out‚49.36083984375 disk.nvme1n1.in‚4 disk.sda.in‚ 0.0078125) proc.memory.availableMB‚ 1296.70703125 disk.sdb2.out‚49.36083984375 disk.rbd2.out‚0 disk.sda2.out‚49.36083984375 disk.nvme0n1.in‚ 0.25390625 proc.cpu.threads‚17 network.recv‚210352 disk.dm-0.out‚ 3643.69140625 disk.sdb1.in‚0 disk.sdb1.out‚0 disk.md0.out‚ 47.8203125 disk.sdb.in‚0# disk./.usageGB‚2977.00772857666 disk.sdb2.in‚0 disk.md0.in‚ 0.0078125 disk.sda.out‚49.36083984375$ memory_percent‚92.08552837371826wýæ?ž’‚(:“ Ö¿èÔ›ôÉ° gpu.0.gpu‚98.0 gpu.0.memory‚17+ gpu.0.memoryAllocated‚47.54493713378906* gpu.0.memoryAllocatedBytes‚ 40840790016 gpu.0.temp‚45.0 gpu.0.powerWatts‚234.448' gpu.0.enforcedPowerLimitWatts‚400.0) gpu.0.powerPercent‚58.611999999999995 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0Õ_rþ“‚(:’ Ö¿èÔ¹†¹± disk.md0.in‚ 0.0078125 disk.sdb.out‚50.70068359375 disk.sda2.out‚50.70068359375 network.recv‚214852 disk.sdb2.out‚50.70068359375 disk.sda1.in‚0 disk.sdb.in‚0 proc.cpu.threads‚17 disk.dm-0.in‚ 4.25390625 disk.sda1.out‚0 disk.md0.out‚49.09375 disk.sda2.in‚ 0.0078125 disk.nvme1n1.in‚4 disk.nvme1n1.out‚ 1221.953125 disk.sda.in‚ 0.0078125 disk.rbd0.in‚0 disk.nvme0n1.in‚ 0.25390625 disk.rbd2.in‚0$ memory_percent‚92.08557605743408 network.sent‚423418 disk.md0p1.in‚ 0.0078125 disk.rbd2.out‚0$ disk./.usageGB‚2976.233352661133 disk.rbd0.out‚0 disk.sdb2.in‚0) proc.memory.availableMB‚ 1296.69921875 disk.sda.out‚50.70068359375 disk.dm-0.out‚3824.625 disk.sdb1.in‚0 disk.md0p1.out‚49.09375* proc.memory.percent‚18.114376068115234 cpu‚25.322634143598698 disk.sdb1.out‚0) disk./.usagePercent‚83.21673019083532 disk.nvme0n1.out‚ 2602.671875! proc.memory.rssMB‚ 2967.859375!·ÿu“”‚(:ˆ å¿èÔǡð gpu.0.gpu‚98.0 gpu.0.memory‚18+ gpu.0.memoryAllocated‚47.54493713378906* gpu.0.memoryAllocatedBytes‚ 40840790016 gpu.0.temp‚45.0 gpu.0.powerWatts‚228.538' gpu.0.enforcedPowerLimitWatts‚400.0 gpu.0.powerPercent‚57.1345 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0iú°Á#•‚(:˜ å¿èÔÀŸ±# memory_percent‚92.0856237411499 disk.sdb1.in‚0 disk.nvme1n1.in‚4* proc.memory.percent‚18.114376068115234 cpu‚25.324224552607728 network.sent‚429626 disk.rbd0.in‚0 disk.rbd0.out‚0 disk.nvme0n1.in‚ 0.25390625 disk.sda1.in‚0 disk.md0.in‚ 0.0078125 disk.md0.out‚ 49.86328125 disk.md0p1.out‚ 49.86328125" disk.nvme0n1.out‚ 2680.30859375 disk.sdb2.out‚ 51.517578125 disk.sda1.out‚0 disk.sda2.out‚ 51.517578125 disk.sdb.in‚0! proc.memory.rssMB‚ 2967.859375 disk.dm-0.in‚ 4.25390625 disk.sdb1.out‚0 disk.sda2.in‚ 0.0078125) proc.memory.availableMB‚ 1296.69140625 proc.cpu.threads‚17 disk.sda.out‚ 51.517578125 disk.sdb.out‚ 51.517578125 network.recv‚219511 disk.sdb2.in‚0 disk.sda.in‚ 0.0078125 disk.dm-0.out‚ 3902.36328125 disk.rbd2.out‚0) disk./.usagePercent‚83.22374941379371% disk./.usageGB‚2976.4843940734863 disk.md0p1.in‚ 0.0078125 disk.rbd2.in‚0! disk.nvme1n1.out‚ 1222.0546875滑)–‚(:ž ô¿èÔþω± disk.sda.out‚52.61083984375 network.recv‚224905# disk./.usageGB‚2976.48335647583 disk.sda1.out‚0 disk.md0p1.out‚ 50.91015625 disk.rbd2.out‚0 disk.rbd0.out‚0 network.sent‚433899 disk.sdb1.in‚0 disk.rbd2.in‚0 disk.sda2.out‚52.61083984375 disk.nvme1n1.in‚4! disk.nvme1n1.out‚ 1222.6015625 proc.cpu.threads‚17 disk.md0p1.in‚ 0.0078125 disk.md0.out‚ 50.91015625* proc.memory.percent‚18.114376068115234 disk.sda2.in‚ 0.0078125" disk.nvme0n1.out‚ 2784.38671875 disk.dm-0.in‚ 4.25390625 disk.sdb1.out‚0! proc.memory.rssMB‚ 2967.859375 cpu‚25.32621178858731 disk.sdb.in‚0 disk.sdb.out‚52.61083984375$ memory_percent‚92.08710193634033 disk.sda1.in‚0 disk.md0.in‚ 0.0078125 disk.sda.in‚ 0.0078125 disk.rbd0.in‚0) disk./.usagePercent‚83.22372040212898 disk.nvme0n1.in‚ 0.25390625 disk.dm-0.out‚ 4006.98828125 disk.sdb2.in‚0 disk.sdb2.out‚52.61083984375) proc.memory.availableMB‚ 1296.44921875rRŸ”—‚(:‰ ô¿èÔý¨°± gpu.0.gpu‚65.0 gpu.0.memory‚15+ gpu.0.memoryAllocated‚47.54493713378906* gpu.0.memoryAllocatedBytes‚ 40840790016 gpu.0.temp‚43.0 gpu.0.powerWatts‚119.701' gpu.0.enforcedPowerLimitWatts‚400.0 gpu.0.powerPercent‚29.92525 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0'|~Ù˜‚(:’ ƒÀèÔÜ›¨° gpu.0.gpu‚97.0 gpu.0.memory‚17+ gpu.0.memoryAllocated‚47.54493713378906* gpu.0.memoryAllocatedBytes‚ 40840790016 gpu.0.temp‚45.0 gpu.0.powerWatts‚227.652' gpu.0.enforcedPowerLimitWatts‚400.0( gpu.0.powerPercent‚56.91299999999999 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0“CJ&™‚(:› ƒÀèÔá•…± network.sent‚439711 disk.md0.out‚ 51.9453125 disk.md0p1.in‚ 0.0078125 disk.md0p1.out‚ 51.9453125 disk.sdb.out‚ 53.7080078125 network.recv‚229063% disk./.usageGB‚2976.4833221435547 disk.md0.in‚ 0.0078125 disk.sda.in‚ 0.0078125 disk.sda.out‚ 53.7080078125 disk.rbd0.in‚0 disk.dm-0.in‚ 4.25390625 disk.sdb2.out‚ 53.7080078125 disk.nvme1n1.in‚4 disk.sda1.out‚0) proc.memory.availableMB‚ 1296.43359375 proc.cpu.threads‚17 disk.rbd2.in‚0 disk.dm-0.out‚ 4110.48828125 disk.sda1.in‚0! proc.memory.rssMB‚ 2967.859375 cpu‚25.327535319561505 disk.nvme0n1.in‚ 0.25390625 disk.sda2.out‚ 53.7080078125) disk./.usagePercent‚83.22371944218419 disk.rbd2.out‚0 disk.rbd0.out‚0$ memory_percent‚92.08719730377197 disk.sdb1.out‚0* proc.memory.percent‚18.114376068115234 disk.sdb.in‚0! disk.nvme0n1.out‚ 2887.7578125" disk.nvme1n1.out‚ 1222.73046875 disk.sdb1.in‚0 disk.sdb2.in‚0 disk.sda2.in‚ 0.0078125š‹0š+ _wandbruntime‚437  _runtime‚437izrÆ›À ' lr-AdamW/pg2‚0.00046886816848214024 trainer/global_step‚249 % _timestamp‚1.7884856385442762e+09 _runtime‚ 437.645608556 _step‚16 ' lr-AdamW/pg1‚0.00046886816848214024*%zÊœÄ trainer/global_step‚249 % _timestamp‚1.7884856385442762e+09 _runtime‚ 437.645608556 _step‚16 ' lr-AdamW/pg1‚0.00046886816848214024 ' lr-AdamW/pg2‚0.00046886816848214024à/•Ó0+ _wandbruntime‚437  _runtime‚437+bšÿœžÆO TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚25427968 OCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚3479849 OCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚4598901 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 UBDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated‚54136806716928 VEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 588167908864 YJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 2515700736 `MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚11253075016192 MDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚1533 % _timestamp‚1.7884856385449758e+09 K?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚4598901 \IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚53546123107328 XEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚54136806716928 HDeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚2684 Q>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚54134265588224 XEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚53546097679360 P@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 38610163200 D>DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 MDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚2147 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 SGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚4617291 BDeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚22388 SGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚1098972 RCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 2539791784 XIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 3214934016 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 PDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚2780191 bQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 741891344384 KBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚1533 trainer/global_step‚249 H?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚1299 TEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 1491343872 A8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚3680 YJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 2514363816 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 A8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚3680 RCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 2541128704 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 [HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚51444114995600 _LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚53546123107328 NFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚976 _LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚51444140423568 RBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 40288387072 O?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 40288387072 O@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 2541128704 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚18390 ^MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 741810770944 XEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚52034077696080 VGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 3140732164 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚18396 SDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 3142090752 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚18396 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 P@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 36985717077 OCDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated‚4617291 YFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚11994885787136 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 VFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 37073453056 L@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚8097146 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚6 H?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚1533 _runtime‚ 437.64651767 ]LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 590683609600 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚22687 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚18390 VGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 3142090752 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚22388 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚80573440 ZKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 3214934016 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚615 D8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚8078750 ZIDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 590683609600 SGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚1680604 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 TADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚52031537904296 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚25427968 TDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 35913877504 YIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 37073453056 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚22687 [HDeviceStatsMonitor.on_tÂ0 rain_batch_start/allocated_bytes.large_pool.freed‚53546097679360 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 D;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚3680 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚1681216 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚1098975 WGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 34283657216 OCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚3479855 ]JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚11994995458560 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚3 YHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 588167908864 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚6 dQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚11253104114176 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 TDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 40288387072 NFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚330 VGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 2515700736 TADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚54134265588224 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚612 ]LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 589937272512 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚25427968 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 H?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚2147 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 109671424 YHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 587422908696 K?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚3479849 M=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 38610163200 WGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 35913877504 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚2684 L@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚2779576 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 963120128 [KDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 37073453056 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚29097984 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 553243648 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 F=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚3680 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0ÑÚzÐ'ŸÊO YHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 588167908864 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚6 dQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚11253104114176 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 TDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 40288387072 NFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚330 VGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 2515700736 TADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚54134265588224 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚612 ]LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 589937272512 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚25427968 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 H?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚2147 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 109671424 YHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 587422908696 K?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚3479849 M=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 38610163200 WGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 35913877504 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚2684 L@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚2779576 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 963120128 [KDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 37073453056 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚29097984 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 553243648 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 F=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚3680 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚25427968 OCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚3479849 OCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚4598901 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 UBDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated‚54136806716928 VEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 588167908864 YJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 2515700736 `MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚11253075016192 MDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚1533 % _timestamp‚1.7884856385449758e+09 K?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚4598901 \IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚53546123107328 XEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚54136806716928 HDeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚2684 Q>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚54134265588224 XEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚53546097679360 P@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 38610163200 D>DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 MDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚2147 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 SGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚4617291 BDeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚22388 SGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚1098972 RCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 2539791784 XIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 3214934016 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 PDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚2780191 bQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 741891344384 KBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚1533 trainer/global_step‚249 H?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚1299 TEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 1491343872 A8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚3680 YJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 2514363816 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 A8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚3680 RCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 2541128704 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 [HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚51444114995600 _LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚53546123107328 NFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚976 _LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚51444140423568 RBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 40288387072 O?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 40288387072 O@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 2541128704 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚18390 ^MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 741810770944 XEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚52034077696080 VGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 3140732164 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚18396 SDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 3142090752 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚18396 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 P@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 36985717077 OCDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated‚4617291 YFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚11994885787136 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 VFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 37073453056 L@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚8097146 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚6 H?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚1533 _runtime‚ 437.64651767 ]LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 590683609600 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚22687 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚18390 VGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 3142090752 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚22388 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚80573440 ZKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 3214934016 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚615 D8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚8078750 ZIDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 590683609600 SGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚1680604 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 TADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚52031537904296 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚25427968 TDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 35913877504 YIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 37073453056 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚22687 [HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚53546097679360 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 D;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚3680 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚1681216 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚1098975 WGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 34283657216 OCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚3479855 ]JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚11994995458560 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚3T˜ø)0 + _wandbruntime‚439  _runtime‚439Ñÿí×&¡ÑM C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚25427968 I@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚2147 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚18416 TEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 3142090752 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚22687 NBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚2791089 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚1687748 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 963120128 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 553243648 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1299 RCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 1491343872 PADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 2541142016 R?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚54350567754752 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚22687 WHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 2515714048 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚2684 MADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚4617243 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚74268672 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 F=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚2147 M>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 2541142016 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚25427968 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚601 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 P@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 40288387072 M=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 40288387072 N>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 38610163200 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 VCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚54353108896768 FDeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 FDeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚8129404 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 SDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 3214934016 B6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚8110982 XGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 593039718912 J>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚2790488 N>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 36985717077 E8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 PADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 2539791992 % _timestamp‚1.7884856400469441e+09 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚6 [HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚12042066182144 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 F:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚8129404 WDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚12041962815488 ^KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚11297205341184 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 WFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 589776044430 KBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚1533 _step‚18 YFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚53760043749888 [JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 593039718912 VCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚53760043749888 QBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 3142090752 VCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚52241946492518 UEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 35913877504 \KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 744757474304 ?6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚3680 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚25427968 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 BDeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 ZGDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚53760069177856 RBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 40288387072 MADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚4635659 TEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 2515714048 bODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚11297234439168 VGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 3214934016 ?6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚3680 _runtime‚ 439.148960137 ODeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 2541142016 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚25427968 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚601 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 P@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 40288387072 M=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 40288387072 N>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 38610163200 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 VCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚54353108896768 FDeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 FDeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚8129404 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 SDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 3214934016 B6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚8110982 XGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 593039718912 J>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚2790488 N>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 36985717077 E8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 PADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 2539791992 % _timestamp‚1.7884856400469441e+09 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚6 [HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚12042066182144 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 F:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚8129404 WDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚12041962815488 ^KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚11297205341184 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 WFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 589776044430 KBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚1533 _step‚18 YFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚53760043749888 [JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 593039718912 VCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚53760043749888 QBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 3142090752 VCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚52241946492518 UEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 35913877504 \KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 744757474304 ?6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚3680 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚25427968 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 BDeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 ZGDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚53760069177856 RBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 40288387072 MADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚4635659 TEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 2515714048 bODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚11297234439168 VGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 3214934016 ?6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚3680 _runtime‚ 439.148960137 ODeviceStatsMonitor.on_train_batch_start/allocation.all.current‚18396 OCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚4174349 A8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚3680 VFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 37073453056 RCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 2541128704 HDeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚22687 RCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 2539791784 H?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚2147 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚18390 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚22388 XEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚64243401205760 WGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 35913877504 P@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 36985717077 XEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚64951942871552 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 _runtime‚ 521.719873792 OCDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated‚5537743 YJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 2515700736 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 MDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚2147 $ _timestamp‚1.788485722618321e+09 Q>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚64949401742848 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚617 YJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 2514363816 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 SGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚2014485 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 TDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 35913877504 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 _step‚21 OCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚5519353 VGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 2515700736 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚18396 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 SDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 3142090752 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 TADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚62425004887804 [HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚61719898020400 G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚2684 ZIDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 708516237824 L@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚3331757 dQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚13459620363776 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 UFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 3214934016 O?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 40288387072 P@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 38610163200 O@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 2541128704 SGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚4174355 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 553243648 KBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚2147 YHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 706000537088 YIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 37073453056 OCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚4174355 `MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚13459591265792 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 ]LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 707621231220 UBDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated‚64951942871552 SGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚5537743 XEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚62427544679588‘Ý![Ð'µÊO E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚22687 RCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 2539791784 H?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚2147 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚18390 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚22388 XEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚64243401205760 WGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 35913877504 P@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 36985717077 XEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚64951942871552 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 _runtime‚ 521.719873792 OCDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated‚5537743 YJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 2515700736 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 MDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚2147 $ _timestamp‚1.788485722618321e+09 Q>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚64949401742848 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚617 YJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 2514363816 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 SGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚2014485 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 TDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 35913877504 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 _step‚21 OCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚5519353 VGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 2515700736 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚18396 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 SDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 3142090752 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 TADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚62425004887804 [HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚61719898020400 G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚2684 ZIDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 708516237824 L@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚3331757 dQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚13459620363776 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 UFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 3214934016 O?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 40288387072 P@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 38610163200 O@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 2541128704 SGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚4174355 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 553243648 KBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚2147 YHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 706000537088 YIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 37073453056 OCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚4174355 `MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚13459591265792 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 ]LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 707621231220 UBDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated‚64951942871552 SGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚5537743 XEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚62427544679588 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚6 HDeviceStatsMonitor.on_train_batch_start/allocation.all.current‚18396 OCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚4174349 A8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚3680 VFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 37073453056 RCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 2541128704 HDeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0šÛ0¶+ _wandbruntime‚523  _runtime‚523´i­e·ÐM N>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 38610163200 LDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚977 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 QEDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚4188245 VGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 3214934016 % _timestamp‚1.7884857242020388e+09 F:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚9744356 MADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚4188239 QBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 3142090752 [HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚14395976852480 _runtime‚ 523.30451048 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚18422 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚18416 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚2684 QEDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚5556111 WGDeviceStatsMonitor.on_trai©äyq"n_batch_end/reserved_bytes.large_pool.current‚ 37073453056 ODeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 VCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚64457347276288 EDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 36985717077 YIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 37073453056 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚607 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚3 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚6 BDeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 MADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚4188245 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 J>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚3342841 \KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 892137214976 R?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚62632873684034 I@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚1533 XGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 710872347136 TDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 37073453056 J>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚9744356 YFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚61925413680896 SDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 3214934016 R?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚65165703909376 ^KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚13503721590784 M=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 40288387072 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚1321641 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 F=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚1533 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚6 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 I=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚5537695 M>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 2541142016 WHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 2515714048 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚22687 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚1321638 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚2021807 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚25427968 ]JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚64457372704256 [JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 709974367162 P@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 40288387072 B9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚3680 F=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚2147 S@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚65168245051392 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚25427968 NBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚3343448 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 963120128 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 VCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚62635413476026 ?6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚3680 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚2021203 bODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚13503750688768 D;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚3680 KBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚2147 KBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚1533 TEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 2515714048 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚22687 MADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚5556111 FDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 VCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚64457347276288 EDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 36985717077 YIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 37073453056 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚607 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚3 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚6 BDeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 MADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚4188245 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 J>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚3342841 \KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 892137214976 R?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚62632873684034 I@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚1533 XGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 710872347136 TDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 37073453056 J>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚9744356 YFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚61925413680896 SDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 3214934016 R?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚65165703909376 ^KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚13503721590784 M=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 40288387072 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚1321641 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 F=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚1533 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚6 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 I=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚5537695 M>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 2541142016 WHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 2515714048 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚22687 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚1321638 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚2021807 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚25427968 ]JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚64457372704256 [JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 709974367162 P@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 40288387072 B9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚3680 F=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚2147 S@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚65168245051392 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚25427968 NBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚3343448 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 963120128 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 VCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚62635413476026 ?6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚3680 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚2021203 bODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚13503750688768 D;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚3680 KBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚2147 KBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚1533 TEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 2515714048 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚22687 MADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚5556111 FDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 38610163200 LDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚977 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 QEDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚4188245 VGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 3214934016 % _timestamp‚1.7884857242020388e+09 F:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚9744356 MADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚4188239 QBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 3142090752 [HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚14395976852480 _runtime‚ 523.30451048 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚18422 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚18416J|*ã0¹+ _wandbruntime‚523  _runtime‚523ýg%èQºË /loss/FBt2/detection_loss‚1.0022044678663529 epoch‚0 1loss/SRqqt2/assignment_loss‚2.239778286681093 0loss/FBt1/assignment_loss‚0.2502899819425571 /loss/FRt3/detection_loss‚2.6337563495886953 /loss/FBt4/detection_loss‚4.5907099576372845 _step‚23 2loss/SRqqt3/assignment_loss‚2.7933507553736368 1loss/SRqqt4/assignment_loss‚2.961758189201355 0loss/SRqqt1/detection_loss‚3.143109541514824 /loss/FRt2/detection_loss‚2.3709815967342127 0loss/SRqqt3/detection_loss‚2.879965976079305 /loss/FBt1/detection_loss‚2.7436075116048997 /loss/FRt3/assignment_loss‚6.015482129548725 0loss/FBt2/assignment_loss‚0.2930963458252171 /loss/FRt1/detection_loss‚2.9352506081263225 -loss/FBt3/detection_loss‚4.65601968102985 $ _timestamp‚1.788485724204937e+09 _runtime‚ 523.306273128 /loss/FRt4/detection_loss‚2.2181288770099323 /loss/FRt1/assignment_loss‚6.350805203119914 /loss/FRt4/assignment_loss‚5.646673485988707 0loss/FBt4/assignment_loss‚0.8234762475874863 1loss/SRqqt2/detection_loss‚1.9552554765651966 /loss/FRt2/assignment_loss‚4.147963901188063 2loss/SRqqt1/assignment_loss‚2.0337950561902134 0loss/FBt3/assignment_loss‚0.4945647899940961 %loss/total_loss‚66.60940490566774 trainer/global_step‚299 1loss/SRqqt4/detection_loss‚1.4293804892698037D‰¦¾U»Ï %loss/total_loss‚66.60940490566774 trainer/global_step‚299 1loss/SRqqt4/detection_loss‚1.4293804892698037 /loss/FBt2/detection_loss‚1.0022044678663529 epoch‚0 1loss/SRqqt2/assignment_loss‚2.239778286681093 0loss/FBt1/assignment_loss‚0.2502899819425571 /loss/FRt3/detection_loss‚2.6337563495886953 /loss/FBt4/detection_loss‚4.5907099576372845 _step‚23 2loss/SRqqt3/assignment_loss‚2.7933507553736368 1loss/SRqqt4/assignment_loss‚2.961758189201355 0loss/SRqqt1/detection_loss‚3.143109541514824 /loss/FRt2/detection_loss‚2.3709815967342127 0loss/SRqqt3/detection_loss‚2.879965976079305 /loss/FBt1/detection_loss‚2.7436075116048997 /loss/FRt3/assignment_loss‚6.015482129548725 0loss/FBt2/assignment_loss‚0.2930963458252171 /loss/FRt1/detection_loss‚2.9352506081263225 -loss/FBt3/detection_loss‚4.65601968102985 $ _timestamp‚1.788485724204937e+09 _runtime‚ 523.306273128 /loss/FRt4/detection_loss‚2.2181288770099323 /loss/FRt1/assignment_loss‚6.350805203119914 /loss/FRt4/assignment_loss‚5.646673485988707 0loss/FBt4/assignment_loss‚0.8234762475874863 1loss/SRqqt2/detection_loss‚1.9552554765651966 /loss/FRt2/assignment_loss‚4.147963901188063 2loss/SRqqt1/assignment_loss‚2.0337950561902134 0loss/FBt3/assignment_loss‚0.4945647899940961(—S“¼‚(:ˆ ÝÀèÔ„ŠÛ° gpu.0.gpu‚82.0 gpu.0.memory‚17+ gpu.0.memoryAllocated‚47.54493713378906* gpu.0.memoryAllocatedBytes‚ 40840790016 gpu.0.temp‚42.0 gpu.0.powerWatts‚156.578' gpu.0.enforcedPowerLimitWatts‚400.0 gpu.0.powerPercent‚39.1445 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0® rå ½‚(:• ÝÀèԛ沱 disk.sda1.out‚0" disk.nvme0n1.out‚ 3456.18359375 disk.dm-0.out‚ 4679.96875 disk.sdb2.out‚ 60.814453125 disk.md0.out‚ 58.72265625 disk.md0.in‚ 0.0078125 disk.sdb.in‚0 disk.sda2.in‚ 0.0078125 disk.sda2.out‚ 60.814453125 disk.sda.in‚ 0.0078125$ memory_percent‚92.08917617797852 disk.md0p1.in‚ 0.0078125' proc.memory.availableMB‚ 1296.109375* proc.memory.percent‚18.114376068115234 disk.md0p1.out‚ 58.72265625 disk.rbd2.in‚0 disk.rbd2.out‚0 disk.nvme1n1.in‚4" disk.nvme1n1.out‚ 1223.78515625 network.sent‚531843 disk.dm-0.in‚ 4.25390625 disk.sda1.in‚0 cpu‚25.336637838622924 network.recv‚255913) disk./.usagePercent‚83.22702922515646 disk.nvme0n1.in‚ 0.25390625 disk.sdb1.in‚0 disk.sdb2.in‚0 disk.sdb.out‚ 60.814453125 disk.sda.out‚ 60.814453125! proc.memory.rssMB‚ 2967.859375% disk./.usageGB‚2976.6016960144043 disk.sdb1.out‚0 disk.rbd0.in‚0 disk.rbd0.out‚0 proc.cpu.threads‚17»Ñy’¾‚(:‡ ìÀèÔ™¤° gpu.0.gpu‚64.0 gpu.0.memory‚12+ gpu.0.memoryAllocated‚47.54493713378906* gpu.0.memoryAllocatedBytes‚ 40840790016 gpu.0.temp‚45.0 gpu.0.powerWatts‚236.616' gpu.0.enforcedPowerLimitWatts‚400.0 gpu.0.powerPercent‚59.154 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0û%¤Ú¿‚(:‡ ìÀèÔ¿Œ”± network.sent‚570729 disk.nvme0n1.in‚ 0.25390625 disk.sdb1.out‚0 disk.rbd2.in‚0 disk.nvme1n1.in‚4 proc.cpu.threads‚17 disk.md0p1.in‚ 0.0078125 disk.sda2.out‚ 61.8046875 disk.sda.in‚ 0.0078125$ memory_percent‚92.08812713623047 disk.sdb.in‚0 disk.sda2.in‚ 0.0078125! proc.memory.rssMB‚ 2967.859375 disk.dm-0.in‚ 4.25390625 disk.sdb2.in‚0 disk.sda1.in‚0 disk.rbd0.in‚0 disk.rbd0.out‚0* proc.memory.percent‚18.114376068115234% disk./.usageGB‚2976.6641960144043" disk.nvme0n1.out‚ 3557.57421875 disk.md0.in‚ 0.0078125 disk.rbd2.out‚0 disk.sdb.out‚ 61.8046875 disk.sda.out‚ 61.8046875 network.recv‚261583) disk./.usagePercent‚83.22877675131474 disk.md0.out‚59.65625& proc.memory.availableMB‚ 1296.28125 cpu‚25.337015763144475 disk.dm-0.out‚ 4781.578125 disk.sdb1.in‚0 disk.sdb2.out‚ 61.8046875 disk.sda1.out‚0 disk.md0p1.out‚59.65625" disk.nvme1n1.out‚ 1224.00390625Œª¢ï’À‚(:‡ ûÀèԆ槰 gpu.0.gpu‚94.0 gpu.0.memory‚22+ gpu.0.memoryAllocated‚47.54493713378906* gpu.0.memoryAllocatedBytes‚ 40840790016 gpu.0.temp‚44.0 gpu.0.powerWatts‚318.176' gpu.0.enforcedPowerLimitWatts‚400.0 gpu.0.powerPercent‚79.544 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0ò‘ÍÁ‚(:“ ûÀèÔ†”‘± disk.sdb2.out‚ 63.01171875 disk.md0p1.out‚ 60.80078125! proc.memory.rssMB‚ 2967.859375 disk.sda.in‚ 0.0078125 disk.sda1.in‚0 disk.nvme1n1.in‚4 disk.rbd0.out‚0 disk.dm-0.out‚ 5106.16015625 disk.sdb1.in‚0 disk.sdb.in‚0 proc.cpu.threads‚17 disk.sda.out‚ 63.01171875 disk.sdb2.in‚0 disk.sda1.out‚0 network.recv‚265392$ disk./.usageGB‚2976.787910461426 disk.rbd2.in‚0 disk.sda2.in‚ 0.0078125$ memory_percent‚92.08827018737793! disk.nvme0n1.out‚ 3882.0234375 disk.dm-0.in‚ 4.25390625 disk.md0.out‚ 60.80078125* proc.memory.percent‚18.114376068115234) disk./.usagePercent‚83.23223585903197 disk.md0.in‚ 0.0078125 disk.rbd2.out‚0 disk.sdb.out‚ 63.01171875 disk.sda2.out‚ 63.01171875( proc.memory.availableMB‚ 1296.2578125 cpu‚25.337353363689495 disk.rbd0.in‚0 disk.nvme0n1.in‚ 0.25390625 disk.md0p1.in‚ 0.0078125 network.sent‚574412" disk.nvme1n1.out‚ 1224.13671875 disk.sdb1.out‚0|b‚(:“ ŠÁèÔ–ê° disk.sda1.in‚0 disk.sda1.out‚0( disk./.usagePercent‚83.2244832382547 disk.dm-0.out‚ 5245.21875 disk.rbd2.in‚0 network.recv‚269702 disk.dm-0.in‚ 4.25390625 disk.sdb2.out‚64.00634765625 disk.md0p1.in‚ 0.0078125 disk.md0p1.out‚ 61.734375( proc.memory.availableMB‚ 1296.3515625 proc.cpu.threads‚17$ disk./.usageGB‚2976.510639190674 disk.sdb1.out‚0 network.sent‚582800 disk.nvme1n1.out‚ 1224.296875 disk.sda.in‚ 0.0078125 disk.sda.out‚64.00634765625 disk.sda2.in‚ 0.0078125 disk.nvme1n1.in‚4 disk.nvme0n1.out‚ 4020.921875 disk.sdb1.in‚0 disk.sdb2.in‚0 disk.rbd0.out‚0 disk.rbd2.out‚0 disk.sdb.in‚0 disk.sdb.out‚64.00634765625! proc.memory.rssMB‚ 2967.859375 cpu‚25.33601809101168 disk.nvme0n1.in‚ 0.25390625 disk.sda2.out‚64.00634765625$ memory_percent‚92.08769798278809 disk.md0.in‚ 0.0078125 disk.md0.out‚ 61.734375 disk.rbd0.in‚0* proc.memory.percent‚18.114376068115234órÂ(:’ ŠÁèÔÏâ° gpu.0.gpu‚98.0 gpu.0.memory‚17+ gpu.0.memoryAllocated‚47.54493713378906* gpu.0.memoryAllocatedBytes‚ 40840790016 gpu.0.temp‚45.0 gpu.0.powerWatts‚241.292' gpu.0.enforcedPowerLimitWatts‚400.0( gpu.0.powerPercent‚60.32300000000001 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0 ,”Ä‚(:‰ ™ÁèÔßÚ£° gpu.0.gpu‚96.0 gpu.0.memory‚16+ gpu.0.memoryAllocated‚47.54493713378906* gpu.0.memoryAllocatedBytes‚ 40840790016 gpu.0.temp‚45.0 gpu.0.powerWatts‚230.813' gpu.0.enforcedPowerLimitWatts‚400.0 gpu.0.powerPercent‚57.70325 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0®ÎžÅ‚(:ø ™ÁèÔÚ¾˜± disk.md0.out‚62.84375 disk.md0.in‚ 0.0078125! disk.nvme1n1.out‚ 1224.3984375$ memory_percent‚92.08769798278809( proc.memory.availableMB‚ 1296.3515625 cpu‚25.336334972683705 disk.sda.in‚ 0.0078125 disk.sda.out‚65.1875 disk.sdb1.in‚0 disk.sdb2.in‚0 disk.sdb.out‚65.1875 network.sent‚589584 network.recv‚275090) disk./.usagePercent‚83.22447566535693 disk.rbd0.in‚0 disk.dm-0.in‚ 4.25390625 disk.rbd2.in‚0* proc.memory.percent‚18.114376068115234 disk.rbd2.out‚0 disk.sda2.out‚65.1875$ disk./.usageGB‚2976.510368347168 disk.sdb2.out‚65.1875 disk.md0p1.in‚ 0.0078125 disk.nvme0n1.out‚ 4125.8125 disk.rbd0.out‚0 disk.sdb.in‚0 disk.sda2.in‚ 0.0078125! proc.memory.rssMB‚ 2967.859375 proc.cpu.threads‚17 disk.dm-0.out‚ 5350.2109375 disk.sdb1.out‚0 disk.sda1.in‚0 disk.sda1.out‚0 disk.md0p1.out‚62.84375 disk.nvme1n1.in‚4 disk.nvme0n1.in‚ 0.25390625AMõ”Æ‚(:‰ ¨ÁèÔ’Îù¯ gpu.0.gpu‚72.0 gpu.0.memory‚17+ gpu.0.memoryAllocated‚47.54493713378906* gpu.0.memoryAllocatedBytes‚ 40840790016 gpu.0.temp‚48.0 gpu.0.powerWatts‚306.375' gpu.0.enforcedPowerLimitWatts‚400.0 gpu.0.powerPercent‚76.59375 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0©EÿQ#Ç‚(:˜ ¨ÁèÔˆíË° disk.sdb2.out‚ 66.2568359375 disk.nvme1n1.in‚4 disk.dm-0.out‚ 5545.31640625 disk.sdb1.in‚0$ memory_percent‚92.09375381469727' proc.memory.availableMB‚ 1295.359375! disk.nvme1n1.out‚ 1296.4609375 disk.sda.out‚ 66.2568359375 disk.sdb1.out‚0 disk.sda1.in‚0! proc.memory.rssMB‚ 2967.859375 proc.cpu.threads‚17 disk.md0p1.in‚ 0.0078125 disk.sdb.in‚0 disk.sda2.in‚ 0.0078125 disk.sdb2.in‚0% disk./.usageGB‚2976.5359077453613 disk.rbd0.in‚0 disk.nvme0n1.in‚ 0.25390625 network.recv‚280284 disk.sdb.out‚ 66.2568359375 disk.sda.in‚ 0.0078125" disk.nvme0n1.out‚ 4248.85546875 disk.dm-0.in‚ 4.25390625* proc.memory.percent‚18.114376068115234 disk.sda2.out‚ 66.2568359375 disk.rbd0.out‚0 disk.sda1.out‚0 network.sent‚593947) disk./.usagePercent‚83.22518975761949 disk.md0p1.out‚ 63.8671875 cpu‚25.33751044561189 disk.md0.in‚ 0.0078125 disk.md0.out‚ 63.8671875 disk.rbd2.in‚0 disk.rbd2.out‚0TB.Ñ0È+ _wandbruntime‚608  _runtime‚608”k‡«ÁÉ» % lr-AdamW/pg1‚0.000657168637752076 % lr-AdamW/pg2‚0.000657168637752076 trainer/global_step‚349 $ _timestamp‚1.788485809200826e+09 _runtime‚ 608.302276822 _step‚24L“éÅÊ¿ _runtime‚ 608.302276822 _step‚24 % lr-AdamW/pg1‚0.000657168637752076 % lr-AdamW/pg2‚0.000657168637752076 trainer/global_step‚349 $ _timestamp‚1.788485809200826e+09û_¥j0Ë+ _wandbruntime‚608  _runtime‚608ILgOÌÌO WKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚1535575 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 BDeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚75764537897472 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚22388 trainer/global_step‚349 UFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 3214934016 _LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚74940730160128 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 TADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚72818471871312 XIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 3214934016 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚18390 M@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚11327050 YIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 37073453056 D;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚3680 E;DeviceStatsMonitor.on_train_ba£rƒ tch_start/allocation.all.peak‚22687 PDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚3886461 A8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚3680 ]LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 825305189928 OCDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated‚6458195 TDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 35913877504 OCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚6439805 UHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚97088512 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 963120128 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚25427968 F=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚3680 [KDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 37073453056 E8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚11308654 [HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚74940704732160 VGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 3142090752 SGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚6458195 % _timestamp‚1.7884858092021337e+09 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚6 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚22388 XEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚75767079026176 TDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 40288387072 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 KBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚2147 K?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚6439805 D>DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 RBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 40288387072 OCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚4868855 XEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚74940704732160 RCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 2539791784 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚22687 G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚2684 SGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚4868855 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚18390 H?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚1300 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚2350886 _runtime‚ 608.303710929 _step‚25 IDeviceStatsMonitor.on_train_batch_start/allocation.all.current‚18396 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚6 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚3 dQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚15666136613376 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 YFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚16703245266432 [HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚71995681045200 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 H?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚2147 ZIDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 826348866048 `MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚15666107515392 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 L@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚3885892 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚566 A8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚3680ò&ËÖ'ÍÐO BDeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚75764537897472 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚22388 trainer/global_step‚349 UFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 3214934016 _LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚74940730160128 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 TADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚72818471871312 XIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 3214934016 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚18390 M@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚11327050 YIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 37073453056 D;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚3680 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚22687 PDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚3886461 A8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚3680 ]LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 825305189928 OCDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated‚6458195 TDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 35913877504 OCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚6439805 UHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚97088512 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 963120128 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚25427968 F=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚3680 [KDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 37073453056 E8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚11308654 [HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚74940704732160 VGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 3142090752 SGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚6458195 % _timestamp‚1.7884858092021337e+09 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚6 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚22388 XEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚75767079026176 TDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 40288387072 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 KBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚2147 K?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚6439805 D>DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 RBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 40288387072 OCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚4868855 XEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚74940704732160 RCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 2539791784 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚22687 G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚2684 SGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚4868855 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚18390 H?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚1300 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚2350886 _runtime‚ 608.303710929 _step‚25 IDeviceStatsMonitor.on_train_batch_start/allocation.all.current‚18396 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚6 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚3 dQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚15666136613376 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 YFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚16703245266432 [HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚71995681045200 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 H?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚2147 ZIDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 826348866048 `MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚15666107515392 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 L@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚3885892 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚566 A8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚3680 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚1535575 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0…ϬÝ0Î+ _wandbruntime‚609  _runtime‚609€óK”Ü&ÏÖM C6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚11340886 LDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚330 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 WHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 2514364024 G:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚11359308 MADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚4882745 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚22687 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚560 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚29097984 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 553243648 _runtime‚ 609.869910557 QEDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚4882745 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 N>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 36985717077 S@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚75983381206016 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚18416 SFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚97075200 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 K;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 38610163200 N>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 38610163200 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚6 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 M=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 40288387072 VGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 3214934016 VCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚75154650802688 UEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 35913877504 QEDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚6476563 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚1539941 LDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚977 ^KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚15710237840384 VCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚73028880459534 RCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 1493440000 PADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 2539791992 R?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚75980840064000 ?6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚3680 WGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 37073453056 ZGDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚75154676230656 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 UEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 34283657216 RBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 40288387072 M>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 2541142016 QBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 3142090752 YFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚72201196705696 I@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚2147 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚25427968 K>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚11359308 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 963120128 VCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚75983381206016 P@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 40288387072 _step‚26 I=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚4882739 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚2684 aODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 1040171831296 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚67977216 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 KBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚2147 G:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚11340886 WDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚16750341694464 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 MADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚6458147 trainer/global_step‚349 FDeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚3897017 ]KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 1040103854080 MADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚6476563 TEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 2515714048 F=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚2147 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚18422 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚1539938 ]JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚72201222133664 TDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 37073453056 SDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 3214934016 D;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚3680 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚18416 I=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚6458147 TEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 3140732164 B9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚3680 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1300 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚2357639 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 NBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚3897580 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 TCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 826189261312 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚22388 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚6 WFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 826189261312 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚563 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚25427968 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 PADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 2541142016 TEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 3142090752 [HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚16750438769664 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 R?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚73026340667542 I@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚1533 % _timestamp‚1.7884858107669616e+09 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚3 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 [JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 827658325870 YIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 37073453056 ODeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚3897017 ]KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 1040103854080 MADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚6476563 TEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 2515714048 F=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚2147 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚18422 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚1539938 ]JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚72201222133664 TDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 37073453056 SDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 3214934016 D;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚3680 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚18416 I=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚6458147 TEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 3140732164 B9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚3680 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1300 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚2357639 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 NBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚3897580 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 TCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 826189261312 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚22388 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚6 WFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 826189261312 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚563 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚25427968 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 PADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 2541142016 TEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 3142090752 [HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚16750438769664 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 R?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚73026340667542 I@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚1533 % _timestamp‚1.7884858107669616e+09 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚3 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 [JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 827658325870 YIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 37073453056 ODeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 36985717077 S@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚75983381206016 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚18416 SFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚97075200 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 K;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 38610163200 N>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 38610163200 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚6 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 M=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 40288387072 VGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 3214934016 VCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚75154650802688 UEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 35913877504 QEDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚6476563 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚1539941 LDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚977 ^KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚15710237840384 VCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚73028880459534 RCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 1493440000 PADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 2539791992 R?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚75980840064000 ?6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚3680 WGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 37073453056 ZGDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚75154676230656 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 UEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 34283657216 RBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 40288387072 M>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 2541142016 QBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 3142090752 YFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚72201196705696 I@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚2147 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚25427968 K>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚11359308 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 963120128 VCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚75983381206016 P@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 40288387072 _step‚26 I=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚4882739 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚2684 aODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 1040171831296 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚67977216 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 KBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚2147@½Ä0Ñ+ _wandbruntime‚609  _runtime‚609ó‡PÐRÒÌ trainer/global_step‚349 /loss/FRt1/assignment_loss‚5.474980930848555 2loss/SRqqt3/assignment_loss‚2.5887296443254177 1loss/FBt2/assignment_loss‚0.27882975095272344 0loss/SRqqt2/detection_loss‚1.412776098266911 /loss/FRt1/detection_loss‚2.8592927726832302 /loss/FRt4/assignment_loss‚5.024143578789451 0loss/FBt3/assignment_loss‚0.7115365021995136 1loss/SRqqt1/detection_loss‚2.6658134470725883 .loss/FBt3/detection_loss‚3.841625843729292 .loss/FRt3/detection_loss‚2.856731738249461 /loss/FBt2/detection_loss‚0.7613711131181907 epoch‚0 % _timestamp‚1.7884858107705758e+09 0loss/FRt2/assignment_loss‚3.1349902575214705 0loss/SRqqt4/detection_loss‚1.798405641893836 .loss/FBt4/detection_loss‚4.736053729057312 %loss/total_loss‚61.90360858770837 /loss/FRt3/assignment_loss‚5.472204942703247 2loss/SRqqt1/assignment_loss‚2.4333130566981334 1loss/SRqqt2/assignment_loss‚2.384677655661284 0loss/FBt1/assignment_loss‚0.5004148496410222 /loss/FRt2/detection_loss‚2.1779929040931165 _step‚27 2loss/SRqqt4/assignment_loss‚2.5939058591849613 0loss/FBt4/assignment_loss‚0.7389077682048082 .loss/FRt4/detection_loss‚2.321672930365259 0loss/SRqqt3/detection_loss‚2.539894700050354 /loss/FBt1/detection_loss‚2.5953428723982404 _runtime‚ 609.871907486(u–)VÓÐ /loss/FRt3/assignment_loss‚5.472204942703247 2loss/SRqqt1/assignment_loss‚2.4333130566981334 1loss/SRqqt2/assignment_loss‚2.384677655661284 0loss/FBt1/assignment_loss‚0.5004148496410222 /loss/FRt2/detection_loss‚2.1779929040931165 _step‚27 2loss/SRqqt4/assignment_loss‚2.5939058591849613 0loss/FBt4/assignment_loss‚0.7389077682048082 .loss/FRt4/detection_loss‚2.321672930365259 0loss/SRqqt3/detection_loss‚2.539894700050354 /loss/FBt1/detection_loss‚2.5953428723982404 _runtime‚ 609.871907486 trainer/global_step‚349 /loss/FRt1/assignment_loss‚5.474980930848555 2loss/SRqqt3/assignment_loss‚2.5887296443254177 1loss/FBt2/assignment_loss‚0.27882975095272344 0loss/SRqqt2/detection_loss‚1.412776098266911 /loss/FRt1/detection_loss‚2.8592927726832302 /loss/FRt4/assignment_loss‚5.024143578789451 0loss/FBt3/assignment_loss‚0.7115365021995136 1loss/SRqqt1/detection_loss‚2.6658134470725883 .loss/FBt3/detection_loss‚3.841625843729292 .loss/FRt3/detection_loss‚2.856731738249461 /loss/FBt2/detection_loss‚0.7613711131181907 epoch‚0 % _timestamp‚1.7884858107705758e+09 0loss/FRt2/assignment_loss‚3.1349902575214705 0loss/SRqqt4/detection_loss‚1.798405641893836 .loss/FBt4/detection_loss‚4.736053729057312 %loss/total_loss‚61.90360858770837•u{¯žÔ‚(:“ ·ÁèÔÂÞ° gpu.0.gpu‚98.0 gpu.0.memory‚18+ gpu.0.memoryAllocated‚47.54493713378906* gpu.0.memoryAllocatedBytes‚ 40840790016 gpu.0.temp‚42.0 gpu.0.powerWatts‚119.368' gpu.0.enforcedPowerLimitWatts‚400.0) gpu.0.powerPercent‚29.841999999999995 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0Ý|mAÕ‚(:‡ ·ÁèÔŠ§ž² disk.dm-0.in‚ 4.25390625 disk.sdb1.out‚0 disk.sdb2.in‚0 disk.rbd0.out‚0* proc.memory.percent‚18.114376068115234 disk.md0.in‚ 0.0078125 disk.md0.out‚ 65.1015625 disk.sda2.in‚ 0.0078125 disk.sdb1.in‚0 disk.rbd2.in‚0 disk.nvme1n1.in‚4 disk.sda.out‚ 67.5546875) disk./.usagePercent‚83.23922852351788 disk.nvme0n1.in‚ 0.25390625 disk.sda1.out‚0 disk.md0p1.in‚ 0.0078125 disk.md0p1.out‚ 65.1015625 disk.rbd2.out‚0 disk.nvme1n1.out‚ 1298.828125 network.sent‚599802# disk./.usageGB‚2977.03800201416 disk.sda1.in‚0 disk.sda2.out‚ 67.5546875 disk.sda.in‚ 0.0078125! disk.nvme0n1.out‚ 4310.1015625 disk.dm-0.out‚ 5608.9296875 disk.sdb2.out‚ 67.5546875' proc.memory.availableMB‚ 1296.015625 proc.cpu.threads‚17 disk.sdb.in‚0 disk.sdb.out‚ 67.5546875 disk.rbd0.in‚0$ memory_percent‚92.08974838256836! proc.memory.rssMB‚ 2967.859375 cpu‚25.33688542981361 network.recv‚284555} “Ö‚(:ˆ ÆÁèÔêϤ® gpu.0.gpu‚30.0 gpu.0.memory‚5+ gpu.0.memoryAllocated‚47.54493713378906* gpu.0.memoryAllocatedBytes‚ 40840790016 gpu.0.temp‚44.0 gpu.0.powerWatts‚222.027' gpu.0.enforcedPowerLimitWatts‚400.0 gpu.0.powerPercent‚55.50675 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0Š(&(ׂ(: ÆÁèÔ»ªÝ® disk.nvme1n1.out‚1376.625 disk.sdb1.in‚0 disk.md0p1.out‚ 66.1171875 cpu‚25.337469379029116 disk.rbd2.in‚0 disk.nvme1n1.in‚4 disk.sda1.out‚0 disk.sdb.in‚0 disk.sda.out‚68.63134765625 disk.rbd0.in‚0 disk.dm-0.in‚ 4.25390625 disk.dm-0.out‚5748.25 network.recv‚289638% disk./.usageGB‚2977.1629638671875 disk.md0.in‚ 0.0078125 disk.rbd2.out‚0 disk.sda.in‚ 0.0078125 disk.md0.out‚ 66.1171875) proc.memory.availableMB‚ 1295.94140625* proc.memory.percent‚18.114376068115234 disk.sda2.out‚68.63134765625 disk.nvme0n1.out‚4371.625 disk.sda1.in‚0 proc.cpu.threads‚17 disk.sdb1.out‚0 disk.md0p1.in‚ 0.0078125$ memory_percent‚92.09020137786865! proc.memory.rssMB‚ 2967.859375 disk.sdb.out‚68.63134765625 disk.rbd0.out‚0 disk.nvme0n1.in‚ 0.25390625 disk.sdb2.in‚0 disk.sdb2.out‚68.63134765625 network.sent‚648744) disk./.usagePercent‚83.24272250922913 disk.sda2.in‚ 0.0078125våj”Ø‚(:‰ ÕÁèÔ¡Ä™° gpu.0.gpu‚71.0 gpu.0.memory‚17+ gpu.0.memoryAllocated‚47.54493713378906* gpu.0.memoryAllocatedBytes‚ 40840790016 gpu.0.temp‚45.0 gpu.0.powerWatts‚141.087' gpu.0.enforcedPowerLimitWatts‚400.0 gpu.0.powerPercent‚35.27175 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0¦r ¦)Ù‚(:ž ÕÁèÔþ›Ï± network.sent‚654950 disk.md0.in‚ 0.0078125 disk.md0.out‚ 67.22265625 disk.rbd2.out‚0 disk.sda.in‚ 0.0078125$ memory_percent‚92.08979606628418 disk.nvme0n1.in‚ 0.25390625 disk.sdb2.out‚69.79345703125 disk.sda1.in‚0 disk.sda1.out‚0 disk.md0p1.in‚ 0.0078125* proc.memory.percent‚18.114376068115234) disk./.usagePercent‚83.24621702824302$ disk./.usageGB‚2977.287944793701 disk.nvme0n1.out‚ 4494.078125 disk.sdb.out‚69.79345703125! proc.memory.rssMB‚ 2967.859375 proc.cpu.threads‚17 disk.sdb.in‚0 disk.sda2.in‚ 0.0078125 disk.nvme1n1.in‚4 disk.rbd0.in‚0( proc.memory.availableMB‚ 1296.0078125 cpu‚25.337895252371343 disk.dm-0.out‚ 5974.08984375 disk.md0p1.out‚ 67.22265625 disk.rbd2.in‚0 disk.rbd0.out‚0 network.recv‚294538 disk.sdb1.in‚0 disk.sdb1.out‚0 disk.sda2.out‚69.79345703125" disk.nvme1n1.out‚ 1480.01171875 disk.sda.out‚69.79345703125 disk.dm-0.in‚ 4.25390625 disk.sdb2.in‚0ÖGè]žÚ‚(:“ äÁèÔ›Çâ® gpu.0.gpu‚98.0 gpu.0.memory‚17+ gpu.0.memoryAllocated‚47.54493713378906* gpu.0.memoryAllocatedBytes‚ 40840790016 gpu.0.temp‚45.0 gpu.0.powerWatts‚236.007' gpu.0.enforcedPowerLimitWatts‚400.0) gpu.0.powerPercent‚59.001749999999994 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0Yf¶ŸÛ‚(:‘ äÁèÔ¡Á˜¯ network.recv‚299234 disk.sdb1.in‚0 disk.md0p1.in‚ 0.0078125 disk.md0p1.out‚68.40625 disk.sdb.out‚ 71.0185546875 disk.nvme1n1.in‚4 disk.nvme1n1.out‚ 1516.34375 disk.rbd0.in‚0 network.sent‚660970 disk.sdb1.out‚0 disk.sdb.in‚0 disk.sda.in‚ 0.0078125* proc.memory.percent‚18.114376068115234 cpu‚25.337617608615666 disk.dm-0.in‚ 4.25390625 disk.md0.out‚68.40625 disk.sda1.in‚0 disk.sda2.in‚ 0.0078125) disk./.usagePercent‚83.25323561123822 disk.md0.in‚ 0.0078125 disk.rbd2.out‚0 disk.sda2.out‚ 71.0185546875$ memory_percent‚92.09132194519043$ disk./.usageGB‚2977.538963317871! disk.nvme0n1.out‚ 4555.3828125 disk.dm-0.out‚ 6071.7265625 disk.sdb2.in‚0 disk.sda1.out‚0 disk.rbd0.out‚0( proc.memory.availableMB‚ 1295.7578125! proc.memory.rssMB‚ 2967.859375 disk.nvme0n1.in‚ 0.25390625 disk.sdb2.out‚ 71.0185546875 disk.rbd2.in‚0 disk.sda.out‚ 71.0185546875 proc.cpu.threads‚17mQÜÜ‚(:’ óÁèÔ‚½—° gpu.0.gpu‚89.0 gpu.0.memory‚20+ gpu.0.memoryAllocated‚47.54493713378906* gpu.0.memoryAllocatedBytes‚ 40840790016 gpu.0.temp‚45.0 gpu.0.powerWatts‚243.83' gpu.0.enforcedPowerLimitWatts‚400.0) gpu.0.powerPercent‚60.957499999999996 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0¼ÙËþÝ‚(:Ž óÁèÔþˆÊ° cpu‚25.338134870349815 disk.sdb1.in‚0 disk.sdb1.out‚0$ memory_percent‚92.09389686584473 disk.dm-0.in‚ 4.25390625 disk.sdb2.in‚0 disk.md0.in‚ 0.0078125! proc.memory.rssMB‚ 2967.859375$ disk./.usageGB‚2977.537883758545 disk.sda2.out‚72.16357421875 disk.rbd0.out‚0 disk.dm-0.out‚ 6200.359375 disk.sdb.in‚0) disk./.usagePercent‚83.25320542630764 disk.sda.in‚ 0.0078125 disk.nvme0n1.in‚ 0.25390625 disk.rbd2.out‚0 network.sent‚666395 network.recv‚302714 disk.sda1.in‚0 disk.sda1.out‚0 disk.md0.out‚69.5 disk.md0p1.in‚ 0.0078125* proc.memory.percent‚18.114376068115234 disk.sdb2.out‚72.16357421875 proc.cpu.threads‚17 disk.nvme1n1.in‚4 disk.rbd0.in‚0! disk.nvme0n1.out‚ 4616.6328125 disk.rbd2.in‚0 disk.sdb.out‚72.16357421875 disk.sda2.in‚ 0.0078125! disk.nvme1n1.out‚ 1583.7265625 disk.sda.out‚72.16357421875 disk.md0p1.out‚69.5( proc.memory.availableMB‚ 1295.3359375/LÞ‚(:’ ‚ÂèÔĊа gpu.0.gpu‚97.0 gpu.0.memory‚18+ gpu.0.memoryAllocated‚47.54493713378906* gpu.0.memoryAllocatedBytes‚ 40840790016 gpu.0.temp‚45.0 gpu.0.powerWatts‚245.672' gpu.0.enforcedPowerLimitWatts‚400.0( gpu.0.powerPercent‚61.41799999999999 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0R›TÎ*ß‚(:Ÿ ‚ÂèÔ™Îϱ" disk.nvme0n1.out‚ 4677.90234375 disk.md0.out‚ 70.58203125 disk.rbd2.in‚0 disk.rbd2.out‚0! disk.nvme1n1.out‚ 1764.6484375 disk.sda.in‚ 0.0078125 disk.dm-0.in‚ 4.25390625 disk.sda1.out‚0 cpu‚25.336909924930474 disk.sdb2.in‚0 disk.md0p1.out‚ 70.58203125 disk.sda2.out‚73.30322265625$ memory_percent‚92.09389686584473 disk.nvme1n1.in‚4 disk.rbd0.in‚0 disk.sdb1.out‚0 disk.md0p1.in‚ 0.0078125 proc.cpu.threads‚17 network.recv‚306910 disk.nvme0n1.in‚ 0.25390625 disk.sda1.in‚0( proc.memory.availableMB‚ 1295.3359375) disk./.usagePercent‚83.25320382639966 disk.sda.out‚73.30322265625 disk.dm-0.out‚ 6442.55078125 disk.md0.in‚ 0.0078125! proc.memory.rssMB‚ 2967.859375 network.sent‚674516$ disk./.usageGB‚2977.537826538086 disk.rbd0.out‚0 disk.sdb.in‚0 disk.sdb.out‚73.30322265625 disk.sda2.in‚ 0.0078125* proc.memory.percent‚18.114376068115234 disk.sdb1.in‚0 disk.sdb2.out‚73.30322265625ëý«0à+ _wandbruntime‚695  _runtime‚695‡Ó~´Âá¼ % _timestamp‚1.7884858962522895e+09 _runtime‚ 695.353642602 _step‚28 % lr-AdamW/pg1‚0.000751318872387044 % lr-AdamW/pg2‚0.000751318872387044 trainer/global_step‚399"[I»ÆâÀ trainer/global_step‚399 % _timestamp‚1.7884858962522895e+09 _runtime‚ 695.353642602 _step‚28 % lr-AdamW/pg1‚0.000751318872387044 % lr-AdamW/pg2‚0.000751318872387044Dà µ0ã+ _wandbruntime‚695  _runtime‚695Ô1‘Ó'äÍO O@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 2541128704 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚6 PDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚4440600 VFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 37073453056 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚22687 OCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚7360257 TADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚86579674052096 OCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚5563349 H?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚1300 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 K?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚7360257 XEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚85638008258560 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚22687 ]LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 942989148636 VGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 3140732164 D;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚3680 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 H?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚1533 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 \IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚85638033686528 ZIDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 944181494272 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚2684 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 _runtime‚ 695.354934338 OCDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated‚7378647 _LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚85638033686528 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚25427968 D>DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 TADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚83211938854820 NFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚330 WGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 34283657216 O?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 40288387072 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 VGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 2515700736 M@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚12942002 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚29097984 ]JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚19057759197184 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 MDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚2147 RCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 2541128704 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 963120128 P@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 36985717077 Q>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚86579674052096 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚1753875 NFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚977 cQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 1185106334208 BDeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚22388 YJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 2515700736 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚18390 SGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚2686105 A8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚3680 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 [HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚85638008258560 VGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 3142090752 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 YHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 940474784820 UFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 3214934016 MDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚1533 G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚2684 TDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 40288387072 SGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚1753872 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚74281984 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚25427968 RBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 40288387072 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 K?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚5563349 SGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚5563355 L@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚4439977 dQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚17872652862976 _MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 1185032052224 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 WGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 35913877504 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚18396 XIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 3214934016 SDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 3142090752 E8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚12923606 SGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚7378647 KBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚1533 P@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 38610163200 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚25427968 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 XEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚86582215180800 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 553472512 YIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 37073453056 A8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚3680 OCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚5563355 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚3 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 [KDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 37073453056 TDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 35913877504 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚623 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚620 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 103379968G„ŸmåÑO VGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 3140732164 D;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚3680 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 H?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚1533 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 \IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚85638033686528 ZIDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 944181494272 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚2684 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 _runtime‚ 695.354934338 OCDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated‚7378647 _LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚85638033686528 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚25427968 D>DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 TADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚83211938854820 NFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚330 WGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 34283657216 O?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 40288387072 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 VGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 2515700736 M@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚12942002 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚29097984 ]JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚19057759197184 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 MDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚2147 RCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 2541128704 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 963120128 P@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 36985717077 Q>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚86579674052096 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚1753875 NFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚977 cQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 1185106334208 BDeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚22388 YJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 2515700736 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚18390 SGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚2686105 A8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚3680 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 [HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚85638008258560 VGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 3142090752 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 YHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 940474784820 UFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 3214934016 MDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚1533 G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚2684 TDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 40288387072 SGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚1753872 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚74281984 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚25427968 RBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 40288387072 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 K?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚5563349 SGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚5563355 L@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚4439977 dQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚17872652862976 _MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 1185032052224 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 WGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 35913877504 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚18396 XIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 3214934016 SDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 3142090752 E8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚12923606 SGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚7378647 KBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚1533 P@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 38610163200 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚25427968 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 XEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚86582215180800 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 553472512 YIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 37073453056 A8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚3680 OCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚5563355 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚3 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 [KDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 37073453056 TDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 35913877504 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚623 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚620 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 103379968 O@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 2541128704 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚6 PDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚4440600 VFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 37073453056 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚22687 OCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚7360257 TADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚86579674052096 OCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚5563349 H?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚1300 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 K?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚7360257 XEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚85638008258560 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚22687 ]LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 942989148636)§v80æ+ _wandbruntime‚697  _runtime‚697¤\½Ý&ç×M ?6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚3680 G:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚12974260 FDeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚4451105 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 963120128 TEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 3140732164 P@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 40288387072 MADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚5577239 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 I=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚7378599 [HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚19104857843200 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 R?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚83419807651050 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚22687 MADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚7397015 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚18416 LDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚977 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 YFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚85851954329088 I@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚1533 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1300 KBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚2147 I@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚2147 VCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚86798517360640 MADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚5577245 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚25427968 ]JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚85851979757056 WFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 944021889536 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 [JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 945342284578 R?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚86795976218624 trainer/global_step‚399 C6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚12955838 ZGDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚85851979757056 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 [JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 946537603584 ]KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 1188002483712 WGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 37073453056 PADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 2541142016 UEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 35913877504 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚29097984 M>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 2541142016 XGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 946537603584 aODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 1188074655232 BDeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚12974260 WFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 942827920554 EDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 38610163200 QEDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚5577245 LDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚330 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚2693467 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚18416 N>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 36985717077 YFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚82476979730496 ?6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚3680 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚6 _step‚30 TEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 3142090752 WDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚19104756573696 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 553472512 RBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 40288387072 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚603 XIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 3214934016 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 FDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 F=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚1533 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚22687 bODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚17916783187968 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚25427968 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 SDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 3214934016 _runtime‚ 697.151302984 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚3 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 101269504 M=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 40288387072 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚25427968 QEDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚7397015 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚2692867Éi{¡á&èÛM BDeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚12974260 WFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 942827920554 EDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 38610163200 QEDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚5577245 LDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚330 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚2693467 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚18416 N>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 36985717077 YFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚82476979730496 ?6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚3680 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚6 _step‚30 TEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 3142090752 WDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚19104756573696 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 553472512 RBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 40288387072 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚603 XIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 3214934016 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 FDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 F=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚1533 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚22687 bODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚17916783187968 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚25427968 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 SDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 3214934016 _runtime‚ 697.151302984 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚3 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 101269504 M=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 40288387072 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚25427968 QEDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚7397015 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚2692867 ?6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚3680 G:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚12974260 FDeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚4451105 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 963120128 TEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 3140732164 P@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 40288387072 MADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚5577239 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 I=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚7378599 [HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚19104857843200 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 R?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚83419807651050 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚22687 MADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚7397015 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚18416 LDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚977 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 YFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚85851954329088 I@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚1533 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1300 KBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚2147 I@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚2147 VCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚86798517360640 MADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚5577245 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚25427968 ]JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚85851979757056 WFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 944021889536 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 [JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 945342284578 R?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚86795976218624 trainer/global_step‚399 C6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚12955838 ZGDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚85851979757056 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 [JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 946537603584 ]KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 1188002483712 WGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 37073453056 PADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 2541142016 UEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 35913877504 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚29097984 M>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 2541142016 XGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 946537603584 aODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 1188074655232ùÆB0é+ _wandbruntime‚697  _runtime‚697Måj9RêÌ 0loss/FBt2/assignment_loss‚0.2526499106341942 0loss/FBt3/assignment_loss‚0.4516890243976377 /loss/FRt2/detection_loss‚1.9742049548811125 1loss/SRqqt1/detection_loss‚2.5978513662429417 1loss/SRqqt2/assignment_loss‚1.787600650467345 /loss/FRt3/detection_loss‚2.6038268551230432 1loss/SRqqt2/detection_loss‚1.3444787958850626 1loss/SRqqt4/detection_loss‚1.7927141682707148 .loss/FBt3/detection_loss‚4.864840316772461 trainer/global_step‚399 0loss/FBt1/assignment_loss‚0.5310471122249055 0loss/FBt4/assignment_loss‚0.3110946726202118 .loss/FRt4/detection_loss‚2.353780597448349 &loss/total_loss‚57.414493373263845 2loss/SRqqt1/assignment_loss‚2.6040870420196476 2loss/SRqqt3/assignment_loss‚2.2660183844769874 1loss/SRqqt3/detection_loss‚2.3436939331089577 % _timestamp‚1.7884858980514507e+09 /loss/FRt1/assignment_loss‚5.480306445335855 .loss/FRt4/assignment_loss‚4.28831814458737 _runtime‚ 697.15265602 /loss/FRt3/assignment_loss‚4.610236825048924 2loss/SRqqt4/assignment_loss‚2.4354366437136696 /loss/FRt1/detection_loss‚2.4695210383862864 /loss/FBt1/detection_loss‚3.2203549653291703 .loss/FBt4/detection_loss‚3.894087011163885 _step‚31 epoch‚0 0loss/FRt2/assignment_loss‚2.0572522276813543 -loss/FBt2/detection_loss‚0.87940228744376çÏÊVëÐ &loss/total_loss‚57.414493373263845 2loss/SRqqt1/assignment_loss‚2.6040870420196476 2loss/SRqqt3/assignment_loss‚2.2660183844769874 1loss/SRqqt3/detection_loss‚2.3436939331089577 % _timestamp‚1.7884858980514507e+09 /loss/FRt1/assignment_loss‚5.480306445335855 .loss/FRt4/assignment_loss‚4.28831814458737 _runtime‚ 697.15265602 /loss/FRt3/assignment_loss‚4.610236825048924 2loss/SRqqt4/assignment_loss‚2.4354366437136696 /loss/FRt1/detection_loss‚2.4695210383862864 /loss/FBt1/detection_loss‚3.2203549653291703 .loss/FBt4/detection_loss‚3.894087011163885 _step‚31 epoch‚0 0loss/FRt2/assignment_loss‚2.0572522276813543 -loss/FBt2/detection_loss‚0.87940228744376 0loss/FBt2/assignment_loss‚0.2526499106341942 0loss/FBt3/assignment_loss‚0.4516890243976377 /loss/FRt2/detection_loss‚1.9742049548811125 1loss/SRqqt1/detection_loss‚2.5978513662429417 1loss/SRqqt2/assignment_loss‚1.787600650467345 /loss/FRt3/detection_loss‚2.6038268551230432 1loss/SRqqt2/detection_loss‚1.3444787958850626 1loss/SRqqt4/detection_loss‚1.7927141682707148 .loss/FBt3/detection_loss‚4.864840316772461 trainer/global_step‚399 0loss/FBt1/assignment_loss‚0.5310471122249055 0loss/FBt4/assignment_loss‚0.3110946726202118 .loss/FRt4/detection_loss‚2.353780597448349<"H"ì‚(:— ‘ÂèÔñÓ»¯) proc.memory.availableMB‚ 1295.69921875 proc.cpu.threads‚17 disk.dm-0.out‚ 6544.03125 disk.sdb.out‚74.74755859375 disk.sda1.out‚0 cpu‚25.337676345240194 disk.md0.out‚71.96875 network.recv‚311164$ disk./.usageGB‚2977.010772705078" disk.nvme1n1.out‚ 1804.81640625 disk.sda.in‚ 0.0078125 disk.dm-0.in‚ 4.25390625 disk.sdb1.out‚0 network.sent‚691131 disk.sdb.in‚0 disk.sda2.in‚ 0.0078125 disk.sda.out‚74.74755859375 disk.rbd0.out‚0" disk.nvme0n1.out‚ 4739.21484375( disk./.usagePercent‚83.2384671806396 disk.sda2.out‚74.74755859375 disk.sdb2.in‚0 disk.sda1.in‚0 disk.md0p1.in‚ 0.0078125 disk.md0p1.out‚71.96875 disk.rbd2.out‚0* proc.memory.percent‚18.114376068115234 disk.nvme1n1.in‚4 disk.rbd0.in‚0 disk.rbd2.in‚0! proc.memory.rssMB‚ 2967.859375 disk.sdb2.out‚74.74755859375 disk.md0.in‚ 0.0078125$ memory_percent‚92.09167957305908 disk.nvme0n1.in‚ 0.25390625 disk.sdb1.in‚0óe¦’í‚(:‡ ‘ÂèÔ¤õ° gpu.0.gpu‚97.0 gpu.0.memory‚17+ gpu.0.memoryAllocated‚47.54493713378906* gpu.0.memoryAllocatedBytes‚ 40840790016 gpu.0.temp‚45.0 gpu.0.powerWatts‚245.644' gpu.0.enforcedPowerLimitWatts‚400.0 gpu.0.powerPercent‚61.411 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0ˆt“î‚(:ˆ  ÂèÔ­‰° gpu.0.gpu‚11.0 gpu.0.memory‚5+ gpu.0.memoryAllocated‚47.54493713378906* gpu.0.memoryAllocatedBytes‚ 40840790016 gpu.0.temp‚45.0 gpu.0.powerWatts‚283.453' gpu.0.enforcedPowerLimitWatts‚400.0 gpu.0.powerPercent‚70.86325 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0'<¹"ï‚(:—  ÂèÔûî° network.sent‚726911 disk.sdb2.in‚0 disk.md0p1.in‚ 0.0078125 disk.nvme1n1.out‚ 1848.734375 disk.sda.out‚75.75927734375 disk.rbd0.in‚0 disk.nvme0n1.in‚ 0.25390625 disk.sdb1.in‚0 disk.md0p1.out‚ 72.9296875! proc.memory.rssMB‚ 2967.859375 disk.nvme1n1.in‚4 disk.sdb.out‚75.75927734375 disk.sda2.in‚ 0.0078125 disk.sda2.out‚75.75927734375 disk.dm-0.in‚ 4.25390625$ memory_percent‚92.08974838256836% disk./.usageGB‚2977.0105018615723 disk.sda1.out‚0 disk.sda.in‚ 0.0078125' proc.memory.availableMB‚ 1296.015625* proc.memory.percent‚18.114376068115234 disk.md0.out‚ 72.9296875 disk.nvme0n1.out‚ 4923.40625 disk.dm-0.out‚ 6772.140625 cpu‚25.339503756418818 proc.cpu.threads‚17) disk./.usagePercent‚83.23845960774182 disk.sdb2.out‚75.75927734375 disk.rbd2.out‚0 disk.rbd0.out‚0 network.recv‚316273 disk.sdb1.out‚0 disk.sda1.in‚0 disk.rbd2.in‚0 disk.sdb.in‚0 disk.md0.in‚ 0.0078125“Ïâ’ð‚(:‡ ¯ÂèÔ§ŒÎ° gpu.0.gpu‚97.0 gpu.0.memory‚17+ gpu.0.memoryAllocated‚47.54493713378906* gpu.0.memoryAllocatedBytes‚ 40840790016 gpu.0.temp‚42.0 gpu.0.powerWatts‚177.688' gpu.0.enforcedPowerLimitWatts‚400.0 gpu.0.powerPercent‚44.422 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0 YØî"ñ‚(:— ¯Âèԛ斱 disk.sdb2.out‚ 76.939453125! proc.memory.rssMB‚ 2967.859375 disk.sdb1.in‚0 disk.sdb1.out‚0$ memory_percent‚92.08979606628418% disk./.usageGB‚2977.0360221862793 disk.md0.out‚ 74.05078125 disk.rbd2.in‚0 disk.sda2.out‚ 76.939453125 disk.sda.out‚ 76.939453125* proc.memory.percent‚18.114376068115234 network.recv‚319863" disk.nvme0n1.out‚ 4984.64453125 disk.sdb.in‚0 disk.sda2.in‚ 0.0078125( proc.memory.availableMB‚ 1296.0078125 proc.cpu.threads‚17 disk.dm-0.in‚ 4.25390625 disk.sdb2.in‚0 disk.md0p1.in‚ 0.0078125 disk.md0p1.out‚ 74.05078125 disk.rbd0.in‚0 cpu‚25.339689075361562 network.sent‚734349( disk./.usagePercent‚83.2391731667017 disk.nvme0n1.in‚ 0.25390625 disk.rbd2.out‚0 disk.sdb.out‚ 76.939453125" disk.nvme1n1.out‚ 1921.26953125 disk.rbd0.out‚0 disk.dm-0.out‚ 6905.9140625 disk.sda1.in‚0 disk.sda1.out‚0 disk.nvme1n1.in‚4 disk.sda.in‚ 0.0078125 disk.md0.in‚ 0.0078125‘î-“ò‚(:ˆ ¾ÂèÔàù¹° gpu.0.gpu‚92.0 gpu.0.memory‚16+ gpu.0.memoryAllocated‚47.54493713378906* gpu.0.memoryAllocatedBytes‚ 40840790016 gpu.0.temp‚45.0 gpu.0.powerWatts‚242.186' gpu.0.enforcedPowerLimitWatts‚400.0 gpu.0.powerPercent‚60.5465 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0vO"t'ó‚(:œ ¾ÂèÔ–Ùù° disk.sdb.in‚0$ memory_percent‚92.09129810333252* proc.memory.percent‚18.114376068115234 proc.cpu.threads‚17% disk./.usageGB‚2977.0381355285645 disk.rbd0.in‚0 cpu‚25.34119479683363 disk.md0.out‚ 74.984375 disk.rbd2.in‚0 disk.sda2.out‚77.93310546875 disk.nvme1n1.in‚4! disk.nvme1n1.out‚ 1923.6328125 disk.rbd0.out‚0 disk.nvme0n1.in‚ 0.25390625) proc.memory.availableMB‚ 1295.76171875" disk.nvme0n1.out‚ 5045.95703125 disk.sdb2.out‚77.93310546875 disk.md0p1.out‚ 74.984375 disk.sda.in‚ 0.0078125 disk.sdb1.out‚0! proc.memory.rssMB‚ 2967.859375 disk.dm-0.out‚ 6969.58984375 network.recv‚324497 disk.sda2.in‚ 0.0078125 disk.dm-0.in‚ 4.25390625 network.sent‚740372 disk.sdb.out‚77.93310546875 disk.sda.out‚77.93310546875 disk.sdb1.in‚0 disk.sda1.in‚0 disk.sda1.out‚0 disk.rbd2.out‚0) disk./.usagePercent‚83.23923225663651 disk.sdb2.in‚0 disk.md0.in‚ 0.0078125 disk.md0p1.in‚ 0.0078125ù©¤Ý”ô‚(:‰ ÍÂèÔïа gpu.0.gpu‚71.0 gpu.0.memory‚17+ gpu.0.memoryAllocated‚47.54493713378906* gpu.0.memoryAllocatedBytes‚ 40840790016 gpu.0.temp‚48.0 gpu.0.powerWatts‚324.349' gpu.0.enforcedPowerLimitWatts‚400.0 gpu.0.powerPercent‚81.08725 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0oT‹Áõ‚(:‹ ÍÂèÔŒ°±$ disk./.usageGB‚2977.163074493408 disk.dm-0.in‚ 4.25390625 disk.md0p1.in‚ 0.0078125 disk.rbd2.out‚0 disk.nvme1n1.in‚4! proc.memory.rssMB‚ 2967.859375 disk.nvme0n1.in‚ 0.25390625 disk.sdb1.in‚0 disk.sdb1.out‚0 disk.sda1.in‚0 disk.md0.in‚ 0.0078125 disk.rbd0.out‚0 network.recv‚328513 disk.sdb2.out‚ 79.5185546875 disk.md0.out‚76.5 disk.sdb.in‚0" disk.nvme1n1.out‚ 1993.61328125 disk.rbd0.in‚0 cpu‚25.34228405725514 disk.sda1.out‚0 disk.rbd2.in‚0 disk.sdb2.in‚0 disk.sdb.out‚ 79.5185546875 disk.sda.out‚ 79.5185546875 network.sent‚746148 disk.nvme0n1.out‚ 5134.09375 disk.dm-0.out‚ 7127.70703125) disk./.usagePercent‚83.24272560238455 disk.md0p1.out‚76.5 disk.sda2.out‚ 79.5185546875 disk.sda.in‚ 0.0078125$ memory_percent‚92.09001064300537) proc.memory.availableMB‚ 1295.97265625* proc.memory.percent‚18.114376068115234 proc.cpu.threads‚17 disk.sda2.in‚ 0.0078125—@’ö‚(:‡ ÜÂèÔ¾üÝ° gpu.0.gpu‚97.0 gpu.0.memory‚17+ gpu.0.memoryAllocated‚47.54493713378906* gpu.0.memoryAllocatedBytes‚ 40840790016 gpu.0.temp‚45.0 gpu.0.powerWatts‚226.37' gpu.0.enforcedPowerLimitWatts‚400.0 gpu.0.powerPercent‚56.5925 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0'ýl÷‚(:“ ÜÂèÔãðű disk.sdb2.in‚0$ disk./.usageGB‚2977.288074493408 disk.sda1.in‚0 disk.md0p1.in‚ 0.0078125! proc.memory.rssMB‚ 2967.859375 proc.cpu.threads‚17 disk.sda2.out‚ 80.935546875* proc.memory.percent‚18.114376068115234) disk./.usagePercent‚83.24622065470112 disk.dm-0.out‚ 7301.26953125 disk.rbd2.out‚0 disk.sdb1.in‚0 disk.md0.in‚ 0.0078125 disk.sda2.in‚ 0.0078125 disk.sda.in‚ 0.0078125( proc.memory.availableMB‚ 1295.7265625 network.recv‚333131 disk.rbd0.out‚0 disk.dm-0.in‚ 4.25390625$ memory_percent‚92.09151268005371 cpu‚25.342726890812774 network.sent‚752341 disk.nvme0n1.in‚ 0.25390625 disk.sdb1.out‚0 disk.sda1.out‚0 disk.rbd2.in‚0 disk.sdb.out‚ 80.935546875 disk.sda.out‚ 80.935546875 disk.rbd0.in‚0" disk.nvme0n1.out‚ 5229.80078125 disk.md0.out‚ 77.8515625 disk.md0p1.out‚ 77.8515625 disk.nvme1n1.out‚ 2071.46875 disk.sdb2.out‚ 80.935546875 disk.sdb.in‚0 disk.nvme1n1.in‚4þšä0ø+ _wandbruntime‚782  _runtime‚782=Ä^Äù¾ & lr-AdamW/pg1‚0.0008454691070220119 & lr-AdamW/pg2‚0.0008454691070220119 trainer/global_step‚449 % _timestamp‚1.7884859830483358e+09 _runtime‚ 782.149881712 _step‚32¨ö'Èú & lr-AdamW/pg1‚0.0008454691070220119 & lr-AdamW/pg2‚0.0008454691070220119 trainer/global_step‚449 % _timestamp‚1.7884859830483358e+09 _runtime‚ 782.149881712 _step‚32 ½ã_0û+ _wandbruntime‚782  _runtime‚782•ûØ'üÒO UFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 3214934016 SDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 3142090752 VGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 3140732164 RBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 40288387072 MDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚2147 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚18396 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚18390 OCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚6257855 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚6 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚594 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 YJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 2514363816 D;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚3680 IDeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚22388 VGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 2515700736 `MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚20079140014592 _runtime‚ 782.15171418 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚25427968 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚2684 dQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚20079169112576 D>DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 _step‚33 UBDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated‚97397351335424 ZHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 1059498421760 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚3022775 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 M=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 38610163200 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚1972175 BDeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚97394810206720 NFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚977 XEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚96335311784960 M@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚14556954 TDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 35913877504 ^LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 1062014122496 [HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚92547247094800 [KDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 37073453056 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 101282816 TDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 40288387072 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚18396 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 YIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 37073453056 VFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 37073453056 F=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚3680 PDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚4994950 TEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 1495537152 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 YJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 2515700736 SGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚6257855 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚25427968 E8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚14538558 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚18390 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚22388 SGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚3022184 A8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚3680 MDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚1533 G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚2684 WGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 35913877504 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚22687 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚591 ]JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚21411708756992 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚29097984 TADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚93605405838328 O?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 40288387072 [IDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 1062014122496 WEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 1059498421760 SGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚8299099 OCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚8280709 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 P@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 38610163200 YFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚21411607474176 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 XEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚93607945630112 H?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚1533 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚22687 \IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚96335337212928 L@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚4994356 A8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚3680 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 ZHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 1058158743528 XIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 3214934016 H?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚2147 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚72184832 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 _LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚92547272522768 TADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚97394810206720 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚25427968 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 WGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 34283657216 % _timestamp‚1.7884859830498788e+09 XEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚97397351335424 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 trainer/global_step‚449 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚6 [HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚96335311784960 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 P@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 36985717077 O@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 2541128704 VGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 3142090752 H?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚1300 NFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚330 ^LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 1060673107344 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 KBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚1533 _LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚96335337212928 OCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚6257849 _MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 1332467459584 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 KBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚2147 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 963120128 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 561701376 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 K?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚6257849 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚3 IDeviceStatsMonitor.on_train_batch_start/allocation.all.current‚18396 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚18390 OCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚6257855 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚6 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚594 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 YJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 2514363816 D;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚3680 IDeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚22388 VGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 2515700736 `MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚20079140014592 _runtime‚ 782.15171418 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚25427968 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚2684 dQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚20079169112576 D>DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 _step‚33 UBDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated‚97397351335424 ZHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 1059498421760 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚3022775 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 M=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 38610163200 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚1972175 BDeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚97394810206720 NFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚977 XEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚96335311784960 M@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚14556954 TDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 35913877504 ^LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 1062014122496 [HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚92547247094800 [KDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 37073453056 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 101282816 TDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 40288387072 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚18396 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 YIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 37073453056 VFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 37073453056 F=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚3680 PDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚4994950 TEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 1495537152 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 YJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 2515700736 SGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚6257855 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚25427968 E8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚14538558 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚18390 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚22388 SGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚3022184 A8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚3680 MDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚1533 G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚2684 WGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 35913877504 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚22687 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚591 ]JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚21411708756992 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚29097984 TADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚93605405838328 O?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 40288387072 [IDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 1062014122496 WEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 1059498421760 SGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚8299099 OCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚8280709 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 P@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 38610163200 YFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚21411607474176 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 XEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚93607945630112 H?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚1533 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚22687 \IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚96335337212928 L@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚4994356 A8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚3680 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 ZHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 1058158743528 XIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 3214934016 H?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚2147 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚72184832 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 _LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚92547272522768 TADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚97394810206720 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚25427968 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 WGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 34283657216 % _timestamp‚1.7884859830498788e+09 XEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚97397351335424 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 trainer/global_step‚449 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚6 [HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚96335311784960 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 P@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 36985717077!Ãsè0þ+ _wandbruntime‚783  _runtime‚783²Þ•"ºÿÝM >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 XIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 3214934016 VGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 3214934016 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚22388 F=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚1533 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚6 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚18416 aODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 1335488369152 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚18422 QBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 3142090752 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 RBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 40288387072 ?6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚3680 I=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚6271739 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 QEDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚6271745 R?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚97611112373248 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚18416 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚25427968 \JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 1064370231808 VCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚93815814426550 MADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚6271745 S@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚97613653515264 G:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚14570790 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚2684 WGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 37073453056 F=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚2147 RBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 35913877504 ]JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚92752788183264 P@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 40288387072 M>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 2541142016 ]JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚96549283283456 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚22687 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚3028903 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 TEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 2515714048 FDeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚14589212 LDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚977 bODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚20123299437568 XFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 1060511879262 B9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚3680 M=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 40288387072 QEDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚8317467 ^KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚20123270339584 N>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 36985717077 \JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 1063026243286 _runtime‚ 783.749838634 EDeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 NBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚5006032 N>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 38610163200 C6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚14570790 UEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 35913877504 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚588 ]KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 1335416197632 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 WHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 2514364024 VCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚96549257855488 PADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 2541142016 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 101269504 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚72171520 YIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 37073453056 % _timestamp‚1.7884859846474664e+09 _step‚34 UHDeviíÌÀ{) ceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚25427968 SDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 3214934016 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 561701376 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 YFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚92752762755296 I@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚2147 KBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚1533 G:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚14589212 MADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚8317467 UCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 1061854517760 MADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚6271739 ?6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚3680 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 D;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚3680 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚22687 YFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚96549257855488 WHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 2515714048 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1300 I@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚1533 ODeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚5005441 VCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚97613653515264 [HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚21458787806720 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 R?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚93813274634558 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚25427968 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚6 LDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚330Yå½ ç&€áM SDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 3214934016 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 561701376 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 YFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚92752762755296 I@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚2147 KBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚1533 G:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚14589212 MADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚8317467 UCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 1061854517760 MADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚6271739 ?6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚3680 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 D;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚3680 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚22687 YFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚96549257855488 WHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 2515714048 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1300 I@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚1533 ODeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚5005441 VCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚97613653515264 [HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚21458787806720 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 R?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚93813274634558 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚25427968 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚6 LDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚330 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 XIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 3214934016 VGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 3214934016 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚22388 F=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚1533 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚6 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚18416 aODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 1335488369152 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚18422 QBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 3142090752 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 RBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 40288387072 ?6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚3680 I=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚6271739 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 QEDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚6271745 R?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚97611112373248 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚18416 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚25427968 \JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 1064370231808 VCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚93815814426550 MADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚6271745 S@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚97613653515264 G:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚14570790 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚2684 WGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 37073453056 F=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚2147 RBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 35913877504 ]JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚92752788183264 P@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 40288387072 M>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 2541142016 ]JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚96549283283456 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚22687 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚3028903 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 TEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 2515714048 FDeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚14589212 LDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚977 bODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚20123299437568 XFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 1060511879262 B9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚3680 M=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 40288387072 QEDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚8317467 ^KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚20123270339584 N>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 36985717077 \JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 1063026243286 _runtime‚ 783.749838634 EDeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 NBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚5006032 N>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 38610163200 C6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚14570790 UEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 35913877504 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚588 ]KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 1335416197632 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 WHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 2514364024 VCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚96549257855488 PADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 2541142016 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 101269504 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚72171520 YIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 37073453056 % _timestamp‚1.7884859846474664e+09 _step‚34 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚25427968"À‰Ù“0+ _wandbruntime‚783  _runtime‚783aúhS‚Í 1loss/SRqqt4/detection_loss‚1.9321458359726338 trainer/global_step‚449 _runtime‚ 783.751072089 %loss/total_loss‚52.05757656947645 epoch‚0 /loss/FRt1/assignment_loss‚5.362291197566425 0loss/FRt2/assignment_loss‚1.8800900005158925 /loss/FRt4/assignment_loss‚3.271229480649089 1loss/SRqqt4/assignment_loss‚2.038284577116245 /loss/FBt4/assignment_loss‚0.640169675482244 .loss/FRt3/detection_loss‚2.867429471960882 /loss/FRt4/detection_loss‚2.3187770588057384 % _timestamp‚1.7884859846498032e+09 _step‚35 2loss/SRqqt1/assignment_loss‚2.1238175911562784 1loss/SRqqt2/assignment_loss‚1.291483865339097 /loss/SRqqt3/detection_loss‚2.62439672259184 /loss/FBt1/detection_loss‚3.0160441916921865 2loss/SRqqt3/assignment_loss‚1.8144989256675428 1loss/FBt1/assignment_loss‚0.43812787746276666 .loss/FRt1/detection_loss‚2.790845884119763 1loss/SRqqt2/detection_loss‚1.1602795684664715 .loss/FBt4/detection_loss‚3.165553298435713 0loss/FRt3/assignment_loss‚4.9054366044881865 1loss/FBt2/assignment_loss‚0.22586520983731695 .loss/FBt3/detection_loss‚2.606526404619217 /loss/FBt2/detection_loss‚0.9030800024405281 1loss/FBt3/assignment_loss‚0.43810089687463005 /loss/FRt2/detection_loss‚1.8959580672638756 0loss/SRqqt1/detection_loss‚2.347144160951887O! ’WƒÑ /loss/FRt1/assignment_loss‚5.362291197566425 0loss/FRt2/assignment_loss‚1.8800900005158925 /loss/FRt4/assignment_loss‚3.271229480649089 1loss/SRqqt4/assignment_loss‚2.038284577116245 /loss/FBt4/assignment_loss‚0.640169675482244 .loss/FRt3/detection_loss‚2.867429471960882 /loss/FRt4/detection_loss‚2.3187770588057384 % _timestamp‚1.7884859846498032e+09 _step‚35 2loss/SRqqt1/assignment_loss‚2.1238175911562784 1loss/SRqqt2/assignment_loss‚1.291483865339097 /loss/SRqqt3/detection_loss‚2.62439672259184 /loss/FBt1/detection_loss‚3.0160441916921865 2loss/SRqqt3/assignment_loss‚1.8144989256675428 1loss/FBt1/assignment_loss‚0.43812787746276666 .loss/FRt1/detection_loss‚2.790845884119763 1loss/SRqqt2/detection_loss‚1.1602795684664715 .loss/FBt4/detection_loss‚3.165553298435713 0loss/FRt3/assignment_loss‚4.9054366044881865 1loss/FBt2/assignment_loss‚0.22586520983731695 .loss/FBt3/detection_loss‚2.606526404619217 /loss/FBt2/detection_loss‚0.9030800024405281 1loss/FBt3/assignment_loss‚0.43810089687463005 /loss/FRt2/detection_loss‚1.8959580672638756 0loss/SRqqt1/detection_loss‚2.347144160951887 1loss/SRqqt4/detection_loss‚1.9321458359726338 trainer/global_step‚449 _runtime‚ 783.751072089 %loss/total_loss‚52.05757656947645 epoch‚0#¤J¸H„‚(:’ ëÂèÔŒ„Õ° gpu.0.gpu‚64.0 gpu.0.memory‚14+ gpu.0.memoryAllocated‚47.54493713378906* gpu.0.memoryAllocatedBytes‚ 40840790016 gpu.0.temp‚45.0 gpu.0.powerWatts‚235.768' gpu.0.enforcedPowerLimitWatts‚400.0( gpu.0.powerPercent‚58.94200000000001 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0ª®fÎ)…‚(:ž ëÂèÔ‹Ïܱ disk.md0.in‚ 0.0078125" disk.nvme1n1.out‚ 2130.87890625 disk.rbd0.in‚0 disk.sdb1.out‚0 disk.sdb2.in‚0 disk.md0.out‚ 79.07421875 disk.rbd2.out‚0 disk.sda.in‚ 0.0078125 disk.rbd0.out‚0 disk.nvme0n1.in‚ 0.25390625 disk.sdb1.in‚0( proc.memory.availableMB‚ 1295.8828125 network.recv‚336427) disk./.usagePercent‚83.25323923769632" disk.nvme0n1.out‚ 5291.03515625 network.sent‚768238 disk.rbd2.in‚0 disk.sdb.in‚0 disk.sdb.out‚82.21728515625 disk.nvme1n1.in‚4 disk.dm-0.in‚ 4.25390625* proc.memory.percent‚18.114376068115234$ disk./.usageGB‚2977.539093017578 disk.md0p1.in‚ 0.0078125 disk.sdb2.out‚82.21728515625 disk.sda2.in‚ 0.0078125 disk.sda2.out‚82.21728515625 disk.sda.out‚82.21728515625 disk.sda1.out‚0! proc.memory.rssMB‚ 2967.859375 cpu‚25.343763352790504 proc.cpu.threads‚17 disk.md0p1.out‚ 79.07421875 disk.dm-0.out‚ 7421.9140625 disk.sda1.in‚0# memory_percent‚92.0905590057373Ó¸ a‘†‚(:† úÂèÔªÛ·° gpu.0.gpu‚95.0 gpu.0.memory‚16+ gpu.0.memoryAllocated‚47.54493713378906* gpu.0.memoryAllocatedBytes‚ 40840790016 gpu.0.temp‚45.0 gpu.0.powerWatts‚229.82' gpu.0.enforcedPowerLimitWatts‚400.0 gpu.0.powerPercent‚57.455 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0aw}~*‡‚(:Ÿ úÂèÔÉìú° disk.sdb1.out‚0 disk.sdb2.in‚0* proc.memory.percent‚18.114376068115234 disk.sdb.out‚83.23681640625" disk.nvme0n1.out‚ 5352.36328125 disk.sda.out‚83.23681640625 disk.rbd0.in‚0 disk.sdb2.out‚83.23681640625 proc.cpu.threads‚17$ disk./.usageGB‚2977.538074493408 disk.dm-0.out‚ 7638.33984375 disk.md0p1.out‚ 80.04296875! proc.memory.rssMB‚ 2967.859375 cpu‚25.343924320006085 network.sent‚804509 network.recv‚341237) disk./.usagePercent‚83.25321075933425 disk.sdb.in‚0 disk.sda2.out‚83.23681640625! disk.nvme1n1.out‚ 2285.9765625 disk.nvme0n1.in‚ 0.25390625 disk.sda2.in‚ 0.0078125 disk.rbd0.out‚0 disk.sda1.in‚0 disk.sda1.out‚0 disk.rbd2.in‚0 disk.sdb1.in‚0 disk.md0.in‚ 0.0078125 disk.md0.out‚ 80.04296875 disk.rbd2.out‚0$ memory_percent‚92.09208488464355 disk.nvme1n1.in‚4 disk.sda.in‚ 0.0078125 disk.md0p1.in‚ 0.0078125( proc.memory.availableMB‚ 1295.6328125 disk.dm-0.in‚ 4.25390625;U¨“ˆ‚(:ˆ ‰ÃèÔ÷ûØ° gpu.0.gpu‚97.0 gpu.0.memory‚23+ gpu.0.memoryAllocated‚47.54493713378906* gpu.0.memoryAllocatedBytes‚ 40840790016 gpu.0.temp‚45.0 gpu.0.powerWatts‚149.442' gpu.0.enforcedPowerLimitWatts‚400.0 gpu.0.powerPercent‚37.3605 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0R([U‰‚(:’ ‰ÃèÔóÚ³±! disk.nvme1n1.out‚ 2415.1640625) proc.memory.availableMB‚ 1295.88671875 network.sent‚812452 disk.sdb2.out‚ 84.560546875 disk.md0p1.out‚ 81.29296875 disk.rbd2.out‚0 disk.nvme0n1.out‚ 5513.65625 cpu‚25.34518140534537 disk.sdb2.in‚0 disk.md0p1.in‚ 0.0078125 disk.sda2.in‚ 0.0078125 disk.sda.out‚ 84.560546875# memory_percent‚92.0905351638794) disk./.usagePercent‚83.25321075933425 disk.sda1.in‚0 disk.sdb.out‚ 84.560546875 disk.sda2.out‚ 84.560546875! proc.memory.rssMB‚ 2967.859375 disk.sdb1.in‚0 disk.nvme1n1.in‚4 disk.nvme0n1.in‚ 0.25390625* proc.memory.percent‚18.114376068115234 disk.sda1.out‚0 disk.md0.in‚ 0.0078125 disk.sdb.in‚0 disk.rbd0.in‚0 disk.rbd0.out‚0 proc.cpu.threads‚17 disk.dm-0.in‚ 4.25390625 disk.md0.out‚ 81.29296875 disk.sda.in‚ 0.0078125 network.recv‚345784$ disk./.usageGB‚2977.538074493408 disk.dm-0.out‚ 7928.8203125 disk.sdb1.out‚0 disk.rbd2.in‚0I•ôó’Š‚(:‡ ˜ÃèÔ¬ùÚ° gpu.0.gpu‚97.0 gpu.0.memory‚17+ gpu.0.memoryAllocated‚47.54493713378906* gpu.0.memoryAllocatedBytes‚ 40840790016 gpu.0.temp‚45.0 gpu.0.powerWatts‚227.312' gpu.0.enforcedPowerLimitWatts‚400.0 gpu.0.powerPercent‚56.828 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0Ï—ÿ?"‹‚(:— ˜ÃèÔà×ö°) disk./.usagePercent‚83.23846003438395 disk.md0.in‚ 0.0078125 disk.md0p1.in‚ 0.0078125 disk.sdb.in‚0 disk.nvme0n1.in‚ 0.25390625 disk.sdb2.out‚86.02099609375 disk.rbd2.in‚0 disk.sda2.out‚86.02099609375 disk.sda.in‚ 0.0078125 disk.rbd0.out‚0" disk.nvme0n1.out‚ 5536.11328125 disk.dm-0.in‚ 4.25390625 disk.sda1.in‚0 disk.rbd0.in‚0 disk.sdb2.in‚0! proc.memory.rssMB‚ 2967.859375 cpu‚25.345583221059037 disk.sda1.out‚0 disk.md0p1.out‚82.6875 disk.sdb.out‚86.02099609375 disk.dm-0.out‚ 7994.96484375# memory_percent‚92.0905590057373( proc.memory.availableMB‚ 1295.8828125 network.recv‚350390% disk./.usageGB‚2977.0105171203613 disk.md0.out‚82.6875 disk.sda.out‚86.02099609375 disk.sdb1.out‚0* proc.memory.percent‚18.114376068115234 disk.sda2.in‚ 0.0078125 disk.nvme1n1.in‚4 disk.sdb1.in‚0 network.sent‚816562 disk.rbd2.out‚0! disk.nvme1n1.out‚ 2458.8515625 proc.cpu.threads‚17JHw6žŒ‚(:“ §ÃèÔ¢¾º° gpu.0.gpu‚98.0 gpu.0.memory‚23+ gpu.0.memoryAllocated‚47.54493713378906* gpu.0.memoryAllocatedBytes‚ 40840790016 gpu.0.temp‚45.0 gpu.0.powerWatts‚230.204' gpu.0.enforcedPowerLimitWatts‚400.0) gpu.0.powerPercent‚57.550999999999995 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0Øt ý!‚(:– §ÃèÔ¤Æü® disk.nvme1n1.out‚ 2458.984375) proc.memory.availableMB‚ 1295.87890625 disk.rbd0.in‚0 disk.nvme0n1.in‚ 0.25390625 disk.md0.in‚ 0.0078125 disk.md0p1.in‚ 0.0078125 cpu‚25.346344081307052 disk.rbd2.in‚0 disk.sdb.out‚ 87.154296875 disk.sda.out‚ 87.154296875! disk.nvme0n1.out‚ 5597.8828125$ memory_percent‚92.09058284759521% disk./.usageGB‚2976.9663276672363 disk.sda2.in‚ 0.0078125 disk.nvme1n1.in‚4 disk.dm-0.out‚ 8056.8671875 proc.cpu.threads‚17 disk.sda2.out‚ 87.154296875 disk.dm-0.in‚ 4.25390625 disk.sdb1.in‚0 disk.sda1.out‚0* proc.memory.percent‚18.114376068115234 disk.sda1.in‚0 disk.rbd2.out‚0 disk.sdb.in‚0 disk.rbd0.out‚0 disk.sdb1.out‚0 disk.md0p1.out‚ 83.75390625! proc.memory.rssMB‚ 2967.859375 network.recv‚354046) disk./.usagePercent‚83.23722447877985 disk.sda.in‚ 0.0078125 disk.sdb2.in‚0 disk.sdb2.out‚ 87.154296875 disk.md0.out‚ 83.75390625 network.sent‚821847àâ„¿“Ž‚(:ˆ ¶ÃèÔ³Ž¬° gpu.0.gpu‚98.0 gpu.0.memory‚18+ gpu.0.memoryAllocated‚47.54493713378906* gpu.0.memoryAllocatedBytes‚ 40840790016 gpu.0.temp‚45.0 gpu.0.powerWatts‚253.402' gpu.0.enforcedPowerLimitWatts‚400.0 gpu.0.powerPercent‚63.3505 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚00êèw!‚(:– ¶ÃèÔ½›í° disk.dm-0.out‚ 8251.43359375 disk.md0p1.in‚ 0.0078125 disk.rbd0.in‚0* proc.memory.percent‚18.114376068115234) disk./.usagePercent‚83.23917327336224 disk.dm-0.in‚ 4.25390625 disk.sdb1.in‚0 disk.sdb2.out‚88.29638671875 disk.rbd2.in‚0 network.recv‚358339 disk.sda.in‚ 0.0078125! disk.nvme0n1.out‚ 5792.2578125 disk.sdb2.in‚0" disk.nvme1n1.out‚ 2459.17578125 disk.sda.out‚88.29638671875 disk.rbd0.out‚0 disk.md0p1.out‚84.84375 disk.sdb.in‚0! proc.memory.rssMB‚ 2967.859375% disk./.usageGB‚2977.0360260009766 disk.nvme0n1.in‚ 0.25390625 disk.sda1.in‚0 disk.md0.out‚84.84375 disk.rbd2.out‚0 disk.nvme1n1.in‚4 proc.cpu.threads‚17 disk.md0.in‚ 0.0078125 disk.sdb.out‚88.29638671875 disk.sda2.in‚ 0.0078125$ memory_percent‚92.09060668945312 cpu‚25.346573448036832 network.sent‚827599 disk.sdb1.out‚0 disk.sda1.out‚0 disk.sda2.out‚88.29638671875$ proc.memory.availableMB‚1295.875^X”]0+ _wandbruntime‚870  _runtime‚870 ηđ¾ _runtime‚ 870.275304034 _step‚36 & lr-AdamW/pg1‚0.0009396193416569798 & lr-AdamW/pg2‚0.0009396193416569798 trainer/global_step‚499 % _timestamp‚1.7884860711737592e+09CJÚ È’Â _runtime‚ 870.275304034 _step‚36 & lr-AdamW/pg1‚0.0009396193416569798 & lr-AdamW/pg2‚0.0009396193416569798 trainer/global_step‚499 % _timestamp‚1.7884860711737592e+09$ñEæ0“+ _wandbruntime‚870  _runtime‚870´”'d8”ÝO SGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚6952355 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 561701376 UADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚103998872821836 UFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 3214934016 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 VGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 2515700736 SDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 3142090752 VGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 3142090752 YJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 2514363816 TDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 40288387072 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚22388 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚586 SGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚3358944 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 IDeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚108209946361344 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚22388 TEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 1495537152 cQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 1480129681920 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚70087680 KBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚2147 E8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚16153510 L@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚5549416 K?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚6952349 UADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚108209946361344 ZHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 1177331049984 BDeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚2684 P@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 38610163200 \HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚107032615311360 `LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚102823055547568 O@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 2541128704 YEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚108212487490048 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚25427968 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚6 YFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚23765715858432 ZHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 1175842702236 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 WGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 35913877504 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚18390 A8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚3680 O?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 40288387072 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚22687 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚2190475 VBDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated‚108212487490048 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚2684 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚3 _runtime‚ 870.277038132 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚18396 WEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 1177331049984 F=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚3680 [IDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 1179846750720 M@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚16171906 OCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚9201161 PDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚5550005 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚25427968 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 963120128 YEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚104001412613620 H?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚2147 XIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 3214934016 OCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚6952355 SGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚9219551 WGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 34283657216 ZKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 3214934016 $ _timestamp‚1.788486071175244e+09 RCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 2541128704 `LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚107032640739328 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚18396 SGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚2190472 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 [KDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 37073453056 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 H?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚1533 ]IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚107032640739328 TDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 35913877504 OCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚6952349 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚25427968 D;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚3680 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚6 K?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚9201161 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 KBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚1533 YEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚107032615311360 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚3359530 YJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 2515700736 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚589 MDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚1533 ]JDeviceStatsMonitor.on_train_batch_start/gó¡à«inactive_split_bytes.all.allocated‚23765815044096 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 YIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 37073453056 VFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 37073453056 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 M=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 38610163200 NFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚330 `MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚22285656264192 MDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚2147 NFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚977 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 D>DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 P@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 36985717077 VGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 3140732164 A8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚3680 trainer/global_step‚499 H?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚1300 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 ^LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 1178357066052 _step‚37 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚22687 IDeviceStatsMonitor.on_train_batch_start/allocation.all.current‚18396 SGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚2190472 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 [KDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 37073453056 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 H?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚1533 ]IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚107032640739328 TDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 35913877504 OCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚6952349 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚25427968 D;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚3680 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚6 K?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚9201161 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 KBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚1533 YEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚107032615311360 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚3359530 YJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 2515700736 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚589 MDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚1533 ]JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚23765815044096 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 YIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 37073453056 VFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 37073453056 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 M=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 38610163200 NFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚330 `MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚22285656264192 MDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚2147 NFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚977 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 D>DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 P@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 36985717077 VGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 3140732164 A8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚3680 trainer/global_step‚499 H?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚1300 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 ^LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 1178357066052 _step‚37 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚22687 IDeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚22388 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚586 SGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚3358944 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 IDeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚108209946361344 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚22388 TEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 1495537152 cQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 1480129681920 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚70087680 KBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚2147 E8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚16153510 L@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚5549416 K?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚6952349 UADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚108209946361344 ZHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 1177331049984 BDeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚2684 P@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 38610163200 \HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚107032615311360 `LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚102823055547568 O@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 2541128704 YEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚108212487490048 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚25427968 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚6 YFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚23765715858432 ZHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 1175842702236 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 WGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 35913877504 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚18390 A8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚3680 O?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 40288387072 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚22687 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚2190475 VBDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated‚108212487490048 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚2684 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚3%ÕQ0–+ _wandbruntime‚871  _runtime‚871hÚÔŽí&—çM KBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚2147 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 _step‚38 I=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚9219503 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚22388 UEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 34283657216 KBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚1533 G:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚16204164 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚25427968 MADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚9219503 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚72171520 M=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 40288387072 QBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 3142090752 WCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚108428789669888 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚2684 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 561701376 F=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚1533 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚22687 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚22687 QEDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚6966245 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚2194841 aODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 1483096054272 TEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 3140732164 PADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 2541142016 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 \JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 1180710201994 I=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚6966239 K;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 38610163200 MADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚6966239 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚6 PDeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚16204164 ^KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚22329786589184 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 FDeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚5560330 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 C6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚16185742 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 TDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 37073453056 $ _timestamp‚1.788486072779489e+09 MADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚6966245 ZFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚107246561381888 PADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 2539791992 S?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚104206741618066 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚25427968 WHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 2514364024 YGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 1182202860032 B9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚3680 RBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 35913877504 N>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 38610163200 XIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 3214934016 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚18416 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚3 S?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚108426248527872 RCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 1495537152 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 trainer/global_step‚499 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚599 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 BDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 36985717077 M>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 2541142016 XFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 1179687145984 LDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚977 ?6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚3680 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 YIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 37073453056 _runtime‚ 871.88212835 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚18416 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚3365492 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 WGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 37073453056 TEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 2515714048 UEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 35913877504 [HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚23812911741440 RBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 40288387072 F=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚2147 EDeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 _step‚38 I=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚9219503 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚22388 UEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 34283657216 KBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚1533 G:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚16204164 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚25427968 MADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚9219503 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚72171520 M=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 40288387072 QBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 3142090752 WCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚108428789669888 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚2684 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 561701376 F=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚1533 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚22687 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚22687 QEDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚6966245 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚2194841 aODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 1483096054272 TEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 3140732164 PADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 2541142016 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 \JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 1180710201994 I=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚6966239 K;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 38610163200 MADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚6966239 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚6 PDeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚16204164 ^KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚22329786589184 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 FDeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚5560330 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 C6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚16185742 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 TDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 37073453056 $ _timestamp‚1.788486072779489e+09 MADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚6966245 ZFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚107246561381888 PADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 2539791992 S?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚104206741618066 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚25427968 WHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 2514364024 YGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 1182202860032 B9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚3680 RBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 35913877504 N>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 38610163200 XIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 3214934016 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚18416 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚3 S?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚108426248527872 RCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 1495537152 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 trainer/global_step‚499 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚599 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 BDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 36985717077 M>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 2541142016 XFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 1179687145984 LDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚977 ?6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚3680 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 YIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 37073453056 _runtime‚ 871.88212835 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚18416 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚3365492 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 WGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 37073453056&_´"i0™+ _wandbruntime‚871  _runtime‚871ÚÇ£/ÿšË 1loss/SRqqt1/assignment_loss‚2.105851496111101 0loss/SRqqt3/detection_loss‚2.217291188390949 .loss/FBt1/detection_loss‚3.340232640504837 %loss/total_loss‚48.98305735924012 0loss/FRt2/assignment_loss‚1.8013593789110793 .loss/FR¨uÜRt3/detection_loss‚2.594319172454684 .loss/FRt4/detection_loss‚2.559616576793582 .loss/FBt4/detection_loss‚3.517627847194672 2loss/SRqqt3/assignment_loss‚1.6733474463601656 2loss/SRqqt4/assignment_loss‚1.7275868894515234 0loss/FBt1/assignment_loss‚0.2630756498308204 0loss/FBt2/assignment_loss‚0.1821025119868643 /loss/FRt1/detection_loss‚2.4349497149630293 _runtime‚ 871.884883763 /loss/FRt4/assignment_loss‚4.152994449055472 2loss/SRqqt2/assignment_loss‚0.9464118789446397 .loss/SRqqt4/detection_loss‚2.0363448839122 epoch‚0 1loss/FBt3/assignment_loss‚0.26146830528920295 1loss/FBt4/assignment_loss‚0.16011713727299745 .loss/FRt2/detection_loss‚1.727128549736548 1loss/SRqqt1/detection_loss‚2.4100516327281496 /loss/FBt2/detection_loss‚0.7285131067142442 trainer/global_step‚499 % _timestamp‚1.7884860727833333e+09 /loss/FRt1/assignment_loss‚4.402127489811036 /loss/FRt3/assignment_loss‚4.369667196541689 /loss/FBt3/detection_loss‚2.3907053633169695 1loss/SRqqt2/detection_loss‚0.9801668529636669 _step‚39©¾4U›Ï 1loss/FBt3/assignment_loss‚0.26146830528920295 1loss/FBt4/assignment_loss‚0.16011713727299745 .loss/FRt2/detection_loss‚1.727128549736548 1loss/SRqqt1/detection_loss‚2.4100516327281496 /loss/FBt2/detection_loss‚0.7285131067142442 trainer/global_step‚499 % _timestamp‚1.7884860727833333e+09 /loss/FRt1/assignment_loss‚4.402127489811036 /loss/FRt3/assignment_loss‚4.369667196541689 /loss/FBt3/detection_loss‚2.3907053633169695 1loss/SRqqt2/detection_loss‚0.9801668529636669 _step‚39 1loss/SRqqt1/assignment_loss‚2.105851496111101 0loss/SRqqt3/detection_loss‚2.217291188390949 .loss/FBt1/detection_loss‚3.340232640504837 %loss/total_loss‚48.98305735924012 0loss/FRt2/assignment_loss‚1.8013593789110793 .loss/FRt3/detection_loss‚2.594319172454684 .loss/FRt4/detection_loss‚2.559616576793582 .loss/FBt4/detection_loss‚3.517627847194672 2loss/SRqqt3/assignment_loss‚1.6733474463601656 2loss/SRqqt4/assignment_loss‚1.7275868894515234 0loss/FBt1/assignment_loss‚0.2630756498308204 0loss/FBt2/assignment_loss‚0.1821025119868643 /loss/FRt1/detection_loss‚2.4349497149630293 _runtime‚ 871.884883763 /loss/FRt4/assignment_loss‚4.152994449055472 2loss/SRqqt2/assignment_loss‚0.9464118789446397 .loss/SRqqt4/detection_loss‚2.0363448839122 epoch‚0'Ù*§”œ‚(:‰ ÅÃèÔòlj° gpu.0.gpu‚97.0 gpu.0.memory‚17+ gpu.0.memoryAllocated‚47.54493713378906* gpu.0.memoryAllocatedBytes‚ 40840790016 gpu.0.temp‚45.0 gpu.0.powerWatts‚227.929' gpu.0.enforcedPowerLimitWatts‚400.0 gpu.0.powerPercent‚56.98225 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0Ô D)‚(:ž ÅÃèÔƒ¦‚± disk.sda.out‚89.63916015625 disk.nvme0n1.in‚ 0.25390625 disk.nvme1n1.in‚4 disk.sdb1.out‚0 disk.sda1.out‚0 disk.rbd2.in‚0 disk.sdb.out‚89.63916015625" disk.nvme1n1.out‚ 2459.33203125& proc.memory.availableMB‚ 1295.78125 cpu‚25.347719427482367 disk.dm-0.in‚ 4.25390625) disk./.usagePercent‚83.24010313988128 disk.sda1.in‚0 disk.rbd2.out‚0 disk.sdb.in‚0 proc.cpu.threads‚17 disk.sda.in‚ 0.0078125 disk.rbd0.out‚0 disk.sdb2.out‚89.63916015625 disk.md0.out‚ 86.12890625 network.recv‚362803 disk.md0p1.in‚ 0.0078125! proc.memory.rssMB‚ 2967.859375 network.sent‚846453 disk.rbd0.in‚0* proc.memory.percent‚18.114376068115234 disk.sdb2.in‚0 disk.md0.in‚ 0.0078125 disk.md0p1.out‚ 86.12890625 disk.sda2.out‚89.63916015625$ memory_percent‚92.09117889404297 disk.sda2.in‚ 0.0078125" disk.nvme0n1.out‚ 5881.50390625 disk.dm-0.out‚ 8340.8359375 disk.sdb1.in‚0% disk./.usageGB‚2977.06928253173838(b’ž‚(:‡ ÔÃèÔ󎉰 gpu.0.gpu‚96.0 gpu.0.memory‚23+ gpu.0.memoryAllocated‚47.54493713378906* gpu.0.memoryAllocatedBytes‚ 40840790016 gpu.0.temp‚44.0 gpu.0.powerWatts‚257.116' gpu.0.enforcedPowerLimitWatts‚400.0 gpu.0.powerPercent‚64.279 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0Ý7‰&Ÿ‚(:› ÔÃèÔëè’± disk.rbd0.in‚0 disk.dm-0.out‚ 8541.1953125 disk.sdb2.in‚0 disk.md0.in‚ 0.0078125 disk.sdb.in‚0 disk.sdb.out‚ 90.7763671875 disk.sda2.out‚ 90.7763671875 disk.sdb2.out‚ 90.7763671875 disk.nvme1n1.in‚4 cpu‚25.348290202269055 proc.cpu.threads‚17 disk.sda1.out‚0 disk.rbd2.in‚0 disk.sda2.in‚ 0.0078125 network.recv‚368664 disk.dm-0.in‚ 4.25390625 disk.sdb1.out‚0 disk.rbd2.out‚0 disk.sda.in‚ 0.0078125 disk.nvme0n1.in‚ 0.25390625 disk.md0.out‚ 87.1953125" disk.nvme1n1.out‚ 2459.58203125 disk.rbd0.out‚0 disk.sda1.in‚0! proc.memory.rssMB‚ 2967.859375" disk.nvme0n1.out‚ 6081.61328125 disk.md0p1.in‚ 0.0078125 disk.sda.out‚ 90.7763671875* proc.memory.percent‚18.114376068115234) disk./.usagePercent‚83.24628753085476% disk./.usageGB‚2977.2904663085938 disk.sdb1.in‚0 disk.md0p1.out‚ 87.1953125$ memory_percent‚92.09115505218506) proc.memory.availableMB‚ 1295.78515625 network.sent‚885181|³éˆ” ‚(:‰ ãÃèÔ©™–° gpu.0.gpu‚97.0 gpu.0.memory‚18+ gpu.0.memoryAllocated‚47.54493713378906* gpu.0.memoryAllocatedBytes‚ 40840790016 gpu.0.temp‚45.0 gpu.0.powerWatts‚212.105' gpu.0.enforcedPowerLimitWatts‚400.0 gpu.0.powerPercent‚53.02625 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0ŸÃ0>(¡‚(: ãÃèÔ–Ðô° network.recv‚372003 disk.sdb.out‚91.95947265625 disk.sda.in‚ 0.0078125' proc.memory.availableMB‚ 1295.765625 disk.md0p1.in‚ 0.0078125 disk.md0p1.out‚ 88.32421875 disk.rbd2.in‚0 disk.rbd2.out‚0 disk.sdb.in‚0 disk.sda2.in‚ 0.0078125$ memory_percent‚92.09127426147461! proc.memory.rssMB‚ 2967.859375 disk.nvme0n1.in‚ 0.25390625% disk./.usageGB‚2977.2899284362793 disk.sda2.out‚91.95947265625 network.sent‚888409 disk.sda1.out‚0 disk.md0.out‚ 88.32421875 disk.nvme0n1.out‚ 6246.171875 disk.dm-0.out‚ 8705.94140625 disk.sdb2.out‚91.95947265625 cpu‚25.34859704881002) disk./.usagePercent‚83.24627249171974" disk.nvme1n1.out‚ 2459.76953125 disk.rbd0.in‚0 disk.rbd0.out‚0 disk.dm-0.in‚ 4.25390625 disk.sdb2.in‚0 disk.sda1.in‚0 disk.nvme1n1.in‚4 proc.cpu.threads‚17 disk.md0.in‚ 0.0078125 disk.sda.out‚91.95947265625 disk.sdb1.in‚0 disk.sdb1.out‚0* proc.memory.percent‚18.114376068115234V°O“¢‚(:ˆ òÃèÔ¾£÷¯ gpu.0.gpu‚97.0 gpu.0.memory‚16+ gpu.0.memoryAllocated‚47.54493713378906* gpu.0.memoryAllocatedBytes‚ 40840790016 gpu.0.temp‚45.0 gpu.0.powerWatts‚231.046' gpu.0.enforcedPowerLimitWatts‚400.0 gpu.0.powerPercent‚57.7615 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0jPBÝ£‚(:‹ òÃèÔ½½Û°! proc.memory.rssMB‚ 2967.859375 network.recv‚376338( disk./.usagePercent‚83.2532650495451% disk./.usageGB‚2977.5400161743164 disk.dm-0.out‚ 8891.96875 disk.md0p1.in‚ 0.0078125 disk.rbd2.in‚0 disk.sda2.out‚ 93.095703125) proc.memory.availableMB‚ 1295.51953125 disk.sdb2.out‚ 93.095703125 disk.rbd0.in‚0$ memory_percent‚92.09277629852295 cpu‚25.34941526693567 disk.dm-0.in‚ 4.25390625 proc.cpu.threads‚17 disk.sdb2.in‚0 disk.nvme0n1.in‚ 0.25390625 disk.sda.in‚ 0.0078125 disk.md0p1.out‚89.40625 disk.sdb.out‚ 93.095703125 disk.sda2.in‚ 0.0078125 disk.nvme1n1.out‚ 2460.015625 disk.nvme0n1.out‚ 6439.828125* proc.memory.percent‚18.114376068115234 network.sent‚896466 disk.sda1.in‚0 disk.md0.in‚ 0.0078125 disk.sdb.in‚0 disk.rbd0.out‚0 disk.sda1.out‚0 disk.rbd2.out‚0 disk.nvme1n1.in‚4 disk.sdb1.in‚0 disk.sdb1.out‚0 disk.md0.out‚89.40625 disk.sda.out‚ 93.0957031258 ¤“¤‚(:ˆ ÄèÔÏí…° gpu.0.gpu‚99.0 gpu.0.memory‚23+ gpu.0.memoryAllocated‚47.54493713378906* gpu.0.memoryAllocatedBytes‚ 40840790016 gpu.0.temp‚45.0 gpu.0.powerWatts‚248.454' gpu.0.enforcedPowerLimitWatts‚400.0 gpu.0.powerPercent‚62.1135 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0ÜoÊ¥‚(: ÄèÔÄÝ° disk.sda1.out‚0! proc.memory.rssMB‚ 2967.859375 disk.sdb.out‚ 94.513671875) proc.memory.availableMB‚ 1295.75390625) disk./.usagePercent‚83.25326814270053 disk.sdb2.in‚0! disk.nvme1n1.out‚ 2460.1015625 disk.rbd0.in‚0 disk.dm-0.in‚ 4.25390625 proc.cpu.threads‚17 disk.nvme0n1.in‚ 0.25390625 disk.md0p1.out‚ 90.7734375$ disk./.usageGB‚2977.540126800537 disk.sdb2.out‚ 94.513671875 disk.md0p1.in‚ 0.0078125 disk.sdb.in‚0 disk.sda.in‚ 0.0078125 disk.sda.out‚ 94.513671875 network.recv‚380791 disk.md0.in‚ 0.0078125 disk.rbd2.out‚0 disk.sda2.out‚ 94.513671875 disk.nvme1n1.in‚4* proc.memory.percent‚18.114376068115234 network.sent‚902330 disk.sdb1.in‚0 disk.sdb1.out‚0 disk.sda1.in‚0 disk.md0.out‚ 90.7734375 disk.sda2.in‚ 0.0078125! disk.nvme0n1.out‚ 6575.4453125 cpu‚25.350196599597 disk.rbd2.in‚0 disk.dm-0.out‚ 9035.546875$ memory_percent‚92.09134578704834 disk.rbd0.out‚0ìº0¦+ _wandbruntime‚955  _runtime‚955RÖLp§¼ % lr-AdamW/pg2‚0.001033769576291948 trainer/global_step‚549 % _timestamp‚1.7884861565441954e+09 _runtime‚ 955.646567734 _step‚40 % lr-AdamW/pg1‚0.001033769576291948×&@‘ƨÀ _step‚40 % lr-AdamW/pg1‚0.001033769576291948 % lr-AdamW/pg2‚0.001033769576291948 trainer/global_step‚549 % _timestamp‚1.7884861565441954e+09 _runtime‚ 955.646567734(<{©‚0©+ _wandbruntime‚955  _runtime‚955Ùñ$˜é'ªãO VFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 37073453056 A8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚3680 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚18396 ]IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚117729944265728 NFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚330 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚29097984 XIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 3214934016 UADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚119025082515968 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚2684 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚2408775 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 WEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 1295163678208 H?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚2147 MDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚1533 VGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 3142090752 ]JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚26119889137152 dQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚24492201611776 KBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚2147 SGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚7646855 cQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 1627687525376 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚22388 O@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 2541128704 ZKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 3214934016 SGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚3695900 NFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚977 BDeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚119025082515968 TGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚10140003 \HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚113098813144400 UFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 3214934016 % _timestamp‚1.7884861565450017e+09 M=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 38610163200 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 YEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚114394879597128 YIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 37073453056 P@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 38610163200 OCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚7646849 SGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚2408772 IDeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 UADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚114392339805344 ^LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 1297679378944 YJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 2515700736 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚3 TDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 40288387072 D;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚3680 IDeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚2684 YEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚117729918837760 P@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 36985717077 MDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚2147 KBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚1533 RCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 2541128704 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚6 TEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 1497634304 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 `LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚113098838572368 ^LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 1296041024760 H?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚1533 WGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 35913877504 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚18396 H?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚1300 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 101282816 [KDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 37073453056Ô1üí'«çO M@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚17786858 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚22687 RBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 40288387072 O?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 40288387072 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚18390 SDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 3142090752 YFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚26119787854336 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚25427968 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚607 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 E8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚17768462 L@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚6104672 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚6 D>DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 UADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚114392339805344 ^LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 1297679378944 YJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 2515700736 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚3 TDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 40288387072 D;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚3680 IDeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚2684 YEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚117729918837760 P@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 36985717077 MDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚2147 KBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚1533 RCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 2541128704 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚6 TEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 1497634304 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 `LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚113098838572368 ^LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 1296041024760 H?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚1533 WGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 35913877504 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚18396 H?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚1300 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 101282816 [KDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 37073453056 VFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 37073453056 A8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚3680 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚18396 ]IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚117729944265728 NFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚330 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚29097984 XIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 3214934016 UADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚119025082515968 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚2684 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚2408775 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 WEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 1295163678208 H?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚2147 MDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚1533 VGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 3142090752 ]JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚26119889137152 dQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚24492201611776 KBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚2147 SGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚7646855 cQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 1627687525376 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚22388 O@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 2541128704 ZKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 3214934016 SGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚3695900 NFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚977 BDeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚119025082515968 TGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚10140003 \HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚113098813144400 UFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 3214934016 % _timestamp‚1.7884861565450017e+09 M=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 38610163200 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 YEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚114394879597128 YIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 37073453056 P@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 38610163200 OCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚7646849 SGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚2408772 IDeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚6115728 UEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 34283657216 G:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚17800694 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 [HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚26166965539328 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 WHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 2514364024 _runtime‚ 957.149517219 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚22687 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚2684 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚3 C6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚17800694 REDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚10158371 NADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚10139955 PADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 2539791992 D;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚3680 F=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚2147 MADZÜ…œeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚7660739 ?6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚3680 ?6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚3680 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚72171520 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 RBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 40288387072 M=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 40288387072 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 QBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 3142090752 aODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 1630633602560 J=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚10139955 TEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 3142090752 WGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 37073453056 trainer/global_step‚549 NADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚10158371 T@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚119243925824512 TEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 2515714048 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1300 YIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 37073453056 FDeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚17819116 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚22687 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 S?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚114600208601574 \JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 1298394160702 XIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 3214934016 B9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚3680 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚2413138 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 SDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 3214934016 M>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 2541142016 NBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚6116327 WCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚114602748393566 TDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 37073453056 WCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚119243925824512 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚25427968 WHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 2515714048 I@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚2147 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚6 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚3702590 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 101269504 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚25427968 _step‚42 G:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚17819116 ^KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚24536302838784 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 MADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚7660745 UEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 35913877504 LDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚977 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 561701376 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 VGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 3214934016 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚18416 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚25427968 F=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚1533 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚29097984 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚6 WDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚26166864269824 RCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 1497634304 bODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚24536331936768 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 KBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚2147 YGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 1300035488256 N>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 38610163200 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚596 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚18422 QEDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚7660745 ]KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 1630561431040 I=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚7660739 PDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 % _timestamp‚1.7884861580476298e+09 EDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 36985717077 KBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚1533 PADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 2541142016òãáÎ÷&®ñM WCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚119243925824512 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚25427968 WHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 2515714048 I@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚2147 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚6 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚3702590 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 101269504 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚25427968 _step‚42 G:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚17819116 ^KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚24536302838784 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 MADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚7660745 UEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 35913877504 LDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚977 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 561701376 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 VGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 3214934016 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚18416 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚25427968 F=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚1533 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚29097984 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚6 WDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚26166864269824 RCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 1497634304 bODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚24536331936768 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 KBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚2147 YGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 1300035488256 N>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 38610163200 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚596 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚18422 QEDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚7660745 ]KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 1630561431040 I=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚7660739 PDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 % _timestamp‚1.7884861580476298e+09 EDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 36985717077 KBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚1533 PADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 2541142016 LDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚330 BDeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚6115728 UEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 34283657216 G:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚17800694 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 [HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚26166965539328 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 WHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 2514364024 _runtime‚ 957.149517219 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚22687 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚2684 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚3 C6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚17800694 REDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚10158371 NADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚10139955 PADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 2539791992 D;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚3680 F=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚2147 MADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚7660739 ?6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚3680 ?6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚3680 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚72171520 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 RBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 40288387072 M=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 40288387072 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 QBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 3142090752 aODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 1630633602560 J=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚10139955 TEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 3142090752 WGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 37073453056 trainer/global_step‚549 NADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚10158371 T@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚119243925824512 TEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 2515714048 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1300 YIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 37073453056 FDeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚17819116 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚22687 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 S?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚114600208601574 \JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 1298394160702 XIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 3214934016 B9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚3680 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚2413138 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 SDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 3214934016 M>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 2541142016 NBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚6116327 WCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚114602748393566 TDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 37073453056*þ!t´0¯+ _wandbruntime‚957  _runtime‚957‘¿ZUV°Ð 0loss/FBt2/assignment_loss‚0.1573075573735166 /loss/FRt3/detection_loss‚2.9428855607758706 .loss/FBt2/detection_loss‚0.692782561196034 %loss/total_loss‚49.42105731568199 trainer/global_step‚549 _runtime‚ 957.151024675 0loss/FRt4/assignment_loss‚3.4270837116858055 2loss/SRqqt2/assignment_loss‚0.9751392005487765 2loss/SRqqt3/assignment_loss‚2.1907944254080456 1loss/FBt1/assignment_loss‚0.41647923809068743 0loss/SRqqt3/detection_loss‚2.473212838570277 /loss/FRt3/assignment_loss‚5.339679607704504 0loss/SRqqt1/assignment_loss‚2.32095675966511 /loss/FRt1/detection_loss‚2.3574419102885504 epoch‚0 .loss/FRt4/detection_loss‚2.140002799959018 .loss/FBt1/detection_loss‚2.724481374025345 1loss/SRqqt1/detection_loss‚2.4692857134832096 % _timestamp‚1.7884861580498657e+09 _step‚43 /loss/FRt2/detection_loss‚1.6641193221662647 1loss/SRqqt2/detection_loss‚1.1203014114089596 /loss/FBt3/detection_loss‚2.9930708536079953 /loss/FBt4/detection_loss‚3.1029237070504356 0loss/FRt2/assignment_loss‚1.3246801010916167 1loss/FBt4/assignment_loss‚0.18989886561073088 /loss/FRt1/assignment_loss‚4.821153846654025 1loss/SRqqt4/assignment_loss‚1.720038879164465 1loss/FBt3/assignment_loss‚0.26301449217966627 1loss/SRqqt4/detection_loss‚1.5943225779730803¥«Þ–Z±Ô 0loss/FRt4/assignment_loss‚3.4270837116858055 2loss/SRqqt2/assignment_loss‚0.9751392005487765 2loss/SRqqt3/assignment_loss‚2.1907944254080456 1loss/FBt1/assignment_loss‚0.41647923809068743 0loss/SRqqt3/detection_loss‚2.473212838570277 /loss/FRt3/assignment_loss‚5.339679607704504 0loss/SRqqt1/assignment_loss‚2.32095675966511 /loss/FRt1/detection_loss‚2.3574419102885504 epoch‚0 .loss/FRt4/detection_loss‚2.140002799959018 .loss/FBt1/detection_loss‚2.724481374025345 1loss/SRqqt1/detection_loss‚2.4692857134832096 % _timestamp‚1.7884861580498657e+09 _step‚43 /loss/FRt2/detection_loss‚1.6641193221662647 1loss/SRqqt2/detection_loss‚1.1203014114089596 /loss/FBt3/detection_loss‚2.9930708536079953 /loss/FBt4/detection_loss‚3.1029237070504356 0loss/FRt2/assignment_loss‚1.3246801010916167 1loss/FBt4/assignment_loss‚0.18989886561073088 /loss/FRt1/assignment_loss‚4.821153846654025 1loss/SRqqt4/assignment_loss‚1.720038879164465 1loss/FBt3/assignment_loss‚0.26301449217966627 1loss/SRqqt4/detection_loss‚1.5943225779730803 0loss/FBt2/assignment_loss‚0.1573075573735166 /loss/FRt3/detection_loss‚2.9428855607758706 .loss/FBt2/detection_loss‚0.692782561196034 %loss/total_loss‚49.42105731568199 trainer/global_step‚549 _runtime‚ 957.151024675+"Z«N”²‚(:‰ ÄèÔŠ…þ° gpu.0.gpu‚97.0 gpu.0.memory‚17+ gpu.0.memoryAllocated‚47.54493713378906* gpu.0.memoryAllocatedBytes‚ 40840790016 gpu.0.temp‚45.0 gpu.0.powerWatts‚226.473' gpu.0.enforcedPowerLimitWatts‚400.0 gpu.0.powerPercent‚56.61825 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚06&tL*³‚(:Ÿ ÄèÔʇÀ± disk.sdb2.in‚0 network.sent‚919116 disk.md0p1.in‚ 0.0078125 disk.dm-0.out‚ 9196.17578125 network.recv‚385754 disk.md0.out‚ 91.80859375 disk.nvme1n1.out‚ 2460.390625$ memory_percent‚92.09578037261963 proc.cpu.threads‚17 disk.md0p1.out‚ 91.80859375 disk.sdb.in‚0 disk.rbd0.out‚0 disk.dm-0.in‚ 4.25390625) proc.memory.availableMB‚ 1295.02734375$ disk./.usageGB‚2976.486042022705 disk.md0.in‚ 0.0078125 disk.sdb1.out‚0 disk.sdb2.out‚95.60888671875) disk./.usagePercent‚83.22379549114359 disk.sdb.out‚95.60888671875 disk.sda2.out‚95.60888671875 disk.nvme1n1.in‚4 disk.nvme0n1.in‚ 0.25390625* proc.memory.percent‚18.114376068115234 cpu‚25.349898604517037 disk.rbd2.out‚0 disk.sda2.in‚ 0.0078125 disk.sda.in‚ 0.0078125 disk.rbd0.in‚0" disk.nvme0n1.out‚ 6735.78515625 disk.sda.out‚95.60888671875 disk.sdb1.in‚0! proc.memory.rssMB‚ 2967.859375 disk.rbd2.in‚0 disk.sda1.in‚0 disk.sda1.out‚00ªš’´‚(:‡ ŸÄèÔ©Û£° gpu.0.gpu‚98.0 gpu.0.memory‚18+ gpu.0.memoryAllocated‚47.54493713378906* gpu.0.memoryAllocatedBytes‚ 40840790016 gpu.0.temp‚45.0 gpu.0.powerWatts‚225.248' gpu.0.enforcedPowerLimitWatts‚400.0 gpu.0.powerPercent‚56.312 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0i†&×(µ‚(: ŸÄèÔ¡ÓÞ° disk.nvme0n1.out‚ 6959.34375 disk.sdb1.in‚0 disk.md0.in‚ 0.0078125 disk.md0.out‚ 93.06640625 network.sent‚956974 network.recv‚390238 disk.dm-0.out‚ 9419.8984375 disk.sdb2.out‚96.94091796875 disk.md0p1.out‚ 93.06640625 disk.nvme1n1.in‚4! proc.memory.rssMB‚ 2967.859375 cpu‚25.348939872162344 disk.rbd2.in‚0) proc.memory.availableMB‚ 1294.86328125* proc.memory.percent‚18.114376068115234 disk.sdb2.in‚0 disk.sda1.in‚0 disk.md0p1.in‚ 0.0078125! disk.nvme1n1.out‚ 2460.5546875$ memory_percent‚92.09678173065186 disk.sda1.out‚0 disk.sda2.in‚ 0.0078125 disk.sdb1.out‚0) disk./.usagePercent‚83.20903090032411% disk./.usageGB‚2975.9579887390137 disk.sda.in‚ 0.0078125 disk.rbd0.out‚0 disk.dm-0.in‚ 4.25390625 disk.rbd2.out‚0 disk.sda.out‚96.94091796875 disk.nvme0n1.in‚ 0.25390625 disk.sdb.in‚0 disk.sda2.out‚96.94091796875 disk.sdb.out‚96.94091796875 proc.cpu.threads‚17 disk.rbd0.in‚0j}‘¶‚(:† ®ÄèÔÊÚ§® gpu.0.gpu‚44.0 gpu.0.memory‚11+ gpu.0.memoryAllocated‚47.54493713378906* gpu.0.memoryAllocatedBytes‚ 40840790016 gpu.0.temp‚47.0 gpu.0.powerWatts‚323.74' gpu.0.enforcedPowerLimitWatts‚400.0 gpu.0.powerPercent‚80.935 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚035ìM'·‚(:œ ®ÄèÔŽ¹Û®" disk.nvme1n1.out‚ 2460.71484375$ memory_percent‚92.09668636322021* proc.memory.percent‚18.114376068115234 cpu‚25.34881525418404) disk./.usagePercent‚83.20778083888764 disk.sdb2.out‚98.53955078125 disk.sda.in‚ 0.0078125 disk.rbd0.out‚0 disk.sdb2.in‚0 disk.sda1.in‚0 disk.sda1.out‚0 disk.md0.in‚ 0.0078125 disk.md0.out‚ 94.59765625 disk.rbd2.in‚0) proc.memory.availableMB‚ 1294.87890625 network.recv‚393694 disk.sdb1.in‚0 disk.sdb1.out‚0 disk.rbd2.out‚0 disk.sdb.in‚0 disk.rbd0.in‚0 disk.dm-0.out‚9420.875 disk.sdb.out‚98.53955078125 disk.sda.out‚98.53955078125% disk./.usageGB‚2975.9132804870605 disk.nvme0n1.in‚ 0.25390625 disk.sda2.in‚ 0.0078125 proc.cpu.threads‚17 disk.md0p1.out‚ 94.59765625 disk.sda2.out‚98.53955078125 disk.md0p1.in‚ 0.0078125 disk.nvme1n1.in‚4! proc.memory.rssMB‚ 2967.859375 network.sent‚961923" disk.nvme0n1.out‚ 6960.16015625 disk.dm-0.in‚ 4.25390625ëÍÏM ¸‚(:• ½ÄèÔâ畯! disk.nvme0n1.out‚ 7033.4296875 disk.sdb1.in‚0 disk.sda2.in‚ 0.0078125 disk.md0.in‚ 0.0078125 disk.md0.out‚ 95.640625 disk.rbd2.out‚0 disk.sda.in‚ 0.0078125 disk.rbd0.in‚0 disk.sda2.out‚99.65087890625 disk.sda.out‚99.65087890625$ proc.memory.availableMB‚1295.375! proc.memory.rssMB‚ 2967.859375* proc.memory.percent‚18.114376068115234) disk./.usagePercent‚83.20688019735441 disk.dm-0.in‚ 4.25390625 disk.dm-0.out‚ 9494.234375 disk.sdb2.out‚99.65087890625 disk.nvme1n1.in‚4% disk./.usageGB‚2975.8810691833496 disk.nvme0n1.in‚ 0.25390625 disk.sda1.in‚0 disk.sda1.out‚0 disk.md0p1.out‚ 95.640625 disk.rbd2.in‚0$ memory_percent‚92.09365844726562 network.sent‚967759 disk.sdb.out‚99.65087890625! disk.nvme1n1.out‚ 2460.8046875 proc.cpu.threads‚17 disk.sdb1.out‚0 disk.sdb2.in‚0 cpu‚25.348080593042976 network.recv‚397664 disk.md0p1.in‚ 0.0078125 disk.sdb.in‚0 disk.rbd0.out‚0DÑ–”¹‚(:‰ ½ÄèÔƃ™° gpu.0.gpu‚98.0 gpu.0.memory‚17+ gpu.0.memoryAllocated‚47.54493713378906* gpu.0.memoryAllocatedBytes‚ 40840790016 gpu.0.temp‚45.0 gpu.0.powerWatts‚242.273' gpu.0.enforcedPowerLimitWatts‚400.0 gpu.0.powerPercent‚60.56825 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0F÷Ôžº‚(:“ ÌÄèÔ« ±¯ gpu.0.gpu‚98.0 gpu.0.memory‚18+ gpu.0.memoryAllocated‚47.54493713378906* gpu.0.memoryAllocatedBytes‚ 40840790016 gpu.0.temp‚45.0 gpu.0.powerWatts‚222.968' gpu.0.enforcedPowerLimitWatts‚400.0) gpu.0.powerPercent‚55.742000000000004 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0Ô6\©8»‚(:­ ÌÄèÔ㻄± disk.sda2.in‚ 0.0078125 disk.rbd0.out‚ 0.07421875 disk.dm-0.in‚ 4.25390625! disk.sdb2.out‚100.77587890625 disk.md0p1.out‚96.71875* proc.memory.percent‚18.114376068115234 proc.cpu.threads‚17 disk.rbd2.in‚0 disk.sdb.in‚0 disk.sdb.out‚100.77587890625 disk.sda.out‚100.77587890625! disk.sda2.out‚100.77587890625 disk.nvme1n1.in‚4 disk.nvme0n1.in‚ 0.25390625 disk.md0p1.in‚ 0.0078125) proc.memory.availableMB‚ 1295.12890625 disk.sdb1.out‚0 disk.md0.out‚96.71875$ memory_percent‚92.09516048431396 network.sent‚971810$ disk./.usageGB‚2976.004814147949 disk.sdb1.in‚0 disk.sda1.in‚0 cpu‚25.347295492518473 disk.sda1.out‚0 network.recv‚402206( disk./.usagePercent‚83.2103401583559! disk.nvme0n1.out‚ 7109.9765625! proc.memory.rssMB‚ 2967.859375" disk.nvme1n1.out‚ 2461.12890625 disk.rbd0.in‚0 disk.sda.in‚ 0.0078125 disk.dm-0.out‚ 9571.10546875 disk.sdb2.in‚0 disk.md0.in‚ 0.0078125 disk.rbd2.out‚0.015625ìoꔼ‚(:‰ ÛÄèÔטհ gpu.0.gpu‚74.0 gpu.0.memory‚17+ gpu.0.memoryAllocated‚47.54493713378906* gpu.0.memoryAllocatedBytes‚ 40840790016 gpu.0.temp‚48.0 gpu.0.powerWatts‚320.009' gpu.0.enforcedPowerLimitWatts‚400.0 gpu.0.powerPercent‚80.00225 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0|À|o1½‚(:¦ ÛÄèÔ•î°$ proc.memory.availableMB‚1295.375* proc.memory.percent‚18.114376068115234$ disk./.usageGB‚2976.129814147949 disk.md0.in‚ 0.0078125 disk.md0p1.out‚ 98.4296875 disk.sdb.out‚102.5419921875 disk.sda.in‚ 0.0078125 disk.sda.out‚102.5419921875 proc.cpu.threads‚17) disk./.usagePercent‚83.21383521067247 disk.sdb2.in‚0 disk.rbd0.out‚ 0.07421875 cpu‚25.346398962357597 disk.sdb1.out‚0 disk.md0p1.in‚ 0.0078125 disk.sda2.in‚ 0.0078125 disk.sda2.out‚102.5419921875 network.sent‚977565" disk.nvme0n1.out‚ 7187.55078125 disk.sda1.in‚0 disk.sda1.out‚0 disk.md0.out‚ 98.4296875 disk.sdb1.in‚0 disk.rbd0.in‚0 disk.dm-0.in‚ 4.25390625 disk.dm-0.out‚ 9648.8125 disk.sdb2.out‚102.5419921875 disk.rbd2.out‚0.078125 disk.sdb.in‚0" disk.nvme1n1.out‚ 2461.26171875$ memory_percent‚92.09365844726562! proc.memory.rssMB‚ 2967.859375 disk.nvme1n1.in‚4 network.recv‚406215 disk.nvme0n1.in‚ 0.25390625 disk.rbd2.in‚0u^žþ2¾- _wandbruntime‚1037  _runtime‚1037ÞÃlGÄ¿¾ & lr-AdamW/pg1‚0.0011279198109269157 & lr-AdamW/pg2‚0.0011279198109269157 trainer/global_step‚599 $ _timestamp‚1.788486238774199e+09 _runtime‚1037.875895579 _step‚44§=ÈÀ & lr-AdamW/pg2‚0.0011279198109269157 trainer/global_step‚599 $ _timestamp‚1.788486238774199e+09 _runtime‚1037.875895579 _step‚44 & lr-AdamW/pg1‚0.0011279198109269157,aëÎ82Á- _wandbruntime‚1037  _runtime‚1037ˆXœ„ ÂãO SDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 3142090752 YEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚129842759799296 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚590 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 E8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚19383414 G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚2684 ^LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 1415512007168 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚593 `MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚26698688763392 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 [KDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 37073453056 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚25427968 \HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚128427222364160 D;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚3680 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚22687 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚2627075 NFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚977 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚70087680 MDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚1533 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚18396 M=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 38610163200 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚2684 SGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚2627072 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚25427968 UFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 3214934016 ]IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚128427247792128 P@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 38610163200 cQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 1775388839424 @:DeviceStatsM³leonitor.on_train_batch_start/num_oom_rejections‚0 `LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚128427247792128 IDeviceStatsMonitor.on_train_batch_start/allocation.all.current‚18396 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 VFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 37073453056 R>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚129840218670592 WGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 35913877504 D>DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 YIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 37073453056 YEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚128427222364160 UADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚129840218670592 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 A8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚3680 KBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚2147 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚29097984 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 UADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚124785806788852 PCDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated‚11060455 VBDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated‚129842759799296 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 H?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚2147 WGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 34283657216 TDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 40288387072 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚18390 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚22388 TEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 1497634304 O?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 40288387072 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚6 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚4034220 MDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚2147 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 ZHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 1411210619652 XIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 3214934016 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚3 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚6 VGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 3140732164 _step‚45 O@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 2541128704 RCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 2541128704 M@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚19401810 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚18390 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 \HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚123374596169200 YJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 2514363816 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚22687 NFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚330 ]JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚28474106700800 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 963120128 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 _MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 1775318751744 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 KBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚1533 % _timestamp‚1.7884862387752364e+09 VGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 3142090752 P@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 36985717077 _runtime‚1037.877039795 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0›äóí'ÃçO ZKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 3214934016 IDeviceStatsMonitor.on_train_batch_start/allocation.all.current‚18396 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 VFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 37073453056 R>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚129840218670592 WGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 35913877504 D>DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 YIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 37073453056 YEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚128427222364160 UADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚129840218670592 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 A8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚3680 KBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚2147 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚29097984 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 UADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚124785806788852 PCDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated‚11060455 VBDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated‚129842759799296 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 H?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚2147 WGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 34283657216 TDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 40288387072 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚18390 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚22388 TEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 1497634304 O?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 40288387072 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚6 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚4034220 MDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚2147 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 ZHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 1411210619652 XIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 3214934016 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚3 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚6 VGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 3140732164 _step‚45 O@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 2541128704 RCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 2541128704 M@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚19401810 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚18390 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 \HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚123374596169200 YJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 2514363816 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚22687 NFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚330 ]JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚28474106700800 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 963120128 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 _MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 1775318751744 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 KBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚1533 % _timestamp‚1.7884862387752364e+09 VGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 3142090752 P@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 36985717077 _runtime‚1037.877039795 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 SDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 3142090752 YEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚129842759799296 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚590 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 E8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚19383414 G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚2684 ^LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 1415512007168 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚593 `MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚26698688763392 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 [KDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 37073453056 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚25427968 \HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚128427222364160 D;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚3680 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚22687 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚2627075 NFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚977 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚70087680 MDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚1533 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚18396 M=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 38610163200 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚2684 SGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚2627072 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚25427968 UFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 3214934016 ]IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚128427247792128 P@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 38610163200 cQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 1775388839424 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 `LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚128427247792128 IDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 38610163200 QEDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚8355245 NADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚11060407 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚580 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚70074368 TDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 37073453056 VGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 3214934016 aODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 1778339181056 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 N>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 36985717077 WHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 2514364024 S?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚130056520837120 ^JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚128641193862656 $ _timestamp‚1.788486240282426e+09 EDeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 M>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 2541142016 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚6 F=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚2147 F=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚1533 UEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 35913877504 RCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 1497634304 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 B9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚3680 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚22687 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚2631441 \JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 1417868116480 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚18416 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 J=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚11060407 T@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚130059061979136 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚2684 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚3 WDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚28521088195072 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 963120128 ]KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 1778269106688 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 PDeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚19434068 ?6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚3680 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 trainer/global_step‚599 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚18422 K;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 38610163200 J>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚6671749 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚4040311 SDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 3214934016 TEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 3142090752 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚2631438 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 YGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 1417868116480 QBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 3142090752 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 TEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 2515714048 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚25427968 KBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚1533 FDeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 C6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚19415646 LDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚330 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 UCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 1415352402432 NBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚6672329 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚577 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 _runtime‚1039.384573771 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚25427968 XFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 1415352402432 G:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚19415646 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1300 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 KBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚2147 SFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚99172352 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 B8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 BDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 38610163200 QEDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚8355245 NADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚11060407 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚580 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚70074368 TDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 37073453056 VGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 3214934016 aODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 1778339181056 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 N>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 36985717077 WHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 2514364024 S?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚130056520837120 ^JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚128641193862656 $ _timestamp‚1.788486240282426e+09 EDeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 M>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 2541142016 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚6 F=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚2147 F=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚1533 UEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 35913877504 RCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 1497634304 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 B9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚3680 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚22687 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚2631441 \JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 1417868116480 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚18416 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 J=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚11060407 T@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚130059061979136 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚2684 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚3 WDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚28521088195072 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 963120128 ]KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 1778269106688 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 PDeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚19434068 ?6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚3680 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 trainer/global_step‚599 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚18422 K;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 38610163200 J>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚6671749 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚4040311 SDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 3214934016 TEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 3142090752 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚2631438 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 YGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 1417868116480 QBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 3142090752 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 TEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 2515714048 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚25427968 KBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚1533 FDeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 C6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚19415646 LDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚330 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 UCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 1415352402432.éc¼S2Ç- _wandbruntime‚1039  _runtime‚1039¥‹?ƒXÈÒ 0loss/SRqqt2/detection_loss‚1.025323919736403 0loss/SRqqt3/detection_loss‚2.467832362279296 0loss/FRt2/assignment_loss‚1.3678934471360569 /loss/FRt2/detection_loss‚1.9109305425964553 /loss/FRt3/detection_loss‚2.7901893193657332 .loss/FBt3/detection_loss‚3.248075598478317 _step‚47 /loss/FBt2/detection_loss‚0.6330523880422171 epoch‚0 2loss/SRqqt4/assignment_loss‚1.8212813285645097 _runtime‚ 1039.38643374 /loss/FRt1/assignment_loss‚4.582735051001821 /loss/FRt3/assignment_loss‚4.708382964134216 2loss/SRqqt2/assignment_loss‚1.3569259585982019 1loss/SRqqt1/detection_loss‚2.4148645900391243 2loss/SRqqt1/assignment_loss‚2.1003710928599575 1loss/FBt1/assignment_loss‚0.18677978104223375 0loss/FBt3/assignment_loss‚0.4566894770599902 /loss/FRt1/detection_loss‚2.3152128216766177 % _timestamp‚1.7884862402843592e+09 1loss/FBt2/assignment_loss‚0.20921187789721224 /loss/FRt4/detection_loss‚2.2589539119175504 0loss/SRqqt4/detection_loss‚1.821417092345655 /loss/FBt4/detection_loss‚3.0909402929246426 &loss/total_loss‚49.583785001364596 /loss/FRt4/assignment_loss‚3.502278140613011 /loss/FBt1/detection_loss‚2.6115784541420313 trainer/global_step‚599 2loss/SRqqt3/assignment_loss‚1.9144957477692515 0loss/FBt4/assignment_loss‚0.7883688411440914¹$$\ÉÖ 1loss/SRqqt1/detection_loss‚2.4148645900391243 2loss/SRqqt1/assignment_loss‚2.1003710928599575 1loss/FBt1/assignment_loss‚0.18677978104223375 0loss/FBt3/assignment_loss‚0.4566894770599902 /loss/FRt1/detection_loss‚2.3152128216766177 % _timestamp‚1.7884862402843592e+09 1loss/FBt2/assignment_loss‚0.20921187789721224 /loss/FRt4/detection_loss‚2.2589539119175504 0loss/SRqqt4/detection_loss‚1.821417092345655 /loss/FBt4/detection_loss‚3.0909402929246426 &loss/total_loss‚49.583785001364596 /loss/FRt4/assignment_loss‚3.502278140613011 /loss/FBt1/detection_loss‚2.6115784541420313 trainer/global_step‚599 2loss/SRqqt3/assignment_loss‚1.9144957477692515 0loss/FBt4/assignment_loss‚0.7883688411440914 0loss/SRqqt2/detection_loss‚1.025323919736403 0loss/SRqqt3/detection_loss‚2.467832362279296 0loss/FRt2/assignment_loss‚1.3678934471360569 /loss/FRt2/detection_loss‚1.9109305425964553 /loss/FRt3/detection_loss‚2.7901893193657332 .loss/FBt3/detection_loss‚3.248075598478317 _step‚47 /loss/FBt2/detection_loss‚0.6330523880422171 epoch‚0 2loss/SRqqt4/assignment_loss‚1.8212813285645097 _runtime‚ 1039.38643374 /loss/FRt1/assignment_loss‚4.582735051001821 /loss/FRt3/assignment_loss‚4.708382964134216 2loss/SRqqt2/assignment_loss‚1.3569259585982019/Aï|”Ê‚(:‰ êÄèÔ‹ëÉ° gpu.0.gpu‚99.0 gpu.0.memory‚22+ gpu.0.memoryAllocated‚47.54493713378906* gpu.0.memoryAllocatedBytes‚ 40840790016 gpu.0.temp‚45.0 gpu.0.powerWatts‚317.227' gpu.0.enforcedPowerLimitWatts‚400.0 gpu.0.powerPercent‚79.30675 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0ÆÉD{<Ë‚(:± êÄèÔöÆß±" disk.nvme0n1.out‚ 7268.80859375 network.recv‚410781 disk.sda1.out‚0 disk.rbd2.in‚0 disk.md0.in‚ 0.0078125) disk./.usagePercent‚83.22086936610536" disk.nvme1n1.out‚ 2461.50390625 disk.sda.in‚ 0.0078125 disk.rbd0.out‚ 0.17626953125 disk.nvme0n1.in‚ 0.25390625 disk.sdb1.in‚0 disk.sdb.out‚103.63037109375# disk./.usageGB‚2976.38138961792 disk.sda2.in‚ 0.0078125 disk.dm-0.out‚ 9730.3125 disk.sdb1.out‚0 disk.sdb2.in‚0 disk.md0p1.out‚ 99.46484375 cpu‚25.35617614883561 disk.md0p1.in‚ 0.0078125 disk.sdb.in‚0$ memory_percent‚92.92402267456055# proc.memory.rssMB‚ 3115.71484375) proc.memory.percent‚19.01681423187256 disk.md0.out‚ 99.46484375 disk.rbd2.out‚0.078125 proc.cpu.threads‚17! disk.sdb2.out‚103.63037109375 disk.sda1.in‚0 disk.nvme1n1.in‚4 disk.sda.out‚103.63037109375' proc.memory.availableMB‚ 1159.328125 network.sent‚983603! disk.sda2.out‚103.63037109375 disk.rbd0.in‚0 disk.dm-0.in‚ 4.25390625¾$Že”Ì‚(:‰ ùÄèÔ™Žû® gpu.0.gpu‚91.0 gpu.0.memory‚20+ gpu.0.memoryAllocated‚47.54493713378906* gpu.0.memoryAllocatedBytes‚ 40840790016 gpu.0.temp‚46.0 gpu.0.powerWatts‚265.139' gpu.0.enforcedPowerLimitWatts‚400.0 gpu.0.powerPercent‚66.28475 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0Hv(IÍ‚(:¾ ùÄèÔ㙫¯ disk.rbd2.out‚ 0.16552734375 disk.sda.out‚105.1337890625 disk.sdb1.in‚0 disk.sdb2.out‚105.1337890625* proc.memory.percent‚18.583035469055176 disk.sda2.in‚ 0.0078125 disk.nvme0n1.in‚ 0.25390625 disk.sdb.in‚0 disk.nvme1n1.in‚4 disk.sda1.out‚0 cpu‚25.426185913225925$ disk./.usageGB‚2976.535842895508 disk.rbd0.in‚0 disk.sdb2.in‚0 disk.md0.in‚ 0.0078125 disk.md0p1.in‚ 0.0078125$ memory_percent‚92.56489276885986 network.recv‚415942" disk.nvme0n1.out‚ 7451.69140625 disk.sdb1.out‚0# proc.memory.rssMB‚ 3044.64453125 disk.sdb.out‚105.1337890625" disk.nvme1n1.out‚ 2461.62109375 disk.rbd0.out‚ 0.17626953125 disk.dm-0.in‚ 4.25390625 disk.sda1.in‚0 network.sent‚ 1.03484e+06) disk./.usagePercent‚83.22518794439044 disk.sda2.out‚105.1337890625 disk.sda.in‚ 0.0078125 disk.md0.out‚ 100.90234375) proc.memory.availableMB‚ 1218.16796875 proc.cpu.threads‚17 disk.rbd2.in‚0 disk.dm-0.out‚ 9913.3125 disk.md0p1.out‚ 100.90234375ë#‘΂(:† ˆÅèÔû”Í® gpu.0.gpu‚90.0 gpu.0.memory‚19+ gpu.0.memoryAllocated‚47.54493713378906* gpu.0.memoryAllocatedBytes‚ 40840790016 gpu.0.temp‚47.0 gpu.0.powerWatts‚325.02' gpu.0.enforcedPowerLimitWatts‚400.0 gpu.0.powerPercent‚81.255 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0iùN¤bI &Purity/2FRt2SRqqt1FBt/event_proportion"trainer/global_step2:JPž“*€;¥b6 jet/accuracy_2_of_5"trainer/global_step2:JPöX—Ã@¦b; particle/accuracy_2_of_4"trainer/global_step2:JPq”ŒõJ§bE "Purity/1FRt2SRqqt2FBt/event_purity"trainer/global_step2:JPØØ£J¨bE "Purity/1FRt*SRqqt2FBt/SRqqt_purity"trainer/global_step2:JPÏì`K@©b; particle/accuracy_1_of_2"trainer/global_step2:JP ÃÂÆJªbE "Purity/1FRt2SRqqt2FBt/SRqqt_purity"trainer/global_step2:JPéêwH«bC Purity/*FRt*SRqqt3FBt/FBt_purity"trainer/global_step2:JP]eqH¬bC Purity/*FRt3SRqqt1FBt/FBt_purity"trainer/global_step2:JPèjÀuJ­bE "Purity/3FRt2SRqqt1FBt/SRqqt_purity"trainer/global_step2:JPÃÔÅé@®b; particle/accuracy_7_of_7"trainer/global_step2:JP£ŸýJ¯bE "Purity/4FRt1SRqqt*FBt/SRqqt_purity"trainer/global_step2:JP†¡ÉN°bI &Purity/*FRt2SRqqt*FBt/event_proportion"trainer/global_step2:JP¦N±bI &Purity/0FRt*SRqqt4FBt/event_proportion"trainer/global_step2:JP l©ôJ²bE "Purity/0FRt2SRqqt1FBt/SRqqt_purity"trainer/global_step2:JP6d²lN³bI &Purity/0FRt4SRqqt3FBt/event_proportion"trainer/global_step2:JP š;J´bE "Purity/1FRt1SRqqt1FBt/event_purity"trainer/global_step2:JP ô‡NµbI &Purity/4FRt1SRqqt4FBt/event_proportion"trainer/global_step2:JPϤ,ÌN¶bI &Purity/*FRt*SRqqt2FBt/event_proportion"trainer/global_step2:JP#ä.H·bC Purity/*FRt*SRqqt2FBt/FRt_purity"trainer/global_step2:JPž;ÖgJ¸bE "Purity/*FRt0SRqqt0FBt/event_purity"trainer/global_step2:JP»M˜¨H¹bC Purity/*FRt3SRqqt1FBt/FRt_purity"trainer/global_step2:JP{uxINºbI &Purity/1FRt2SRqqt2FBt/event_proportion"trainer/global_step2:JPgd¨ÍH»bC Purity/3FRt0SRqqt*FBt/FBt_purity"trainer/global_step2:JP gÁ•H¼bC Purity/3FRt0SRqqt0FBt/FRt_purity"trainer/global_step2:JPpï—tJ½bE "Purity/2FRt4SRqqt0FBt/event_purity"trainer/global_step2:JP®txH¾bC Purity/0FRt1SRqqt*FBt/FBt_purity"trainer/global_step2:JP2(¦óH¿bC Purity/1FRt0SRqqt3FBt/FRt_purity"trainer/global_step2:JPë…êNÀbI &Purity/1FRt3SRqqt0FBt/event_proportion"trainer/global_step2:JP*wâFJÁbE "Purity/2FRt*SRqqt1FBt/event_purity"trainer/global_step2:JP_t¤øJÂbE "Purity/1FRt*SRqqt4FBt/event_purity"trainer/global_step2:JP θãJÃbE "Purity/*FRt2SRqqt0FBt/event_purity"trainer/global_step2:JPã DNÄbI &Purity/2FRt*SRqqt2FBt/event_proportion"trainer/global_step2:JPè·ˆEJÅbE "Purity/2FRt2SRqqt0FBt/event_purity"trainer/global_step2:JP!•^âJÆbE "Purity/2FRt3SRqqt*FBt/SRqqt_purity"trainer/global_step2:JPHÄMéNÇbI &Purity/2FRt3SRqqt4FBt/event_proportion"trainer/global_step2:JPƬYNÈbI &Purity/*FRt4SRqqt3FBt/event_proportion"trainer/global_step2:JPµL[¦HÉbC Purity/3FRt1SRqqt*FBt/FBt_purity"trainer/global_step2:JPù87 HÊbC Purity/3FRt1SRqqt0FBt/FRt_purity"trainer/global_step2:JPèWê;Ëb6 jet/accuracy_3_of_6"trainer/global_step2:JP anÖJÌbE "Purity/1FRt4SRqqt0FBt/SRqqt_purity"trainer/global_step2:JP’UyúNÍbI &Purity/0FRt*SRqqt0FBt/event_proportion"trainer/global_step2:JPÍünšNÎbI &Purity/2FRt4SRqqt*FBt/event_proportion"trainer/global_step2:JPsu¹NÏbI &Purity/3FRt2SRqqt2FBt/event_proportion"trainer/global_step2:JP ØëHÐbC Purity/*FRt*SRqqt4FBt/FBt_purity"trainer/global_step2:JP¥0µÃHÑbC Purity/1FRt2SRqqt2FBt/FBt_purity"trainer/global_step2:JP?ÂH¼NÒbI &Purity/0FRt1SRqqt0FBt/event_proportion"trainer/global_step2:JPãÞvHÓbC Purity/2FRt4SRqqt0FBt/FRt_purity"trainer/global_step2:JPN~¼JÔbE "Purity/0FRt0SRqqt*FBt/event_purity"trainer/global_step2:JPYˆÏ]NÕbI &Purity/0FRt0SRqqt2FBt/event_proportion"trainer/global_step2:JPHùNÖbI &Purity/0FRt1SRqqt3FBt/event_proportion"trainer/global_step2:JP´ÔáãJ×bE "Purity/1FRt*SRqqt1FBt/event_purity"trainer/global_step2:JP3ÑéÎNØbI &Purity/2FRt4SRqqt1FBt/event_proportion"trainer/global_step2:JPŠ „±JÙbE "Purity/*FRt4SRqqt1FBt/event_purity"trainer/global_step2:JPþL÷HÚbC Purity/2FRt*SRqqt3FBt/FRt_purity"trainer/global_step2:JP¤zT¾HÛbC Purity/0FRt1SRqqt3FBt/FBt_purity"trainer/global_step2:JP_FD‡NÜbI &Purity/0FRt*SRqqt*FBt/event_proportion"trainer/global_step2:JP)©·NÝbI &Purity/4FRt0SRqqt1FBt/event_proportion"trainer/global_step2:JP>[ìªNÞbI &Purity/4FRt3SRqqt1FBt/event_proportion"trainer/global_step2:JPëXQÿ<ßb7 particle/sensitivity"trainer/global_step2:JPaPó“JàbE "Purity/1FRt4SRqqt*FBt/SRqqt_purity"trainer/global_step2:JPãñ½JábE "Purity/1FRt*SRqqt4FBt/SRqqt_purity"trainer/global_step2:JPMaþNâbI &Purity/*FRt*SRqqt0FBt/event_proportion"trainer/global_step2:JP(†“JãbE "Purity/0FRt*SRqqt2FBt/event_purity"trainer/global_step2:JP=ðHäbC Purity/*FRt1SRqqt3FBt/FRt_purity"trainer/global_step2:JPJLÈEHåbC Purity/4FRt1SRqqt0FBt/FRt_purity"trainer/global_step2:JP©˜¤NæbI &Purity/2FRt3SRqqt0FBt/event_proportion"trainer/global_step2:JP/墄JçbE "Purity/0FRt*SRqqt2FBt/SRqqt_purity"trainer/global_step2:JP$ GÈNèbI &Purity/*FRt3SRqqt0FBt/event_proportion"trainer/global_step2:JPæ^¤NébI &Purity/*FRt4SRqqt1FBt/event_proportion"trainer/global_step2:JPí6]FHêbC Purity/*FRt*SRqqt*FBt/FBt_purity"trainer/global_step2:JP0ù3¯HëbC Purity/*FRt*SRqqt0FBt/FRt_purity"trainer/global_step2:JP‘`Î7NìbI &Purity/*FRt1SRqqt0FBt/event_proportion"trainer/global_step2:JPùüQwHíbC Purity/3FRt0SRqqt2FBt/FRt_purity"trainer/global_step2:JP¨†_mHîbC Purity/1FRt*SRqqt3FBt/FBt_purity"trainer/global_step2:JP 1ëžHïbC Purity/1FRt0SRqqt2FBt/FBt_purity"trainer/global_step2:JP­Ks´;ðb6 jet/accuracy_2_of_2"trainer/global_step2:JPÐ fjJñbE "Purity/1FRt0SRqqt4FBt/event_purity"trainer/global_step2:JPñxJòbE "Purity/*FRt0SRqqt2FBt/event_purity"trainer/global_step2:JP{ºÓFJóbE "Purity/*FRt1SRqqt*FBt/event_purity"trainer/global_step2:JP=G¯TJôbE "Purity/*FRt3SRqqt1FBt/event_purity"trainer/global_step2:JPS®_ÄNõbI &Purity/1FRt4SRqqt3FBt/event_proportion"trainer/global_step2:JPéÞ?;öb6 jet/accuracy_4_of_6"trainer/global_step2:JP_dÏ!J÷bE "Purity/4FRt2SRqqt*FBt/event_purity"trainer/global_step2:JPŠöeÈJøbE "Purity/*FRt0SRqqt1FBt/event_purity"trainer/global_step2:JP?º™KJùbE "Purity/0FRt2SRqqt0FBt/SRqqt_purity"trainer/global_step2:JPweNúbI &Purity/0FRt2SRqqt2FBt/event_proportion"trainer/global_step2:JPÚz‰HûbC Purity/1FRt*SRqqt2FBt/FRt_purity"trainer/global_step2:JP[¥êNübI &Purity/1FRt1SRqqt2FBt/event_proportion"trainer/global_step2:JP89¾7NýbI &Purity/2FRt4SRqqt3FBt/event_proportion"trainer/global_step2:JPñjkûJþbE "Purity/0FRt3SRqqt*FBt/SRqqt_purity"trainer/global_step2:JP}ó„NÿbI &Purity/*FRt2SRqqt4FBt/event_proportion"trainer/global_step2:JP&ÑñN€bI &Purity/*FRt3SRqqt2FBt/event_proportion"trainer/global_step2:JPû=e6HbC Purity/0FRt*SRqqt2FBt/FBt_purity"trainer/global_step2:JPÆÙN‚bI &Purity/3FRt1SRqqt0FBt/event_proportion"trainer/global_step2:JPUšCÄ;ƒb6 jet/accuracy_7_of_7"trainer/global_step2:JPV>!`H„bC Purity/*FRt1SRqqt4FBt/FRt_purity"trainer/global_step2:JP&ËMN…bI &Purity/1FRt*SRqqt4FBt/event_proportion"trainer/global_step2:JPMù+N†bI &Purity/1FRt2SRqqt3FBt/event_proportion"trainer/global_step2:JP€¨N‡bI &Purity/3FRt1SRqqt1FBt/event_proportion"trainer/global_step2:JPO›ÇJˆbE "Purity/3FRt2SRqqt*FBt/event_purity"trainer/global_step2:JPõÉÂN–bI &Purity/0FRt3SRqqt4FBt/event_proportion"trainer/global_step2:JP1wq‘N—bI &Purity/3FRt2SRqqt*FBt/event_proportion"trainer/global_step2:JPGxsN˜bI &Purity/3FRt2SRqqt3FBt/event_proportion"trainer/global_step2:JP8X9ØgJÓbE "Purity/2FRt2SRqqt1FBt/event_purity"trainer/global_step2:JP¯„NÔbI &Purity/*FRt*SRqqt*FBt/event_proportion"trainer/global_step2:JPÓ7[~HÕbC Purity/*FRt*SRqqt1FBt/FRt_purity"trainer/global_step2:JPŒ¢ÍJÖbE "Purity/*FRt2SRqqt1FBt/event_purity"trainer/global_step2:JPvÂH×bC Purity/1FRt1SRqqt2FBt/FBt_purity"trainer/global_step2:JPå)SJØbE "Purity/2FRt0SRqqt2FBt/event_purity"trainer/global_step2:JPQþXÝNÙbI &Purity/3FRt0SRqqt4FBt/event_proportion"trainer/global_step2:JPÍgagJÚbE "Purity/*FRt1SRqqt4FBt/event_purity"trainer/global_step2:JPgY'øHÛbC Purity/*FRt3SRqqt2FBt/FRt_purity"trainer/global_step2:JPZ6“TNÜbI &Purity/*FRt2SRqqt3FBt/event_proportion"trainer/global_step2:JP;”.æJÝbE "Purity/2FRt1SRqqt2FBt/SRqqt_purity"trainer/global_step2:JPUo FNÞbI &Purity/3FRt*SRqqt0FBt/event_proportion"trainer/global_step2:JP3­ü%HßbC Purity/3FRt*SRqqt2FBt/FBt_purity"trainer/global_step2:JPÜ›JàbE "Purity/0FRt1SRqqt3FBt/event_purity"trainer/global_step2:JPÇO¨ŒJábE "Purity/1FRt1SRqqt4FBt/event_purity"trainer/global_step2:JPø`"3JâbE "Purity/*FRt1SRqqt0FBt/SRqqt_purity"trainer/global_step2:JPñâsJãbE "Purity/2FRt*SRqqt1FBt/SRqqt_purity"trainer/global_step2:JP¢ö8JäbE "Purity/2FRt1SRqqt2FBt/event_purity"trainer/global_step2:JPøø‘åHåbC Purity/3FRt1SRqqt1FBt/FRt_purity"trainer/global_step2:JP²—MJæbE "Purity/3FRt2SRqqt0FBt/SRqqt_purity"trainer/global_step2:JP>ø|iJçbE "Purity/0FRt2SRqqt3FBt/SRqqt_purity"trainer/global_step2:JPG\*úHèbC Purity/*FRt0SRqqt3FBt/FRt_purity"trainer/global_step2:JP™èŸwJébE "Purity/0FRt*SRqqt3FBt/event_purity"trainer/global_step2:JP0$ JêbE "Purity/3FRt2SRqqt1FBt/event_purity"trainer/global_step2:JP'¼¡NëbI &Purity/*FRt*SRqqt4FBt/event_proportion"trainer/global_step2:JPXÔJìbE "Purity/0FRt2SRqqt1FBt/event_purity"trainer/global_step2:JP³`œNíbI &Purity/0FRt4SRqqt2FBt/event_proportion"trainer/global_step2:JPŠV:•JîbE "Purity/1FRt0SRqqt0FBt/event_purity"trainer/global_step2:JPŽ¥…JïbE "Purity/1FRt1SRqqt1FBt/SRqqt_purity"trainer/global_step2:JP£0G"JðbE "Purity/1FRt2SRqqt0FBt/SRqqt_purity"trainer/global_step2:JPñáÓNñbI &Purity/2FRt*SRqqt1FBt/event_proportion"trainer/global_step2:JPqâeNòbI &Purity/2FRt4SRqqt4FBt/event_proportion"trainer/global_step2:JPbeƒHóbC Purity/*FRt2SRqqt*FBt/FRt_purity"trainer/global_step2:JPîÙhHôbC Purity/1FRt2SRqqt*FBt/FBt_purity"trainer/global_step2:JPÒ!·HõbC Purity/1FRt2SRqqt0FBt/FRt_purity"trainer/global_step2:JPAkiMNöbI &Purity/2FRt0SRqqt3FBt/event_proportion"trainer/global_step2:JPYhþˆN÷bI &Purity/2FRt0SRqqt4FBt/event_proportion"trainer/global_step2:JPE¡}HøbC Purity/2FRt3SRqqt*FBt/FRt_purity"trainer/global_step2:JP±ò·”HùbC Purity/3FRt*SRqqt1FBt/FBt_purity"trainer/global_step2:JP»4HúbC Purity/3FRt1SRqqt1FBt/FBt_purity"trainer/global_step2:JPÍRXNûbI &Purity/2FRt1SRqqt*FBt/event_proportion"trainer/global_step2:JP©«0NübI &Purity/2FRt4SRqqt2FBt/event_proportion"trainer/global_step2:JPú!x#JýbE "Purity/*FRt4SRqqt0FBt/event_purity"trainer/global_step2:JP6æüš;þb6 jet/accuracy_1_of_6"trainer/global_step2:JPÁ¯«i;ÿb6 jet/accuracy_3_of_7"trainer/global_step2:JPäÈüà@€b; particle/accuracy_1_of_7"trainer/global_step2:JPF׫NbI &Purity/*FRt0SRqqt1FBt/event_proportion"trainer/global_step2:JP«YŒóN‚bI &Purity/0FRt3SRqqt*FBt/event_proportion"trainer/global_step2:JPªñ»JƒbE "Purity/1FRt2SRqqt*FBt/event_purity"trainer/global_step2:JPêÎP³J„bE "Purity/2FRt1SRqqt*FBt/event_purity"trainer/global_step2:JP©Aú=J…bE "Purity/2FRt1SRqqt0FBt/SRqqt_purity"trainer/global_step2:JP¿=q}H†bC Purity/3FRt*SRqqt1FBt/FRt_purity"trainer/global_step2:JPw ‡ÄN‡bI &Purity/3FRt4SRqqt0FBt/event_proportion"trainer/global_step2:JP\ØÜ;ˆb6 jet/accuracy_6_of_6"trainer/global_step2:JP~ùû³J‰bE "Purity/1FRt0SRqqt2FBt/event_purity"trainer/global_step2:JPÑÖJŠbE "Purity/1FRt2SRqqt1FBt/event_purity"trainer/global_step2:JP¢ªÄmN‹bI &Purity/2FRt*SRqqt4FBt/event_proportion"trainer/global_step2:JP3¤yNŒbI &Purity/3FRt0SRqqt0FBt/event_proportion"trainer/global_step2:JPÃÀLïHbC Purity/3FRt1SRqqt2FBt/FBt_purity"trainer/global_step2:JPüri6JŽbE "Purity/1FRt*SRqqt3FBt/SRqqt_purity"trainer/global_step2:JP€ÊJbE "Purity/2FRt*SRqqt*FBt/SRqqt_purity"trainer/global_step2:JP­Y+ NbI &Purity/4FRt1SRqqt2FBt/event_proportion"trainer/global_step2:JPfŠp…;‘b6 jet/accuracy_1_of_5"trainer/global_step2:JP?åkó@’b; particle/accuracy_3_of_6"trainer/global_step2:JP¡Q^fJ“bE "Purity/3FRt0SRqqt2FBt/event_purity"trainer/global_step2:JP¢Û°ÓJ”bE "Purity/4FRt*SRqqt0FBt/event_purity"trainer/global_step2:JP¸AcUH•bC Purity/*FRt*SRqqt2FBt/FBt_purity"trainer/global_step2:JP™Þ‘3H–bC Purity/1FRt1SRqqt1FBt/FRt_purity"trainer/global_step2:JPw´r©N—bI &Purity/1FRt4SRqqt0FBt/event_proportion"trainer/global_step2:JPÕ"_¢J˜bE "Purity/3FRt0SRqqt*FBt/event_purity"trainer/global_step2:JP mHJ™bE "Purity/3FRt1SRqqt*FBt/event_purity"trainer/global_step2:JPôUæNšbI &Purity/3FRt1SRqqt*FBt/event_proportion"trainer/global_step2:JP‰ç·ÊH›bC Purity/*FRt4SRqqt*FBt/FBt_purity"trainer/global_step2:JPçäÞ’HœbC Purity/*FRt4SRqqt0FBt/FRt_purity"trainer/global_step2:JPÎñ›[JbE "Purity/*FRt*SRqqt1FBt/event_purity"trainer/global_step2:JP.Ã:HžbC Purity/1FRt*SRqqt3FBt/FRt_purity"trainer/global_step2:JPåHŸbC Purity/1FRt1SRqqt1FBt/FBt_purity"trainer/global_step2:JP1~J bE "Purity/3FRt1SRqqt1FBt/SRqqt_purity"trainer/global_step2:JPZ|ˆï@¡b; particle/accuracy_1_of_5"trainer/global_step2:JPÛëÿ@¢b; particle/accuracy_5_of_5"trainer/global_step2:JP¸«9J£bE "Purity/3FRt4SRqqt0FBt/SRqqt_purity"trainer/global_step2:JP©…Ó›N¤bI &Purity/0FRt4SRqqt4FBt/event_proportion"trainer/global_step2:JP¨`˜HûbC Purity/1FRt3SRqqt1FBt/FRt_purity"trainer/global_step2:JPñÞ§.HübC Purity/2FRt0SRqqt1FBt/FRt_purity"trainer/global_step2:JP€†aHýbC Purity/*FRt1SRqqt4FBt/FBt_purity"trainer/global_step2:JP …Ý:HþbC Purity/4FRt2SRqqt*FBt/FRt_purity"trainer/global_step2:JPe!a×HÿbC Purity/*FRt0SRqqt1FBt/FRt_purity"trainer/global_step2:JP¹³ÏN€ bI &Purity/0FRt*SRqqt2FBt/event_proportion"trainer/global_step2:JPP2 ìH bC Purity/0FRt0SRqqt3FBt/FBt_purity"trainer/global_step2:JP1_fòN‚ bI &Purity/4FRt2SRqqt0FBt/event_proportion"trainer/global_step2:JPèÙ\ô;ƒ b6 jet/accuracy_4_of_4"trainer/global_step2:JPRðÅmN„ bI &Purity/*FRt3SRqqt*FBt/event_proportion"trainer/global_step2:JP’T¶ÚJ… bE "Purity/0FRt1SRqqt*FBt/SRqqt_purity"trainer/global_step2:JP¢Ð¹H† bC Purity/1FRt1SRqqt*FBt/FRt_purity"trainer/global_step2:JP°9#J‡ bE "Purity/1FRt1SRqqt0FBt/SRqqt_purity"trainer/global_step2:JP•~þHˆ bC Purity/2FRt*SRqqt2FBt/FRt_purity"trainer/global_step2:JPö( N‰ bI &Purity/2FRt0SRqqt2FBt/event_proportion"trainer/global_step2:JPìâgJŠ bE "Purity/2FRt1SRqqt1FBt/event_purity"trainer/global_step2:JPâàð-J‹ bE "Purity/*FRt*SRqqt4FBt/event_purity"trainer/global_step2:JP¢øÝJŒ bE "Purity/*FRt3SRqqt*FBt/event_purity"trainer/global_step2:JPôeø'H bC Purity/0FRt*SRqqt*FBt/FBt_purity"trainer/global_step2:JPùUÿJŽ bE "Purity/0FRt*SRqqt1FBt/event_purity"trainer/global_step2:JP ®-"N bI &Purity/1FRt4SRqqt*FBt/event_proportion"trainer/global_step2:JPz;ŠN bI &Purity/1FRt4SRqqt1FBt/event_proportion"trainer/global_step2:JP "çN‘ bI &Purity/2FRt0SRqqt1FBt/event_proportion"trainer/global_step2:JPiÞN’ bI &Purity/4FRt0SRqqt2FBt/event_proportion"trainer/global_step2:JPßßéýH“ bC Purity/0FRt0SRqqt4FBt/FBt_purity"trainer/global_step2:JP_â“ÐH” bC Purity/*FRt1SRqqt2FBt/FRt_purity"trainer/global_step2:JPÔ×ÐH• bC Purity/1FRt0SRqqt*FBt/FBt_purity"trainer/global_step2:JP˜£ãVH– bC Purity/1FRt0SRqqt0FBt/FRt_purity"trainer/global_step2:JP'Ç úN— bI &Purity/4FRt4SRqqt2FBt/event_proportion"trainer/global_step2:JP®þ@‡@˜ b; particle/accuracy_6_of_7"trainer/global_step2:JPÅ7ÄH™ bC Purity/2FRt*SRqqt3FBt/FBt_purity"trainer/global_step2:JP uŒJš bE "Purity/*FRt*SRqqt3FBt/event_purity"trainer/global_step2:JP!üg4N› bI &Purity/2FRt1SRqqt4FBt/event_proportion"trainer/global_step2:JP6…Nœ bI &Purity/4FRt1SRqqt1FBt/event_proportion"trainer/global_step2:JP(XfH bC Purity/1FRt*SRqqt4FBt/FRt_purity"trainer/global_step2:JPy4möHž bC Purity/*FRt1SRqqt3FBt/FBt_purity"trainer/global_step2:JP‰pÁ|HŸ bC Purity/3FRt3SRqqt0FBt/FRt_purity"trainer/global_step2:JPÒH  bC Purity/0FRt0SRqqt*FBt/FBt_purity"trainer/global_step2:JPn9N¡ bI &Purity/1FRt2SRqqt4FBt/event_proportion"trainer/global_step2:JPà N¢ bI &Purity/2FRt*SRqqt*FBt/event_proportion"trainer/global_step2:JP<,¼aH£ bC Purity/0FRt*SRqqt4FBt/FBt_purity"trainer/global_step2:JP¾[ÆH¤ bC Purity/*FRt2SRqqt1FBt/FRt_purity"trainer/global_step2:JPæšÿN¥ bI &Purity/2FRt3SRqqt1FBt/event_proportion"trainer/global_step2:JP –ÄžN¦ bI &Purity/3FRt0SRqqt3FBt/event_proportion"trainer/global_step2:JP¼°Ti;§ b6 jet/accuracy_2_of_3"trainer/global_step2:JPÂ÷àèH¨ bC Purity/2FRt3SRqqt1FBt/FRt_purity"trainer/global_step2:JPpŠ•'J© bE "Purity/3FRt3SRqqt*FBt/SRqqt_purity"trainer/global_step2:JP7·WNª bI &Purity/2FRt2SRqqt3FBt/event_proportion"trainer/global_step2:JPž¢æN« bI &Purity/3FRt*SRqqt4FBt/event_proportion"trainer/global_step2:JPO9máN¬ bI &Purity/4FRt3SRqqt2FBt/event_proportion"trainer/global_step2:JP=²²H­ bC Purity/*FRt2SRqqt1FBt/FBt_purity"trainer/global_step2:JPz ßHN® bI &Purity/*FRt1SRqqt*FBt/event_proportion"trainer/global_step2:JPÜ|4èJ¯ bE "Purity/2FRt0SRqqt*FBt/event_purity"trainer/global_step2:JP¿+©H° bC Purity/3FRt*SRqqt*FBt/FRt_purity"trainer/global_step2:JPòu²øJ± bE "Purity/3FRt*SRqqt1FBt/SRqqt_purity"trainer/global_step2:JPš†ûN² bI &Purity/4FRt4SRqqt1FBt/event_proportion"trainer/global_step2:JPÚÃaçH³ bC Purity/2FRt3SRqqt1FBt/FBt_purity"trainer/global_step2:JPBdŽšH´ bC Purity/1FRt0SRqqt4FBt/FBt_purity"trainer/global_step2:JP«Øà"Jµ bE "Purity/1FRt*SRqqt*FBt/event_purity"trainer/global_step2:JPË7¤H¶ bC Purity/3FRt1SRqqt*FBt/FRt_purity"trainer/global_step2:JPù¦[Ø9· b4 particle/accuracy"trainer/global_step2:JPŠÌ1J¸ bE "Purity/2FRt4SRqqt0FBt/SRqqt_purity"trainer/global_step2:JP”Áö;¹ b6 jet/accuracy_2_of_6"trainer/global_step2:JPæÃÞáJº bE "Purity/0FRt3SRqqt1FBt/event_purity"trainer/global_step2:JP [ûJ» bE "Purity/4FRt*SRqqt*FBt/event_purity"trainer/global_step2:JP*yÍJ¼ bE "Purity/0FRt0SRqqt2FBt/event_purity"trainer/global_step2:JPÁÂiJ½ bE "Purity/1FRt1SRqqt3FBt/event_purity"trainer/global_step2:JPo>N¾ bI &Purity/2FRt0SRqqt0FBt/event_proportion"trainer/global_step2:JPµ}^N¿ bI &Purity/2FRt4SRqqt0FBt/event_proportion"trainer/global_step2:JP\6±JÀ bE "Purity/*FRt4SRqqt*FBt/SRqqt_purity"trainer/global_step2:JPU㩇HÁ bC Purity/2FRt4SRqqt*FBt/FRt_purity"trainer/global_step2:JPZ¤ŠJ bE "Purity/0FRt2SRqqt*FBt/event_purity"trainer/global_step2:JP™¥èNà bI &Purity/1FRt3SRqqt1FBt/event_proportion"trainer/global_step2:JP‡ÈË6@Ä b; particle/accuracy_3_of_5"trainer/global_step2:JP£3¸XHÅ bC Purity/*FRt2SRqqt2FBt/FBt_purity"trainer/global_step2:JPZpHÆ bC Purity/*FRt0SRqqt4FBt/FRt_purity"trainer/global_step2:JPcJÇ bE "Purity/*FRt0SRqqt*FBt/event_purity"trainer/global_step2:JPf­åJJÈ bE "Purity/1FRt3SRqqt0FBt/SRqqt_purity"trainer/global_step2:JP-\®@É b; particle/accuracy_1_of_4"trainer/global_step2:JP_zÓ@Ê b; particle/accuracy_4_of_7"trainer/global_step2:JP‘üCÃJË bE "Purity/2FRt2SRqqt2FBt/SRqqt_purity"trainer/global_step2:JPLmå÷NÌ bI &Purity/0FRt3SRqqt0FBt/event_proportion"trainer/global_step2:JPDÛJÍ bE "Purity/1FRt0SRqqt3FBt/event_purity"trainer/global_step2:JPµJ,1@Î b; particle/accuracy_2_of_5"trainer/global_step2:JP–gÔJÏ bE "Purity/2FRt2SRqqt2FBt/event_purity"trainer/global_step2:JPI3ìÔ Ð bC Pur¡á>ity/*FRt*SRqqt*FBt/FRt_purity"trainer/global_step2:JPk‡žNÑ bI &Purity/1FRt0SRqqt2FBt/event_proportion"trainer/global_step2:JPðº÷ºNÒ bI &Purity/*FRt1SRqqt3FBt/event_proportion"trainer/global_step2:JP¤<÷âJÓ bE "Purity/3FRt*SRqqt*FBt/event_purity"trainer/global_step2:JPв²/HÔ bC Purity/4FRt1SRqqt*FBt/FRt_purity"trainer/global_step2:JPé+Š1JÕ bE "Purity/4FRt2SRqqt0FBt/SRqqt_purity"trainer/global_step2:JP©X§NÖ bI &Purity/0FRt4SRqqt0FBt/event_proportion"trainer/global_step2:JPfÙPJ× bE "Purity/1FRt0SRqqt*FBt/event_purity"trainer/global_step2:JPçk¾ºHØ bC Purity/2FRt0SRqqt2FBt/FBt_purity"trainer/global_step2:JPž=§NÙ bI &Purity/2FRt1SRqqt2FBt/event_proportion"trainer/global_step2:JPê²âVJÚ bE "Purity/3FRt2SRqqt*FBt/SRqqt_purity"trainer/global_step2:JP™ƒ&JÛ bE "Purity/2FRt4SRqqt*FBt/SRqqt_purity"trainer/global_step2:JPªö¹HÜ bC Purity/1FRt3SRqqt2FBt/FBt_purity"trainer/global_step2:JP¼E#JÝ bE "Purity/*FRt*SRqqt*FBt/SRqqt_purity"trainer/global_step2:JP¹=DGHÞ bC Purity/*FRt0SRqqt2FBt/FBt_purity"trainer/global_step2:JP$SrVNß bI &Purity/0FRt0SRqqt*FBt/event_proportion"trainer/global_step2:JPÂRyðNà bI &Purity/0FRt0SRqqt1FBt/event_proportion"trainer/global_step2:JPñæí9Ná bI &Purity/1FRt3SRqqt3FBt/event_proportion"trainer/global_step2:JP^Ú;Hâ bC Purity/3FRt2SRqqt*FBt/FRt_purity"trainer/global_step2:JPí_! Nã bI &Purity/3FRt3SRqqt4FBt/event_proportion"trainer/global_step2:JPæ>½¥Nä bI &Purity/4FRt*SRqqt1FBt/event_proportion"trainer/global_step2:JP»{Jå bE "Purity/0FRt4SRqqt*FBt/event_purity"trainer/global_step2:JP€H>ÇJæ bE "Purity/*FRt3SRqqt1FBt/SRqqt_purity"trainer/global_step2:JPtû•Nç bI &Purity/0FRt2SRqqt3FBt/event_proportion"trainer/global_step2:JPv TNè bI &Purity/4FRt0SRqqt0FBt/event_proportion"trainer/global_step2:JP¡ÞG@é b; particle/accuracy_1_of_3"trainer/global_step2:JP˜š¾­Hê bC Purity/1FRt1SRqqt3FBt/FRt_purity"trainer/global_step2:JPÄ“ª¢Jë bE "Purity/1FRt3SRqqt2FBt/SRqqt_purity"trainer/global_step2:JPlû˜Jì bE "Purity/*FRt1SRqqt*FBt/SRqqt_purity"trainer/global_step2:JPL»Hí bC Purity/1FRt*SRqqt1FBt/FBt_purity"trainer/global_step2:JP\Wš;î b6 jet/accuracy_5_of_6"trainer/global_step2:JPÃaŽµJï bE "Purity/1FRt3SRqqt2FBt/event_purity"trainer/global_step2:JPÿ¿}PJð bE "Purity/4FRt2SRqqt*FBt/SRqqt_purity"trainer/global_step2:JPæ(PˆJñ bE "Purity/*FRt2SRqqt0FBt/SRqqt_purity"trainer/global_step2:JPÿÃt»Jò bE "Purity/0FRt0SRqqt0FBt/event_purity"trainer/global_step2:JPÌ&Purity/1FRt2SRqqt0FBt/event_proportion‚0.04731602966785431 ("Purity/2FRt2SRqqt1FBt/event_purity‚0 =&Purity/*FRt*SRqqt*FBt/event_proportion‚0.9591733813285828 7 Purity/*FRt*SRqqt1FBt/FRt_purity‚0.2773731052875519 ;"Purity/*FRt2SRqqt1FBt/event_purity‚0.006182795390486717 7 Purity/1FRt1SRqqt2FBt/FBt_purity‚0.9726190567016602 :"Purity/2FRt0SRqqt2FBt/event_purity‚0.01785714365541935 ,&Purity/3FRt0SRqqt4FBt/event_proportion‚0 ("Purity/*FRt1SRqqt4FBt/event_purity‚0 & Purity/*FRt3SRqqt2FBt/FRt_purity‚1 @&Purity/*FRt2SRqqt3FBt/event_proportion‚6.300403038039804e-05 9"Purity/2FRt1SRqqt2FBt/SRqqt_purity‚0.6710526347160339 @&Purity/3FRt*SRqqt0FBt/event_proportion‚0.0071824598126113415 & Purity/3FRt*SRqqt2FBt/FBt_purity‚1 ("Purity/0FRt1SRqqt3FBt/event_purity‚0 ("Purity/1FRt1SRqqt4FBt/event_purity‚0 9"Purity/*FRt1SRqqt0FBt/SRqqt_purity‚0.3397771418094635 9"Purity/2FRt*SRqqt1FBt/SRqqt_purity‚0.3888493478298187 ("Purity/2FRt1SRqqt2FBt/event_purity‚0 7 Purity/3FRt1SRqqt1FBt/FRt_purity‚0.2820512652397156 :"Purity/3FRt2SRqqt0FBt/SRqqt_purity‚0.38768112659454346 *"Purity/0FRt2SRqqt3FBt/SRqqt_purity‚0.5 ( Purity/*FRt0SRqqt3FBt/FRt_purity‚0.5 :"Purity/0FRt*SRqqt3FBt/event_purity‚0.18209877610206604 ("Purity/3FRt2SRqqt1FBt/event_purity‚0 @&Purity/*FRt*SRqqt4FBt/event_proportion‚0.0003780241822823882 ;"Purity/0FRt2SRqqt1FBt/event_purity‚0.011648745276033878 ,&Purity/0FRt4SRqqt2FBt/event_proportion‚0 9"Purity/1FRt0SRqqt0FBt/event_purity‚0.1496870219707489 9"Purity/1FRt1SRqqt1FBt/SRqqt_purity‚0.4611673355102539 :"Purity/1FRt2SRqqt0FBt/SRqqt_purity‚0.34311607480049133 ?&Purity/2FRt*SRqqt1FBt/event_proportion‚0.021169355139136314 ,&Purity/2FRt4SRqqt4FBt/event_proportion‚0 8 Purity/*FRt2SRqqt*FBt/FRt_purity‚0.20096346735954285 6 Purity/1FRt2SRqqt*FBt/FBt_purity‚0.981566846370697 7 Purity/1FRt2SRqqt0FBt/FRt_purity‚0.1898135393857956 @&Purity/2FRt0SRqqt3FBt/event_proportion‚6.300403038039804e-05 ,&Purity/2FRt0SRqqt4FBt/event_proportion‚0 7 Purity/2FRt3SRqqt*FBt/FRt_purity‚0.1780952513217926 & Purity/3FRt*SRqqt1FBt/FBt_purity‚1 & Purity/3FRt1SRqqt1FBt/FBt_purity‚1 >&Purity/2FRt1SRqqt*FBt/event_proportion‚0.04158266261219978 ,&Purity/2FRt4SRqqt2FBt/event_proportion‚0 ("Purity/*FRt4SRqqt0FBt/event_purity‚0 jet/accuracy_1_of_6‚1 jet/accuracy_3_of_7‚0 particle/accuracy_1_of_7‚1 >&Purity/*FRt0SRqqt1FBt/event_proportion‚0.14591734111309052 ?&Purity/0FRt3SRqqt*FBt/event_proportion‚0.006741431541740894 ("Purity/1FRt2SRqqt*FBt/event_purity‚0 ("Purity/2FRt1SRqqt*FBt/event_purity‚0 9"Purity/2FRt1SRqqt0FBt/SRqqt_purity‚0.3509160280227661 7 Purity/3FRt*SRqqt1FBt/FRt_purity‚0.2904040813446045 A&Purity/3FRt4SRqqt0FBt/event_proportion‚0.00012600806076079607 jet/accuracy_6_of_6‚0 :"Purity/1FRt0SRqqt2FBt/event_purity‚0.07879630476236343 ("Purity/1FRt2SRqqt1FBt/event_purity‚0 ,&Purity/2FRt*SRqqt4FBt/event_proportion‚0 @&Purity/3FRt0SRqqt0FBt/event_proportion‚0.0015120967291295528 & Purity/3FRt1SRqqt2FBt/FBt_purity‚1 ("Purity/1FRt*SRqqt3FBt/SRqqt_purity‚1 9"Purity/2FRt*SRqqt*FBt/SRqqt_purity‚0.3667474091053009 ,&Purity/4FRt1SRqqt2FBt/event_proportion‚0 *jet/accuracy_1_of_5‚0.9197626709938049 particle/accuracy_3_of_6‚1 ("Purity/3FRt0SRqqt2FBt/event_purity‚0 ("Purity/4FRt*SRqqt0FBt/event_purity‚0 7 Purity/*FRt*SRqqt2FBt/FBt_purity‚0.9780328869819641 7 Purity/1FRt1SRqqt1FBt/FRt_purity‚0.2410500943660736 @&Purity/1FRt4SRqqt0FBt/event_proportion‚0.0003780241822823882 ("Purity/3FRt0SRqqt*FBt/event_purity‚0 ("Purity/3FRt1SRqqt*FBt/event_purity‚0 ?&Purity/3FRt1SRqqt*FBt/event_proportion‚0.003717237850651145 & Purity/*FRt4SRqqt*FBt/FBt_purity‚1 6 Purity/*FRt4SRqqt0FBt/FRt_purity‚0.236111119389534 :"Purity/*FRt*SRqqt1FBt/event_purity‚0.38116568326950073 + Purity/1FRt*SRqqt3FBt/FRt_purity‚0.4375 7 Purity/1FRt1SRqqt1FBt/FBt_purity‚0.9949740171432495 *"Purity/3FRt1SRqqt1FBt/SRqqt_purity‚0.5 particle/accuracy_1_of_5‚1 particle/accuracy_5_of_5‚1 +"Purity/3FRt4SRqqt0FBt/SRqqt_purity‚0.25 ,&Purity/0FRt4SRqqt4FBt/event_proportion‚0 ,&Purity/3FRt*SRqqt3FBt/event_proportion‚0 particle/accuracy_4_of_4‚1 jet/accuracy_2_of_7‚0 :"Purity/*FRt1SRqqt1FBt/event_purity‚0.07138144224882126 @&Purity/2FRt*SRqqt3FBt/event_proportion‚6.300403038039804e-05 ?&Purity/2FRt0SRqqt*FBt/event_proportion‚0.037046369165182114 ,&Purity/4FRt4SRqqt4FBt/event_proportion‚0 *"Purity/0FRt*SRqqt4FBt/event_purity‚0.5 7 Purity/1FRt2SRqqt2FBt/FRt_purity‚0.3888889253139496 7 Purity/1FRt4SRqqt*FBt/FRt_purity‚0.1666666716337204 ( Purity/2FRt0SRqqt3FBt/FRt_purity‚0.5 9"Purity/*FRt*SRqqt0FBt/SRqqt_purity‚0.3411872684955597 8 Purity/*FRt0SRqqt*FBt/FRt_purity‚0.21240629255771637 9"Purity/1FRt3SRqqt*FBt/SRqqt_purity‚0.3328213095664978 <"Purity/2FRt*SRqqt0FBt/event_purity‚0.0027786199934780598 ?&Purity/2FRt1SRqqt1FBt/event_proportion‚0.007875503972172737 @&Purity/3FRt0SRqqt*FBt/event_proportion‚0.0027091733645647764 ("Purity/3FRt4SRqqt*FBt/event_purity‚0 7 Purity/0FRt2SRqqt1FBt/FBt_purity‚0.9792627096176147 :"Purity/1FRt*SRqqt0FBt/SRqqt_purity‚0.34288355708122253 8 Purity/1FRt3SRqqt*FBt/FRt_purity‚0.20784690976142883 8 Purity/2FRt0SRqqt*FBt/FRt_purity‚0.20727728307247162 ("Purity/3FRt1SRqqt0FBt/event_purity‚0 @&Purity/4FRt*SRqqt*FBt/event_proportion‚0.0001890120911411941 particle/accuracy_1_of_1‚1 *jet/accuracy_3_of_4‚0.0471128448843956 particle/accuracy_2_of_7‚1 & Purity/3FRt0SRqqt2FBt/FBt_purity‚1 >&Purity/*FRt2SRqqt0FBt/event_proportion‚0.10855595022439957 7 Purity/0FRt1SRqqt2FBt/FBt_purity‚0.9621295928955078 ("Purity/1FRt*SRqqt3FBt/event_purity‚0 7 Purity/2FRt1SRqqt*FBt/FRt_purity‚0.2224024534225464 9"Purity/2FRt2SRqqt*FBt/SRqqt_purity‚0.3756571114063263 7 Purity/2FRt2SRqqt1FBt/FBt_purity‚0.9821428656578064 ("Purity/3FRt0SRqqt1FBt/event_purity‚0 A&Purity/3FRt4SRqqt*FBt/event_proportion‚0.00012600806076079607 =&Purity/0FRt1SRqqt*FBt/event_proportion‚0.2305317521095276 ?&Purity/2FRt3SRqqt*FBt/event_proportion‚0.004032257944345474 *jet/accuracy_2_of_4‚0.2931746244430542 & Purity/0FRt4SRqqt*FBt/FBt_purity‚1 jet/accuracy_6_of_7‚0 & Purity/4FRt*SRqqt0FBt/FRt_purity‚0 ;"Purity/1FRt1SRqqt*FBt/event_purity‚0.010837228037416935 7 Purity/2FRt2SRqqt1FBt/FRt_purity‚0.2395833283662796 :"Purity/0FRt1SRqqt1FBt/SRqqt_purity‚0.42248860001564026 ("Purity/*FRt3SRqqt0FBt/event_purity‚0 ,jet/accuracy_3_of_3‚0.014111964032053947 9"Purity/0FRt4SRqqt0FBt/SRqqt_purity‚0.1428571492433548 ( Purity/*FRt*SRqqt4FBt/FRt_purity‚0.5 @&Purity/1FRt0SRqqt4FBt/event_proportion‚6.300403038039804e-05 8 Purity/2FRt1SRqqt1FBt/FRt_purity‚0.27142855525016785 ("Purity/4FRt0SRqqt*FBt/event_purity‚0 @&Purity/*FRt0SRqqt4FBt/event_proportion‚0.0003150201519019902 9"Purity/0FRt0SRqqt3FBt/event_purity‚0.2433333396911621 7 Purity/1FRt*SRqqt2FBt/FBt_purity‚0.9775258302688599 7 Purity/1FRt0SRqqt3FBt/FBt_purity‚0.9285714030265808 7 Purity/2FRt*SRqqt*FBt/FBt_purity‚0.9907397627830505 8 Purity/2FRt*SRqqt0FBt/FRt_purity‚0.18108418583869934 8 Purity/*FRt*SRqqt3FBt/FRt_purity‚0.44117647409439087 =&Purity/*FRt1SRqqt1FBt/event_proportion‚0.1001134067773819 7 Purity/0FRt*SRqqt3FBt/FBt_purity‚0.8971194624900818 & Purity/2FRt1SRqqt1FBt/FBt_purity‚1 6 Purity/2FRt2SRqqt*FBt/FRt_purity‚0.205313041806221 ("Purity/2FRt*SRqqt3FBt/event_purity‚0 ("Purity/*FRt*SRqqt4FBt/SRqqt_purity‚1 7 Purity/*FRt1SRqqt*FBt/FBt_purity‚0.9864656925201416 8 Purity/*FRt1SRqqt0FBt/FRt_purity‚0.21613827347755432 9"Purity/*FRt2SRqqt*FBt/SRqqt_purity‚0.3504430651664734 :"Purity/0FRt*SRqqt0FBt/SRqqt_purity‚0.33446353673934937 >&Purity/1FRt*SRqqt1FBt/event_proportion‚0.09929435700178146 7 Purity/1FRt0SRqqt2FBt/FRt_purity‚0.4749508798122406 :"Purity/2FRt3SRqqt1FBt/SRqqt_purity‚0.30000001192092896 9"Purity/0FRt*SRqqt*FBt/event_purity‚0.3947468101978302 & Purity/1FRt3SRqqt1FBt/FBt_purity‚1 ;"Purity/2FRt0SRqqt0FBt/event_purity‚0.009447004646062851 & Purity/2FRt0SRqqt1FBt/FBt_purity‚1 @&Purity/2FRt2SRqqt2FBt/event_proportion‚6.300403038039804e-05 ("Purity/0FRt3SRqqt0FBt/event_purity‚0 ("Purity/0FRt4SRqqt1FBt/event_purity‚0 ("Purity/4FRt1SRqqt0FBt/SRqqt_purity‚1 7 Purity/*FRt0SRqqt1FBt/FBt_purity‚0.9983914494514465 ?&Purity/*FRt1SRqqt2FBt/event_proportion‚0.017011089250445366 ,&Purity/*FRt4SRqqt4FBt/event_proportion‚0 >&Purity/1FRt2SRqqt*FBt/event_proportion‚0.05727066472172737 7 Purity/3FRt4SRqqt0FBt/FRt_purity‚0.1666666716337204 8"Purity/*FRt*SRqqt3FBt/SRqqt_purity‚0.714285671710968 7 Purity/1FRt3SRqqt1FBt/FRt_purity‚0.2888889014720917 8 Purity/2FRt0SRqqt1FBt/FRt_purity‚0.29182541370391846 ) Purity/*FRt1SRqqt4FBt/FBt_purity‚0.75 & Purity/4FRt2SRqqt*FBt/FRt_purity‚0 8 Purity/*FRt0SRqqt1FBt/FRt_purity‚0.30453720688819885 >&Purity/0FRt*SRqqt2FBt/event_proportion‚0.03944052383303642 7 Purity/0FRt0SRqqt3FBt/FBt_purity‚0.9066668152809143 @&Purity/4FRt2SRqqt0FBt/event_proportion‚6.300403038039804e-05 ,jet/accuracy_4_of_4‚0.004535453859716654 ?&Purity/*FRt3SRqqt*FBt/event_proportion‚0.020476309582591057 9"Purity/0FRt1SRqqt*FBt/SRqqt_purity‚0.3565128445625305 7 Purity/1FRt1SRqqt*FBt/FRt_purity‚0.2348003387451172 :"Purity/1FRt1SRqqt0FBt/SRqqt_purity‚0.34526321291923523 7 Purity/2FRt*SRqqt2FBt/FRt_purity‚0.4102778136730194 ?&Purity/2FRt0SRqqt2FBt/event_proportion‚0.003465221729129553 ("Purity/2FRt1SRqqt1FBt/event_purity‚0 9"Purity/*FRt*SRqqt4FBt/event_purity‚0.3333333432674408 ("Purity/*FRt3SRqqt*FBt/event_purity‚0 7 Purity/0FRt*SRqqt*FBt/FBt_purity‚0.9850422739982605 9"Purity/0FRt*SRqqt1FBt/event_purity‚0.6256648898124695 @&Purity/1FRt4SRqqt*FBt/event_proportion‚0.0003780241822823882 ,&Purity/1FRt4SRqqt1FBt/event_proportion‚0 ?&Purity/2FRt0SRqqt1FBt/event_proportion‚0.008946572430431843 ,&Purity/4FRt0SRqqt2FBt/event_proportion‚0 * Purity/0FRt0SRqqt4FBt/FBt_purity‚0.875 8 Purity/*FRt1SRqqt2FBt/FRt_purity‚0.35599538683891296 7 Purity/1FRt0SRqqt*FBt/FBt_purity‚0.9877117872238159 7 Purity/1FRt0SRqqt0FBt/FRt_purity‚0.1496870219707489 ,&Purity/4FRt4SRqqt2FBt/event_proportion‚0 particle/accuracy_6_of_7‚1 & Purity/2FRt*SRqqt3FBt/FBt_purity‚1 :"Purity/*FRt*SRqqt3FBt/event_purity‚0.10632183402776718 ,&Purity/2FRt1SRqqt4FBt/event_proportion‚0 ,&Purity/4FRt1SRqqt1FBt/event_proportion‚0 ( Purity/1FRt*SRqqt4FBt/FRt_purity‚0.5 5 Purity/*FRt1SRqqt3FBt/FBt_purity‚0.89682537317276 7 Purity/3FRt3SRqqt0FBt/FRt_purity‚0.2142857164144516 7 Purity/0FRt0SRqqt*FBt/FBt_purity‚0.9858734607696533 ,&Purity/1FRt2SRqqt4FBt/event_proportion‚0 >&Purity/2FRt*SRqqt*FBt/event_proportion‚0.10244455933570862 * Purity/0FRt*SRqqt4FBt/FBt_purity‚0.875 8 Purity/*FRt2SRqqt1FBt/FRt_purity‚0.25665760040283203 @&Purity/2FRt3SRqqt1FBt/event_proportion‚0.0003780241822823882 ,&Purity/3FRt0SRqqt3FBt/event_proportion‚0 +jet/accuracy_2_of_3‚0.20174075663089752 8 Purity/2FRt3SRqqt1FBt/FRt_purity‚0.10000000149011612 9"Purity/3FRt3SRqqt*FBt/SRqqt_purity‚0.5476190447807312 ,&Purity/2FRt2SRqqt3FBt/event_proportion‚0 ,&Purity/3FRt*SRqqt4FBt/event_proportion‚0 ,&Purity/4FRt3SRqqt2FBt/event_proportion‚0 7 Purity/*FRt2SRqqt1FBt/FBt_purity‚0.9828523993492126 =&Purity/*FRt1SRqqt*FBt/event_proportion‚0.4482736885547638 ;"Purity/2FRt0SRqqt*FBt/event_purity‚0.011417882516980171 8 Purity/3FRt*SRqqt*FBt/FRt_purity‚0.21296724677085876 9"Purity/3FRt*SRqqt1FBt/SRqqt_purity‚0.4777778089046478 ,&Purity/4FRt4SRqqt1FBt/event_proportion‚0 & Purity/2FRt3SRqqt1FBt/FBt_purity‚1 & Purity/1FRt0SRqqt4FBt/FBt_purity‚1 :"Purity/1FRt*SRqqt*FBt/event_purity‚0.07448069751262665 6 Purity/3FRt1SRqqt*FBt/FRt_purity‚0.194444477558136 (particle/accuracy‚0.8436554074287415 +"Purity/2FRt4SRqqt0FBt/SRqqt_purity‚0.25 *jet/accuracy_2_of_6‚0.4298245906829834 ("Purity/0FRt3SRqqt1FBt/event_purity‚0 ("Purity/4FRt*SRqqt*FBt/event_purity‚0 :"Purity/0FRt0SRqqt2FBt/event_purity‚0.32851898670196533 ("Purity/1FRt1SRqqt3FBt/event_purity‚0 ?&Purity/2FRt0SRqqt0FBt/event_proportion‚0.024571573361754417 @&Purity/2FRt4SRqqt0FBt/event_proportion‚0.0003150201519019902 /"Purity/*FRt4SRqqt*FBt/SRqqt_purity‚0.265625 8 Purity/2FRt4SRqqt*FBt/FRt_purity‚0.30000001192092896 :"Purity/0FRt2SRqqt*FBt/event_purity‚0.03395435959100723 @&Purity/1FRt3SRqqt1FBt/event_proportion‚0.0011340725468471646 particle/accuracy_3_of_5‚1 7 Purity/*FRt2SRqqt2FBt/FBt_purity‚0.9912281036376953 & Purity/*FRt0SRqqt4FBt/FRt_purity‚0 :"Purity/*FRt0SRqqt*FBt/event_purity‚0.35872897505760193 :"Purity/1FRt3SRqqt0FBt/SRqqt_purity‚0.32388633489608765 particle/accuracy_1_of_4‚1 particle/accuracy_4_of_7‚1 ("Purity/2FRt2SRqqt2FBt/SRqqt_purity‚0 ?&Purity/0FRt3SRqqt0FBt/event_proportion‚0.006111391354352236 ("Purity/1FRt0SRqqt3FBt/event_purity‚0 particle/accuracy_2_of_5‚1 ("Purity/2FRt2SRqqt2FBt/event_purity‚0 5 Purity/*FRt*SRqqt*FBt/FRt_purity‚0.21834696829319 ?&Purity/1FRt0SRqqt2FBt/event_proportion‚0.015120967291295528 @&Purity/*FRt1SRqqt3FBt/event_proportion‚0.0006930443341843784 ("Purity/3FRt*SRqqt*FBt/event_purity‚0 & Purity/4FRt1SRqqt*FBt/FRt_purity‚0 ("Purity/4FRt2SRqqt0FBt/SRqqt_purity‚1 A&Purity/0FRt4SRqqt0FBt/event_proportion‚0.00044102821266278625 :"Purity/1FRt0SRqqt*FBt/event_purity‚0.15538960695266724 7 Purity/2FRt0SRqqt2FBt/FBt_purity‚0.9940475821495056 ?&Purity/2FRt1SRqqt2FBt/event_proportion‚0.001890120911411941 9"Purity/3FRt2SRqqt*FBt/SRqqt_purity‚0.3836805522441864 +"Purity/2FRt4SRqqt*FBt/SRqqt_purity‚0.25 & Purity/1FRt3SRqqt2FBt/FBt_purity‚1 9"Purity/*FRt*SRqqt*FBt/SRqqt_purity‚0.3560960292816162 7 Purity/*FRt0SRqqt2FBt/FBt_purity‚0.9806344509124756 >&Purity/0FRt0SRqqt*FBt/event_proportion‚0.11857359111309052 >&Purity/0FRt0SRqqt1FBt/event_proportion‚0.08606351166963577 ,&Purity/1FRt3SRqqt3FBt/event_proportion‚0 8 Purity/3FRt2SRqqt*FBt/FRt_purity‚0.19560183584690094 ,&Purity/3FRt3SRqqt4FBt/event_proportion‚0 ,&Purity/4FRt*SRqqt1FBt/event_proportion‚0 ("Purity/0FRt4SRqqt*FBt/event_purity‚0 :"Purity/*FRt3SRqqt1FBt/SRqqt_purity‚0.33068785071372986 @&Purity/0FRt2SRqqt3FBt/event_proportion‚6.300403038039804e-05 @&Purity/4FRt0SRqqt0FBt/event_proportion‚6.300403038039804e-05 particle/accuracy_1_of_3‚1 _runtime‚1095.020157554 ) Purity/1FRt1SRqqt3FBt/FRt_purity‚0.25 9"Purity/1FRt3SRqqt2FBt/SRqqt_purity‚0.3333333432674408 9"Purity/*FRt1SRqqt*FBt/SRqqt_purity‚0.3666665852069855 7 Purity/1FRt*SRqqt1FBt/FBt_purity‚0.9940850138664246 jet/accuracy_5_of_6‚0 ("Purity/1FRt3SRqqt2FBt/event_purity‚0 ("Purity/4FRt2SRqqt*FBt/SRqqt_purity‚1 9"Purity/*FRt2SRqqt0FBt/SRqqt_purity‚0.3505835235118866 ("Purity/0FRt0SRqqt0FBt/event_purity‚1 ,&Purity/0FRt3SRqqt2FBt/event_proportion‚0 ;"Purity/2FRt*SRqqt2FBt/event_purity‚0.011111111380159855 7 Purity/2FRt1SRqqt2FBt/FBt_purity‚0.9868420958518982 :"Purity/2FRt2SRqqt0FBt/SRqqt_purity‚0.38297781348228455 ,&Purity/4FRt2SRqqt3FBt/event_proportion‚0 particle/accuracy_3_of_4‚1 8 Purity/1FRt*SRqqt1FBt/FRt_purity‚0.26739901304244995 9"Purity/1FRt0SRqqt1FBt/event_purity‚0.2009960412979126 @&Purity/1FRt0SRqqt3FBt/event_proportion‚0.0010710685746744275 ("Purity/1FRt3SRqqt1FBt/event_purity‚0 9"Purity/2FRt*SRqqt2FBt/SRqqt_purity‚0.6359648704528809 7 Purity/3FRt0SRqqt*FBt/FRt_purity‚0.2196969836950302 ,&Purity/4FRt3SRqqt3FBt/event_proportion‚0 'particle/f_score‚0.3976779878139496 ?&Purity/*FRt*SRqqt3FBt/event_proportion‚0.004473286215215921 @&Purity/3FRt*SRqqt2FBt/event_proportion‚0.0003150201519019902 ("Purity/1FRt4SRqqt0FBt/event_purity‚0 ("Purity/1FRt1SRqqt3FBt/SRqqt_purity‚1 7 Purity/1FRt1SRqqt*FBt/FBt_purity‚0.9850632548332214 8 Purity/1FRt1SRqqt0FBt/FRt_purity‚0.22795183956623077 ("Purity/2FRt2SRqqt*FBt/event_purity‚0 particle/accuracy_4_of_5‚1 ("Purity/0FRt3SRqqt*FBt/event_purity‚0 particle/accuracy_4_of_6‚1 * Purity/0FRt3SRqqt1FBt/FBt_purity‚0.875 =&Purity/*FRt0SRqqt2FBt/event_proportion‚0.0483240932226181 A&Purity/1FRt1SRqqt3FBt/event_proportion‚0.00025201612152159214 @&Purity/4FRt2SRqqt*FBt/event_proportion‚6.300403038039804e-05 9"Purity/0FRt3SRqqt0FBt/SRqqt_purity‚0.3204365074634552 particle/accuracy_2_of_6‚1 7 Purity/3FRt3SRqqt*FBt/FRt_purity‚0.2142857164144516 ("Purity/4FRt1SRqqt0FBt/event_purity‚0 ( Purity/2FRt2SRqqt2FBt/FRt_purity‚0.5 :"Purity/*FRt1SRqqt1FBt/SRqqt_purity‚0.43834149837493896 =&Purity/1FRt*SRqqt0FBt/event_proportion‚0.3135710656642914 6 Purity/1FRt0SRqqt*FBt/FRt_purity‚0.211908221244812 9"Purity/1FRt1SRqqt*FBt/SRqqt_purity‚0.3763994574546814 ?&Purity/2FRt2SRqqt0FBt/event_proportion‚0.015435988083481789 & Purity/*FRt4SRqqt1FBt/FBt_purity‚1 *"Purity/*FRt2SRqqt3FBt/SRqqt_purity‚0.5 =&Purity/1FRt1SRqqt*FBt/event_proportion‚0.1723790317773819 8 Purity/1FRt2SRqqt1FBt/FRt_purity‚0.23348693549633026 particle/accuracy_6_of_6‚1 ("Purity/*FRt1SRqqt3FBt/event_purity‚0 ) Purity/1FRt1SRqqt4FBt/FBt_purity‚0.75 9"Purity/0FRt*SRqqt0FBt/event_purity‚0.2664025127887726 :"Purity/0FRt1SRqqt1FBt/event_purity‚0.12602807581424713 ?&Purity/1FRt0SRqqt1FBt/event_proportion‚0.049773186445236206 @&Purity/4FRt1SRqqt0FBt/event_proportion‚6.300403038039804e-05 *jet/accuracy_3_of_5‚0.0911157876253128 ("Purity/2FRt0SRqqt3FBt/event_purity‚0 7 Purity/1FRt2SRqqt1FBt/FBt_purity‚0.9806451201438904 ("Purity/1FRt3SRqqt*FBt/event_purity‚0 ?&Purity/1FRt3SRqqt*FBt/event_proportion‚0.009135584346950054 & Purity/3FRt*SRqqt*FBt/FBt_purity‚1 7 Purity/3FRt*SRqqt0FBt/FRt_purity‚0.1829100400209427 +"Purity/3FRt4SRqqt*FBt/SRqqt_purity‚0.25 7 Purity/0FRt2SRqqt3FBt/FBt_purity‚0.6666666865348816 ,&Purity/2FRt1SRqqt3FBt/event_proportion‚0 ?&Purity/2FRt2SRqqt*FBt/event_proportion‚0.019468246027827263 ,&Purity/4FRt*SRqqt3FBt/event_proportion‚0 particle/accuracy_3_of_3‚1 9"Purity/*FRt1SRqqt2FBt/SRqqt_purity‚0.4528195559978485 8 Purity/1FRt1SRqqt2FBt/FRt_purity‚0.38222792744636536 ;"Purity/2FRt*SRqqt*FBt/event_purity‚0.003873026231303811 8 Purity/2FRt*SRqqt*FBt/FRt_purity‚0.21636322140693665 ,&Purity/3FRt4SRqqt2FBt/event_proportion‚0 ,&Purity/3FRt4SRqqt4FBt/event_proportion‚0 & Purity/*FRt3SRqqt2FBt/FBt_purity‚1 # _timestamp‚1.78848629591639e+09 :"Purity/*FRt1SRqqt2FBt/event_purity‚0.03143997862935066 :"Purity/1FRt2SRqqt1FBt/SRqqt_purity‚0.32803377509117126 ,&Purity/1FRt3SRqqt4FBt/event_proportion‚0 :"Purity/3FRt1SRqqt*FBt/SRqqt_purity‚0.39814817905426025 ,&Purity/4FRt4SRqqt*FBt/event_proportion‚0 8 Purity/*FRt1SRqqt*FBt/FRt_purity‚0.23056207597255707 >&Purity/0FRt1SRqqt1FBt/event_proportion‚0.05185231938958168 @&Purity/0FRt4SRqqt1FBt/event_proportion‚6.300403038039804e-05 ?&Purity/1FRt*SRqqt2FBt/event_proportion‚0.022303426638245583 *jet/accuracy_1_of_2‚0.6902841329574585 +"Purity/*FRt2SRqqt2FBt/SRqqt_purity‚0.25 9"Purity/*FRt1SRqqt3FBt/SRqqt_purity‚0.7619047164916992 7 Purity/1FRt1SRqqt3FBt/FBt_purity‚0.8333333730697632 7 Purity/2FRt2SRqqt*FBt/FBt_purity‚0.9821428656578064 8 Purity/2FRt2SRqqt0FBt/FRt_purity‚0.17782025039196014 ,&Purity/3FRt1SRqqt3FBt/event_proportion‚0 @&Purity/4FRt*SRqqt0FBt/event_proportion‚0.0001890120911411941 ,&Purity/4FRt2SRqqt4FBt/event_proportion‚0 & Purity/0FRt4SRqqt1FBt/FBt_purity‚1 ("Purity/*FRt2SRqqt2FBt/event_purity‚0 :"Purity/*FRt*SRqqt1FBt/SRqqt_purity‚0.40597864985466003 @&Purity/4FRt1SRqqt*FBt/event_proportion‚6.300403038039804e-05 7 Purity/3FRt4SRqqt*FBt/FRt_purity‚0.1666666716337204 ("Purity/3FRt1SRqqt2FBt/event_purity‚0 ("Purity/*FRt2SRqqt3FBt/event_purity‚0 @&Purity/*FRt4SRqqt0FBt/event_proportion‚0.0012600806076079607 >&Purity/0FRt2SRqqt*FBt/event_proportion‚0.05361643061041832 7 Purity/0FRt2SRqqt*FBt/FBt_purity‚0.9685100317001343 ("Purity/3FRt1SRqqt1FBt/event_purity‚0 *"Purity/0FRt0SRqqt4FBt/event_purity‚0.5 *"Purity/3FRt1SRqqt2FBt/SRqqt_purity‚0.5 >&Purity/*FRt0SRqqt0FBt/event_proportion‚0.15757308900356293 :"Purity/0FRt1SRqqt2FBt/SRqqt_purity‚0.38585978746414185 ;"Purity/1FRt1SRqqt0FBt/event_purity‚0.008950828574597836 :"Purity/1FRt2SRqqt*FBt/SRqqt_purity‚0.34632670879364014 9"Purity/2FRt1SRqqt*FBt/SRqqt_purity‚0.3825775980949402 @&Purity/3FRt0SRqqt1FBt/event_proportion‚0.0011340725468471646 ,&Purity/4FRt*SRqqt2FBt/event_proportion‚0 7validation_average_jet_accuracy‚0.37975460290908813 ,&Purity/*FRt3SRqqt4FBt/event_proportion‚0 =&Purity/*FRt0SRqqt*FBt/event_proportion‚0.3558467626571655 ;"Purity/0FRt1SRqqt2FBt/event_purity‚0.051428571343421936 ?&Purity/3FRt*SRqqt*FBt/event_proportion‚0.009954636916518211 8 Purity/3FRt*SRqqt2FBt/FRt_purity‚0.20000000298023224 ("Purity/4FRt0SRqqt0FBt/event_purity‚0 & Purity/4FRt2SRqqt0FBt/FRt_purity‚0 ;"Purity/2FRt0SRqqt1FBt/event_purity‚0.009259259328246117 9"Purity/2FRt2SRqqt1FBt/SRqqt_purity‚0.3538690507411957 @&Purity/3FRt0SRqqt2FBt/event_proportion‚6.300403038039804e-05 8 Purity/3FRt2SRqqt0FBt/FRt_purity‚0.18719805777072906 ,&Purity/4FRt4SRqqt0FBt/event_proportion‚0 & Purity/3FRt2SRqqt*FBt/FBt_purity‚1 7 Purity/3FRt1SRqqt2FBt/FRt_purity‚0.1666666716337204 :"Purity/*FRt2SRqqt1FBt/SRqqt_purity‚0.35507386922836304 >&Purity/2FRt*SRqqt0FBt/event_proportion‚0.07579384744167328 ,&Purity/4FRt*SRqqt4FBt/event_proportion‚0 ("Purity/4FRt2SRqqt0FBt/event_purity‚0 :"Purity/1FRt*SRqqt2FBt/event_purity‚0.05309804528951645 ("Purity/1FRt3SRqqt0FBt/event_purity‚0 ?&Purity/2FRt2SRqqt1FBt/event_proportion‚0.003969253972172737 +jet/accuracy_2_of_5‚0.44650667905807495 particle/accuracy_2_of_4‚1 ("Purity/1FRt2SRqqt2FBt/event_purity‚0 9"Purity/1FRt*SRqqt2FBt/SRqqt_purity‚0.3604966700077057 particle/accuracy_1_of_2‚1 9"Purity/1FRt2SRqqt2FBt/SRqqt_purity‚0.2986111342906952 7 Purity/*FRt*SRqqt3FBt/FBt_purity‚0.9028735756874084 6 Purity/*FRt3SRqqt1FBt/FBt_purity‚0.976190447807312 9"Purity/3FRt2SRqqt1FBt/SRqqt_purity‚0.3333333432674408 particle/accuracy_7_of_7‚1 ("Purity/4FRt1SRqqt*FBt/SRqqt_purity‚1 =&Purity/*FRt2SRqqt*FBt/event_proportion‚0.1332535296678543 A&Purity/0FRt*SRqqt4FBt/event_proportion‚0.00025201612152159214 9"Purity/0FRt2SRqqt1FBt/SRqqt_purity‚0.3843318223953247 ,&Purity/0FRt4SRqqt3FBt/event_proportion‚0 :"Purity/1FRt1SRqqt1FBt/event_purity‚0.01746870018541813 ,&Purity/4FRt1SRqqt4FBt/event_proportion‚0 >&Purity/*FRt*SRqqt2FBt/event_proportion‚0.06747731566429138 8 Purity/*FRt*SRqqt2FBt/FRt_purity‚0.41108572483062744 :"Purity/*FRt0SRqqt0FBt/event_purity‚0.12620827555656433 8 Purity/*FRt3SRqqt1FBt/FRt_purity‚0.24444444477558136 @&Purity/1FRt2SRqqt2FBt/event_proportion‚0.0010080644860863686 & Purity/3FRt0SRqqt*FBt/FBt_purity‚1 8 Purity/3FRt0SRqqt0FBt/FRt_purity‚0.09803921729326248 ("Purity/2FRt4SRqqt0FBt/event_purity‚0 7 Purity/0FRt1SRqqt*FBt/FBt_purity‚0.9857016205787659 ( Purity/1FRt0SRqqt3FBt/FRt_purity‚0.5 ?&Purity/1FRt3SRqqt0FBt/event_proportion‚0.007938507944345474 ;"Purity/2FRt*SRqqt1FBt/event_purity‚0.004385964944958687 ("Purity/1FRt*SRqqt4FBt/event_purity‚0 :"Purity/*FRt2SRqqt0FBt/event_purity‚0.01597520150244236 ?&Purity/2FRt*SRqqt2FBt/event_proportion‚0.005418346729129553 ("Purity/2FRt2SRqqt0FBt/event_purity‚0 9"Purity/2FRt3SRqqt*FBt/SRqqt_purity‚0.2994285821914673 ,&Purity/2FRt3SRqqt4FBt/event_proportion‚0 ,&Purity/*FRt4SRqqt3FBt/event_proportion‚0 & Purity/3FRt1SRqqt*FBt/FBt_purity‚1 8 Purity/3FRt1SRqqt0FBt/FRt_purity‚0.16111113131046295 ,jet/accuracy_3_of_6‚0.061403512954711914 9"Purity/1FRt4SRqqt0FBt/SRqqt_purity‚0.4166666567325592 >&Purity/0FRt*SRqqt0FBt/event_proportion‚0.21931703388690948 @&Purity/2FRt4SRqqt*FBt/event_proportion‚0.0003150201519019902 ,&Purity/3FRt2SRqqt2FBt/event_proportion‚0 * Purity/*FRt*SRqqt4FBt/FBt_purity‚0.875 & Purity/1FRt2SRqqt2FBt/FBt_purity‚1 =&Purity/0FRt1SRqqt0FBt/event_proportion‚0.1694808453321457 8 Purity/2FRt4SRqqt0FBt/FRt_purity‚0.30000001192092896 9"Purity/0FRt0SRqqt*FBt/event_purity‚0.8139540553092957 ?&Purity/0FRt0SRqqt2FBt/event_proportion‚0.029674898833036423 A&Purity/0FRt1SRqqt3FBt/event_proportion‚0.00044102821266278625 :"Purity/1FRt*SRqqt1FBt/event_purity‚0.10717689245939255 ,&Purity/2FRt4SRqqt1FBt/event_proportion‚0 ("Purity/*FRt4SRqqt1FBt/event_purity‚0 ( Purity/2FRt*SRqqt3FBt/FRt_purity‚0.5 7 Purity/0FRt1SRqqt3FBt/FBt_purity‚0.8999999761581421 >&Purity/0FRt*SRqqt*FBt/event_proportion‚0.40996724367141724 ,&Purity/4FRt0SRqqt1FBt/event_proportion‚0 ,&Purity/4FRt3SRqqt1FBt/event_proportion‚0 +particle/sensitivity‚0.3305012285709381 9"Purity/1FRt4SRqqt*FBt/SRqqt_purity‚0.4166666567325592 ("Purity/1FRt*SRqqt4FBt/SRqqt_purity‚1 =&Purity/*FRt*SRqqt0FBt/event_proportion‚0.6160534024238586 9"Purity/0FRt*SRqqt2FBt/event_purity‚0.2624504864215851 ) Purity/*FRt1SRqqt3FBt/FRt_purity‚0.25 & Purity/4FRt1SRqqt0FBt/FRt_purity‚0 ?&Purity/2FRt3SRqqt0FBt/event_proportion‚0.003654233878478408 9"Purity/0FRt*SRqqt2FBt/SRqqt_purity‚0.3739814758300781 >&Purity/*FRt3SRqqt0FBt/event_proportion‚0.01827116869390011 @&Purity/*FRt4SRqqt1FBt/event_proportion‚6.300403038039804e-05 7 Purity/*FRt*SRqqt*FBt/FBt_purity‚0.9864770770072937 8 Purity/*FRt*SRqqt0FBt/FRt_purity‚0.18591664731502533 >&Purity/*FRt1SRqqt0FBt/event_proportion‚0.33039313554763794 7 Purity/3FRt0SRqqt2FBt/FRt_purity‚0.3333333432674408 7 Purity/1FRt*SRqqt3FBt/FBt_purity‚0.9270833134651184 7 Purity/1FRt0SRqqt2FBt/FBt_purity‚0.9801872372627258 +jet/accuracy_2_of_2‚0.08364889025688171 ("Purity/1FRt0SRqqt4FBt/event_purity‚0 9"Purity/*FRt0SRqqt2FBt/event_purity‚0.2297607809305191 :"Purity/*FRt1SRqqt*FBt/event_purity‚0.14601264894008636 ("Purity/*FRt3SRqqt1FBt/event_pÊóšö`-urity‚0 ,&Purity/1FRt4SRqqt3FBt/event_proportion‚0 jet/accuracy_4_of_6‚0 ("Purity/4FRt2SRqqt*FBt/event_purity‚0 9"Purity/*FRt0SRqqt1FBt/event_purity‚0.6592440009117126 :"Purity/0FRt2SRqqt0FBt/SRqqt_purity‚0.34718528389930725 @&Purity/0FRt2SRqqt2FBt/event_proportion‚0.0010080644860863686 7 Purity/1FRt*SRqqt2FBt/FRt_purity‚0.4516954720020294 ?&Purity/1FRt1SRqqt2FBt/event_proportion‚0.006111391354352236 ,&Purity/2FRt4SRqqt3FBt/event_proportion‚0 :"Purity/0FRt3SRqqt*FBt/SRqqt_purity‚0.31226375699043274 ,&Purity/*FRt2SRqqt4FBt/event_proportion‚0 @&Purity/*FRt3SRqqt2FBt/event_proportion‚6.300403038039804e-05 7 Purity/0FRt*SRqqt2FBt/FBt_purity‚0.9759090542793274 @&Purity/3FRt1SRqqt0FBt/event_proportion‚0.0025201612152159214 jet/accuracy_7_of_7‚0 & Purity/*FRt1SRqqt4FBt/FRt_purity‚1 A&Purity/1FRt*SRqqt4FBt/event_proportion‚0.00012600806076079607 ,&Purity/1FRt2SRqqt3FBt/event_proportion‚0 @&Purity/3FRt1SRqqt1FBt/event_proportion‚0.0009450604557059705 ("Purity/3FRt2SRqqt*FBt/event_purity‚0 ("Purity/2FRt4SRqqt*FBt/event_purity‚0 :"Purity/*FRt*SRqqt*FBt/event_purity‚0.20322614908218384 :"Purity/0FRt1SRqqt0FBt/event_purity‚0.33452585339546204 8"Purity/1FRt3SRqqt1FBt/SRqqt_purity‚0.307407408952713 ("Purity/3FRt*SRqqt0FBt/event_purity‚0 ("Purity/3FRt3SRqqt0FBt/event_purity‚0 7 Purity/0FRt0SRqqt2FBt/FBt_purity‚0.9796414971351624 ,&Purity/0FRt1SRqqt4FBt/event_proportion‚0 @&Purity/0FRt4SRqqt*FBt/event_proportion‚0.0005040322430431843 8"Purity/2FRt3SRqqt0FBt/SRqqt_purity‚0.283189058303833 ."Purity/0FRt4SRqqt*FBt/SRqqt_purity‚0.15625 7 Purity/1FRt4SRqqt0FBt/FRt_purity‚0.1666666716337204 & Purity/4FRt*SRqqt*FBt/FRt_purity‚0 ,&Purity/0FRt3SRqqt4FBt/event_proportion‚0 @&Purity/3FRt2SRqqt*FBt/event_proportion‚0.0028351815417408943 ,&Purity/3FRt2SRqqt3FBt/event_proportion‚0 9"Purity/*FRt0SRqqt4FBt/event_purity‚0.4000000059604645 >&Purity/0FRt0SRqqt0FBt/event_proportion‚0.04082661122083664 8"Purity/0FRt0SRqqt1FBt/event_purity‚0.999193549156189 =&Purity/1FRt*SRqqt*FBt/event_proportion‚0.4366179406642914 >&Purity/1FRt0SRqqt0FBt/event_proportion‚0.13142640888690948 * Purity/0FRt3SRqqt*FBt/FBt_purity‚0.875 7 Purity/*FRt1SRqqt2FBt/FBt_purity‚0.9684198498725891 ("Purity/1FRt2SRqqt0FBt/event_purity‚0 'jet/accuracy_1_of_1‚0.4051533639431 @&Purity/3FRt*SRqqt1FBt/event_proportion‚0.0024571572430431843 9"Purity/0FRt3SRqqt1FBt/SRqqt_purity‚0.2083333432674408 & Purity/1FRt0SRqqt4FBt/FRt_purity‚0 9"Purity/*FRt1SRqqt0FBt/event_purity‚0.1748521625995636 8 Purity/*FRt3SRqqt*FBt/FRt_purity‚0.18883544206619263 7 Purity/1FRt*SRqqt*FBt/FBt_purity‚0.9865187406539917 8 Purity/1FRt*SRqqt0FBt/FRt_purity‚0.18954649567604065 7 Purity/2FRt*SRqqt2FBt/FBt_purity‚0.9911110997200012 9"Purity/3FRt*SRqqt*FBt/SRqqt_purity‚0.3652161955833435 ("Purity/3FRt2SRqqt0FBt/event_purity‚0 ,&Purity/3FRt3SRqqt2FBt/event_proportion‚0 @&Purity/*FRt1SRqqt4FBt/event_proportion‚6.300403038039804e-05 >&Purity/0FRt*SRqqt1FBt/event_proportion‚0.14787046611309052 9"Purity/1FRt*SRqqt*FBt/SRqqt_purity‚0.3603418469429016 ,&Purity/4FRt3SRqqt*FBt/event_proportion‚0 +particle/specificity‚0.9386556148529053 :"Purity/0FRt2SRqqt2FBt/SRqqt_purity‚0.23076923191547394 >&Purity/1FRt0SRqqt*FBt/event_proportion‚0.19745463132858276 ("Purity/0FRt2SRqqt2FBt/event_purity‚0 ,&Purity/0FRt3SRqqt3FBt/event_proportion‚0 ("Purity/*FRt4SRqqt*FBt/event_purity‚0 ("Purity/3FRt*SRqqt2FBt/event_purity‚0 * Purity/1FRt*SRqqt4FBt/FBt_purity‚0.875 & Purity/4FRt0SRqqt*FBt/FRt_purity‚0 ?&Purity/*FRt2SRqqt2FBt/event_proportion‚0.002079133177176118 ?&Purity/*FRt0SRqqt3FBt/event_proportion‚0.003717237850651145 9"Purity/0FRt2SRqqt*FBt/SRqqt_purity‚0.3465501070022583 ,&Purity/1FRt4SRqqt2FBt/event_proportion‚0 ,&Purity/3FRt4SRqqt3FBt/event_proportion‚0 particle/accuracy_2_of_2‚1 *"Purity/3FRt*SRqqt2FBt/SRqqt_purity‚0.5 >&Purity/*FRt*SRqqt1FBt/event_proportion‚0.27079132199287415 ?&Purity/0FRt0SRqqt3FBt/event_proportion‚0.002583165420219302 <"Purity/1FRt1SRqqt2FBt/event_purity‚0.0071428571827709675 7 Purity/*FRt2SRqqt2FBt/FRt_purity‚0.3974359333515167 ,&Purity/0FRt2SRqqt4FBt/event_proportion‚0 9"Purity/1FRt1SRqqt2FBt/SRqqt_purity‚0.3561224639415741 ?&Purity/1FRt2SRqqt1FBt/event_proportion‚0.008946572430431843 ("Purity/3FRt0SRqqt0FBt/event_purity‚0 @&Purity/4FRt0SRqqt*FBt/event_proportion‚6.300403038039804e-05 ,&Purity/4FRt4SRqqt3FBt/event_proportion‚0 ("Purity/3FRt4SRqqt0FBt/event_purity‚0 ("Purity/3FRt3SRqqt*FBt/event_purity‚0 & Purity/1FRt3SRqqt*FBt/FBt_purity‚1 8 Purity/1FRt3SRqqt0FBt/FRt_purity‚0.23260369896888733 7 Purity/2FRt0SRqqt*FBt/FBt_purity‚0.9967741966247559 8 Purity/2FRt0SRqqt0FBt/FRt_purity‚0.14567704498767853 >&Purity/2FRt1SRqqt0FBt/event_proportion‚0.03181703761219978 9"Purity/2FRt1SRqqt1FBt/SRqqt_purity‚0.4020737111568451 ("Purity/4FRt*SRqqt0FBt/SRqqt_purity‚1 7 Purity/*FRt2SRqqt3FBt/FBt_purity‚0.6666666865348816 7 Purity/*FRt0SRqqt*FBt/FBt_purity‚0.9871624708175659 8 Purity/*FRt0SRqqt0FBt/FRt_purity‚0.14835508167743683 >&Purity/*FRt2SRqqt1FBt/event_proportion‚0.02255544438958168 :"Purity/0FRt*SRqqt1FBt/SRqqt_purity‚0.40107712149620056 ?&Purity/0FRt1SRqqt2FBt/event_proportion‚0.008757560513913631 ,&Purity/3FRt4SRqqt1FBt/event_proportion‚0 particle/accuracy_5_of_6‚1 ("Purity/4FRt*SRqqt*FBt/SRqqt_purity‚1 :"Purity/0FRt1SRqqt*FBt/event_purity‚0.27562132477760315 =&Purity/1FRt1SRqqt0FBt/event_proportion‚0.1265120953321457 7 Purity/2FRt1SRqqt*FBt/FBt_purity‚0.9964157938957214 ("Purity/2FRt1SRqqt0FBt/event_purity‚0 8 Purity/2FRt1SRqqt0FBt/FRt_purity‚0.19673781096935272 6 Purity/*FRt1SRqqt1FBt/FBt_purity‚0.995071291923523 :"Purity/*FRt3SRqqt*FBt/SRqqt_purity‚0.32137051224708557 8 Purity/2FRt*SRqqt1FBt/FRt_purity‚0.29634109139442444 ("Purity/3FRt*SRqqt1FBt/event_purity‚0 particle/accuracy_3_of_7‚1 ,&Purity/3FRt3SRqqt3FBt/event_proportion‚0 7 Purity/*FRt0SRqqt4FBt/FBt_purity‚0.8999999761581421 /"Purity/*FRt4SRqqt0FBt/SRqqt_purity‚0.265625 7 Purity/*FRt1SRqqt1FBt/FRt_purity‚0.2541731894016266 A&Purity/0FRt0SRqqt4FBt/event_proportion‚0.00025201612152159214 >&Purity/1FRt1SRqqt1FBt/event_proportion‚0.03944052383303642 7 Purity/2FRt*SRqqt1FBt/FBt_purity‚0.9946236610412598 ,&Purity/4FRt0SRqqt4FBt/event_proportion‚0 jet/accuracy_5_of_5‚0 7 Purity/*FRt0SRqqt3FBt/FBt_purity‚0.9093254804611206 ,&Purity/*FRt4SRqqt2FBt/event_proportion‚0 9"Purity/0FRt*SRqqt*FBt/SRqqt_purity‚0.3483651876449585 ,&Purity/2FRt3SRqqt3FBt/event_proportion‚0 ,&Purity/4FRt0SRqqt3FBt/event_proportion‚0 *jet/accuracy_1_of_3‚0.8177136778831482 jet/accuracy_5_of_7‚0 7 Purity/*FRt2SRqqt*FBt/FBt_purity‚0.9811604619026184 8 Purity/*FRt2SRqqt0FBt/FRt_purity‚0.18745605647563934 :"Purity/*FRt3SRqqt0FBt/SRqqt_purity‚0.32116320729255676 8 Purity/1FRt2SRqqt*FBt/FRt_purity‚0.19866220653057098 8 Purity/2FRt3SRqqt0FBt/FRt_purity‚0.20519481599330902 ("Purity/0FRt4SRqqt0FBt/event_purity‚0 & Purity/2FRt3SRqqt*FBt/FBt_purity‚1 ("Purity/2FRt3SRqqt1FBt/event_purity‚0 7 Purity/*FRt0SRqqt2FBt/FRt_purity‚0.4521593153476715 6 Purity/0FRt0SRqqt1FBt/FBt_purity‚0.999193549156189 @&Purity/1FRt1SRqqt4FBt/event_proportion‚6.300403038039804e-05 -jet/accuracy_4_of_5‚0.0021505376789718866 & Purity/2FRt0SRqqt3FBt/FBt_purity‚1 9"Purity/*FRt0SRqqt3FBt/event_purity‚0.1577381044626236 8 Purity/2FRt0SRqqt2FBt/FRt_purity‚0.46071428060531616 ,&Purity/3FRt2SRqqt4FBt/event_proportion‚0 & Purity/1FRt3SRqqt2FBt/FRt_purity‚1 :"Purity/*FRt*SRqqt0FBt/event_purity‚0.12886430323123932 7 Purity/1FRt0SRqqt1FBt/FBt_purity‚0.9966126084327698 9"Purity/2FRt*SRqqt0FBt/SRqqt_purity‚0.3539843261241913 A&Purity/3FRt1SRqqt2FBt/event_proportion‚0.00025201612152159214 6 Purity/*FRt4SRqqt*FBt/FRt_purity‚0.236111119389534 9"Purity/1FRt*SRqqt0FBt/event_purity‚0.0663265734910965 8 Purity/2FRt1SRqqt2FBt/FRt_purity‚0.37719297409057617 ,"Purity/3FRt1SRqqt0FBt/SRqqt_purity‚0.375 @&Purity/3FRt3SRqqt*FBt/event_proportion‚0.0005670362734235823 7 Purity/1FRt0SRqqt1FBt/FRt_purity‚0.2966076135635376 ,&Purity/1FRt4SRqqt4FBt/event_proportion‚0 ,&Purity/3FRt1SRqqt4FBt/event_proportion‚0 +"Purity/0FRt4SRqqt1FBt/SRqqt_purity‚0.25 ;"Purity/*FRt2SRqqt*FBt/event_purity‚0.013820252381265163 7 Purity/0FRt*SRqqt1FBt/FBt_purity‚0.9960033297538757 :"Purity/*FRt*SRqqt2FBt/SRqqt_purity‚0.42012831568717957 @&Purity/0FRt3SRqqt1FBt/event_proportion‚0.0006300403038039804 ("Purity/2FRt3SRqqt0FBt/event_purity‚0 & Purity/3FRt0SRqqt1FBt/FBt_purity‚1 @&Purity/3FRt2SRqqt1FBt/event_proportion‚0.0003780241822823882 *jet/accuracy_1_of_4‚0.8711963295936584 9"Purity/0FRt*SRqqt3FBt/SRqqt_purity‚0.4333333373069763 ("Purity/*FRt3SRqqt2FBt/event_purity‚0 :"Purity/*FRt*SRqqt2FBt/event_purity‚0.17307427525520325 7 Purity/0FRt1SRqqt1FBt/FBt_purity‚0.9942485094070435 particle/accuracy_1_of_6‚1 jet/accuracy_1_of_7‚1 jet/accuracy_4_of_7‚0 ("Purity/1FRt4SRqqt*FBt/event_purity‚0 ("Purity/*FRt1SRqqt4FBt/SRqqt_purity‚1 & Purity/1FRt1SRqqt4FBt/FRt_purity‚1 7 Purity/*FRt3SRqqt*FBt/FBt_purity‚0.9772727489471436 8 Purity/*FRt3SRqqt0FBt/FRt_purity‚0.20790085196495056 ?&Purity/0FRt2SRqqt1FBt/event_proportion‚0.009261592291295528 8 Purity/1FRt*SRqqt*FBt/FRt_purity‚0.22017589211463928 ,&Purity/2FRt2SRqqt4FBt/event_proportion‚0 ,&Purity/2FRt3SRqqt2FBt/event_proportion‚0 7 Purity/3FRt0SRqqt1FBt/FRt_purity‚0.3461538553237915 particle/accuracy_5_of_7‚1 ?&Purity/*FRt3SRqqt1FBt/event_proportion‚0.002142137149348855 ,&Purity/*FRt3SRqqt3FBt/event_proportion‚0 :"Purity/0FRt1SRqqt0FBt/SRqqt_purity‚0.33452585339546204 @&Purity/1FRt3SRqqt2FBt/event_proportion‚6.300403038039804e-05 8"Purity/3FRt*SRqqt0FBt/SRqqt_purity‚0.358956903219223 @&Purity/3FRt3SRqqt0FBt/event_proportion‚0.0005670362734235823 ,&Purity/4FRt1SRqqt3FBt/event_proportion‚0 *"Purity/0FRt1SRqqt3FBt/SRqqt_purity‚0.5 *loss/val_total_loss‚49.371364115333826 ;"Purity/0FRt2SRqqt0FBt/event_purity‚0.040243521332740784 ("Purity/2FRt3SRqqt*FBt/event_purity‚0 7 Purity/3FRt2SRqqt1FBt/FRt_purity‚0.2222222238779068 9"Purity/3FRt3SRqqt0FBt/SRqqt_purity‚0.5476190447807312 9"Purity/*FRt3SRqqt2FBt/SRqqt_purity‚0.3333333432674408 & Purity/2FRt2SRqqt2FBt/FBt_purity‚1 ("Purity/0FRt2SRqqt3FBt/event_purity‚0 @&Purity/0FRt*SRqqt3FBt/event_proportion‚0.0030871976632624865 ,&Purity/3FRt3SRqqt1FBt/event_proportion‚0 _step‚48 :"Purity/1FRt*SRqqt1FBt/SRqqt_purity‚0.41674521565437317 @&Purity/3FRt2SRqqt0FBt/event_proportion‚0.0024571572430431843 +"Purity/*FRt4SRqqt1FBt/SRqqt_purity‚0.25 & Purity/3FRt2SRqqt1FBt/FBt_purity‚1 ("Purity/1FRt1SRqqt4FBt/SRqqt_purity‚1 trainer/global_step‚603 @&Purity/*FRt4SRqqt*FBt/event_proportion‚0.0013230846961960196 ,&Purity/4FRt2SRqqt2FBt/event_proportion‚0 7 Purity/0FRt2SRqqt2FBt/FBt_purity‚0.9807692170143127 & Purity/4FRt0SRqqt0FBt/FRt_purity‚0 ?&Purity/0FRt2SRqqt0FBt/event_proportion‚0.043283771723508835 @&Purity/1FRt*SRqqt3FBt/event_proportion‚0.0013230846961960196 ,&Purity/4FRt2SRqqt1FBt/event_proportion‚0 particle/accuracy_2_of_3‚1 ("Purity/4FRt1SRqqt*FBt/event_purity‚0 epoch‚0 7 Purity/*FRt*SRqqt1FBt/FBt_purity‚0.9955952167510986 ,&Purity/4FRt3SRqqt0FBt/event_proportion‚0 ,&Purity/4FRt3SRqqt4FBt/event_proportion‚0:Dë’R£ ¯§ 7 Purity/2FRt0SRqqt*FBt/FBt_purity‚0.9967741966247559 8 Purity/2FRt0SRqqt0FBt/FRt_purity‚0.14567704498767853 >&Purity/2FRt1SRqqt0FBt/event_proportion‚0.03181703761219978 9"Purity/2FRt1SRqqt1FBt/SRqqt_purity‚0.4020737111568451 ("Purity/4FRt*SRqqt0FBt/SRqqt_purity‚1 7 Purity/*FRt2SRqqt3FBt/FBt_purity‚0.6666666865348816 7 Purity/*FRt0SRqqt*FBt/FBt_purity‚0.9871624708175659 8 Purity/*FRt0SRqqt0FBt/FRt_purity‚0.14835508167743683 >&Purity/*FRt2SRqqt1FBt/event_proportion‚0.02255544438958168 :"Purity/0FRt*SRqqt1FBt/SRqqt_purity‚0.40107712149620056 ?&Purity/0FRt1SRqqt2FBt/event_proportion‚0.008757560513913631 ,&Purity/3FRt4SRqqt1FBt/event_proportion‚0 particle/accuracy_5_of_6‚1 ("Purity/4FRt*SRqqt*FBt/SRqqt_purity‚1 :"Purity/0FRt1SRqqt*FBt/event_purity‚0.27562132477760315 =&Purity/1FRt1SRqqt0FBt/event_proportion‚0.1265120953321457 7 Purity/2FRt1SRqqt*FBt/FBt_purity‚0.9964157938957214 ("Purity/2FRt1SRqqt0FBt/event_purity‚0 8 Purity/2FRt1SRqqt0FBt/FRt_purity‚0.19673781096935272 6 Purity/*FRt1SRqqt1FBt/FBt_purity‚0.995071291923523 :"Purity/*FRt3SRqqt*FBt/SRqqt_purity‚0.32137051224708557 8 Purity/2FRt*SRqqt1FBt/FRt_purity‚0.29634109139442444 ("Purity/3FRt*SRqqt1FBt/event_purity‚0 particle/accuracy_3_of_7‚1 ,&Purity/3FRt3SRqqt3FBt/event_proportion‚0 7 Purity/*FRt0SRqqt4FBt/FBt_purity‚0.8999999761581421 /"Purity/*FRt4SRqqt0FBt/SRqqt_purity‚0.265625 7 Purity/*FRt1SRqqt1FBt/FRt_purity‚0.2541731894016266 A&Purity/0FRt0SRqqt4FBt/event_proportion‚0.00025201612152159214 >&Purity/1FRt1SRqqt1FBt/event_proportion‚0.03944052383303642 7 Purity/2FRt*SRqqt1FBt/FBt_purity‚0.9946236610412598 ,&Purity/4FRt0SRqqt4FBt/event_proportion‚0 jet/accuracy_5_of_5‚0 7 Purity/*FRt0SRqqt3FBt/FBt_purity‚0.9093254804611206 ,&Purity/*FRt4SRqqt2FBt/event_proportion‚0 9"Purity/0FRt*SRqqt*FBt/SRqqt_purity‚0.3483651876449585 ,&Purity/2FRt3SRqqt3FBt/event_proportion‚0 ,&Purity/4FRt0SRqqt3FBt/event_proportion‚0 *jet/accuracy_1_of_3‚0.8177136778831482 jet/accuracy_5_of_7‚0 7 Purity/*FRt2SRqqt*FBt/FBt_purity‚0.9811604619026184 8 Purity/*FRt2SRqqt0FBt/FRt_purity‚0.18745605647563934 :"Purity/*FRt3SRqqt0FBt/SRqqt_purity‚0.32116320729255676 8 Purity/1FRt2SRqqt*FBt/FRt_purity‚0.19866220653057098 8 Purity/2FRt3SRqqt0FBt/FRt_purity‚0.20519481599330902 ("Purity/0FRt4SRqqt0FBt/event_purity‚0 & Purity/2FRt3SRqqt*FBt/FBt_purity‚1 ("Purity/2FRt3SRqqt1FBt/event_purity‚0 7 Purity/*FRt0SRqqt2FBt/FRt_purity‚0.4521593153476715 6 Purity/0FRt0SRqqt1FBt/FBt_purity‚0.999193549156189 @&Purity/1FRt1SRqqt4FBt/event_proportion‚6.300403038039804e-05 -jet/accuracy_4_of_5‚0.0021505376789718866 & Purity/2FRt0SRqqt3FBt/FBt_purity‚1 9"Purity/*FRt0SRqqt3FBt/event_purity‚0.1577381044626236 8 Purity/2FRt0SRqqt2FBt/FRt_purity‚0.46071428060531616 ,&Purity/3FRt2SRqqt4FBt/event_proportion‚0 & Purity/1FRt3SRqqt2FBt/FRt_purity‚1 :"Purity/*FRt*SRqqt0FBt/event_purity‚0.12886430323123932 7 Purity/1FRt0SRqqt1FBt/FBt_purity‚0.9966126084327698 9"Purity/2FRt*SRqqt0FBt/SRqqt_purity‚0.3539843261241913 A&Purity/3FRt1SRqqt2FBt/event_proportion‚0.00025201612152159214 6 Purity/*FRt4SRqqt*FBt/FRt_purity‚0.236111119389534 9"Purity/1FRt*SRqqt0FBt/event_purity‚0.0663265734910965 8 Purity/2FRt1SRqqt2FBt/FRt_purity‚0.37719297409057617 ,"Purity/3FRt1SRqqt0FBt/SRqqt_purity‚0.375 @&Purity/3FRt3SRqqt*FBt/event_proportion‚0.0005670362734235823 7 Purity/1FRt0SRqqt1FBt/FRt_purity‚0.2966076135635376 ,&Purity/1FRt4SRqqt4FBt/event_proportion‚0 ,&Purity/3FRt1SRqqt4FBt/event_proportion‚0 +"Purity/0FRt4SRqqt1FBt/SRqqt_purity‚0.25 ;"Purity/*FRt2SRqqt*FBt/event_purity‚0.013820252381265163 7 Purity/0FRt*SRqqt1FBt/FBt_purity‚0.9960033297538757 :"Purity/*FRt*SRqqt2FBt/SRqqt_purity‚0.42012831568717957 @&Purity/0FRt3SRqqt1FBt/event_proportion‚0.0006300403038039804 ("Purity/2FRt3SRqqt0FBt/event_purity‚0 & Purity/3FRt0SRqqt1FBt/FBt_purity‚1 @&Purity/3FRt2SRqqt1FBt/event_proportion‚0.0003780241822823882 *jet/accuracy_1_of_4‚0.8711963295936584 9"Purity/0FRt*SRqqt3FBt/SRqqt_purity‚0.4333333373069763 ("Purity/*FRt3SRqqt2FBt/event_purity‚0 :"Purity/*FRt*SRqqt2FBt/event_purity‚0.17307427525520325 7 Purity/0FRt1SRqqt1FBt/FBt_purity‚0.9942485094070435 particle/accuracy_1_of_6‚1 jet/accuracy_1_of_7‚1 jet/accuracy_4_of_7‚0 ("Purity/1FRt4SRqqt*FBt/event_purity‚0 ("Purity/*FRt1SRqqt4FBt/SRqqt_purity‚1 & Purity/1FRt1SRqqt4FBt/FRt_purity‚1 7 Purity/*FRt3SRqqt*FBt/FBt_purity‚0.9772727489471436 8 Purity/*FRt3SRqqt0FBt/FRt_purity‚0.20790085196495056 ?&Purity/0FRt2SRqqt1FBt/event_proportion‚0.009261592291295528 8 Purity/1FRt*SRqqt*FBt/FRt_purity‚0.22017589211463928 ,&Purity/2FRt2SRqqt4FBt/event_proportion‚0 ,&Purity/2FRt3SRqqt2FBt/event_proportion‚0 7 Purity/3FRt0SRqqt1FBt/FRt_purity‚0.3461538553237915 particle/accuracy_5_of_7‚1 ?&Purity/*FRt3SRqqt1FBt/event_proportion‚0.002142137149348855 ,&Purity/*FRt3SRqqt3FBt/event_proportion‚0 :"Purity/0FRt1SRqqt0FBt/SRqqt_purity‚0.33452585339546204 @&Purity/1FRt3SRqqt2FBt/event_proportion‚6.300403038039804e-05 8"Purity/3FRt*SRqqt0FBt/SRqqt_purity‚0.358956903219223 @&Purity/3FRt3SRqqt0FBt/event_proportion‚0.0005670362734235823 ,&Purity/4FRt1SRqqt3FBt/event_proportion‚0 *"Purity/0FRt1SRqqt3FBt/SRqqt_purity‚0.5 *loss/val_total_loss‚49.371364115333826 ;"Purity/0FRt2SRqqt0FBt/event_purity‚0.040243521332740784 ("Purity/2FRt3SRqqt*FBt/event_purity‚0 7 Purity/3FRt2SRqqt1FBt/FRt_purity‚0.2222222238779068 9"Purity/3FRt3SRqqt0FBt/SRqqt_purity‚0.5476190447807312 9"Purity/*FRt3SRqqt2FBt/SRqqt_purity‚0.3333333432674408 & Purity/2FRt2SRqqt2FBt/FBt_purity‚1 ("Purity/0FRt2SRqqt3FBt/event_purity‚0 @&Purity/0FRt*SRqqt3FBt/event_proportion‚0.0030871976632624865 ,&Purity/3FRt3SRqqt1FBt/event_proportion‚0 _step‚48 :"Purity/1FRt*SRqqt1FBt/SRqqt_purity‚0.41674521565437317 @&Purity/3FRt2SRqqt0FBt/event_proportion‚0.0024571572430431843 +"Purity/*FRt4SRqqt1FBt/SRqqt_purity‚0.25 & Purity/3FRt2SRqqt1FBt/FBt_purity‚1 ("Purity/1FRt1SRqqt4FBt/SRqqt_purity‚1 trainer/global_step‚603 @&Purity/*FRt4SRqqt*FBt/event_proportion‚0.0013230846961960196 ,&Purity/4FRt2SRqqt2FBt/event_proportion‚0 7 Purity/0FRt2SRqqt2FBt/FBt_purity‚0.9807692170143127 & Purity/4FRt0SRqqt0FBt/FRt_purity‚0 ?&Purity/0FRt2SRqqt0FBt/event_proportion‚0.043283771723508835 @&Purity/1FRt*SRqqt3FBt/event_proportion‚0.0013230846961960196 ,&Purity/4FRt2SRqqt1FBt/event_proportion‚0 particle/accuracy_2_of_3‚1 ("Purity/4FRt1SRqqt*FBt/event_purity‚0 epoch‚0 7 Purity/*FRt*SRqqt1FBt/FBt_purity‚0.9955952167510986 ,&Purity/4FRt3SRqqt0FBt/event_proportion‚0 ,&Purity/4FRt3SRqqt4FBt/event_proportion‚0 >&Purity/1FRt2SRqqt0FBt/event_proportion‚0.04731602966785431 ("Purity/2FRt2SRqqt1FBt/event_purity‚0 =&Purity/*FRt*SRqqt*FBt/event_proportion‚0.9591733813285828 7 Purity/*FRt*SRqqt1FBt/FRt_purity‚0.2773731052875519 ;"Purity/*FRt2SRqqt1FBt/event_purity‚0.006182795390486717 7 Purity/1FRt1SRqqt2FBt/FBt_purity‚0.9726190567016602 :"Purity/2FRt0SRqqt2FBt/event_purity‚0.01785714365541935 ,&Purity/3FRt0SRqqt4FBt/event_proportion‚0 ("Purity/*FRt1SRqqt4FBt/event_purity‚0 & Purity/*FRt3SRqqt2FBt/FRt_purity‚1 @&Purity/*FRt2SRqqt3FBt/event_proportion‚6.300403038039804e-05 9"Purity/2FRt1SRqqt2FBt/SRqqt_purity‚0.6710526347160339 @&Purity/3FRt*SRqqt0FBt/event_proportion‚0.0071824598126113415 & Purity/3FRt*SRqqt2FBt/FBt_purity‚1 ("Purity/0FRt1SRqqt3FBt/event_purity‚0 ("Purity/1FRt1SRqqt4FBt/event_purity‚0 9"Purity/*FRt1SRqqt0FBt/SRqqt_purity‚0.3397771418094635 9"Purity/2FRt*SRqqt1FBt/SRqqt_purity‚0.3888493478298187 ("Purity/2FRt1SRqqt2FBt/event_purity‚0 7 Purity/3FRt1SRqqt1FBt/FRt_purity‚0.2820512652397156 :"Purity/3FRt2SRqqt0FBt/SRqqt_purity‚0.38768112659454346 *"Purity/0FRt2SRqqt3FBt/SRqqt_purity‚0.5 ( Purity/*FRt0SRqqt3FBt/FRt_purity‚0.5 :"Purity/0FRt*SRqqt3FBt/event_purity‚0.18209877610206604 ("Purity/3FRt2SRqqt1FBt/event_purity‚0 @&Purity/*FRt*SRqqt4FBt/event_proportion‚0.0003780241822823882 ;"Purity/0FRt2SRqqt1FBt/event_purity‚0.011648745276033878 ,&Purity/0FRt4SRqqt2FBt/event_proportion‚0 9"Purity/1FRt0SRqqt0FBt/event_purity‚0.1496870219707489 9"Purity/1FRt1SRqqt1FBt/SRqqt_purity‚0.4611673355102539 :"Purity/1FRt2SRqqt0FBt/SRqqt_purity‚0.34311607480049133 ?&Purity/2FRt*SRqqt1FBt/event_proportion‚0.021169355139136314 ,&Purity/2FRt4SRqqt4FBt/event_proportion‚0 8 Purity/*FRt2SRqqt*FBt/FRt_purity‚0.20096346735954285 6 Purity/1FRt2SRqqt*FBt/FBt_purity‚0.981566846370697 7 Purity/1FRt2SRqqt0FBt/FRt_purity‚0.1898135393857956 @&Purity/2FRt0SRqqt3FBt/event_proportion‚6.300403038039804e-05 ,&Purity/2FRt0SRqqt4FBt/event_proportion‚0 7 Purity/2FRt3SRqqt*FBt/FRt_purity‚0.1780952513217926 & Purity/3FRt*SRqqt1FBt/FBt_purity‚1 & Purity/3FRt1SRqqt1FBt/FBt_purity‚1 >&Purity/2FRt1SRqqt*FBt/event_proportion‚0.04158266261219978 ,&Purity/2FRt4SRqqt2FBt/event_proportion‚0 ("Purity/*FRt4SRqqt0FBt/event_purity‚0 jet/accuracy_1_of_6‚1 jet/accuracy_3_of_7‚0 particle/accuracy_1_of_7‚1 >&Purity/*FRt0SRqqt1FBt/event_proportion‚0.14591734111309052 ?&Purity/0FRt3SRqqt*FBt/event_proportion‚0.006741431541740894 ("Purity/1FRt2SRqqt*FBt/event_purity‚0 ("Purity/2FRt1SRqqt*FBt/event_purity‚0 9"Purity/2FRt1SRqqt0FBt/SRqqt_purity‚0.3509160280227661 7 Purity/3FRt*SRqqt1FBt/FRt_purity‚0.2904040813446045 A&Purity/3FRt4SRqqt0FBt/event_proportion‚0.00012600806076079607 jet/accuracy_6_of_6‚0 :"Purity/1FRt0SRqqt2FBt/event_purity‚0.07879630476236343 ("Purity/1FRt2SRqqt1FBt/event_purity‚0 ,&Purity/2FRt*SRqqt4FBt/event_proportion‚0 @&Purity/3FRt0SRqqt0FBt/event_proportion‚0.0015120967291295528 & Purity/3FRt1SRqqt2FBt/FBt_purity‚1 ("Purity/1FRt*SRqqt3FBt/SRqqt_purity‚1 9"Purity/2FRt*SRqqt*FBt/SRqqt_purity‚0.3667474091053009 ,&Purity/4FRt1SRqqt2FBt/event_proportion‚0 *jet/accuracy_1_of_5‚0.9197626709938049 particle/accuracy_3_of_6‚1 ("Purity/3FRt0SRqqt2FBt/event_purity‚0 ("Purity/4FRt*SRqqt0FBt/event_purity‚0 7 Purity/*FRt*SRqqt2FBt/FBt_purity‚0.9780328869819641 7 Purity/1FRt1SRqqt1FBt/FRt_purity‚0.2410500943660736 @&Purity/1FRt4SRqqt0FBt/event_proportion‚0.0003780241822823882 ("Purity/3FRt0SRqqt*FBt/event_purity‚0 ("Purity/3FRt1SRqqt*FBt/event_purity‚0 ?&Purity/3FRt1SRqqt*FBt/event_proportion‚0.003717237850651145 & Purity/*FRt4SRqqt*FBt/FBt_purity‚1 6 Purity/*FRt4SRqqt0FBt/FRt_purity‚0.236111119389534 :"Purity/*FRt*SRqqt1FBt/event_purity‚0.38116568326950073 + Purity/1FRt*SRqqt3FBt/FRt_purity‚0.4375 7 Purity/1FRt1SRqqt1FBt/FBt_purity‚0.9949740171432495 *"Purity/3FRt1SRqqt1FBt/SRqqt_purity‚0.5 particle/accuracy_1_of_5‚1 particle/accuracy_5_of_5‚1 +"Purity/3FRt4SRqqt0FBt/SRqqt_purity‚0.25 ,&Purity/0FRt4SRqqt4FBt/event_proportion‚0 ,&Purity/3FRt*SRqqt3FBt/event_proportion‚0 particle/accuracy_4_of_4‚1 jet/accuracy_2_of_7‚0 :"Purity/*FRt1SRqqt1FBt/event_purity‚0.07138144224882126 @&Purity/2FRt*SRqqt3FBt/event_proportion‚6.300403038039804e-05 ?&Purity/2FRt0SRqqt*FBt/event_proportion‚0.037046369165182114 ,&Purity/4FRt4SRqqt4FBt/event_proportion‚0 *"Purity/0FRt*SRqqt4FBt/event_purity‚0.5 7 Purity/1FRt2SRqqt2FBt/FRt_purity‚0.3888889253139496 7 Purity/1FRt4SRqqt*FBt/FRt_purity‚0.1666666716337204 ( Purity/2FRt0SRqqt3FBt/FRt_purity‚0.5 9"Purity/*FRt*SRqqt0FBt/SRqqt_purity‚0.3411872684955597 8 Purity/*FRt0SRqqt*FBt/FRt_purity‚0.21240629255771637 9"Purity/1FRt3SRqqt*FBt/SRqqt_purity‚0.3328213095664978 <"Purity/2FRt*SRqqt0FBt/event_purity‚0.0027786199934780598 ?&Purity/2FRt1SRqqt1FBt/event_proportion‚0.007875503972172737 @&Purity/3FRt0SRqqt*FBt/event_proportion‚0.0027091733645647764 ("Purity/3FRt4SRqqt*FBt/event_purity‚0 7 Purity/0FRt2SRqqt1FBt/FBt_purity‚0.9792627096176147 :"Purity/1FRt*SRqqt0FBt/SRqqt_purity‚0.34288355708122253 8 Purity/1FRt3SRqqt*FBt/FRt_purity‚0.20784690976142883 8 Purity/2FRt0SRqqt*FBt/FRt_purity‚0.20727728307247162 ("Purity/3FRt1SRqqt0FBt/event_purity‚0 @&Purity/4FRt*SRqqt*FBt/event_proportion‚0.0001890120911411941 particle/accuracy_1_of_1‚1 *jet/accuracy_3_of_4‚0.0471128448843956 particle/accuracy_2_of_7‚1 & Purity/3FRt0SRqqt2FBt/FBt_purity‚1 >&Purity/*FRt2SRqqt0FBt/event_proportion‚0.10855595022439957 7 Purity/0FRt1SRqqt2FBt/FBt_purity‚0.9621295928955078 ("Purity/1FRt*SRqqt3FBt/event_purity‚0 7 Purity/2FRt1SRqqt*FBt/FRt_purity‚0.2224024534225464 9"Purity/2FRt2SRqqt*FBt/SRqqt_purity‚0.3756571114063263 7 Purity/2FRt2SRqqt1FBt/FBt_purity‚0.9821428656578064 ("Purity/3FRt0SRqqt1FBt/event_purity‚0 A&Purity/3FRt4SRqqt*FBt/event_proportion‚0.00012600806076079607 =&Purity/0FRt1SRqqt*FBt/event_proportion‚0.2305317521095276 ?&Purity/2FRt3SRqqt*FBt/event_proportion‚0.004032257944345474 *jet/accuracy_2_of_4‚0.2931746244430542 & Purity/0FRt4SRqqt*FBt/FBt_purity‚1 jet/accuracy_6_of_7‚0 & Purity/4FRt*SRqqt0FBt/FRt_purity‚0 ;"Purity/1FRt1SRqqt*FBt/event_purity‚0.010837228037416935 7 Purity/2FRt2SRqqt1FBt/FRt_purity‚0.2395833283662796 :"Purity/0FRt1SRqqt1FBt/SRqqt_purity‚0.42248860001564026 ("Purity/*FRt3SRqqt0FBt/event_purity‚0 ,jet/accuracy_3_of_3‚0.014111964032053947 9"Purity/0FRt4SRqqt0FBt/SRqqt_purity‚0.1428571492433548 ( Purity/*FRt*SRqqt4FBt/FRt_purity‚0.5 @&Purity/1FRt0SRqqt4FBt/event_proportion‚6.300403038039804e-05 8 Purity/2FRt1SRqqt1FBt/FRt_purity‚0.27142855525016785 ("Purity/4FRt0SRqqt*FBt/event_purity‚0 @&Purity/*FRt0SRqqt4FBt/event_proportion‚0.0003150201519019902 9"Purity/0FRt0SRqqt3FBt/event_purity‚0.2433333396911621 7 Purity/1FRt*SRqqt2FBt/FBt_purity‚0.9775258302688599 7 Purity/1FRt0SRqqt3FBt/FBt_purity‚0.9285714030265808 7 Purity/2FRt*SRqqt*FBt/FBt_purity‚0.9907397627830505 8 Purity/2FRt*SRqqt0FBt/FRt_purity‚0.18108418583869934 8 Purity/*FRt*SRqqt3FBt/FRt_purity‚0.44117647409439087 =&Purity/*FRt1SRqqt1FBt/event_proportion‚0.1001134067773819 7 Purity/0FRt*SRqqt3FBt/FBt_purity‚0.8971194624900818 & Purity/2FRt1SRqqt1FBt/FBt_purity‚1 6 Purity/2FRt2SRqqt*FBt/FRt_purity‚0.205313041806221 ("Purity/2FRt*SRqqt3FBt/event_purity‚0 ("Purity/*FRt*SRqqt4FBt/SRqqt_purity‚1 7 Purity/*FRt1SRqqt*FBt/FBt_purity‚0.9864656925201416 8 Purity/*FRt1SRqqt0FBt/FRt_purity‚0.21613827347755432 9"Purity/*FRt2SRqqt*FBt/SRqqt_purity‚0.3504430651664734 :"Purity/0FRt*SRqqt0FBt/SRqqt_purity‚0.33446353673934937 >&Purity/1FRt*SRqqt1FBt/event_proportion‚0.09929435700178146 7 Purity/1FRt0SRqqt2FBt/FRt_purity‚0.4749508798122406 :"Purity/2FRt3SRqqt1FBt/SRqqt_purity‚0.30000001192092896 9"Purity/0FRt*SRqqt*FBt/event_purity‚0.3947468101978302 & Purity/1FRt3SRqqt1FBt/FBt_purity‚1 ;"Purity/2FRt0SRqqt0FBt/event_purity‚0.009447004646062851 & Purity/2FRt0SRqqt1FBt/FBt_purity‚1 @&Purity/2FRt2SRqqt2FBt/event_proportion‚6.300403038039804e-05 ("Purity/0FRt3SRqqt0FBt/event_purity‚0 ("Purity/0FRt4SRqqt1FBt/event_purity‚0 ("Purity/4FRt1SRqqt0FBt/SRqqt_purity‚1 7 Purity/*FRt0SRqqt1FBt/FBt_purity‚0.9983914494514465 ?&Purity/*FRt1SRqqt2FBt/event_proportion‚0.017011089250445366 ,&Purity/*FRt4SRqqt4FBt/event_proportion‚0 >&Purity/1FRt2SRqqt*FBt/event_proportion‚0.05727066472172737 7 Purity/3FRt4SRqqt0FBt/FRt_purity‚0.1666666716337204 8"Purity/*FRt*SRqqt3FBt/SRqqt_purity‚0.714285671710968 7 Purity/1FRt3SRqqt1FBt/FRt_purity‚0.2888889014720917 8 Purity/2FRt0SRqqt1FBt/FRt_purity‚0.29182541370391846 ) Purity/*FRt1SRqqt4FBt/FBt_purity‚0.75 & Purity/4FRt2SRqqt*FBt/FRt_purity‚0 8 Purity/*FRt0SRqqt1FBt/FRt_purity‚0.30453720688819885 >&Purity/0FRt*SRqqt2FBt/event_proportion‚0.03944052383303642 7 Purity/0FRt0SRqqt3FBt/FBt_purity‚0.9066668152809143 @&Purity/4FRt2SRqqt0FBt/event_proportion‚6.300403038039804e-05 ,jet/accuracy_4_of_4‚0.004535453859716654 ?&Purity/*FRt3SRqqt*FBt/event_proportion‚0.020476309582591057 9"Purity/0FRt1SRqqt*FBt/SRqqt_purity‚0.3565128445625305 7 Purity/1FRt1SRqqt*FBt/FRt_purity‚0.2348003387451172 :"Purity/1FRt1SRqqt0FBt/SRqqt_purity‚0.34526321291923523 7 Purity/2FRt*SRqqt2FBt/FRt_purity‚0.4102778136730194 ?&Purity/2FRt0SRqqt2FBt/event_proportion‚0.003465221729129553 ("Purity/2FRt1SRqqt1FBt/event_purity‚0 9"Purity/*FRt*SRqqt4FBt/event_purity‚0.3333333432674408 ("Purity/*FRt3SRqqt*FBt/event_purity‚0 7 Purity/0FRt*SRqqt*FBt/FBt_purity‚0.9850422739982605 9"Purity/0FRt*SRqqt1FBt/event_purity‚0.6256648898124695 @&Purity/1FRt4SRqqt*FBt/event_proportion‚0.0003780241822823882 ,&Purity/1FRt4SRqqt1FBt/event_proportion‚0 ?&Purity/2FRt0SRqqt1FBt/event_proportion‚0.008946572430431843 ,&Purity/4FRt0SRqqt2FBt/event_proportion‚0 * Purity/0FRt0SRqqt4FBt/FBt_purity‚0.875 8 Purity/*FRt1SRqqt2FBt/FRt_purity‚0.35599538683891296 7 Purity/1FRt0SRqqt*FBt/FBt_purity‚0.9877117872238159 7 Purity/1FRt0SRqqt0FBt/FRt_purity‚0.1496870219707489 ,&Purity/4FRt4SRqqt2FBt/event_proportion‚0 particle/accuracy_6_of_7‚1 & Purity/2FRt*SRqqt3FBt/FBt_purity‚1 :"Purity/*FRt*SRqqt3FBt/event_purity‚0.10632183402776718 ,&Purity/2FRt1SRqqt4FBt/event_proportion‚0 ,&Purity/4FRt1SRqqt1FBt/event_proportion‚0 ( Purity/1FRt*SRqqt4FBt/FRt_purity‚0.5 5 Purity/*FRt1SRqqt3FBt/FBt_purity‚0.89682537317276 7 Purity/3FRt3SRqqt0FBt/FRt_purity‚0.2142857164144516 7 Purity/0FRt0SRqqt*FBt/FBt_purity‚0.9858734607696533 ,&Purity/1FRt2SRqqt4FBt/event_proportion‚0 >&Purity/2FRt*SRqqt*FBt/event_proportion‚0.10244455933570862 * Purity/0FRt*SRqqt4FBt/FBt_purity‚0.875 8 Purity/*FRt2SRqqt1FBt/FRt_purity‚0.25665760040283203 @&Purity/2FRt3SRqqt1FBt/event_proportion‚0.0003780241822823882 ,&Purity/3FRt0SRqqt3FBt/event_proportion‚0 +jet/accuracy_2_of_3‚0.20174075663089752 8 Purity/2FRt3SRqqt1FBt/FRt_purity‚0.10000000149011612 9"Purity/3FRt3SRqqt*FBt/SRqqt_purity‚0.5476190447807312 ,&Purity/2FRt2SRqqt3FBt/event_proportion‚0 ,&Purity/3FRt*SRqqt4FBt/event_proportion‚0 ,&Purity/4FRt3SRqqt2FBt/event_proportion‚0 7 Purity/*FRt2SRqqt1FBt/FBt_purity‚0.9828523993492126 =&Purity/*FRt1SRqqt*FBt/event_proportion‚0.4482736885547638 ;"Purity/2FRt0SRqqt*FBt/event_purity‚0.011417882516980171 8 Purity/3FRt*SRqqt*FBt/FRt_purity‚0.21296724677085876 9"Purity/3FRt*SRqqt1FBt/SRqqt_purity‚0.4777778089046478 ,&Purity/4FRt4SRqqt1FBt/event_proportion‚0 & Purity/2FRt3SRqqt1FBt/FBt_purity‚1 & Purity/1FRt0SRqqt4FBt/FBt_purity‚1 :"Purity/1FRt*SRqqt*FBt/event_purity‚0.07448069751262665 6 Purity/3FRt1SRqqt*FBt/FRt_purity‚0.194444477558136 (particle/accuracy‚0.8436554074287415 +"Purity/2FRt4SRqqt0FBt/SRqqt_purity‚0.25 *jet/accuracy_2_of_6‚0.4298245906829834 ("Purity/0FRt3SRqqt1FBt/event_purity‚0 ("Purity/4FRt*SRqqt*FBt/event_purity‚0 :"Purity/0FRt0SRqqt2FBt/event_purity‚0.32851898670196533 ("Purity/1FRt1SRqqt3FBt/event_purity‚0 ?&Purity/2FRt0SRqqt0FBt/event_proportion‚0.024571573361754417 @&Purity/2FRt4SRqqt0FBt/event_proportion‚0.0003150201519019902 /"Purity/*FRt4SRqqt*FBt/SRqqt_purity‚0.265625 8 Purity/2FRt4SRqqt*FBt/FRt_purity‚0.30000001192092896 :"Purity/0FRt2SRqqt*FBt/event_purity‚0.03395435959100723 @&Purity/1FRt3SRqqt1FBt/event_proportion‚0.0011340725468471646 particle/accuracy_3_of_5‚1 7 Purity/*FRt2SRqqt2FBt/FBt_purity‚0.9912281036376953 & Purity/*FRt0SRqqt4FBt/FRt_purity‚0 :"Purity/*FRt0SRqqt*FBt/event_purity‚0.35872897505760193 :"Purity/1FRt3SRqqt0FBt/SRqqt_purity‚0.32388633489608765 particle/accuracy_1_of_4‚1 particle/accuracy_4_of_7‚1 ("Purity/2FRt2SRqqt2FBt/SRqqt_purity‚0 ?&Purity/0FRt3SRqqt0FBt/event_proportion‚0.006111391354352236 ("Purity/1FRt0SRqqt3FBt/event_purity‚0 particle/accuracy_2_of_5‚1 ("Purity/2FRt2SRqqt2FBt/event_purity‚0 5 Purity/*FRt*SRqqt*FBt/FRt_purity‚0.21834696829319 ?&Purity/1FRt0SRqqt2FBt/event_proportion‚0.015120967291295528 @&Purity/*FRt1SRqqt3FBt/event_proportion‚0.0006930443341843784 ("Purity/3FRt*SRqqt*FBt/event_purity‚0 & Purity/4FRt1SRqqt*FBt/FRt_purity‚0 ("Purity/4FRt2SRqqt0FBt/SRqqt_purity‚1 A&Purity/0FRt4SRqqt0FBt/event_proportion‚0.00044102821266278625 :"Purity/1FRt0SRqqt*FBt/event_purity‚0.15538960695266724 7 Purity/2FRt0SRqqt2FBt/FBt_purity‚0.9940475821495056 ?&Purity/2FRt1SRqqt2FBt/event_proportion‚0.001890120911411941 9"Purity/3FRt2SRqqt*FBt/SRqqt_purity‚0.3836805522441864 +"Purity/2FRt4SRqqt*FBt/SRqqt_purity‚0.25 & Purity/1FRt3SRqqt2FBt/FBt_purity‚1 9"Purity/*FRt*SRqqt*FBt/SRqqt_purity‚0.3560960292816162 7 Purity/*FRt0SRqqt2FBt/FBt_purity‚0.9806344509124756 >&Purity/0FRt0SRqqt*FBt/event_proportion‚0.11857359111309052 >&Purity/0FRt0SRqqt1FBt/event_proportion‚0.08606351166963577 ,&Purity/1FRt3SRqqt3FBt/event_proportion‚0 8 Purity/3FRt2SRqqt*FBt/FRt_purity‚0.19560183584690094 ,&Purity/3FRt3SRqqt4FBt/event_proportion‚¤_“$A0 ,&Purity/4FRt*SRqqt1FBt/event_proportion‚0 ("Purity/0FRt4SRqqt*FBt/event_purity‚0 :"Purity/*FRt3SRqqt1FBt/SRqqt_purity‚0.33068785071372986 @&Purity/0FRt2SRqqt3FBt/event_proportion‚6.300403038039804e-05 @&Purity/4FRt0SRqqt0FBt/event_proportion‚6.300403038039804e-05 particle/accuracy_1_of_3‚1 _runtime‚1095.020157554 ) Purity/1FRt1SRqqt3FBt/FRt_purity‚0.25 9"Purity/1FRt3SRqqt2FBt/SRqqt_purity‚0.3333333432674408 9"Purity/*FRt1SRqqt*FBt/SRqqt_purity‚0.3666665852069855 7 Purity/1FRt*SRqqt1FBt/FBt_purity‚0.9940850138664246 jet/accuracy_5_of_6‚0 ("Purity/1FRt3SRqqt2FBt/event_purity‚0 ("Purity/4FRt2SRqqt*FBt/SRqqt_purity‚1 9"Purity/*FRt2SRqqt0FBt/SRqqt_purity‚0.3505835235118866 ("Purity/0FRt0SRqqt0FBt/event_purity‚1 ,&Purity/0FRt3SRqqt2FBt/event_proportion‚0 ;"Purity/2FRt*SRqqt2FBt/event_purity‚0.011111111380159855 7 Purity/2FRt1SRqqt2FBt/FBt_purity‚0.9868420958518982 :"Purity/2FRt2SRqqt0FBt/SRqqt_purity‚0.38297781348228455 ,&Purity/4FRt2SRqqt3FBt/event_proportion‚0 particle/accuracy_3_of_4‚1 8 Purity/1FRt*SRqqt1FBt/FRt_purity‚0.26739901304244995 9"Purity/1FRt0SRqqt1FBt/event_purity‚0.2009960412979126 @&Purity/1FRt0SRqqt3FBt/event_proportion‚0.0010710685746744275 ("Purity/1FRt3SRqqt1FBt/event_purity‚0 9"Purity/2FRt*SRqqt2FBt/SRqqt_purity‚0.6359648704528809 7 Purity/3FRt0SRqqt*FBt/FRt_purity‚0.2196969836950302 ,&Purity/4FRt3SRqqt3FBt/event_proportion‚0 'particle/f_score‚0.3976779878139496 ?&Purity/*FRt*SRqqt3FBt/event_proportion‚0.004473286215215921 @&Purity/3FRt*SRqqt2FBt/event_proportion‚0.0003150201519019902 ("Purity/1FRt4SRqqt0FBt/event_purity‚0 ("Purity/1FRt1SRqqt3FBt/SRqqt_purity‚1 7 Purity/1FRt1SRqqt*FBt/FBt_purity‚0.9850632548332214 8 Purity/1FRt1SRqqt0FBt/FRt_purity‚0.22795183956623077 ("Purity/2FRt2SRqqt*FBt/event_purity‚0 particle/accuracy_4_of_5‚1 ("Purity/0FRt3SRqqt*FBt/event_purity‚0 particle/accuracy_4_of_6‚1 * Purity/0FRt3SRqqt1FBt/FBt_purity‚0.875 =&Purity/*FRt0SRqqt2FBt/event_proportion‚0.0483240932226181 A&Purity/1FRt1SRqqt3FBt/event_proportion‚0.00025201612152159214 @&Purity/4FRt2SRqqt*FBt/event_proportion‚6.300403038039804e-05 9"Purity/0FRt3SRqqt0FBt/SRqqt_purity‚0.3204365074634552 particle/accuracy_2_of_6‚1 7 Purity/3FRt3SRqqt*FBt/FRt_purity‚0.2142857164144516 ("Purity/4FRt1SRqqt0FBt/event_purity‚0 ( Purity/2FRt2SRqqt2FBt/FRt_purity‚0.5 :"Purity/*FRt1SRqqt1FBt/SRqqt_purity‚0.43834149837493896 =&Purity/1FRt*SRqqt0FBt/event_proportion‚0.3135710656642914 6 Purity/1FRt0SRqqt*FBt/FRt_purity‚0.211908221244812 9"Purity/1FRt1SRqqt*FBt/SRqqt_purity‚0.3763994574546814 ?&Purity/2FRt2SRqqt0FBt/event_proportion‚0.015435988083481789 & Purity/*FRt4SRqqt1FBt/FBt_purity‚1 *"Purity/*FRt2SRqqt3FBt/SRqqt_purity‚0.5 =&Purity/1FRt1SRqqt*FBt/event_proportion‚0.1723790317773819 8 Purity/1FRt2SRqqt1FBt/FRt_purity‚0.23348693549633026 particle/accuracy_6_of_6‚1 ("Purity/*FRt1SRqqt3FBt/event_purity‚0 ) Purity/1FRt1SRqqt4FBt/FBt_purity‚0.75 9"Purity/0FRt*SRqqt0FBt/event_purity‚0.2664025127887726 :"Purity/0FRt1SRqqt1FBt/event_purity‚0.12602807581424713 ?&Purity/1FRt0SRqqt1FBt/event_proportion‚0.049773186445236206 @&Purity/4FRt1SRqqt0FBt/event_proportion‚6.300403038039804e-05 *jet/accuracy_3_of_5‚0.0911157876253128 ("Purity/2FRt0SRqqt3FBt/event_purity‚0 7 Purity/1FRt2SRqqt1FBt/FBt_purity‚0.9806451201438904 ("Purity/1FRt3SRqqt*FBt/event_purity‚0 ?&Purity/1FRt3SRqqt*FBt/event_proportion‚0.009135584346950054 & Purity/3FRt*SRqqt*FBt/FBt_purity‚1 7 Purity/3FRt*SRqqt0FBt/FRt_purity‚0.1829100400209427 +"Purity/3FRt4SRqqt*FBt/SRqqt_purity‚0.25 7 Purity/0FRt2SRqqt3FBt/FBt_purity‚0.6666666865348816 ,&Purity/2FRt1SRqqt3FBt/event_proportion‚0 ?&Purity/2FRt2SRqqt*FBt/event_proportion‚0.019468246027827263 ,&Purity/4FRt*SRqqt3FBt/event_proportion‚0 particle/accuracy_3_of_3‚1 9"Purity/*FRt1SRqqt2FBt/SRqqt_purity‚0.4528195559978485 8 Purity/1FRt1SRqqt2FBt/FRt_purity‚0.38222792744636536 ;"Purity/2FRt*SRqqt*FBt/event_purity‚0.003873026231303811 8 Purity/2FRt*SRqqt*FBt/FRt_purity‚0.21636322140693665 ,&Purity/3FRt4SRqqt2FBt/event_proportion‚0 ,&Purity/3FRt4SRqqt4FBt/event_proportion‚0 & Purity/*FRt3SRqqt2FBt/FBt_purity‚1 # _timestamp‚1.78848629591639e+09 :"Purity/*FRt1SRqqt2FBt/event_purity‚0.03143997862935066 :"Purity/1FRt2SRqqt1FBt/SRqqt_purity‚0.32803377509117126 ,&Purity/1FRt3SRqqt4FBt/event_proportion‚0 :"Purity/3FRt1SRqqt*FBt/SRqqt_purity‚0.39814817905426025 ,&Purity/4FRt4SRqqt*FBt/event_proportion‚0 8 Purity/*FRt1SRqqt*FBt/FRt_purity‚0.23056207597255707 >&Purity/0FRt1SRqqt1FBt/event_proportion‚0.05185231938958168 @&Purity/0FRt4SRqqt1FBt/event_proportion‚6.300403038039804e-05 ?&Purity/1FRt*SRqqt2FBt/event_proportion‚0.022303426638245583 *jet/accuracy_1_of_2‚0.6902841329574585 +"Purity/*FRt2SRqqt2FBt/SRqqt_purity‚0.25 9"Purity/*FRt1SRqqt3FBt/SRqqt_purity‚0.7619047164916992 7 Purity/1FRt1SRqqt3FBt/FBt_purity‚0.8333333730697632 7 Purity/2FRt2SRqqt*FBt/FBt_purity‚0.9821428656578064 8 Purity/2FRt2SRqqt0FBt/FRt_purity‚0.17782025039196014 ,&Purity/3FRt1SRqqt3FBt/event_proportion‚0 @&Purity/4FRt*SRqqt0FBt/event_proportion‚0.0001890120911411941 ,&Purity/4FRt2SRqqt4FBt/event_proportion‚0 & Purity/0FRt4SRqqt1FBt/FBt_purity‚1 ("Purity/*FRt2SRqqt2FBt/event_purity‚0 :"Purity/*FRt*SRqqt1FBt/SRqqt_purity‚0.40597864985466003 @&Purity/4FRt1SRqqt*FBt/event_proportion‚6.300403038039804e-05 7 Purity/3FRt4SRqqt*FBt/FRt_purity‚0.1666666716337204 ("Purity/3FRt1SRqqt2FBt/event_purity‚0 ("Purity/*FRt2SRqqt3FBt/event_purity‚0 @&Purity/*FRt4SRqqt0FBt/event_proportion‚0.0012600806076079607 >&Purity/0FRt2SRqqt*FBt/event_proportion‚0.05361643061041832 7 Purity/0FRt2SRqqt*FBt/FBt_purity‚0.9685100317001343 ("Purity/3FRt1SRqqt1FBt/event_purity‚0 *"Purity/0FRt0SRqqt4FBt/event_purity‚0.5 *"Purity/3FRt1SRqqt2FBt/SRqqt_purity‚0.5 >&Purity/*FRt0SRqqt0FBt/event_proportion‚0.15757308900356293 :"Purity/0FRt1SRqqt2FBt/SRqqt_purity‚0.38585978746414185 ;"Purity/1FRt1SRqqt0FBt/event_purity‚0.008950828574597836 :"Purity/1FRt2SRqqt*FBt/SRqqt_purity‚0.34632670879364014 9"Purity/2FRt1SRqqt*FBt/SRqqt_purity‚0.3825775980949402 @&Purity/3FRt0SRqqt1FBt/event_proportion‚0.0011340725468471646 ,&Purity/4FRt*SRqqt2FBt/event_proportion‚0 7validation_average_jet_accuracy‚0.37975460290908813 ,&Purity/*FRt3SRqqt4FBt/event_proportion‚0 =&Purity/*FRt0SRqqt*FBt/event_proportion‚0.3558467626571655 ;"Purity/0FRt1SRqqt2FBt/event_purity‚0.051428571343421936 ?&Purity/3FRt*SRqqt*FBt/event_proportion‚0.009954636916518211 8 Purity/3FRt*SRqqt2FBt/FRt_purity‚0.20000000298023224 ("Purity/4FRt0SRqqt0FBt/event_purity‚0 & Purity/4FRt2SRqqt0FBt/FRt_purity‚0 ;"Purity/2FRt0SRqqt1FBt/event_purity‚0.009259259328246117 9"Purity/2FRt2SRqqt1FBt/SRqqt_purity‚0.3538690507411957 @&Purity/3FRt0SRqqt2FBt/event_proportion‚6.300403038039804e-05 8 Purity/3FRt2SRqqt0FBt/FRt_purity‚0.18719805777072906 ,&Purity/4FRt4SRqqt0FBt/event_proportion‚0 & Purity/3FRt2SRqqt*FBt/FBt_purity‚1 7 Purity/3FRt1SRqqt2FBt/FRt_purity‚0.1666666716337204 :"Purity/*FRt2SRqqt1FBt/SRqqt_purity‚0.35507386922836304 >&Purity/2FRt*SRqqt0FBt/event_proportion‚0.07579384744167328 ,&Purity/4FRt*SRqqt4FBt/event_proportion‚0 ("Purity/4FRt2SRqqt0FBt/event_purity‚0 :"Purity/1FRt*SRqqt2FBt/event_purity‚0.05309804528951645 ("Purity/1FRt3SRqqt0FBt/event_purity‚0 ?&Purity/2FRt2SRqqt1FBt/event_proportion‚0.003969253972172737 +jet/accuracy_2_of_5‚0.44650667905807495 particle/accuracy_2_of_4‚1 ("Purity/1FRt2SRqqt2FBt/event_purity‚0 9"Purity/1FRt*SRqqt2FBt/SRqqt_purity‚0.3604966700077057 particle/accuracy_1_of_2‚1 9"Purity/1FRt2SRqqt2FBt/SRqqt_purity‚0.2986111342906952 7 Purity/*FRt*SRqqt3FBt/FBt_purity‚0.9028735756874084 6 Purity/*FRt3SRqqt1FBt/FBt_purity‚0.976190447807312 9"Purity/3FRt2SRqqt1FBt/SRqqt_purity‚0.3333333432674408 particle/accuracy_7_of_7‚1 ("Purity/4FRt1SRqqt*FBt/SRqqt_purity‚1 =&Purity/*FRt2SRqqt*FBt/event_proportion‚0.1332535296678543 A&Purity/0FRt*SRqqt4FBt/event_proportion‚0.00025201612152159214 9"Purity/0FRt2SRqqt1FBt/SRqqt_purity‚0.3843318223953247 ,&Purity/0FRt4SRqqt3FBt/event_proportion‚0 :"Purity/1FRt1SRqqt1FBt/event_purity‚0.01746870018541813 ,&Purity/4FRt1SRqqt4FBt/event_proportion‚0 >&Purity/*FRt*SRqqt2FBt/event_proportion‚0.06747731566429138 8 Purity/*FRt*SRqqt2FBt/FRt_purity‚0.41108572483062744 :"Purity/*FRt0SRqqt0FBt/event_purity‚0.12620827555656433 8 Purity/*FRt3SRqqt1FBt/FRt_purity‚0.24444444477558136 @&Purity/1FRt2SRqqt2FBt/event_proportion‚0.0010080644860863686 & Purity/3FRt0SRqqt*FBt/FBt_purity‚1 8 Purity/3FRt0SRqqt0FBt/FRt_purity‚0.09803921729326248 ("Purity/2FRt4SRqqt0FBt/event_purity‚0 7 Purity/0FRt1SRqqt*FBt/FBt_purity‚0.9857016205787659 ( Purity/1FRt0SRqqt3FBt/FRt_purity‚0.5 ?&Purity/1FRt3SRqqt0FBt/event_proportion‚0.007938507944345474 ;"Purity/2FRt*SRqqt1FBt/event_purity‚0.004385964944958687 ("Purity/1FRt*SRqqt4FBt/event_purity‚0 :"Purity/*FRt2SRqqt0FBt/event_purity‚0.01597520150244236 ?&Purity/2FRt*SRqqt2FBt/event_proportion‚0.005418346729129553 ("Purity/2FRt2SRqqt0FBt/event_purity‚0 9"Purity/2FRt3SRqqt*FBt/SRqqt_purity‚0.2994285821914673 ,&Purity/2FRt3SRqqt4FBt/event_proportion‚0 ,&Purity/*FRt4SRqqt3FBt/event_proportion‚0 & Purity/3FRt1SRqqt*FBt/FBt_purity‚1 8 Purity/3FRt1SRqqt0FBt/FRt_purity‚0.16111113131046295 ,jet/accuracy_3_of_6‚0.061403512954711914 9"Purity/1FRt4SRqqt0FBt/SRqqt_purity‚0.4166666567325592 >&Purity/0FRt*SRqqt0FBt/event_proportion‚0.21931703388690948 @&Purity/2FRt4SRqqt*FBt/event_proportion‚0.0003150201519019902 ,&Purity/3FRt2SRqqt2FBt/event_proportion‚0 * Purity/*FRt*SRqqt4FBt/FBt_purity‚0.875 & Purity/1FRt2SRqqt2FBt/FBt_purity‚1 =&Purity/0FRt1SRqqt0FBt/event_proportion‚0.1694808453321457 8 Purity/2FRt4SRqqt0FBt/FRt_purity‚0.30000001192092896 9"Purity/0FRt0SRqqt*FBt/event_purity‚0.8139540553092957 ?&Purity/0FRt0SRqqt2FBt/event_proportion‚0.029674898833036423 A&Purity/0FRt1SRqqt3FBt/event_proportion‚0.00044102821266278625 :"Purity/1FRt*SRqqt1FBt/event_purity‚0.10717689245939255 ,&Purity/2FRt4SRqqt1FBt/event_proportion‚0 ("Purity/*FRt4SRqqt1FBt/event_purity‚0 ( Purity/2FRt*SRqqt3FBt/FRt_purity‚0.5 7 Purity/0FRt1SRqqt3FBt/FBt_purity‚0.8999999761581421 >&Purity/0FRt*SRqqt*FBt/event_proportion‚0.40996724367141724 ,&Purity/4FRt0SRqqt1FBt/event_proportion‚0 ,&Purity/4FRt3SRqqt1FBt/event_proportion‚0 +particle/sensitivity‚0.3305012285709381 9"Purity/1FRt4SRqqt*FBt/SRqqt_purity‚0.4166666567325592 ("Purity/1FRt*SRqqt4FBt/SRqqt_purity‚1 =&Purity/*FRt*SRqqt0FBt/event_proportion‚0.6160534024238586 9"Purity/0FRt*SRqqt2FBt/event_purity‚0.2624504864215851 ) Purity/*FRt1SRqqt3FBt/FRt_purity‚0.25 & Purity/4FRt1SRqqt0FBt/FRt_purity‚0 ?&Purity/2FRt3SRqqt0FBt/event_proportion‚0.003654233878478408 9"Purity/0FRt*SRqqt2FBt/SRqqt_purity‚0.3739814758300781 >&Purity/*FRt3SRqqt0FBt/event_proportion‚0.01827116869390011 @&Purity/*FRt4SRqqt1FBt/event_proportion‚6.300403038039804e-05 7 Purity/*FRt*SRqqt*FBt/FBt_purity‚0.9864770770072937 8 Purity/*FRt*SRqqt0FBt/FRt_purity‚0.18591664731502533 >&Purity/*FRt1SRqqt0FBt/event_proportion‚0.33039313554763794 7 Purity/3FRt0SRqqt2FBt/FRt_purity‚0.3333333432674408 7 Purity/1FRt*SRqqt3FBt/FBt_purity‚0.9270833134651184 7 Purity/1FRt0SRqqt2FBt/FBt_purity‚0.9801872372627258 +jet/accuracy_2_of_2‚0.08364889025688171 ("Purity/1FRt0SRqqt4FBt/event_purity‚0 9"Purity/*FRt0SRqqt2FBt/event_purity‚0.2297607809305191 :"Purity/*FRt1SRqqt*FBt/event_purity‚0.14601264894008636 ("Purity/*FRt3SRqqt1FBt/event_purity‚0 ,&Purity/1FRt4SRqqt3FBt/event_proportion‚0 jet/accuracy_4_of_6‚0 ("Purity/4FRt2SRqqt*FBt/event_purity‚0 9"Purity/*FRt0SRqqt1FBt/event_purity‚0.6592440009117126 :"Purity/0FRt2SRqqt0FBt/SRqqt_purity‚0.34718528389930725 @&Purity/0FRt2SRqqt2FBt/event_proportion‚0.0010080644860863686 7 Purity/1FRt*SRqqt2FBt/FRt_purity‚0.4516954720020294 ?&Purity/1FRt1SRqqt2FBt/event_proportion‚0.006111391354352236 ,&Purity/2FRt4SRqqt3FBt/event_proportion‚0 :"Purity/0FRt3SRqqt*FBt/SRqqt_purity‚0.31226375699043274 ,&Purity/*FRt2SRqqt4FBt/event_proportion‚0 @&Purity/*FRt3SRqqt2FBt/event_proportion‚6.300403038039804e-05 7 Purity/0FRt*SRqqt2FBt/FBt_purity‚0.9759090542793274 @&Purity/3FRt1SRqqt0FBt/event_proportion‚0.0025201612152159214 jet/accuracy_7_of_7‚0 & Purity/*FRt1SRqqt4FBt/FRt_purity‚1 A&Purity/1FRt*SRqqt4FBt/event_proportion‚0.00012600806076079607 ,&Purity/1FRt2SRqqt3FBt/event_proportion‚0 @&Purity/3FRt1SRqqt1FBt/event_proportion‚0.0009450604557059705 ("Purity/3FRt2SRqqt*FBt/event_purity‚0 ("Purity/2FRt4SRqqt*FBt/event_purity‚0 :"Purity/*FRt*SRqqt*FBt/event_purity‚0.20322614908218384 :"Purity/0FRt1SRqqt0FBt/event_purity‚0.33452585339546204 8"Purity/1FRt3SRqqt1FBt/SRqqt_purity‚0.307407408952713 ("Purity/3FRt*SRqqt0FBt/event_purity‚0 ("Purity/3FRt3SRqqt0FBt/event_purity‚0 7 Purity/0FRt0SRqqt2FBt/FBt_purity‚0.9796414971351624 ,&Purity/0FRt1SRqqt4FBt/event_proportion‚0 @&Purity/0FRt4SRqqt*FBt/event_proportion‚0.0005040322430431843 8"Purity/2FRt3SRqqt0FBt/SRqqt_purity‚0.283189058303833 ."Purity/0FRt4SRqqt*FBt/SRqqt_purity‚0.15625 7 Purity/1FRt4SRqqt0FBt/FRt_purity‚0.1666666716337204 & Purity/4FRt*SRqqt*FBt/FRt_purity‚0 ,&Purity/0FRt3SRqqt4FBt/event_proportion‚0 @&Purity/3FRt2SRqqt*FBt/event_proportion‚0.0028351815417408943 ,&Purity/3FRt2SRqqt3FBt/event_proportion‚0 9"Purity/*FRt0SRqqt4FBt/event_purity‚0.4000000059604645 >&Purity/0FRt0SRqqt0FBt/event_proportion‚0.04082661122083664 8"Purity/0FRt0SRqqt1FBt/event_purity‚0.999193549156189 =&Purity/1FRt*SRqqt*FBt/event_proportion‚0.4366179406642914 >&Purity/1FRt0SRqqt0FBt/event_proportion‚0.13142640888690948 * Purity/0FRt3SRqqt*FBt/FBt_purity‚0.875 7 Purity/*FRt1SRqqt2FBt/FBt_purity‚0.9684198498725891 ("Purity/1FRt2SRqqt0FBt/event_purity‚0 'jet/accuracy_1_of_1‚0.4051533639431 @&Purity/3FRt*SRqqt1FBt/event_proportion‚0.0024571572430431843 9"Purity/0FRt3SRqqt1FBt/SRqqt_purity‚0.2083333432674408 & Purity/1FRt0SRqqt4FBt/FRt_purity‚0 9"Purity/*FRt1SRqqt0FBt/event_purity‚0.1748521625995636 8 Purity/*FRt3SRqqt*FBt/FRt_purity‚0.18883544206619263 7 Purity/1FRt*SRqqt*FBt/FBt_purity‚0.9865187406539917 8 Purity/1FRt*SRqqt0FBt/FRt_purity‚0.18954649567604065 7 Purity/2FRt*SRqqt2FBt/FBt_purity‚0.9911110997200012 9"Purity/3FRt*SRqqt*FBt/SRqqt_purity‚0.3652161955833435 ("Purity/3FRt2SRqqt0FBt/event_purity‚0 ,&Purity/3FRt3SRqqt2FBt/event_proportion‚0 @&Purity/*FRt1SRqqt4FBt/event_proportion‚6.300403038039804e-05 >&Purity/0FRt*SRqqt1FBt/event_proportion‚0.14787046611309052 9"Purity/1FRt*SRqqt*FBt/SRqqt_purity‚0.3603418469429016 ,&Purity/4FRt3SRqqt*FBt/event_proportion‚0 +particle/specificity‚0.9386556148529053 :"Purity/0FRt2SRqqt2FBt/SRqqt_purity‚0.23076923191547394 >&Purity/1FRt0SRqqt*FBt/event_proportion‚0.19745463132858276 ("Purity/0FRt2SRqqt2FBt/event_purity‚0 ,&Purity/0FRt3SRqqt3FBt/event_proportion‚0 ("Purity/*FRt4SRqqt*FBt/event_purity‚0 ("Purity/3FRt*SRqqt2FBt/event_purity‚0 * Purity/1FRt*SRqqt4FBt/FBt_purity‚0.875 & Purity/4FRt0SRqqt*FBt/FRt_purity‚0 ?&Purity/*FRt2SRqqt2FBt/event_proportion‚0.002079133177176118 ?&Purity/*FRt0SRqqt3FBt/event_proportion‚0.003717237850651145 9"Purity/0FRt2SRqqt*FBt/SRqqt_purity‚0.3465501070022583 ,&Purity/1FRt4SRqqt2FBt/event_proportion‚0 ,&Purity/3FRt4SRqqt3FBt/event_proportion‚0 particle/accuracy_2_of_2‚1 *"Purity/3FRt*SRqqt2FBt/SRqqt_purity‚0.5 >&Purity/*FRt*SRqqt1FBt/event_proportion‚0.27079132199287415 ?&Purity/0FRt0SRqqt3FBt/event_proportion‚0.002583165420219302 <"Purity/1FRt1SRqqt2FBt/event_purity‚0.0071428571827709675 7 Purity/*FRt2SRqqt2FBt/FRt_purity‚0.3974359333515167 ,&Purity/0FRt2SRqqt4FBt/event_proportion‚0 9"Purity/1FRt1SRqqt2FBt/SRqqt_purity‚0.3561224639415741 ?&Purity/1FRt2SRqqt1FBt/event_proportion‚0.008946572430431843 ("Purity/3FRt0SRqqt0FBt/event_purity‚0 @&Purity/4FRt0SRqqt*FBt/event_proportion‚6.300403038039804e-05 ,&Purity/4FRt4SRqqt3FBt/event_proportion‚0 ("Purity/3FRt4SRqqt0FBt/event_purity‚0 ("Purity/3FRt3SRqqt*FBt/event_purity‚0 & Purity/1FRt3SRqqt*FBt/FBt_purity‚1 8 Purity/1FRt3SRqqt0FBt/FRt_purity‚0.232603698968887330äqqÍK¤ ‚(:À ¦ÅèÔó躲 disk.sda1.out‚0 disk.rbd2.in‚0 disk.rbd2.out‚ 0.228515625 disk.md0p1.in‚ 0.0078125 network.sent‚ 1.050177e+06 network.recv‚428766 disk.nvme1n1.out‚ 5006.234375 disk.sda.in‚ 0.0078125 disk.dm-0.out‚13944.48828125 disk.dm-0.in‚ 4.25390625 disk.sdb1.out‚0! disk.sdb2.out‚109.07568359375! disk.sda2.out‚109.07568359375 disk.rbd0.in‚0" disk.nvme0n1.out‚ 8938.25390625 disk.sdb2.in‚0 disk.sdb.in‚0 disk.sda2.in‚ 0.0078125% disk./.usageGB‚2979.5695152282715 disk.nvme0n1.in‚ 0.25390625 disk.md0.out‚ 104.66796875$ memory_percent‚77.84786224365234# proc.memory.rssMB‚ 3050.70703125 cpu‚25.54597098506068 proc.cpu.threads‚17 disk.md0p1.out‚ 104.66796875 disk.sdb.out‚109.07568359375 disk.rbd0.out‚ 0.27880859375 disk.sdb1.in‚0 disk.md0.in‚ 0.0078125) disk./.usagePercent‚83.31001069250502 disk.nvme1n1.in‚4 disk.sda.out‚109.07568359375) proc.memory.percent‚18.62003803253174 disk.sda1.in‚0& proc.memory.availableMB‚ 3629.40625î†Ãô’¥ ‚(:‡ ¦ÅèÔäÝ©² gpu.0.gpu‚98.0 gpu.0.memory‚17+ gpu.0.memoryAllocated‚47.54493713378906* gpu.0.memoryAllocatedBytes‚ 40840790016 gpu.0.temp‚42.0 gpu.0.powerWatts‚116.916' gpu.0.enforcedPowerLimitWatts‚400.0 gpu.0.powerPercent‚29.229 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0YøŸ’¦ ‚(:‡ µÅèÔÙÌž° gpu.0.gpu‚9.0 gpu.0.memory‚5+ gpu.0.memoryAllocated‚47.54493713378906* gpu.0.memoryAllocatedBytes‚ 40840790016 gpu.0.temp‚40.0 gpu.0.powerWatts‚116.583' gpu.0.enforcedPowerLimitWatts‚400.0 gpu.0.powerPercent‚29.14575 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0°ÈK–:§ ‚(:¯ µÅèÔ‹õ ± disk.nvme1n1.in‚4 disk.nvme1n1.out‚ 6154.546875 disk.sda.out‚ 110.318359375 disk.sdb2.in‚0 disk.sdb2.out‚ 110.318359375 network.recv‚434069( disk./.usagePercent‚83.3126916049799 disk.sdb1.in‚0 disk.rbd2.out‚ 0.29150390625" proc.memory.rssMB‚ 3050.7734375 network.sent‚ 1.154355e+06 disk.rbd0.in‚0 disk.sda1.out‚0 proc.cpu.threads‚17$ disk./.usageGB‚2979.665397644043 disk.nvme0n1.in‚ 0.25390625 disk.sda.in‚ 0.0078125 disk.md0.out‚ 105.84375 disk.rbd2.in‚0 cpu‚25.54237599140409" disk.nvme0n1.out‚ 8999.50390625 disk.md0.in‚ 0.0078125 disk.sda2.in‚ 0.0078125 disk.sda2.out‚ 110.318359375) proc.memory.percent‚18.62044334411621 disk.dm-0.in‚ 4.25390625$ proc.memory.availableMB‚3629.375 disk.rbd0.out‚ 0.27880859375 disk.sdb1.out‚0 disk.sdb.out‚ 110.318359375$ memory_percent‚77.84805297851562 disk.dm-0.out‚15154.05078125 disk.sda1.in‚0 disk.md0p1.in‚ 0.0078125 disk.md0p1.out‚ 105.84375 disk.sdb.in‚0¦Bdž”¨ ‚(:‰ ÄÅèÔÀ£ç° gpu.0.gpu‚97.0 gpu.0.memory‚17+ gpu.0.memoryAllocated‚47.54493713378906* gpu.0.memoryAllocatedBytes‚ 40840790016 gpu.0.temp‚45.0 gpu.0.powerWatts‚236.339' gpu.0.enforcedPowerLimitWatts‚400.0 gpu.0.powerPercent‚59.08475 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0ý~ª¯G© ‚(:¼ ÄÅèÔœÕÖ±) disk./.usagePercent‚83.32407537691272 disk.sdb2.in‚0 disk.sda1.in‚0 disk.rbd2.in‚0 disk.sdb.out‚111.83447265625 disk.rbd0.out‚ 0.341796875 network.recv‚438868! disk.sdb2.out‚111.83447265625 disk.md0.out‚ 107.30078125 disk.sdb1.out‚0 disk.md0.in‚ 0.0078125 disk.md0p1.in‚ 0.0078125 disk.sda2.in‚ 0.0078125! disk.sda2.out‚111.83447265625 disk.nvme0n1.in‚ 0.25390625$ memory_percent‚77.84199714660645( proc.memory.availableMB‚ 3630.3671875 disk.sda1.out‚0 disk.sda.in‚ 0.0078125! disk.nvme0n1.out‚ 9060.9609375 disk.dm-0.out‚ 15293.375) proc.memory.percent‚18.61443519592285 network.sent‚ 1.160725e+06$ disk./.usageGB‚2980.072536468506 disk.rbd2.out‚ 0.29150390625 disk.sdb.in‚0 disk.dm-0.in‚ 4.25390625" proc.memory.rssMB‚ 3049.7890625 cpu‚25.540308656921155 proc.cpu.threads‚17 disk.md0p1.out‚ 107.30078125 disk.nvme1n1.in‚4 disk.rbd0.in‚0! disk.nvme1n1.out‚ 6232.4140625 disk.sdb1.in‚0 disk.sda.out‚111.834472656250ì蔪 ‚(:‰ ÓÅèÔÍÆ…® gpu.0.gpu‚98.0 gpu.0.memory‚23+ gpu.0.memoryAllocated‚47.54493713378906* gpu.0.memoryAllocatedBytes‚ 40840790016 gpu.0.temp‚45.0 gpu.0.powerWatts‚322.515' gpu.0.enforcedPowerLimitWatts‚400.0 gpu.0.powerPercent‚80.62875 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0æŒi©D« ‚(:¹ ÓÅèÔ¦ÿû® disk.sdb1.out‚0 disk.rbd2.out‚ 0.29150390625 disk.sda.in‚ 0.0078125! disk.nvme0n1.out‚ 9122.1953125 disk.sda.out‚ 113.318359375$ memory_percent‚77.84640789031982) proc.memory.percent‚18.61443519592285$ disk./.usageGB‚2980.071506500244 disk.sda2.in‚ 0.0078125 cpu‚25.537587390940157 proc.cpu.threads‚17" disk.nvme1n1.out‚ 6310.01953125 network.recv‚442504 disk.rbd0.in‚0 disk.sda1.out‚0 disk.md0p1.out‚ 108.7109375 disk.sdb.in‚0 disk.sda2.out‚ 113.318359375) proc.memory.availableMB‚ 3629.64453125) disk./.usagePercent‚83.32404657856904 disk.nvme0n1.in‚ 0.25390625 disk.dm-0.out‚15432.21484375 disk.sdb2.in‚0 disk.sdb2.out‚ 113.318359375 disk.md0.out‚ 108.7109375 disk.md0p1.in‚ 0.0078125" proc.memory.rssMB‚ 3049.7890625 network.sent‚ 1.163821e+06 disk.dm-0.in‚ 4.25390625 disk.sdb1.in‚0 disk.sdb.out‚ 113.318359375 disk.nvme1n1.in‚4 disk.sda1.in‚0 disk.rbd2.in‚0 disk.rbd0.out‚ 0.341796875 disk.md0.in‚ 0.0078125N­Ù•’¬ ‚(:‡ âÅèÔ¤‚ú¯ gpu.0.gpu‚35.0 gpu.0.memory‚10+ gpu.0.memoryAllocated‚47.54493713378906* gpu.0.memoryAllocatedBytes‚ 40840790016 gpu.0.temp‚41.0 gpu.0.powerWatts‚163.09' gpu.0.enforcedPowerLimitWatts‚400.0 gpu.0.powerPercent‚40.7725 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0#F«§;­ ‚(:° âÅèÔر± network.recv‚446870$ disk./.usageGB‚2980.571506500244 disk.sda1.in‚0 disk.rbd2.in‚0 disk.sda2.in‚ 0.0078125! disk.nvme1n1.out‚ 6439.2890625 disk.sdb1.out‚0) proc.memory.percent‚18.61443519592285 disk.dm-0.in‚ 4.25390625 disk.dm-0.out‚ 15684.0078125( disk./.usagePercent‚83.3380267878353 disk.md0p1.out‚ 109.734375 disk.sda2.out‚ 114.392578125 disk.sda.out‚ 114.392578125' proc.memory.availableMB‚ 3629.140625 cpu‚25.534496738021858 network.sent‚ 1.171954e+06 disk.sda1.out‚0$ memory_percent‚77.84948348999023 disk.md0p1.in‚ 0.0078125 disk.sda.in‚ 0.0078125" proc.memory.rssMB‚ 3049.7890625 disk.sdb.in‚0 disk.sdb2.in‚0 disk.sdb1.in‚0 disk.md0.in‚ 0.0078125 disk.sdb.out‚ 114.392578125 disk.rbd0.in‚0 disk.rbd0.out‚ 0.341796875 disk.md0.out‚ 109.734375 disk.rbd2.out‚ 0.29150390625 proc.cpu.threads‚17 disk.nvme1n1.in‚4 disk.nvme0n1.in‚ 0.25390625 disk.nvme0n1.out‚ 9244.71875 disk.sdb2.out‚ 114.392578125Xði-2® - _wandbruntime‚1182  _runtime‚1182t¦Ôhį ¾ & lr-AdamW/pg2‚0.0011371425386125776 trainer/global_step‚649 $ _timestamp‚1.788486383116339e+09 _runtime‚1182.217992283 _step‚49 & lr-AdamW/pg1‚0.0011371425386125776šG}È°  & lr-AdamW/pg2‚0.0011371425386125776 trainer/global_step‚649 $ _timestamp‚1.788486383116339e+09 _runtime‚1182.217992283 _step‚49 & lr-AdamW/pg1‚0.00113714253861257761À€[2± - _wandbruntime‚1182  _runtime‚1182Ã:³S² ãO OCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚9365069 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚2908986 A8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚3680 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 O?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 40288387072 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚25427968 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 UFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 3214934016 MDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚1533 [IDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 1564156550656 YJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 2516362240 NFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚330 K?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚9365069 VBDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated‚145150056328704 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚23979 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚71523328 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 VFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 37073453056 PDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚7430289 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚2684 ]IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚143585899778048 YEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚143585874350080 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚23680 L@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚7429710 L?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚12206029 SGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚2908983 NFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚977 UADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚139406161913525 D;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚3680 SGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚4520727 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚19688 dQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚29031436548096 YEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚145150056328704 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 `LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚137846527875312 \HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚137846502447344 WGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 35913877504 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚23979 RCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 2541790208 ^LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 1564156550656 YJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 2514370132 [KDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 37073453056 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚23680 VGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 3142752256 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚2684 TGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚12225711 cQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 1955466816512 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 F=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚3680 \HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚143585874350080 ^LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 1562173836313 ZKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 3214934016 A8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚3680 H?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚1533 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚6 ]JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚30986903364608 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 100621312 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 RBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 40288387072 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚25427968 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 O@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 2541790208 R>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚145147514538496 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 561701376 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 XIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 3214934016 _runtime‚1182.220642534 `MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚29031407450112 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 MDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚2147 IDeviceStatsMonitor.on_train_batch_start/allocation.all.current‚19688 M=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 38610163200 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚576 OCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚9365075 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚29097984 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 963120128 _step‚50 UADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚145147514538496 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 WEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 1561640188416 ZHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 1559659466181 KBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚1533 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚19682 PCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚12206029 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚579 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚3 YIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 37073453056 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 H?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚1300 YEDeviceStatsMonitor.oð——g–n_train_batch_start/requested_bytes.all.allocated‚139408701711625 SDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 3142752256 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚19682 TEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 1497634304 BDeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 VGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 3140738480 E8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚21571098 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚6 VGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 2516362240 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚25427968 KBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚2147 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 PCDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated‚12225711 P@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 38610163200 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚4521303 WGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 34283657216 TDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 40288387072 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 H?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚2147 `LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚143585899778048 M@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚21590786 YFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚30986802743296 _MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 1955395293184 RCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 2539798100 IDeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚2684 ]IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚143585899778048 YEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚143585874350080 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚23680 L@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚7429710 L?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚12206029 SGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚2908983 NFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚977 UADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚139406161913525 D;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚3680 SGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚4520727 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚19688 dQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚29031436548096 YEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚145150056328704 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 `LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚137846527875312 \HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚137846502447344 WGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 35913877504 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚23979 RCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 2541790208 ^LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 1564156550656 YJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 2514370132 [KDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 37073453056 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚23680 VGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 3142752256 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚2684 TGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚12225711 cQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 1955466816512 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 F=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚3680 \HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚143585874350080 ^LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 1562173836313 ZKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 3214934016 A8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚3680 H?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚1533 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚6 ]JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚30986903364608 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 100621312 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 RBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 40288387072 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚25427968 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 O@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 2541790208 R>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚145147514538496 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 561701376 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 XIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 3214934016 _runtime‚1182.220642534 `MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚29031407450112 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 MDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚2147 IDeviceStatsMonitor.on_train_batch_start/allocation.all.current‚19688 M=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 38610163200 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚576 OCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚9365075 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚29097984 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 963120128 _step‚50 UADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚145147514538496 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 WEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 1561640188416 ZHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 1559659466181 KBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚1533 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚19682 PCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚12206029 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚579 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚3 YIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 37073453056 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 H?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚1300 YEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚139408701711625 SDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 3142752256 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚19682 TEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 1497634304 BDeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 VGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 3140738480 E8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚21571098 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚6 VGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 2516362240 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚25427968 KBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚2147 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 PCDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated‚12225711 P@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 38610163200 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚4521303 WGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 34283657216 TDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 40288387072 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 H?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚2147 `LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚143585899778048 M@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚21590786 YFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚30986802743296 _MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 1955395293184 RCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 25397981002ŒÃ2´ - _wandbruntime‚1183  _runtime‚1183cû1ô&µ îM UIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚2913352 RBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 35913877504 REDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚12244079 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚19708 XFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 1563996284416 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 B9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚3680 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚6 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚71510016 RBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 40288387072 M=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 40288387072 MADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚9378959 LDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚977 ]KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 1958346647552 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚19714 UCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 1563996284416 I@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚2147 _runtime‚1183.824378178 QBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 3142752256 NBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚7441129 ^JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚138052043535808 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 NADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚12224371 [HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚31033985030656 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 P@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 40288387072 I@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚1533 XIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 3214934016 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1300 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚4527201 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 F=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚1533 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚6 K>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚21623044 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚29097984 aODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 1958418157568 S?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚139614030709755 ?6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚3680 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 NADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚12244079 RCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 1497634304 F=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚2147 % _timestamp‚1.7884863847220519e+09 WCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚143799820420608 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚4527777 KBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚2147 _step‚51 N>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 38610163200 LDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚330 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 WCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚145366358508544 C6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚21603330 T@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚145366358508544 K;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 38610163200 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 D;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚3680 PADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 2541803520 FDeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚7440550 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚2913349 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 963120128 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚23979 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚19708 ^JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚143799845848576 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚579 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚25427968 QEDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚9378965 bODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚29075566873088 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚23680 VGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 3214934016 YGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 1566512659968 WHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 2516375552 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 WHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 2514370340 XFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 1562012601915 TDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 37073453056 TEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 2516375552 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚25427968 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 100608000 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 MADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚9378965 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚576 ?6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚3680 N>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 36985723393 SDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 3214934016 I=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚9378959 M>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 2541803520 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚23979 PADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 2539798308 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 WCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚139616570508063 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚3 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 S?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚145363816705024 EDeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚21623044 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚29097984 aODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 1958418157568 S?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚139614030709755 ?6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚3680 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 NADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚12244079 RCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 1497634304 F=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚2147 % _timestamp‚1.7884863847220519e+09 WCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚143799820420608 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚4527777 KBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚2147 _step‚51 N>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 38610163200 LDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚330 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 WCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚145366358508544 C6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚21603330 T@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚145366358508544 K;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 38610163200 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 D;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚3680 PADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 2541803520 FDeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚7440550 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚2913349 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 963120128 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚23979 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚19708 ^JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚143799845848576 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚579 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚25427968 QEDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚9378965 bODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚29075566873088 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚23680 VGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 3214934016 YGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 1566512659968 WHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 2516375552 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 WHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 2514370340 XFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 1562012601915 TDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 37073453056 TEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 2516375552 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚25427968 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 100608000 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 MADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚9378965 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚576 ?6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚3680 N>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 36985723393 SDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 3214934016 I=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚9378959 M>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 2541803520 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚23979 PADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 2539798308 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 WCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚139616570508063 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚3 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 S?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚145363816705024 E8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 B9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚3680 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚6 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚71510016 RBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 40288387072 M=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 40288387072 MADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚9378959 LDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚977 ]KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 1958346647552 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚19714 UCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 1563996284416 I@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚2147 _runtime‚1183.824378178 QBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 3142752256 NBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚7441129 ^JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚138052043535808 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚03øe ‰2· - _wandbruntime‚1183  _runtime‚1183ÈÒyÆü¸ Ó 0loss/FRt2/assignment_loss‚1.4689481454559519 /loss/FRt4/assignment_loss‚3.451673293259086 1loss/FBt4/assignment_loss‚0.43826684920010545 /loss/FRt4/detection_loss‚2.5534764386531785 /loss/FBt2/detection_loss‚0.83973111964100TÏÝ>]03 % _timestamp‚1.7884863847247903e+09 0loss/SRqqt3/assignment_loss‚1.99271291951221 .loss/FBt3/detection_loss‚2.732342793827965 0loss/FRt3/assignment_loss‚3.9665490917012662 /loss/FRt2/detection_loss‚1.9032724110197417 1loss/SRqqt4/detection_loss‚1.6588540128362712 1loss/FBt1/assignment_loss‚0.21131590055301785 1loss/FBt3/assignment_loss‚0.02864547640534251 /loss/FRt3/detection_loss‚2.5977249500117723 1loss/SRqqt2/detection_loss‚0.9636341782130358 .loss/FBt1/detection_loss‚3.607478896776835 epoch‚1 _step‚52 0loss/FRt1/assignment_loss‚4.0955716947491245 2loss/SRqqt4/assignment_loss‚2.1099117807487944 1loss/FBt2/assignment_loss‚0.18740791080100003 trainer/global_step‚649 2loss/SRqqt1/assignment_loss‚1.8453772882930934 0loss/SRqqt2/assignment_loss‚1.09629846180128 -loss/FRt1/detection_loss‚2.28752826974633 1loss/SRqqt1/detection_loss‚2.5027504842728376 &loss/total_loss‚47.755801401326565 1loss/SRqqt3/detection_loss‚2.2549551483747123 /loss/FBt4/detection_loss‚2.9613738854726157 _runtime‚1183.8262262665|]¹ × 0loss/FRt2/assignment_loss‚1.4689481454559519 /loss/FRt4/assignment_loss‚3.451673293259086 1loss/FBt4/assignment_loss‚0.43826684920010545 /loss/FRt4/detection_loss‚2.5534764386531785 /loss/FBt2/detection_loss‚0.8397311196410003 % _timestamp‚1.7884863847247903e+09 0loss/SRqqt3/assignment_loss‚1.99271291951221 .loss/FBt3/detection_loss‚2.732342793827965 0loss/FRt3/assignment_loss‚3.9665490917012662 /loss/FRt2/detection_loss‚1.9032724110197417 1loss/SRqqt4/detection_loss‚1.6588540128362712 1loss/FBt1/assignment_loss‚0.21131590055301785 1loss/FBt3/assignment_loss‚0.02864547640534251 /loss/FRt3/detection_loss‚2.5977249500117723 1loss/SRqqt2/detection_loss‚0.9636341782130358 .loss/FBt1/detection_loss‚3.607478896776835 epoch‚1 _step‚52 0loss/FRt1/assignment_loss‚4.0955716947491245 2loss/SRqqt4/assignment_loss‚2.1099117807487944 1loss/FBt2/assignment_loss‚0.18740791080100003 trainer/global_step‚649 2loss/SRqqt1/assignment_loss‚1.8453772882930934 0loss/SRqqt2/assignment_loss‚1.09629846180128 -loss/FRt1/detection_loss‚2.28752826974633 1loss/SRqqt1/detection_loss‚2.5027504842728376 &loss/total_loss‚47.755801401326565 1loss/SRqqt3/detection_loss‚2.2549551483747123 /loss/FBt4/detection_loss‚2.9613738854726157 _runtime‚1183.8262262664¨•ÜKº ‚(:À ñÅèÔ¾ÆƱ! disk.nvme0n1.out‚ 9305.9765625 disk.dm-0.out‚ 15793.421875 disk.sdb1.in‚0 disk.sdb2.in‚0( proc.memory.availableMB‚ 3628.9765625" proc.memory.rssMB‚ 3049.7890625 disk.sda2.in‚ 0.0078125! disk.nvme1n1.out‚ 6487.4453125 network.sent‚ 1.175597e+06 disk.sdb.out‚115.65869140625 disk.sda1.out‚0% disk./.usageGB‚2980.5717697143555 disk.nvme1n1.in‚4 disk.rbd2.in‚0 disk.sdb.in‚0 disk.nvme0n1.in‚ 0.25390625 disk.dm-0.in‚ 4.25390625 disk.md0.out‚ 110.93359375) proc.memory.percent‚18.61443519592285 disk.rbd2.out‚ 0.29150390625! disk.sda2.out‚115.65869140625 disk.sda.out‚115.65869140625 disk.md0.in‚ 0.0078125 disk.rbd0.in‚0 disk.rbd0.out‚ 0.341796875 disk.sda.in‚ 0.0078125 disk.sda1.in‚0 cpu‚25.533234287301106! disk.sdb2.out‚115.65869140625 network.recv‚450781) disk./.usagePercent‚83.33803414741202 disk.md0p1.in‚ 0.0078125 disk.md0p1.out‚ 110.93359375 disk.sdb1.out‚0$ memory_percent‚77.85048484802246 proc.cpu.threads‚17¤ëßÿ» ‚(:’ ñÅèÔ‹©Û± gpu.0.gpu‚97.0 gpu.0.memory‚17+ gpu.0.memoryAllocated‚47.54493713378906* gpu.0.memoryAllocatedBytes‚ 40840790016 gpu.0.temp‚45.0 gpu.0.powerWatts‚227.652' gpu.0.enforcedPowerLimitWatts‚400.0( gpu.0.powerPercent‚56.91299999999999 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0žÒÐ{’¼ ‚(:‡ €ÆèÔÌ•°° gpu.0.gpu‚80.0 gpu.0.memory‚18+ gpu.0.memoryAllocated‚47.54493713378906* gpu.0.memoryAllocatedBytes‚ 40840790016 gpu.0.temp‚45.0 gpu.0.powerWatts‚124.044' gpu.0.enforcedPowerLimitWatts‚400.0 gpu.0.powerPercent‚31.011 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0ŒbÚN½ ‚(:à €ÆèÔîüÏ° proc.cpu.threads‚17 disk.dm-0.out‚16023.08203125 disk.sda1.in‚0$ memory_percent‚77.85747051239014 disk.sda.in‚ 0.0078125 disk.md0.in‚ 0.0078125 disk.md0p1.out‚ 112.29296875" proc.memory.rssMB‚ 3049.7890625 network.recv‚456770 disk.sda2.in‚ 0.0078125! disk.sda2.out‚117.07666015625 disk.sda.out‚117.07666015625 disk.rbd2.out‚ 0.29150390625 disk.sdb.out‚117.07666015625) proc.memory.percent‚18.61443519592285$ disk./.usageGB‚2981.073844909668! disk.nvme1n1.out‚ 6594.6171875 disk.nvme0n1.in‚ 0.25390625 disk.sda1.out‚0 disk.sdb.in‚0 network.sent‚ 1.264441e+06) disk./.usagePercent‚83.35207238000775 disk.nvme1n1.in‚4 disk.rbd0.out‚ 0.341796875! disk.sdb2.out‚117.07666015625 disk.rbd0.in‚0" disk.nvme0n1.out‚ 9428.46484375 disk.dm-0.in‚ 4.25390625 disk.sdb1.out‚0 cpu‚25.531438918794304 disk.sdb1.in‚0 disk.sdb2.in‚0 disk.md0p1.in‚ 0.0078125 disk.md0.out‚ 112.29296875 disk.rbd2.in‚0) proc.memory.availableMB‚ 3627.83203125î¥\锾 ‚(:‰ ÆèÔöƒÄ° gpu.0.gpu‚97.0 gpu.0.memory‚18+ gpu.0.memoryAllocated‚47.54493713378906* gpu.0.memoryAllocatedBytes‚ 40840790016 gpu.0.temp‚45.0 gpu.0.powerWatts‚247.561' gpu.0.enforcedPowerLimitWatts‚400.0 gpu.0.powerPercent‚61.89025 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0Äð*BM¿ ‚(: ÆèÔºµ± disk.rbd2.out‚ 0.29150390625 disk.sdb.in‚0) proc.memory.percent‚18.61443519592285 disk.md0p1.out‚ 113.38671875 disk.dm-0.in‚ 4.25390625 disk.md0.in‚ 0.0078125 proc.cpu.threads‚17! disk.sdb2.out‚118.23779296875 disk.sda1.out‚0 disk.md0.out‚ 113.38671875" disk.nvme1n1.out‚ 6638.24609375 disk.sda.out‚118.23779296875) proc.memory.availableMB‚ 3627.81640625" proc.memory.rssMB‚ 3049.7890625! disk.sda2.out‚118.23779296875 disk.sda.in‚ 0.0078125 disk.dm-0.out‚16127.95703125 network.recv‚460411 cpu‚25.529630022115537 network.sent‚ 1.269875e+06 disk.md0p1.in‚ 0.0078125 disk.rbd2.in‚0 disk.sdb.out‚118.23779296875 disk.rbd0.in‚0 disk.sdb1.in‚0 disk.sdb1.out‚0$ memory_percent‚77.85756587982178 disk.rbd0.out‚ 0.341796875 disk.nvme0n1.in‚ 0.25390625! disk.nvme0n1.out‚ 9489.7109375( disk./.usagePercent‚83.3372613918568 disk.sda2.in‚ 0.0078125 disk.nvme1n1.in‚4$ disk./.usageGB‚2980.544132232666 disk.sdb2.in‚0 disk.sda1.in‚07î#_JÀ ‚(:¿ žÆèÔ¥Ÿ°° disk.nvme1n1.in‚4 disk.rbd0.in‚0 disk.sdb2.in‚0 disk.sda1.in‚0 disk.rbd2.out‚ 0.29150390625 disk.sdb.in‚0 disk.sdb.out‚119.49853515625! disk.nvme1n1.out‚ 6638.9140625 disk.rbd0.out‚ 0.341796875 disk.dm-0.out‚ 16189.9375 disk.sdb1.in‚0 disk.sda1.out‚0 disk.sda2.in‚ 0.0078125! disk.nvme0n1.out‚ 9565.7734375 disk.sdb1.out‚0 disk.rbd2.in‚0) proc.memory.availableMB‚ 3627.56640625 proc.cpu.threads‚17 network.sent‚ 1.275746e+06 disk.sda.in‚ 0.0078125 disk.nvme0n1.in‚ 0.25390625 disk.md0.in‚ 0.0078125$ memory_percent‚77.85909175872803 disk.sda.out‚119.49853515625 disk.md0p1.out‚ 114.58984375 network.recv‚464843) disk./.usagePercent‚83.33601474355736 cpu‚25.528306173107996% disk./.usageGB‚2980.4995460510254! disk.sda2.out‚119.49853515625 disk.dm-0.in‚ 4.25390625! disk.sdb2.out‚119.49853515625 disk.md0.out‚ 114.58984375 disk.md0p1.in‚ 0.0078125" proc.memory.rssMB‚ 3049.7890625) proc.memory.percent‚18.614435195922852ƒ:ó”Á ‚(:‰ žÆèÔÜÄ÷° gpu.0.gpu‚92.0 gpu.0.memory‚21+ gpu.0.memoryAllocated‚47.54493713378906* gpu.0.memoryAllocatedBytes‚ 40840790016 gpu.0.temp‚45.0 gpu.0.powerWatts‚236.377' gpu.0.enforcedPowerLimitWatts‚400.0 gpu.0.powerPercent‚59.09425 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0óV'á“ ‚(:ˆ ­ÆèÔ…“Ï° gpu.0.gpu‚43.0 gpu.0.memory‚11+ gpu.0.memoryAllocated‚47.54493713378906* gpu.0.memoryAllocatedBytes‚ 40840790016 gpu.0.temp‚47.0 gpu.0.powerWatts‚321.566' gpu.0.enforcedPowerLimitWatts‚400.0 gpu.0.powerPercent‚80.3915 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0ó^`@à ‚(:µ ­ÆèÔ㱆±" disk.nvme0n1.out‚ 9673.44140625 disk.md0p1.out‚ 115.9609375" disk.nvme1n1.out‚ 6710.91015625 disk.sda.in‚ 0.0078125 disk.rbd0.out‚ 0.341796875) proc.memory.availableMB‚ 3628.01953125" proc.memory.rssMB‚ 3049.7890625 disk.sda2.in‚ 0.0078125 disk.nvme1n1.in‚4 disk.sda.out‚ 120.92578125 disk.sda1.out‚0 disk.sdb1.in‚0 disk.sdb.out‚ 120.92578125$ memory_percent‚77.85632610321045 disk.sdb2.in‚0 disk.md0p1.in‚ 0.0078125 disk.rbd2.out‚ 0.29150390625 disk.sdb.in‚0) proc.memory.percent‚18.61443519592285$ disk./.usageGB‚2980.571865081787 disk.sdb1.out‚0 disk.sda1.in‚0 disk.sda2.out‚ 120.92578125 cpu‚25.526959128660394) disk./.usagePercent‚83.33803681392533 disk.nvme0n1.in‚ 0.25390625 disk.dm-0.out‚ 16384.3515625 disk.md0.in‚ 0.0078125 disk.md0.out‚ 115.9609375 disk.rbd2.in‚0 network.recv‚468390 disk.dm-0.in‚ 4.25390625 disk.sdb2.out‚ 120.92578125 disk.rbd0.in‚0 proc.cpu.threads‚17 network.sent‚ 1.280813e+06âðG¹Ä ‚(:’ ¼ÆèÔ³´¯° gpu.0.gpu‚98.0 gpu.0.memory‚17+ gpu.0.memoryAllocated‚47.54493713378906* gpu.0.memoryAllocatedBytes‚ 40840790016 gpu.0.temp‚45.0 gpu.0.powerWatts‚221.742' gpu.0.enforcedPowerLimitWatts‚400.0( gpu.0.powerPercent‚55.43549999999999 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0Ûö&AÅ ‚(:¶ ¼ÆèÔÝÙ˜± network.recv‚472998 disk.md0.out‚ 117.015625 disk.sda2.out‚122.0380859375 disk.dm-0.in‚ 4.25390625" proc.memory.rssMB‚ 3049.7890625 cpu‚25.52488687889976 disk.sdb2.in‚0 disk.nvme1n1.in‚4) proc.memory.percent‚18.61443519592285 disk.sda1.out‚0 disk.md0.in‚ 0.0078125 disk.sdb.in‚0 disk.rbd0.in‚0 disk.nvme0n1.in‚ 0.25390625! disk.nvme0n1.out‚ 9734.6796875) proc.memory.availableMB‚ 3628.02734375 disk.rbd2.out‚ 0.29150390625 disk.rbd0.out‚ 0.341796875 disk.sdb1.in‚0$ memory_percent‚77.85627841949463 disk.sdb2.out‚122.0380859375 disk.sda1.in‚0 disk.md0p1.in‚ 0.0078125 disk.dm-0.out‚ 16473.6875 disk.sdb1.out‚0% disk./.usageGB‚2980.6971473693848 disk.sdb.out‚122.0380859375 disk.sda.out‚122.0380859375) disk./.usagePercent‚83.34153975912126! disk.nvme1n1.out‚ 6739.0078125 network.sent‚ 1.284798e+06 disk.md0p1.out‚ 117.015625 disk.rbd2.in‚0 disk.sda2.in‚ 0.0078125 disk.sda.in‚ 0.0078125 proc.cpu.threads‚17Í#^¨2Æ - _wandbruntime‚1268  _runtime‚1268<–´ÃÇ ½ % lr-AdamW/pg1‚0.001136477979430123 % lr-AdamW/pg2‚0.001136477979430123 trainer/global_step‚699 % _timestamp‚1.7884864696933959e+09 _runtime‚1268.794883741 _step‚53L.ÊÇÈ Á % lr-AdamW/pg2‚0.001136477979430123 trainer/global_step‚699 % _timestamp‚1.7884864696933959e+09 _runtime‚1268.794883741 _step‚53 % lr-AdamW/pg1‚0.0011364779794301235(Ó2É - _wandbruntime‚1268  _runtime‚1268k¢ÆNí'Ê çO E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚23979 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚29097984 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚23979 PCDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated‚13146163 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚23680 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚572 [IDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 1681989178880 \HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚148122285472144 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 L?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚13126481 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚25427968 M@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚23205738 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚3127286 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 ^LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 1679857795021 $ _timestamp‚1.788486469694655e+09 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚19688 PCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚10059569 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 102718464 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 _step‚54 ZKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 3214934016 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 H?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚1300 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚19688 TGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚10059575 TGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚13146163 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚4848816 KBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚1533 D>DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 RCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 2541790208 PCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚13126481 PDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚7976102 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚6 `LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚154283203304448 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚19682 NFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚330 WGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 34283657216 ZHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 1677343424889 dQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚31237952797696 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 MDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚1533 G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚2684 WGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 35913877504 cQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 2102576255488 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 WEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 1679472816640 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚3 O?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 40288387072 [KDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 37073453056 A8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚3680 KBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚2147 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚23680 E8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚23186050 L?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚10059569 VFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 37073453056 MDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚2147 H?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚1533 L@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚7975530 TEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 1497634304 IDeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚155962650693120 SDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 3142752256 `LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚148122310900112 TDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 40288387072 M=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 38610163200 A8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚3680 P@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 36985723393 F=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚3680 VBDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated‚155965192483328 TDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 35913877504 \HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚154283177876480 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚2684 UFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 3214934016 IDeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚155962650693120 SDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 3142752256 `LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚148122310900112 TDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 40288387072 M=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 38610163200 A8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚3680 P@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 36985723393 F=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚3680 VBDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated‚155965192483328 TDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 35913877504 \HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚154283177876480 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚2684 UFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 3214934016 IDeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚23680 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚572 [IDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 1681989178880 \HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚148122285472144 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 L?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚13126481 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚25427968 M@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚23205738 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚3127286 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 ^LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 1679857795021 $ _timestamp‚1.788486469694655e+09 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚19688 PCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚10059569 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 102718464 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 _step‚54 ZKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 3214934016 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 H?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚1300 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚19688 TGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚10059575 TGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚13146163 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚4848816 KBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚1533 D>DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 RCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 2541790208 PCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚13126481 PDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚7976102 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚6 `LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚154283203304448 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚19682 NFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚330 WGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 34283657216 ZHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 1677343424889 dQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚31237952797696 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 MDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚1533 G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚2684 WGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 35913877504 cQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 2102576255488 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 WEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 1679472816640 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚3 O?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 40288387072 [KDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 37073453056 A8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚3680 KBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚21476ëú¹Ù2Ì - _wandbruntime‚1270  _runtime‚1270«\÷,vÍ ñM XFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 1679696560623 TEDeviceStatsMonitorÿMM¶&.on_train_batch_end/requested_bytes.small_pool.peak‚ 3140738480 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚10073459 PDeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚23237996 UEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 34283657216 P@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 40288387072 K;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 38610163200 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 REDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚10073465 NADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚13144823 ?6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚3680 RBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 40288387072 _runtime‚1270.471397453 aODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 2105500756480 YGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 1684345288192 UCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 1681828912640 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 B9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚3680 E8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚25427968 T@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚156181494663168 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚25427968 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚3131649 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 104802304 RCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 1497634304 NADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚10073459 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚19714 NADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚13164531 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚6 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚4855297 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚578 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚6 WCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚154497123947008 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚75704320 BDeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚7986368 G:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚23237996 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚23979 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚25427968 PADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 2539798308 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 F=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚2147 M>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 2541803520 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚23680 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 561701376 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 KBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚2147 ?6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚3680 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 WGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 37073453056 ZFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚154497123947008 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚23979 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 SDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 3214934016 PADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 2541803520 ^JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚154497149374976 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚581 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 S?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚150007497693263 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 J=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚13144823 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚19708 ^KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚31282054024704 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 XFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 1681828912640 FDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 36985723393 F=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚1533 $ _timestamp‚1.788486471369454e+09 NADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚10073465 N>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 38610163200 \JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 1684345288192 ]KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 2105425052160 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 ^JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚148327826560608 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 [HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚33387583879168 WCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚150010037491571 TDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 37073453056 _step‚55 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚19708 S?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚156178952859648 WHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 2514370340 YIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 37073453056 C6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚23218282 bODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚31282083122688 LDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚977 ZFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚148327801132640 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚3 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚29097984 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 VGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 3214934016 [GDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚154497149374976 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚4854719 M=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 40288387072 trainer/global_step‚699 D;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚3680 NBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚7986949 LDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚330 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 UEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 35913877504 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1300Jæýtû&Î õM \JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 1682210930963 XIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 3214934016 I@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚2147 REDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚13164531 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚3131652 I@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚1533 TEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 2516375552 FDeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚7986368 G:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚23237996 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚23979 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚25427968 PADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 2539798308 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 F=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚2147 M>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 2541803520 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚23680 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 561701376 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 KBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚2147 ?6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚3680 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 WGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 37073453056 ZFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚154497123947008 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚23979 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 SDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 3214934016 PADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 2541803520 ^JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚154497149374976 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚581 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 S?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚150007497693263 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 J=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚13144823 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚19708 ^KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚31282054024704 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 XFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 1681828912640 FDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 36985723393 F=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚1533 $ _timestamp‚1.788486471369454e+09 NADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚10073465 N>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 38610163200 \JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 1684345288192 ]KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 2105425052160 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 ^JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚148327826560608 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 [HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚33387583879168 WCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚150010037491571 TDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 37073453056 _step‚55 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚19708 S?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚156178952859648 WHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 2514370340 YIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 37073453056 C6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚23218282 bODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚31282083122688 LDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚977 ZFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚148327801132640 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚3 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚29097984 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 VGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 3214934016 [GDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚154497149374976 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚4854719 M=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 40288387072 trainer/global_step‚699 D;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚3680 NBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚7986949 LDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚330 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 UEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 35913877504 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1300 XFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 1679696560623 TEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 3140738480 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚10073459 PDeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚23237996 UEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 34283657216 P@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 40288387072 K;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 38610163200 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 REDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚10073465 NADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚13144823 ?6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚3680 RBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 40288387072 _runtime‚1270.471397453 aODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 2105500756480 YGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 1684345288192 UCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 1681828912640 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 B9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚3680 E8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚25427968 T@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚156181494663168 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚25427968 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚3131649 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 104802304 RCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 1497634304 NADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚10073459 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚19714 NADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚13164531 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚6 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚4855297 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚578 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚6 WCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚154497123947008 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚75704320 B“Ú ‚(:ˆ ‡ÇèÔ‚ˆÑ® gpu.0.gpu‚80.0 gpu.0.memory‚18+ gpu.0.memoryAllocated‚47.54493713378906* gpu.0.memoryAllocatedBytes‚ 40840790016 gpu.0.temp‚45.0 gpu.0.powerWatts‚122.486' gpu.0.enforcedPowerLimitWatts‚400.0 gpu.0.powerPercent‚30.6215 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0ž²CïNÛ ‚(:à ‡ÇèÔÛÁæ¯# disk.nvme0n1.out‚10224.82421875 disk.sdb1.in‚0 disk.md0.out‚ 123.51171875 disk.md0p1.in‚ 0.0078125 disk.nvme1n1.in‚4.4375 disk.rbd0.in‚0 network.sent‚ 1.39404e+06 network.recv‚494781 disk.rbd2.in‚0" proc.memory.rssMB‚ 3049.7890625 proc.cpu.threads‚17) disk./.usagePercent‚83.33725915198563 disk.sdb2.in‚0 disk.sda1.in‚0 disk.sda1.out‚0 disk.sda2.in‚ 0.0078125% proc.memory.availableMB‚ 8.1640625 disk.nvme0n1.in‚ 0.25390625 disk.dm-0.in‚ 4.69140625 disk.sdb1.out‚0 disk.sdb.out‚128.82763671875 disk.sda.out‚128.82763671875) proc.memory.percent‚18.61443519592285 disk.dm-0.out‚ 17447.53125 disk.sda.in‚ 0.0078125% disk./.usageGB‚2980.5440521240234 disk.md0p1.out‚ 123.51171875 disk.md0.in‚ 0.0078125 disk.rbd2.out‚ 0.29150390625 disk.sdb.in‚0! disk.sda2.out‚128.82763671875 cpu‚25.517685641209802! disk.sdb2.out‚128.82763671875" disk.nvme1n1.out‚ 7222.70703125 disk.rbd0.out‚ 0.341796875$ memory_percent‚99.95017051696777=8EÇ”Ü ‚(:‰ –ÇèÔ…Ÿ»® gpu.0.gpu‚48.0 gpu.0.memory‚12+ gpu.0.memoryAllocated‚47.54493713378906* gpu.0.memoryAllocatedBytes‚ 40840790016 gpu.0.temp‚48.0 gpu.0.powerWatts‚320.681' gpu.0.enforcedPowerLimitWatts‚400.0 gpu.0.powerPercent‚80.17025 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0$ TCÝ ‚(:¸ –ÇèÔÞ¾é¯ disk.sdb2.in‚0 disk.md0.in‚ 0.0078125 disk.rbd2.out‚ 0.29150390625 disk.nvme0n1.in‚ 0.25390625 disk.rbd0.out‚ 0.341796875& proc.memory.availableMB‚ 8.28515625 disk.sda.out‚ 129.95703125 disk.sdb2.out‚ 129.95703125 disk.dm-0.out‚17570.04296875 disk.sdb.out‚ 129.95703125 disk.sdb1.out‚0 disk.rbd2.in‚0 disk.sdb.in‚0) proc.memory.percent‚18.61443519592285 network.recv‚498906 disk.md0p1.in‚ 0.0078125 disk.sda2.in‚ 0.0078125 disk.sda2.out‚ 129.95703125 disk.nvme1n1.in‚4.4375 disk.rbd0.in‚0 disk.dm-0.in‚ 4.69140625 disk.sda.in‚ 0.0078125 cpu‚25.516581866590247% disk./.usageGB‚2980.5440406799316! disk.nvme0n1.out‚ 10347.234375 disk.sdb1.in‚0 disk.md0.out‚ 124.5859375" disk.nvme1n1.out‚ 7222.80859375 proc.cpu.threads‚17 disk.sda1.out‚0 disk.md0p1.out‚ 124.5859375$ memory_percent‚99.94943141937256" proc.memory.rssMB‚ 3049.7890625 disk.sda1.in‚0 network.sent‚ 1.402103e+06) disk./.usagePercent‚83.33725883200404¹<_ 2Þ - _wandbruntime‚1354  _runtime‚1354»k»Åß ¿ & lr-AdamW/pg2‚0.0011353393375500734 trainer/global_step‚749 % _timestamp‚1.7884865556645918e+09 _runtime‚1354.769288226 _step‚57 & lr-AdamW/pg1‚0.0011353393375500734W»´oÉà à & lr-AdamW/pg2‚0.0011353393375500734 trainer/global_step‚749 % _timestamp‚1.7884865556645918e+09 _runtime‚1354.769288226 _step‚57 & lr-AdamW/pg1‚0.00113533933755007349šòcÄ2á - _wandbruntime‚1354  _runtime‚1354ïG8 â èO BDeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 L?DeviceStatsMonitor.on_train_batch_start/active.large_po)´5GÎ%ol.freed‚10754069 SDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 3142752256 A8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚3680 ^LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 1797541753729 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 E8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚24801002 O@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 2541790208 VGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 3142752256 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚25427968 IDeviceStatsMonitor.on_train_batch_start/allocation.all.current‚19688 PCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚14046933 PDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚8520899 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚565 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚3345586 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚5175313 RBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 40288387072 VGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 2516362240 cQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 2249562502656 WGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 34283657216 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚25427968 ZKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 3214934016 VBDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated‚166780328637952 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚19682 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 963120128 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 UFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 3214934016 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚6 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚23680 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 H?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚1533 P@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 38610163200 XIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 3214934016 H?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚2147 _runtime‚1354.770431892 L@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚8520334 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚562 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 YEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚164980481402880 H?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚1300 R>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚166777786847744 YFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚35693930928640 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 YJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 2516362240 NFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚330 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 P@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 36985723393 trainer/global_step‚749 _step‚58 SGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚3345583 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚23979 ]JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚35694031549952 TDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 40288387072 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 IDeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚23680 WEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 1797305444864 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚23979 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚19682 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚25427968 [IDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 1799821807104 RCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 2541790208 WGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 35913877504 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚2684 UADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚160193095880541 M@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚24820690 TGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚10754075 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 561701376 \HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚158398068496944 KBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚2147 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚19688 G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚2684 NFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚977 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 YEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚160195635678641 YJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 2514370132 VFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 37073453056 UADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚166777786847744 `LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚164980506830848 \HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚164980481402880 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 VGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 3140738480 D;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚3680 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 100621312 dQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚33444469047296 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚29097984 F=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚3680 SGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚5174751 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚71523328 PCDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated‚14066615 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 RCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 2539798100 TEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 1497634304 _MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 2249490979328 ZHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 1795027383597 [KDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 37073453056 YIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 37073453056 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚6 MDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚1533 L?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚14046933 M=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 38610163200 ^LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 1799821807104 ZHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 1797305444864 PCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚10754069H‘£Xò'ã ìO E8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚24801002 O@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 2541790208 VGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 3142752256 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚25427968 IDeviceStatsMonitor.on_train_batch_start/allocation.all.current‚19688 PCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚14046933 PDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚8520899 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚565 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚3345586 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚5175313 RBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 40288387072 VGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 2516362240 cQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 2249562502656 WGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 34283657216 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚25427968 ZKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 3214934016 VBDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated‚166780328637952 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚19682 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 963120128 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 UFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 3214934016 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚6 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚23680 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 H?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚1533 P@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 38610163200 XIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 3214934016 H?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚2147 _runtime‚1354.770431892 L@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚8520334 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚562 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 YEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚164980481402880 H?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚1300 R>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚166777786847744 YFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚35693930928640 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 YJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 2516362240 NFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚330 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 P@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 36985723393 trainer/global_step‚749 _step‚58 SGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚3345583 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚23979 ]JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚35694031549952 TDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 40288387072 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 IDeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚23680 WEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 1797305444864 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚23979 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚19682 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚25427968 [IDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 1799821807104 RCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 2541790208 WGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 35913877504 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚2684 UADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚160193095880541 M@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚24820690 TGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚10754075 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 561701376 \HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚158398068496944 KBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚2147 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚19688 G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚2684 NFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚977 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 YEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚160195635678641 YJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 2514370132 VFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 37073453056 UADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚166777786847744 `LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚164980506830848 \HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚164980481402880 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 VGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 3140738480 D;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚3680 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 100621312 dQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚33444469047296 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚29097984 F=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚3680 SGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚5174751 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚71523328 PCDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated‚14066615 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 RCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 2539798100 TEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 1497634304 _MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 2249490979328 ZHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 1795027383597 [KDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 37073453056 YIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 37073453056 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚6 MDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚1533 L?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚14046933 M=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 38610163200 ^LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 1799821807104 ZHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 1797305444864 PCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚10754069 BDeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 L?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚10754069 SDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 3142752256 A8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚3680 ^LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 1797541753729 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0:¦|?2ä - _wandbruntime‚1356  _runtime‚1356úûyZø&å òM _step‚59 TEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 2516375552 \JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 1802177916416 WDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚35741005665792 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚23979 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 WGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 37073453056 NADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚10767965 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 ?6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚3680 N>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 36985723393 ZFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚158603584157440 J=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚14065275 FDeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚24852948 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 FDeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 KBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚1533 WCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚166996630817792 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚23680 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚3349952 LDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚330 KBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚2147 F=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚2147 _runtime‚1356.289890227 G:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚24833234 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 963120128 ]KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 2252435391488 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 XFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 1797380519331 TDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 37073453056 ^JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚165194452901376 J>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚8531157 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1300 XIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 3214934016 RBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 35913877504 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 SDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 3214934016 D;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚3680 N>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 38610163200 WHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 2516375552 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 P@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 40288387072 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚19714 EDeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 2541803520 S?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚166994089014272 \JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 1799894889671 K;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 38610163200 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚6 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 561701376 PADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 2539798308 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚25427968 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚6 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚5181773 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚565 RCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 1497634304 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 F=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚1533 LDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚977 [HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚35741110468096 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 NBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚8531725 XFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 1799661540864 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚2684 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚5181208 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 TEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 3140738480 T@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚166996630817792 UCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 1799661540864 PADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 2541803520 UEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 34283657216 REDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚10767965 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 trainer/global_step‚749 % _timestamp‚1.7884865571876273e+09 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚25427968 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 104802304 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 BDeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚24852948 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 FDeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 KBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚1533 WCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚166996630817792 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚23680 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚3349952 LDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚330 KBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚2147 F=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚2147 _runtime‚1356.289890227 G:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚24833234 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 963120128 ]KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 2252435391488 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 XFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 1797380519331 TDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.pe°2 ak‚ 37073453056 ^JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚165194452901376 J>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚8531157 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1300 XIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 3214934016 RBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 35913877504 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 SDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 3214934016 D;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚3680 N>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 38610163200 WHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 2516375552 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 P@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 40288387072 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚19714 EDeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 2541803520 S?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚166994089014272 \JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 1799894889671 K;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 38610163200 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚6 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 561701376 PADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 2539798308 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚25427968 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚6 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚5181773 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚565 RCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 1497634304 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 F=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚1533 LDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚977 [HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚35741110468096 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 NBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚8531725 XFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 1799661540864 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚2684 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚5181208 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 TEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 3140738480 T@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚166996630817792 UCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 1799661540864 PADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 2541803520 UEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 34283657216 REDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚10767965 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 trainer/global_step‚749 % _timestamp‚1.7884865571876273e+09 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚25427968 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 104802304 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 BDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 36985723393 ZFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚158603584157440 J=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚14065275 Fžò ‚(:“ áÇèÔàÕ™° gpu.0.gpu‚97.0 gpu.0.memory‚17+ gpu.0.memoryAllocated‚47.54737854003906* gpu.0.memoryAllocatedBytes‚ 40842887168 gpu.0.temp‚45.0 gpu.0.powerWatts‚221.409' gpu.0.enforcedPowerLimitWatts‚400.0) gpu.0.powerPercent‚55.352250000000005 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0~§ü†Gó ‚(:¼ áÇèԆÀ± network.recv‚520569 disk.sdb.in‚0 disk.sdb.out‚136.74755859375$ memory_percent‚99.95057582855225( disk./.usagePercent‚83.3520413417929 disk.rbd0.out‚ 0.341796875 disk.sda2.in‚ 0.0078125& proc.memory.availableMB‚ 8.09765625" proc.memory.rssMB‚ 3049.7890625 proc.cpu.threads‚17 disk.nvme0n1.in‚ 0.25390625 disk.dm-0.out‚18362.66015625 disk.sda1.in‚0 disk.sda1.out‚0 disk.rbd2.out‚ 0.44091796875) proc.memory.percent‚18.61443519592285 disk.rbd0.in‚0 disk.nvme0n1.out‚ 10831.34375! disk.sda2.out‚136.74755859375 cpu‚25.507057367886507 disk.md0.in‚ 0.0078125 disk.md0p1.in‚ 0.0078125 disk.nvme1n1.in‚4.4375 network.sent‚ 1.511938e+06 disk.sda.out‚136.74755859375! disk.sdb2.out‚136.74755859375 disk.md0p1.out‚131.0625 disk.sda.in‚ 0.0078125 disk.sdb1.in‚0 disk.sdb1.out‚0 disk.sdb2.in‚0 disk.md0.out‚131.0625" disk.nvme1n1.out‚ 7531.31640625% disk./.usageGB‚2981.0727348327637 disk.dm-0.in‚ 4.69140625 disk.rbd2.in‚0îGf‹2ô - _wandbruntime‚1439  _runtime‚1439qjÆÐÄõ ¾ % _timestamp‚1.7884866404092824e+09 _runtime‚ 1439.51089033 _step‚61 & lr-AdamW/pg1‚0.0011337275636599468 & lr-AdamW/pg2‚0.0011337275636599468 trainer/global_step‚799ŽûÄôÈö  _runtime‚ 1439.51089033 _step‚61 & lr-AdamW/pg1‚0.0011337275636599468 & lr-AdamW/pg2‚0.0011337275636599468 trainer/global_step‚799 % _timestamp‚1.7884866404092824e+09=š0ÿÁ2÷ - _wandbruntime‚1439  _runtime‚1439´¢cí'ø çO NFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚330 RCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 2539798100 VFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 37073453056 UFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 3217031168 IDeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚23680 M=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 38610163200 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚19688 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 MDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚2147 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚19688 $ _timestamp‚1.788486640410007e+09 [KDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 37073453056 R>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚177592923002368 TDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 35913877504 WEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 1915138073088 M@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚26435642 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 YEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚170589102662149 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚25427968 PCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚11448575 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚25427968 H?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚1300 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 P@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 36985723393 RBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 40290484224 ZHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 1915138073088 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 `LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚168673876949712 VBDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated‚177595464792576 O@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 2541790208 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚3563886 YFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚38047258947072 _MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 2396302748160 ZHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 1912711342305 ]JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚38047370054144 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 KBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚1534 L?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚11448569 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚19682 `MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚35650956198912 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 VGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 3140738480 A8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚3681 E8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚26415954 G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚2684 SDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 3142752256 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 111107072 YJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 2516362240 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚23979 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 963120128 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚82009088 UADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚170586562864049 \HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚168673851521744 \HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚175677784929280 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 P@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 38610163200 NFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚977 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 561701376 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 F=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚3681 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚25427968 WGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 35913877504 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚2684 A8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚3681 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 ZKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 3217031168 UADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚177592923002368 IDeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 YEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚175677784929280 L@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚9065040 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚3 TDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 40290484224 _runtime‚1439.514932238 [IDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 1917654435328 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚6 YEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚177595464792576 PDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚9065634 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚594 ^LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 1915225712437 L?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚14967385 ]IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚175677810357248 ^LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 1917654435328 TGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚14987067 KBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚2147 ¼q™_ù ëO UFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 3217031168 IDeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚23680 M=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 38610163200 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚19688 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 MDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚2147 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚19688 $ _timestamp‚1.788486640410007e+09 [KDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 37073453056 R>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚177592923002368 TDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 35913877504 WEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 1915138073088 M@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚26435642 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 YEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚170589102662149 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚25427968 PCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚11448575 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚25427968 H?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚1300 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 P@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 36985723393 RBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 40290484224 ZHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 1915138073088 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 `LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚168673876949712 VBDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated‚177595464792576 O@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 2541790208 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚3563886 YFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚38047258947072 _MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 2396302748160 ZHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 1912711342305 ]JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚38047370054144 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 KBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚1534 L?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚11448569 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚19682 `MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚35650956198912 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 VGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 3140738480 A8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚3681 E8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚26415954 G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚2684 SDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 3142752256 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 111107072 YJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 2516362240 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚23979 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 963120128 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚82009088 UADeviceStatsMonitor.on_tr††‰ ’ain_batch_start/requested_bytes.all.freed‚170586562864049 \HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚168673851521744 \HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚175677784929280 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 P@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 38610163200 NFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚977 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 561701376 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 F=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚3681 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚25427968 WGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 35913877504 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚2684 A8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚3681 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 ZKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 3217031168 UADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚177592923002368 IDeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 YEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚175677784929280 L@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚9065040 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚3 TDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 40290484224 _runtime‚1439.514932238 [IDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 1917654435328 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚6 YEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚177595464792576 PDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚9065634 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚594 ^LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 1915225712437 L?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚14967385 ]IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚175677810357248 ^LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 1917654435328 TGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚14987067 KBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚2147 NFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚330 RCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 2539798100 VFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 37073453056>v¨1ëLú ‚(:Á ðÇèÔŽôϱ disk.sda.out‚137.98193359375 disk.rbd0.in‚0 disk.nvme0n1.out‚ 10898.875 disk.dm-0.in‚ 4.69140625 disk.sdb1.in‚0$ memory_percent‚99.95126724243164 network.recv‚525108 disk.rbd2.out‚ 0.44091796875 disk.sdb.in‚0 disk.nvme0n1.in‚ 0.25390625 cpu‚25.506109256937776 disk.rbd2.in‚0 disk.sdb.out‚137.98193359375 disk.sda2.in‚ 0.0078125" disk.nvme1n1.out‚ 7660.52734375 disk.sda.in‚ 0.0078125 disk.sdb1.out‚0" proc.memory.rssMB‚ 3049.7890625% disk./.usageGB‚2981.0718269348145 disk.md0.in‚ 0.0078125! disk.sdb2.out‚137.98193359375 disk.md0p1.out‚ 132.23046875 disk.nvme1n1.in‚4.4375) proc.memory.percent‚18.61443519592285 proc.cpu.threads‚17 network.sent‚ 1.515923e+06 disk.sda1.in‚0 disk.sda1.out‚0 disk.md0p1.in‚ 0.0078125 disk.dm-0.out‚18559.40234375 disk.md0.out‚ 132.23046875! disk.sda2.out‚137.98193359375 disk.rbd0.out‚ 0.341796875$ proc.memory.availableMB‚7.984375) disk./.usagePercent‚83.35201595658627 disk.sdb2.in‚0w‡1.”û ‚(:‰ ðÇèԵŴ gpu.0.gpu‚94.0 gpu.0.memory‚22+ gpu.0.memoryAllocated‚47.54737854003906* gpu.0.memoryAllocatedBytes‚ 40842887168 gpu.0.temp‚45.0 gpu.0.powerWatts‚124.661' gpu.0.enforcedPowerLimitWatts‚400.0 gpu.0.powerPercent‚31.16525 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0ïd'Ÿ2ü - _wandbruntime‚1441  _runtime‚1441E ¥ø&ý òM PDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 38610163200 K>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚26467900 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 561701376 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 S?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚170794431660279 XIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 3217031168 T@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚177811766972416 BDeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 TEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 2516375552 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 WGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 37073453056 G:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚26467900 M>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 2541803520 S?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚177809225168896 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1300 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚19714 C6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚26448186 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚23979 ?6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚3681 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚11462459 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚77801472 TDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 37073453056 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚23680 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚3568252 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 106899456 I@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚2147 NADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚15005435 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚25427968 F8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 D;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚3681 B9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚3681 F=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚2147 PADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 2541803520 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚25427968 \JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 1920010544640 VGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 3217031168 ZFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚175891730999808 UEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 35913877504 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚23979 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚25427968 % _timestamp‚1.7884866419522622e+09 EDeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 RBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 35913877504 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚3 PADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 2539798308 F=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚1534 XFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 1917494169088 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚19708 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚590 J>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚9075959 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 N>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 36985723393 WHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 2514370340 ^JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚175891756427776 ]KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 2399234115072 KBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚2147 QBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 3142752256 WDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚38094320638976 WCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚170796971458587 \JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 1917578848379 YGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 1920010544640 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚6 NADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚14985727 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚587 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 963120128 aODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 2399311916544 TEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 3140738480 ?6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚3681 NADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚11462465 UCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 1917494169088 TEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 3142752256 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚5508297 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 M=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 40290484224 I@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚1534#sôUü&þ öM WHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 2516375552 ^KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚35695086523904 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 D;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚3681 B9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚3681 F=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚2147 PADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 2541803520 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚25427968 \JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 1920010544640 VGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 3217031168 ZFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚175891730999808 UEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 35913877504 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚23979 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚25427968 % _timestamp‚1.7884866419522622e+09 EDeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 RBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 35913877504 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚3 PADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 2539798308 F=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚1534 XFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 1917494169088 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚19708 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚590 J>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚9075959 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 N>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 36985723393 WHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 2514370340 ^JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚175891756427776 ]KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 2399234115072 KBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚2147 QBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 3142752256 WDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚38094320638976 WCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚170796971458587 \JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 1917578848379 YGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 1920010544640 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚6 NADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚14985727 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚587 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 963120128 aODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 2399311916544 TEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 3140738480 ?6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚3681 NADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚11462465 UCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 1917494169088 TEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 3142752256 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚5508297 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 M=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 40290484224 I@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚1534 PDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 38610163200 K>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚26467900 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 561701376 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 S?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚170794431660279 XIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 3217031168 T@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚177811766972416 BDeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 TEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 2516375552 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 WGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 37073453056 G:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚26467900 M>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 2541803520 S?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚177809225168896 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1300 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚19714 C6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚26448186 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚23979 ?6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚3681 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚11462459 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚77801472 TDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 37073453056 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚23680 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚3568252 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 106899456 I@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚2147 NADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚15005435 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚25427968 FDeviceStatsMonitor.on_train_batch_start/allocation.all.current‚19688 NFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚977 UADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚180980029847557 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 PCDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated‚15907519 YJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 2516362240 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚3782186 BDeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 dQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚37857501546496 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚69426176 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚23979 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚19682 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚23680 TGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚12143075 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚2684 UHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚98524160 P@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 36985723393 YJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 2514370132 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 VFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 37073453056 PDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚9611966 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 MDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚2147 YEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚186375088455680 H?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚2147 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚25427968 PCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚12143069 WGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 34283657216 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚19688 ZHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 2032970701312 PCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚15887837 A8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚3681 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 \HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚178949634546544 A8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚3681 IDeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚2684 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚567 TEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 1513750016 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 L?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚15887837 SDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 3142752256 IDeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚188408059156992 ^LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 2035487063552 M@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚28050594 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚6 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 963120128 cQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 2543056349696 _runtime‚1526.995017206 PCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚12143075 VGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 3142752256 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚3 O?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 40290484224 H?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚1300 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 YEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚188410600947200 P@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 38610163200 RCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 2539798100 UFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 3217031168 TGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚15907519 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 _step‚66 þñ'‘ ëO TDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 40290484224 YIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 37073453056 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 [IDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 2035487063552 RCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 2541790208 E8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚28030906 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚25427968 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚23979 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚19682 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚23680 YFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚40400459372032 MDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚1534 `LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚186375113883648 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 L?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚12143069 ]IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚186375113883648 NFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚330 D>DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 dQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚37857501546496 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚69426176 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚23979 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚19682 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚23680 TGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚12143075 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚2684 UHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚98524160 P@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 36985723393 YJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 2514370132 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 VFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 37073453056 PDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚9611966 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 MDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚2147 YEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚186375088455680 H?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚2147 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚25427968 PCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚12143069 WGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 34283657216 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚19688 ZHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 2032970701312 PCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚15887837 A8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚3681 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 \HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚178949634546544 A8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚3681 IDeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚2684 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚567 TEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 1513750016 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 L?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚15887837 SDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 3142752256 IDeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚188408059156992 ^LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 2035487063552 M@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚28050594 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚6 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 963120128 cQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 2543056349696 _runtime‚1526.995017206 PCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚12143075 VGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 3142752256 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚3 O?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 40290484224 H?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚1300 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 YEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚188410600947200 P@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 38610163200 RCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 2539798100 UFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 3217031168 TGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚15907519 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 _step‚66 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚29097984 ZKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 3217031168 H?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚1534 TDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 35913877504 WEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 2032970701312 UADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚188408059156992 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚19688 NFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚977 UADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚180980029847557 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 PCDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated‚15907519 YJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 2516362240 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚3782186 BDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 36985723393 ZFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚179155150207040 D;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚3681 I@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚2147 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚25427968 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚69412864 I@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚1534 NADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚12156965 K>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚28082852 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 WGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 37073453056 _step‚67 XFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 2035326797312 BDeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 2541803520 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚6 J=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚15906179 PADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 2541803520 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 T@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚188626903127040 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 YIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 37073453056 QBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 3142752256 NADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚12156959 SFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚98510848 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 TDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 37073453056 VGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 3217031168 SDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 3217031168 WHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 2516375552 ^KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚37901602773504 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 577553920 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚3786552 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚5836354 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 P@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 40290484224 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 WCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚186589034526208 aODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 2546036313600 PADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 2539798308 XIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 3217031168 REDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚12156965 NBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚9622906 \JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 2035262807087 YGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 2037843172864 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚6 REDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚15925887 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚29097984 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 ?6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚3681 F=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚2147 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 F=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚1534 NADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚15925887 PDeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚9622347 XFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 2032748436747 KBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚2147 WCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚181190438442095 ^JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚179155175635008 LDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚33Ûh§– 0 G:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚28082852 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚12156959 WHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 2514370340 _runtime‚1528.580645835 TEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 2516375552 UCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 2035326797312 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1300 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 ?6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚3681 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 % _timestamp‚1.7884867294787989e+09 EDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 38610163200 ^JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚186589059954176 TEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 3142752256 FDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚23979 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚559 M=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 40290484224 B9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚3681 [GDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚186589059954176 WCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚188626903127040 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 9631201286ÄIû&” õM D;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚3681 I@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚2147 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚25427968 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚69412864 I@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚1534 NADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚12156965 K>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚28082852 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 WGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 37073453056 _step‚67 XFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 2035326797312 BDeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 2541803520 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚6 J=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚15906179 PADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 2541803520 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 T@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚188626903127040 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 YIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 37073453056 QBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 3142752256 NADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚12156959 SFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚98510848 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 TDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 37073453056 VGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 3217031168 SDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 3217031168 WHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 2516375552 ^KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚37901602773504 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 577553920 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚3786552 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚5836354 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 P@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 40290484224 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 WCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚186589034526208 aODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 2546036313600 PADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 2539798308 XIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 3217031168 REDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚12156965 NBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚9622906 \JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 2035262807087 YGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 2037843172864 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚6 REDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚15925887 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚29097984 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 ?6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚3681 F=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚2147 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 F=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚1534 NADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚15925887 PDeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚9622347 XFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 2032748436747 KBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚2147 WCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚181190438442095 ^JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚179155175635008 LDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚330 G:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚28082852 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚12156959 WHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 2514370340 _runtime‚1528.580645835 TEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 2516375552 UCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 2035326797312 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1300 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 ?6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚3681 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 % _timestamp‚1.7884867294787989e+09 EDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 38610163200 ^JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚186589059954176 TEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 3142752256 FDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚23979 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚559 M=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 40290484224 B9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚3681 [GDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚186589059954176 WCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚188626903127040 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 963120128 RBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 35913877504 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚3786549 [HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚40447668185088 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 N>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 36985723393 ZFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚179155150207040CUö¦^2• - _wandbruntime‚1528  _runtime‚1528æ½B]– × /loss/FRt3/assignment_loss‚3.630931577125153 2loss/SRqqt2/assignment_loss‚1.0833508112974115 1loss/SRqqt4/detection_loss‚1.7566269172295448 2loss/SRqqt3/assignment_loss‚1.9708137509982977 1loss/FBt1/assignment_loss‚0.36463723886005867 1loss/SRqqt2/detection_loss‚0.9163687509678354 /loss/FBt2/detection_loss‚0.6695167636045966 2loss/SRqqt4/assignment_loss‚1.7472643495327043 .loss/FRt3/detection_loss‚2.888730793804317 /loss/FRt4/detection_loss‚2.3995209971354123 /loss/FBt4/detection_loss‚3.1490739782651267 $ _timestamp‚1.788486729480477e+09 2loss/SRqqt1/assignment_loss‚1.5576876975399883 epoch‚1 trainer/global_step‚849 _runtime‚1528.581690959 1loss/FBt3/assignment_loss‚0.24196878552538692 1loss/SRqqt3/detection_loss‚2.5668435336938544 &loss/total_loss‚46.008897982527515 /loss/FRt4/assignment_loss‚3.050339398639543 /loss/FRt1/assignment_loss‚3.737524664402008 1loss/FBt4/assignment_loss‚0.07618681449313651 /loss/FRt1/detection_loss‚2.3079841494560243 1loss/SRqqt1/detection_loss‚2.4342357014354907 /loss/FBt1/detection_loss‚2.9348152371553273 _step‚68 1loss/FBt2/assignment_loss‚0.14357242640424345 /loss/FBt3/detection_loss‚3.2848967711130777 /loss/FRt2/detection_loss‚1.6949293234330767 0loss/FRt2/assignment_loss‚1.4010775504158992ÉØÐa— Û /loss/FBt4/detection_loss‚3.1490739782651267 $ _timestamp‚1.788486729480477e+09 2loss/SRqqt1/assignment_loss‚1.5576876975399883 epoch‚1 trainer/global_step‚849 _runtime‚1528.581690959 1loss/FBt3/assignment_loss‚0.24196878552538692 1loss/SRqqt3/detection_loss‚2.5668435336938544 &loss/total_loss‚46.008897982527515 /loss/FRt4/assignment_loss‚3.050339398639543 /loss/FRt1/assignment_loss‚3.737524664402008 1loss/FBt4/assignment_loss‚0.07618681449313651 /loss/FRt1/detection_loss‚2.3079841494560243 1loss/SRqqt1/detection_loss‚2.4342357014354907 /loss/FBt1/detection_loss‚2.9348152371553273 _step‚68 1loss/FBt2/assignment_loss‚0.14357242640424345 /loss/FBt3/detection_loss‚3.2848967711130777 /loss/FRt2/detection_loss‚1.6949293234330767 0loss/FRt2/assignment_loss‚1.4010775504158992 /loss/FRt3/assignment_loss‚3.630931577125153 2loss/SRqqt2/assignment_loss‚1.0833508112974115 1loss/SRqqt4/detection_loss‚1.7566269172295448 2loss/SRqqt3/assignment_loss‚1.9708137509982977 1loss/FBt1/assignment_loss‚0.36463723886005867 1loss/SRqqt2/detection_loss‚0.9163687509678354 /loss/FBt2/detection_loss‚0.6695167636045966 2loss/SRqqt4/assignment_loss‚1.7472643495327043 .loss/FRt3/detection_loss‚2.888730793804317 /loss/FRt4/detection_loss‚2.3995209971354123DâÎ ”˜ ‚(:‰ ÊÈèÔ¹—í¯ gpu.0.gpu‚97.0 gpu.0.memory‚17+ gpu.0.memoryAllocated‚47.54737854003906* gpu.0.memoryAllocatedBytes‚ 40842887168 gpu.0.temp‚45.0 gpu.0.powerWatts‚227.929' gpu.0.enforcedPowerLimitWatts‚400.0 gpu.0.powerPercent‚56.98225 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0Û„J™ ‚(:¿ ÊÈèÔÛ»¡± disk.md0.out‚ 140.02734375 disk.md0p1.in‚ 0.0078125 disk.rbd2.out‚ 0.50390625" disk.nvme1n1.out‚ 8085.31640625 disk.sda.out‚146.14892578125 disk.dm-0.in‚ 4.69140625) disk./.usagePercent‚83.34502542531101 disk.sdb1.out‚0 disk.sdb2.in‚0 disk.sda1.in‚0 disk.sdb.out‚146.14892578125 disk.nvme1n1.in‚4.4375 disk.rbd0.in‚0 disk.nvme0n1.in‚ 0.25390625 disk.md0p1.out‚ 140.02734375 disk.rbd0.out‚ 0.341796875 disk.nvme0n1.out‚ 11450.625$ memory_percent‚99.95341300964355 proc.cpu.threads‚17% disk./.usageGB‚2980.8218116760254 disk.sdb1.in‚0 disk.sda1.out‚0 disk.sda2.in‚ 0.0078125" proc.memory.rssMB‚ 3049.7890625! disk.sda2.out‚146.14892578125 disk.sda.in‚ 0.0078125 network.sent‚ 1.701865e+06 network.recv‚550459 disk.rbd2.in‚0% proc.memory.availableMB‚ 7.6328125) proc.memory.percent‚18.61443519592285 cpu‚25.498852953121688 disk.md0.in‚ 0.0078125! disk.sdb2.out‚146.14892578125 disk.sdb.in‚0 disk.dm-0.out‚19535.94140625×f„”š ‚(:‰ ÙÈèԣΚ° gpu.0.gpu‚70.0 gpu.0.memory‚16+ gpu.0.memoryAllocated‚47.54737854003906* gpu.0.memoryAllocatedBytes‚ 40842887168 gpu.0.temp‚47.0 gpu.0.powerWatts‚320.349' gpu.0.enforcedPowerLimitWatts‚400.0 gpu.0.powerPercent‚80.08725 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0!.¨Ò=› ‚(:² ÙÈèÔÄÌÀ± network.sent‚ 1.777792e+06( disk./.usagePercent‚83.3225744499047 disk.sdb2.out‚ 147.5703125 disk.sdb.out‚ 147.5703125 disk.rbd0.in‚0 disk.sdb2.in‚0 disk.sda1.in‚0 disk.sda1.out‚0 disk.md0p1.in‚ 0.0078125 disk.sdb1.out‚0 disk.sda.out‚ 147.5703125 disk.dm-0.out‚19678.80078125 disk.sdb1.in‚0 disk.md0p1.out‚ 141.37109375 disk.sdb.in‚0 disk.sda.in‚ 0.0078125 disk.nvme0n1.in‚ 0.25390625) proc.memory.percent‚18.61443519592285 proc.cpu.threads‚17 disk.rbd2.out‚ 0.50390625 disk.sda2.out‚ 147.5703125$ memory_percent‚99.95195865631104$ disk./.usageGB‚2980.018856048584 disk.sda2.in‚ 0.0078125 disk.nvme1n1.in‚4.4375 disk.rbd0.out‚ 0.341796875 disk.dm-0.in‚ 4.69140625& proc.memory.availableMB‚ 7.87109375" proc.memory.rssMB‚ 3049.7890625 disk.md0.out‚ 141.37109375! disk.nvme1n1.out‚ 8166.7421875# disk.nvme0n1.out‚11512.05859375 network.recv‚555942 disk.md0.in‚ 0.0078125 disk.rbd2.in‚0 cpu‚25.498059543201464W¦s¨žœ ‚(:“ èÈèÔ惛° gpu.0.gpu‚98.0 gpu.0.memory‚23+ gpu.0.memoryAllocated‚47.54737854003906* gpu.0.memoryAllocatedBytes‚ 40842887168 gpu.0.temp‚45.0 gpu.0.powerWatts‚127.106' gpu.0.enforcedPowerLimitWatts‚400.0) gpu.0.powerPercent‚31.776499999999995 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0˜+‚H ‚(:½ èÈèÔàŒÔ°) disk./.usagePercent‚83.32253338559984 disk.md0.out‚ 143.01953125 disk.nvme1n1.in‚4.4375 disk.nvme1n1.out‚ 8321.765625 disk.rbd0.out‚ 0.341796875 disk.sdb1.out‚0% proc.memory.availableMB‚ 7.9296875 network.recv‚560643 disk.rbd0.in‚0 disk.sdb2.out‚149.2724609375 proc.cpu.threads‚17 disk.rbd2.in‚0 disk.nvme0n1.in‚ 0.25390625 disk.sdb2.in‚0 disk.sda1.in‚0 disk.sda1.out‚0 disk.md0p1.out‚ 143.01953125" proc.memory.rssMB‚ 3049.7890625 network.sent‚ 1.781777e+06 disk.sda2.in‚ 0.0078125 disk.sdb1.in‚0 disk.md0.in‚ 0.0078125 cpu‚25.49621311160203 disk.sda2.out‚149.2724609375 disk.sda.in‚ 0.0078125 disk.dm-0.in‚ 4.69140625 disk.md0p1.in‚ 0.0078125$ memory_percent‚99.95160102844238) proc.memory.percent‚18.61443519592285% disk./.usageGB‚2980.0173873901367 disk.rbd2.out‚ 0.50390625# disk.nvme0n1.out‚11615.05859375 disk.dm-0.out‚19936.82421875 disk.sdb.in‚0 disk.sdb.out‚149.2724609375 disk.sda.out‚149.2724609375=\užž ‚(:“ ÷ÈèÔÌåì¯ gpu.0.gpu‚97.0 gpu.0.memory‚17+ gpu.0.memoryAllocated‚47.54737854003906* gpu.0.memoryAllocatedBytes‚ 40842887168 gpu.0.temp‚45.0 gpu.0.powerWatts‚230.204' gpu.0.enforcedPowerLimitWatts‚400.0) gpu.0.powerPercent‚57.550999999999995 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0;‹1êHŸ ‚(:½ ÷ÈèÔÍæ° network.sent‚ 1.789468e+06 disk.sda1.out‚0 disk.rbd2.out‚ 0.50390625 disk.sdb2.in‚0% proc.memory.availableMB‚ 7.2734375" proc.memory.rssMB‚ 3049.7890625 disk.md0.out‚ 144.2421875 disk.rbd2.in‚0! disk.sda2.out‚150.56787109375 cpu‚25.494524704145636 proc.cpu.threads‚17 disk.sdb.in‚0 disk.sdb1.out‚0 disk.md0p1.out‚ 144.2421875 disk.sdb1.in‚0 disk.md0.in‚ 0.0078125 disk.sdb.out‚150.56787109375 disk.sda2.in‚ 0.0078125 disk.nvme1n1.in‚4.4375 disk.rbd0.in‚0) disk./.usagePercent‚83.30778831365774 disk.sda1.in‚0 disk.sda.in‚ 0.0078125 disk.sda.out‚150.56787109375 disk.rbd0.out‚ 0.341796875) proc.memory.percent‚18.61443519592285 disk.md0p1.in‚ 0.0078125 disk.nvme0n1.in‚ 0.25390625" disk.nvme0n1.out‚ 11615.2890625 disk.dm-0.in‚ 4.69140625 disk.dm-0.out‚ 20033.3046875$ memory_percent‚99.95560646057129 network.recv‚564014$ disk./.usageGB‚2979.490032196045 disk.nvme1n1.out‚ 8418.015625! disk.sdb2.out‚150.56787109375c%ßQ‘  ‚(:† †ÉèÔùŽ©° gpu.0.gpu‚32.0 gpu.0.memory‚9+ gpu.0.memoryAllocated‚47.54737854003906* gpu.0.memoryAllocatedBytes‚ 40842887168 gpu.0.temp‚45.0 gpu.0.powerWatts‚116.916' gpu.0.enforcedPowerLimitWatts‚400.0 gpu.0.powerPercent‚29.229 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚03zîA¡ ‚(:¶ †ÉèԙŰ) disk./.usagePercent‚83.30639212729226 disk.sdb2.out‚152.2001953125 disk.sdb.out‚152.2001953125 disk.nvme1n1.out‚ 8462.140625 disk.sda.out‚152.2001953125 disk.rbd0.out‚ 0.341796875 disk.dm-0.out‚ 20077.4921875 disk.sda1.in‚0 disk.md0.out‚ 145.8203125$ memory_percent‚99.95613098144531) proc.memory.percent‚18.61443519592285 disk.sda1.out‚0 disk.md0.in‚ 0.0078125 disk.md0p1.out‚ 145.8203125" proc.memory.availableMB‚7.1875" proc.memory.rssMB‚ 3049.7890625 cpu‚25.492726215666373 network.sent‚ 1.795383e+06$ disk./.usageGB‚2979.440097808838 disk.sda2.in‚ 0.0078125 network.recv‚568132 disk.rbd0.in‚0" disk.nvme0n1.out‚ 11615.3515625 disk.sdb.in‚0 proc.cpu.threads‚17 disk.sda.in‚ 0.0078125 disk.nvme0n1.in‚ 0.25390625 disk.dm-0.in‚ 4.69140625 disk.sdb1.out‚0 disk.nvme1n1.in‚4.4375 disk.sdb1.in‚0 disk.sdb2.in‚0 disk.rbd2.in‚0 disk.rbd2.out‚ 0.50390625 disk.sda2.out‚152.2001953125 disk.md0p1.in‚ 0.0078125Z¹G¢ ‚(:¼ •ÉèÔ装± network.sent‚ 1.801291e+06) disk./.usagePercent‚83.30554012296167 disk.sda1.in‚0 disk.sda1.out‚0 disk.rbd2.in‚0" disk.nvme0n1.out‚ 11616.7890625 cpu‚25.49093572882298" proc.memory.rssMB‚ 3049.7890625 network.recv‚572213! disk.sdb2.out‚153.87646484375 disk.md0p1.in‚ 0.0078125 disk.sda2.in‚ 0.0078125! disk.sda2.out‚153.87646484375 disk.nvme1n1.in‚4.4375 disk.nvme1n1.out‚ 8534.265625) proc.memory.percent‚18.61443519592285 disk.sdb2.in‚0 disk.md0.out‚ 147.4296875 disk.md0p1.out‚ 147.4296875 disk.rbd0.in‚0 disk.rbd0.out‚ 0.341796875 disk.sda.in‚ 0.0078125 disk.sdb1.out‚0 proc.cpu.threads‚17# disk./.usageGB‚2979.40962600708 disk.nvme0n1.in‚ 0.25390625 disk.dm-0.in‚ 4.69140625$ memory_percent‚99.95462894439697 disk.md0.in‚ 0.0078125 disk.rbd2.out‚ 0.50390625 disk.sdb.in‚0 disk.sdb.out‚153.87646484375 disk.sda.out‚153.87646484375 disk.dm-0.out‚ 20151.0546875 disk.sdb1.in‚0& proc.memory.availableMB‚ 7.433593751 ‚(:’ •ÉèÔ¿û–± gpu.0.gpu‚97.0 gpu.0.memory‚17+ gpu.0.memoryAllocated‚47.54737854003906* gpu.0.memoryAllocatedBytes‚ 40842887168 gpu.0.temp‚46.0 gpu.0.powerWatts‚243.136' gpu.0.enforcedPowerLimitWatts‚400.0( gpu.0.powerPercent‚60.78399999999999 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0¶Àã\2¤ - _wandbruntime‚1609  _runtime‚1609¢4òÅ¥ ¿ & lr-AdamW/pg1‚0.0011290903966390995 & lr-AdamW/pg2‚0.0011290903966390995 trainer/global_step‚899 % _timestamp‚1.7884868100431576e+09 _runtime‚1609.144610379 _step‚69§‡‘‰É¦ à & lr-AdamW/pg1‚0.0011290903966390995 & lr-AdamW/pg2‚0.0011290903966390995 trainer/global_step‚899 % _timestamp‚1.7884868100431576e+09 _runtime‚1609.144610379 _step‚69E·z2§ - _wandbruntime‚1609  _runtime‚1609Ê6u¨ êO \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚73620480 D>DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 NFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚330 IDeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚2684 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚23680 WGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 34283657216 YJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 2514370132 VGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 3140738480 D;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚3681 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚25427968 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚6159279 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 H?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚2147 RCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 2541790208 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚599 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚29097984 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 577553920 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 PCDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated‚16827971 H?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚1300 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 ZHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 2148079259721 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚19688 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 XIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 3217031168 MDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚1534 _MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 2690388902400 [KDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 37073453056 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚19682 dQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚40064017796096 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 H?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚1534 O@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 2541790208 QDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚10159765 YFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚42754377600512 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚19688 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚23680 YJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 2516362240 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 BDeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚199223195311616 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚6 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚3 M@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚29665546 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚4000486 KBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚2147 ]JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚42754480318976 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚25427968 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚6 M=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 38610163200 SDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 3142752256 WGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 35913877504 VGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 3142752256 SGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚6158683Y`ô'© îO O@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 2541790208 QDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚10159765 YFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚42754377600512 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚19688 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚23680 YJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 2516362240 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 BDeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚199223195311616 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚6 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚3 M@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚29665546 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚4000486 KBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚2147 ]JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚42754480318976 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚25427968 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚6 M=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 38610163200 SDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 3142752256 WGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 35913877504 VGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 3142752256 SGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚6158683 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚73620480 D>DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 NFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚330 IDeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚2684 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚23680 WGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 34283657216 YJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 2514370132 VGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 3140738480 D;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚3681 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚25427968 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚6159279 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 H?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚2147 RCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 2541790208 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚599 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚29097984 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 577553920 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 PCDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated‚16827971 H?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚1300 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 ZHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 2148079259721 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚19688 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 XIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 3217031168 MDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚1534 _MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 2690388902400 [KDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 37073453056 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚19682 dQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚40064017796096 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 H?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚1534FϤ—o2ª - _wandbruntime‚1610  _runtime‚1610ÐÕ ù&« óM ZFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚189430933231840 RBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 40290484224 J=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚16826631 FDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 36985723393 NADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚12851465 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 577553920 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 TEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 3140738480 FDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 G:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚29697804 PADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 2541803520 S?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚199439497478144 ^KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚40108119023104 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚6165295 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 P@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 40290484224 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 C6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚29678090 WHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 2516375552 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 UEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 34283657216 \JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 2152946765795 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚19714 trainer/global_step‚899 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚6 YIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 37073453056 WCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚197286338052608 ZFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚197286338052608 OBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚10170728 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚581 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 SDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 3217031168 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 \JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 2155675801088 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚3 XIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 3217031168 KBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚1534 NADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚12851459 K>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚10170144 WHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 2514370340 $ _timestamp‚1.788486811552762e+09 K;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 38610163200 N>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 38610163200 XFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 2153159425536 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚25427968 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 TEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 2516375552 TEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 3142752256 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚2684 ]KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 2693335513600 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚23979 REDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚12851465 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1300 aODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 2693411217920 BDeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚29697804 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 M=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 40290484224 KBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚2147 M>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 2541803520 [GDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚197286363480576 UCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 2153159425536 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 963120128 WGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 37073453056 B9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚3681 UEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 35913877504 VGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 3217031168 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚4004852 XFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 2150432395455 F=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚2147 T@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚199442039281664 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚25427968 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚23979 NADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚16826631Ò ‰$¬ ÷M =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 M=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 40290484224 KBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚2147 M>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 2541803520 [GDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚197286363480576 UCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 2153159425536 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 963120128 WGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 37073453056 B9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚3681 UEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 35913877504 VGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 3217031168 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚4004852 XFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 2150432395455 F=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚2147 T@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚199442039281664 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚25427968 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚23979 NADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚16826631 ZFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚189430933231840 RBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 40290484224 J=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚16826631 FDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 36985723393 NADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚12851465 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 577553920 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 TEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 3140738480 FDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 G:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚29697804 PADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 2541803520 S?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚199439497478144 ^KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚40108119023104 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚6165295 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 P@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 40290484224 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 C6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚29678090 WHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 2516375552 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 UEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 34283657216 \JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 2152946765795 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚19714 trainer/global_step‚899 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚6 YIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 37073453056 WCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚197286338052608 ZFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚197286338052608 OBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚10170728 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚581 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 SDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 3217031168 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 \JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 2155675801088 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚3 XIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 3217031168 KBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚1534 NADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚12851459 K>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚10170144 WHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 2514370340 $ _timestamp‚1.788486811552762e+09 K;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 38610163200 N>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 38610163200 XFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 2153159425536 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚25427968 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 TEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 2516375552 TEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 3142752256 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚2684 ]KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 2693335513600 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚23979 REDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚12851465 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1300 aODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 2693411217920 BDeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚29697804G4eýu2­ - _wandbruntime‚1610  _runtime‚1610-WùT® Î 2loss/SRqqt3/assignment_loss‚1.4871601316322018 1loss/FBt1/assignment_loss‚0.24479204536732546 1loss/FBt2/assignment_loss‚0.17140038161364632 .loss/FBt1/detection_loss‚3.345628274811639 .loss/FBt3/detection_loss‚2.611769851885344 $loss/total_loss‚44.2292535283534 /loss/FRt3/assignment_loss‚3.429412621847341 0loss/FBt3/assignment_loss‚0.0838559973306378 .loss/FRt1/detection_loss‚2.272280209685025 .loss/FRt2/detection_loss‚1.879484666342085 trainer/global_step‚899 _step‚72 0loss/FRt2/assignment_loss‚1.4869173074310476 2loss/SRqqt2/assignment_loss‚1.1394566473626677 1loss/SRqqt4/assignment_loss‚1.799894195701927 1loss/SRqqt4/detection_loss‚1.8293192714452744 epoch‚1 $ _timestamp‚1.788486811555124e+09 _runtime‚1610.656274442 0loss/FRt1/assignment_loss‚3.9092637259666234 /loss/FRt4/assignment_loss‚2.903165863765465 1loss/SRqqt2/detection_loss‚0.9783048291704548 1loss/SRqqt1/detection_loss‚2.3152054524099506 1loss/SRqqt3/detection_loss‚2.2654862680296968 /loss/FBt2/detection_loss‚0.7859976522814847 -loss/FBt4/detection_loss‚2.36261764482448 2loss/SRqqt1/assignment_loss‚1.6767754897072509 1loss/FBt4/assignment_loss‚0.09993278137163113 /loss/FRt3/detection_loss‚2.7201387714332257 .loss/FRt4/detection_loss‚2.430993446936974˜ê3ñX¯ Ò /loss/FRt3/assignment_loss‚3.429412621847341 0loss/FBt3/assignment_loss‚0.0838559973306378 .loss/FRt1/detection_loss‚2.272280209685025 .loss/FRt2/detection_loss‚1.879484666342085 trainer/global_step‚899 _step‚72 0loss/FRt2/assignment_loss‚1.4869173074310476 2loss/SRqqt2/assignment_loss‚1.1394566473626677 1loss/SRqqt4/assignment_loss‚1.799894195701927 1loss/SRqqt4/detection_loss‚1.8293192714452744 epoch‚1 $ _timestamp‚1.788486811555124e+09 _runtime‚1610.656274442 0loss/FRt1/assignment_loss‚3.9092637259666234 /loss/FRt4/assignment_loss‚2.903165863765465 1loss/SRqqt2/detection_loss‚0.9783048291704548 1loss/SRqqt1/detection_loss‚2.3152054524099506 1loss/SRqqt3/detection_loss‚2.2654862680296968 /loss/FBt2/detection_loss‚0.7859976522814847 -loss/FBt4/detection_loss‚2.36261764482448 2loss/SRqqt1/assignment_loss‚1.6767754897072509 1loss/FBt4/assignment_loss‚0.09993278137163113 /loss/FRt3/detection_loss‚2.7201387714332257 .loss/FRt4/detection_loss‚2.430993446936974 2loss/SRqqt3/assignment_loss‚1.4871601316322018 1loss/FBt1/assignment_loss‚0.24479204536732546 1loss/FBt2/assignment_loss‚0.17140038161364632 .loss/FBt1/detection_loss‚3.345628274811639 .loss/FBt3/detection_loss‚2.611769851885344 $loss/total_loss‚44.2292535283534H#UKfC° ‚(:¸ ¤ÉèÔÈïó® disk.dm-0.in‚ 4.69140625% disk./.usageGB‚2979.4746170043945 disk.nvme0n1.in‚ 0.25390625 disk.dm-0.out‚ 20179.296875" proc.memory.rssMB‚ 3049.7890625) proc.memory.percent‚18.61443519592285 cpu‚25.489111672953918 disk.sdb2.in‚0 disk.md0p1.out‚ 148.6953125 disk.rbd2.out‚ 0.50390625 disk.rbd0.in‚0 disk.sdb1.in‚0 proc.cpu.threads‚17 network.recv‚576683 disk.sda1.out‚0 disk.sdb.in‚0 disk.sda.in‚ 0.0078125 disk.nvme1n1.in‚4.4375 disk.rbd0.out‚ 0.341796875$ memory_percent‚99.95689392089844) disk./.usagePercent‚83.30735729844744 disk.md0.in‚ 0.0078125 disk.rbd2.in‚0 disk.sdb.out‚155.21142578125 disk.nvme0n1.out‚ 11616.84375 disk.sda2.in‚ 0.0078125 disk.nvme1n1.out‚ 8562.453125 disk.sdb1.out‚0 network.sent‚ 1.805234e+06! disk.sdb2.out‚155.21142578125 disk.sda1.in‚0 disk.md0p1.in‚ 0.0078125! disk.sda2.out‚155.21142578125" proc.memory.availableMB‚7.0625 disk.md0.out‚ 148.6953125 disk.sda.out‚155.21142578125HX."”± ‚(:‰ ¤ÉèÔÛÞ­¯ gpu.0.gpu‚98.0 gpu.0.memory‚17+ gpu.0.memoryAllocated‚47.54737854003906* gpu.0.memoryAllocatedBytes‚ 40842887168 gpu.0.temp‚45.0 gpu.0.powerWatts‚248.089' gpu.0.enforcedPowerLimitWatts‚400.0 gpu.0.powerPercent‚62.02225 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0Û™ ² ‚(:’ ³ÉèÔ•¾Õ® gpu.0.gpu‚79.0 gpu.0.memory‚18+ gpu.0.memoryAllocated‚47.54737854003906* gpu.0.memoryAllocatedBytes‚ 40842887168 gpu.0.temp‚41.0 gpu.0.powerWatts‚117.81' gpu.0.enforcedPowerLimitWatts‚400.0) gpu.0.powerPercent‚29.452499999999997 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0tQ Ý=³ ‚(:² ³ÉèԞˣ¯ disk.sdb.out‚156.51123046875 disk.dm-0.in‚ 4.69140625! disk.sdb2.out‚156.51123046875 disk.sda1.in‚0% disk./.usageGB‚2979.8174629211426 disk.rbd0.in‚0 disk.sdb2.in‚0 disk.rbd2.in‚0 disk.nvme1n1.out‚ 8745.59375 disk.sdb1.in‚0 disk.md0.in‚ 0.0078125 cpu‚25.48754121309883 disk.md0p1.in‚ 0.0078125 disk.nvme1n1.in‚4.4375 disk.rbd0.out‚ 0.341796875 disk.dm-0.out‚ 20362.8125) proc.memory.percent‚18.61443519592285 proc.cpu.threads‚17 network.recv‚581221" proc.memory.rssMB‚ 3049.7890625 disk.md0p1.out‚149.9375 disk.sda2.in‚ 0.0078125! disk.sda2.out‚156.51123046875 disk.sda.in‚ 0.0078125 disk.sda.out‚156.51123046875 disk.nvme0n1.in‚ 0.25390625 disk.sdb1.out‚0& proc.memory.availableMB‚ 6.78515625 disk.md0.out‚149.9375$ memory_percent‚99.95858669281006 network.sent‚ 1.891148e+06 disk.rbd2.out‚ 0.50390625 disk.sdb.in‚0 disk.nvme0n1.out‚ 11617.21875 disk.sda1.out‚0) disk./.usagePercent‚83.31694341377187ï@È ”´ ‚(:‰ ÂÉèÔíѶ® gpu.0.gpu‚60.0 gpu.0.memory‚14+ gpu.0.memoryAllocated‚47.54737854003906* gpu.0.memoryAllocatedBytes‚ 40842887168 gpu.0.temp‚42.0 gpu.0.powerWatts‚123.435' gpu.0.enforcedPowerLimitWatts‚400.0 gpu.0.powerPercent‚30.85875 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0_Ù‹›Gµ ‚(:¼ ÂÉèÔ²—ù® disk.sda1.in‚0 disk.sda.in‚ 0.0078125 network.sent‚ 1.897042e+06 disk.md0p1.in‚ 0.0078125 disk.md0p1.out‚ 151.62109375 disk.rbd2.in‚0 disk.nvme1n1.in‚4.4375 disk.nvme0n1.in‚ 0.25390625 disk.sdb1.in‚0 disk.sdb1.out‚0 disk.sda2.out‚158.2646484375 disk.sdb2.out‚158.2646484375 disk.sda1.out‚0! disk.nvme0n1.out‚ 11617.265625 disk.dm-0.in‚ 4.69140625 disk.md0.in‚ 0.0078125 disk.rbd2.out‚ 0.50390625 disk.sdb.in‚0" proc.memory.rssMB‚ 3049.7890625 cpu‚25.486319892540873) disk./.usagePercent‚83.32392434559924 disk.dm-0.out‚ 20571.5078125 disk.sda2.in‚ 0.0078125 disk.rbd0.out‚ 0.341796875% proc.memory.availableMB‚ 6.7890625 proc.cpu.threads‚17 disk.md0.out‚ 151.62109375 disk.sdb.out‚158.2646484375 disk.rbd0.in‚0$ memory_percent‚99.95856285095215) proc.memory.percent‚18.61443519592285 network.recv‚585580! disk.nvme1n1.out‚ 8954.2421875 disk.sda.out‚158.2646484375% disk./.usageGB‚2980.0671348571777 disk.sdb2.in‚0î`C`¶ ‚(:„ ÑÉèÔ‡õ»® gpu.0.gpu‚0.0 gpu.0.memory‚0+ gpu.0.memoryAllocated‚47.54737854003906* gpu.0.memoryAllocatedBytes‚ 40842887168 gpu.0.temp‚45.0 gpu.0.powerWatts‚281.62' gpu.0.enforcedPowerLimitWatts‚400.0 gpu.0.powerPercent‚70.405 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0OÌÕ×F· ‚(:» ÑÉèԣ兯 disk.md0.in‚ 0.0078125 disk.sda.out‚160.0634765625 disk.sdb1.in‚0 disk.sdb2.in‚0 disk.rbd2.in‚0 disk.nvme1n1.in‚4.4375 disk.rbd0.in‚0 network.sent‚ 1.902957e+06 disk.dm-0.out‚ 20706.6953125 disk.sda1.out‚0 disk.md0p1.out‚ 153.34765625 disk.rbd0.out‚ 0.341796875 network.recv‚589923$ disk./.usageGB‚2980.072124481201 disk.nvme0n1.in‚ 0.25390625" disk.nvme0n1.out‚ 11617.4609375 disk.sdb1.out‚0 disk.nvme1n1.out‚ 9089.234375% proc.memory.availableMB‚ 6.7734375 cpu‚25.484662456529037 disk.rbd2.out‚ 0.50390625 disk.sdb.in‚0 disk.sda2.in‚ 0.0078125 disk.sda2.out‚160.0634765625) disk./.usagePercent‚83.32406385757525 disk.sda1.in‚0 disk.md0.out‚ 153.34765625 proc.cpu.threads‚17 disk.dm-0.in‚ 4.69140625 disk.sda.in‚ 0.0078125) proc.memory.percent‚18.61443519592285 disk.md0p1.in‚ 0.0078125 disk.sdb.out‚160.0634765625$ memory_percent‚99.95865821838379" proc.memory.rssMB‚ 3049.7890625 disk.sdb2.out‚160.0634765625â…>á’¸ ‚(:‡ àÉèÔÙö® gpu.0.gpu‚97.0 gpu.0.memory‚17+ gpu.0.memoryAllocated‚47.54737854003906* gpu.0.memoryAllocatedBytes‚ 40842887168 gpu.0.temp‚45.0 gpu.0.powerWatts‚225.144' gpu.0.enforcedPowerLimitWatts‚400.0 gpu.0.powerPercent‚56.286 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0Âi¶{9¹ ‚(:® àÉèÔ×µï¯ disk.sdb2.in‚0 disk.sdb2.out‚ 161.28515625 disk.md0.in‚ 0.0078125 disk.md0p1.out‚ 154.515625 disk.sda.in‚ 0.0078125" proc.memory.rssMB‚ 3049.7890625$ disk./.usageGB‚2980.134792327881" disk.nvme0n1.out‚ 11617.5234375 disk.rbd2.in‚0 disk.sdb.out‚ 161.28515625 cpu‚25.482865499705465 disk.nvme1n1.in‚4.4375 disk.sda2.in‚ 0.0078125 disk.sda.out‚ 161.28515625 disk.rbd2.out‚ 0.50390625 disk.sda2.out‚ 161.28515625# memory_percent‚99.9575138092041 disk.sda1.in‚0! disk.nvme1n1.out‚ 9089.3359375) disk./.usagePercent‚83.32581607679694 disk.nvme0n1.in‚ 0.25390625 disk.dm-0.in‚ 4.69140625 disk.sdb1.out‚0 disk.md0.out‚ 154.515625 disk.md0p1.in‚ 0.0078125) proc.memory.percent‚18.61443519592285 disk.rbd0.in‚0 disk.dm-0.out‚ 20706.859375 disk.sdb1.in‚0 disk.sda1.out‚0 disk.sdb.in‚0 proc.cpu.threads‚17 disk.rbd0.out‚ 0.341796875% proc.memory.availableMB‚ 6.9609375 network.sent‚ 1.909165e+06 network.recv‚594621lìÃ>2º - _wandbruntime‚1692  _runtime‚1692UŠ—» ¼ % lr-AdamW/pg1‚0.001126068875223343 % lr-AdamW/pg2‚0.001126068875223343 trainer/global_step‚949 $ _timestamp‚1.788486893355301e+09 _runtime‚1692.457050789 _step‚73ýp^‰Æ¼ À trainer/global_step‚949 $ _timestamp‚1.788486893355301e+09 _runtime‚1692.457050789 _step‚73 % lr-AdamW/pg1‚0.001126068875223343 % lr-AdamW/pg2‚0.001126068875223343I—-©$2½ - _wandbruntime‚1692  _runtime‚1692®Àð'¾ êO A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚23979 PCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚13532075 O@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 2541790208 YJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 2516362240 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚6 PCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚17728741 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚23680 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 YEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚210040873256448 WGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 35913877504 IDeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚2684 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚6486846 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚19688 RCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 2541790208 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚587 ZHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 2265763218429 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚19682 `MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚42270504947712 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 [KDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 37073453056 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚23680 _MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 2837017525760 D>DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 IDeviceStatsMonitor.on_train_batch_start/allocation.all.current‚19688 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 577553920 TDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 40290484224 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 L?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚17728741 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚23979 NFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚330 `LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚199501226024112 [IDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 2271152320000 dQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚42270534045696 RCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 2539798100 VFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 37073453056 trainer/global_step‚949 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚3 BDeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚210038331466240 M@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚31280498 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 109009920 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚25427968 % _timestamp‚1.7884868933567345e+09 ]IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚207769720936448 YEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚207769695508480 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚25427968 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚4218786 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 _step‚74 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚2684 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 KBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚2147 H?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚1534 TDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 35913877504 SDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 3142752256 UADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚201766963814573LŠÏëô'¿ îO VFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 37073453056 trainer/global_step‚949 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚3 BDeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚210038331466240 M@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚31280498 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 109009920 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚25427968 % _timestamp‚1.7884868933567345e+09 ]IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚207769720936448 YEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚207769695508480 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚25427968 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚4218786 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 _step‚74 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚2684 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 KBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚2147 H?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚1534 TDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 35913877504 SDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 3142752256 UADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚201766963814573 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚23979 PCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚13532075 O@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 2541790208 YJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 2516362240 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚6 PCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚17728741 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚23680 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 YEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚210040873256448 WGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 35913877504 IDeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚2684 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚6486846 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚19688 RCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 2541790208 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚587 ZHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 2265763218429 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚19682 `MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚42270504947712 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 [KDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 37073453056 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚23680 _MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 2837017525760 D>DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 IDeviceStatsMonitor.on_train_batch_start/allocation.all.current‚19688 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 577553920 TDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 40290484224 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 L?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚17728741 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚23979 NFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚330 `LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚199501226024112 [IDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 2271152320000 dQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚42270534045696 RCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 2539798100JøxJ2À - _wandbruntime‚1694  _runtime‚1694%7$–íÁ óM =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 _step‚75 ZFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚207983641579008 REDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚13545965 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚6492756 S?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚201974832610803 P@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 40290484224 ?6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚3681 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚19708 PADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 2541803520 I@DeviceStatsMonwKÍG $itor.on_train_batch_end/segment.large_pool.current‚2147 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 577553920 WHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 2514370340 F=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚2147 NADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚13545965 T@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚210257175436288 K;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 38610163200 [GDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚207983667006976 OBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚10716496 K>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚10715905 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 ^JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚199706741684608 FDeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚19708 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚29097984 RBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 35913877504 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚4223149 WGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 37073453056 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 F=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚1534 XFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 2270992053760 TEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 3142752256 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 BDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 36985723393 trainer/global_step‚949 % _timestamp‚1.7884868949614978e+09 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚19714 G:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚31293042 ^KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚42314635272704 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 D;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚3681 aODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 2840033802240 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 N>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 38610163200 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚2684 RCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 1515847168 TDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 37073453056 PDeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 TEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 3140738480 XIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 3217031168 I@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚1534 J=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚17747083 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚3 bODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚42314664370688 ]KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 2839951806464 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 VGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 3217031168 KBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚2147 C6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚31293042 [HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚45154698172928 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚4223152 ?6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚3681 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 ZFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚199706716256640 RBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 40290484224 LDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚330 WCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚210257175436288 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚23979 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚591 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 YIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 37073453056 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1309 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚23979 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚13545959 \JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 2273508429312 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚6 REDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚17766791 NADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚17747083 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚588 FDeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚31312756 NADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚13545959 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚23680 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 UEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 34283657216 YGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 2273508429312 WCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚201977372409111 EDeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 2541803520 WHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 2516375552 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 PADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 2539798308KqX ý& ÷M NADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚17766791 M>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 2541803520 WHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 2516375552 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 PADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 2539798308 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 _step‚75 ZFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚207983641579008 REDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚13545965 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚6492756 S?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚201974832610803 P@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 40290484224 ?6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚3681 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚19708 PADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 2541803520 I@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚2147 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 577553920 WHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 2514370340 F=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚2147 NADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚13545965 T@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚210257175436288 K;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 38610163200 [GDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚207983667006976 OBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚10716496 K>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚10715905 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 ^JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚199706741684608 FDeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚19708 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚29097984 RBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 35913877504 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚4223149 WGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 37073453056 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 F=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚1534 XFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 2270992053760 TEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 3142752256 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 BDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 36985723393 trainer/global_step‚949 % _timestamp‚1.7884868949614978e+09 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚19714 G:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚31293042 ^KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚42314635272704 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 D;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚3681 aODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 2840033802240 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 N>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 38610163200 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚2684 RCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 1515847168 TDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 37073453056 PDeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 TEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 3140738480 XIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 3217031168 I@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚1534 J=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚17747083 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚3 bODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚42314664370688 ]KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 2839951806464 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 VGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 3217031168 KBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚2147 C6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚31293042 [HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚45154698172928 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚4223152 ?6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚3681 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 ZFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚199706716256640 RBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 40290484224 LDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚330 WCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚210257175436288 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚23979 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚591 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 YIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 37073453056 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1309 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚23979 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚13545959 \JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 2273508429312 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚6 REDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚17766791 NADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚17747083 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚588 FDeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚31312756 NADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚13545959 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚23680 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 UEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 34283657216 YGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 2273508429312 WCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚201977372409111 E!JÍ ‚(:¿ œÊèÔ¿íÿ® disk.nvme0n1.in‚ 0.33984375$ disk./.usageGB‚2980.134754180908! disk.sda2.out‚167.04052734375 disk.sda.in‚ 0.0078125 disk.md0.out‚ 159.9921875$ memory_percent‚99.95565414428711" proc.memory.rssMB‚ 3049.7890625 network.sent‚ 2.012359e+06 disk.md0p1.out‚ 159.9921875 disk.sdb1.out‚0 disk.md0p1.in‚ 0.0078125 disk.dm-0.out‚21383.46484375 disk.sdb2.in‚0 disk.sda1.in‚0$ proc.memory.availableMB‚7.265625) proc.memory.percent‚18.61443519592285 network.recv‚611556 disk.rbd2.in‚0 disk.rbd2.out‚ 0.50390625 disk.sdb.in‚0 disk.nvme1n1.in‚4.4375 disk.dm-0.in‚ 4.77734375 disk.sda1.out‚0 disk.md0.in‚ 0.0078125) disk./.usagePercent‚83.32581501019163 disk.sdb.out‚167.04052734375 disk.sda2.in‚ 0.0078125 disk.rbd0.in‚0 disk.rbd0.out‚ 0.341796875 cpu‚25.476815499459153 proc.cpu.threads‚17! disk.sdb2.out‚167.04052734375! disk.nvme1n1.out‚ 9665.6171875 disk.sda.out‚167.04052734375# disk.nvme0n1.out‚11717.84765625 disk.sdb1.in‚0Yè¯Ñ“Î ‚(:ˆ «ÊèÔÛ÷ø° gpu.0.gpu‚12.0 gpu.0.memory‚4+ gpu.0.memoryAllocated‚47.54737854003906* gpu.0.memoryAllocatedBytes‚ 40842887168 gpu.0.temp‚46.0 gpu.0.powerWatts‚315.393' gpu.0.enforcedPowerLimitWatts‚400.0 gpu.0.powerPercent‚78.84825 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0DZ&2GÏ ‚(:¼ «ÊèÔ—Ãȱ# disk.nvme0n1.out‚11821.40234375# disk./.usageGB‚2980.82413482666 disk.md0p1.in‚ 0.0078125 disk.rbd2.in‚0! disk.sda2.out‚168.87060546875 disk.nvme1n1.out‚ 9727.078125 disk.sda.out‚168.87060546875! proc.memory.availableMB‚7.125 disk.dm-0.in‚ 4.77734375 disk.dm-0.out‚21548.48046875 disk.md0p1.out‚ 161.75390625 disk.sdb.in‚0 disk.rbd0.out‚ 0.341796875 disk.sdb1.in‚0$ memory_percent‚99.95651245117188" proc.memory.rssMB‚ 3049.7890625 disk.nvme1n1.in‚4.4375 proc.cpu.threads‚17 network.sent‚ 2.019873e+06 network.recv‚615175 disk.sdb2.in‚0 disk.sda1.in‚0 disk.sdb1.out‚0) proc.memory.percent‚18.61443519592285) disk./.usagePercent‚83.34509038157508! disk.sdb2.out‚168.87060546875 disk.rbd2.out‚ 0.50390625 disk.sda.in‚ 0.0078125 disk.md0.in‚ 0.0078125 disk.sda2.in‚ 0.0078125 disk.rbd0.in‚0 disk.sda1.out‚0 disk.sdb.out‚168.87060546875 disk.md0.out‚ 161.75390625 disk.nvme0n1.in‚ 0.33984375 cpu‚25.475389552735816µŽîU”Ð ‚(:‰ ºÊèԹǀ· gpu.0.gpu‚99.0 gpu.0.memory‚18+ gpu.0.memoryAllocated‚47.54737854003906* gpu.0.memoryAllocatedBytes‚ 40842887168 gpu.0.temp‚45.0 gpu.0.powerWatts‚173.059' gpu.0.enforcedPowerLimitWatts‚400.0 gpu.0.powerPercent‚43.26475 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0¢wëåGÑ ‚(:¼ ºÊèÔÚ»¹ disk.sda.in‚ 0.0078125& proc.memory.availableMB‚ 7.00390625! disk.sda2.out‚170.10595703125% disk./.usageGB‚2981.0718879699707 disk.nvme0n1.in‚ 0.33984375" disk.nvme0n1.out‚ 11950.5546875 disk.sda1.in‚0 disk.md0p1.out‚162.9375 cpu‚25.473830314890815) disk./.usagePercent‚83.35201766315478 disk.md0.in‚ 0.0078125 disk.sdb.in‚0 disk.nvme1n1.in‚4.4375 disk.rbd0.in‚0 disk.rbd0.out‚ 0.341796875 disk.sdb1.in‚0 disk.md0.out‚162.9375" proc.memory.rssMB‚ 3049.7890625 disk.rbd2.in‚0 proc.cpu.threads‚17" disk.nvme1n1.out‚ 9849.52734375 disk.sda.out‚170.10595703125 disk.sdb1.out‚0 disk.md0p1.in‚ 0.0078125 disk.sdb.out‚170.10595703125$ memory_percent‚99.95725154876709 network.sent‚ 2.025771e+06 disk.dm-0.out‚21800.08203125 disk.sda2.in‚ 0.0078125) proc.memory.percent‚18.61443519592285 network.recv‚619600 disk.sdb2.in‚0 disk.dm-0.in‚ 4.77734375! disk.sdb2.out‚170.10595703125 disk.sda1.out‚0 disk.rbd2.out‚ 0.50390625Ç)@„2Ò - _wandbruntime‚1779  _runtime‚1779­‡»¥ÄÓ ¾ & lr-AdamW/pg1‚0.0011225819619948218 & lr-AdamW/pg2‚0.0011225819619948218 trainer/global_step‚999 $ _timestamp‚1.788486980050834e+09 _runtime‚1779.152266122 _step‚778€ÒÈÔ Â trainer/global_step‚999 $ _timestamp‚1.788486980050834e+09 _runtime‚1779.152266122 _step‚77 & lr-AdamW/pg1‚0.0011225819619948218 & lr-AdamW/pg2‚0.0011225819619948218M<è*ž2Õ - _wandbruntime‚1779  _runtime‚1779ÆÈvÝfÖ êO TDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 35913877504 RCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 2541790208 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚3 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 100621312 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚25427968 KBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚1534 SGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚6813563 YFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚47461259750912 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 trainer/global_step‚999 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚4437086 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 VGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 3140738480 MDDeviceStatsM(q £Š#onitor.on_train_batch_start/segment.small_pool.allocated‚1534 G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚2684 VBDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated‚220856009411072 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚19682 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 YEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚212162970596181 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚23979 _MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 2984238553600 H?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚2147 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚23680 IDeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚220853467620864 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚6 NFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚986 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 H?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚1534 IDeviceStatsMonitor.on_train_batch_start/allocation.all.current‚19688 TGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚14226575 ]JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚47461360372224 TEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 1515847168 BDeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 RBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 40290484224 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚2684 E8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚32875762 L?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚18649193 M=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 38610163200 ^LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 2388984948224 TDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 40290484224 `LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚218467024462848 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚25427968 M@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚32895450 _step‚78 SDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 3142752256 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 MDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚2147´]¹ô'× îO H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚23680 IDeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚220853467620864 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚6 NFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚986 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 H?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚1534 IDeviceStatsMonitor.on_train_batch_start/allocation.all.current‚19688 TGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚14226575 ]JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚47461360372224 TEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 1515847168 BDeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 RBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 40290484224 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚2684 E8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚32875762 L?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚18649193 M=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 38610163200 ^LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 2388984948224 TDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 40290484224 `LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚218467024462848 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚25427968 M@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚32895450 _step‚78 SDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 3142752256 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 MDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚2147 TDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 35913877504 RCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 2541790208 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚3 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 100621312 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚25427968 KBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚1534 SGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚6813563 YFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚47461259750912 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 trainer/global_step‚999 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚4437086 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 VGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 3140738480 MDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚1534 G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚2684 VBDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated‚220856009411072 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚19682 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 YEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚212162970596181 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚23979 _MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 2984238553600 H?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚2147NÚ‹ª2Ø - _wandbruntime‚1780  _runtime‚1780=Óäaú&Ù ôM REDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚18687243 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚19714 ^KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚44521151522304 ZFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚209982499281440 C6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚32907994 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚19708 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚588 RCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 1515847168 VGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 3217031168 ^JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚209982524709408 D;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚3681 % _timestamp‚1.7884869816691353e+09 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚14240459 K>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚11261438 LDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚330 aODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 2987255403008 PADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 2539798308 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚25427968 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 FDeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚32927708 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚29097984 T@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚221072311590912 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚6 J=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚18667535 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 577553920 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 B9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚3681 NADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚14240465 BDeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 F=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚1534 QBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 3142752256 EDeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 2541803520 WCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚221072311590912 [HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚47508436023296 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 963120128 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 TDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 37073453056 _step‚79 REDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚14240465 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 NADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚18687243 \JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 2391341057536 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚6820577 RBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 40290484224 M=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 40290484224 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚23979 FDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 36985723393 \JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 2388314683211 XFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 2388824681984 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚23680 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚71510016 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 G:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚32927708 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚25427968 WCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚218680945105408 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 TEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 3140738480 YIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 37073453056 KBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚2147 ^JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚218680970533376 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚25427968 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚2684 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 WGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 37073453056 PADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 2541803520 N>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 38610163200 NADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚18667535 WCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚212370839392619 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 F=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚2147 RBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 35913877504Ü>°Ø3 Ú øM E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚6 J=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚18667535 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 577553920 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 B9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚3681 NADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚14240465 BDeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 F=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚1534 QBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 3142752256 EDeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 2541803520 WCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚221072311590912 [HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚47508436023296 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 963120128 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 TDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 37073453056 _step‚79 REDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚14240465 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 NADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚18687243 \JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 2391341057536 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚6820577 RBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 40290484224 M=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 40290484224 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚23979 FDeviceStatsMonitor.on_train_batch_end/»Õ[jËrequested_bytes.all.peak‚ 36985723393 \JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 2388314683211 XFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 2388824681984 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚23680 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚71510016 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 G:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚32927708 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚25427968 WCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚218680945105408 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 TEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 3140738480 YIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 37073453056 KBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚2147 ^JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚218680970533376 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚25427968 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚2684 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 WGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 37073453056 PADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 2541803520 N>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 38610163200 NADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚18667535 WCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚212370839392619 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 F=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚2147 RBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 35913877504 REDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚18687243 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚19714 ^KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚44521151522304 ZFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚209982499281440 C6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚32907994 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚19708 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚588 RCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 1515847168 VGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 3217031168 ^JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚209982524709408 D;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚3681 % _timestamp‚1.7884869816691353e+09 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚14240459 K>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚11261438 LDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚330 aODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 2987255403008 PADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 2539798308 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚25427968 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 FDeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚32927708 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚29097984 T@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚221072311590912O{­à2Û - _wandbruntime‚1780  _runtime‚1780’+ôWÜ Ñ /loss/FRt4/assignment_loss‚2.927149730743748 0loss/FBt1/assignment_loss‚0.3344422205859287 .loss/FBt4/detection_loss‚2.854506341616313 0loss/FBt4/assignment_loss‚0.1272082143618415 1loss/SRqqt3/detection_loss‚2.4961530344955847 _step‚80 /loss/FRt2/detection_loss‚1.7447129691827972 % _timestamp‚1.7884869816713378e+09 /loss/FRt1/assignment_loss‚3.840488575398922 -loss/FRt1/detection_loss‚2.35575931177785 .loss/FRt3/detection_loss‚2.824528906742732 2loss/SRqqt3/assignment_loss‚1.9003895549950274 1loss/FBt3/assignment_loss‚0.18470099250165126 /loss/FBt1/detection_loss‚2.3085091493346472 trainer/global_step‚999 0loss/FRt2/assignment_loss‚1.4046178150851771 1loss/SRqqt1/assignment_loss‚1.813337346042196 1loss/FBt2/assignment_loss‚0.09573230334084518 0loss/SRqqt1/detection_loss‚2.318807853857676 1loss/SRqqt4/detection_loss‚1.8037760114970327 /loss/FBt3/detection_loss‚3.4095825254917145 2loss/SRqqt2/assignment_loss‚0.9948987291996659 /loss/FRt4/detection_loss‚2.3731229443951407 /loss/FBt2/detection_loss‚0.5895979192535246 &loss/total_loss‚45.324991620882784 epoch‚1 2loss/SRqqt4/assignment_loss‚1.5411871024603103 1loss/SRqqt2/detection_loss‚1.0392058051741906 _runtime‚1780.772489474 /loss/FRt3/assignment_loss‚4.042576263348262¢¬«[Ý Õ 2loss/SRqqt2/assignment_loss‚0.9948987291996659 /loss/FRt4/detection_loss‚2.3731229443951407 /loss/FBt2/detection_loss‚0.5895979192535246 &loss/total_loss‚45.324991620882784 epoch‚1 2loss/SRqqt4/assignment_loss‚1.5411871024603103 1loss/SRqqt2/detection_loss‚1.0392058051741906 _runtime‚1780.772489474 /loss/FRt3/assignment_loss‚4.042576263348262 /loss/FRt4/assignment_loss‚2.927149730743748 0loss/FBt1/assignment_loss‚0.3344422205859287 .loss/FBt4/detection_loss‚2.854506341616313 0loss/FBt4/assignment_loss‚0.1272082143618415 1loss/SRqqt3/detection_loss‚2.4961530344955847 _step‚80 /loss/FRt2/detection_loss‚1.7447129691827972 % _timestamp‚1.7884869816713378e+09 /loss/FRt1/assignment_loss‚3.840488575398922 -loss/FRt1/detection_loss‚2.35575931177785 .loss/FRt3/detection_loss‚2.824528906742732 2loss/SRqqt3/assignment_loss‚1.9003895549950274 1loss/FBt3/assignment_loss‚0.18470099250165126 /loss/FBt1/detection_loss‚2.3085091493346472 trainer/global_step‚999 0loss/FRt2/assignment_loss‚1.4046178150851771 1loss/SRqqt1/assignment_loss‚1.813337346042196 1loss/FBt2/assignment_loss‚0.09573230334084518 0loss/SRqqt1/detection_loss‚2.318807853857676 1loss/SRqqt4/detection_loss‚1.8037760114970327 /loss/FBt3/detection_loss‚3.4095825254917145P©"ÎMÞ ‚(: ÉÊèÔ‰þ–¯ disk.sdb.out‚171.42529296875 disk.sda.in‚ 0.0078125 disk.nvme0n1.in‚ 0.33984375& proc.memory.availableMB‚ 6.90234375 cpu‚25.472551753235237 disk.sda1.in‚0% disk./.usageGB‚2981.0723457336426 disk.nvme1n1.in‚4.4375 disk.rbd0.in‚0 disk.rbd0.out‚ 0.341796875 disk.dm-0.out‚21991.26953125 disk.md0p1.in‚ 0.0078125 disk.rbd2.in‚0 disk.sdb1.out‚0! disk.sdb2.out‚171.42529296875 disk.sdb.in‚0# disk.nvme0n1.out‚12080.16796875 proc.cpu.threads‚17! disk.nvme1n1.out‚ 9911.1015625 disk.dm-0.in‚ 4.77734375 disk.sda1.out‚0 disk.sda2.in‚ 0.0078125! disk.sda2.out‚171.42529296875 disk.sda.out‚171.42529296875 disk.md0p1.out‚ 164.1953125 network.sent‚ 2.029477e+06) disk./.usagePercent‚83.35203046241864 disk.sdb1.in‚0 disk.md0.in‚ 0.0078125 disk.md0.out‚ 164.1953125) proc.memory.percent‚18.61443519592285 network.recv‚623751 disk.rbd2.out‚ 0.50390625 disk.sdb2.in‚0$ memory_percent‚99.95787143707275" proc.memory.rssMB‚ 3049.7890625êÇß ‚(:’ ÉÊèÔ¾™±¯ gpu.0.gpu‚98.0 gpu.0.memory‚17+ gpu.0.memoryAllocated‚47.54737854003906* gpu.0.memoryAllocatedBytes‚ 40842887168 gpu.0.temp‚45.0 gpu.0.powerWatts‚239.491' gpu.0.enforcedPowerLimitWatts‚400.0( gpu.0.powerPercent‚59.87275000000001 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0k^=l“à ‚(:ˆ ØÊèÔœ‹â¯ gpu.0.gpu‚38.0 gpu.0.memory‚9+ gpu.0.memoryAllocated‚47.54737854003906* gpu.0.memoryAllocatedBytes‚ 40842887168 gpu.0.temp‚47.0 gpu.0.powerWatts‚325.297' gpu.0.enforcedPowerLimitWatts‚400.0 gpu.0.powerPercent‚81.32425 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0Y·¾ƒJá ‚(:¿ ØÊèÔÍÌÚ° disk.rbd0.in‚0# disk.nvme0n1.out‚12184.30078125 disk.sdb2.in‚0! disk.sdb2.out‚172.67626953125 disk.md0.out‚ 165.3828125$ memory_percent‚99.95808601379395 cpu‚25.47136221691467 network.sent‚ 2.118451e+06$ disk./.usageGB‚2980.502471923828 disk.sdb.out‚172.67626953125 disk.nvme1n1.in‚4.4375 disk.sdb1.in‚0 network.recv‚629799 disk.rbd2.out‚ 0.50390625" proc.memory.rssMB‚ 3049.7890625) proc.memory.percent‚18.61443519592285 disk.sdb.in‚0 disk.sda.in‚ 0.0078125 disk.rbd0.out‚ 0.341796875 disk.sda1.in‚0 disk.sda2.in‚ 0.0078125! disk.sda2.out‚172.67626953125 disk.nvme0n1.in‚ 0.33984375 disk.dm-0.out‚22217.85546875 disk.sda1.out‚0 proc.cpu.threads‚17 disk.rbd2.in‚0 disk.dm-0.in‚ 4.77734375 disk.sdb1.out‚0 disk.md0p1.in‚ 0.0078125 disk.md0p1.out‚ 165.3828125% proc.memory.availableMB‚ 6.8671875( disk./.usagePercent‚83.3360965521855 disk.sda.out‚172.67626953125 disk.md0.in‚ 0.0078125" disk.nvme1n1.out‚ 10033.5546875·eô”â ‚(:‰ çÊèÔåšÁ° gpu.0.gpu‚98.0 gpu.0.memory‚18+ gpu.0.memoryAllocated‚47.54737854003906* gpu.0.memoryAllocatedBytes‚ 40842887168 gpu.0.temp‚44.0 gpu.0.powerWatts‚251.845' gpu.0.enforcedPowerLimitWatts‚400.0 gpu.0.powerPercent‚62.96125 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0]ý’«Hã ‚(:½ çÊèÔ£Àþ± disk.dm-0.out‚ 22322.890625! disk.sda2.out‚174.38720703125$ memory_percent‚99.95803833007812" proc.memory.rssMB‚ 3049.7890625 cpu‚25.46965664126234 network.sent‚ 2.123778e+06$ disk./.usageGB‚2980.544780731201 disk.sdb1.out‚0 disk.md0.in‚ 0.0078125 disk.md0.out‚ 167.03515625! disk.sdb2.out‚174.38720703125 disk.rbd2.out‚ 0.50390625 disk.rbd0.out‚ 0.341796875! proc.memory.availableMB‚6.875 proc.cpu.threads‚17 disk.sda.out‚174.38720703125 disk.sda.in‚ 0.0078125 disk.md0p1.in‚ 0.0078125 disk.md0p1.out‚ 167.03515625 disk.sdb.in‚0 disk.sda2.in‚ 0.0078125 disk.rbd0.in‚0 disk.sdb.out‚174.38720703125 disk.sda1.in‚0 disk.sdb2.in‚0 disk.rbd2.in‚0) disk./.usagePercent‚83.33727952414726# disk.nvme1n1.out‚10094.99609375) proc.memory.percent‚18.61443519592285 disk.sda1.out‚0 disk.nvme0n1.in‚ 0.33984375# disk.nvme0n1.out‚12227.89453125 disk.sdb1.in‚0 disk.nvme1n1.in‚4.4375 network.recv‚633370 disk.dm-0.in‚ 4.77734375G¿æ’ä ‚(:‡ öÊèÔ×…³° gpu.0.gpu‚97.0 gpu.0.memory‚17+ gpu.0.memoryAllocated‚47.54737854003906* gpu.0.memoryAllocatedBytes‚ 40842887168 gpu.0.temp‚45.0 gpu.0.powerWatts‚226.75' gpu.0.enforcedPowerLimitWatts‚400.0 gpu.0.powerPercent‚56.6875 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0SxåMå ‚(: öÊèÔàçÁ± network.recv‚637202 disk.nvme0n1.in‚ 0.33984375# disk.nvme0n1.out‚12228.44140625! disk.sdb2.out‚175.60107421875 disk.md0.out‚ 168.17578125 disk.sdb.in‚0 disk.sda2.in‚ 0.0078125 disk.dm-0.out‚22445.94921875 disk.sdb1.out‚0 disk.md0p1.out‚ 168.17578125 disk.dm-0.in‚ 4.77734375 disk.md0.in‚ 0.0078125! disk.sda2.out‚175.60107421875 cpu‚25.46843254700274 disk.rbd0.out‚ 0.341796875 disk.sda1.out‚0) proc.memory.percent‚18.61443519592285 disk.nvme1n1.in‚4.4375 disk.sda.out‚175.60107421875 disk.sdb1.in‚0$ memory_percent‚99.95801448822021 proc.cpu.threads‚17 disk.rbd0.in‚0 disk.sda1.in‚0 disk.sdb.out‚175.60107421875" proc.memory.rssMB‚ 3049.7890625 network.sent‚ 2.129466e+06( disk./.usagePercent‚83.3380018292708$ disk./.usageGB‚2980.570613861084 disk.sdb2.in‚0 disk.rbd2.in‚0 disk.rbd2.out‚ 0.50390625& proc.memory.availableMB‚ 6.87890625" disk.nvme1n1.out‚ 10217.5078125 disk.sda.in‚ 0.0078125 disk.md0p1.in‚ 0.0078125‰Ï‡ƒ”æ ‚(:‰ …ËèÔ…ß¾® gpu.0.gpu‚83.0 gpu.0.memory‚19+ gpu.0.memoryAllocated‚47.54737854003906* gpu.0.memoryAllocatedBytes‚ 40842887168 gpu.0.temp‚45.0 gpu.0.powerWatts‚230.097' gpu.0.enforcedPowerLimitWatts‚400.0 gpu.0.powerPercent‚57.52425 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0òèz8Oç ‚(:Ä …ËèÔù‹¯ disk.sdb.out‚176.87060546875 disk.sdb1.out‚0 disk.rbd2.out‚ 0.50390625 disk.nvme0n1.in‚ 0.33984375! disk.nvme0n1.out‚ 12228.703125) proc.memory.percent‚18.61443519592285 disk.md0.in‚ 0.0078125 disk.md0p1.out‚ 169.38671875 disk.sda2.in‚ 0.0078125# disk.nvme1n1.out‚10352.94921875 disk.sda.out‚176.87060546875 proc.cpu.threads‚17 disk.sda1.in‚0 disk.sdb.in‚0! disk.sda2.out‚176.87060546875& proc.memory.availableMB‚ 7.01171875" proc.memory.rssMB‚ 3049.7890625 disk.sdb1.in‚0 disk.md0.out‚ 169.38671875 disk.rbd0.out‚ 0.341796875 disk.dm-0.out‚22581.65234375$ memory_percent‚99.95720386505127 cpu‚25.467283922608352) disk./.usagePercent‚83.33805217304196 network.recv‚641426 disk.nvme1n1.in‚4.4375 disk.rbd0.in‚0 disk.sdb2.in‚0! disk.sdb2.out‚176.87060546875 disk.sda1.out‚0 disk.sda.in‚ 0.0078125 disk.dm-0.in‚ 4.77734375 network.sent‚ 2.133343e+06% disk./.usageGB‚2980.5724143981934 disk.md0p1.in‚ 0.0078125 disk.rbd2.in‚0òš”è ‚(:‰ ”ËèÔ¯àÄ° gpu.0.gpu‚97.0 gpu.0.memory‚17+ gpu.0.memoryAllocated‚47.54737854003906* gpu.0.memoryAllocatedBytes‚ 40842887168 gpu.0.temp‚42.0 gpu.0.powerWatts‚243.745' gpu.0.enforcedPowerLimitWatts‚400.0 gpu.0.powerPercent‚60.93625 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0\ò±"Eé ‚(:º ”ËèÔýƒÏ±" proc.memory.rssMB‚ 3049.7890625 network.recv‚646216( disk./.usagePercent‚83.3450549702784 disk.md0.in‚ 0.0078125 disk.sdb.in‚0! disk.nvme0n1.out‚ 12228.765625 disk.dm-0.in‚ 4.77734375 disk.nvme0n1.in‚ 0.33984375 disk.sda1.in‚0 disk.rbd2.in‚0 disk.nvme1n1.in‚4.4375$ proc.memory.availableMB‚6.953125 disk.rbd2.out‚ 0.50390625 disk.sda2.in‚ 0.0078125! disk.sda2.out‚178.52197265625" disk.nvme1n1.out‚ 10552.8828125 disk.sda.in‚ 0.0078125 disk.dm-0.out‚ 22781.6484375 disk.sdb1.out‚0 disk.sdb2.in‚0 disk.rbd0.in‚0) proc.memory.percent‚18.61443519592285 proc.cpu.threads‚17 network.sent‚ 2.141672e+06$ disk./.usageGB‚2980.822868347168 disk.md0p1.in‚ 0.0078125 disk.md0p1.out‚ 170.984375 disk.sda.out‚178.52197265625 disk.rbd0.out‚ 0.341796875 disk.sdb1.in‚0 disk.md0.out‚ 170.984375 disk.sdb.out‚178.52197265625! disk.sdb2.out‚178.52197265625 cpu‚25.465748486362738 disk.sda1.out‚0$ memory_percent‚99.95756149291992}º‘Ü2ê - _wandbruntime‚1865  _runtime‚1865ä#ìÄë ¾ % _timestamp‚1.7884870662946482e+09 _runtime‚1865.396141322 _step‚81 % lr-AdamW/pg1‚0.001118632568285835 % lr-AdamW/pg2‚0.001118632568285835 trainer/global_step‚1049å¿Èì  _step‚81 % lr-AdamW/pg1‚0.001118632568285835 % lr-AdamW/pg2‚0.001118632568285835 trainer/global_step‚1049 % _timestamp‚1.7884870662946482e+09 _runtime‚1865.396141322Q†{ûÆ2í - _wandbruntime‚1865  _runtime‚1865R|Ÿñ'î ëO SGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚7140787 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚88300544 % _timestamp‚1.7884870662961402e+09 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚23680 VBDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated‚231671145565696 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚3 dQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚46683566544896 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚19688 VGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 3142752256 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 M=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 38610163200 O@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 2541790208 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚25427968 ]JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚49815019000320 VGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 3140738480 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 L?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚19569645 WEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 2504301214208 NFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚986 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 D;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚3681 KBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚2147 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚23979 PCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚19569645 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚4655386 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 117398528 BDeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 IDeviceStatsMonitor.on_train_batch_start/allocation.all.current‚19688 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 ^LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 2506817576448 QDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚11796751 _MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 3131364154880 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 [IDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 2506817576448 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚19682 TDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 40290484224 VGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 2516362240 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚25427968 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚2684 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 963120128 MDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚2147 H?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚1309 \HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚229164302561280 TGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚14921075 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 RBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 40290484224 H?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚2147 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚19682 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚6 TGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚19589327 WGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 34283657216 O?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 40290484224 ZKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 3217031168 PCDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated‚19589327 YFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚49814901601792 cQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 3131452455424 XIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 3217031168 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 E8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚34490714 PCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚14921075 L?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚14921069 G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚2684 WGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 35913877504 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚25427968 YEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚229164302561280 TDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 35913877504 YEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚222556437579689 YIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 37073453056 RCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 2541790208 YJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 2516362240 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 `LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚229164327989248 M@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚34510402 ZHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 2501131135845 [KDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 37073453056 YEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚231671145565696 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 _step‚82 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚581 ]IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚229164327989248 TEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 1515847168 P@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 36985723393 trainer/global_step‚1049 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚23680 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 NFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚330 IDeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚231668603775488 SDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 3142752256 A8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚3681 H?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚1534 P@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 38610163200 ZHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 2504301214208 PCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚14921069 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚29097984 `LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚220052792073712 _runtime‚1865.397640618 UADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚231668603775488 Ï5±˜ ï ïO ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 963120128 MDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚2147 H?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚1309 \HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚229164302561280 TGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚14921075 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 RBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 40290484224 H?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚2147 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚19682 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚6 TGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚19589327 WGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 34283657216 O?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 40290484224 ZKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 3217031168 PCDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated‚19589327 YFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚49814901601792 cQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 3131452455424 XIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 3217031168 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 E8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚34490714 PCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚14921075 L?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚14921069 G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚2684 WGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 35913877504 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚25427968 YEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚229164302561280 TDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 35913877504 YEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚222556437579689 YIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 37073453056 RCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 2541790208 YJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 2516362240 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 `LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚229164327989248 M@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚34510402 ZHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 2501131135845 [KDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 37073453056 YEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚231671145565696 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 _step‚82 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚581 ]IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚229164327989248 TEDeviceStatsMonitorÚO(Q].on_train_batch_start/inactive_split_bytes.all.peak‚ 1515847168 P@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 36985723393 trainer/global_step‚1049 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚23680 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 NFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚330 IDeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚231668603775488 SDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 3142752256 A8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚3681 H?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚1534 P@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 38610163200 ZHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 2504301214208 PCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚14921069 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚29097984 `LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚220052792073712 _runtime‚1865.397640618 UADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚231668603775488 SGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚7140787 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚88300544 % _timestamp‚1.7884870662961402e+09 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚23680 VBDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated‚231671145565696 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚3 dQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚46683566544896 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚19688 VGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 3142752256 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 M=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 38610163200 O@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 2541790208 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚25427968 ]JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚49815019000320 VGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 3140738480 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 L?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚19569645 WEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 2504301214208 NFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚986 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 D;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚3681 KBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚2147 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚23979 PCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚19569645 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚4655386 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 117398528 BDeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 IDeviceStatsMonitor.on_train_batch_start/allocation.all.current‚19688 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 ^LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 2506817576448 QDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚11796751 _MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 3131364154880 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 [IDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 2506817576448 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚19682 TDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 40290484224 VGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 2516362240 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚25427968 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚2684R O¨2ð - _wandbruntime‚1866  _runtime‚1866JÖRú&ñ ôM ?6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚3681 \JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 2505998641919 WGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 37073453056 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 WHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 2514370340 N>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 38610163200 NADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚14934965 E8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 PDeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚11806983 LDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚330 KBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚2147 TEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 3142752256 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1309 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚571 WDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚49861943237632 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚29097984 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 ^JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚229378274059776 K>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚34542660 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 F=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚2147 UCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 2506657310208 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚2684 RCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 1515847168 UEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 34283657216 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 J=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚19587987 [HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚49862056428544 bODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚46727696869888 KBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚1534 _step‚83 YGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 2509173685760 NADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚19587987 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 BDeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 2541803520 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 113190912 ]KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 3134275465728 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚14934959 T@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚231887447745536 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚6 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚7147234 ^KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚46727667771904 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 SDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 3217031168 S?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚231884905942016 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚23680 I@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚2147 trainer/global_step‚1049 TEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 2516375552 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚25427968 WHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 2516375552 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 577553920 N>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 36985723393 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚25427968 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚19714 \JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 2509173685760 S?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚222761766577819 TEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 3140738480 VGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 3217031168 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚4659749 NADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚14934959 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 963120128 F8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 PDeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚11806983 LDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚330 KBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚2147 TEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 3142752256 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1309 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚571 WDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚49861943237632 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚29097984 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 ^JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚229378274059776 K>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚34542660 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 F=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚2147 UCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 2506657310208 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚2684 RCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 1515847168 UEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 34283657216 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 J=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚19587987 [HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚49862056428544 bODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚46727696869888 KBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚1534 _step‚83 YGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 2509173685760 NADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚19587987 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 BDeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 2541803520 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 113190912 ]KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 3134275465728 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚14934959 T@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚231887447745536 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚6 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚7147234 ^KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚46727667771904 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 SDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 3217031168 S?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚231884905942016 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚23680 I@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚2147 trainer/global_step‚1049 TEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 2516375552 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚25427968 WHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 2516375552 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 577553920 N>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 36985723393 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚25427968 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚19714 \JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 2509173685760 S?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚222761766577819 TEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 3140738480 VGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 3217031168 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚4659749 NADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚14934959 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 963120128 FDeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 WHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 2514370340 N>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 38610163200 NADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚14934965 EDeviceStatsMonitor.on_train_batch_start/allocation.all.current‚19688 _MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 3278631890432 RBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 40290484224 YEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚239861606087680 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚6 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚3 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 YEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚232949904563197 WGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 34283657216 [KDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 37073453056 MDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚1534 VGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 2516362240 ]JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚52168784111104 D>DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 YIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 37073453056 MDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚2147 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 trainer/global_step‚1099 IDeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚2684 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 WEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 2622133842432 `MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚48890053696512 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 577553920 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚25427968 A8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚3681 _runtime‚1951.907082259 M@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚36125354 D;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚3681 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚29097984 H?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚1309 YJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 2514370132 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 KBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚1534 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚25427968 SGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚4873683 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚23680 R>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚242483739930112 SDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 3142752256 VGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 3142752256 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚69426176 RCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 2539798100 E8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚36105666 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚19682 VBDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated‚242486281720320 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚23979 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 963120128 VFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 37073453056 RCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 2541790208 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚23680 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚575 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 BDeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚23680 R>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚242483739930112 SDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 3142752256 VGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 3142752256 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚69426176 RCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 2539798100 E8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚36105666 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚19682 VBDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated‚242486281720320 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚23979 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 963120128 VFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 37073453056 RCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 2541790208 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚23680 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚575 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 BDeviceStatsMonitor.on_train_batch_start/allocation.all.current‚19688 _MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 3278631890432 RBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 40290484224 YEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚239861606087680 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚6 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚3 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 YEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚232949904563197 WGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 34283657216 [KDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 37073453056 MDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚1534 VGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 2516362240 ]JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚52168784111104 D>DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 YIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 37073453056 MDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚2147 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 trainer/global_step‚1099 IDeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚2684 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 WEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 2622133842432 `MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚48890053696512 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 577553920 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚25427968 A8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚3681 _runtime‚1951.907082259 M@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚36125354 D;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚3681 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚29097984VµF¾g2ˆ - _wandbruntime‚1953  _runtime‚1953s‰¬Â‰ õM F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1309 UCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 2624489938432 WCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚242702583900160 \JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 2627006313984 REDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚15629465 \JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 2623682600627 % _timestamp‚1.7884871544109857e+09 _runtime‚1953.513163551 WHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 2516375552 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 I@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚2147 trainer/global_step‚1099 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚19708 K;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 38610163200 WCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚240075552158208 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚581 XFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 2624489938432 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 ?6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚3681 I@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚1534 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚15629459 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚29097984 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚25427968 TDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 37073453056 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 B9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚3681 REDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚20528147 RBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 40290484224 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 XIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 3217031168 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚2684 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 XFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 2621168230287 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚25427968 UEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 35913877504 OBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚12354253 UEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 34283657216 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚19714 T@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚242702583900160 N>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 38610163200 K>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚12353669 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚71510016 M>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 2541803520 S?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚242700042096640 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚23979 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚3 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 N>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 36985723393 VGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 3217031168 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚584 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 SDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 3217031168 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚23979 PADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 2541803520 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚23680 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 WCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚233157773359635 LDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚330 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 963120128 F=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚2147 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 TEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 2516375552 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚6 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 577553920 WGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 37073453056 NADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚20508439 [GDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚240075577586176 QBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 3142752256 NADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚15629459 P@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 40290484224 PDeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚36157612 [HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚52215860846592 RCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 1515847168 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 ZFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚240075552158208 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 100608000 TEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 3142752256 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚19708 aODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 3281647727104 TEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 3140738480 F=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚1534 NADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚15629465 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚4878052 ^JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚240075577586176 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 NADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚20528147 RBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 35913877504 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 _step‚87 C6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚36137898 YGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 2627006313984 B8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 ?6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚3681 I@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚1534 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚15629459 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚29097984 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚25427968 TDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 37073453056 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 B9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚3681 REDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚20528147 RBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 40290484224 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 XIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 3217031168 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚2684 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 XFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 2621168230287 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚25427968 UEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 35913877504 OBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚12354253 UEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 34283657216 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚19714 T@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚242702583900160 N>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 38610163200 K>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚12353669 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚71510016 M>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 2541803520 S?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚242700042096640 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚23979 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚3 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 N>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 36985723393 VGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 3217031168 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚584 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 SDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 3217031168 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚23979 PADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 2541803520 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚23680 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 WCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚233157773359635 LDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚330 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 963120128 F=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚2147 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 TEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 2516375552 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚6 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 577553920 WGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 37073453056 NADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚20508439 [GDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚240075577586176 QBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 3142752256 NADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚15629459 P@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 40290484224 PDeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚36157612 [HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚52215860846592 RCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 1515847168 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 ZFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚240075552158208 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 100608000 TEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 3142752256 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚19708 aODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 3281647727104 TEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 3140738480 F=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚1534 NADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚15629465 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚4878052 ^JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚240075577586176 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 NADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚20528147 RBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 35913877504 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 _step‚87 C6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚36137898 YGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 2627006313984 BDeviceStatsMonitor.on_train_batch_start/allocation.all.current‚19688 YEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚243343371546705 ^LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 2739013423393 YJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 2514370132 MDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚2147 PCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚16310075 NFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚330 MDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚1534 TDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 35913877504 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 963120128 BDeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚23680 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚19682 WGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 35913877504 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 trainer/global_step‚1149 YEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚250558909614080 VFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 37073453056 TEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 1515847168 \HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚240604332695344 RBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 40290484224 UADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚253298876084736 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚25427968 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚6 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 577553920 RCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 2539798100 SDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 3142752256 O?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 40290484224 ZKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 3217031168 UFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 3217031168 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚6 ]JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚54522814064128 D>DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 P@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 36985723393 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚19688 G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚2684 YJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 2516362240 VGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 3142752256 M@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚37740306 TGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚16310075 KBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚2147 KBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚1534 E8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚37720618 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚5091986 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 106912768 SGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚7796893 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 \HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚250558909614080 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚7797480 M=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 38610163200 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚587 dQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚51096599044096 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 RCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 2541790208 TGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚21430231 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 PCDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated‚21430231 `LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚240604358123312 R>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚253298876084736 VGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 2516362240 WEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 2739966470656 YEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚253301417874944 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚77814784 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚25427968 TDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 40290484224 IDeviceStatsMonitor.on_train_batch_start/allocation.all.current‚19688 YEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚243343371546705 ^LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 2739013423393 YJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 2514370132 MDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚2147 PCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚16310075 NFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚330 MDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚1534 TDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 35913877504 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 963120128 BDeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚23680 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚19682 WGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 35913877504 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 trainer/global_step‚1149 YEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚250558909614080 VFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 37073453056 TEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 1515847168 \HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚240604332695344 RBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 40290484224 UADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚253298876084736 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚25427968 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚6 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 577553920 RCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 2539798100 SDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 3142752256 O?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 40290484224 ZKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 3217031168 UFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 3217031168 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚6 ]JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚54522814064128 D>DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 P@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 36985723393 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚19688 G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚2684 YJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 2516362240 VGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 3142752256 M@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚37740306 TGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚16310075 KBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚2147 KBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚1534 E8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚37720618 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚5091986 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 106912768 SGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚7796893 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 \HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚250558909614080 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚7797480 M=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 38610163200 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚587 dQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚51096599044096 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 RCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 2541790208 TGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚21430231 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 PCDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated‚21430231 `LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚240604358123312 R>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚253298876084736 VGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 2516362240 WEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 2739966470656 YEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚253301417874944 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚77814784 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚25427968 TDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 40290484224 I8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 NADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚21428891 [HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚54569866108928 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚23979 UEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 35913877504 ^KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚51140700271104 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 WCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚250772855684608 TEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 3142752256 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚6 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 ?6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚3681 N>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 36985723393 PADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 2541803520 WHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 2516375552 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚5096349 XFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 2738852188995 M=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 40290484224 WGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 37073453056 J=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚21428891 FDeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 2541803520 PDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 SDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 3217031168 I@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚1534 ZFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚250772855684608 REDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚21448599 BDeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 XIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 3217031168 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚2684 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚585 PADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 2539798308 S?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚243548700544835 ^JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚240809873783808 NADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚21448599 ^JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚250772881112576 REDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚16323965 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚29097984 UEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 34283657216 VGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 3217031168 TEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 3140738480 $ _timestamp‚1.788487237610576e+09 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚23680 aODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 3429136739840 \JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 2744838942208 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚25427968 [GDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚250772881112576 bODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚51140729369088 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚23979 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 N>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 38610163200 G:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚37772564 EDeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚12899746 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 577553920 T@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚253517720054784 TEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 2516375552 QBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 3142752256 XFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 2742322566656 K>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚37772564 TDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 37073453056 UCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 2742322566656 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚7803982 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 B9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚3681 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚3 NADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚16323959 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚7803397 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚79898624 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 _step‚91 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 YIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 37073453056d·8éç&  øM C6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚37752850 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚19708 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 ZFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚240809848355840 WHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 2514370340 F=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚1534 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 KBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚2147 trainer/global_step‚1149 FDeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚12899746 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 577553920 T@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚253517720054784 TEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 2516375552 QBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 3142752256 XFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 2742322566656 K>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚37772564 TDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 37073453056 UCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 2742322566656 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚7803982 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 B9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚3681 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚3 NADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚16323959 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚7803397 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚79898624 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 _step‚91 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 YIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 37073453056 _runtime‚2036.713656883 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 I@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚2147 F=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚2147 YGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 2744838942208 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚19708 LDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚986 ]KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 3429056841216 KBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚1534 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1309 S?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚253515178251264 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚6 G:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚37752850 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 NADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚21428891 [HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚54569866108928 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚23979 UEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 35913877504 ^KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚51140700271104 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 WCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚250772855684608 TEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 3142752256 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚6 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 ?6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚3681 N>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 36985723393 PADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 2541803520 WHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 2516375552 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚5096349 XFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 2738852188995 M=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 40290484224 WGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 37073453056 J=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚21428891 FDeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 2541803520 PDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 SDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 3217031168 I@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚1534 ZFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚250772855684608 REDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚21448599 BDeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 XIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 3217031168 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚2684 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚585 PADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 2539798308 S?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚243548700544835 ^JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚240809873783808 NADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚21448599 ^JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚250772881112576 REDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚16323965 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚29097984 UEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 34283657216 VGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 3217031168 TEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 3140738480 $ _timestamp‚1.788487237610576e+09 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚23680 aODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 3429136739840 \JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 2744838942208 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚25427968 [GDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚250772881112576 bODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚51140729369088 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚23979 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 N>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 38610163200 G:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚37772564 E/ž§ ‚(:“ ×ÌèÔ½¤ø° gpu.0.gpu‚99.0 gpu.0.memory‚18+ gpu.0.memoryAllocated‚47.54737854003906* gpu.0.memoryAllocatedBytes‚ 40842887168 gpu.0.temp‚45.0 gpu.0.powerWatts‚237.326' gpu.0.enforcedPowerLimitWatts‚400.0) gpu.0.powerPercent‚59.331500000000005 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0pº„=I¨ ‚(:¾ æÌèÔ°îû¯) proc.memory.percent‚18.61443519592285 disk.rbd2.in‚0 disk.sdb.out‚199.26318359375 disk.sda2.in‚ 0.0078125 disk.sda.out‚199.26318359375 disk.rbd0.out‚ 0.341796875 disk.sdb1.out‚0) disk./.usagePercent‚83.30554854914371 disk.md0p1.in‚ 0.0078125 disk.sdb.in‚0 disk.rbd0.in‚0 disk.dm-0.out‚24807.76953125 disk.sdb2.in‚0! disk.sda2.out‚199.26318359375 disk.nvme1n1.in‚4.4375! disk.nvme0n1.out‚ 12303.265625 cpu‚25.44774120648152$ disk./.usageGB‚2979.409927368164 disk.sda.in‚ 0.0078125 disk.sdb1.in‚0 network.sent‚ 2.461671e+06 disk.sda1.in‚0 disk.nvme0n1.in‚ 4.33984375# memory_percent‚99.9596118927002 proc.cpu.threads‚17 network.recv‚708272 disk.md0.in‚ 0.0078125 disk.md0.out‚ 190.8515625 disk.md0p1.out‚ 190.8515625# disk.nvme1n1.out‚12504.50390625 disk.dm-0.in‚ 8.77734375 disk.sda1.out‚0 disk.rbd2.out‚ 0.50390625! disk.sdb2.out‚199.26318359375% proc.memory.availableMB‚ 6.6171875" proc.memory.rssMB‚ 3049.7890625"’© ‚(:‡ æÌèÔ Ûæ¯ gpu.0.gpu‚49.0 gpu.0.memory‚13+ gpu.0.memoryAllocated‚47.54737854003906* gpu.0.memoryAllocatedBytes‚ 40842887168 gpu.0.temp‚49.0 gpu.0.powerWatts‚324.688' gpu.0.enforcedPowerLimitWatts‚400.0 gpu.0.powerPercent‚81.172 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0ÿ—î[ª ‚(:’ õÌèÔ¶ìù¯ gpu.0.gpu‚89.0 gpu.0.memory‚15+ gpu.0.memoryAllocated‚47.54737854003906* gpu.0.memoryAllocatedBytes‚ 40842887168 gpu.0.temp‚45.0 gpu.0.powerWatts‚238.452' gpu.0.enforcedPowerLimitWatts‚400.0( gpu.0.powerPercent‚59.61300000000001 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0NpùCH« ‚(:½ õÌèÔòê° network.sent‚ 2.469758e+06) disk./.usagePercent‚83.30735740510796 disk.sdb1.out‚0 disk.sdb.out‚200.6474609375# disk.nvme1n1.out‚12504.63671875& proc.memory.availableMB‚ 7.10546875 proc.cpu.threads‚17! disk.nvme0n1.out‚ 12331.453125 disk.md0.in‚ 0.0078125$ memory_percent‚99.95663166046143$ disk./.usageGB‚2979.474620819092 disk.dm-0.in‚ 8.77734375 disk.sda1.in‚0 disk.sdb.in‚0 disk.sda2.out‚200.6474609375 disk.sdb1.in‚0 disk.sda.out‚200.6474609375 disk.rbd0.out‚ 0.341796875 cpu‚25.446714928248735 disk.dm-0.out‚24836.08984375 disk.sdb2.out‚200.6474609375 disk.md0.out‚ 192.1796875 disk.md0p1.in‚ 0.0078125 disk.md0p1.out‚ 192.1796875 disk.rbd2.in‚0 disk.rbd0.in‚0 disk.nvme0n1.in‚ 4.33984375 disk.sdb2.in‚0 disk.sda1.out‚0 disk.rbd2.out‚ 0.50390625 disk.sda2.in‚ 0.0078125 disk.nvme1n1.in‚4.4375 network.recv‚712471 disk.sda.in‚ 0.0078125" proc.memory.rssMB‚ 3049.7890625) proc.memory.percent‚18.61443519592285a>\€M¬ ‚(: „ÍèԲ픰 disk.rbd2.in‚0 disk.nvme1n1.in‚4.4375 disk.sdb.in‚0 disk.sda.in‚ 0.0078125$ disk./.usageGB‚2979.739814758301) proc.memory.percent‚18.61443519592285 disk.dm-0.out‚25019.17578125! disk.sdb2.out‚202.08154296875 disk.sda1.out‚0 disk.md0.in‚ 0.0078125 disk.md0.out‚ 193.53515625 disk.rbd0.in‚0" proc.memory.rssMB‚ 3049.7890625 cpu‚25.445649861289525) disk./.usagePercent‚83.31477233864052" disk.nvme0n1.out‚ 12514.3984375 disk.sdb2.in‚0 disk.md0p1.out‚ 193.53515625 disk.sda.out‚202.08154296875 disk.dm-0.in‚ 8.77734375 disk.sdb1.in‚0 disk.sdb.out‚202.08154296875! disk.sda2.out‚202.08154296875$ proc.memory.availableMB‚6.671875 network.sent‚ 2.473802e+06 disk.sdb1.out‚0 disk.rbd2.out‚ 0.50390625 disk.rbd0.out‚ 0.341796875$ memory_percent‚99.95927810668945 proc.cpu.threads‚17 disk.sda2.in‚ 0.0078125# disk.nvme1n1.out‚12504.77734375 network.recv‚717269 disk.sda1.in‚0 disk.md0p1.in‚ 0.0078125 disk.nvme0n1.in‚ 4.33984375ÚôÛl’­ ‚(:‡ „ÍèÔÜÛˆ° gpu.0.gpu‚98.0 gpu.0.memory‚17+ gpu.0.memoryAllocated‚47.54737854003906* gpu.0.memoryAllocatedBytes‚ 40842887168 gpu.0.temp‚45.0 gpu.0.powerWatts‚254.608' gpu.0.enforcedPowerLimitWatts‚400.0 gpu.0.powerPercent‚63.652 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0 Œ/~ž® ‚(:“ “ÍèÔ…‡½® gpu.0.gpu‚98.0 gpu.0.memory‚18+ gpu.0.memoryAllocated‚47.54737854003906* gpu.0.memoryAllocatedBytes‚ 40842887168 gpu.0.temp‚45.0 gpu.0.powerWatts‚224.194' gpu.0.enforcedPowerLimitWatts‚400.0) gpu.0.powerPercent‚56.048500000000004 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚01Žbº2¯ ‚(:§ “ÍèÔÞíù® disk.sdb1.out‚0 disk.sdb2.in‚0 disk.sda2.in‚ 0.0078125 disk.nvme0n1.in‚ 4.33984375 disk.sdb2.out‚203.8125 disk.md0.in‚ 0.0078125 disk.sda2.out‚203.8125 disk.dm-0.in‚ 8.77734375$ memory_percent‚99.95701313018799 disk.sdb1.in‚0# disk.nvme0n1.out‚12719.58984375 disk.sda1.out‚0 disk.sdb.out‚203.8125 disk.sda.in‚ 0.0078125 disk.md0p1.out‚ 195.19921875 disk.dm-0.out‚25224.74609375" proc.memory.rssMB‚ 3049.7890625) proc.memory.percent‚18.61443519592285 cpu‚25.444892558810853 disk.rbd2.in‚0 disk.sdb.in‚0 disk.sda.out‚203.8125 disk.rbd0.out‚ 0.341796875% disk./.usageGB‚2980.0684204101562 disk.sda1.in‚0 disk.md0.out‚ 195.19921875 disk.rbd2.out‚ 0.50390625 proc.cpu.threads‚17 disk.md0p1.in‚ 0.0078125 disk.nvme1n1.in‚4.4375 disk.nvme1n1.out‚ 12505.15625 disk.rbd0.in‚0& proc.memory.availableMB‚ 7.04296875 network.sent‚ 2.479166e+06 network.recv‚721076) disk./.usagePercent‚83.32396029019856)MJ2° - _wandbruntime‚2116  _runtime‚2116.wpı ¾ % lr-AdamW/pg1‚0.001104044392850051 % lr-AdamW/pg2‚0.001104044392850051 trainer/global_step‚1199 % _timestamp‚1.7884873177467973e+09 _runtime‚2116.848218953 _step‚93´ñ–—Ȳ  _runtime‚2116.848218953 _step‚93 % lr-AdamW/pg1‚0.001104044392850051 % lr-AdamW/pg2‚0.001104044392850051 trainer/global_step‚1199 % _timestamp‚1.7884873177467973e+09]]:2³ - _wandbruntime‚2116  _runtime‚2116d†Ï^ñ'´ ëO TGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚17004575 ]JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚56876451744256 TEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 1515847168 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚23979 P@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 38610163200 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚6 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 PCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚22331001 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚23680 YFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚56876351122944 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 963120128 \HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚250880115720144 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 100621312 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 VFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 37073453056 MDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚1534 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 YEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚253736838530213 RBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 40290484224 ZKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 3217031168 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚2684 NFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚986 dQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚53303115293696 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚71523328 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 577553920 A8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚3681 VGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 3140738480 D;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚3681 `MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚53303086195712 trainer/global_step‚1199 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚19688 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚23680 M=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 38610163200 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚25427968 VGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 2516362240 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚2684 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 RCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 2539798100 TGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚22350683 IDeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚264114012239360 IDeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 UADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚253734298732113 SGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚5310283 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚6 P@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 36985723393 H?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚1534 YEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚261256213140480 RCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 2541790208 VGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 3142752256 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 XIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 3217031168 VBDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated‚264116554029568 UADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚264114012239360 M@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚13434438 O?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 40290484224 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚5310286 SGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚8124155 F=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚3681 H?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚2147 NFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚330 ]IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚261256238568448 [IDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 2860315461120 SDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 3142752256 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚23979 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚8124721 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚25427968 % _timestamp‚1.7884873177478766e+09 _runtime‚2116.849375729 TDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 35913877504 M@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚39355258 WGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 34283657216 ^LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 2856697382101 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚3 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 TDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 40290484224 [KDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 37073453056 KBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚1534 L?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚22331001 UFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 3217031168 \HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚261256213140480 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 WGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 35913877504 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 L?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚17004569 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚19682 `LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚261256238568448 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚19688 PCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚17004569 `LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚250880141148112 YIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 37073453056ÖxEæY'µ ïO UADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚264114012239360 M@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚13434438 O?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 40290484224 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚5310286 SGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚8124155 F=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚3681 H?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚2147 NFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚330 ]IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚261256238568448 [IDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 2860315461120 SDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 3142752256 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚23979 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚8124721 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚25427968 % _timestamp‚1.7884873177478766e+09 _runtime‚2116.849375729 TDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 35913877504 M@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚39355258 WGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 34283657216 ^LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 2856697382101 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚3 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 TDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 40290484224 [KDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 37073453056 KBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚1534 L?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚22331001 UFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 3217031168 \HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚261256213140480 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 WGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 35913877504 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 L?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚17004569 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚19682 `LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚261256238568448 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚19688 PCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚17004569 `LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚250880141148112 YIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 37073453056 TGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚17004575 ]JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚56876451744256 TEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 1515847168 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚23979 P@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 38610163200 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚6 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 PCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚22331001 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚23680 YFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚56876351122944 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 963120128 \HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚250880115720144 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 100621312 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 VFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 37073453056 MDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚1534 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 YEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚253736838530213 RBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 40290484224 ZKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 3217031168 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚2684 NFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚986 dQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚53303115293696 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚71523328 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 577553920 A8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚3681 VGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 3140738480 D;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚3681 `MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚53303086195712 trainer/global_step‚1199 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚19688 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚23680 M=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 38610163200 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚25427968 VGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 2516362240 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚2684 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 RCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 2539798100 TGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚22350683 IDeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚264114012239360 IDeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 UADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚253734298732113 SGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚5310283 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚6 P@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 36985723393 H?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚1534 YEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚261256213140480 RCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 2541790208 VGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 3142752256 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 XIDeviceStatsMonitor.on_úiïœtrain_batch_start/reserved_bytes.small_pool.current‚ 3217031168 VBDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated‚264116554029568^¡Åï!2¶ - _wandbruntime‚2118  _runtime‚2118æ†ähû&· õM D;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚3681 SDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 3217031168 NADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚17018459 ]KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 3576187068928 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 I@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚2147 ^JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚261470184638976 bODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚53347245618688 RBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 40290484224 M=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 40290484224 NADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚22369051 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1309 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚3 LDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚330 K>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚39387516 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚5314652 J=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚22349343 UEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 35913877504 XIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 3217031168 KBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚2147 KBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚1534 WCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚264332856209408 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚25427968 B9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚3681 aODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 3576264870400 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 trainer/global_step‚1199 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚6 XFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 2860155194880 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 577553920 I@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚1534 TEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 2516375552 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚5314649 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 P@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 40290484224 PADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 2541803520 G:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚39367802 REDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚22369051 ?6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚3681 ^JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚251085656808608 NADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚22349343 C6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚39367802 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚17018459 REDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚17018465 XFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 2856536147703 _step‚95 G:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚39387516 EDeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚19714 NADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚17018465 T@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚264332856209408 M>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 2541803520 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚19708 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚23680 WDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚56923403589632 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚29097984 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚23979 UEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 34283657216 YGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 2862671570432 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 WCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚261470159211008 TEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 3142752256 K>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚13445231 BDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 36985723393 \JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 2859050518043 F=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚2147 WHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 2516375552 OBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚13445811 ^KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚53347216520704 S?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚253942167528343 ?6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚3681 UCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 2860155194880 [HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚56923510489088 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 RBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 35913877504 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚8131159 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 ZFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚251085631380640 N>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 38610163200 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 963120128 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚23979 FDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 ZFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚261470159211008 LDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚986 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚25427968 WHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 2514370340 VGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 3217031168 % _timestamp‚1.7884873192604942e+09 [GDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚261470184638976 S?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚264330314405888 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚577 TEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 3140738480Šÿ&¸ ùM KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 F=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚1534 _runtime‚2118.362397408 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚19708 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚580 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 N>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 36985723393 \JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 2859050518043 F=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚2147 WHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 2516375552 OBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚13445811 ^KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚53347216520704 S?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚253942167528343 ?6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚3681 UCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 2860155194880 [HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚56923510489088 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 RBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 35913877504 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚8131159 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 ZFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚251085631380640 N>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 38610163200 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 963120128 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚23979 FDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 ZFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚261470159211008 LDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚986 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚25427968 WHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 2514370340 VGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 3217031168 % _timestamp‚1.7884873192604942e+09 [GDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚261470184638976 S?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚264330314405888 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚577 TEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 3140738480 D;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚3681 SDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 3217031168 NADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚17018459 ]KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 3576187068928 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 I@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚2147 ^JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚261470184638976 bODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚53347245618688 RBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 40290484224 M=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 40290484224 NADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚22369051 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1309 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚3 LDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚330 K>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚39387516 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚5314652 J=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚22349343 UEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 35913877504 XIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 3217031168 KBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚2147 KBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚1534 WCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚264332856209408 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚25427968 B9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚3681 aODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 3576264870400 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 trainer/global_step‚1199 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚6 XFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 2860155194880 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 577553920 I@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚1534 TEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 2516375552 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚5314649 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 P@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 40290484224 PADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 2541803520 G:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚39367802 REDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚22369051 ?6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚3681 ^JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚251085656808608 NADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚22349343 C6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚39367802 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚17018459 REDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚17018465 XFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 2856536147703 _step‚95 G:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚39387516 EDeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚19714 NADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚17018465 T@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚264332856209408 M>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 2541803520 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚19708 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚23680 WDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚56923403589632 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚29097984 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚23979 UEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 34283657216 YGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 2862671570432 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 WCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚261470159211008 TEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 3142752256 K>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚13445231 B&Purity/2FRt1SRqqt0FBt/event_proportion‚0.03181703761219978 ?&Purity/2FRt1SRqqt2FBt/event_proportion‚0.001890120911411941 ("Purity/2FRt2SRqqt0FBt/event_purity‚0 ?&Purity/2FRt3SRqqt*FBt/event_proportion‚0.004032257944345474 >&Purity/0FRt2SRqqt*FBt/event_proportion‚0.05361643061041832 9"Purity/1FRt1SRqqt1FBt/event_purity‚0.0332222655415535 7 Purity/3FRt0SRqqt*FBt/FRt_purity‚0.2929293215274811 ,&Purity/4FRt*SRqqt1FBt/event_proportion‚0 ,&Purity/4FRt0SRqqt1FBt/event_proportion‚0 particle/accuracy_2_of_4‚1 ("Purity/1FRt*SRqqt4FBt/event_purity‚0 ?&Purity/1FRt*SRqqt2FBt/event_proportion‚0.022303426638245583 7 Purity/*FRt1SRqqt0FBt/FRt_purity‚0.2850886881351471 ,&Purity/*FRt4SRqqt2FBt/event_proportion‚0 @&Purity/1FRt4SRqqt*FBt/event_proportion‚0.0003780241822823882 ,&Purity/4FRt*SRqqt2FBt/event_proportion‚0 ("Purity/1FRt0SRqqt4FBt/event_purity‚0 ("Purity/2FRt*SRqqt3FBt/event_purity‚0 7 Purity/1FRt1SRqqt3FBt/FBt_purity‚0.8333333730697632 % _timestamp‚1.7884873831279972e+09 ?&Purity/*FRt3SRqqt1FBt/event_proportion‚0.002142137149348855 9"Purity/2FRt*SRqqt0FBt/SRqqt_purity‚0.3663597106933594 ("Purity/3FRt1SRqqt0FBt/event_purity‚0 ,&Purity/3FRt2SRqqt3FBt/event_proportion‚0 @&Purity/4FRt1SRqqt0FBt/event_proportion‚6.300403038039804e-05 +"Purity/*FRt4SRqqt1FBt/SRqqt_purity‚0.25 7 Purity/1FRt2SRqqt2FBt/FRt_purity‚0.4305555820465088 jet/accuracy_7_of_7‚0 @&Purity/0FRt2SRqqt3FBt/event_proportion‚6.300403038039804e-05 8 Purity/2FRt2SRqqt*FBt/FRt_purity‚0.27208051085472107 particle/accuracy_4_of_6‚1 7 Purity/*FRt2SRqqt2FBt/FBt_purity‚0.9912281036376953 =&Purity/*FRt*SRqqt*FBt/event_proportion‚0.9591733813285828 8 Purity/2FRt1SRqqt*FBt/FRt_purity‚0.29895952343940735 particle/accuracy_2_of_3‚1 ("Purity/4FRt1SRqqt*FBt/event_purity‚0 7 Purity/*FRt3SRqqt1FBt/FBt_purity‚0.9523809552192688 ,&Purity/*FRt3SRqqt4FBt/event_proportion‚0 :"Purity/0FRt0SRqqt2FBt/event_purity‚0.30155453085899353 7 Purity/2FRt1SRqqt0FBt/FRt_purity‚0.2757420837879181 ;"Purity/*FRt2SRqqt1FBt/event_purity‚0.004378923214972019 ,&Purity/0FRt3SRqqt4FBt/event_proportion‚0 =&Purity/1FRt1SRqqt*FBt/event_proportion‚0.1723790317773819 7 Purity/2FRt2SRqqt0FBt/FRt_purity‚0.2805858254432678 ("Purity/2FRt3SRqqt*FBt/event_purity‚0 @&Purity/3FRt0SRqqt0FBt/event_proportion‚0.0015120967291295528 @&Purity/4FRt2SRqqt*FBt/event_proportion‚6.300403038039804e-05 & Purity/2FRt0SRqqt3FBt/FBt_purity‚1 7 Purity/*FRt0SRqqt3FBt/FRt_purity‚0.4000000059604645 9"Purity/*FRt2SRqqt*FBt/SRqqt_purity‚0.3802308142185211 ?&Purity/2FRt*SRqqt2FBt/event_proportion‚0.005418346729129553 ,&Purity/3FRt3SRqqt3FBt/event_proportion‚0 particle/accuracy_1_of_3‚1 8 Purity/*FRt*SRqqt3FBt/FRt_purity‚0.44117647409439087 :"Purity/*FRt*SRqqt1FBt/event_purity‚0.41015782952308655 8 Purity/*FRt1SRqqt*FBt/FRt_purity‚0.29629623889923096 @&Purity/*FRt4SRqqt0FBt/event_proportion‚0.0012600806076079607 :"Purity/0FRt*SRqqt0FBt/SRqqt_purity‚0.33687901496887207 :"Purity/0FRt1SRqqt2FBt/event_purity‚0.06740740686655045 @&Purity/1FRt3SRqqt1FBt/event_proportion‚0.0011340725468471646 ,jet/accuracy_3_of_3‚0.022896582260727882 :"Purity/*FRt1SRqqt*FBt/event_purity‚0.15045492351055145 >&Purity/*FRt3SRqqt0FBt/event_proportion‚0.01827116869390011 >&Purity/1FRt*SRqqt1FBt/event_proportion‚0.09929435700178146 7 Purity/1FRt1SRqqt*FBt/FBt_purity‚0.9840551614761353 @&Purity/2FRt2SRqqt2FBt/event_propºò$ùortion‚6.300403038039804e-05 +jet/accuracy_3_of_6‚0.17543859779834747 :"Purity/2FRt3SRqqt1FBt/SRqqt_purity‚0.23333331942558289 ?&Purity/*FRt2SRqqt2FBt/event_proportion‚0.002079133177176118 9"Purity/*FRt*SRqqt3FBt/SRqqt_purity‚0.9047619104385376 ("Purity/0FRt2SRqqt3FBt/SRqqt_purity‚1 8 Purity/1FRt3SRqqt*FBt/FRt_purity‚0.24082182347774506 9"Purity/2FRt1SRqqt1FBt/SRqqt_purity‚0.5210061073303223 7 Purity/3FRt0SRqqt0FBt/FRt_purity‚0.1993464082479477 -"Purity/*FRt4SRqqt0FBt/SRqqt_purity‚0.3125 & Purity/*FRt4SRqqt1FBt/FBt_purity‚1 7 Purity/1FRt0SRqqt3FBt/FRt_purity‚0.3928571343421936 ,&Purity/2FRt3SRqqt2FBt/event_proportion‚0 @&Purity/3FRt3SRqqt*FBt/event_proportion‚0.0005670362734235823 @&Purity/4FRt0SRqqt*FBt/event_proportion‚6.300403038039804e-05 particle/accuracy_1_of_2‚1 9"Purity/*FRt*SRqqt*FBt/SRqqt_purity‚0.3837721347808838 ("Purity/0FRt0SRqqt0FBt/event_purity‚1 ?&Purity/2FRt*SRqqt1FBt/event_proportion‚0.021169355139136314 7 Purity/2FRt*SRqqt1FBt/FRt_purity‚0.3549676239490509 :"Purity/2FRt3SRqqt0FBt/SRqqt_purity‚0.35349929332733154 8 Purity/3FRt2SRqqt*FBt/FRt_purity‚0.26273152232170105 ,&Purity/3FRt2SRqqt4FBt/event_proportion‚0 ,&Purity/4FRt4SRqqt*FBt/event_proportion‚0 ;"Purity/*FRt2SRqqt0FBt/event_purity‚0.016516922041773796 ?&Purity/0FRt0SRqqt2FBt/event_proportion‚0.029674898833036423 ?&Purity/1FRt0SRqqt1FBt/event_proportion‚0.049773186445236206 ,&Purity/1FRt4SRqqt4FBt/event_proportion‚0 7 Purity/2FRt0SRqqt2FBt/FRt_purity‚0.4940476417541504 ?&Purity/2FRt2SRqqt*FBt/event_proportion‚0.019468246027827263 particle/accuracy_4_of_5‚1 7validation_average_jet_accuracy‚0.39937517046928406 :"Purity/*FRt*SRqqt3FBt/event_purity‚0.10057471692562103 7 Purity/*FRt2SRqqt*FBt/FRt_purity‚0.2737419605255127 :"Purity/0FRt0SRqqt3FBt/event_purity‚0.15333333611488342 ?&Purity/0FRt3SRqqt*FBt/event_proportion‚0.006741431541740894 @&Purity/0FRt3SRqqt1FBt/event_proportion‚0.0006300403038039804 ,&Purity/1FRt3SRqqt4FBt/event_proportion‚0 ,&Purity/4FRt4SRqqt0FBt/event_proportion‚0 7 Purity/*FRt0SRqqt4FBt/FBt_purity‚0.8999999761581421 =&Purity/*FRt0SRqqt*FBt/event_proportion‚0.3558467626571655 7 Purity/0FRt0SRqqt1FBt/FBt_purity‚0.9985072016716003 ,&Purity/1FRt3SRqqt3FBt/event_proportion‚0 7 Purity/2FRt1SRqqt2FBt/FBt_purity‚0.9868420958518982 ,&Purity/3FRt0SRqqt4FBt/event_proportion‚0 * Purity/*FRt*SRqqt4FBt/FBt_purity‚0.875 * Purity/0FRt*SRqqt4FBt/FBt_purity‚0.875 & Purity/2FRt3SRqqt1FBt/FBt_purity‚1 7 Purity/0FRt1SRqqt1FBt/FBt_purity‚0.9933437705039978 jet/accuracy_1_of_7‚1 ( Purity/*FRt*SRqqt4FBt/FRt_purity‚0.5 ("Purity/4FRt0SRqqt*FBt/event_purity‚0 9"Purity/1FRt*SRqqt1FBt/SRqqt_purity‚0.5021957159042358 7 Purity/1FRt2SRqqt*FBt/FBt_purity‚0.9976958632469177 @&Purity/2FRt4SRqqt0FBt/event_proportion‚0.0003150201519019902 @&Purity/3FRt*SRqqt0FBt/event_proportion‚0.0071824598126113415 @&Purity/4FRt*SRqqt*FBt/event_proportion‚0.0001890120911411941 ,&Purity/4FRt0SRqqt3FBt/event_proportion‚0 & Purity/1FRt0SRqqt4FBt/FRt_purity‚1 & Purity/3FRt0SRqqt2FBt/FBt_purity‚1 ,&Purity/0FRt3SRqqt3FBt/event_proportion‚0 ,&Purity/4FRt*SRqqt4FBt/event_proportion‚0 ("Purity/*FRt4SRqqt*FBt/event_purity‚0 & Purity/*FRt0SRqqt4FBt/FRt_purity‚1 + Purity/0FRt3SRqqt1FBt/FBt_purity‚0.8125 & Purity/4FRt1SRqqt*FBt/FRt_purity‚0 ("Purity/2FRt2SRqqt2FBt/SRqqt_purity‚0 ("Purity/4FRt2SRqqt0FBt/event_purity‚0 :"Purity/0FRt3SRqqt*FBt/event_purity‚0.01785714365541935 ("Purity/2FRt4SRqqt*FBt/event_purity‚0 & Purity/1FRt0SRqqt4FBt/FBt_purity‚1 ("Purity/4FRt2SRqqt*FBt/SRqqt_purity‚1 particle/accuracy_1_of_4‚1 ("Purity/4FRt*SRqqt0FBt/event_purity‚0 & Purity/2FRt2SRqqt2FBt/FBt_purity‚1 ("Purity/0FRt2SRqqt3FBt/event_purity‚0 >&Purity/*FRt1SRqqt0FBt/event_proportion‚0.33039313554763794 ,&Purity/4FRt*SRqqt3FBt/event_proportion‚0 particle/accuracy_3_of_4‚1 ("Purity/*FRt0SRqqt4FBt/event_purity‚0 >&Purity/*FRt2SRqqt0FBt/event_proportion‚0.10855595022439957 >&Purity/0FRt*SRqqt0FBt/event_proportion‚0.21931703388690948 ,&Purity/1FRt4SRqqt2FBt/event_proportion‚0 ,&Purity/1FRt4SRqqt3FBt/event_proportion‚0 ,&Purity/2FRt2SRqqt4FBt/event_proportion‚0 & Purity/3FRt1SRqqt*FBt/FBt_purity‚1 ,&Purity/3FRt1SRqqt4FBt/event_proportion‚0 +jet/accuracy_3_of_5‚0.09869074076414108 9"Purity/*FRt*SRqqt2FBt/SRqqt_purity‚0.5561156868934631 :"Purity/0FRt1SRqqt1FBt/event_purity‚0.15148796141147614 8 Purity/1FRt0SRqqt0FBt/FRt_purity‚0.21827785670757294 9"Purity/2FRt*SRqqt*FBt/SRqqt_purity‚0.3997207581996918 9"Purity/2FRt0SRqqt*FBt/event_purity‚0.0334082692861557 ,&Purity/2FRt4SRqqt2FBt/event_proportion‚0 ("Purity/3FRt0SRqqt1FBt/event_purity‚0 & Purity/3FRt*SRqqt2FBt/FBt_purity‚1 :"Purity/0FRt*SRqqt1FBt/SRqqt_purity‚0.49202239513397217 7 Purity/1FRt*SRqqt*FBt/FBt_purity‚0.9874643087387085 *jet/accuracy_1_of_5‚0.9576897621154785 ("Purity/1FRt1SRqqt4FBt/event_purity‚0 ("Purity/*FRt2SRqqt3FBt/event_purity‚0 >&Purity/*FRt0SRqqt0FBt/event_proportion‚0.15757308900356293 7 Purity/*FRt2SRqqt1FBt/FBt_purity‚0.9925950169563293 @&Purity/1FRt3SRqqt2FBt/event_proportion‚6.300403038039804e-05 8 Purity/2FRt3SRqqt*FBt/FRt_purity‚0.31300002336502075 ("Purity/3FRt1SRqqt*FBt/event_purity‚0 A&Purity/3FRt1SRqqt2FBt/event_proportion‚0.00025201612152159214 @&Purity/3FRt2SRqqt0FBt/event_proportion‚0.0024571572430431843 ,&Purity/4FRt3SRqqt2FBt/event_proportion‚0 6 Purity/*FRt0SRqqt0FBt/FRt_purity‚0.222781702876091 ,&Purity/*FRt4SRqqt4FBt/event_proportion‚0 @&Purity/1FRt1SRqqt4FBt/event_proportion‚6.300403038039804e-05 ,&Purity/2FRt*SRqqt4FBt/event_proportion‚0 ?&Purity/2FRt0SRqqt0FBt/event_proportion‚0.024571573361754417 & Purity/2FRt*SRqqt3FBt/FBt_purity‚1 6 Purity/*FRt*SRqqt0FBt/FRt_purity‚0.258646160364151 :"Purity/0FRt2SRqqt1FBt/SRqqt_purity‚0.46701231598854065 ,&Purity/2FRt3SRqqt3FBt/event_proportion‚0 @&Purity/4FRt*SRqqt0FBt/event_proportion‚0.0001890120911411941 & Purity/3FRt2SRqqt1FBt/FBt_purity‚1 particle/accuracy_5_of_7‚1 ("Purity/3FRt1SRqqt2FBt/event_purity‚0 7 Purity/1FRt0SRqqt2FBt/FBt_purity‚0.9809930324554443 ?&Purity/2FRt0SRqqt*FBt/event_proportion‚0.037046369165182114 @&Purity/2FRt0SRqqt3FBt/event_proportion‚6.300403038039804e-05 7 Purity/3FRt*SRqqt0FBt/FRt_purity‚0.2666666805744171 A&Purity/3FRt4SRqqt0FBt/event_proportion‚0.00012600806076079607 +"Purity/0FRt4SRqqt1FBt/SRqqt_purity‚0.25 *loss/val_total_loss‚43.154410914983565 @&Purity/0FRt4SRqqt1FBt/event_proportion‚6.300403038039804e-05 ?&Purity/1FRt3SRqqt0FBt/event_proportion‚0.007938507944345474 & Purity/1FRt3SRqqt1FBt/FBt_purity‚1 7 Purity/*FRt4SRqqt*FBt/FRt_purity‚0.3611111342906952 jet/accuracy_3_of_7‚0 & Purity/4FRt2SRqqt*FBt/FRt_purity‚0 9"Purity/*FRt1SRqqt1FBt/SRqqt_purity‚0.5316541790962219 ?&Purity/0FRt3SRqqt0FBt/event_proportion‚0.006111391354352236 =&Purity/1FRt*SRqqt*FBt/event_proportion‚0.4366179406642914 ,&Purity/3FRt0SRqqt3FBt/event_proportion‚0 particle/accuracy_1_of_7‚1 particle/accuracy_7_of_7‚1 ?&Purity/0FRt1SRqqt2FBt/event_proportion‚0.008757560513913631 :"Purity/1FRt*SRqqt*FBt/event_purity‚0.11178943514823914 :"Purity/2FRt2SRqqt0FBt/SRqqt_purity‚0.35229483246803284 ,&Purity/2FRt4SRqqt4FBt/event_proportion‚0 9"Purity/3FRt1SRqqt1FBt/SRqqt_purity‚0.6153846383094788 particle/accuracy_5_of_5‚1 ."Purity/0FRt4SRqqt*FBt/SRqqt_purity‚0.21875 7 Purity/*FRt*SRqqt2FBt/FBt_purity‚0.9772515892982483 7 Purity/*FRt1SRqqt1FBt/FBt_purity‚0.9942842721939087 @&Purity/1FRt*SRqqt3FBt/event_proportion‚0.0013230846961960196 ("Purity/1FRt3SRqqt*FBt/event_purity‚0 @&Purity/2FRt4SRqqt*FBt/event_proportion‚0.0003150201519019902 ("Purity/1FRt1SRqqt4FBt/SRqqt_purity‚1 ("Purity/*FRt2SRqqt3FBt/SRqqt_purity‚1 >&Purity/*FRt*SRqqt2FBt/event_proportion‚0.06747731566429138 7 Purity/*FRt0SRqqt2FBt/FBt_purity‚0.9814485311508179 ,&Purity/3FRt4SRqqt1FBt/event_proportion‚0 particle/accuracy_2_of_5‚1 ,&Purity/0FRt3SRqqt2FBt/event_proportion‚0 8 Purity/2FRt*SRqqt2FBt/FRt_purity‚0.47583335638046265 ,&Purity/3FRt3SRqqt1FBt/event_proportion‚0 7 Purity/*FRt3SRqqt*FBt/FRt_purity‚0.2614314556121826 7 Purity/0FRt*SRqqt2FBt/FBt_purity‚0.9741479754447937 7 Purity/0FRt2SRqqt1FBt/FBt_purity‚0.9953917264938354 :"Purity/2FRt*SRqqt0FBt/event_purity‚0.00949862226843834 7 Purity/2FRt0SRqqt1FBt/FRt_purity‚0.3909126818180084 :"Purity/3FRt1SRqqt0FBt/SRqqt_purity‚0.40833330154418945 & Purity/4FRt*SRqqt*FBt/FRt_purity‚0 7 Purity/*FRt3SRqqt0FBt/FRt_purity‚0.2844630479812622 7 Purity/0FRt0SRqqt2FBt/FBt_purity‚0.9796778559684753 & Purity/2FRt1SRqqt1FBt/FBt_purity‚1 ) Purity/*FRt1SRqqt4FBt/FBt_purity‚0.75 & Purity/4FRt*SRqqt0FBt/FRt_purity‚0 :"Purity/0FRt*SRqqt0FBt/event_purity‚0.26281315088272095 7 Purity/2FRt2SRqqt1FBt/FBt_purity‚0.9821428656578064 ,&Purity/3FRt*SRqqt4FBt/event_proportion‚0 particle/accuracy_5_of_6‚1 ("Purity/3FRt4SRqqt*FBt/event_purity‚0 A&Purity/0FRt*SRqqt4FBt/event_proportion‚0.00025201612152159214 8 Purity/1FRt1SRqqt1FBt/FRt_purity‚0.30853188037872314 ("Purity/3FRt*SRqqt*FBt/event_purity‚0 ) Purity/3FRt1SRqqt2FBt/FRt_purity‚0.25 & Purity/*FRt1SRqqt4FBt/FRt_purity‚0 @&Purity/3FRt0SRqqt2FBt/event_proportion‚6.300403038039804e-05 ,&Purity/4FRt1SRqqt2FBt/event_proportion‚0 ,&Purity/4FRt2SRqqt3FBt/event_proportion‚0 ,&Purity/4FRt3SRqqt*FBt/event_proportion‚0 ("Purity/0FRt0SRqqt4FBt/event_purity‚0 ("Purity/3FRt2SRqqt*FBt/event_purity‚0 :"Purity/*FRt3SRqqt1FBt/SRqqt_purity‚0.33862432837486267 7 Purity/1FRt*SRqqt2FBt/FRt_purity‚0.5199726223945618 :"Purity/1FRt0SRqqt2FBt/event_purity‚0.12273846566677094 9"Purity/3FRt*SRqqt1FBt/SRqqt_purity‚0.6611111164093018 ("Purity/*FRt4SRqqt0FBt/event_purity‚0 *"Purity/3FRt2SRqqt1FBt/SRqqt_purity‚0.5 @&Purity/*FRt*SRqqt4FBt/event_proportion‚0.0003780241822823882 9"Purity/0FRt1SRqqt2FBt/SRqqt_purity‚0.5231084227561951 >&Purity/2FRt*SRqqt0FBt/event_proportion‚0.07579384744167328 *jet/accuracy_1_of_4‚0.9090997576713562 particle/accuracy_4_of_7‚1 >&Purity/1FRt1SRqqt1FBt/event_proportion‚0.03944052383303642 & Purity/3FRt0SRqqt1FBt/FBt_purity‚1 7 Purity/*FRt4SRqqt0FBt/FRt_purity‚0.3611111342906952 & Purity/4FRt2SRqqt0FBt/FRt_purity‚0 7 Purity/0FRt1SRqqt2FBt/FBt_purity‚0.9537036418914795 :"Purity/1FRt0SRqqt0FBt/event_purity‚0.21827785670757294 8 Purity/3FRt*SRqqt*FBt/FRt_purity‚0.28674259781837463 ("Purity/0FRt3SRqqt1FBt/event_purity‚0 7 Purity/*FRt*SRqqt*FBt/FRt_purity‚0.2930887043476105 @&Purity/3FRt1SRqqt0FBt/event_proportion‚0.0025201612152159214 ( Purity/*FRt1SRqqt3FBt/FRt_purity‚0.5 ("Purity/4FRt*SRqqt0FBt/SRqqt_purity‚1 7 Purity/*FRt0SRqqt*FBt/FRt_purity‚0.2972099483013153 ,&Purity/2FRt4SRqqt1FBt/event_proportion‚0 8 Purity/2FRt3SRqqt0FBt/FRt_purity‚0.33338746428489685 ,&Purity/2FRt3SRqqt4FBt/event_proportion‚0 ("Purity/1FRt3SRqqt1FBt/event_purity‚0 ("Purity/2FRt3SRqqt0FBt/event_purity‚0 ("Purity/4FRt2SRqqt0FBt/SRqqt_purity‚1 =&Purity/*FRt*SRqqt0FBt/event_proportion‚0.6160534024238586 >&Purity/*FRt0SRqqt1FBt/event_proportion‚0.14591734111309052 9"Purity/*FRt2SRqqt0FBt/SRqqt_purity‚0.3623194396495819 >&Purity/*FRt2SRqqt1FBt/event_proportion‚0.02255544438958168 >&Purity/0FRt*SRqqt1FBt/event_proportion‚0.14787046611309052 7 Purity/1FRt2SRqqt1FBt/FRt_purity‚0.3320276439189911 ,&Purity/1FRt2SRqqt3FBt/event_proportion‚0 & Purity/*FRt3SRqqt2FBt/FBt_purity‚1 7 Purity/1FRt*SRqqt3FBt/FBt_purity‚0.9270833134651184 8 Purity/1FRt0SRqqt*FBt/FRt_purity‚0.29792314767837524 9"Purity/1FRt1SRqqt*FBt/SRqqt_purity‚0.4011145234107971 +jet/accuracy_2_of_5‚0.39338332414627075 particle/accuracy_4_of_4‚1 jet/accuracy_2_of_7‚0.5 :"Purity/0FRt1SRqqt*FBt/SRqqt_purity‚0.37868180871009827 =&Purity/0FRt1SRqqt0FBt/event_proportion‚0.1694808453321457 ?&Purity/2FRt1SRqqt1FBt/event_proportion‚0.007875503972172737 & Purity/0FRt4SRqqt1FBt/FBt_purity‚1 @&Purity/1FRt4SRqqt0FBt/event_proportion‚0.0003780241822823882 7 Purity/2FRt*SRqqt*FBt/FBt_purity‚0.9907397627830505 ,&Purity/3FRt4SRqqt2FBt/event_proportion‚0 ,&Purity/4FRt4SRqqt1FBt/event_proportion‚0 ("Purity/2FRt2SRqqt2FBt/event_purity‚0 >&Purity/0FRt*SRqqt2FBt/event_proportion‚0.03944052383303642 7 Purity/*FRt3SRqqt*FBt/FBt_purity‚0.9545454382896423 9"Purity/0FRt*SRqqt2FBt/SRqqt_purity‚0.5198147892951965 9"Purity/0FRt0SRqqt*FBt/event_purity‚0.8052219152450562 9"Purity/2FRt*SRqqt1FBt/SRqqt_purity‚0.5078203678131104 ("Purity/2FRt2SRqqt1FBt/event_purity‚0 8 Purity/3FRt0SRqqt1FBt/FRt_purity‚0.42307692766189575 ,&Purity/3FRt1SRqqt3FBt/event_proportion‚0 jet/accuracy_5_of_7‚0 :"Purity/*FRt*SRqqt*FBt/event_purity‚0.22082605957984924 @&Purity/3FRt*SRqqt1FBt/event_proportion‚0.0024571572430431843 ,&Purity/3FRt4SRqqt4FBt/event_proportion‚0 +jet/accuracy_2_of_3‚0.22940385341644287 +particle/specificity‚0.9380465745925903 ("Purity/0FRt4SRqqt0FBt/event_purity‚0 ("Purity/1FRt2SRqqt0FBt/event_purity‚0 ,&Purity/1FRt2SRqqt4FBt/event_proportion‚0 @&Purity/2FRt*SRqqt3FBt/event_proportion‚6.300403038039804e-05 @&Purity/3FRt2SRqqt1FBt/event_proportion‚0.0003780241822823882 9"Purity/*FRt1SRqqt3FBt/event_purity‚0.0714285746216774 9"Purity/*FRt3SRqqt2FBt/SRqqt_purity‚0.6666666865348816 7 Purity/1FRt0SRqqt2FBt/FRt_purity‚0.5651761293411255 ;"Purity/1FRt1SRqqt0FBt/event_purity‚0.012274806387722492 ?&Purity/2FRt0SRqqt2FBt/event_proportion‚0.003465221729129553 ,&Purity/3FRt3SRqqt2FBt/event_proportion‚0 7 Purity/1FRt1SRqqt1FBt/FBt_purity‚0.9949740171432495 +jet/accuracy_2_of_2‚0.10095174610614777 7 Purity/3FRt3SRqqt0FBt/FRt_purity‚0.3333333432674408 7 Purity/2FRt*SRqqt2FBt/FBt_purity‚0.9911110997200012 *jet/accuracy_1_of_3‚0.8526424169540405 trainer/global_step‚1207 9"Purity/*FRt0SRqqt1FBt/event_purity‚0.6989607810974121 7 Purity/*FRt0SRqqt2FBt/FRt_purity‚0.5266392230987549 8 Purity/2FRt1SRqqt1FBt/FRt_purity‚0.33045312762260437 & Purity/*FRt4SRqqt*FBt/FBt_purity‚1 9"Purity/0FRt3SRqqt*FBt/SRqqt_purity‚0.3203420639038086 :"Purity/2FRt4SRqqt*FBt/SRqqt_purity‚0.30000001192092896 ("Purity/2FRt4SRqqt0FBt/event_purity‚0 ("Purity/4FRt2SRqqt*FBt/event_purity‚0 7 Purity/*FRt*SRqqt2FBt/FRt_purity‚0.4761085510253906 @&Purity/*FRt2SRqqt3FBt/event_proportion‚6.300403038039804e-05 ,&Purity/0FRt2SRqqt4FBt/event_proportion‚0 7 Purity/2FRt2SRqqt1FBt/FRt_purity‚0.2690476179122925 -"Purity/*FRt4SRqqt*FBt/SRqqt_purity‚0.3125 ( Purity/2FRt4SRqqt*FBt/FRt_purity‚0.5 8"Purity/3FRt3SRqqt0FBt/SRqqt_purity‚0.380952388048172 >&Purity/0FRt*SRqqt*FBt/event_proportion‚0.40996724367141724 9"Purity/1FRt2SRqqt1FBt/SRqqt_purity‚0.4277649521827698 :"Purity/2FRt3SRqqt*FBt/SRqqt_purity‚0.33590471744537354 @&Purity/3FRt1SRqqt1FBt/event_proportion‚0.0009450604557059705 particle/accuracy_2_of_7‚1 7 Purity/3FRt*SRqqt2FBt/FRt_purity‚0.2666666805744171 + Purity/0FRt3SRqqt*FBt/FBt_purity‚0.8125 ;"Purity/2FRt1SRqqt0FBt/event_purity‚0.002481389557942748 ("Purity/1FRt2SRqqt2FBt/event_purity‚0 ?&Purity/0FRt2SRqqt1FBt/event_proportion‚0.009261592291295528 9"Purity/1FRt1SRqqt1FBt/SRqqt_purity‚0.5551432371139526 ?&Purity/1FRt3SRqqt*FBt/event_proportion‚0.009135584346950054 @&Purity/4FRt1SRqqt*FBt/event_proportion‚6.300403038039804e-05 *jet/accuracy_2_of_4‚0.3503338098526001 & Purity/2FRt3SRqqt*FBt/FBt_purity‚1 +"Purity/3FRt4SRqqt0FBt/SRqqt_purity‚0.25 ;"Purity/*FRt2SRqqt*FBt/event_purity‚0.014280671253800392 7 Purity/0FRt0SRqqt*FBt/FBt_purity‚0.9847189784049988 7 Purity/1FRt*SRqqt*FBt/FRt_purity‚0.2943839430809021 7 Purity/1FRt0SRqqt3FBt/FBt_purity‚0.9285714030265808 ("Purity/0FRt*SRqqt4FBt/event_purity‚0 +jet/accuracy_4_of_6‚0.02631578966975212 ("Purity/1FRt4SRqqt0FBt/event_purity‚0 7 Purity/1FRt4SRqqt0FBt/FRt_purity‚0.1666666716337204 7 Purity/0FRt*SRqqt3FBt/FBt_purity‚0.8724279999732971 7 Purity/0FRt1SRqqt*FBt/FBt_purity‚0.9818498492240906 9"Purity/*FRt1SRqqt*FBt/SRqqt_purity‚0.3919236660003662 :"Purity/1FRt*SRqqt2FBt/event_purity‚0.08720673620700836 7 Purity/1FRt2SRqqt1FBt/FBt_purity‚0.9967741966247559 8 Purity/3FRt1SRqqt*FBt/FRt_purity‚0.25617286562919617 8"Purity/3FRt2SRqqt0FBt/SRqqt_purity‚0.364130437374115 ("Purity/*FRt4SRqqt1FBt/event_purity‚0 ( Purity/2FRt0SRqqt3FBt/FRt_purity‚0.5 9"Purity/*FRt1SRqqt3FBt/SRqqt_purity‚0.9047619104385376 7 Purity/*FRt0SRqqt3FBt/FBt_purity‚0.9013890027999878 7 Purity/2FRt*SRqqt0FBt/FRt_purity‚0.2658703923225403 9"Purity/3FRt*SRqqt0FBt/SRqqt_purity‚0.3478032946586609 @&Purity/3FRt2SRqqt*FBt/event_proportion‚0.0028351815417408943 7 Purity/*FRt*SRqqt3FBt/FBt_purity‚0.8932951092720032 >&Purity/1FRt2SRqqt*FBt/event_proportion‚0.05727066472172737 9"Purity/1FRt3SRqqt0FBt/SRqqt_purity‚0.3606502413749695 @&Purity/3FRt3SRqqt0FBt/event_proportion‚0.0005670362734235823 ( Purity/3FRt4SRqqt*FBt/FRt_purity‚0.5 ("Purity/4FRt1SRqqt0FBt/event_purity‚0 9"Purity/0FRt*SRqqt*FBt/event_purity‚0.3940039575099945 ("Purity/1FRt2SRqqt*FBt/event_purity‚0 ,&Purity/4FRt3SRqqt0FBt/event_proportion‚0 ( Purity/3FRt4SRqqt0FBt/FRt_purity‚0.5 & Purity/*FRt3SRqqt2FBt/FRt_purity‚0 ( Purity/1FRt1SRqqt3FBt/FRt_purity‚0.5 epoch‚1 ?&Purity/*FRt1SRqqt2FBt/event_proportion‚0.017011089250445366 :"Purity/*FRt2SRqqt1FBt/SRqqt_purity‚0.46048206090927124 ("Purity/1FRt*SRqqt3FBt/event_purity‚0 8 Purity/2FRt*SRqqt*FBt/FRt_purity‚0.29335853457450867 7 Purity/3FRt2SRqqt1FBt/FRt_purity‚0.2222222238779068 ("Purity/3FRt3SRqqt*FBt/event_purity‚0 >&Purity/0FRt0SRqqt1FBt/event_proportion‚0.08606351166963577 7 Purity/3FRt1SRqqt0FBt/FRt_purity‚0.2361111342906952 9"Purity/*FRt*SRqqt1FBt/SRqqt_purity‚0.5007259249687195 :"Purity/*FRt*SRqqt0FBt/SRqqt_purity‚0.34769973158836365 7 Purity/*FRt2SRqqt1FBt/FRt_purity‚0.3186286389827728 6 Purity/0FRt0SRqqt3FBt/FBt_purity‚0.893333375453949 =&Purity/0FRt1SRqqt*FBt/event_proportion‚0.2305317521095276 7 Purity/1FRt0SRqqt*FBt/FBt_purity‚0.9882248044013977 ,&Purity/4FRt1SRqqt3FBt/event_proportion‚0 ,&Purity/4FRt2SRqqt4FBt/event_proportion‚0 7 Purity/0FRt*SRqqt*FBt/FBt_purity‚0.9832651615142822 ,&Purity/0FRt4SRqqt2FBt/event_proportion‚0 8 Purity/1FRt*SRqqt0FBt/FRt_purity‚0.25606444478034973 ,&Purity/4FRt4SRqqt2FBt/event_proportion‚0 ("Purity/2FRt3SRqqt1FBt/event_purity‚0 7 Purity/1FRt4SRqqt*FBt/FRt_purity‚0.1666666716337204 7 Purity/0FRt1SRqqt3FBt/FBt_purity‚0.8333333730697632 :"Purity/*FRt3SRqqt0FBt/SRqqt_purity‚0.35146406292915344 ?&Purity/2FRt3SRqqt0FBt/event_proportion‚0.003654233878478408 7 Purity/*FRt0SRqqt*FBt/FBt_purity‚0.9869210720062256 9"Purity/1FRt*SRqqt0FBt/SRqqt_purity‚0.3531774580478668 9"Purity/1FRt0SRqqt*FBt/event_purity‚0.2315816879272461 ("Purity/2FRt1SRqqt1FBt/event_purity‚0 +jet/accuracy_3_of_4‚0.06716153025627136 :"Purity/*FRt2SRqqt2FBt/SRqqt_purity‚0.37280702590942383 ("Purity/4FRt1SRqqt*FBt/SRqqt_purity‚1 ("Purity/1FRt0SRqqt3FBt/event_purity‚0 7 Purity/*FRt*SRqqt*FBt/FBt_purity‚0.9858607053756714 ;"Purity/*FRt1SRqqt2FBt/event_purity‚0.044637616723775864 ?&Purity/0FRt0SRqqt3FBt/event_proportion‚0.002583165420219302 A&Purity/1FRt1SRqqt3FBt/event_proportion‚0.00025201612152159214 ,&Purity/2FRt1SRqqt3FBt/event_proportion‚0 jet/accuracy_6_of_6‚0 8"Purity/*FRt0SRqqt2FBt/event_purity‚0.226539745926857 9"Purity/*FRt1SRqqt0FBt/SRqqt_purity‚0.3377826511859894 :"Purity/*FRt3SRqqt*FBt/SRqqt_purity‚0.35035985708236694 >&Purity/1FRt0SRqqt0FBt/event_proportion‚0.13142640888690948 ,&Purity/2FRt0SRqqt4FBt/event_proportion‚0 >&Purity/2FRt1SRqqt*FBt/event_proportion‚0.04158266261219978 ("Purity/2FRt2SRqqt*FBt/event_purity‚0 & Purity/3FRt*SRqqt*FBt/FBt_purity‚1 :"Purity/1FRt0SRqqt1FBt/event_purity‚0.30789756774902344 & Purity/2FRt0SRqqt1FBt/FBt_purity‚1 @&Purity/3FRt0SRqqt*FBt/event_proportion‚0.0027091733645647764 ( Purity/2FRt4SRqqt0FBt/FRt_purity‚0.5 & Purity/3FRt1SRqqt2FBt/FBt_purity‚1 ) Purity/1FRt1SRqqt4FBt/FBt_purity‚0.75 ("Purity/4FRt*SRqqt*FBt/SRqqt_purity‚1 @&Purity/1FRt0SRqqt3FBt/event_proportion‚0.0010710685746744275 >&Purity/*FRt*SRqqt1FBt/event_proportion‚0.27079132199287415 7 Purity/*FRt1SRqqt1FBt/FRt_purity‚0.3159247934818268 ?&Purity/*FRt3SRqqt*FBt/event_proportion‚0.020476309582591057 >&Purity/1FRt2SRqqt0FBt/event_proportion‚0.04731602966785431 >&Purity/2FRt*SRqqt*FBt/event_proportion‚0.10244455933570862 & Purity/1FRt1SRqqt4FBt/FRt_purity‚0 9"Purity/0FRt2SRqqt*FBt/SRqqt_purity‚0.3772554099559784 ,&Purity/4FRt0SRqqt2FBt/event_proportion‚0 8"Purity/0FRt3SRqqt0FBt/SRqqt_purity‚0.313492089509964 9"Purity/0FRt2SRqqt2FBt/SRqqt_purity‚0.4423076808452606 *"Purity/3FRt*SRqqt2FBt/SRqqt_purity‚0.5 7 Purity/3FRt3SRqqt*FBt/FRt_purity‚0.3333333432674408 9"Purity/1FRt3SRqqt2FBt/SRqqt_purity‚0.6666666865348816 9"Purity/0FRt1SRqqt0FBt/event_purity‚0.3298182785511017 @&Purity/0FRt4SRqqt*FBt/event_proportion‚0.0005040322430431843 9"Purity/2FRt*SRqqt2FBt/SRqqt_purity‚0.6710526347160339 _runtime‚2182.231633732 :"Purity/1FRt*SRqqt1FBt/event_purity‚0.16590532660484314 8"Purity/0FRt2SRqqt0FBt/SRqqt_purity‚0.360020250082016 9"Purity/2FRt1SRqqt2FBt/SRqqt_purity‚0.7061403393745422 @&Purity/4FRt0SRqqt0FBt/event_proportion‚6.300403038039804e-05 ("Purity/1FRt*SRqqt3FBt/SRqqt_purity‚1 ?&Purity/1FRt0SRqqt2FBt/event_proportion‚0.015120967291295528 >&Purity/0FRt1SRqqt1FBt/event_proportion‚0.05185231938958168 6 Purity/1FRt*SRqqt2FBt/FBt_purity‚0.977053165435791 9"Purity/1FRt1SRqqt2FBt/event_purity‚0.0357142873108387 8 Purity/1FRt3SRqqt1FBt/FRt_purity‚0.24444444477558136 <"Purity/2FRt1SRqqt*FBt/event_purity‚0.0015360983088612556 9"Purity/1FRt4SRqqt*FBt/SRqqt_purity‚0.4166666567325592 + Purity/1FRt*SRqqt3FBt/FRt_purity‚0.4375 :"Purity/1FRt2SRqqt0FBt/SRqqt_purity‚0.36267656087875366 @&Purity/1FRt2SRqqt2FBt/event_proportion‚0.0010080644860863686 7 Purity/2FRt1SRqqt*FBt/FBt_purity‚0.9964157938957214 :"Purity/1FRt1SRqqt0FBt/SRqqt_purity‚0.34054625034332275 7 Purity/2FRt2SRqqt*FBt/FBt_purity‚0.9821428656578064 particle/accuracy_3_of_7‚1 :"Purity/0FRt1SRqqt3FBt/event_purity‚0.10000000149011612 ("Purity/2FRt0SRqqt3FBt/event_purity‚0 =&Purity/*FRt1SRqqt1FBt/event_proportion‚0.1001134067773819 @&Purity/0FRt*SRqqt3FBt/event_proportion‚0.0030871976632624865 9"Purity/0FRt1SRqqt1FBt/SRqqt_purity‚0.5103722214698792 ,&Purity/2FRt1SRqqt4FBt/event_proportion‚0 @&Purity/3FRt0SRqqt1FBt/event_proportion‚0.0011340725468471646 9"Purity/0FRt4SRqqt0FBt/SRqqt_purity‚0.2142857164144516 9"Purity/0FRt*SRqqt1FBt/event_purity‚0.6347508430480957 9"Purity/0FRt1SRqqt*FBt/event_purity‚0.2789357006549835 =&Purity/1FRt*SRqqt0FBt/event_proportion‚0.3135710656642914 7 Purity/1FRt1SRqqt0FBt/FRt_purity‚0.2933848798274994 :"Purity/2FRt4SRqqt0FBt/SRqqt_purity‚0.30000001192092896 7 Purity/*FRt1SRqqt3FBt/FBt_purity‚0.8730158805847168 ("Purity/*FRt3SRqqt2FBt/event_purity‚0 9"Purity/0FRt0SRqqt1FBt/event_purity‚0.9985072016716003 ;"Purity/1FRt1SRqqt*FBt/event_purity‚0.017811637371778488 8 Purity/*FRt2SRqqt2FBt/FRt_purity‚0.43589746952056885 ("Purity/3FRt3SRqqt0FBt/event_purity‚0 7 Purity/*FRt3SRqqt1FBt/FRt_purity‚0.2388889044523239 :"Purity/0FRt*SRqqt2FBt/event_purity‚0.24378076195716858 7 Purity/2FRt0SRqqt*FBt/FRt_purity‚0.2973017394542694 & Purity/3FRt0SRqqt*FBt/FBt_purity‚1 A&Purity/3FRt4SRqqt*FBt/event_proportion‚0.00012600806076079607 *jet/accuracy_1_of_6‚0.9736841917037964 & Purity/1FRt2SRqqt2FBt/FBt_purity‚1 jet/accuracy_4_of_7‚0 ?&Purity/*FRt*SRqqt3FBt/event_proportion‚0.004473286215215921 >&Purity/0FRt0SRqqt*FBt/event_proportion‚0.11857359111309052 ;"Purity/0FRt2SRqqt1FBt/event_purity‚0.008960573934018612 @&Purity/4FRt2SRqqt0FBt/event_proportion‚6.300403038039804e-05 ("Purity/*FRt*SRqqt4FBt/event_purity‚0 7 Purity/0FRt2SRqqt3FBt/FBt_purity‚0.6666666865348816 ,&Purity/0FRt1SRqqt4FBt/event_proportion‚0 ("Purity/0FRt4SRqqt1FBt/event_purity‚0 +"Purity/3FRt4SRqqt*FBt/SRqqt_purity‚0.25 ("Purity/3FRt4SRqqt0FBt/event_purity‚0 * Purity/1FRt*SRqqt4FBt/FBt_purity‚0.875 ,&Purity/4FRt3SRqqt1FBt/event_proportion‚0 particle/accuracy_1_of_6‚1 particle/accuracy_6_of_7‚1 9"Purity/1FRt4SRqqt0FBt/SRqqt_purity‚0.4166666567325592 & Purity/1FRt3SRqqt2FBt/FRt_purity‚0 :"Purity/*FRt1SRqqt1FBt/event_purity‚0.09183076024055481 ,&Purity/3FRt4SRqqt3FBt/event_proportion‚0 particle/accuracy_1_of_1‚1 ( Purity/1FRt*SRqqt4FBt/FRt_purity‚0.5 7"Purity/1FRt*SRqqt2FBt/SRqqt_purity‚0.55345618724823 :"Purity/2FRt1SRqqt0FBt/SRqqt_purity‚0.37399375438690186 9"Purity/3FRt*SRqqt*FBt/SRqqt_purity‚0.4082101583480835 ("Purity/3FRt1SRqqt1FBt/event_purity‚0 (particle/f_score‚0.40630847215652466 & Purity/4FRt0SRqqt*FBt/FRt_purity‚0 8 Purity/*FRt1SRqqt2FBt/FRt_purity‚0.36194783449172974 ("Purity/3FRt*SRqqt0FBt/event_purity‚0 ,&Purity/4FRt2SRqqt1FBt/event_proportion‚0 particle/accuracy_3_of_6‚1 ) Purity/2FRt3SRqqt1FBt/FRt_purity‚0.25 ("Purity/4FRt0SRqqt0FBt/event_purity‚0 ?&Purity/0FRt2SRqqt0FBt/event_proportion‚0.043283771723508835 =&Purity/1FRt1SRqqt0FBt/event_proportion‚0.1265120953321457 7 Purity/1FRt2SRqqt0FBt/FRt_purity‚0.2609463632106781 ("Purity/1FRt3SRqqt0FBt/event_purity‚0 9"Purity/3FRt2SRqqt*FBt/SRqqt_purity‚0.3836805820465088 ,&Purity/4FRt4SRqqt3FBt/event_proportion‚0 jet/accuracy_5_of_5‚0 & Purity/0FRt4SRqqt*FBt/FBt_purity‚1 7 Purity/2FRt*SRqqt1FBt/FBt_purity‚0.9946236610412598 :"Purity/2FRt0SRqqt1FBt/event_purity‚0.06703703850507736 8"Purity/0FRt1SRqqt3FBt/SRqqt_purity‚0.800000011920929 ("Purity/4FRt1SRqqt0FBt/SRqqt_purity‚1 7 Purity/*FRt2SRqqt3FBt/FBt_purity‚0.6666666865348816 ?&Purity/*FRt0SRqqt3FBt/event_proportion‚0.003717237850651145 >&Purity/0FRt0SRqqt0FBt/event_proportion‚0.04082661122083664 6 Purity/1FRt1SRqqt2FBt/FBt_purity‚0.968154788017273 ,&Purity/1FRt4SRqqt1FBt/event_proportion‚0 ("Purity/3FRt2SRqqt0FBt/event_purity‚0 ,&Purity/4FRt4SRqqt4FBt/event_proportion‚0 (particle/accuracy‚0.8446897864341736 -"Purity/1FRt2SRqqt2FBt/SRqqt_purity‚0.3125 8 Purity/*FRt0SRqqt1FBt/FRt_purity‚0.41688114404678345 7 Purity/2FRt1SRqqt2FBt/FRt_purity‚0.4385965168476105 ?&Purity/3FRt*SRqqt*FBt/event_proportion‚0.009954636916518211 @&Purity/3FRt*SRqqt2FBt/event_proportion‚0.0003150201519019902 ,&Purity/4FRt2SRqqt2FBt/event_proportion‚0 ("Purity/*FRt2SRqqt2FBt/event_purity‚0 ("Purity/*FRt*SRqqt4FBt/SRqqt_purity‚1 8 Purity/*FRt*SRqqt1FBt/FRt_purity‚0.36288997530937195 :"Purity/*FRt*SRqqt0FBt/event_purity‚0.14373420178890228 9"Purity/*FRt1SRqqt2FBt/SRqqt_purity‚0.5854530334472656 @&Purity/*FRt1SRqqt4FBt/event_proportion‚6.300403038039804e-05 A&Purity/1FRt*SRqqt4FBt/event_proportion‚0.00012600806076079607 @&Purity/1FRt0SRqqt4FBt/event_proportion‚6.300403038039804e-05 8"Purity/0FRt*SRqqt3FBt/SRqqt_purity‚0.800000011920929 ( Purity/2FRt2SRqqt2FBt/FRt_purity‚0.5 9"Purity/1FRt*SRqqt*FBt/SRqqt_purity‚0.3892556130886078 7 Purity/1FRt2SRqqt*FBt/FRt_purity‚0.2749495804309845 7 Purity/2FRt0SRqqt2FBt/FBt_purity‚0.9940475821495056 & Purity/3FRt1SRqqt1FBt/FBt_purity‚1 ("Purity/0FRt4SRqqt*FBt/event_purity‚0 8"Purity/*FRt0SRqqt*FBt/event_purity‚0.400392085313797 ?&Purity/1FRt2SRqqt1FBt/event_proportion‚0.008946572430431843 ,&Purity/4FRt0SRqqt4FBt/event_proportion‚0 jet/accuracy_5_of_6‚0 ,&Purity/3FRt3SRqqt4FBt/event_proportion‚0 @&Purity/*FRt0SRqqt4FBt/event_proportion‚0.0003150201519019902 ("Purity/*FRt3SRqqt1FBt/event_purity‚0 ,&Purity/*FRt3SRqqt3FBt/event_proportion‚0 @&Purity/*FRt4SRqqt*FBt/event_proportion‚0.0013230846961960196 @&Purity/0FRt2SRqqt2FBt/event_proportion‚0.0010080644860863686 :"Purity/1FRt*SRqqt0FBt/event_purity‚0.09659674018621445 7 Purity/1FRt*SRqqt1FBt/FBt_purity‚0.9956594705581665 7 Purity/*FRt2SRqqt*FBt/FBt_purity‚0.9885477423667908 8 Purity/1FRt0SRqqt1FBt/FRt_purity‚0.42507821321487427 9"Purity/1FRt3SRqqt*FBt/SRqqt_purity‚0.3688470721244812 ,&Purity/2FRt2SRqqt3FBt/event_proportion‚0 ("Purity/3FRt*SRqqt1FBt/event_purity‚0 *jet/accuracy_1_of_1‚0.4200058877468109 particle/accuracy_3_of_3‚1 particle/accuracy_1_of_5‚1 ?&Purity/2FRt2SRqqt1FBt/event_proportion‚0.003969253972172737 ,&Purity/3FRt*SRqqt3FBt/event_proportion‚0 ,&Purity/4FRt3SRqqt4FBt/event_proportion‚0 jet/accuracy_6_of_7‚0 7 Purity/3FRt0SRqqt2FBt/FRt_purity‚0.3333333432674408 ;"Purity/*FRt3SRqqt0FBt/event_purity‚0.004608294926583767 ("Purity/3FRt0SRqqt0FBt/event_purity‚0 ,jet/accuracy_4_of_5‚0.013685239478945732 & Purity/3FRt2SRqqt*FBt/FBt_purity‚1 ("Purity/3FRt2SRqqt1FBt/event_purity‚0 9"Purity/0FRt3SRqqt1FBt/SRqqt_purity‚0.4583333432674408 ("Purity/3FRt0SRqqt2FBt/event_purity‚0 ("Purity/*FRt1SRqqt4FBt/event_purity‚0 ,&Purity/*FRt2SRqqt4FBt/event_proportion‚0 ("Purity/2FRt1SRqqt2FBt/event_purity‚0 _step‚97 A&Purity/0FRt4SRqqt0FBt/event_proportion‚0.00044102821266278625 ,&Purity/0FRt4SRqqt4FBt/event_proportion‚0 9"Purity/1FRt1SRqqt2FBt/SRqqt_purity‚0.5925169587135315 & Purity/1FRt3SRqqt*FBt/FBt_purity‚1 8 Purity/2FRt0SRqqt0FBt/FRt_purity‚0.24054308235645294 :"Purity/0FRt3SRqqt0FBt/event_purity‚0.01785714365541935 >&Purity/1FRt0SRqqt*FBt/event_proportion‚0.19745463132858276 =&Purity/*FRt0SRqqt2FBt/event_proportion‚0.0483240932226181 =&Purity/*FRt1SRqqt*FBt/event_proportion‚0.4482736885547638 :"Purity/*FRt1SRqqt0FBt/event_purity‚0.17396295070648193 ,&Purity/*FRt4SRqqt3FBt/event_proportion‚0 ?&Purity/1FRt1SRqqt2FBt/event_proportion‚0.006111391354352236 :"Purity/1FRt3SRqqt1FBt/SRqqt_purity‚0.30000004172325134 ;"Purity/2FRt*SRqqt*FBt/event_purity‚0.013833863660693169 9"Purity/*FRt0SRqqt0FBt/event_purity‚0.1856137067079544 @&Purity/*FRt3SRqqt2FBt/event_proportion‚6.300403038039804e-05 8 Purity/1FRt1SRqqt*FBt/FRt_purity‚0.29857590794563293 9"Purity/2FRt*SRqqt2FBt/event_purity‚0.0277777798473835 ?&Purity/2FRt2SRqqt0FBt/event_proportion‚0.015435988083481789 ,&Purity/4FRt1SRqqt1FBt/event_proportion‚0 ( Purity/2FRt*SRqqt3FBt/FRt_purity‚0.5 ("Purity/1FRt1SRqqt3FBt/event_purity‚0 7 Purity/*FRt1SRqqt*FBt/FBt_purity‚0.9838809967041016 =&Purity/*FRt2SRqqt*FBt/event_proportion‚0.1332535296678543 @&Purity/*FRt4SRqqt1FBt/event_proportion‚6.300403038039804e-05 :"Purity/0FRt2SRqqt0FBt/event_purity‚0.04079851135611534 ,&Purity/0FRt4SRqqt3FBt/event_proportion‚0 9"Purity/3FRt1SRqqt*FBt/SRqqt_purity‚0.4660493731498718 ,&Purity/3FRt2SRqqt2FBt/event_proportion‚0 ,&Purity/4FRt1SRqqt4FBt/event_proportion‚0 ("Purity/3FRt0SRqqt*FBt/event_purity‚0 A&Purity/0FRt1SRqqt3FBt/event_proportion‚0.00044102821266278625 9"Purity/2FRt0SRqqt2FBt/event_purity‚0.0535714291036129 particle/accuracy_2_of_2‚1 ("Purity/0FRt2SRqqt2FBt/event_purity‚0 ("Purity/3FRt*SRqqt2FBt/event_purity‚0 ("Purity/*FRt1SRqqt4FBt/SRqqt_purity‚1 ("Purity/1FRt3SRqqt2FBt/event_purity‚0 :"Purity/0FRt*SRqqt3FBt/event_purity‚0.15432098507881165 9"Purity/0FRt1SRqqt0FBt/SRqqt_purity‚0.3298182785511017 ,&Purity/2FRt4SRqqt3FBt/event_proportion‚0 7 Purity/3FRt*SRqqt1FBt/FRt_purity‚0.3090909421443939 ,&Purity/4FRt3SRqqt3FBt/event_proportion‚0 ("Purity/1FRt1SRqqt3FBt/SRqqt_purity‚1 & Purity/4FRt0SRqqt0FBt/FRt_purity‚0 6 Purity/0FRt2SRqqt*FBt/FBt_purity‚0.984639048576355 particle/accuracy_2_of_6‚1 A&Purity/0FRt0SRqqt4FBt/event_proportion‚0.00025201612152159214 ;"Purity/2FRt*SRqqt1FBt/event_purity‚0.031320054084062576 9"Purity/2FRt2SRqqt1FBt/SRqqt_purity‚0.4690476059913635 *jet/accuracy_1_of_2‚0.7170048952102661 ("Purity/4FRt*SRqqt*FBt/event_purity‚0 ;"Purity/*FRt3SRqqt*FBt/event_purity‚0.004608294926583767 9"Purity/2FRt2SRqqt*FBt/SRqqt_purity‚0.3841923773288727 *"Purity/3FRt1SRqqt2FBt/SRqqt_purity‚0.5 ("Purity/1FRt*SRqqt4FBt/SRqqt_purity‚1 & Purity/4FRt1SRqqt0FBt/FRt_purity‚0 :"Purity/0FRt2SRqqt*FBt/event_purity‚0.03501037880778313 7 Purity/1FRt*SRqqt1FBt/FRt_purity‚0.3685687184333801 ,particle/sensitivity‚0.34033629298210144 particle/accuracy_6_of_6‚1 & Purity/1FRt3SRqqt2FBt/FBt_purity‚1 9"Purity/*FRt0SRqqt3FBt/e»|£Áèvent_purity‚0.1130952388048172 7 Purity/1FRt0SRqqt1FBt/FBt_purity‚0.9966126084327698 7 Purity/0FRt*SRqqt1FBt/FBt_purity‚0.9953582882881165 7 Purity/1FRt1SRqqt2FBt/FRt_purity‚0.3760204315185547 8 Purity/3FRt1SRqqt1FBt/FRt_purity‚0.26923075318336487 ,jet/accuracy_4_of_4‚0.005122719798237085 particle/accuracy_3_of_5‚1 +jet/accuracy_2_of_6‚0.49122804403305054 7 Purity/0FRt2SRqqt2FBt/FBt_purity‚0.9807692170143127 9"Purity/*FRt*SRqqt2FBt/event_purity‚0.1741527020931244 7 Purity/*FRt2SRqqt0FBt/FRt_purity‚0.2678324282169342 ?&Purity/2FRt0SRqqt1FBt/event_proportion‚0.008946572430431843 7 Purity/*FRt0SRqqt1FBt/FBt_purity‚0.9979555010795593 @&Purity/*FRt1SRqqt3FBt/event_proportion‚0.0006930443341843784 ;"Purity/2FRt0SRqqt0FBt/event_purity‚0.024096736684441566 9"Purity/2FRt1SRqqt*FBt/SRqqt_purity‚0.4205760955810547 ("Purity/1FRt4SRqqt*FBt/event_purity‚0 7 Purity/*FRt*SRqqt1FBt/FBt_purity‚0.9956392049789429 7 Purity/*FRt1SRqqt2FBt/FBt_purity‚0.9632813334465027 @&Purity/2FRt3SRqqt1FBt/event_proportion‚0.0003780241822823882 ?&Purity/3FRt1SRqqt*FBt/event_proportion‚0.003717237850651145 7 Purity/3FRt2SRqqt0FBt/FRt_purity‚0.2777777910232544 * Purity/0FRt0SRqqt4FBt/FBt_purity‚0.875 ("Purity/1FRt2SRqqt1FBt/event_purity‚0 & Purity/3FRt*SRqqt1FBt/FBt_purity‚1 8"Purity/3FRt3SRqqt*FBt/SRqqt_purity‚0.380952388048172 :"Purity/0FRt*SRqqt*FBt/SRqqt_purity‚0.37402716279029846 9"Purity/1FRt2SRqqt*FBt/SRqqt_purity‚0.3777875006198883Yɤ zÆ ¤¨ ,&Purity/0FRt3SRqqt2FBt/event_proportion‚0 8 Purity/2FRt*SRqqt2FBt/FRt_purity‚0.47583335638046265 ,&Purity/3FRt3SRqqt1FBt/event_proportion‚0 7 Purity/*FRt3SRqqt*FBt/FRt_purity‚0.2614314556121826 7 Purity/0FRt*SRqqt2FBt/FBt_purity‚0.9741479754447937 7 Purity/0FRt2SRqqt1FBt/FBt_purity‚0.9953917264938354 :"Purity/2FRt*SRqqt0FBt/event_purity‚0.00949862226843834 7 Purity/2FRt0SRqqt1FBt/FRt_purity‚0.3909126818180084 :"Purity/3FRt1SRqqt0FBt/SRqqt_purity‚0.40833330154418945 & Purity/4FRt*SRqqt*FBt/FRt_purity‚0 7 Purity/*FRt3SRqqt0FBt/FRt_purity‚0.2844630479812622 7 Purity/0FRt0SRqqt2FBt/FBt_purity‚0.9796778559684753 & Purity/2FRt1SRqqt1FBt/FBt_purity‚1 ) Purity/*FRt1SRqqt4FBt/FBt_purity‚0.75 & Purity/4FRt*SRqqt0FBt/FRt_purity‚0 :"Purity/0FRt*SRqqt0FBt/event_purity‚0.26281315088272095 7 Purity/2FRt2SRqqt1FBt/FBt_purity‚0.9821428656578064 ,&Purity/3FRt*SRqqt4FBt/event_proportion‚0 particle/accuracy_5_of_6‚1 ("Purity/3FRt4SRqqt*FBt/event_purity‚0 A&Purity/0FRt*SRqqt4FBt/event_proportion‚0.00025201612152159214 8 Purity/1FRt1SRqqt1FBt/FRt_purity‚0.30853188037872314 ("Purity/3FRt*SRqqt*FBt/event_purity‚0 ) Purity/3FRt1SRqqt2FBt/FRt_purity‚0.25 & Purity/*FRt1SRqqt4FBt/FRt_purity‚0 @&Purity/3FRt0SRqqt2FBt/event_proportion‚6.300403038039804e-05 ,&Purity/4FRt1SRqqt2FBt/event_proportion‚0 ,&Purity/4FRt2SRqqt3FBt/event_proportion‚0 ,&Purity/4FRt3SRqqt*FBt/event_proportion‚0 ("Purity/0FRt0SRqqt4FBt/event_purity‚0 ("Purity/3FRt2SRqqt*FBt/event_purity‚0 :"Purity/*FRt3SRqqt1FBt/SRqqt_purity‚0.33862432837486267 7 Purity/1FRt*SRqqt2FBt/FRt_purity‚0.5199726223945618 :"Purity/1FRt0SRqqt2FBt/event_purity‚0.12273846566677094 9"Purity/3FRt*SRqqt1FBt/SRqqt_purity‚0.6611111164093018 ("Purity/*FRt4SRqqt0FBt/event_purity‚0 *"Purity/3FRt2SRqqt1FBt/SRqqt_purity‚0.5 @&Purity/*FRt*SRqqt4FBt/event_proportion‚0.0003780241822823882 9"Purity/0FRt1SRqqt2FBt/SRqqt_purity‚0.5231084227561951 >&Purity/2FRt*SRqqt0FBt/event_proportion‚0.07579384744167328 *jet/accuracy_1_of_4‚0.9090997576713562 particle/accuracy_4_of_7‚1 >&Purity/1FRt1SRqqt1FBt/event_proportion‚0.03944052383303642 & Purity/3FRt0SRqqt1FBt/FBt_purity‚1 7 Purity/*FRt4SRqqt0FBt/FRt_purity‚0.3611111342906952 & Purity/4FRt2SRqqt0FBt/FRt_purity‚0 7 Purity/0FRt1SRqqt2FBt/FBt_purity‚0.9537036418914795 :"Purity/1FRt0SRqqt0FBt/event_purity‚0.21827785670757294 8 Purity/3FRt*SRqqt*FBt/FRt_purity‚0.28674259781837463 ("Purity/0FRt3SRqqt1FBt/event_purity‚0 7 Purity/*FRt*SRqqt*FBt/FRt_purity‚0.2930887043476105 @&Purity/3FRt1SRqqt0FBt/event_proportion‚0.0025201612152159214 ( Purity/*FRt1SRqqt3FBt/FRt_purity‚0.5 ("Purity/4FRt*SRqqt0FBt/SRqqt_purity‚1 7 Purity/*FRt0SRqqt*FBt/FRt_purity‚0.2972099483013153 ,&Purity/2FRt4SRqqt1FBt/event_proportion‚0 8 Purity/2FRt3SRqqt0FBt/FRt_purity‚0.33338746428489685 ,&Purity/2FRt3SRqqt4FBt/event_proportion‚0 ("Purity/1FRt3SRqqt1FBt/event_purity‚0 ("Purity/2FRt3SRqqt0FBt/event_purity‚0 ("Purity/4FRt2SRqqt0FBt/SRqqt_purity‚1 =&Purity/*FRt*SRqqt0FBt/event_proportion‚0.6160534024238586 >&Purity/*FRt0SRqqt1FBt/event_proportion‚0.14591734111309052 9"Purity/*FRt2SRqqt0FBt/SRqqt_purity‚0.3623194396495819 >&Purity/*FRt2SRqqt1FBt/event_proportion‚0.02255544438958168 >&Purity/0FRt*SRqqt1FBt/event_proportion‚0.14787046611309052 7 Purity/1FRt2SRqqt1FBt/FRt_purity‚0.3320276439189911 ,&Purity/1FRt2SRqqt3FBt/event_proportion‚0 & Purity/*FRt3SRqqt2FBt/FBt_purity‚1 7 Purity/1FRt*SRqqt3FBt/FBt_purity‚0.9270833134651184 8 Purity/1FRt0SRqqt*FBt/FRt_purity‚0.29792314767837524 9"Purity/1FRt1SRqqt*FBt/SRqqt_purity‚0.4011145234107971 +jet/accuracy_2_of_5‚0.39338332414627075 particle/accuracy_4_of_4‚1 jet/accuracy_2_of_7‚0.5 :"Purity/0FRt1SRqqt*FBt/SRqqt_purity‚0.37868180871009827 =&Purity/0FRt1SRqqt0FBt/event_proportion‚0.1694808453321457 ?&Purity/2FRt1SRqqt1FBt/event_proportion‚0.007875503972172737 & Purity/0FRt4SRqqt1FBt/FBt_purity‚1 @&Purity/1FRt4SRqqt0FBt/event_proportion‚0.0003780241822823882 7 Purity/2FRt*SRqqt*FBt/FBt_purity‚0.9907397627830505 ,&Purity/3FRt4SRqqt2FBt/event_proportion‚0 ,&Purity/4FRt4SRqqt1FBt/event_proportion‚0 ("Purity/2FRt2SRqqt2FBt/event_purity‚0 >&Purity/0FRt*SRqqt2FBt/event_proportion‚0.03944052383303642 7 Purity/*FRt3SRqqt*FBt/FBt_purity‚0.9545454382896423 9"Purity/0FRt*SRqqt2FBt/SRqqt_purity‚0.5198147892951965 9"Purity/0FRt0SRqqt*FBt/event_purity‚0.8052219152450562 9"Purity/2FRt*SRqqt1FBt/SRqqt_purity‚0.5078203678131104 ("Purity/2FRt2SRqqt1FBt/event_purity‚0 8 Purity/3FRt0SRqqt1FBt/FRt_purity‚0.42307692766189575 ,&Purity/3FRt1SRqqt3FBt/event_proportion‚0 jet/accuracy_5_of_7‚0 :"Purity/*FRt*SRqqt*FBt/event_purity‚0.22082605957984924 @&Purity/3FRt*SRqqt1FBt/event_proportion‚0.0024571572430431843 ,&Purity/3FRt4SRqqt4FBt/event_proportion‚0 +jet/accuracy_2_of_3‚0.22940385341644287 +particle/specificity‚0.9380465745925903 ("Purity/0FRt4SRqqt0FBt/event_purity‚0 ("Purity/1FRt2SRqqt0FBt/event_purity‚0 ,&Purity/1FRt2SRqqt4FBt/event_proportion‚0 @&Purity/2FRt*SRqqt3FBt/event_proportion‚6.300403038039804e-05 @&Purity/3FRt2SRqqt1FBt/event_proportion‚0.0003780241822823882 9"Purity/*FRt1SRqqt3FBt/event_purity‚0.0714285746216774 9"Purity/*FRt3SRqqt2FBt/SRqqt_purity‚0.6666666865348816 7 Purity/1FRt0SRqqt2FBt/FRt_purity‚0.5651761293411255 ;"Purity/1FRt1SRqqt0FBt/event_purity‚0.012274806387722492 ?&Purity/2FRt0SRqqt2FBt/event_proportion‚0.003465221729129553 ,&Purity/3FRt3SRqqt2FBt/event_proportion‚0 7 Purity/1FRt1SRqqt1FBt/FBt_purity‚0.9949740171432495 +jet/accuracy_2_of_2‚0.10095174610614777 7 Purity/3FRt3SRqqt0FBt/FRt_purity‚0.3333333432674408 7 Purity/2FRt*SRqqt2FBt/FBt_purity‚0.9911110997200012 *jet/accuracy_1_of_3‚0.8526424169540405 trainer/global_step‚1207 9"Purity/*FRt0SRqqt1FBt/event_purity‚0.6989607810974121 7 Purity/*FRt0SRqqt2FBt/FRt_purity‚0.5266392230987549 8 Purity/2FRt1SRqqt1FBt/FRt_purity‚0.33045312762260437 & Purity/*FRt4SRqqt*FBt/FBt_purity‚1 9"Purity/0FRt3SRqqt*FBt/SRqqt_purity‚0.3203420639038086 :"Purity/2FRt4SRqqt*FBt/SRqqt_purity‚0.30000001192092896 ("Purity/2FRt4SRqqt0FBt/event_purity‚0 ("Purity/4FRt2SRqqt*FBt/event_purity‚0 7 Purity/*FRt*SRqqt2FBt/FRt_purity‚0.4761085510253906 @&Purity/*FRt2SRqqt3FBt/event_proportion‚6.300403038039804e-05 ,&Purity/0FRt2SRqqt4FBt/event_proportion‚0 7 Purity/2FRt2SRqqt1FBt/FRt_purity‚0.2690476179122925 -"Purity/*FRt4SRqqt*FBt/SRqqt_purity‚0.3125 ( Purity/2FRt4SRqqt*FBt/FRt_purity‚0.5 8"Purity/3FRt3SRqqt0FBt/SRqqt_purity‚0.380952388048172 >&Purity/0FRt*SRqqt*FBt/event_proportion‚0.40996724367141724 9"Purity/1FRt2SRqqt1FBt/SRqqt_purity‚0.4277649521827698 :"Purity/2FRt3SRqqt*FBt/SRqqt_purity‚0.33590471744537354 @&Purity/3FRt1SRqqt1FBt/event_proportion‚0.0009450604557059705 particle/accuracy_2_of_7‚1 7 Purity/3FRt*SRqqt2FBt/FRt_purity‚0.2666666805744171 + Purity/0FRt3SRqqt*FBt/FBt_purity‚0.8125 ;"Purity/2FRt1SRqqt0FBt/event_purity‚0.002481389557942748 ("Purity/1FRt2SRqqt2FBt/event_purity‚0 ?&Purity/0FRt2SRqqt1FBt/event_proportion‚0.009261592291295528 9"Purity/1FRt1SRqqt1FBt/SRqqt_purity‚0.5551432371139526 ?&Purity/1FRt3SRqqt*FBt/event_proportion‚0.009135584346950054 @&Purity/4FRt1SRqqt*FBt/event_proportion‚6.300403038039804e-05 *jet/accuracy_2_of_4‚0.3503338098526001 & Purity/2FRt3SRqqt*FBt/FBt_purity‚1 +"Purity/3FRt4SRqqt0FBt/SRqqt_purity‚0.25 ;"Purity/*FRt2SRqqt*FBt/event_purity‚0.014280671253800392 7 Purity/0FRt0SRqqt*FBt/FBt_purity‚0.9847189784049988 7 Purity/1FRt*SRqqt*FBt/FRt_purity‚0.2943839430809021 7 Purity/1FRt0SRqqt3FBt/FBt_purity‚0.9285714030265808 ("Purity/0FRt*SRqqt4FBt/event_purity‚0 +jet/accuracy_4_of_6‚0.02631578966975212 ("Purity/1FRt4SRqqt0FBt/event_purity‚0 7 Purity/1FRt4SRqqt0FBt/FRt_purity‚0.1666666716337204 7 Purity/0FRt*SRqqt3FBt/FBt_purity‚0.8724279999732971 7 Purity/0FRt1SRqqt*FBt/FBt_purity‚0.9818498492240906 9"Purity/*FRt1SRqqt*FBt/SRqqt_purity‚0.3919236660003662 :"Purity/1FRt*SRqqt2FBt/event_purity‚0.08720673620700836 7 Purity/1FRt2SRqqt1FBt/FBt_purity‚0.9967741966247559 8 Purity/3FRt1SRqqt*FBt/FRt_purity‚0.25617286562919617 8"Purity/3FRt2SRqqt0FBt/SRqqt_purity‚0.364130437374115 ("Purity/*FRt4SRqqt1FBt/event_purity‚0 ( Purity/2FRt0SRqqt3FBt/FRt_purity‚0.5 9"Purity/*FRt1SRqqt3FBt/SRqqt_purity‚0.9047619104385376 7 Purity/*FRt0SRqqt3FBt/FBt_purity‚0.9013890027999878 7 Purity/2FRt*SRqqt0FBt/FRt_purity‚0.2658703923225403 9"Purity/3FRt*SRqqt0FBt/SRqqt_purity‚0.3478032946586609 @&Purity/3FRt2SRqqt*FBt/event_proportion‚0.0028351815417408943 7 Purity/*FRt*SRqqt3FBt/FBt_purity‚0.8932951092720032 >&Purity/1FRt2SRqqt*FBt/event_proportion‚0.05727066472172737 9"Purity/1FRt3SRqqt0FBt/SRqqt_purity‚0.3606502413749695 @&Purity/3FRt3SRqqt0FBt/event_proportion‚0.0005670362734235823 ( Purity/3FRt4SRqqt*FBt/FRt_purity‚0.5 ("Purity/4FRt1SRqqt0FBt/event_purity‚0 9"Purity/0FRt*SRqqt*FBt/event_purity‚0.3940039575099945 ("Purity/1FRt2SRqqt*FBt/event_purity‚0 ,&Purity/4FRt3SRqqt0FBt/event_proportion‚0 ( Purity/3FRt4SRqqt0FBt/FRt_purity‚0.5 & Purity/*FRt3SRqqt2FBt/FRt_purity‚0 ( Purity/1FRt1SRqqt3FBt/FRt_purity‚0.5 epoch‚1 ?&Purity/*FRt1SRqqt2FBt/event_proportion‚0.017011089250445366 :"Purity/*FRt2SRqqt1FBt/SRqqt_purity‚0.46048206090927124 ("Purity/1FRt*SRqqt3FBt/event_purity‚0 8 Purity/2FRt*SRqqt*FBt/FRt_purity‚0.29335853457450867 7 Purity/3FRt2SRqqt1FBt/FRt_purity‚0.2222222238779068 ("Purity/3FRt3SRqqt*FBt/event_purity‚0 >&Purity/0FRt0SRqqt1FBt/event_proportion‚0.08606351166963577 7 Purity/3FRt1SRqqt0FBt/FRt_purity‚0.2361111342906952 9"Purity/*FRt*SRqqt1FBt/SRqqt_purity‚0.5007259249687195 :"Purity/*FRt*SRqqt0FBt/SRqqt_purity‚0.34769973158836365 7 Purity/*FRt2SRqqt1FBt/FRt_purity‚0.3186286389827728 6 Purity/0FRt0SRqqt3FBt/FBt_purity‚0.893333375453949 =&Purity/0FRt1SRqqt*FBt/event_proportion‚0.2305317521095276 7 Purity/1FRt0SRqqt*FBt/FBt_purity‚0.9882248044013977 ,&Purity/4FRt1SRqqt3FBt/event_proportion‚0 ,&Purity/4FRt2SRqqt4FBt/event_proportion‚0 7 Purity/0FRt*SRqqt*FBt/FBt_purity‚0.9832651615142822 ,&Purity/0FRt4SRqqt2FBt/event_proportion‚0 8 Purity/1FRt*SRqqt0FBt/FRt_purity‚0.25606444478034973 ,&Purity/4FRt4SRqqt2FBt/event_proportion‚0 ("Purity/2FRt3SRqqt1FBt/event_purity‚0 7 Purity/1FRt4SRqqt*FBt/FRt_purity‚0.1666666716337204 7 Purity/0FRt1SRqqt3FBt/FBt_purity‚0.8333333730697632 :"Purity/*FRt3SRqqt0FBt/SRqqt_purity‚0.35146406292915344 ?&Purity/2FRt3SRqqt0FBt/event_proportion‚0.003654233878478408 7 Purity/*FRt0SRqqt*FBt/FBt_purity‚0.9869210720062256 9"Purity/1FRt*SRqqt0FBt/SRqqt_purity‚0.3531774580478668 9"Purity/1FRt0SRqqt*FBt/event_purity‚0.2315816879272461 ("Purity/2FRt1SRqqt1FBt/event_purity‚0 +jet/accuracy_3_of_4‚0.06716153025627136 :"Purity/*FRt2SRqqt2FBt/SRqqt_purity‚0.37280702590942383 ("Purity/4FRt1SRqqt*FBt/SRqqt_purity‚1 ("Purity/1FRt0SRqqt3FBt/event_purity‚0 7 Purity/*FRt*SRqqt*FBt/FBt_purity‚0.9858607053756714 ;"Purity/*FRt1SRqqt2FBt/event_purity‚0.044637616723775864 ?&Purity/0FRt0SRqqt3FBt/event_proportion‚0.002583165420219302 A&Purity/1FRt1SRqqt3FBt/event_proportion‚0.00025201612152159214 ,&Purity/2FRt1SRqqt3FBt/event_proportion‚0 jet/accuracy_6_of_6‚0 8"Purity/*FRt0SRqqt2FBt/event_purity‚0.226539745926857 9"Purity/*FRt1SRqqt0FBt/SRqqt_purity‚0.3377826511859894 :"Purity/*FRt3SRqqt*FBt/SRqqt_purity‚0.35035985708236694 >&Purity/1FRt0SRqqt0FBt/event_proportion‚0.13142640888690948 ,&Purity/2FRt0SRqqt4FBt/event_proportion‚0 >&Purity/2FRt1SRqqt*FBt/event_proportion‚0.04158266261219978 ("Purity/2FRt2SRqqt*FBt/event_purity‚0 & Purity/3FRt*SRqqt*FBt/FBt_purity‚1 :"Purity/1FRt0SRqqt1FBt/event_purity‚0.30789756774902344 & Purity/2FRt0SRqqt1FBt/FBt_purity‚1 @&Purity/3FRt0SRqqt*FBt/event_proportion‚0.0027091733645647764 ( Purity/2FRt4SRqqt0FBt/FRt_purity‚0.5 & Purity/3FRt1SRqqt2FBt/FBt_purity‚1 ) Purity/1FRt1SRqqt4FBt/FBt_purity‚0.75 ("Purity/4FRt*SRqqt*FBt/SRqqt_purity‚1 @&Purity/1FRt0SRqqt3FBt/event_proportion‚0.0010710685746744275 >&Purity/*FRt*SRqqt1FBt/event_proportion‚0.27079132199287415 7 Purity/*FRt1SRqqt1FBt/FRt_purity‚0.3159247934818268 ?&Purity/*FRt3SRqqt*FBt/event_proportion‚0.020476309582591057 >&Purity/1FRt2SRqqt0FBt/event_proportion‚0.04731602966785431 >&Purity/2FRt*SRqqt*FBt/event_proportion‚0.10244455933570862 & Purity/1FRt1SRqqt4FBt/FRt_purity‚0 9"Purity/0FRt2SRqqt*FBt/SRqqt_purity‚0.3772554099559784 ,&Purity/4FRt0SRqqt2FBt/event_proportion‚0 8"Purity/0FRt3SRqqt0FBt/SRqqt_purity‚0.313492089509964 9"Purity/0FRt2SRqqt2FBt/SRqqt_purity‚0.4423076808452606 *"Purity/3FRt*SRqqt2FBt/SRqqt_purity‚0.5 7 Purity/3FRt3SRqqt*FBt/FRt_purity‚0.3333333432674408 9"Purity/1FRt3SRqqt2FBt/SRqqt_purity‚0.6666666865348816 9"Purity/0FRt1SRqqt0FBt/event_purity‚0.3298182785511017 @&Purity/0FRt4SRqqt*FBt/event_proportion‚0.0005040322430431843 9"Purity/2FRt*SRqqt2FBt/SRqqt_purity‚0.6710526347160339 _runtime‚2182.231633732 :"Purity/1FRt*SRqqt1FBt/event_purity‚0.16590532660484314 8"Purity/0FRt2SRqqt0FBt/SRqqt_purity‚0.360020250082016 9"Purity/2FRt1SRqqt2FBt/SRqqt_purity‚0.7061403393745422 @&Purity/4FRt0SRqqt0FBt/event_proportion‚6.300403038039804e-05 ("Purity/1FRt*SRqqt3FBt/SRqqt_purity‚1 ?&Purity/1FRt0SRqqt2FBt/event_proportion‚0.015120967291295528 >&Purity/0FRt1SRqqt1FBt/event_proportion‚0.05185231938958168 6 Purity/1FRt*SRqqt2FBt/FBt_purity‚0.977053165435791 9"Purity/1FRt1SRqqt2FBt/event_purity‚0.0357142873108387 8 Purity/1FRt3SRqqt1FBt/FRt_purity‚0.24444444477558136 <"Purity/2FRt1SRqqt*FBt/event_purity‚0.0015360983088612556 9"Purity/1FRt4SRqqt*FBt/SRqqt_purity‚0.4166666567325592 + Purity/1FRt*SRqqt3FBt/FRt_purity‚0.4375 :"Purity/1FRt2SRqqt0FBt/SRqqt_purity‚0.36267656087875366 @&Purity/1FRt2SRqqt2FBt/event_proportion‚0.0010080644860863686 7 Purity/2FRt1SRqqt*FBt/FBt_purity‚0.9964157938957214 :"Purity/1FRt1SRqqt0FBt/SRqqt_purity‚0.34054625034332275 7 Purity/2FRt2SRqqt*FBt/FBt_purity‚0.9821428656578064 particle/accuracy_3_of_7‚1 :"Purity/0FRt1SRqqt3FBt/event_purity‚0.10000000149011612 ("Purity/2FRt0SRqqt3FBt/event_purity‚0 =&Purity/*FRt1SRqqt1FBt/event_proportion‚0.1001134067773819 @&Purity/0FRt*SRqqt3FBt/event_proportion‚0.0030871976632624865 9"Purity/0FRt1SRqqt1FBt/SRqqt_purity‚0.5103722214698792 ,&Purity/2FRt1SRqqt4FBt/event_proportion‚0 @&Purity/3FRt0SRqqt1FBt/event_proportion‚0.0011340725468471646 9"Purity/0FRt4SRqqt0FBt/SRqqt_purity‚0.2142857164144516 9"Purity/0FRt*SRqqt1FBt/event_purity‚0.6347508430480957 9"Purity/0FRt1SRqqt*FBt/event_purity‚0.2789357006549835 =&Purity/1FRt*SRqqt0FBt/event_proportion‚0.3135710656642914 7 Purity/1FRt1SRqqt0FBt/FRt_purity‚0.2933848798274994 :"Purity/2FRt4SRqqt0FBt/SRqqt_purity‚0.30000001192092896 7 Purity/*FRt1SRqqt3FBt/FBt_purity‚0.8730158805847168 ("Purity/*FRt3SRqqt2FBt/event_purity‚0 9"Purity/0FRt0SRqqt1FBt/event_purity‚0.9985072016716003 ;"Purity/1FRt1SRqqt*FBt/event_purity‚0.017811637371778488 8 Purity/*FRt2SRqqt2FBt/FRt_purity‚0.43589746952056885 ("Purity/3FRt3SRqqt0FBt/event_purity‚0 7 Purity/*FRt3SRqqt1FBt/FRt_purity‚0.2388889044523239 :"Purity/0FRt*SRqqt2FBt/event_purity‚0.24378076195716858 7 Purity/2FRt0SRqqt*FBt/FRt_purity‚0.2973017394542694 & Purity/3FRt0SRqqt*FBt/FBt_purity‚1 A&Purity/3FRt4SRqqt*FBt/event_proportion‚0.00012600806076079607 *jet/accuracy_1_of_6‚0.9736841917037964 & Purity/1FRt2SRqqt2FBt/FBt_purity‚1 jet/accuracy_4_of_7‚0 ?&Purity/*FRt*SRqqt3FBt/event_proportion‚0.004473286215215921 >&Purity/0FRt0SRqqt*FBt/event_proportion‚0.11857359111309052 ;"Purity/0FRt2SRqqt1FBt/event_purity‚0.008960573934018612 @&Purity/4FRt2SRqqt0FBt/event_proportion‚6.300403038039804e-05 ("Purity/*FRt*SRqqt4FBt/event_purity‚0 7 Purity/0FRt2SRqqt3FBt/FBt_purity‚0.6666666865348816 ,&Purity/0FRt1SRqqt4FBt/event_proportion‚0 ("Purity/0FRt4SRqqt1FBt/event_purity‚0 +"Purity/3FRt4SRqqt*FBt/SRqqt_purity‚0.25 ("Purity/3FRt4SRqqt0FBt/event_purity‚0 * Purity/1FRt*SRqqt4FBt/FBt_purity‚0.875 ,&Purity/4FRt3SRqqt1FBt/event_proportion‚0 particle/accuracy_1_of_6‚1 particle/accuracy_6_of_7‚1 9"Purity/1FRt4SRqqt0FBt/SRqqt_purity‚0.4166666567325592 & Purity/1FRt3SRqqt2FBt/FRt_purity‚0 :"Purity/*FRt1SRqqt1FBt/event_purity‚0.09183076024055481 ,&Purity/3FRt4SRqqt3FBt/event_proportion‚0 particle/accuracy_1_of_1‚1 ( Purity/1FRt*SRqqt4FBt/FRt_purity‚0.5 7"Purity/1FRt*SRqqt2FBt/SRqqt_purity‚0.55345618724823 :"Purity/2FRt1SRqqt0FBt/SRqqt_purity‚0.37399375438690186 9"Purity/3FRt*SRqqt*FBt/SRqqt_purity‚0.4082101583480835 ("Purity/3FRt1SRqqt1FBt/event_purity‚0 (particle/f_score‚0.40630847215652466 & Purity/4FRt0SRqqt*FBt/FRt_purity‚0 8 Purity/*FRt1SRqqt2FBt/FRt_purity‚0.36194783449172974 ("Purity/3FRt*SRqqt0FBt/event_purity‚0 ,&Purity/4FRt2SRqqt1FBt/event_proportion‚0 particle/accuracy_3_of_6‚1 ) Purity/2FRt3SRqqt1FBt/FRt_purity‚0.25 ("Purity/4FRt0SRqqt0FBt/event_purity‚0 ?&Purity/0FRt2SRqqt0FBt/event_proportion‚0.043283771723508835 =&Purity/1FRt1SRqqt0FBt/event_proportion‚0.1265120953321457 7 Purity/1FRt2SRqqt0FBt/FRt_purity‚0.2609463632106781 ("Purity/1FRt3SRqqt0FBt/event_purity‚0 9"Purity/3FRt2SRqqt*FBt/SRqqt_purity‚0.3836805820465088 ,&Purity/4FRt4SRqqt3FBt/event_proportion‚0 jet/accuracy_5_of_5‚0 & Purity/0FRt4SRqqt*FBt/FBt_purity‚1 7 Purity/2FRt*SRqqt1FBt/FBt_purity‚0.9946236610412598 :"Purity/2FRt0SRqqt1FBt/event_purity‚0.06703703850507736 8"Purity/0FRt1SRqqt3FBt/SRqqt_purity‚0.800000011920929 ("Purity/4FRt1SRqqt0FBt/SRqqt_purity‚1 7 Purity/*FRt2SRqqt3FBt/FBt_purity‚0.6666666865348816 ?&Purity/*FRt0SRqqt3FBt/event_proportion‚0.003717237850651145 >&Purity/0FRt0SRqqt0FBt/event_proportion‚0.04082661122083664 6 Purity/1FRt1SRqqt2FBt/FBt_purity‚0.968154788017273 ,&Purity/1FRt4SRqqt1FBt/event_proportion‚0 ("Purity/3FRt2SRqqt0FBt/event_purity‚0 ,&Purity/4FRt4SRqqt4FBt/event_proportion‚0 (particle/accuracy‚0.8446897864341736 -"Purity/1FRt2SRqqt2FBt/SRqqt_purity‚0.3125 8 Purity/*FRt0SRqqt1FBt/FRt_purity‚0.41688114404678345 7 Purity/2FRt1SRqqt2FBt/FRt_purity‚0.4385965168476105 ?&Purity/3FRt*SRqqt*FBt/event_proportion‚0.009954636916518211 @&Purity/3FRt*SRqqt2FBt/event_proportion‚0.0003150201519019902 ,&Purity/4FRt2SRqqt2FBt/event_proportion‚0 ("Purity/*FRt2SRqqt2FBt/event_purity‚0 ("Purity/*FRt*SRqqt4FBt/SRqqt_purity‚1 8 Purity/*FRt*SRqqt1FBt/FRt_purity‚0.36288997530937195 :"Purity/*FRt*SRqqt0FBt/event_purity‚0.14373420178890228 9"Purity/*FRt1SRqqt2FBt/SRqqt_purity‚0.5854530334472656 @&Purity/*FRt1SRqqt4FBt/event_proportion‚6.300403038039804e-05 A&Purity/1FRt*SRqqt4FBt/event_proportion‚0.00012600806076079607 @&Purity/1FRt0SRqqt4FBt/event_proportion‚6.300403038039804e-05 8"Purity/0FRt*SRqqt3FBt/SRqqt_purity‚0.800000011920929 ( Purity/2FRt2SRqqt2FBt/FRt_purity‚0.5 9"Purity/1FRt*SRqqt*FBt/SRqqt_purity‚0.3892556130886078 7 Purity/1FRt2SRqqt*FBt/FRt_purity‚0.2749495804309845 7 Purity/2FRt0SRqqt2FBt/FBt_purity‚0.9940475821495056 & Purity/3FRt1SRqqt1FBt/FBt_purity‚1 ("Purity/0FRt4SRqqt*FBt/event_purity‚0 8"Purity/*FRt0SRqqt*FBt/event_purity‚0.400392085313797 ?&Purity/1FRt2SRqqt1FBt/event_proportion‚0.008946572430431843 ,&Purity/4FRt0SRqqt4FBt/event_proportion‚0 jet/accuracy_5_of_6‚0 ,&Purity/3FRt3SRqqt4FBt/event_proportion‚0 @&Purity/*FRt0SRqqt4FBt/event_proportion‚0.0003150201519019902 ("Purity/*FRt3SRqqt1FBt/event_purity‚0 ,&Purity/*FRt3SRqqt3FBt/event_proportion‚0 @&Purity/*FRt4SRqqt*FBt/event_proportion‚0.0013230846961960196 @&Purity/0FRt2SRqqt2FBt/event_proportion‚0.0010080644860863686 :"Purity/1FRt*SRqqt0FBt/event_purity‚0.09659674018621445 7 Purity/1FRt*SRqqt1FBt/FBt_purity‚0.9956594705581665 7 Purity/*FRt2SRqqt*FBt/FBt_purity‚0.9885477423667908 8 Purity/1FRt0SRqqt1FBt/FRt_purity‚0.42507821321487427 9"Purity/1FRt3SRqqt*FBt/SRqqt_purity‚0.3688470721244812 ,&Purity/2FRt2SRqqt3FBt/event_proportion‚0 ("Purity/3FRt*SRqqt1FBt/event_purity‚0 *jet/accuracy_1_of_1‚0.4200058877468109 particle/accuracy_3_of_3‚1 particle/accuracy_1_of_5‚1 ?&Purity/2FRt2SRqqt1FBt/event_proportion‚0.003969253972172737 ,&Purity/3FRt*SRqqt3FBt/event_proportion‚0 ,&Purity/4FRt3SRqqt4FBt/event_proportion‚0 jet/accuracy_6_of_7‚0 7 Purity/3FRt0SRqqt2FBt/FRt_purity‚0.3333333432674408 ;"Purity/*FRt3SRqqt0FBt/event_purity‚0.004608294926583767 ("Purity/3FRt0SRqqt0FBt/event_purity‚0 ,jet/accuracy_4_of_5‚0.013685239478945732 & Purity/3FRt2SRqqt*FBt/FBt_purity‚1 ("Purity/3FRt2SRqqt1FBt/event_purity‚0 9"Purity/0FRt3SRqqt1FBt/SRqqt_purity‚0.4583333432674408 ("Purity/3FRt0SRqqt2FBt/event_purity‚0 ("Purity/*FRt1SRqqt4FBt/event_purity‚0 ,&Purity/*FRt2SRqqt4FBt/event_proportion‚0 ("Purity/2FRt1SRqqt2FBt/event_purity‚0 _step‚97 A&Purity/0FRt4SRqqt0FBt/event_proportion‚0.00044102821266278625 ,&Purity/0FRt4SRqqt4FBt/event_proportion‚0 9"Purity/1FRt1SRqqt2FBt/SRqqt_purity‚0.5925169587135315 & Purity/1FRt3SRqqt*FBt/FBt_purity‚1 8 Purity/2FRt0SRqqt0FBt/FRt_purity‚0.24054308235645294 :"Purity/0FRt3SRqqt0FBt/event_purity‚0.01785714365541935 >&Purity/1FRt0SRqqt*FBt/event_proportion‚0.19745463132858276 =&Purity/*FRt0SRqqt2FBt/event_proportion‚0.0483240932226181 =&Purity/*FRt1SRqqt*FBt/event_proportion‚0.4482736885547638 :"Purity/*FRt1SRqqt0FBt/event_purity‚0.17396295070648193 ,&Purity/*FRt4SRqqt3FBt/event_proportion‚0 ?&Purity/1FRt1SRqqt2FBt/event_proportion‚0.006111391354352236 :"Purity/1FRt3SRqqt1FBt/SRqqt_purity‚0.30000004172325134 ;"Purity/2FRt*SRqqt*FBt/event_purity‚0.013833863660693169 9"Purity/*FRt0SRqqt0FBt/event_purity‚0.1856137067079544 @&Purity/*FRt3SRqqt2FBt/event_proportion‚6.300403038039804e-05 8 Purity/1FRt1SRqqt*FBt/FRt_purity‚0.29857590794563293 9"Purity/2FRt*SRqqt2FBt/event_purity‚0.0277777798473835 ?&Purity/2FRt2SRqqt0FBt/event_proportion‚0.015435988083481789 ,&Purity/4FRt1SRqqt1FBt/event_proportion‚0 ( Purity/2FRt*SRqqt3FBt/FRt_purity‚0.5 ("Purity/1FRt1SRqqt3FBt/event_purity‚0 7 Purity/*FRt1SRqqt*FBt/FBt_purity‚0.9838809967041016 =&Purity/*FRt2SRqqt*FBt/event_proportion‚0.1332535296678543 @&Purity/*FRt4SRqqt1FBt/event_proportion‚6.300403038039804e-05 :"Purity/0FRt2SRqqt0FBt/event_purity‚0.04079851135611534 ,&Purity/0FRt4SRqqt3FBt/event_proportion‚0 9"Purity/3FRt1SRqqt*FBt/SRqqt_purity‚0.4660493731498718 ,&Purity/3FRt2SRqqt2FBt/event_proportion‚0 ,&Purity/4FRt1SRqqt4FBt/event_proportion‚0 ("Purity/3FRt0SRqqt*FBt/event_purity‚0 A&Purity/0FRt1SRqqt3FBt/event_proportion‚0.00044102821266278625 9"Purity/2FRt0SRqqt2FBt/event_purity‚0.0535714291036129 particle/accuracy_2_of_2‚1 ("Purity/0FRt2SRqqt2FBt/event_purity‚0 ("Purity/3FRt*SRqqt2FBt/event_purity‚0 ("Purity/*FRt1SRqqt4FBt/SRqqt_purity‚1 ("Purity/1FRt3SRqqt2FBt/event_purity‚0 :"Purity/0FRt*SRqqt3FBt/event_purity‚0.15432098507881165 9"Purity/0FRt1SRqqt0FBt/SRqqt_purity‚0.3298182785511017 ,&Purity/2FRt4SRqqt3FBt/event_proportion‚0 7 Purity/3FRt*SRqqt1FBt/FRt_purity‚0.3090909421443939 ,&Purity/4FRt3SRqqt3FBt/event_proportion‚0 ("Purity/1FRt1SRqqt3FBt/SRqqt_purity‚1 & Purity/4FRt0SRqqt0FBt/FRt_purity‚0 6 Purity/0FRt2SRqqt*FBt/FBt_purity‚0.984639048576355 particle/accuracy_2_of_6‚1 A&Purity/0FRt0SRqqt4FBt/event_proportion‚0.00025201612152159214 ;"Purity/2FRt*SRqqt1FBt/event_purity‚0.031320054084062576 9"Purity/2FRt2SRqqt1FBt/SRqqt_purity‚0.4690476059913635 *jet/accuracy_1_of_2‚0.7170048952102661 ("Purity/4FRt*SRqqt*FBt/event_purity‚0 ;"Purity/*FRt3SRqqt*FBt/event_purity‚0.004608294926583767 9"Purity/2FRt2SRqqt*FBt/SRqqt_purity‚0.3841923773288727 *"Purity/3FRt1SRqqt2FBt/SRqqt_purity‚0.5 ("Purity/1FRt*SRqqt4FBt/SRqqt_purity‚1 & Purity/4FRt1SRqqt0FBt/FRt_purity‚0 :"Purity/0FRt2SRqqt*FBt/event_purity‚0.03501037880778313 7 Purity/1FRt*SRqqt1FBt/FRt_purity‚0.3685687184333801 ,particle/sensitivity‚0.34033629298210144 particle/accuracy_6_of_6‚1 & Purity/1FRt3SRqqt2FBt/FBt_purity‚1 9"Purity/*FRt0SRqqt3FBt/event_purity‚0.1130952388048172 7 Purity/1FRt0SRqqt1FBt/FBt_purity‚0.9966126084327698 7 Purity/0FRt*SRqqt1FBt/FBt_purity‚0.9953582882881165 7 Purity/1FRt1SRqqt2FBt/FRt_purity‚0.3760204315185547 8 Purity/3FRt1SRqqt1FBt/FRt_purity‚0.26923075318336487 ,jet/accuracy_4_of_4‚0.005122719798237085 particle/accuracy_3_of_5‚1 +jet/accuracy_2_of_6‚0.49122804403305054 7 Purity/0FRt2SRqqt2FBt/FBt_purity‚0.9807692170143127 9"Purity/*FRt*SRqqt2FBt/event_purity‚0.1741527020931244 7 Purity/*FRt2SRqqt0FBt/FRt_purity‚0.2678324282169342 ?&Purity/2FRt0SRqqt1FBt/event_proportion‚0.008946572430431843 7 Purity/*FRt0SRqqt1FBt/FBt_purity‚0.9979555010795593 @&Purity/*FRt1SRqqt3FBt/event_proportion‚0.0006930443341843784 ;"Purity/2FRt0SRqqt0FBt/event_purity‚0.024096736684441566 9"Purity/2FRt1SRqqt*FBt/SRqqt_purity‚0.4205760955810547 ("Purity/1FRt4SRqqt*FBt/event_purity‚0 7 Purity/*FRt*SRqqt1FBt/FBt_purity‚0.9956392049789429 7 Purity/*FRt1SRqqt2FBt/FBt_purity‚0.9632813334465027 @&Purity/2FRt3SRqqt1FBt/event_proportion‚0.0003780241822823882 ?&Purity/3FRt1SRqqt*FBt/event_proportion‚0.003717237850651145 7 Purity/3FRt2SRqqt0FBt/FRt_purity‚0.2777777910232544 * Purity/0FRt0SRqqt4FBt/FBt_purity‚0.875 ("Purity/1FRt2SRqqt1FBt/event_purity‚0 & Purity/3FRt*SRqqt1FBt/FBt_purity‚1 8"Purity/3FRt3SRqqt*FBt/SRqqt_purity‚0.380952388048172 :"Purity/0FRt*SRqqt*FBt/SRqqt_purity‚0.37402716279029846 9"Purity/1FRt2SRqqt*FBt/SRqqt_purity‚0.3777875006198883 8 Purity/1FRt3SRqqt0FBt/FRt_purity‚0.26478493213653564 7 Purity/2FRt0SRqqt*FBt/FBt_purity‚0.9967741966247559 >&Purity/2FRt1SRqqt0FBt/event_proportion‚0.03181703761219978 ?&Purity/2FRt1SRqqt2FBt/event_proportion‚0.001890120911411941 ("Purity/2FRt2SRqqt0FBt/event_purity‚0 ?&Purity/2FRt3SRqqt*FBt/event_proportion‚0.004032257944345474 >&Purity/0FRt2SRqqt*FBt/event_proportion‚0.05361643061041832 9"Purity/1FRt1SRqqt1FBt/event_purity‚0.0332222655415535 7 Purity/3FRt0SRqqt*FBt/FRt_purity‚0.2929293215274811 ,&Purity/4FRt*SRqqt1FBt/event_proportion‚0 ,&Purity/4FRt0SRqqt1FBt/event_proportion‚0 particle/accuracy_2_of_4‚1 ("Purity/1FRt*SRqqt4FBt/event_purity‚0 ?&Purity/1FRt*SRqqt2FBt/event_proportion‚0.022303426638245583 7 Purity/*FRt1SRqqt0FBt/FRt_purity‚0.2850886881351471 ,&Purity/*FRt4SRqqt2FBt/event_proportion‚0 @&Purity/1FRt4SRqqt*FBt/event_proportion‚0.0003780241822823882 ,&Purity/4FRt*SRqqt2FBt/event_proportion‚0 ("Purity/1FRt0SRqqt4FBt/event_purity‚0 ("Purity/2FRt*SRqqt3FBt/event_purity‚0 7 Purity/1FRt1SRqqt3FBt/FBt_purity‚0.8333333730697632 % _timestamp‚1.7884873831279972e+09 ?&Purity/*FRt3SRqqt1FBt/event_proportion‚0.002142137149348855 9"Purity/2FRt*SRqqt0FBt/SRqqt_purity‚0.3663597106933594 ("Purity/3FRt1SRqqt0FBt/event_purity‚0 ,&Purity/3FRt2SRqqt3FBt/event_proportion‚0 @&Purity/4FRt1SRqqt0FBt/event_proportion‚6.300403038039804e-05 +"Purity/*FRt4SRqqt1FBt/SRqqt_purity‚0.25 7 Purity/1FRt2SRqqt2FBt/FRt_purity‚0.4305555820465088 jet/accuracy_7_of_7‚0 @&Purity/0FRt2SRqqt3FBt/event_proportion‚6.300403038039804e-05 8 Purity/2FRt2SRqqt*FBt/FRt_purity‚0.27208051085472107 particle/accuracy_4_of_6‚1 7 Purity/*FRt2SRqqt2FBt/FBt_purity‚0.9912281036376953 =&Purity/*FRt*SRqqt*FBt/event_proportion‚0.9591733813285828 8 Purity/2FRt1SRqqt*FBt/FRt_purity‚0.29895952343940735 particle/accuracy_2_of_3‚1 ("Purity/4FRt1SRqqt*FBt/event_purity‚0 7 Purity/*FRt3SRqqt1FBt/FBt_purity‚0.9523809552192688 ,&Purity/*FRt3SRqqt4FBt/event_proportion‚0 :"Purity/0FRt0SRqqt2FBt/event_purity‚0.30155453085899353 7 Purity/2FRt1SRqqt0FBt/FRt_purity‚0.2757420837879181 ;"Purity/*FRt2SRqqt1FBt/event_purity‚0.004378923214972019 ,&Purity/0FRt3SRqqt4FBt/event_proportion‚0 =&Purity/1FRt1SRqqt*FBt/event_proportion‚0.1723790317773819 7 Purity/2FRt2SRqqt0FBt/FRt_purity‚0.2805858254432678 ("Purity/2FRt3SRqqt*FBt/event_purity‚0 @&Purity/3FRt0SRqqt0FBt/event_proportion‚0.0015120967291295528 @&Purity/4FRt2SRqqt*FBt/event_proportion‚6.300403038039804e-05 & Purity/2FRt0SRqqt3FBt/FBt_purity‚1 7 Purity/*FRt0SRqqt3FBt/FRt_purity‚0.4000000059604645 9"Purity/*FRt2SRqqt*FBt/SRqqt_purity‚0.3802308142185211 ?&Purity/2FRt*SRqqt2FBt/event_proportion‚0.005418346729129553 ,&Purity/3FRt3SRqqt3FBt/event_proportion‚0 particle/accuracy_1_of_3‚1 8 Purity/*FRt*SRqqt3FBt/FRt_purity‚0.44117647409439087 :"Purity/*FRt*SRqqt1FBt/event_purity‚0.41015782952308655 8 Purity/*FRt1SRqqt*FBt/FRt_purity‚0.29629623889923096 @&Purity/*FRt4SRqqt0FBt/event_proportion‚0.0012600806076079607 :"Purity/0FRt*SRqqt0FBt/SRqqt_purity‚0.33687901496887207 :"Purity/0FRt1SRqqt2FBt/event_purity‚0.06740740686655045 @&Purity/1FRt3SRqqt1FBt/event_proportion‚0.0011340725468471646 ,jet/accuracy_3_of_3‚0.022896582260727882 :"Purity/*FRt1SRqqt*FBt/event_purity‚0.15045492351055145 >&Purity/*FRt3SRqqt0FBt/event_proportion‚0.01827116869390011 >&Purity/1FRt*SRqqt1FBt/event_proportion‚0.09929435700178146 7 Purity/1FRt1SRqqt*FBt/FBt_purity‚0.9840551614761353 @&Purity/2FRt2SRqqt2FBt/event_proportion‚6.300403038039804e-05 +jet/accuracy_3_of_6‚0.17543859779834747 :"Purity/2FRt3SRqqt1FBt/SRqqt_purity‚0.23333331942558289 ?&Purity/*FRt2SRqqt2FBt/event_proportion‚0.002079133177176118 9"Purity/*FRt*SRqqt3FBt/SRqqt_purity‚0.9047619104385376 ("Purity/0FRt2SRqqt3FBt/SRqqt_purity‚1 8 Purity/1FRt3SRqqt*FBt/FRt_purity‚0.24082182347774506 9"Purity/2FRt1SRqqt1FBt/SRqqt_purity‚0.5210061073303223 7 Purity/3FRt0SRqqt0FBt/FRt_purity‚0.1993464082479477 -"Purity/*FRt4SRqqt0FBt/SRqqt_purity‚0.3125 & Purity/*FRt4SRqqt1FBt/FBt_purity‚1 7 Purity/1FRt0SRqqt3FBt/FRt_purity‚0.3928571343421936 ,&Purity/2FRt3SRqqt2FBt/event_proportion‚0 @&Purity/3FRt3SRqqt*FBt/event_proportion‚0.0005670362734235823 @&Purity/4FRt0SRqqt*FBt/event_proportion‚6.300403038039804e-05 particle/accuracy_1_of_2‚1 9"Purity/*FRt*SRqqt*FBt/SRqqt_purity‚0.3837721347808838 ("Purity/0FRt0SRqqt0FBt/event_purity‚1 ?&Purity/2FRt*SRqqt1FBt/event_proportion‚0.021169355139136314 7 Purity/2FRt*SRqqt1FBt/FRt_purity‚0.3549676239490509 :"Purity/2FRt3SRqqt0FBt/SRqqt_purity‚0.35349929332733154 8 Purity/3FRt2SRqqt*FBt/FRt_purity‚0.26273152232170105 ,&Purity/3FRt2SRqqt4FBt/event_proportion‚0 ,&Purity/4FRt4SRqqt*FBt/event_proportion‚0 ;"Purity/*FRt2SRqqt0FBt/event_purity‚0.016516922041773796 ?&Purity/0FRt0SRqqt2FBt/event_proportion‚0.029674898833036423 ?&Purity/1FRt0SRqqt1FBt/event_proportion‚0.049773186445236206 ,&Purity/1FRt4SRqqt4FBt/eèö@Y!vent_proportion‚0 7 Purity/2FRt0SRqqt2FBt/FRt_purity‚0.4940476417541504 ?&Purity/2FRt2SRqqt*FBt/event_proportion‚0.019468246027827263 particle/accuracy_4_of_5‚1 7validation_average_jet_accuracy‚0.39937517046928406 :"Purity/*FRt*SRqqt3FBt/event_purity‚0.10057471692562103 7 Purity/*FRt2SRqqt*FBt/FRt_purity‚0.2737419605255127 :"Purity/0FRt0SRqqt3FBt/event_purity‚0.15333333611488342 ?&Purity/0FRt3SRqqt*FBt/event_proportion‚0.006741431541740894 @&Purity/0FRt3SRqqt1FBt/event_proportion‚0.0006300403038039804 ,&Purity/1FRt3SRqqt4FBt/event_proportion‚0 ,&Purity/4FRt4SRqqt0FBt/event_proportion‚0 7 Purity/*FRt0SRqqt4FBt/FBt_purity‚0.8999999761581421 =&Purity/*FRt0SRqqt*FBt/event_proportion‚0.3558467626571655 7 Purity/0FRt0SRqqt1FBt/FBt_purity‚0.9985072016716003 ,&Purity/1FRt3SRqqt3FBt/event_proportion‚0 7 Purity/2FRt1SRqqt2FBt/FBt_purity‚0.9868420958518982 ,&Purity/3FRt0SRqqt4FBt/event_proportion‚0 * Purity/*FRt*SRqqt4FBt/FBt_purity‚0.875 * Purity/0FRt*SRqqt4FBt/FBt_purity‚0.875 & Purity/2FRt3SRqqt1FBt/FBt_purity‚1 7 Purity/0FRt1SRqqt1FBt/FBt_purity‚0.9933437705039978 jet/accuracy_1_of_7‚1 ( Purity/*FRt*SRqqt4FBt/FRt_purity‚0.5 ("Purity/4FRt0SRqqt*FBt/event_purity‚0 9"Purity/1FRt*SRqqt1FBt/SRqqt_purity‚0.5021957159042358 7 Purity/1FRt2SRqqt*FBt/FBt_purity‚0.9976958632469177 @&Purity/2FRt4SRqqt0FBt/event_proportion‚0.0003150201519019902 @&Purity/3FRt*SRqqt0FBt/event_proportion‚0.0071824598126113415 @&Purity/4FRt*SRqqt*FBt/event_proportion‚0.0001890120911411941 ,&Purity/4FRt0SRqqt3FBt/event_proportion‚0 & Purity/1FRt0SRqqt4FBt/FRt_purity‚1 & Purity/3FRt0SRqqt2FBt/FBt_purity‚1 ,&Purity/0FRt3SRqqt3FBt/event_proportion‚0 ,&Purity/4FRt*SRqqt4FBt/event_proportion‚0 ("Purity/*FRt4SRqqt*FBt/event_purity‚0 & Purity/*FRt0SRqqt4FBt/FRt_purity‚1 + Purity/0FRt3SRqqt1FBt/FBt_purity‚0.8125 & Purity/4FRt1SRqqt*FBt/FRt_purity‚0 ("Purity/2FRt2SRqqt2FBt/SRqqt_purity‚0 ("Purity/4FRt2SRqqt0FBt/event_purity‚0 :"Purity/0FRt3SRqqt*FBt/event_purity‚0.01785714365541935 ("Purity/2FRt4SRqqt*FBt/event_purity‚0 & Purity/1FRt0SRqqt4FBt/FBt_purity‚1 ("Purity/4FRt2SRqqt*FBt/SRqqt_purity‚1 particle/accuracy_1_of_4‚1 ("Purity/4FRt*SRqqt0FBt/event_purity‚0 & Purity/2FRt2SRqqt2FBt/FBt_purity‚1 ("Purity/0FRt2SRqqt3FBt/event_purity‚0 >&Purity/*FRt1SRqqt0FBt/event_proportion‚0.33039313554763794 ,&Purity/4FRt*SRqqt3FBt/event_proportion‚0 particle/accuracy_3_of_4‚1 ("Purity/*FRt0SRqqt4FBt/event_purity‚0 >&Purity/*FRt2SRqqt0FBt/event_proportion‚0.10855595022439957 >&Purity/0FRt*SRqqt0FBt/event_proportion‚0.21931703388690948 ,&Purity/1FRt4SRqqt2FBt/event_proportion‚0 ,&Purity/1FRt4SRqqt3FBt/event_proportion‚0 ,&Purity/2FRt2SRqqt4FBt/event_proportion‚0 & Purity/3FRt1SRqqt*FBt/FBt_purity‚1 ,&Purity/3FRt1SRqqt4FBt/event_proportion‚0 +jet/accuracy_3_of_5‚0.09869074076414108 9"Purity/*FRt*SRqqt2FBt/SRqqt_purity‚0.5561156868934631 :"Purity/0FRt1SRqqt1FBt/event_purity‚0.15148796141147614 8 Purity/1FRt0SRqqt0FBt/FRt_purity‚0.21827785670757294 9"Purity/2FRt*SRqqt*FBt/SRqqt_purity‚0.3997207581996918 9"Purity/2FRt0SRqqt*FBt/event_purity‚0.0334082692861557 ,&Purity/2FRt4SRqqt2FBt/event_proportion‚0 ("Purity/3FRt0SRqqt1FBt/event_purity‚0 & Purity/3FRt*SRqqt2FBt/FBt_purity‚1 :"Purity/0FRt*SRqqt1FBt/SRqqt_purity‚0.49202239513397217 7 Purity/1FRt*SRqqt*FBt/FBt_purity‚0.9874643087387085 *jet/accuracy_1_of_5‚0.9576897621154785 ("Purity/1FRt1SRqqt4FBt/event_purity‚0 ("Purity/*FRt2SRqqt3FBt/event_purity‚0 >&Purity/*FRt0SRqqt0FBt/event_proportion‚0.15757308900356293 7 Purity/*FRt2SRqqt1FBt/FBt_purity‚0.9925950169563293 @&Purity/1FRt3SRqqt2FBt/event_proportion‚6.300403038039804e-05 8 Purity/2FRt3SRqqt*FBt/FRt_purity‚0.31300002336502075 ("Purity/3FRt1SRqqt*FBt/event_purity‚0 A&Purity/3FRt1SRqqt2FBt/event_proportion‚0.00025201612152159214 @&Purity/3FRt2SRqqt0FBt/event_proportion‚0.0024571572430431843 ,&Purity/4FRt3SRqqt2FBt/event_proportion‚0 6 Purity/*FRt0SRqqt0FBt/FRt_purity‚0.222781702876091 ,&Purity/*FRt4SRqqt4FBt/event_proportion‚0 @&Purity/1FRt1SRqqt4FBt/event_proportion‚6.300403038039804e-05 ,&Purity/2FRt*SRqqt4FBt/event_proportion‚0 ?&Purity/2FRt0SRqqt0FBt/event_proportion‚0.024571573361754417 & Purity/2FRt*SRqqt3FBt/FBt_purity‚1 6 Purity/*FRt*SRqqt0FBt/FRt_purity‚0.258646160364151 :"Purity/0FRt2SRqqt1FBt/SRqqt_purity‚0.46701231598854065 ,&Purity/2FRt3SRqqt3FBt/event_proportion‚0 @&Purity/4FRt*SRqqt0FBt/event_proportion‚0.0001890120911411941 & Purity/3FRt2SRqqt1FBt/FBt_purity‚1 particle/accuracy_5_of_7‚1 ("Purity/3FRt1SRqqt2FBt/event_purity‚0 7 Purity/1FRt0SRqqt2FBt/FBt_purity‚0.9809930324554443 ?&Purity/2FRt0SRqqt*FBt/event_proportion‚0.037046369165182114 @&Purity/2FRt0SRqqt3FBt/event_proportion‚6.300403038039804e-05 7 Purity/3FRt*SRqqt0FBt/FRt_purity‚0.2666666805744171 A&Purity/3FRt4SRqqt0FBt/event_proportion‚0.00012600806076079607 +"Purity/0FRt4SRqqt1FBt/SRqqt_purity‚0.25 *loss/val_total_loss‚43.154410914983565 @&Purity/0FRt4SRqqt1FBt/event_proportion‚6.300403038039804e-05 ?&Purity/1FRt3SRqqt0FBt/event_proportion‚0.007938507944345474 & Purity/1FRt3SRqqt1FBt/FBt_purity‚1 7 Purity/*FRt4SRqqt*FBt/FRt_purity‚0.3611111342906952 jet/accuracy_3_of_7‚0 & Purity/4FRt2SRqqt*FBt/FRt_purity‚0 9"Purity/*FRt1SRqqt1FBt/SRqqt_purity‚0.5316541790962219 ?&Purity/0FRt3SRqqt0FBt/event_proportion‚0.006111391354352236 =&Purity/1FRt*SRqqt*FBt/event_proportion‚0.4366179406642914 ,&Purity/3FRt0SRqqt3FBt/event_proportion‚0 particle/accuracy_1_of_7‚1 particle/accuracy_7_of_7‚1 ?&Purity/0FRt1SRqqt2FBt/event_proportion‚0.008757560513913631 :"Purity/1FRt*SRqqt*FBt/event_purity‚0.11178943514823914 :"Purity/2FRt2SRqqt0FBt/SRqqt_purity‚0.35229483246803284 ,&Purity/2FRt4SRqqt4FBt/event_proportion‚0 9"Purity/3FRt1SRqqt1FBt/SRqqt_purity‚0.6153846383094788 particle/accuracy_5_of_5‚1 ."Purity/0FRt4SRqqt*FBt/SRqqt_purity‚0.21875 7 Purity/*FRt*SRqqt2FBt/FBt_purity‚0.9772515892982483 7 Purity/*FRt1SRqqt1FBt/FBt_purity‚0.9942842721939087 @&Purity/1FRt*SRqqt3FBt/event_proportion‚0.0013230846961960196 ("Purity/1FRt3SRqqt*FBt/event_purity‚0 @&Purity/2FRt4SRqqt*FBt/event_proportion‚0.0003150201519019902 ("Purity/1FRt1SRqqt4FBt/SRqqt_purity‚1 ("Purity/*FRt2SRqqt3FBt/SRqqt_purity‚1 >&Purity/*FRt*SRqqt2FBt/event_proportion‚0.06747731566429138 7 Purity/*FRt0SRqqt2FBt/FBt_purity‚0.9814485311508179 ,&Purity/3FRt4SRqqt1FBt/event_proportion‚0 particle/accuracy_2_of_5‚1a3.€½Ç ‚(:… ÞÍèÔ²ÐЯ gpu.0.gpu‚6.0 gpu.0.memory‚0+ gpu.0.memoryAllocated‚47.54737854003906* gpu.0.memoryAllocatedBytes‚ 40842887168 gpu.0.temp‚33.0 gpu.0.powerWatts‚86.834' gpu.0.enforcedPowerLimitWatts‚400.0 gpu.0.powerPercent‚21.7085 gpu.0.smClock‚1155 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0[¸Ó*RÈ ‚(:Ç ÞÍèÔ¢š¦° disk.rbd2.out‚ 0.50390625 disk.sda2.in‚ 0.0078125 disk.sda.out‚210.91650390625 disk.dm-0.in‚ 8.77734375! disk.sdb2.out‚210.91650390625 disk.sdb.out‚210.91650390625' proc.memory.availableMB‚ 2388.546875 disk.md0p1.in‚ 0.0078125 disk.rbd0.in‚0* proc.memory.percent‚26.196646690368652 proc.cpu.threads‚17 disk.nvme1n1.in‚4.4375# disk.nvme1n1.out‚13152.02734375 disk.sda.in‚ 0.0078125 disk.rbd0.out‚ 0.341796875$ memory_percent‚85.42146682739258 cpu‚25.54990369104627 disk.sda1.in‚0 disk.sda1.out‚0 disk.md0.in‚ 0.0078125 disk.rbd2.in‚0! disk.sda2.out‚210.91650390625 network.sent‚ 2.601834e+06% disk./.usageGB‚2981.8707160949707 disk.md0.out‚ 201.95703125 disk.sdb.in‚0 network.recv‚743712 disk.nvme0n1.in‚ 4.33984375 disk.dm-0.out‚ 26843.4765625 disk.sdb1.in‚0# proc.memory.rssMB‚ 4292.05859375) disk./.usagePercent‚83.37435323186531# disk.nvme0n1.out‚13691.44921875 disk.sdb1.out‚0 disk.sdb2.in‚0 disk.md0p1.out‚ 201.95703125ÙöžÑRÉ ‚(:Ç íÍèÔ Ðί disk.rbd2.in‚0 disk.sdb2.in‚0 network.sent‚ 2.693617e+06% disk./.usageGB‚2982.0898246765137 disk.sdb1.in‚0! disk.sdb2.out‚212.52392578125 disk.md0.in‚ 0.0078125 disk.sda.out‚212.52392578125# proc.memory.rssMB‚ 3057.48046875! disk.nvme0n1.out‚ 14595.515625 network.recv‚749294) disk./.usagePercent‚83.38047959950933 disk.rbd2.out‚ 0.50390625 disk.sdb.in‚0 disk.sdb.out‚212.52392578125 disk.sda.in‚ 0.0078125 disk.sdb1.out‚0 disk.sda1.in‚0 disk.md0p1.in‚ 0.0078125 disk.sda2.in‚ 0.0078125 disk.nvme1n1.in‚4.4375 disk.rbd0.out‚ 0.341796875 disk.nvme0n1.in‚ 4.33984375 proc.cpu.threads‚17 disk.md0p1.out‚ 203.49609375! disk.sda2.out‚212.52392578125# disk.nvme1n1.out‚16319.77734375 disk.dm-0.in‚ 8.77734375$ memory_percent‚77.88703441619873) proc.memory.percent‚18.66137981414795 cpu‚25.54554792075805 disk.dm-0.out‚30915.29296875 disk.md0.out‚ 203.49609375 disk.rbd0.in‚0) proc.memory.availableMB‚ 3622.98828125 disk.sda1.out‚0±ôM4“Ê ‚(:ˆ íÍèÔ‹©° gpu.0.gpu‚38.0 gpu.0.memory‚10+ gpu.0.memoryAllocated‚47.54737854003906* gpu.0.memoryAllocatedBytes‚ 40842887168 gpu.0.temp‚44.0 gpu.0.powerWatts‚306.098' gpu.0.enforcedPowerLimitWatts‚400.0 gpu.0.powerPercent‚76.5245 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0pâI’Ë ‚(:‡ üÍèÔ«­ù¯ gpu.0.gpu‚98.0 gpu.0.memory‚19+ gpu.0.memoryAllocated‚47.54737854003906* gpu.0.memoryAllocatedBytes‚ 40842887168 gpu.0.temp‚42.0 gpu.0.powerWatts‚120.65' gpu.0.enforcedPowerLimitWatts‚400.0 gpu.0.powerPercent‚30.1625 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0,ŠXEÌ ‚(:º üÍèÔßù® disk.sdb2.in‚0 disk.sda1.out‚0 network.sent‚ 2.699777e+06" disk.nvme0n1.out‚ 14717.9453125 disk.dm-0.out‚31097.05078125 disk.sdb1.in‚0 disk.rbd2.in‚0 proc.cpu.threads‚17$ memory_percent‚77.88703441619873 disk.dm-0.in‚ 8.77734375 disk.sdb1.out‚0 disk.md0p1.out‚205.125! disk.sda2.out‚214.22607421875 disk.sda.in‚ 0.0078125 disk.rbd0.out‚ 0.353515625) disk./.usagePercent‚83.38748388999323 disk.md0p1.in‚ 0.0078125 disk.sdb.out‚214.22607421875 disk.rbd0.in‚0 disk.nvme1n1.in‚4.4375# disk.nvme1n1.out‚16379.10546875 disk.sda.out‚214.22607421875 disk.nvme0n1.in‚ 4.33984375# proc.memory.rssMB‚ 3057.48046875) proc.memory.percent‚18.66137981414795 network.recv‚754068 disk.rbd2.out‚0.515625 disk.sdb.in‚0 disk.sda2.in‚ 0.0078125# disk./.usageGB‚2982.34033203125 disk.md0.in‚ 0.0078125 disk.md0.out‚205.125 cpu‚25.54436817329118! disk.sdb2.out‚214.22607421875 disk.sda1.in‚0) proc.memory.availableMB‚ 3622.98828125w|Ô“Í ‚(:ˆ ‹ÎèÔ‘Èܯ gpu.0.gpu‚97.0 gpu.0.memory‚17+ gpu.0.memoryAllocated‚47.54737854003906* gpu.0.memoryAllocatedBytes‚ 40842887168 gpu.0.temp‚45.0 gpu.0.powerWatts‚235.058' gpu.0.enforcedPowerLimitWatts‚400.0 gpu.0.powerPercent‚58.7645 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0¨pêKÎ ‚(:À ‹ÎèÔžÚÀ° disk.sdb.out‚215.63427734375 disk.rbd2.in‚0 disk.sdb.in‚0$ memory_percent‚77.88705825805664) proc.memory.percent‚18.66137981414795 disk.sda.in‚ 0.0078125 disk.dm-0.out‚31344.69140625 disk.sdb2.in‚0 disk.md0.in‚ 0.0078125 disk.md0.out‚ 206.46484375' proc.memory.availableMB‚ 3622.984375 disk.sda.out‚215.63427734375# proc.memory.rssMB‚ 3057.48046875 proc.cpu.threads‚17) disk./.usagePercent‚83.40143402098943 disk.nvme1n1.in‚4.4375 disk.dm-0.in‚ 8.77734375 disk.sda1.out‚0 disk.rbd2.out‚0.515625 disk.rbd0.in‚0 disk.rbd0.out‚ 0.416015625 disk.nvme0n1.in‚ 4.33984375 disk.sdb1.in‚0! disk.sdb2.out‚215.63427734375 disk.md0p1.out‚ 206.46484375! disk.sda2.out‚215.63427734375 network.recv‚757761$ disk./.usageGB‚2982.839256286621 disk.sda1.in‚0 disk.sda2.in‚ 0.0078125 network.sent‚ 2.705119e+06 disk.nvme0n1.out‚ 14810.3125 disk.sdb1.out‚0# disk.nvme1n1.out‚16534.37890625 disk.md0p1.in‚ 0.0078125 cpu‚25.54337304930759&RÜÏ ‚(:’ šÎèÔ®ˆÖ¯ gpu.0.gpu‚71.0 gpu.0.memory‚14+ gpu.0.memoryAllocated‚47.54737854003906* gpu.0.memoryAllocatedBytes‚ 40842887168 gpu.0.temp‚45.0 gpu.0.powerWatts‚239.491' gpu.0.enforcedPowerLimitWatts‚400.0( gpu.0.powerPercent‚59.87275000000001 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0L£ò7FÐ ‚(:» šÎèÔÈ£ï¯ disk.sda2.in‚ 0.0078125 disk.md0.out‚ 207.69921875 disk.md0p1.in‚ 0.0078125 disk.rbd2.in‚0 disk.sda.in‚ 0.0078125 disk.nvme0n1.in‚ 4.33984375 disk.dm-0.out‚31555.86328125 disk.rbd2.out‚0.578125) proc.memory.percent‚18.66137981414795) disk./.usagePercent‚83.40143562089742 disk.sda2.out‚ 216.916015625" disk.nvme0n1.out‚ 14840.6640625 disk.sdb1.out‚0 disk.sda1.in‚0$ memory_percent‚77.88705825805664' proc.memory.availableMB‚ 3622.984375# proc.memory.rssMB‚ 3057.48046875 network.recv‚761163 disk.sdb.out‚ 216.916015625 disk.rbd0.in‚0 disk.sdb2.in‚0 disk.sda1.out‚0 disk.sdb.in‚0 disk.md0p1.out‚ 207.69921875 disk.md0.in‚ 0.0078125 disk.nvme1n1.in‚4.4375 disk.sdb1.in‚0 cpu‚25.542332663745825 proc.cpu.threads‚17# disk.nvme1n1.out‚16715.19921875 disk.sda.out‚ 216.916015625 disk.rbd0.out‚ 0.416015625 disk.dm-0.in‚ 8.77734375 disk.sdb2.out‚ 216.916015625 network.sent‚ 2.709999e+06# disk./.usageGB‚2982.83931350708 €ßf2Ñ - _wandbruntime‚2263  _runtime‚2263£šXÆÒ À trainer/global_step‚1249 % _timestamp‚1.7884874643785589e+09 _runtime‚2263.484022467 _step‚98 & lr-AdamW/pg1‚0.0010982818701134602 & lr-AdamW/pg2‚0.0010982818701134602uñÿ©ÊÓ Ä _runtime‚2263.484022467 _step‚98 & lr-AdamW/pg1‚0.0010982818701134602 & lr-AdamW/pg2‚0.0010982818701134602 trainer/global_step‚1249 % _timestamp‚1.7884874643785589e+09b<t¹2Ô - _wandbruntime‚2263  _runtime‚2263´`éÍð'Õ êO E8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚41523974 L?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚18028289 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚23681 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚25427968 dQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚55635833980416 `LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚265352047426256 RBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 40290484224 IDeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚2684 TDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 40290484224 F=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚3681 % _timestamp‚1.7884874643796206e+09 ^LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 3008959712256 UHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚98523648 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 YJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 2514370136 H?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚2147 H?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚1534 H?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚1309 ]IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚276414890554368 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚8610970 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 [KDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 37073453056 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚69425664 WEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 3006443349504 SGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚5592194 ]JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚59389445345792 _MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 3753541939712 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚5592197 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 963120128 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚23980 SDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 3142752768 TGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚18028295 PCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚18028289 YFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚59389346822144 D;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚3681 L?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚23495685 VGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 3142752768 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚2684 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚3 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 UFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 3217031168 A8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚3681 MDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚1534 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚19683 `LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚276414890554368 \HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚265352021998288 O@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 2541790720 TDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 35913877504 ZHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 3006443349504 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚23681 SGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚8610415 VGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 3140738484 ZKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 3217031168 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚19689 VBDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated‚279423850266624 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚29097984 A8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚3681 RCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 2539798104 NFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚330 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 WGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 35913877504 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 M=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 38610163200 UADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚279421308475904 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 WGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 34283657216 ^LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 3005146231515 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 BDeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚279421308475904 YEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚279423850266624 TEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 1515847168 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚19689 QDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚14203167 NFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚986 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 VGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 2516362752 M@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚14202609 D>DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 YEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚268357193657771 XIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 3217031168 trainer/global_step‚1249 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚19683 O?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 40290484224 KBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚1534 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚25427968 YEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚276414865126400 RCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 2541790720 M@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚41543663 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 577553920 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 IDeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚279421308475904 YEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚279423850266624 TEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 1515847168 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚19689 QDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚14203167 NFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚986 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 VGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 2516362752 M@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚14202609 D>DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 YEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚268357193657771 XIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 3217031168 trainer/global_step‚1249 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚19683 O?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 40290484224 KBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚1534 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚25427968 YEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚276414865126400 RCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 2541790720 M@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚41543663 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 577553920 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 IDeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚23681 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚25427968 dQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚55635833980416 `LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚265352047426256 RBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 40290484224 IDeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚2684 TDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 40290484224 F=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚3681 % _timestamp‚1.7884874643796206e+09 ^LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 3008959712256 UHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚98523648 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 YJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 2514370136 H?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚2147 H?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚1534 H?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚1309 ]IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚276414890554368 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚8610970 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 [KDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 37073453056 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚69425664 WEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 3006443349504 SGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚5592194 ]JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚59389445345792 _MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 3753541939712 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚5592197 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 963120128cדÓu”× ‚(:‰ ©ÎèÔëùÙ¯ gpu.0.gpu‚97.0 gpu.0.memory‚17+ gpu.0.memoryAllocated‚47.54737854003906* gpu.0.memoryAllocatedBytes‚ 40842887168 gpu.0.temp‚45.0 gpu.0.powerWatts‚229.155' gpu.0.enforcedPowerLimitWatts‚400.0 gpu.0.powerPercent‚57.28875 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0Br³_AØ ‚(:¶ ©ÎèÔÄÐË° disk.sdb1.in‚0 disk.sdb1.out‚0 disk.sda2.out‚219.0205078125 disk.nvme1n1.out‚ 16715.5625 network.sent‚ 2.726754e+06 disk.nvme0n1.in‚ 4.33984375 disk.dm-0.in‚ 8.77734375 disk.sdb.out‚219.0205078125 disk.nvme1n1.in‚4.4375# proc.memory.rssMB‚ 3057.48046875 network.recv‚765815 disk.sdb2.out‚219.0205078125 disk.sda1.out‚0 disk.md0p1.in‚ 0.0078125 disk.sda2.in‚ 0.0078125) proc.memory.percent‚18.66137981414795) disk./.usagePercent‚83.38669321546861! disk.nvme0n1.out‚ 14963.296875 disk.md0p1.out‚ 209.734375# memory_percent‚77.8857946395874) proc.memory.availableMB‚ 3623.19140625 cpu‚25.541274014476038 disk.sda1.in‚0 disk.sda.in‚ 0.0078125 proc.cpu.threads‚17# disk./.usageGB‚2982.31205368042 disk.rbd0.out‚ 0.4951171875 disk.sdb2.in‚0 disk.rbd2.in‚0 disk.rbd2.out‚0.578125 disk.sdb.in‚0 disk.sda.out‚219.0205078125 disk.rbd0.in‚0 disk.dm-0.out‚ 31678.859375 disk.md0.in‚ 0.0078125 disk.md0.out‚ 209.734375^Õxª2Ù - _wandbruntime‚2265  _runtime‚2265cYà&û&Ú õM GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 TEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 2516376064 TEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 3142752768 FDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 ^JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚276628836624896 XFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 3008799445504 S?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚268562522655897 _runtime‚2265.027618611 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚19715 K>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚14213693 ?6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚3681 C6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚41556206 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚19709 ZFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚265557537658784 VGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 3217031168 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 ?6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚3681 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚23980 K;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 38610163200 N>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 38610163200 LDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚330 RCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 1515847168 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚19709 RBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 35913877504 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚25427968 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 \JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 3007499367457 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 G:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚41575921 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚18042179 UCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 3008799445504 N>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 36985723397 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚3 LDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚986 BDeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 KBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚1534 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 NADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚23514027 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚8617133 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 963120128 XFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 3004984997113 M>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 2541804032 UEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 35913877504 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚557 SFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚96413184 ^KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚55679935207424 ZFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚276628811196928 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 QBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 3142752768 UEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 34283657216 RBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 40290484224 I@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚2147 PDeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚41575921 REDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚18042185 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚67315200 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚25427968 P@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 40290484224 F=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚2147 NADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚18042185 PADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 2541804032 _step‚100 TDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 37073453056 D;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚3681 F=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚1534 NADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚23533736 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚6 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 ^JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚265557563086752 FDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 ^JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚276628836624896 XFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 3008799445504 S?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚268562522655897 _runtime‚2265.027618611 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚19715 K>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚14213693 ?6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚3681 C6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚41556206 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚19709 ZFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚265557537658784 VGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 3217031168 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 ?6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚3681 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚23980 K;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 38610163200 N>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 38610163200 LDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚330 RCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 1515847168 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚19709 RBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 35913877504 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚25427968 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 \JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 3007499367457 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 G:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚41575921 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚18042179 UCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 3008799445504 N>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 36985723397 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚3 LDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚986 BDeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 KBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚1534 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 NADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚23514027 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚8617133 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 963120128 XFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 3004984997113 M>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 2541804032 UEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 35913877504 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚557 SFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚96413184 ^KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚55679935207424 ZFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚276628811196928 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 QBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 3142752768 UEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 34283657216 RBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 40290484224 I@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚2147 PDeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚41575921 REDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚18042185 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚67315200 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚25427968 P@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 40290484224 F=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚2147 NADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚18042185 PADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 2541804032 _step‚100 TDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 37073453056 D;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚3681 F=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚1534 NADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚23533736 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚6 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 ^JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚265557563086752 FDeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚290236444630528 TDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 35913877504 [IDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 3126792340480 H?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚1309 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 O?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 40290484224 _step‚103 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚23980 UADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚290236444630528 P@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 38610163200 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚19689 TGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚24435820 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 100620800 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 YEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚278750660641279 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚6 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚25427968 D;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚3681 M=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 38610163200 VGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 3142752768 VFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 37073453056 IDeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚23681 `LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚287112194080768 YJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 2516362752 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚19689 VBDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated‚290238986421248 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 963120128 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 _runtime‚2349.826080666 VGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 2516362752 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 VGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 3140738484 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 PCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚18722795 `MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚57842321132032 BDeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 RCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 2539798104 IDeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚2684 M@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚14751686 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 \HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚287112168652800 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0é°ø¢ö'î ðO _runtime‚2349.826080666 VGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 2516362752 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 VGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 3140738484 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 PCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚18722795 `MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚57842321132032 BDeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 RCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 2539798104 IDeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚2684 M@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚14751686 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 \HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚287112168652800 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚19683 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚23681 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 P@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 36985723397 F=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚3681 % _timestamp‚1.7884875507245798e+09 TGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚18722795 \HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚275627805023088 E8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚43138926 WGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 35913877504 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 YEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚287112168652800 YJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 2514370136 RBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 40290484224 XIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 3217031168 H?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚2147 WEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 3124275977728 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚25427968 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚587 ^LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 3122830190223 MDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚2147 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 YIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 37073453056 KBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚1534 R>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚290236444630528 TDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 35913877504 [IDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 3126792340480 H?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚1309 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 O?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 40290484224 _step‚103 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚23980 UADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚290236444630528 P@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 38610163200 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚19689 TGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚24435820 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 100620800 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 YEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚278750660641279 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚6 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚25427968 D;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚3681 M=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 38610163200 VGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 3142752768 VFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 37073453056 IDeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚23681 `LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚287112194080768 YJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 2516362752 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚19689 VBDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated‚290238986421248 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 963120128 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1gç©¢B2ï - _wandbruntime‚2351  _runtime‚2351o´t«ü&ð öM FDeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 2541804032 K>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚43190873 S?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚278955989639405 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 PADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 2541804032 XFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 3122668955821 trainer/global_step‚1299 ZFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚287326114723328 REDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚24454188 OBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚14763188 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 _runtime‚2351.414614511 QBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 3142752768 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚19709 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚6 bODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚57886480555008 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚5814863 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚3 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚25427968 ZFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚275833320683584 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚23681 XFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 3126632073728 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚8948325 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 \JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 3125183326165 XIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 3217031168 TEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 3142752768 ]KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 3903765146112 PADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 2539798312 S?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚290452746797056 UEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 34283657216 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 aODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 3903838752768 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 WHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 2516376064 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 D;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚3681 \JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 3129148449792 REDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚18736685 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚5814860 M=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 40290484224 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚73606656 ?6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚3681 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚19715 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚8947731 RBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 40290484224 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 F=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚2147 F=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚1534 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚23980 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚6 FDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 38610163200 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚25427968 ^KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚57886451457024 I@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚2147 NADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚24454188 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚18736679 K;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 38610163200 [GDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚287326140151296 WCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚287326114723328 NADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚18736679 LDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚330 RCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 1515847168 YIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 37073453056 TEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 2516376064 ^JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚287326140151296 [HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚61790319307776 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 KBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚2147 ^JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚275833346111552 G:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚43190873 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚25427968 RBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 35913877504 LDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚986 T@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚290455288601088 YGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 3129148449792 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 VGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 3217031168 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1309 EDeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚14762591 WDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚61790216603136 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 BDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 36985723397 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚594 I@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚1534 NADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚24434479 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚597 SDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 3217031168 B9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚3681 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 WHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 2514370344 P@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 40290484224A I ñ úM PADe‹î$Æô&viceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 2541804032 XFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 3122668955821 trainer/global_step‚1299 ZFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚287326114723328 REDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚24454188 OBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚14763188 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 _runtime‚2351.414614511 QBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 3142752768 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚19709 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚6 bODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚57886480555008 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚5814863 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚3 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚25427968 ZFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚275833320683584 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚23681 XFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 3126632073728 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚8948325 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 \JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 3125183326165 XIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 3217031168 TEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 3142752768 ]KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 3903765146112 PADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 2539798312 S?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚290452746797056 UEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 34283657216 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 aODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 3903838752768 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 WHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 2516376064 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 D;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚3681 \JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 3129148449792 REDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚18736685 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚5814860 M=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 40290484224 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚73606656 ?6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚3681 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚19715 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚8947731 RBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 40290484224 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 F=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚2147 F=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚1534 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚23980 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚6 FDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 38610163200 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚25427968 ^KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚57886451457024 I@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚2147 NADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚24454188 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚18736679 K;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 38610163200 [GDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚287326140151296 WCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚287326114723328 NADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚18736679 LDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚330 RCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 1515847168 YIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 37073453056 TEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 2516376064 ^JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚287326140151296 [HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚61790319307776 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 KBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚2147 ^JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚275833346111552 G:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚43190873 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚25427968 RBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 35913877504 LDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚986 T@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚290455288601088 YGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 3129148449792 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 VGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 3217031168 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1309 EDeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚14762591 WDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚61790216603136 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 BDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 36985723397 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚594 I@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚1534 NADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚24434479 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚597 SDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 3217031168 B9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚3681 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 WHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 2514370344 P@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 40290484224 FDeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 2541804032 K>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚43190873 S?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚278955989639405 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0hQ¤2ò - _wandbruntime‚2351  _runtime‚2351^ cZó Ô 2loss/SRqqt4/assignment_loss‚1.5505176828272882 %loss/total_loss‚45.18218412166701 % _timestamp‚1.7884875523155417e+09 _runtime‚2351.417089983 0loss/FRt1/assignment_loss‚3.8618048337789683 0loss/FBt1/assignment_loss‚0.2996334981871769 .loss/FBt3/detection_loss‚2.812167857374464 0loss/FRt3/assignment_loss‚3.5124957370758056 .loss/FRt4/detection_loss‚2.478119479355059 1loss/SRqqt2/detection_loss‚1.1587948415315512 .loss/FBt1/detection_loss‚3.713278941810131 epoch‚2 .loss/FRt2/assignment_loss‚1.37302490696311 1loss/FBt4/assignment_loss‚0.21284577230315854 /loss/FRt3/detection_loss‚2.7509302846590677 1loss/SRqqt1/detection_loss‚2.3986791128462013 /loss/FBt4/detection_loss‚2.3440077943461284 2loss/SRqqt2/assignment_loss‚1.1239876017277834 2loss/SRqqt3/assignment_loss‚1.9359494234762482 .loss/FRt1/detection_loss‚2.202258031337689 2loss/SRqqt1/assignment_loss‚2.1221861514661993 1loss/SRqqt4/detection_loss‚1.8053315950016822 trainer/global_step‚1299 _step‚105 /loss/FRt4/assignment_loss‚2.438842783476177 1loss/FBt2/assignment_loss‚0.14592428089961637 0loss/FBt3/assignment_loss‚0.2315782540816664 /loss/FRt2/detection_loss‚1.6665240993412833 1loss/SRqqt3/detection_loss‚2.2533652500526324 /loss/FBt2/detection_loss‚0.7899359077479208¨3ŠV^ô Ø /loss/FRt4/assignment_loss‚2.438842783476177 1loss/FBt2/assignment_loss‚0.14592428089961637 0loss/FBt3/assignment_loss‚0.2315782540816664 /loss/FRt2/detection_loss‚1.6665240993412833 1loss/SRqqt3/detection_loss‚2.2533652500526324 /loss/FBt2/detection_loss‚0.7899359077479208 2loss/SRqqt4/assignment_loss‚1.5505176828272882 %loss/total_loss‚45.18218412166701 % _timestamp‚1.7884875523155417e+09 _runtime‚2351.417089983 0loss/FRt1/assignment_loss‚3.8618048337789683 0loss/FBt1/assignment_loss‚0.2996334981871769 .loss/FBt3/detection_loss‚2.812167857374464 0loss/FRt3/assignment_loss‚3.5124957370758056 .loss/FRt4/detection_loss‚2.478119479355059 1loss/SRqqt2/detection_loss‚1.1587948415315512 .loss/FBt1/detection_loss‚3.713278941810131 epoch‚2 .loss/FRt2/assignment_loss‚1.37302490696311 1loss/FBt4/assignment_loss‚0.21284577230315854 /loss/FRt3/detection_loss‚2.7509302846590677 1loss/SRqqt1/detection_loss‚2.3986791128462013 /loss/FBt4/detection_loss‚2.3440077943461284 2loss/SRqqt2/assignment_loss‚1.1239876017277834 2loss/SRqqt3/assignment_loss‚1.9359494234762482 .loss/FRt1/detection_loss‚2.202258031337689 2loss/SRqqt1/assignment_loss‚2.1221861514661993 1loss/SRqqt4/detection_loss‚1.8053315950016822 trainer/global_step‚1299 _step‚105i ”Qå“õ ‚(:ˆ ƒÏèÔ¶ÇË° gpu.0.gpu‚97.0 gpu.0.memory‚17+ gpu.0.memoryAllocated‚47.54737854003906* gpu.0.memoryAllocatedBytes‚ 40842887168 gpu.0.temp‚45.0 gpu.0.powerWatts‚228.538' gpu.0.enforcedPowerLimitWatts‚400.0 gpu.0.powerPercent‚57.1345 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0tN_;ö ‚(:° ƒÏèÔŸÄø´ disk.rbd0.out‚ 0.62109375% disk./.usageGB‚2982.8393211364746 disk.sdb2.in‚0 disk.sda1.out‚0 disk.sdb.in‚0 disk.nvme1n1.in‚4.4375 disk.sda.in‚ 0.0078125$ memory_percent‚77.88996696472168( proc.memory.availableMB‚ 3622.5078125 network.sent‚ 2.901451e+06 disk.sdb1.out‚0 disk.md0p1.in‚ 0.0078125 disk.rbd2.out‚ 0.783203125# disk.nvme1n1.out‚16760.81640625) disk./.usagePercent‚83.40143583421847 disk.sdb1.in‚0 disk.md0.in‚ 0.0078125 disk.md0.out‚ 218.171875 disk.sdb.out‚ 227.828125 disk.sda2.in‚ 0.0078125 disk.sda2.out‚ 227.828125 disk.sdb2.out‚ 227.828125 disk.dm-0.in‚ 8.77734375 cpu‚25.532366241118066 proc.cpu.threads‚17 disk.sda1.in‚0 disk.rbd2.in‚0 disk.sda.out‚ 227.828125 disk.rbd0.in‚0) proc.memory.percent‚18.66137981414795 network.recv‚792359 disk.nvme0n1.in‚ 4.33984375# disk.nvme0n1.out‚16150.20703125 disk.dm-0.out‚ 32911.0234375 disk.md0p1.out‚ 218.171875# proc.memory.rssMB‚ 3057.48046875tl÷ ‚(:’ ’ÏèÔ÷×ð® gpu.0.gpu‚98.0 gpu.0.memory‚17+ gpu.0.memoryAllocated‚47.54737854003906* gpu.0.memoryAllocatedBytes‚ 40842887168 gpu.0.temp‚45.0 gpu.0.powerWatts‚235.667' gpu.0.enforcedPowerLimitWatts‚400.0( gpu.0.powerPercent‚58.91674999999999 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0ã[R²Rø ‚(:Ç ’ÏèÔ’ëò¯ network.recv‚797918 disk.nvme0n1.in‚ 4.33984375! disk.sdb2.out‚229.53662109375 disk.md0.in‚ 0.0078125 disk.sda2.in‚ 0.0078125! disk.sda2.out‚229.53662109375 disk.nvme1n1.in‚4.4375 disk.sda.out‚229.53662109375$ disk./.usageGB‚2982.312038421631 disk.sdb1.out‚0 disk.sdb.out‚229.53662109375 disk.rbd0.in‚0 disk.rbd0.out‚ 0.62109375 network.sent‚ 2.977345e+06 disk.rbd2.out‚ 0.783203125# disk.nvme1n1.out‚16760.94140625) proc.memory.availableMB‚ 3622.24609375 cpu‚25.53102369966829) disk./.usagePercent‚83.38669278882648 disk.dm-0.out‚ 33067.3359375 disk.sdb2.in‚0 disk.rbd2.in‚0) proc.memory.percent‚18.66137981414795 disk.sdb1.in‚0 disk.md0p1.in‚ 0.0078125 disk.md0p1.out‚ 219.81640625 disk.sdb.in‚0 disk.sda.in‚ 0.0078125# disk.nvme0n1.out‚16306.39453125 disk.sda1.in‚0 disk.sda1.out‚0# proc.memory.rssMB‚ 3057.48046875 proc.cpu.threads‚17 disk.dm-0.in‚ 8.77734375 disk.md0.out‚ 219.81640625$ memory_percent‚77.89156436920166Sk­Z”ù ‚(:‰ ¡ÏèÔâθ® gpu.0.gpu‚97.0 gpu.0.memory‚23+ gpu.0.memoryAllocated‚47.54737854003906* gpu.0.memoryAllocatedBytes‚ 40842887168 gpu.0.temp‚44.0 gpu.0.powerWatts‚212.445' gpu.0.enforcedPowerLimitWatts‚400.0 gpu.0.powerPercent‚53.11125 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0Ù GœQú ‚(:Æ ¡ÏèÔ¹¢û®! disk.sdb2.out‚230.87548828125 disk.sdb.in‚0 disk.sda2.in‚ 0.0078125) disk./.usagePercent‚83.38543707438181 disk.sda.in‚ 0.0078125 disk.sdb.out‚230.87548828125# proc.memory.rssMB‚ 3057.48046875 disk.md0p1.out‚ 221.09765625 disk.dm-0.out‚ 33130.03125 disk.sdb1.out‚0 disk.sdb2.in‚0 disk.sda1.out‚0$ memory_percent‚77.89044380187988( proc.memory.availableMB‚ 3622.4296875% disk./.usageGB‚2982.2671279907227 disk.nvme1n1.in‚4.4375 disk.dm-0.in‚ 8.77734375 disk.md0.out‚ 221.09765625! disk.sda2.out‚230.87548828125) proc.memory.percent‚18.66137981414795 proc.cpu.threads‚17 disk.md0.in‚ 0.0078125 disk.rbd2.out‚ 0.783203125 network.sent‚ 2.981155e+06 network.recv‚802245 disk.sda.out‚230.87548828125 disk.nvme0n1.in‚ 4.33984375 disk.sdb1.in‚0 disk.md0p1.in‚ 0.0078125 disk.rbd2.in‚0# disk.nvme1n1.out‚16761.17578125 disk.rbd0.in‚0# disk.nvme0n1.out‚16368.85546875 disk.sda1.in‚0 cpu‚25.529606016194652 disk.rbd0.out‚ 0.62109375ä"†.”û ‚(:‰ °ÏèÔ¨À§° gpu.0.gpu‚68.0 gpu.0.memory‚15+ gpu.0.memoryAllocated‚47.54737854003906* gpu.0.memoryAllocatedBytes‚ 40842887168 gpu.0.temp‚42.0 gpu.0.powerWatts‚117.193' gpu.0.enforcedPowerLimitWatts‚400.0 gpu.0.powerPercent‚29.29825 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0mrGOPü ‚(:Å °ÏèÔ³­ì° disk.rbd0.in‚0 disk.sdb1.in‚0 disk.nvme1n1.in‚4.4375" disk.nvme1n1.out‚ 16761.2890625# proc.memory.rssMB‚ 3057.48046875 disk.sdb.out‚232.4873046875 network.sent‚ 2.989591e+06 disk.nvme0n1.in‚ 4.33984375# disk.nvme0n1.out‚16563.33984375 disk.rbd2.in‚0 disk.sdb.in‚0 disk.sda2.out‚232.4873046875) disk./.usagePercent‚83.38746074465776 disk.sda.out‚232.4873046875 disk.dm-0.in‚ 8.77734375 disk.md0.out‚ 222.62890625) proc.memory.percent‚18.66137981414795 disk.sdb1.out‚0 disk.sda1.in‚0) proc.memory.availableMB‚ 3622.31640625 proc.cpu.threads‚17 disk.rbd0.out‚ 0.62109375 disk.sda1.out‚0$ memory_percent‚77.89113521575928 disk.sda2.in‚ 0.0078125% disk./.usageGB‚2982.3395042419434 disk.dm-0.out‚33324.62890625 disk.rbd2.out‚ 0.783203125 network.recv‚806829 disk.sdb2.out‚232.4873046875 disk.md0.in‚ 0.0078125 disk.md0p1.in‚ 0.0078125 disk.sda.in‚ 0.0078125 disk.sdb2.in‚0 disk.md0p1.out‚ 222.62890625 cpu‚25.528350309960118Þ=òÿ”ý ‚(:‰ ¿ÏèÔ‹­® gpu.0.gpu‚97.0 gpu.0.memory‚17+ gpu.0.memoryAllocated‚47.54737854003906* gpu.0.memoryAllocatedBytes‚ 40842887168 gpu.0.temp‚45.0 gpu.0.powerWatts‚227.929' gpu.0.enforcedPowerLimitWatts‚400.0 gpu.0.powerPercent‚56.98225 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0ÄÖ7Oþ ‚(:Ä ¿ÏèÔ—Ùç¯ network.recv‚810262# proc.memory.rssMB‚ 3057.48046875 disk.sda.out‚234.1396484375 disk.sda2.in‚ 0.0078125 disk.sda1.out‚0) disk./.usagePercent‚83.39096027671667 disk.rbd2.out‚ 0.783203125 disk.sda2.out‚234.1396484375 disk.rbd0.out‚ 0.62109375 disk.dm-0.out‚33501.18359375 disk.md0.out‚ 224.23046875 disk.md0p1.out‚ 224.23046875( proc.memory.availableMB‚ 3622.5546875% disk./.usageGB‚2982.4646644592285 disk.rbd2.in‚0 disk.sdb.in‚0 disk.dm-0.in‚ 8.77734375 disk.sdb1.out‚0 disk.sdb2.in‚0$ memory_percent‚77.88968086242676 cpu‚25.526866655402966 disk.nvme1n1.in‚4.4375 disk.sdb1.in‚0 disk.sda1.in‚0 proc.cpu.threads‚17 network.sent‚ 2.994859e+06 disk.sdb.out‚234.1396484375" disk.nvme1n1.out‚ 16761.4765625 disk.rbd0.in‚0 disk.sdb2.out‚234.1396484375) proc.memory.percent‚18.66137981414795 disk.sda.in‚ 0.0078125 disk.nvme0n1.in‚ 4.33984375# disk.nvme0n1.out‚16739.70703125 disk.md0.in‚ 0.0078125 disk.md0p1.in‚ 0.0078125_ûÈ~“ÿ ‚(:ˆ ÎÏèÔ¬³¢° gpu.0.gpu‚98.0 gpu.0.memory‚23+ gpu.0.memoryAllocated‚47.54737854003906* gpu.0.memoryAllocatedBytes‚ 40842887168 gpu.0.temp‚45.0 gpu.0.powerWatts‚321.566' gpu.0.enforcedPowerLimitWatts‚400.0 gpu.0.powerPercent‚80.3915 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0ô”MAK€ ‚(:À ÎÏèÔ雹° network.recv‚814295 disk.dm-0.in‚ 8.77734375 network.sent‚ 2.998424e+06$ disk./.usageGB‚2982.589122772217 disk.md0.out‚ 225.44140625 disk.md0p1.out‚ 225.44140625 disk.sda2.out‚ 235.416015625 disk.nvme1n1.in‚4.4375) proc.memory.percent‚18.66137981414795 disk.sdb1.out‚0 disk.sda1.out‚0 disk.rbd2.in‚0# disk.nvme0n1.out‚16930.04296875 disk.dm-0.out‚33691.61328125 disk.sdb2.out‚ 235.416015625 disk.rbd2.out‚ 0.783203125 disk.sda2.in‚ 0.0078125 disk.rbd0.in‚0$ memory_percent‚77.89037227630615# proc.memory.rssMB‚ 3057.48046875 disk.nvme0n1.in‚ 4.33984375 disk.sdb2.in‚0 disk.sdb.out‚ 235.416015625 disk.sda.in‚ 0.0078125 disk.sda.out‚ 235.416015625) proc.memory.availableMB‚ 3622.44140625 proc.cpu.threads‚17 disk.md0.in‚ 0.0078125) disk./.usagePercent‚83.39444018323768 disk.md0p1.in‚ 0.0078125 disk.sdb.in‚0" disk.nvme1n1.out‚ 16761.5703125 disk.rbd0.out‚ 0.62109375 disk.sdb1.in‚0 disk.sda1.in‚0 cpu‚25.525568332013975ôºu2 - _wandbruntime‚2436  _runtime‚2436[%;Ç‚ Á & lr-AdamW/pg1‚0.0010854354304152022 & lr-AdamW/pg2‚0.0010854354304152022 trainer/global_step‚1349 % _timestamp‚1.7884876373010025e+09 _runtime‚2436.403039566 _step‚106U_Ð˃ Å % _timestamp‚1.7884876373010025e+09 _runtime‚2436.403039566 _step‚106 & lr-AdamW/pg1‚0.0010854354304152022 & lr-AdamW/pg2‚0.0010854354304152022 trainer/global_step‚1349jh#ÞÎ2„ - _wandbruntime‚2436  _runtime‚2436ä·ÿñ'… ëO E8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚44753878 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 trainer/global_step‚1349 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚19683 SDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 3142752768 ^LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 3244624968704 `MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚60048837381632 KBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚2147 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 _step‚107 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚25427968 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚583 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 R>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚301051580785152 ]IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚297809497607168 VGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 3142752768 KBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚1534 \HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚297809472179200 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚19683 cQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 4048371715584 YJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 2516362752 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚23681 QDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚15301589 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 963120128 TDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 40290484224 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚71522816 RCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 2539798104 F=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚3681 D;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚3681 WGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 35913877504 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚29097984 _MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 4048300192768 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 H?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚1534 PCDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated‚25356272 M@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚44773567 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚19689 PCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚25336589 BDeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚2684 YEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚301054122575872 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚25427968 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 577553920 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 XIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 3217031168 VBDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated‚301054122575872 NFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚986 YFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚64097137574400 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 [KDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 37073453056 YIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 37073453056 ZKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 3217031168 TEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 1515847168 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 VGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 3140738484 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚25427968 A8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚3681 PCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚19417295 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚23681 M@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚15301006 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 ^LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 3240514148931 O?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 40290484224 L?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚25336589 ZHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 3242108605952 H?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚1309 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚23980 VGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 2516362752 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 _runtime‚2436.404233482 IDeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 YEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚289144127624787 \HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚285903588047888 [IDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 3244624968704 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚23980 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚6028797 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚9272792 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚19689 TDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 35913877504 UADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚301051580785152 SGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚9272212 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 UFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 3217031168 WGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 34283657216 L?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚19417289 TGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚19417295 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚2684 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚6 YEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚297809472179200 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚3 SGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚6028794 ]JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚64097238195200 UADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚289141587826683 YJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 2514370136 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0<³ÓZ† ïO D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚19689 TDDeviceStatqŸ›'sMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 35913877504 UADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚301051580785152 SGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚9272212 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 UFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 3217031168 WGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 34283657216 L?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚19417289 TGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚19417295 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚2684 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚6 YEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚297809472179200 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚3 SGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚6028794 ]JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚64097238195200 UADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚289141587826683 YJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 2514370136 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 E8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚44753878 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 trainer/global_step‚1349 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚19683 SDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 3142752768 ^LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 3244624968704 `MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚60048837381632 KBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚2147 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 _step‚107 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚25427968 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚583 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 R>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚301051580785152 ]IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚297809497607168 VGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 3142752768 KBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚1534 \HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚297809472179200 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚19683 cQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 4048371715584 YJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 2516362752 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚23681 QDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚15301589 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 963120128 TDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 40290484224 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚71522816 RCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 2539798104 F=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚3681 D;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚3681 WGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 35913877504 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚29097984 _MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 4048300192768 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 H?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚1534 PCDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated‚25356272 M@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚44773567 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚19689 PCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚25336589 BDeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚2684 YEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚301054122575872 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚25427968 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 577553920 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 XIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 3217031168 VBDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated‚301054122575872 NFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚986 YFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚64097137574400 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 [KDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 37073453056 YIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 37073453056 ZKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 3217031168 TEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 1515847168 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 VGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 3140738484 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚25427968 A8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚3681 PCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚19417295 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚23681 M@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚15301006 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 ^LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 3240514148931 O?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 40290484224 L?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚25336589 ZHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 3242108605952 H?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚1309 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚23980 VGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 2516362752 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 _runtime‚2436.404233482 IDeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 YEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚289144127624787 \HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚285903588047888 [IDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 3244624968704 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚23980 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚6028797 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚9272792kÌ_'–2‡ - _wandbruntime‚2437  _runtime‚2437<þfŽü&ˆ öM WHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 2514370344 TDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 37073453056 TEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 3140738484 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚19715 K>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚15311968 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚9279391 bODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚60092996804608 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 I@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚2147 C6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚44786110 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚3 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 100607488 WDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚64144238835712 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 ZFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚286109103708384 M=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 40290484224 WGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 37073453056 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚25427968 RBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 35913877504 ]KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 4051271129088 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 577553920 UEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 35913877504 F=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚1534 [GDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚298023443677696 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 F=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚2147 I@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚1534 ?6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚3681 UCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 3244464701952 REDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚19431185 LDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚986 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 KBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚1534 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1309 G:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚44805825 EDeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚23980 K;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 38610163200 WCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚301270424755712 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚2684 ^KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚60092967706624 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚6 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 P@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 40290484224 K>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚44805825 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚9278808 aODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 4051342638592 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚71509504 WCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚298023418249728 REDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚25374640 \JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 3242867284873 \JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 3246981078016 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 WCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚289351996421225 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 T@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚301270424755712 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚19709 QBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 3142752768 PADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 2541804032 NADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚19431179 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚583 RCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 1515847168 UEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 34283657216 G:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚44786110 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚6033160 NADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚19431185 YGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 3246981078016 XFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 3244464701952 TEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 3142752768 OBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚15312554 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 D;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚3681 P8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 XIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 3217031168 J=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚25354931 N>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 38610163200 ?6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚3681 WHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 2516376064 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚23980 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 BDeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚6033163 RBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 40290484224 KBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚2147 trainer/global_step‚1349 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 YIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 37073453056 % _timestamp‚1.7884876388292925e+09 ^JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚298023443677696 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚25427968 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚23681 PADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 2539798312 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 B9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚3681 _step‚108 N>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 36985723397 M>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 2541804032 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 _runtime‚2437.931099017 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 963120128 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 NADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚25374640 FDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 38610163200 ?6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚3681 WHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 2516376064 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚23980 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 BDeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚6033163 RBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 40290484224 KBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚2147 trainer/global_step‚1349 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 YIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 37073453056 % _timestamp‚1.7884876388292925e+09 ^JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚298023443677696 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚25427968 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚23681 PADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 2539798312 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 B9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚3681 _step‚108 N>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 36985723397 M>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 2541804032 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 _runtime‚2437.931099017 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 963120128 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 NADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚25374640 FDeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚15311968 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚9279391 bODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚60092996804608 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 I@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚2147 C6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚44786110 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚3 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 100607488 WDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚64144238835712 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 ZFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚286109103708384 M=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 40290484224 WGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 37073453056 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚25427968 RBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 35913877504 ]KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 4051271129088 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 577553920 UEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 35913877504 F=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚1534 [GDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚298023443677696 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 F=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚2147 I@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚1534 ?6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚3681 UCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 3244464701952 REDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚19431185 LDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚986 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 KBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚1534 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1309 G:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚44805825 EDeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚23980 K;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 38610163200 WCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚301270424755712 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚2684 ^KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚60092967706624 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚6 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 P@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 40290484224 K>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚44805825 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚9278808 aODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 4051342638592 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚71509504 WCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚298023418249728 REDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚25374640 \JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 3242867284873 \JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 3246981078016 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 WCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚289351996421225 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 T@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚301270424755712 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚19709 QBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 3142752768 PADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 2541804032 NADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚19431179 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚583 RCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 1515847168 UEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 34283657216 G:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚44786110 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚6033160 NADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚19431185 YGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 3246981078016 XFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 3244464701952 TEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 3142752768 OBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚15312554 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 D;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚3681 P8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0lN«kè2Š - _wandbruntime‚2437  _runtime‚2437¡ \‹ Ö 1loss/FBt4/assignment_loss‚0.12262037806649459 /loss/FRt2/detection_loss‚1.6747364594290655 2loss/SRqqt3/assignment_loss‚1.9660154928870757 /loss/FRt4/detection_loss‚2.7533029776654745 /loss/FBt1/detection_loss‚2.4520917887037452 0loss/SRqqt3/detection_loss‚2.653612406089388 .loss/FBt3/detection_loss‚3.090710441271464 % _timestamp‚1.7884876388317373e+09 0loss/FRt2/assignment_loss‚1.1453616432845592 1loss/FBt2/assignment_loss‚0.11192391323633817 1loss/FBt3/assignment_loss‚0.11492248732212743 1loss/SRqqt1/detection_loss‚2.2916406160447655 0loss/SRqqt4/detection_loss‚1.790811792612076 /loss/FBt4/detection_loss‚2.5552665501832963 0loss/FRt3/assignment_loss‚3.8102766880622276 /loss/FRt1/detection_loss‚2.1246060964607056 1loss/SRqqt2/detection_loss‚0.9798231227605632 &loss/total_loss‚44.307046100182035 /loss/FRt4/assignment_loss‚2.976899634066381 1loss/SRqqt1/assignment_loss‚1.883612838096735 /loss/FBt2/detection_loss‚0.6175888122623843 _step‚109 0loss/FRt1/assignment_loss‚3.2856175791649593 .loss/FRt3/detection_loss‚3.003222066622514 trainer/global_step‚1349 epoch‚2 2loss/SRqqt2/assignment_loss‚0.9805833379963808 2loss/SRqqt4/assignment_loss‚1.6106722383499146 _runtime‚2437.932883895 /loss/FBt1/assignment_loss‚0.311126739543397hBxŒ Ú .loss/FRt3/detection_loss‚3.003222066622514 trainer/global_step‚1349 epoch‚2 2loss/SRqqt2/assignment_loss‚0.9805833379963808 2loss/SRqqt4/assignment_loss‚1.6106722383499146 _runtime‚2437.932883895 /loss/FBt1/assignment_loss‚0.311126739543397 1loss/FBt4/assignment_loss‚0.12262037806649459 /loss/FRt2/detection_loss‚1.6747364594290655 2loss/SRqqt3/assignment_loss‚1.9660154928870757 /loss/FRt4/detection_loss‚2.7533029776654745 /loss/FBt1/detection_loss‚2.4520917887037452 0loss/SRqqt3/detection_loss‚2.653612406089388 .loss/FBt3/detection_loss‚3.090710441271464 % _timestamp‚1.7884876388317373e+09 0loss/FRt2/assignment_loss‚1.1453616432845592 1loss/FBt2/assignment_loss‚0.11192391323633817 1loss/FBt3/assignment_loss‚0.11492248732212743 1loss/SRqqt1/detection_loss‚2.2916406160447655 0loss/SRqqt4/detection_loss‚1.790811792612076 /loss/FBt4/detection_loss‚2.5552665501832963 0loss/FRt3/assignment_loss‚3.8102766880622276 /loss/FRt1/detection_loss‚2.1246060964607056 1loss/SRqqt2/detection_loss‚0.9798231227605632 &loss/total_loss‚44.3+lžè07046100182035 /loss/FRt4/assignment_loss‚2.976899634066381 1loss/SRqqt1/assignment_loss‚1.883612838096735 /loss/FBt2/detection_loss‚0.6175888122623843 _step‚109 0loss/FRt1/assignment_loss‚3.2856175791649593m8‰õ©“ ‚(:ˆ ÝÏèÔ¹‹Ä¯ gpu.0.gpu‚86.0 gpu.0.memory‚19+ gpu.0.memoryAllocated‚47.54737854003906* gpu.0.memoryAllocatedBytes‚ 40842887168 gpu.0.temp‚45.0 gpu.0.powerWatts‚232.986' gpu.0.enforcedPowerLimitWatts‚400.0 gpu.0.powerPercent‚58.2465 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0ç8?Ž ‚(:´ ÝÏèÔ˜‘Ò¯# disk.nvme0n1.out‚17140.80859375 disk.sdb2.out‚ 236.8671875$ memory_percent‚77.89013385772705 cpu‚25.524114614490763 disk.md0p1.out‚ 226.828125 disk.sdb1.out‚0) disk./.usagePercent‚83.36917027662794$ disk./.usageGB‚2981.685348510742 disk.sda2.in‚ 0.0078125 disk.rbd0.out‚ 0.62109375 proc.cpu.threads‚17 disk.rbd2.in‚0 disk.rbd2.out‚ 0.783203125" disk.nvme1n1.out‚ 16761.9296875 disk.dm-0.out‚33902.73828125 network.sent‚ 3.004474e+06 disk.md0.out‚ 226.828125 disk.nvme1n1.in‚4.4375 network.recv‚818768 disk.sda1.in‚0 disk.md0.in‚ 0.0078125 disk.md0p1.in‚ 0.0078125 disk.rbd0.in‚0# proc.memory.rssMB‚ 3057.48046875) proc.memory.percent‚18.66137981414795 disk.sdb1.in‚0 disk.sdb.in‚0 disk.sdb.out‚ 236.8671875 disk.sda.in‚ 0.0078125 disk.sda.out‚ 236.8671875) proc.memory.availableMB‚ 3622.48046875 disk.sda1.out‚0 disk.sda2.out‚ 236.8671875 disk.nvme0n1.in‚ 4.33984375 disk.dm-0.in‚ 8.77734375 disk.sdb2.in‚0Ë> _ ‚(:’ ìÏèÔ¸ˆÍ® gpu.0.gpu‚97.0 gpu.0.memory‚17+ gpu.0.memoryAllocated‚47.54737854003906* gpu.0.memoryAllocatedBytes‚ 40842887168 gpu.0.temp‚45.0 gpu.0.powerWatts‚239.491' gpu.0.enforcedPowerLimitWatts‚400.0( gpu.0.powerPercent‚59.87275000000001 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0Z]Æu< ‚(:± ìÏèÔÍÿ‘¯) disk./.usagePercent‚83.37195294325045 disk.sda1.in‚0 disk.md0p1.out‚ 228.66796875 disk.sda.out‚ 238.765625 disk.rbd0.out‚ 0.62109375# disk.nvme0n1.out‚17333.00390625# proc.memory.rssMB‚ 3057.48046875 proc.cpu.threads‚17 disk.sdb1.out‚0 disk.nvme1n1.in‚4.4375 disk.sda2.out‚ 238.765625 cpu‚25.522872629234577 network.recv‚824725 disk.sdb2.out‚ 238.765625 disk.sda.in‚ 0.0078125$ disk./.usageGB‚2981.784870147705 disk.sdb.out‚ 238.765625! disk.nvme1n1.out‚ 16762.046875 disk.rbd0.in‚0' proc.memory.availableMB‚ 3622.234375) proc.memory.percent‚18.66137981414795 network.sent‚ 3.091277e+06 disk.dm-0.out‚34095.05078125 disk.sdb2.in‚0 disk.md0p1.in‚ 0.0078125 disk.rbd2.in‚0 disk.rbd2.out‚ 0.783203125 disk.sdb.in‚0 disk.sdb1.in‚0 disk.sda1.out‚0 disk.md0.in‚ 0.0078125 disk.md0.out‚ 228.66796875 disk.dm-0.in‚ 8.77734375 disk.sda2.in‚ 0.0078125 disk.nvme0n1.in‚ 4.33984375$ memory_percent‚77.89163589477539ûÇ¡W”‘ ‚(:‰ ûÏèÔŒŠß¯ gpu.0.gpu‚98.0 gpu.0.memory‚18+ gpu.0.memoryAllocated‚47.54737854003906* gpu.0.memoryAllocatedBytes‚ 40842887168 gpu.0.temp‚43.0 gpu.0.powerWatts‚123.435' gpu.0.enforcedPowerLimitWatts‚400.0 gpu.0.powerPercent‚30.85875 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0\úL’ ‚(:Á ûÏèÔ­Šõ¯ disk.md0p1.in‚ 0.0078125 disk.rbd2.out‚ 0.783203125 disk.rbd0.out‚ 0.62109375' proc.memory.availableMB‚ 3622.609375 disk.md0.out‚ 229.9453125" disk.nvme1n1.out‚ 16762.2890625 disk.sda.out‚240.09033203125! disk.nvme0n1.out‚ 17465.890625 disk.dm-0.in‚ 8.77734375 disk.sda1.out‚0 disk.sda2.in‚ 0.0078125 cpu‚25.521448688365293 network.sent‚ 3.096759e+06$ disk./.usageGB‚2981.257614135742 disk.dm-0.out‚ 34228.1796875 disk.sdb1.in‚0 disk.sda1.in‚0 disk.sdb.out‚240.09033203125! disk.sda2.out‚240.09033203125 disk.rbd0.in‚0$ memory_percent‚77.88934707641602 proc.cpu.threads‚17 disk.sdb2.in‚0! disk.sdb2.out‚240.09033203125# proc.memory.rssMB‚ 3057.48046875 network.recv‚828450 disk.md0.in‚ 0.0078125 disk.md0p1.out‚ 229.9453125 disk.rbd2.in‚0) disk./.usagePercent‚83.35721064448218 disk.sdb.in‚0 disk.nvme1n1.in‚4.4375 disk.sda.in‚ 0.0078125) proc.memory.percent‚18.66137981414795 disk.nvme0n1.in‚ 4.33984375 disk.sdb1.out‚0|ÜÏ4”“ ‚(:‰ ŠÐèÔËê–° gpu.0.gpu‚98.0 gpu.0.memory‚17+ gpu.0.memoryAllocated‚47.54737854003906* gpu.0.memoryAllocatedBytes‚ 40842887168 gpu.0.temp‚41.0 gpu.0.powerWatts‚117.525' gpu.0.enforcedPowerLimitWatts‚400.0 gpu.0.powerPercent‚29.38125 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚03Ïà¢P” ‚(:Å ŠÐèÔþÚá± disk.sda2.out‚241.5009765625 disk.sda.in‚ 0.0078125) disk./.usagePercent‚83.35594863706612 disk.sdb2.in‚0 disk.rbd2.in‚0 disk.sdb.in‚0 network.recv‚832685 disk.rbd0.in‚0 disk.rbd0.out‚ 0.62109375 disk.sda1.out‚0 network.sent‚ 3.104719e+06% disk./.usageGB‚2981.2124786376953 disk.md0.in‚ 0.0078125 proc.cpu.threads‚17 disk.dm-0.in‚ 8.77734375 disk.dm-0.out‚34272.42578125 disk.sdb.out‚241.5009765625 disk.sda2.in‚ 0.0078125 disk.nvme1n1.in‚4.4375) proc.memory.availableMB‚ 1808.55078125 disk.sda.out‚241.5009765625 disk.sdb2.out‚241.5009765625 disk.nvme0n1.in‚ 4.33984375 disk.sdb1.in‚0 disk.sdb1.out‚0 disk.rbd2.out‚ 0.783203125$ memory_percent‚88.96148204803467# proc.memory.rssMB‚ 3057.48046875) proc.memory.percent‚18.66137981414795 disk.sda1.in‚0" disk.nvme0n1.out‚ 17509.9921875 disk.md0.out‚ 231.29296875 disk.md0p1.in‚ 0.0078125# disk.nvme1n1.out‚16762.43359375 disk.md0p1.out‚ 231.29296875 cpu‚25.519561852316826ÖjÎœž• ‚(:“ ™ÐèÔñ¨ž® gpu.0.gpu‚98.0 gpu.0.memory‚18+ gpu.0.memoryAllocated‚47.54737854003906* gpu.0.memoryAllocatedBytes‚ 40842887168 gpu.0.temp‚45.0 gpu.0.powerWatts‚237.233' gpu.0.enforcedPowerLimitWatts‚400.0) gpu.0.powerPercent‚59.308249999999994 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0uVÀçF– ‚(:» ™ÐèÔ칯 disk.sda1.out‚0 disk.md0p1.out‚232.75) disk./.usagePercent‚83.35495722742004 disk.nvme1n1.in‚4.4375# disk.nvme1n1.out‚16834.51953125 disk.rbd0.out‚ 0.62109375 disk.dm-0.in‚ 8.77734375 disk.sda.out‚243.00732421875 disk.dm-0.out‚34345.83984375 disk.md0.out‚232.75 disk.rbd2.out‚ 0.783203125 cpu‚25.518021162081368 proc.cpu.threads‚17% disk./.usageGB‚2981.1770210266113 disk.md0.in‚ 0.0078125 disk.rbd2.in‚0 disk.sda.in‚ 0.0078125 disk.nvme0n1.in‚ 4.33984375 disk.sdb1.in‚0 disk.sdb2.in‚0" disk.nvme0n1.out‚ 17511.3203125) proc.memory.percent‚18.66137981414795 network.sent‚ 3.108533e+06 network.recv‚837029 disk.sdb.out‚243.00732421875 disk.sda1.in‚0 disk.md0p1.in‚ 0.0078125# memory_percent‚88.9615535736084( proc.memory.availableMB‚ 1808.5390625# proc.memory.rssMB‚ 3057.48046875 disk.sdb.in‚0 disk.sda2.in‚ 0.0078125! disk.sda2.out‚243.00732421875 disk.rbd0.in‚0 disk.sdb1.out‚0! disk.sdb2.out‚243.00732421875 ƒ "2— - _wandbruntime‚2518  _runtime‚2518â¯$Ûǘ Á & lr-AdamW/pg1‚0.0010783622393455699 & lr-AdamW/pg2‚0.0010783622393455699 trainer/global_step‚1399 % _timestamp‚1.7884877191433263e+09 _runtime‚2518.244703467 _step‚110j g"Ë™ Å & lr-AdamW/pg1‚0.0010783622393455699 & lr-AdamW/pg2‚0.0010783622393455699 trainer/global_step‚1399 % _timestamp‚1.7884877191433263e+09 _runtime‚2518.244703467 _step‚110n‹wì\2š - _wandbruntime‚2518  _runtime‚2518 ò'› ìO H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚23681 YEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚311869258730496 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 _runtime‚2518.248729796 IDeviceStatsMonitor.on_train_batch_start/allocation.all.current‚19689 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚25427968 TDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 40290484224 D;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚3681 TGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚20111795 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚6 P@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 36985723397 ZHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 3355683737503 MDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚1534 L?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚20111789 M=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 38610163200 TDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 35913877504 H?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚1309 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚3 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚71522816 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 VGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 3140738484 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚19683 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚2684 SGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚6247094 WGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 34283657216 \HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚308506775705600 ^LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 3362457596928 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 D>DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚19683 _step‚111 PCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚26257041 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚25427968 KBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚2147 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 E8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚46368830 \HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚296179371072688 H?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚1534 TGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚26276724 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚6247097 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 963120128 XIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 3217031168 SDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 3142752768 dQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚62255382729216 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚6 O@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 2541790720 P@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 38610163200 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚23980 SGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚9602191 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 WEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 3359941234176 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 UADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚299535054810191 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 VFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 37073453056 A8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚3681 YJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 2516362752 ZHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 3359941234176 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚23681 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚573 ]JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚66451596664320 cQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 4196213935104 PCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚20111795 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 RCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 2539798104 H?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚2147 M@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚15849285 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚29097984 WGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 35913877504 `MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚62255353631232 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 YEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚299537594608295 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 L?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚26257041 UADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚311866716939776 % _timestamp‚1.7884877191472435e+09 UFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 3217031168 RCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 2541790720 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 VBDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated‚311869258730496 TEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 1515847168 RBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 40290484224 KBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚1534 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 577553920 [KDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 37073453056 MDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚2147 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚19689 R>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚311866716939776 VGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 3142752768 YFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚66451496043520 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 YIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 37073453056 G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚2684 A8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚3681 YJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 2514370136 trainer/global_step‚1399 YEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚308506775705600 NFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚330 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 _MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 4196142412288 ^LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 3358198107639 F=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚3681 QDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚15849861 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚23980 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚576 `LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚296179396500656 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚9602764 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 100620800 BDeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚311866716939776 VGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 3142752768 YFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚66451496043520 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 YIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 37073453056 G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚2684 A8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚3681 YJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 2514370136 trainer/global_step‚1399 YEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚308506775705600 NFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚330 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 _MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 4196142412288 ^LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 3358198107639 F=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚3681 QDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚15849861 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚23980 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚576 `LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚296179396500656 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚9602764 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 100620800 BDeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚23681 YEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚311869258730496 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 _runtime‚2518.248729796 IDeviceStatsMonitor.on_train_batch_start/allocation.all.current‚19689 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚25427968 TDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 40290484224 D;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚3681 TGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚20111795 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚6 P@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 36985723397 ZHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 3355683737503 MDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚1534 L?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚20111789 M=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 38610163200 TDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 35913877504 H?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚1309 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚3 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚71522816 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 VGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 3140738484 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚19683 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚2684 SGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚6247094 WGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 34283657216 \HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚308506775705600 ^LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 3362457596928 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 D>DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚19683 _step‚111 PCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚26257041 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚25427968 KBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚2147 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 E8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚46368830 \HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚296179371072688 H?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚1534 TGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚26276724 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚6247097 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 963120128 XIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 3217031168 SDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 3142752768 dQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚62255382729216 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚6 O@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 2541790720 P@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 38610163200 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚23980 SGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚9602191 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 WEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 3359941234176 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 UADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚299535054810191 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 VFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 37073453056 A8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚3681 YJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 2516362752 ZHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 3359941234176 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚23681 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚573 ]JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚66451596664320o ½æT2 - _wandbruntime‚2519  _runtime‚2519Z='Á0ž õM C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚23980 K>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚15860230 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 \JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 3360551243581 NADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚20125685 NADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚26295092 \JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 3364813706240 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 ZFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚296384886733184 [GDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚308720747204096 UCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 3362297330176 PADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 2541804032 C6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚46401062 OBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚15860804 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 REDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚26295092 N>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 36985723397 J=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚26275383 NADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚20125679 ^KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚62299483956224 P@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 40290484224 WGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 37073453056 ?6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚3681 _runtime‚2519.759555512 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚6 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚23681 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚25427968 T@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚312085560910336 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚2684 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚6251463 F=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚2147 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚6 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚9609341 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚571 SDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 3217031168 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚3 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 963120128 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚25427968 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚9608770 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚23980 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 B9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚3A†÷Ë681 KBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚2147 WCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚308720721776128 WCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚312085560910336 N>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 38610163200 PDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 XIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 3217031168 F=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚1534 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚6251460 XFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 3362297330176 TEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 3142752768 NADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚26275383 LDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚986 TEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 3140738484 M=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 40290484224 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 UEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 35913877504 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 100607488 UEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 34283657216 S?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚312083019106304 S?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚299742923606421 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 XFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 3358036873237 G:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚46420777 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚19709 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 577553920 M>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 2541804032 YGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 3364813706240 bODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚62299513054208 ]KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 4199111006720 I@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚2147 WDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚66498594962944 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 WCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚299745463404733 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚19715 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚20125679 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚71509504 WHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 2514370344 YIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 37073453056 I@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚1534 RBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 35913877504 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚29097984 $ _timestamp‚1.788487720657429e+09 _step‚112 TEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 2516376064 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚25427968 FDeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚46420777 RCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 1515847168 D;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚3681 KBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚1534 ZFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚308720721776128 RBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 40290484224 REDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚20125685 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚574 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1309 ?6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚3681 LDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚330 K;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 38610163200 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚19709v3Àÿ&Ÿ ùM QEDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚6251460 XFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 3362297330176 TEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 3142752768 NADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚26275383 LDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚986 TEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 3140738484 M=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 40290484224 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 UEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 35913877504 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 100607488 UEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 34283657216 S?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚312083019106304 S?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚299742923606421 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 XFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 3358036873237 G:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚46420777 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚19709 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 577553920 M>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 2541804032 YGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 3364813706240 bODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚62299513054208 ]KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 4199111006720 I@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚2147 WDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚66498594962944 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 WCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚299745463404733 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚19715 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚20125679 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚71509504 WHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 2514370344 YIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 37073453056 I@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚1534 RBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 35913877504 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚29097984 $ _timestamp‚1.788487720657429e+09 _step‚112 TEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 2516376064 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚25427968 FDeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚46420777 RCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 1515847168 D;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚3681 KBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚1534 ZFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚308720721776128 RBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 40290484224 REDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚20125685 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚574 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1309 ?6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚3681 LDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚330 K;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 38610163200 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚19709 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚23980 K>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚15860230 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 \JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 3360551243581 NADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚20125685 NADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚26295092 \JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 3364813706240 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 ZFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚296384886733184 [GDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚308720747204096 UCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 3362297330176 PADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 2541804032 C6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚46401062 OBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚15860804 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 REDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚26295092 N>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 36985723397 J=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚26275383 NADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚20125679 ^KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚62299483956224 P@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 40290484224 WGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 37073453056 ?6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚3681 _runtime‚2519.759555512 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚6 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚23681 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚25427968 T@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚312085560910336 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚2684 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚6251463 F=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚2147 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚6 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚9609341 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚571 SDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 3217031168 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚3 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 963120128 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚25427968 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚9608770 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚23980 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 B9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚3681 KBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚2147 WCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚308720721776128 WCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚312085560910336 N>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 38610163200 PDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 XIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 3217031168 F=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚1534pä+X2  - _wandbruntime‚2519  _runtime‚2519ÐúFˆ`¡ Ú 0loss/SRqqt3/detection_loss‚2.557098271416836 /loss/FBt2/detection_loss‚0.8187382753861889 /loss/FRt1/assignment_loss‚3.279590588922684 0loss/FRt2/assignment_loss‚1.3997870429907695 0loss/FRt4/assignment_loss‚2.8352536755272104 /loss/FRt1/detection_loss‚2.2892643274882665 2loss/SRqqt3/assignment_loss‚1.9911394494234538 2loss/SRqqt4/assignment_loss‚1.5485526072606444 1loss/FBt2/assignment_loss‚0.12621114193953026 1loss/FBt3/assignment_loss‚0.27572295885980375 1loss/SRqqt4/detection_loss‚1.8520185092091561 1loss/FBt1/assignment_loss‚0.11875879711962105 1loss/SRqqt2/detection_loss‚1.0570026437618902 _runtime‚2519.761565171 /loss/FRt4/detection_loss‚2.4078701408226277 _step‚113 epoch‚2 2loss/SRqqt1/assignment_loss‚1.8860008617242177 1loss/FBt4/assignment_loss‚0.31277478839895895 .loss/FRt2/detection_loss‚1.654969298529743 /loss/FRt3/detection_loss‚2.8873840499280106 1loss/SRqqt2/assignment_loss‚0.886747860750516 1loss/SRqqt1/detection_loss‚2.2552419884999595 /loss/FBt1/detection_loss‚2.9124259246247157 /loss/FBt3/detection_loss‚2.5551185952989677 /loss/FBt4/detection_loss‚2.7818579574426017 &loss/total_loss‚44.161086322760426 0loss/FRt3/assignment_loss‚3.4715565674340545 trainer/global_step‚1399 % _timestamp‚1.7884877206603835e+09æR“Æd¢ Þ /loss/FBt4/detection_loss‚2.7818579574426017 &loss/total_loss‚44.161086322760426 0loss/FRt3/assignment_loss‚3.4715565674340545 trainer/global_step‚1399 % _timestamp‚1.7884877206603835e+09 0loss/SRqqt3/detection_loss‚2.557098271416836 /loss/FBt2/detection_loss‚0.8187382753861889 /loss/FRt1/assignment_loss‚3.279590588922684 0loss/FRt2/assignment_loss‚1.3997870429907695 0loss/FRt4/assignment_loss‚2.8352536755272104 /loss/FRt1/detection_loss‚2.2892643274882665 2loss/SRqqt3/assignment_loss‚1.9911394494234538 2loss/SRqqt4/assignment_loss‚1.5485526072606444 1loss/FBt2/assignment_loss‚0.12621114193953026 1loss/FBt3/assignment_loss‚0.27572295885980375 1loss/SRqqt4/detection_loss‚1.8520185092091561 1loss/FBt1/assignment_loss‚0.11875879711962105 1loss/SRqqt2/detection_loss‚1.0570026437618902 _runtime‚2519.761565171 /loss/FRt4/detection_loss‚2.4078701408226277 _step‚113 epoch‚2 2loss/SRqqt1/assignment_loss‚1.8860008617242177 1loss/FBt4/assignment_loss‚0.31277478839895895 .loss/FRt2/detection_loss‚1.654969298529743 /loss/FRt3/detection_loss‚2.8873840499280106 1loss/SRqqt2/assignment_loss‚0.886747860750516 1loss/SRqqt1/detection_loss‚2.2552419884999595 /loss/FBt1/detection_loss‚2.9124259246247157 /loss/FBt3/detection_loss‚2.5551185952989677qé=Ä6“£ ‚(:ˆ ¨ÐèÔõÛȯ gpu.0.gpu‚94.0 gpu.0.memory‚20+ gpu.0.memoryAllocated‚47.54737854003906* gpu.0.memoryAllocatedBytes‚ 40842887168 gpu.0.temp‚47.0 gpu.0.powerWatts‚316.618' gpu.0.enforcedPowerLimitWatts‚400.0 gpu.0.powerPercent‚79.1545 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0r8óMO¤ ‚(:Ä ¨ÐèÔ®öï° disk.rbd0.out‚ 0.62109375" disk.nvme0n1.out‚ 17511.8046875 disk.dm-0.out‚ 34374.4140625# proc.memory.rssMB‚ 3057.48046875 network.sent‚ 3.126575e+06) disk./.usagePercent‚83.35853472832795% disk./.usageGB‚2981.3049697875977! disk.nvme1n1.out‚ 16862.609375 disk.dm-0.in‚ 8.77734375$ memory_percent‚88.95933628082275 cpu‚25.516893318436598 proc.cpu.threads‚17 disk.sda.out‚244.51123046875! disk.sdb2.out‚244.51123046875 disk.sdb.in‚0 disk.sda2.in‚ 0.0078125 disk.sda1.in‚0 disk.sda1.out‚0 disk.md0p1.out‚ 234.1953125! disk.sda2.out‚244.51123046875) proc.memory.percent‚18.66137981414795 disk.rbd2.out‚ 0.783203125 disk.sdb.out‚244.51123046875 disk.sda.in‚ 0.0078125 disk.sdb1.out‚0) proc.memory.availableMB‚ 1808.90234375 network.recv‚840802 disk.sdb2.in‚0 disk.md0.in‚ 0.0078125 disk.nvme0n1.in‚ 4.33984375 disk.nvme1n1.in‚4.4375 disk.rbd0.in‚0 disk.sdb1.in‚0 disk.md0.out‚ 234.1953125 disk.md0p1.in‚ 0.0078125 disk.rbd2.in‚0NBŸAP¥ ‚(:Å ·ÐèÔ½›Á± disk.sdb.out‚245.75244140625! disk.sda2.out‚245.75244140625 disk.rbd0.out‚ 0.62109375 disk.dm-0.in‚ 8.77734375 disk.rbd2.out‚ 0.783203125# disk.nvme1n1.out‚16966.23828125# disk.nvme0n1.out‚17591.82421875) proc.memory.availableMB‚ 1809.09765625 network.sent‚ 3.199932e+06 disk.dm-0.out‚ 34558.0625 disk.sdb.in‚0 disk.sda2.in‚ 0.0078125 network.recv‚845288 disk.sdb1.out‚0 disk.sda1.in‚0 disk.sda.in‚ 0.0078125# proc.memory.rssMB‚ 3057.48046875) proc.memory.percent‚18.66137981414795$ disk./.usageGB‚2981.584976196289 disk.nvme0n1.in‚ 4.33984375 disk.sdb1.in‚0 disk.sdb2.in‚0 disk.rbd2.in‚0$ memory_percent‚88.95814418792725) disk./.usagePercent‚83.36636382470675! disk.sdb2.out‚245.75244140625 disk.sda1.out‚0 disk.md0.in‚ 0.0078125 disk.md0.out‚ 235.38671875 disk.md0p1.in‚ 0.0078125 disk.md0p1.out‚ 235.38671875 proc.cpu.threads‚17 disk.nvme1n1.in‚4.4375 disk.sda.out‚245.75244140625 disk.rbd0.in‚0 cpu‚25.515609267629554¦Œ.’¦ ‚(:‡ ·ÐèÔ“ûÔ± gpu.0.gpu‚34.0 gpu.0.memory‚8+ gpu.0.memoryAllocated‚47.54737854003906* gpu.0.memoryAllocatedBytes‚ 40842887168 gpu.0.temp‚47.0 gpu.0.powerWatts‚321.906' gpu.0.enforcedPowerLimitWatts‚400.0 gpu.0.powerPercent‚80.4765 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0þìtÑ“§ ‚(:ˆ ÆÐèÔ´¡ž® gpu.0.gpu‚44.0 gpu.0.memory‚10+ gpu.0.memoryAllocated‚47.54737854003906* gpu.0.memoryAllocatedBytes‚ 40842887168 gpu.0.temp‚48.0 gpu.0.powerWatts‚320.958' gpu.0.enforcedPowerLimitWatts‚400.0 gpu.0.powerPercent‚80.2395 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0½¨šQK¨ ‚(:À ÆÐèÔºÏñ® network.sent‚ 3.205839e+06 disk.md0p1.in‚ 0.0078125 disk.sda.out‚247.00537109375# proc.memory.rssMB‚ 3057.48046875 cpu‚25.514213150907363$ disk./.usageGB‚2981.834648132324 disk.md0p1.out‚ 236.5859375 disk.sdb.in‚0 disk.nvme0n1.in‚ 4.33984375 disk.sdb1.out‚0( proc.memory.availableMB‚ 1809.1015625) disk./.usagePercent‚83.37334475653411! disk.sda2.out‚247.00537109375 disk.md0.in‚ 0.0078125 disk.sda.in‚ 0.0078125 disk.rbd0.in‚0! disk.sdb2.out‚247.00537109375 disk.sda1.out‚0 disk.md0.out‚ 236.5859375 disk.nvme1n1.out‚ 17095.4375 disk.dm-0.out‚ 34766.7578125 disk.sda1.in‚0) proc.memory.percent‚18.66137981414795 disk.rbd2.in‚0 disk.sdb.out‚247.00537109375 disk.sda2.in‚ 0.0078125 disk.nvme1n1.in‚4.4375 disk.rbd0.out‚ 0.62109375 disk.sdb1.in‚0 disk.sdb2.in‚0$ memory_percent‚88.95812034606934 disk.rbd2.out‚ 0.783203125 disk.dm-0.in‚ 8.77734375 proc.cpu.threads‚17 network.recv‚849841" disk.nvme0n1.out‚ 17671.32031255ÂY’© ‚(:‡ ÕÐèÔÌÖ® gpu.0.gpu‚97.0 gpu.0.memory‚23+ gpu.0.memoryAllocated‚47.54737854003906* gpu.0.memoryAllocatedBytes‚ 40842887168 gpu.0.temp‚46.0 gpu.0.powerWatts‚321.29' gpu.0.enforcedPowerLimitWatts‚400.0 gpu.0.powerPercent‚80.3225 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0èqDª ‚(:¹ ÕÐèÔœùÖ¯% disk./.usageGB‚2981.8396034240723 disk.dm-0.out‚34927.58984375 disk.md0.in‚ 0.0078125 disk.nvme1n1.in‚4.4375) proc.memory.percent‚18.66137981414795 proc.cpu.threads‚17 network.sent‚ 3.211612e+06 disk.sdb1.out‚0 disk.md0p1.out‚ 238.390625 disk.rbd0.in‚0 network.recv‚853771 disk.sdb2.in‚0# disk.nvme1n1.out‚17250.84765625 disk.sda.in‚ 0.0078125# proc.memory.rssMB‚ 3057.48046875 cpu‚25.51305288536683) disk./.usagePercent‚83.37348330856533 disk.sda1.out‚0 disk.md0p1.in‚ 0.0078125 disk.sda2.out‚ 248.888671875 disk.rbd0.out‚ 0.62109375 disk.rbd2.in‚0 disk.sdb.out‚ 248.888671875 disk.nvme0n1.in‚ 4.33984375" disk.nvme0n1.out‚ 17676.7421875 disk.dm-0.in‚ 8.77734375 disk.sda1.in‚0 disk.md0.out‚ 238.390625 disk.sda.out‚ 248.888671875& proc.memory.availableMB‚ 1809.09375 disk.sdb1.in‚0 disk.sdb2.out‚ 248.888671875 disk.rbd2.out‚ 0.783203125 disk.sda2.in‚ 0.0078125 disk.sdb.in‚0$ memory_percent‚88.95816802978516Ví“3”« ‚(:‰ äÐèÔù° gpu.0.gpu‚98.0 gpu.0.memory‚18+ gpu.0.memoryAllocated‚47.54737854003906* gpu.0.memoryAllocatedBytes‚ 40842887168 gpu.0.temp‚42.0 gpu.0.powerWatts‚118.751' gpu.0.enforcedPowerLimitWatts‚400.0 gpu.0.powerPercent‚29.68775 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0ªº¬mJ¬ ‚(:¿ äÐèÔÐÀÍ° disk.sdb1.in‚0 disk.nvme1n1.in‚4.4375 disk.sdb.out‚250.5048828125 cpu‚25.511518938257577 disk.sda1.out‚0 disk.rbd2.in‚0 disk.sda.out‚250.5048828125 proc.cpu.threads‚17 network.sent‚ 3.217865e+06 disk.dm-0.out‚35045.03515625 disk.sda2.in‚ 0.0078125 disk.rbd0.in‚0 disk.md0p1.out‚ 239.9453125# disk.nvme1n1.out‚17307.01953125 disk.sda.in‚ 0.0078125 disk.rbd0.out‚ 0.62109375$ memory_percent‚88.95659446716309( proc.memory.availableMB‚ 1809.3515625) proc.memory.percent‚18.66137981414795 disk.dm-0.in‚ 8.77734375 disk.sdb1.out‚0 disk.sdb2.in‚0 disk.md0.in‚ 0.0078125 disk.md0p1.in‚ 0.0078125 disk.rbd2.out‚ 0.783203125# proc.memory.rssMB‚ 3057.48046875! disk.nvme0n1.out‚ 17738.015625 disk.sda1.in‚0 disk.md0.out‚ 239.9453125 disk.sdb.in‚0 disk.sdb2.out‚250.5048828125 network.recv‚858358( disk./.usagePercent‚83.3622416084613$ disk./.usageGB‚2981.437545776367 disk.sda2.out‚250.5048828125 disk.nvme0n1.in‚ 4.33984375¯qC­ ‚(:’ óÐèÔîÙš° gpu.0.gpu‚98.0 gpu.0.memory‚18+ gpu.0.memoryAllocated‚47.54737854003906* gpu.0.memoryAllocatedBytes‚ 40842887168 gpu.0.temp‚45.0 gpu.0.powerWatts‚227.652' gpu.0.enforcedPowerLimitWatts‚400.0( gpu.0.powerPercent‚56.91299999999999 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0˜:¥ÈP® ‚(:Å óÐèÔŽâÕ±! disk.sdb2.out‚251.90869140625 disk.md0p1.out‚ 241.28515625 proc.cpu.threads‚17 disk.md0.out‚ 241.28515625 disk.md0p1.in‚ 0.0078125 disk.sdb.in‚0 disk.rbd0.out‚ 0.62109375$ memory_percent‚88.95666599273682 network.sent‚ 3.225377e+06 disk.sda1.in‚0 disk.md0.in‚ 0.0078125 cpu‚25.510381306104474% disk./.usageGB‚2981.5620079040527 disk.rbd2.in‚0 disk.rbd0.in‚0) proc.memory.percent‚18.66137981414795 disk.rbd2.out‚ 0.783203125 disk.sdb.out‚251.90869140625# disk.nvme1n1.out‚17350.62890625 disk.nvme0n1.out‚ 17860.625 disk.sdb2.in‚0 disk.sda.in‚ 0.0078125 disk.sda.out‚251.90869140625 disk.dm-0.out‚35211.25390625 disk.sdb1.out‚0) proc.memory.availableMB‚ 1809.33984375# proc.memory.rssMB‚ 3057.48046875 network.recv‚862036 disk.sda2.in‚ 0.0078125 disk.nvme0n1.in‚ 4.33984375) disk./.usagePercent‚83.36572162164285 disk.sda1.out‚0! disk.sda2.out‚251.90869140625 disk.nvme1n1.in‚4.4375 disk.dm-0.in‚ 8.77734375 disk.sdb1.in‚07dÉû2¯ - _wandbruntime‚2602  _runtime‚2602ߊùÇ° Á _runtime‚2602.241408927 _step‚114 & lr-AdamW/pg1‚0.0010708634882171668 & lr-AdamW/pg2‚0.0010708634882171668 trainer/global_step‚1449 % _timestamp‚1.7884878031400516e+09\h6/˱ Å _runtime‚2602.241408927 _step‚114 & lr-AdamW/pg1‚0.0010708634882171668 & lr-AdamW/pg2‚0.0010708634882171668 trainer/global_step‚1449 % _timestamp‚1.7884878031400516e+09rÈL:2² - _wandbruntime‚2602  _runtime‚2602¸•c¹m³ ëO ZKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 3217031168 XIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 3217031168 YEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚319204079232000 ZHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 3477773862400 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚9932422 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 TDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 40290484224 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚69425664 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚25427968 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 _runtime‚2602.242543232 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 RCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 2539798104 % _timestamp‚1.7884878031410494e+09 IDeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚2684 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚23681 UHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚98523648 cQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 4343466167808 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 D>DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 WGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 34283657216 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 TDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 35913877504 YJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 2516362752 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚6 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚2684 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 RCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.currenCÏFׄt‚ 2541790720 NFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚330 NFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚986 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 577553920 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 YJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 2514370136 VGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 3140738484 RBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 40290484224 SDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 3142752768 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 `MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚64461869880832 F=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚3681 UFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 3217031168 O@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 2541790720 PCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚27177493 trainer/global_step‚1449 VBDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated‚322684394885120 R>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚322681853094400 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚19689 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚6465397 YFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚68805266622976 KBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚1534 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚19689 PCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚20806295 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚3 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 ^LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 3480290225152 `LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚306455179525456 L?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚20806289 \HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚319204079232000 M@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚48003471 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 YEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚309931061591803 KBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚2147 ]IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚319204104659968 SGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚6465394 TEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 1515847168 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 ZHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 3473367696211 E8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚47983782 P@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 38610163200 VGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 3142752768 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 MDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚2147 L?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚27177493 QDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚16397819 ]JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚68805365146624 MDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚1534 ^LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 3475882066347 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚23980 BDeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚23681 YIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 37073453056 IDeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚23681 YIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 37073453056 IDeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚2684 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚23681 UHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚98523648 cQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 4343466167808 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 D>DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 WGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 34283657216 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 TDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 35913877504 YJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 2516362752 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚6 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚2684 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 RCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 2541790720 NFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚330 NFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚986 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 577553920 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 YJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 2514370136 VGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 3140738484 RBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 40290484224 SDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 3142752768 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 `MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚64461869880832 F=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚3681 UFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 3217031168 O@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 2541790720 PCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚27177493 trainer/global_step‚1449 VBDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated‚322684394885120 R>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚322681853094400 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚19689 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚6465397 YFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚68805266622976 KBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚1534 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚19689 PCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚20806295 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚3 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 ^LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 3480290225152 `LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚306455179525456 L?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚20806289 \HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚319204079232000 M@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚48003471 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 YEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚309931061591803 KBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚2147 ]IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚319204104659968 SGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚6465394 TEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 1515847168 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 ZHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 3473367696211 E8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚47983782 P@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 38610163200 VGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 3142752768 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 MDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚2147 L?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚27177493 QDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚16397819 ]JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚68805365146624sã†02µ - _wandbruntime‚2603  _runtime‚2603r5n+û&¶ õM G:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚48016014 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚6469763 SFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚98510336 ^KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚64506000205824 trainer/global_step‚1449 REDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚20820185 KBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚2147 E8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 [HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚68852455120384 WGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 37073453056 WCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚322900697064960 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 RBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 40290484224 M=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 40290484224 I@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚2147 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚23980 NADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚27215544 WHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 2516376064 XFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 3475720831945 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚566 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚23980 K>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚16408167 ]KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 4346356404224 PDeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 K;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 38610163200 NADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚20820179 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚6469760 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚29097984 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚9938970 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚25427968 WHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 2514370344 F=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚2147 S?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚310136390589929 NADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚20820185 UCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 3480129958400 LDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚330 WDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚68852356610048 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 B9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚3681 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 WCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚310138930388241 I@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚1534 FDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 36985723397 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 ?6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚3681 ^JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚319418050730496 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚6 PADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 2539798312 YGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 3482646334464 RCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 1515847168 ^JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚306660695185952 ZFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚306660669757984 M>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 2541804032 UEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 34283657216 D;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚3681 T@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚322900697064960 G:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚48035729 PADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 2541804032 UEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 35913877504 K>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚48035729 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚19709 OBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚16408733 P@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 40290484224 TEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 2516376064 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚69412352 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 YIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 37073453056 F=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚1534 _runtime‚2603.823593586 N>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 38610163200 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚23681 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 963120128 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 \JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 3478235202289 XIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 3217031168 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0jô’[7· ùM ?6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚3681 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 N>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 36985723397 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 ?6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚3681 ^JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚319418050730496 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚6 PADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 2539798312 YGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 3482646334464 RCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 1515847168 ^JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚306660695185952 ZFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚306660669757984 M>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 2541804032 UEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 34283657216 D;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚3681 T@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚322900697064960 G:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚48035729 PADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 2541804032 UEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 35913877504 K>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚48035729 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚19709 OBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚16408733 P@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 40290484224 TEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 2516376064 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚69412352 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 YIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 37073453056 F=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚1534 _runtime‚2603.823593586 N>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 38610163200 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚23681 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 963120128 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 \JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 3478235202289 XIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 3217031168 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 G:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚48016014 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚6469763 SFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚98510336 ^KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚64506000205824 trainer/global_step‚1449 REDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚20820185 KBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚2147 E8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 [HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚68852455120384 WGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 37073453056 WCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚322900697064960 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 RBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 40290484224 M=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 40290484224 I@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚2147 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚23980 NADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚27215544 WHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 2516376064 XFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 3475720831945 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚566 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚23980 K>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚16408167 ]KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 4346356404224 PDeviceStatsMonik&pÈ tor.on_train_batch_end/segment.small_pool.freed‚0 K;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 38610163200 NADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚20820179 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚6469760 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚29097984 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚9938970 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚25427968 WHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 2514370344 F=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚2147 S?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚310136390589929 NADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚20820185 UCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 3480129958400 LDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚330 WDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚68852356610048 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 B9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚3681 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 WCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚310138930388241 I@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚1534 FDeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 963120128 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 TEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 1515847168 SDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 3142752768 UADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚333496989249024 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚6 YEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚320324528575311 VGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 3140738484 R>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚333496989249024 IDeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚2684 VGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 2516362752 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚23980 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚10262464 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚23681 TDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 40290484224 [KDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 37073453056 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 L?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚28097945 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚23681 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚19683 M@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚16946158 A8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚3681 YIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 37073453056 TDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 35913877504 WEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 3595606490624 \HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚329901382758400 ZHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 3595606490624 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚2684 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 O?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 40290484224 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚25427968 YEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚329901382758400 WGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 34283657216 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 D;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚3681 MDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚1534 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚6 `LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚329901408186368 M@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚49618423 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 XIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 3217031168 % _timestamp‚1.7884878884544878e+09 L?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚21500789 TGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚21500795 cQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 4491176652288 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 BDeviceStatsMonitor.on_train_batch_start/allocation.all.current‚19689 PCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚21500789 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚6683697 IDeviceStatsMonitor.on_train_batch_start/allocation.all.current‚19689 PCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚21500789 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚6683697 IDeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 963120128 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 TEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 1515847168 SDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 3142752768 UADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚333496989249024 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚6 YEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚320324528575311 VGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 3140738484 R>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚333496989249024 IDeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚2684 VGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 2516362752 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚23980 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚10262464 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚23681 TDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 40290484224 [KDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 37073453056 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 L?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚28097945 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚23681 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚19683 M@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚16946158 A8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚3681 YIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 37073453056 TDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 35913877504 WEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 3595606490624 \HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚329901382758400 ZHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 3595606490624 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚2684 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 O?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 40290484224wR8á“2Í - _wandbruntime‚2689  _runtime‚2689ý“kEý&Î ÷M G:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚49630966 LDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚330 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚3 LDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚986 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 KBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚2147 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚23980 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚6 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 F=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚1534 \JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 3600478962688 XFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 3597962586624 NADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚21514679 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚6688060 NADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚28116287 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 RBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 40290484224 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 UCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 3597962586624 K>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚16956955 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚580 bODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚66712545553408 S?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚320529857573437 \JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 3595919160997 % _timestamp‚1.7884878900080297e+09 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚19715 C6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚49630966 [GDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚330115354256896 PADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 2541804032 XFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 3593404790653 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚21514679 WCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚330115328828928 RBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 35913877504 UEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 34283657216 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1309 WCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚320532397371749 _step‚120 UEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 35913877504 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 N>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 38610163200 J=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚28116287 REDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚21514685 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚23681 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚73606656 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 PADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 2539798312 XIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 3217031168 VGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 3217031168 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚25427968 [HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚71206669733376 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 577553920 WGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 37073453056 F=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚2147 ?6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚3681 T@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚333715833219584 M>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 2541804032 WDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚71206567028736 D;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚3681 K;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 38610163200 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚2684 ^JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚316936478210752 OBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚16957538 EDeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚49650681 SDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 3217031168 I@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚1534 YIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 37073453056 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚6688063 S?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚333713291415552 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚6 FDeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 _runtime‚ 2689.11013229 QBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 3142752768 ZFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚330115328828928 REDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚28135996 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚29097984 TEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 3140738484 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 ^JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚330115354256896 BDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 36985723397 M=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 40290484224 NADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚21514685 NADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚28135996 WHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 2516376064 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚583 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚10269475 aODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 4494124179968 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚23980 REDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚10268895 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 FDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 UCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 3597962586624 K>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚16956955 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚580 bODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚66712545553408 S?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚320529857573437 \JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 3595919160997 % _timestamp‚1.7884878900080297e+09 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚19715 C6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚49630966 [GDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚330115354256896 PADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 2541804032 XFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 3593404790653 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚21514679 WCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚330115328828928 RBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 35913877504 UEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 34283657216 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1309 WCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚320532397371749 _step‚120 UEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 35913877504 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 N>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 38610163200 J=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚28116287 REDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚21514685 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚23681 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚73606656 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 PADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 2539798312 XIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 3217031168 VGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 3217031168 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚25427968 [HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚71206669733376 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 577553920 WGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 37073453056 F=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚2147 ?6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚3681 T@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚333715833219584 M>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 2541804032 WDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚71206567028736 D;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚3681 K;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 38610163200 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚2684 ^JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚316936478210752 OBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚16957538 EDeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚49650681 SDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 3217031168 I@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚1534 YIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 37073453056 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚6688063 S?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚333713291415552 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚6 FDeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 _runtime‚ 2689.11013229 QBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 3142752768 ZFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚330115328828928 REDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚28135996 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚29097984 TEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 3140738484 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 ^JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚330115354256896 BDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 36985723397 M=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 40290484224 NADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚21514685 NADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚28135996 WHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 2516376064 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚583 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚10269475 aODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 4494124179968 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚23980 REDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚10268895 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 FDeviceStatsMonitor.on_train_batch_start/allocation.all.current‚19689 TEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 1515847168 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 A8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚3681 F=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚3681 P@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 38610163200 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚25427968 NFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚986 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 H?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚1534 M@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚17493740 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚6901997 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚10592314 M=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 38610163200 UADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚344312125403648 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 L?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚22195289 D>DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 E8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚51213686 TGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚22195295 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚2684 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚571 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 577553920 RCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 2539798104 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚25427968 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 TDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 40290484224 WEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 3713439118848 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚568 YEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚330717995558819 O?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 40290484224 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚6 P@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 36985723397 `LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚327006745575056 IDeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚2684 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 RBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 40290484224 PCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚22195295 ]IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚340598711712768 SGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚6901994 MDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚2147 _step‚123 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚19689 SDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 3142752768 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚19683 YIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 37073453056 % _timestamp‚1.7884879752773356e+09 `MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚68874902380032 UADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚330715455760715 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚29097984 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚25427968 MDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚1534 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚23681 ^LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 3715955481600 _MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 4638985122304 WGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 35913877504 PCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚22195289 [KDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 37073453056 `LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚340598711712768 cQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 4639056645120 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 H?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚2147 QDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚17494311 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 963120128 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 VGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 3140738484 M@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚51233375 ]JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚73513988123136 VFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 37073453056 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 _runtime‚2774.379236192 \HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚327006720147088 WGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 34283657216 UFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 3217031168 YEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚340598686284800 RCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 2541790720 VGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 3142752768 TGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚29038080 D;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚3681 ZKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 3217031168 R>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚344312125403648 H?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚1309 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚3 YFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚73513887502336 dQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚68874931478016 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 NFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚330 YEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚344314667194368 YJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 2516362752 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚6 PCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚29018397 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 100620800 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 ^LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 3711249983763 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚71522816 BDeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚2684 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 RBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 40290484224 PCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚22195295 ]IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚340598711712768 SGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚6901994 MDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚2147 _step‚123 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚19689 SDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 3142752768 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚19683 YIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 37073453056 % _timestamp‚1.7884879752773356e+09 `MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚68874902380032 UADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚330715455760715 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚29097984 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚25427968 MDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚1534 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚23681 ^LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 3715955481600 _MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 4638985122304 WGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 35913877504 PCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚22195289 [KDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 37073453056 `LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚340598711712768 cQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 4639056645120 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 H?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚2147 QDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚17494311 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 963120128 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 VGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 3140738484 M@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚51233375 ]JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚73513988123136 VFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 37073453056 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 _runtime‚2774.379236192 \HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚327006720147088 WGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 34283657216 UFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 3217031168 YEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚340598686284800 RCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 2541790720 VGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 3142752768 TGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚29038080 D;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚3681 ZKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 3217031168 R>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚344312125403648 H?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚1309 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚3 YFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚73513887502336 dQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚68874931478016 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 NFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚330 YEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚344314667194368 YJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 2516362752 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚6 PCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚29018397 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 100620800 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 ^LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 3711249983763 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚71522816 BDeviceStatsMonitor.on_train_batch_start/allocation.all.current‚19689 TEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 1515847168 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 A8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚3681 F=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚3681 P@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 38610163200 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚25427968 NFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚986 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 H?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚1534 M@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚17493740 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚6901997 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚10592314 M=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 38610163200 UADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚344312125403648 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 L?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚22195289 D>DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0{âë_I”ã ‚(:‰ §ÒèÔ”‹° gpu.0.gpu‚67.0 gpu.0.memory‚11+ gpu.0.memoryAllocated‚47.54737854003906* gpu.0.memoryAllocatedBytes‚ 40842887168 gpu.0.temp‚45.0 gpu.0.powerWatts‚230.437' gpu.0.enforcedPowerLimitWatts‚400.0 gpu.0.powerPercent‚57.60925 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0r>ðDeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚51265633 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 REDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚29056448 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚23681 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚577 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚3 REDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚10598308 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 963120128 M=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 40290484224 N>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 38610163200 OBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚17505245 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 100607488 ]KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 4641929251328 WHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 2514370344 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 RBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 35913877504 UCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 3715795214848 WDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚73560961956352 BDeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚17504668 LDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚986 F=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚2147 S?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚330923324556945 TDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 37073453056 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚2684 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚29097984 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚25427968 _step‚124 \JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 3718311590912 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚23980 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 577553920 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 I@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚2147 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 PADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 2539798312 C6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚51245918 NADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚29036739 [HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚73561062563840 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 N>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 36985723397 F=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚1534 WHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 2516376064 ?6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚3681 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 SDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 3217031168 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚6 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚25427968 LDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚330 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 ?6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚3681 % _timestamp‚1.7884879768782477e+09 EDeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 2541804032 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚574 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 PDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 38610163200 OBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚17505245 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 100607488 ]KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 4641929251328 WHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 2514370344 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 RBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 35913877504 UCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 3715795214848 WDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚73560961956352 BDeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚17504668 LDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚986 F=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚2147 S?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚330923324556945 TDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 37073453056 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚2684 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚29097984 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚25427968 _step‚124 \JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 3718311590912 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚23980 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 577553920 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 I@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚2147 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 PADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 2539798312 C6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚51245918 NADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚29036739 [HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚73561062563840 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 N>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 36985723397 F=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚1534 WHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 2516376064 ?6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚3681 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 SDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 3217031168 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚6 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚25427968 LDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚330 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 ?6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚3681 % _timestamp‚1.7884879768782477e+09 EDeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 2541804032 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚574 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 PDeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚51265633 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 REDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚29056448 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚23681|£ó£¡2è - _wandbruntime‚2775  _runtime‚27759Ñ:^é Ø /loss/FRt1/detection_loss‚2.3114953702720613 %loss/total_loss‚43.80496247905076 trainer/global_step‚1549 % _timestamp‚1.7884879768825057e+09 0loss/FRt1/assignment_loss‚3.5807479833779126 0loss/FRt3/assignment_loss‚3.4567543897830264 /loss/FRt4/detection_loss‚2.3471239751323743 1loss/SRqqt2/detection_loss‚1.1671280864642088 1loss/SRqqt4/detection_loss‚1.6059214077354256 epoch‚2 2loss/SRqqt3/assignment_loss‚1.8907500450100219 1loss/SRqqt4/assignment_loss‚1.502110903279162 1loss/FBt1/assignment_loss‚0.32114549788730073 /loss/FRt2/detection_loss‚1.8119047969579696 /loss/FRt3/detection_loss‚2.7895279182514674 1loss/FBt2/assignment_loss‚0.10289469892621152 1loss/FBt4/assignment_loss‚0.24638487984553006 .loss/FBt1/detection_loss‚2.756028536707163 /loss/FBt2/detection_loss‚0.7364494817509937 /loss/FRt4/assignment_loss‚2.336646962673106 1loss/SRqqt3/detection_loss‚2.4315739589078085 0loss/FRt2/assignment_loss‚1.5411914050579072 2loss/SRqqt2/assignment_loss‚0.8938364321460478 /loss/FBt4/detection_loss‚2.2799200696103714 _runtime‚2775.983845036 _step‚125 1loss/FBt3/assignment_loss‚0.17657614224299323 1loss/SRqqt1/detection_loss‚2.4934592879798316 .loss/FBt3/detection_loss‚3.190637819468975 1loss/SRqqt1/assignment_loss‚1.834752429582893r Ø;bê Ü 0loss/FRt1/assignment_loss‚3.5807479833779126 0loss/FRt3/assignment_loss‚3.4567543897830264 /loss/FRt4/detection_loss‚2.3471239751323743 1loss/SRqqt2/detection_loss‚1.1671280864642088 1loss/SRqqt4/detection_loss‚1.6059214077354256 epoch‚2 2loss/SRqqt3/assignment_loss‚1.8907500450100219 1loss/SRqqt4/assignment_loss‚1.502110903279162 1loss/FBt1/assignment_loss‚0.32114549788730073 /loss/FRt2/detection_loss‚1.8119047969579696 /loss/FRt3/detection_loss‚2.7895279182514674 1loss/FBt2/assignment_loss‚0.10289469892621152 1loss/FBt4/assignment_loss‚0.24638487984553006 .loss/FBt1/detection_loss‚2.756028536707163 /loss/FBt2/detection_loss‚0.7364494817509937 /loss/FRt4/assignment_loss‚2.336646962673106 1loss/SRqqt3/detection_loss‚2.4315739589078085 0loss/FRt2/assignment_loss‚1.5411914050579072 2loss/SRqqt2/assignment_loss‚0.8938364321460478 /loss/FBt4/detection_loss‚2.2799200696103714 _runtime‚2775.983845036 _step‚125 1loss/FBt3/assignment_loss‚0.17657614224299323 1loss/SRqqt1/detection_loss‚2.4934592879798316 .loss/FBt3/detection_loss‚3.190637819468975 1loss/SRqqt1/assignment_loss‚1.834752429582893 /loss/FRt1/detection_loss‚2.3114953702720613 %loss/total_loss‚43.80496247905076 trainer/global_step‚1549 % _timestamp‚1.7884879768825057e+09}@'!U”ë ‚(:‰ ¶ÒèÔ¯¦¬® gpu.0.gpu‚98.0 gpu.0.memory‚17+ gpu.0.memoryAllocated‚47.54737854003906* gpu.0.memoryAllocatedBytes‚ 40842887168 gpu.0.temp‚42.0 gpu.0.powerWatts‚146.657' gpu.0.enforcedPowerLimitWatts‚400.0 gpu.0.powerPercent‚36.66425 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0ÉŠöLì ‚(:Á ¶ÒèÔÿ¸Ð° network.sent‚ 3.542355e+06# disk./.usageGB‚2982.83931350708 disk.sda1.out‚0# proc.memory.rssMB‚ 3057.48046875 disk.sda.out‚270.64599609375! disk.sdb2.out‚270.64599609375 disk.rbd2.out‚ 0.783203125 disk.sdb.out‚270.64599609375 disk.sda2.in‚ 0.0078125 disk.sda.in‚ 0.0078125 disk.nvme0n1.in‚4.59375 disk.dm-0.in‚9.109375 disk.md0p1.in‚ 0.0078125 disk.md0p1.out‚ 259.2421875# disk.nvme0n1.out‚20331.47265625 disk.sdb2.in‚0 network.recv‚919706) disk./.usagePercent‚83.40143562089742 disk.sdb.in‚0! disk.sda2.out‚270.64599609375 disk.nvme1n1.in‚4.515625! disk.nvme1n1.out‚ 17353.046875 disk.sda1.in‚0 disk.md0.in‚ 0.0078125 disk.rbd2.in‚0 disk.rbd0.in‚0 disk.rbd0.out‚ 0.62109375 disk.sdb1.in‚0 disk.sdb1.out‚0 disk.md0.out‚ 259.2421875) proc.memory.availableMB‚ 1808.43359375) proc.memory.percent‚18.66137981414795 cpu‚25.495026429510826 proc.cpu.threads‚17 disk.dm-0.out‚37684.51953125$ memory_percent‚88.96219730377197í{]”í ‚(:‰ ÅÒèÔû‘š¯ gpu.0.gpu‚86.0 gpu.0.memory‚15+ gpu.0.memoryAllocated‚47.54737854003906* gpu.0.memoryAllocatedBytes‚ 40842887168 gpu.0.temp‚44.0 gpu.0.powerWatts‚193.227' gpu.0.enforcedPowerLimitWatts‚400.0 gpu.0.powerPercent‚48.30675 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0, #b?î ‚(:´ ÅÒèԽ粯 disk.nvme0n1.in‚4.59375 disk.md0.in‚ 0.0078125 disk.md0.out‚ 260.7265625 disk.sda2.out‚ 272.18359375) proc.memory.availableMB‚ 1808.87109375% disk./.usageGB‚2982.3120307922363 disk.sda1.out‚0 disk.rbd2.in‚0$ memory_percent‚88.95952701568604) disk./.usagePercent‚83.38669257550542 disk.sdb2.in‚0# proc.memory.rssMB‚ 3057.48046875) proc.memory.percent‚18.66137981414795 disk.md0p1.in‚ 0.0078125 cpu‚25.494015791065415 network.sent‚ 3.607986e+06# disk.nvme0n1.out‚20470.45703125 disk.dm-0.out‚ 37823.625 disk.sda1.in‚0 disk.sda.in‚ 0.0078125 disk.sdb1.in‚0 disk.sdb1.out‚0 disk.nvme1n1.in‚4.515625 proc.cpu.threads‚17 disk.dm-0.in‚9.109375 disk.sdb2.out‚ 272.18359375 disk.md0p1.out‚ 260.7265625 disk.rbd2.out‚ 0.783203125 disk.sdb.in‚0 disk.sda2.in‚ 0.0078125 disk.rbd0.in‚0 disk.rbd0.out‚ 0.62109375 disk.sdb.out‚ 272.18359375# disk.nvme1n1.out‚17353.16796875 disk.sda.out‚ 272.18359375 network.recv‚925035ú(¹¹Mï ‚(: ÔÒèÔä蟯$ memory_percent‚88.96105289459229) proc.memory.percent‚18.66137981414795) disk./.usagePercent‚83.35314133186068 disk.sdb2.in‚0 disk.rbd2.in‚0 disk.nvme1n1.in‚4.515625! disk.nvme1n1.out‚ 17353.359375 disk.md0.in‚ 0.0078125 disk.md0.out‚ 261.92578125 disk.sda.in‚ 0.0078125# proc.memory.rssMB‚ 3057.48046875 proc.cpu.threads‚17 disk.dm-0.out‚ 37990.2421875 disk.md0p1.out‚ 261.92578125 network.sent‚ 3.614193e+06 disk.dm-0.in‚9.109375 disk.sdb1.in‚0! disk.sdb2.out‚273.43310546875" disk.nvme0n1.out‚ 20636.8828125 disk.sda1.out‚0! disk.sda2.out‚273.43310546875 disk.rbd0.out‚ 0.62109375) proc.memory.availableMB‚ 1808.62109375 disk.sdb.in‚0 disk.sda2.in‚ 0.0078125 disk.sda.out‚273.43310546875 disk.nvme0n1.in‚4.59375 disk.sdb1.out‚0 disk.sda1.in‚0 disk.md0p1.in‚ 0.0078125 cpu‚25.493083368842566 network.recv‚929659$ disk./.usageGB‚2981.112075805664 disk.rbd2.out‚ 0.783203125 disk.sdb.out‚273.43310546875 disk.rbd0.in‚0ÖÇp(žð ‚(:“ ÔÒèÔ¦© ° gpu.0.gpu‚97.0 gpu.0.memory‚17+ gpu.0.memoryAllocated‚47.54737854003906* gpu.0.memoryAllocatedBytes‚ 40842887168 gpu.0.temp‚45.0 gpu.0.powerWatts‚226.093' gpu.0.enforcedPowerLimitWatts‚400.0) gpu.0.powerPercent‚56.523250000000004 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0Æ=2’ñ ‚(:‡ ãÒèÔãâ® gpu.0.gpu‚98.0 gpu.0.memory‚18+ gpu.0.memoryAllocated‚47.54737854003906* gpu.0.memoryAllocatedBytes‚ 40842887168 gpu.0.temp‚45.0 gpu.0.powerWatts‚226.37' gpu.0.enforcedPowerLimitWatts‚400.0 gpu.0.powerPercent‚56.5925 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0•-ÓÞGò ‚(:¼ ãÒèÔž’¯ disk.sda.out‚275.12158203125( proc.memory.availableMB‚ 1808.8671875 network.sent‚ 3.617379e+06 disk.sdb2.in‚0 disk.sda1.in‚0 disk.md0p1.in‚ 0.0078125 disk.sda2.in‚ 0.0078125 disk.nvme1n1.in‚4.515625$ memory_percent‚88.95955085754395 network.recv‚933062 disk.md0p1.out‚ 263.55859375 disk.sdb.out‚275.12158203125# disk.nvme1n1.out‚17353.77734375) proc.memory.percent‚18.66137981414795 cpu‚25.492159863904075( disk./.usagePercent‚83.3577761586235 disk.rbd2.out‚ 0.783203125 disk.sdb.in‚0# proc.memory.rssMB‚ 3057.48046875 proc.cpu.threads‚17% disk./.usageGB‚2981.2778396606445 disk.dm-0.in‚9.109375 disk.sdb1.out‚0 disk.dm-0.out‚38160.77734375 disk.md0.out‚ 263.55859375 disk.nvme0n1.in‚4.59375! disk.sdb2.out‚275.12158203125 disk.md0.in‚ 0.0078125! disk.sda2.out‚275.12158203125 disk.rbd0.out‚ 0.62109375 disk.nvme0n1.out‚20807 disk.sdb1.in‚0 disk.sda1.out‚0 disk.rbd2.in‚0 disk.sda.in‚ 0.0078125 disk.rbd0.in‚0ó’”ó ‚(:‰ òÒèԪё® gpu.0.gpu‚86.0 gpu.0.memory‚20+ gpu.0.memoryAllocated‚47.54737854003906* gpu.0.memoryAllocatedBytes‚ 40842887168 gpu.0.temp‚49.0 gpu.0.powerWatts‚317.567' gpu.0.enforcedPowerLimitWatts‚400.0 gpu.0.powerPercent‚79.39175 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0ºB‡HIô ‚(:¾ òÒèÔìÑó® disk.md0p1.in‚ 0.0078125# disk.nvme1n1.out‚17353.87109375 disk.sda.in‚ 0.0078125 disk.rbd0.in‚0 cpu‚25.491248537372375 proc.cpu.threads‚17 network.recv‚937088 disk.nvme0n1.in‚4.59375( proc.memory.availableMB‚ 1808.6171875) proc.memory.percent‚18.66137981414795 disk.sdb2.out‚276.5205078125 disk.md0p1.out‚ 264.9140625 disk.nvme1n1.in‚4.515625 disk.rbd0.out‚ 0.62109375# memory_percent‚88.9610767364502# proc.memory.rssMB‚ 3057.48046875 disk.dm-0.out‚ 38208.7578125 disk.sdb1.out‚0 disk.sda1.in‚0 disk.sda.out‚276.5205078125# disk.nvme0n1.out‚20854.88671875 disk.dm-0.in‚9.109375 disk.sdb1.in‚0 disk.sdb2.in‚0 disk.sda1.out‚0 disk.sdb.out‚276.5205078125 network.sent‚ 3.625356e+06% disk./.usageGB‚2981.4049644470215 disk.md0.in‚ 0.0078125 disk.rbd2.in‚0 disk.rbd2.out‚ 0.783203125 disk.sdb.in‚0 disk.sda2.in‚ 0.0078125 disk.sda2.out‚276.5205078125) disk./.usagePercent‚83.36133062085645 disk.md0.out‚ 264.9140625~ð¤b2õ - _wandbruntime‚2857  _runtime‚2857jB¡ÄÆö À & lr-AdamW/pg1‚0.0010458782287942725 & lr-AdamW/pg2‚0.0010458782287942725 trainer/global_step‚1599 % _timestamp‚1.7884880586463737e+09 _runtime‚ 2857.75139214 _step‚126\eþÌÊ÷ Ä trainer/global_step‚1599 % _timestamp‚1.7884880586463737e+09 _runtime‚ 2857.75139214 _step‚126 & lr-AdamW/pg1‚0.0010458782287942725 & lr-AdamW/pg2‚0.0010458782287942725~üè2ø - _wandbruntime‚2857  _runtime‚2857Û>XKð'ù êO QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚569 TEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 1515847168 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚69425664 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 577553920 XIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 3217031168 MDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚1534 L?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚22889789 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚6 \HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚337282503171888 A8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚3681 PCDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated‚29958532 PCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚29938849 trainer/global_step‚1599 YEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚341111462542327 ^LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 3828933942471 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚572 YEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚351295989811200 \HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚351295989811200 ^LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 3833788109824 _MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 4786541333504 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 YJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 2514370136 G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚2684 R>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚355127261558272 UHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚98523648 YFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚75867959963136 BDeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 L?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚29938849 YJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 2516362752 VFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 37073453056 F=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚3681 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚2684 RBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 40290484224 # _timestamp‚1.78848805864711e+09 VGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 2516362752 `LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚351296015239168 `LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚337282528599856 ZHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 3826419572335 H?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚1534 SGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚7120294 TDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 40290484224 [KDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 37073453056 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚19689 ZHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 3831271747072 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 O@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 2541790720 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚23681 UADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚341108922744223 H?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚2147 _runtime‚ 2857.75148926 TGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚29958532 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚10921908 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚29097984 UADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚355127261558272 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚25427968 SDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 3142752768 M@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚18042202 MDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚2147 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚23980 ]JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚75868058486784 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 IDeviceStatsMonitor.on_train_batch_start/allocation.all.current‚19689 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚7120297 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚10922477 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 ZKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 3217031168 PCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚22889795 VBDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated‚355129803348992 UFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 3217031168 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚6 ]IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚351296015239168 H?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚1309 RCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 2539798104 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚25427968 WGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 34283657216 P@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 38610163200 M@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚52848327 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 NFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚986 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 YIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 37073453056 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚23681 YEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚355129803348992 E8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚52828638 QDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚18042774 A8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚3681 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚23980 [IDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 3833788109824 PCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚22889789 cQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 4786610759168 P@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 36985723397 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 VGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 3140738484 D;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚3681 KBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚1534 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚19683 TDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 35913877504 IDeviceStatsMonitor.on_train_batch_start/allocation.all.current‚19689 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚7120297 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚10922477 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 ZKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 3217031168 PCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚22889795 VBDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated‚355129803348992 UFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 3217031168 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚6 ]IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚351296015239168 H?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚1309 RCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 2539798104 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚25427968 WGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 34283657216 P@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 38610163200 M@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚52848327 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 NFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚986 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 YIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 37073453056 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚23681 YEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚355129803348992 E8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚52828638 QDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚18042774 A8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚3681 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚23980 [IDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 3833788109824 PCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚22889789 cQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 4786610759168 P@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 36985723397 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 VGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 3140738484 D;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚3681 KBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚1534 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚19683 TDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 35913877504 IDeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚2684 R>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚355127261558272 UHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚98523648 YFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚75867959963136 BDeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0è1¨2û - _wandbruntime‚2859  _runtime‚2859·¢¿ýý&ü ÷M REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚25427968 WCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚341319331338765 PADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 2541804032 FDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚19709 \JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 3836144219136 K>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚52880585 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚29097984 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 D;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚3681 G:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚52880585 ^JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚351509961309696 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 REDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚10928430 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 NADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚22903679 M>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 2541804032 NADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚22903685 bODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚71125578052608 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 XFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 3828772708069 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 XFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 3833627843072 REDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚29976900 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚69412352 N>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 36985723397 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 F=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚2147 F=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚1534 EDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 38610163200 WHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 2516376064 REDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚22903685 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 577553920 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 WCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚351509935881728 UCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 3833627843072 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚22903679 J=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚29957191 [GDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚351509961309696 TEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 3140738484 TDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 37073453056 TEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 2516376064 RCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 1515847168 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚6 OBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚18053652 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚3 trainer/global_step‚1599 P8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 K;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 38610163200 _runtime‚2859.263163374 _step‚128 UEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 35913877504 FDeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚18053090 M=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 40290484224 ?6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚3681 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚2684 ?6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚3681 BDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚19709 \JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 3836144219136 K>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚52880585 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚29097984 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 D;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚3681 G:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚52880585 ^JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚351509961309696 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 REDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚10928430 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 NADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚22903679 M>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 2541804032 NADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚22903685 bODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚71125578052608 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 XFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 3828772708069 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 XFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 3833627843072 REDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚29976900 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚69412352 N>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 36985723397 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 F=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚2147 F=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚1534 EDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 38610163200 WHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 2516376064 REDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚22903685 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 577553920 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 WCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚351509935881728 UCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 3833627843072 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚22903679 J=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚29957191 [GDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚351509961309696 TEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 3140738484 TDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 37073453056 TEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 2516376064 RCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 1515847168 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚6 OBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚18053652 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚3 trainer/global_step‚1599 P8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 K;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 38610163200 _runtime‚2859.263163374 _step‚128 UEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 35913877504 FDeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚18053090 M=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 40290484224 ?6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚3681 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚2684 ?6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚3681 BDeviceStatsMonitor.on_train_batch_start/allocation.all.current‚19689 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚3 L?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚30859301 M@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚54463279 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚2684 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚19683 _MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 4933943286784 VFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 37073453056 TDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 35913877504 PCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚23584289 NFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚986 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 963120128 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚19689 PCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚23584295 VBDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated‚365944939503616 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚6 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 111106560 YFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚78221878166016 XIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 3217031168 VGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 3140738484 E8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚54443590 TGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚30878984 A8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚3681 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 YIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 37073453056 SDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 3142752768 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚25427968 L?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚23584289 TGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚23584295 `LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚347558311624656 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚23980 VGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 2516362752 WEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 3949104375296 WGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 35913877504 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚23681 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚11252155 UADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚365942397712896 \HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚347558286196688 D;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚3681 RCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 2541790720 IDeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 F=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚3681 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚6 YEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚361993293337600 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚25427968 NFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚330 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚19683 [IDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 3951620738048 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚598 RCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 2539798104 RBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 40290484224 YJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 2516362752 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚11252750 TDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 40290484224 trainer/global_step‚1649 ZHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 3949104375296 YEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚351504929525835 UADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚351502389727731 IDeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚365942397712896 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚595 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 WGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 34283657216 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 MDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚2147 KBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚2147 `LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚361993318765568 \HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚361993293337600 `MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚73287934879232 PCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚30859301 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚7338597 _step‚131 G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚2684 PCDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated‚30878984 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚23980 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 KBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚1534 VGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 3142752768 SGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚7338594 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 H?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚2147 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 _runtime‚2942.367045731 YEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚365944939503616 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚29097984 cQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 4934025295360 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚23681£³“Ðù'’óO PCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚30859301 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚7338597 _step‚131 G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚2684 PCDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated‚30878984 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚23980 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 KBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚1534 VGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 3142752768 SGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚7338594 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 H?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚2147 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 _runtime‚2942.367045731 YEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚365944939503616 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚29097984 cQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 4934025295360 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚23681 P@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 38610163200 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 577553920 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚25427968 ^LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 3951620738048 QDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚18591347 TEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 1515847168 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 P@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 36985723397 O@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 2541790720 H?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚1309 ]JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚78221989272576 BDeviceStatsMonitor.on_train_batch_start/allocation.all.current‚19689 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚3 L?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚30859301 M@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚54463279 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚2684 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚19683 _MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 4933943286784 VFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 37073453056 TDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 35913877504 PCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚23584289 NFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚986 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 963120128 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚19689 PCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚23584295 VBDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated‚365944939503616 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚6 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 111106560 YFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚78221878166016 XIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 3217031168 VGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 3140738484 E8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚54443590 TGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚30878984 A8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚3681 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 YIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 37073453056 SDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 3142752768 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚25427968 L?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚23584289 TGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚23584295 `LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚347558311624656 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚23980 VGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 2516362752 WEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 3949104375296 WGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 35913877504 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚23681 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚11252155 UADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚365942397712896 \HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚347558286196688 D;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚3681 RCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 2541790720 IDeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 F=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚3681 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚6 YEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚361993293337600 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚25427968 NFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚330 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚19683 [IDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 3951620738048 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚598 RCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 2539798104 RBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 40290484224 YJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 2516362752 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚11252750 TDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 40290484224 trainer/global_step‚1649 ZHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 3949104375296 YEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚351504929525835 UADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚351502389727731 IDeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚365942397712896 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚595 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 WGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 34283657216 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 MDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚2147 KBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚2147 `LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚361993318765568 \HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚361993293337600 `MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚73287934879232ƒ}2“- _wandbruntime‚2943  _runtime‚2943«’]¦ ”øM EDeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 FDeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚54495537 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 100607488 ]KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 4936905789440 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚3 ^JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚362207264836096 % _timestamp‚1.7884881448200734e+09 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚6 PDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 36985723397 UEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 34283657216 KBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚2147 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 WCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚351712798322273 B9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚3681 \JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 3948971037121 YIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 37073453056 VGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 3217031168 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚19709 T@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚366161241683456 UCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 3951460471296 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚25427968 TEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 3140738484 NADeviceSt|styXatsMonitor.on_train_batch_end/active.large_pool.allocated‚23598185 aODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 4936977298944 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 trainer/global_step‚1649 _runtime‚2943.921858853 J=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚30877643 ?6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚3681 KBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚1534 F=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚1534 _step‚132 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚11259245 ZFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚347763801857184 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚7342963 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 TEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 2516376064 ZFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚362207239408128 REDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚11258671 [HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚78269071601152 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 577553920 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 ?6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚3681 RBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 35913877504 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚577 XFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 3946456666777 XFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 3951460471296 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 K>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚18601631 N>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 38610163200 WCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚362207239408128 TEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 3142752768 F=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚2147 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚25427968 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚574 LDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚330 C6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚54475822 NADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚30897352 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚25427968 UEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 35913877504 RBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 40290484224 G:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚54495537 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚23598179 S?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚351710258523961 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚29097984 D;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚3681 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 YGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 3953976847360 WCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚366161241683456 FDeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 2541804032 QBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 3142752768 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚23980 I@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚1534 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚23980 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚23681 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 ^JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚347763827285152 WGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 37073453056 XIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 3217031168X;'•ýM D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 YGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 3953976847360 WCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚366161241683456 FDeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 2541804032 QBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 3142752768 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚23980 I@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚1534 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚23980 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚23681 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 ^JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚347763827285152 WGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 37073453056 XIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 3217031168 EDeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 FDeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚54495537 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 100607488 ]KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 4936905789440 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚3 ^JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚362207264836096 % _timestamp‚1.7884881448200734e+09 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚6 PDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 36985723397 UEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 34283657216 KBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚2147 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 WCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚351712798322273 B9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚3681 \JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 3948971037121 YIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 37073453056 VGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 3217031168 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚19709 T@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚366161241683456 UCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 3951460471296 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚25427968 TEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 3140738484 NADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚23598185 aODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 4936977298944 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 trainer/global_step‚1649 _runtime‚2943.921858853 J=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚30877643 ?6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚3681 KBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚1534 F=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚1534 _step‚132 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚11259245 ZFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚347763801857184 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚7342963 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 TEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 2516376064 ZFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚362207239408128 REDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚11258671 [HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚78269071601152 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 577553920 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 ?6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚3681 RBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 35913877504 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚577 XFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 3946456666777 XFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 3951460471296 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 K>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚18601631 N>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 38610163200 WCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚362207239408128 TEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 3142752768 F=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚2147 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚25427968 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚574 LDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚330 C6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚54475822 NADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚30897352 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚25427968 UEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 35913877504 RBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 40290484224 G:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚54495537 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚23598179 S?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚351710258523961 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚29097984 D;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚3681„Šä&R2–- _wandbruntime‚2943  _runtime‚2943Œöj²U—Ï /loss/FBt4/detection_loss‚2.3790480287178704 trainer/global_step‚1649 /loss/FRt4/assignment_loss‚2.728572338387586 2loss/SRqqt3/assignment_loss‚1.9533701986074448 2loss/SRqqt4/assignment_loss‚1.6129384054943467 .loss/FRt3/detection_loss‚2.919348657131195 0loss/FRt1/assignment_loss‚3.2815449810495565 1loss/SRqqt2/assignment_loss‚1.196131495787113 /loss/FRt4/detection_loss‚2.7058247556720953 0loss/SRqqt4/detection_loss‚1.631259978720636 /loss/FRt3/assignment_loss‚3.314557296889169 0loss/FBt2/assignment_loss‚0.1147558837690094 /loss/FRt2/detection_loss‚1.5547837987542152 &loss/total_loss‚42.870489232361344 _runtime‚2943.922826358 % _timestamp‚1.7884881448216834e+09 -loss/FBt3/detection_loss‚2.48800988407696 epoch‚2 _step‚133 0loss/FBt1/assignment_loss‚0.5727200902917463 0loss/FBt3/assignment_loss‚0.1064919848799678 1loss/SRqqt1/assignment_loss‚1.646453846810442 1loss/SRqqt3/detection_loss‚2.5446577741474403 /loss/FBt1/detection_loss‚2.7155714423760124 .loss/FBt2/detection_loss‚0.602476268965337 0loss/FRt2/assignment_loss‚1.0991548013228636 1loss/FBt4/assignment_loss‚0.10143592030204751 /loss/FRt1/detection_loss‚2.1613494200157186 0loss/SRqqt1/detection_loss‚2.514155338659431 /loss/SRqqt2/detection_loss‚0.92587664153314SÔgZ˜Ô 0loss/SRqqt4/detection_loss‚1.631259978720636 /loss/FRt3/assignment_loss‚3.314557296889169 0loss/FBt2/assignment_loss‚0.1147558837690094 /loss/FRt2/detection_loss‚1.5547837987542152 &loss/total_loss‚42.870489232361344 _runtime‚2943.922826358 % _timestamp‚1.7884881448216834e+09 -loss/FBt3/detection_loss‚2.48800988407696 epoch‚2 _step‚133 0loss/FBt1/assignment_loss‚0.5727200902917463 0loss/FBt3/assignment_loss‚0.1064919848799678 1loss/SRqqt1/assignment_loss‚1.646453846810442 1loss/SRqqt3/detection_loss‚2.5446577741474403 /loss/FBt1/detection_loss‚2.7155714423760124 .loss/FBt2/detection_loss‚0.602476268965337 0loss/FRt2/assignment_loss‚1.0991548013228636 1loss/FBt4/assignment_loss‚0.10143592030204751 /loss/FRt1/detection_loss‚2.1613494200157186 0loss/SRqqt1/detection_loss‚2.514155338659431 /loss/SRqqt2/detection_loss‚0.92587664153314 /loss/FBt4/detection_loss‚2.3790480287178704 trainer/global_step‚1649 /loss/FRt4/assignment_loss‚2.728572338387586 2loss/SRqqt3/assignment_loss‚1.9533701986074448 2loss/SRqqt4/assignment_loss‚1.6129384054943467 .loss/FRt3/detection_loss‚2.919348657131195 0loss/FRt1/assignment_loss‚3.2815449810495565 1loss/SRqqt2/assignment_loss‚1.196131495787113 /loss/FRt4/detection_loss‚2.7058247556720953…ÝÀÁ¢N™‚(:à ÛÓèÔûˆ°) proc.memory.availableMB‚ 1808.52734375 cpu‚25.48507557139487 proc.cpu.threads‚17) disk./.usagePercent‚83.36809396519828% disk./.usageGB‚2981.6468544006348 disk.sda.out‚287.07568359375 disk.rbd0.in‚0 disk.sda1.in‚0 disk.sda1.out‚0$ memory_percent‚88.96162509918213 disk.rbd0.out‚ 0.62109375 disk.nvme0n1.in‚4.59375 disk.dm-0.out‚39032.11328125 disk.nvme1n1.out‚ 17355.28125 disk.sda.in‚ 0.0078125# disk.nvme0n1.out‚21676.83203125 disk.sdb1.in‚0) proc.memory.percent‚18.66137981414795 disk.sda2.in‚ 0.0078125! disk.sda2.out‚287.07568359375! disk.sdb2.out‚287.07568359375 disk.md0p1.out‚ 275.05859375 disk.sdb1.out‚0 disk.md0.in‚ 0.0078125 disk.md0p1.in‚ 0.0078125 disk.rbd2.in‚0 disk.rbd2.out‚ 0.783203125 disk.sdb.in‚0 disk.sdb.out‚287.07568359375# proc.memory.rssMB‚ 3057.48046875 network.recv‚968459 disk.md0.out‚ 275.05859375 disk.dm-0.in‚9.109375 disk.sdb2.in‚0 network.sent‚ 3.757655e+06 disk.nvme1n1.in‚4.515625‹¥”š‚(:‰ ÛÓèÔñË° gpu.0.gpu‚96.0 gpu.0.memory‚16+ gpu.0.memoryAllocated‚47.54737854003906* gpu.0.memoryAllocatedBytes‚ 40842887168 gpu.0.temp‚45.0 gpu.0.powerWatts‚236.377' gpu.0.enforcedPowerLimitWatts‚400.0 gpu.0.powerPercent‚59.09425 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0Ü…·‘›‚(:† êÓèÔ£¤Ú® gpu.0.gpu‚16.0 gpu.0.memory‚5+ gpu.0.memoryAllocated‚47.54737854003906* gpu.0.memoryAllocatedBytes‚ 40842887168 gpu.0.temp‚48.0 gpu.0.powerWatts‚309.212' gpu.0.enforcedPowerLimitWatts‚400.0 gpu.0.powerPercent‚77.303 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0ݯDœ‚(:¹ êÓèÔ׼ɯ" disk.nvme0n1.out‚ 21840.9296875 disk.rbd2.out‚ 0.783203125 disk.sdb.in‚0 disk.sdb.out‚288.3349609375" proc.memory.availableMB‚1808.5 disk.dm-0.in‚9.109375 disk.sdb1.in‚0 disk.sdb2.out‚288.3349609375# proc.memory.rssMB‚ 3057.48046875 disk.rbd2.in‚0 disk.sda2.out‚288.3349609375 cpu‚25.484157423620832 disk.rbd0.out‚ 0.62109375 disk.nvme0n1.in‚4.59375 disk.md0.out‚ 276.26171875 disk.sda.out‚288.3349609375 disk.sdb2.in‚0 disk.sda1.out‚0# memory_percent‚88.9617919921875 network.recv‚974263% disk./.usageGB‚2982.0852584838867 disk.nvme1n1.in‚4.515625 disk.sda.in‚ 0.0078125 disk.sdb1.out‚0 disk.md0.in‚ 0.0078125 disk.md0p1.out‚ 276.26171875 network.sent‚ 3.83374e+06) disk./.usagePercent‚83.38035192685238# disk.nvme1n1.out‚17355.37890625 disk.dm-0.out‚39196.30859375 disk.sda1.in‚0 disk.md0p1.in‚ 0.0078125) proc.memory.percent‚18.66137981414795 disk.rbd0.in‚0 disk.sda2.in‚ 0.0078125 proc.cpu.threads‚17iHï·ž‚(:“ ùÓèÔåÕ’® gpu.0.gpu‚98.0 gpu.0.memory‚17+ gpu.0.memoryAllocated‚47.54737854003906* gpu.0.memoryAllocatedBytes‚ 40842887168 gpu.0.temp‚45.0 gpu.0.powerWatts‚228.878' gpu.0.enforcedPowerLimitWatts‚400.0) gpu.0.powerPercent‚57.219500000000004 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0w´±­Hž‚(:½ ùÓèÔ﮺° disk.rbd2.out‚ 0.783203125 disk.nvme1n1.in‚4.515625" disk.nvme1n1.out‚ 17355.7109375 disk.sda.out‚289.71142578125 disk.rbd0.in‚0 cpu‚25.483140806172766 proc.cpu.threads‚17% disk./.usageGB‚2982.0926399230957) proc.memory.availableMB‚ 1808.73046875# proc.memory.rssMB‚ 3057.48046875 disk.sdb2.in‚0 disk.md0p1.in‚ 0.0078125 disk.dm-0.in‚9.109375! disk.sda2.out‚289.71142578125 disk.sda.in‚ 0.0078125# disk.nvme0n1.out‚22015.75390625 disk.dm-0.out‚39371.46484375 disk.sdb1.out‚0 network.sent‚ 3.8395e+06 disk.sda1.out‚0 disk.md0.out‚ 277.59375 disk.rbd2.in‚0 disk.sdb.in‚0# memory_percent‚88.9603853225708) disk./.usagePercent‚83.38055831498204 disk.md0p1.out‚ 277.59375 network.recv‚978314 disk.sda2.in‚ 0.0078125 disk.rbd0.out‚ 0.62109375 disk.nvme0n1.in‚4.59375) proc.memory.percent‚18.66137981414795! disk.sdb2.out‚289.71142578125 disk.sda1.in‚0 disk.sdb.out‚289.71142578125 disk.sdb1.in‚0 disk.md0.in‚ 0.0078125îýq‡žŸ‚(:“ ˆÔèÔÚ·Þ¯ gpu.0.gpu‚98.0 gpu.0.memory‚17+ gpu.0.memoryAllocated‚47.54737854003906* gpu.0.memoryAllocatedBytes‚ 40842887168 gpu.0.temp‚45.0 gpu.0.powerWatts‚227.422' gpu.0.enforcedPowerLimitWatts‚400.0) gpu.0.powerPercent‚56.855500000000006 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0ãù~LG ‚(:¼ ˆÔèÔ‹ï™°$ disk./.usageGB‚2982.092872619629 disk.nvme1n1.in‚4.515625 disk.sda.in‚ 0.0078125 disk.sda.out‚291.35595703125 disk.rbd0.out‚ 0.62109375" disk.nvme0n1.out‚ 22230.2109375) proc.memory.availableMB‚ 1808.71484375 network.recv‚982997 disk.sdb1.out‚0! disk.sdb2.out‚291.35595703125 disk.md0p1.in‚ 0.0078125 disk.md0p1.out‚ 279.15625 disk.rbd0.in‚0 disk.rbd2.in‚0 disk.sdb.out‚291.35595703125 disk.dm-0.in‚9.109375 disk.dm-0.out‚ 39586.1328125! disk.nvme1n1.out‚ 17355.921875$ memory_percent‚88.96048069000244 network.sent‚ 3.845792e+06 disk.md0.out‚ 279.15625 disk.rbd2.out‚ 0.783203125) proc.memory.percent‚18.66137981414795 disk.sdb2.in‚0! disk.sda2.out‚291.35595703125 cpu‚25.482402343087898 proc.cpu.threads‚17) disk./.usagePercent‚83.38056482127449 disk.sdb1.in‚0 disk.sda1.out‚0 disk.md0.in‚ 0.0078125 disk.sda2.in‚ 0.0078125 disk.nvme0n1.in‚4.59375# proc.memory.rssMB‚ 3057.48046875 disk.sda1.in‚0 disk.sdb.in‚0ÖKà‘¡‚(:† —ÔèÔÐÅŽ° gpu.0.gpu‚0.0 gpu.0.memory‚0+ gpu.0.memoryAllocated‚47.54737854003906* gpu.0.memoryAllocatedBytes‚ 40842887168 gpu.0.temp‚39.0 gpu.0.powerWatts‚116.306' gpu.0.enforcedPowerLimitWatts‚400.0 gpu.0.powerPercent‚29.0765 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0ù%©2¢‚(:§ —ÔèÔ³½÷°! disk.nvme1n1.out‚ 17356.140625& proc.memory.availableMB‚ 1808.46875 network.sent‚ 3.851021e+06% disk./.usageGB‚2982.5928535461426 disk.rbd0.in‚0 disk.dm-0.in‚9.109375 disk.md0.out‚ 280.4140625 disk.md0p1.in‚ 0.0078125 disk.dm-0.out‚39794.69921875 disk.sdb1.in‚0 disk.md0.in‚ 0.0078125 disk.rbd2.in‚0 disk.rbd2.out‚ 0.783203125 disk.sdb.in‚0 disk.sda1.in‚0 disk.sdb.out‚ 292.65625 disk.sda2.out‚ 292.65625 disk.sda.out‚ 292.65625# proc.memory.rssMB‚ 3057.48046875) disk./.usagePercent‚83.39454449723809 disk.sdb2.in‚0 disk.sdb1.out‚0 disk.sdb2.out‚ 292.65625$ memory_percent‚88.96198272705078 proc.cpu.threads‚17 disk.rbd0.out‚ 0.62109375# disk.nvme0n1.out‚22438.55859375 disk.sda.in‚ 0.0078125 cpu‚25.48157168055258 disk.nvme0n1.in‚4.59375 disk.md0p1.out‚ 280.4140625 disk.nvme1n1.in‚4.515625) proc.memory.percent‚18.66137981414795 network.recv‚986431 disk.sda1.out‚0 disk.sda2.in‚ 0.0078125ž Žx2£- _wandbruntime‚3028  _runtime‚3028/Pz¤Ã¤½ $ lr-AdamW/pg1‚0.00102721759969405 $ lr-AdamW/pg2‚0.00102721759969405 trainer/global_step‚1699 % _timestamp‚1.7884882295656404e+09 _runtime‚3028.667062395 _step‚134ã»Ånȥ trainer/global_step‚1699 % _timestamp‚1.7884882295656404e+09 _runtime‚3028.667062395 _step‚134 $ lr-AdamW/pg1‚0.00102721759969405 $ lr-AdamW/pg2‚0.00102721759969405†’%§2¦- _wandbruntime‚3028  _runtime‚3028È4"ê§íO A8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚3681 UADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚376757533867520 UHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚98523648 TGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚24278795 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚3 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚25427968 VGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 3140738484 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 [KDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 37073453056 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚25427968 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚7556897 G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚2684 dQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚75494480226816 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 577553920 ZKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 3217031168 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 TEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 1515847168 \HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚357834069221488 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚577 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚23980 VGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 2516362752 SDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 3142752768 `LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚372690622291968 IDeviceStatsMonitor.on_train_batch_start/allocation.all.current‚19689 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚11582334 NFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚986 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 XIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 3217031168 PCDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated‚31799436 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 WGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 35913877504 _MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 5082380802560 E8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚56058542 R>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚376757533867520 YEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚376760075658240 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 D>DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 YIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 37073453056 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚23681 RCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 2541790720 YJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 2516362752 VGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 3142752768 cQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 5082450228224 YJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 2514370136 MDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚2147 KBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚2147 SGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚7556894 PCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚24278795 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚25427968 KBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚1534 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚23980 QDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚19139805 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚574 TDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 40290484224 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 _step‚135 P@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 38610163200 UADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚361895856711239 ^LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 4064301859887 VFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 37073453056 _runtime‚3028.668835323 L?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚24278789 [IDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 4069453366272 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚2684 MDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚1534 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚6 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 RCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 2539798104 UFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 3217031168 D;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚3681 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚19689 ZHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 4066937003520 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚6 O?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 40290484224 trainer/global_step‚1699 ]IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚372690622291968 TGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚31799436 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚11582908 A8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚3681 ^LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 4069453366272 `LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚357834094649456 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0¡…Xø'¨òO YFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚80576831931392 RBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 40290484224 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 NFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚330 P@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 36985723397 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚29097984 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 963120128 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 F=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚3681 WEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 4066937003520 M@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚56078231 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚19683 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 YEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚361898396509343 ]JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚80576930455040 BDeviceStatsMonitor.on_train_batch_start/allocation.all.current‚19689 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚11582334 NFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚986 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 XIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 3217031168 PCDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated‚31799436 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 WGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 35913877504 _MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 5082380802560 E8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚56058542 R>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚376757533867520 YEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚376760075658240 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 D>DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 YIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 37073453056 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚23681 RCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 2541790720 YJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 2516362752 VGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 3142752768 cQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 5082450228224 YJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 2514370136 MDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚2147 KBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚2147 SGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚7556894 PCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚24278795 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚25427968 KBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚1534 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚23980 QDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚19139805 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚574 TDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 40290484224 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 _step‚135 P@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 38610163200 UADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚361895856711239 ^LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 4064301859887 VFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 37073453056 _runtime‚3028.668835323 L?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚24278789 [IDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 4069453366272 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚2684 MDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚1534 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚6 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 RCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 2539798104 UFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 3217031168 D;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚3681 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚19689 ZHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 4066937003520 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚6 O?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 40290484224 trainer/global_step‚1699 ]IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚372690622291968 TGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚31799436 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚11582908 A8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚3681 ^LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 4069453366272 `LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚357834094649456 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 A8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚3681 UADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚376757533867520 UHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚98523648 TGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚24278795 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚3 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚25427968 VGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 3140738484 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 [KDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 37073453056 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚25427968 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚7556897 G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚2684 dQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚75494480226816 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 577553920 ZKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 3217031168 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 TEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 1515847168 \HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚357834069221488 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚577 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚23980 VGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 2516362752 SDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 3142752768 `LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚372690622291968 IDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 ZFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚358039584881984 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚7561263 REDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚11589017 WDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚80623923794432 N>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 36985723397 QBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 3142752768 FDeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚19150277 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚7561260 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 963120128 TEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 3140738484 PDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 38610163200 S?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚362103725507469 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚6 WCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚376976377838080 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 WHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 2514370344 RBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 40290484224 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 TEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 3142752768 G:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚56090774 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚23980 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 577553920 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚25427968 P@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 40290484224 KBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚2147 I@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚2147 KBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚1534 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 WHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 2516376064 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚11589584 ?6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚3681 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 NADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚24292685 J=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚31798095 ^JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚372904568362496 K>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚56110489 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚570 ^JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚358039610309952 M>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 2541804032 S?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚376973836034048 LDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚986 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚6 LDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚330 RBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 35913877504 SDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 3217031168 REDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚31817804 NADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚31798095 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 C6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚56090774 bODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚75538610551808 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 BDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 ZFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚358039584881984 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚7561263 REDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚11589017 WDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚80623923794432 N>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 36985723397 QBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 3142752768 FDeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚19150277 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚7561260 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 963120128 TEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 3140738484 PDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 38610163200 S?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚362103725507469 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚6 WCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚376976377838080 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 WHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 2514370344 RBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 40290484224 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 TEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 3142752768 G:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚56090774 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚23980 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 577553920 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚25427968 P@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 40290484224 KBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚2147 I@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚2147 KBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚1534 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 WHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 2516376064 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚11589584 ?6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚3681 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 NADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚24292685 J=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚31798095 ^JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚372904568362496 K>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚56110489 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚570 ^JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚358039610309952 M>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 2541804032 S?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚376973836034048 LDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚986 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚6 LDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚330 RBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 35913877504 SDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 3217031168 REDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚31817804 NADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚31798095 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 C6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚56090774ˆC2®- _wandbruntime‚3030  _runtime‚3030¥”w°Z¯Ô epoch‚2 0loss/FRt1/assignment_loss‚3.7586833024024964 0loss/FRt4/assignment_loss‚3.0644366207510925 0loss/FBt1/assignment_loss‚0.2539163270137376 .loss/FRt1/detection_loss‚2.200339477285743 /loss/FBt1/detection_loss‚2.8695278233951993 .loss/FBt4/detection_loss‚3.082997371753057 1loss/SRqqt2/assignment_loss‚1.068267149472987 1loss/FBt4/assignment_loss‚0.17336408830427294 /loss/FRt4/detection_loss‚2.3152595297541727 1loss/SRqqt2/detection_loss‚0.9342057290975447 0loss/SRqqt4/detection_loss‚1.710409030644563 /loss/FBt3/detection_loss‚2.7520996928215027 _runtime‚3030.259182746 1loss/SRqqt1/detection_loss‚2.4936206672872814 _step‚137 %loss/total_loss‚44.62242152090401 trainer/global_step‚1699 0loss/FRt2/assignment_loss‚1.6306595636738672 2loss/SRqqt1/assignment_loss‚1.5715902275272777 .loss/FRt3/detection_loss‚2.575541316769844 0loss/SRqqt3/detection_loss‚2.321419891052776 0loss/FRt3/assignment_loss‚3.1145743582137797 2loss/SRqqt3/assignment_loss‚1.8747657656669616 2loss/SRqqt4/assignment_loss‚1.7431390435591232 1loss/FBt3/assignment_loss‚0.33935316566203255 /loss/FRt2/detection_loss‚1.6876427578843303 /loss/FBt2/detection_loss‚0.8497167444729712 $ _timestamp‚1.788488231157935e+09 0loss/FBt2/assignment_loss‚0.2368918764373988ï“Ø_°Ù trainer/global_step‚1699 0loss/FRt2/assignment_loss‚1.6306595636738672 2loss/SRqqt1/assignment_loss‚1.5715902275272777 .loss/FRt3/detection_loss‚2.575541316769844 0loss/SRqqt3/detection_loss‚2.321419891052776 0loss/FRt3/assignment_loss‚3.1145743582137797 2loss/SRqqt3/assignment_loss‚1.8747657656669616 2loss/SRqqt4/assignment_loss‚1.7431390435591232 1loss/FBt3/assignment_loss‚0.33935316566203255 /loss/FRt2/detection_loss‚1.6876427578843303 /loss/FBt2/detection_loss‚0.8497167444729712 $ _timestamp‚1.788488231157935e+09 0loss/FBt2/assignment_loss‚0.2368918764373988 epoch‚2 0loss/FRt1/assignment_loss‚3.7586833024024964 0loss/FRt4/assignment_loss‚3.0644366207510925 0loss/FBt1/assignment_loss‚0.2539163270137376 .loss/FRt1/detection_loss‚2.200339477285743 /loss/FBt1/detection_loss‚2.8695278233951993 .loss/FBt4/detection_loss‚3.082997371753057 1loss/SRqqt2/assignment_loss‚1.068267149472987 1loss/FBt4/assignment_loss‚0.17336408830427294 /loss/FRt4/detection_loss‚2.3152595297541727 1loss/SRqqt2/detection_loss‚0.9342057290975447 0loss/SRqqt4/detection_loss‚1.710409030644563 /loss/FBt3/detection_loss‚2.7520996928215027 _runtime‚3030.259182746 1loss/SRqqt1/detection_loss‚2.4936206672872814 _step‚137 %loss/total_loss‚44.62242152090401‰ŸÑáêž±‚(:“ µÔèÔ©Ö¯ gpu.0.gpu‚85.0 gpu.0.memory‚15+ gpu.0.memoryAllocated‚47.54737854003906* gpu.0.memoryAllocatedBytes‚ 40842887168 gpu.0.temp‚45.0 gpu.0.powerWatts‚231.323' gpu.0.enforcedPowerLimitWatts‚400.0) gpu.0.powerPercent‚57.830749999999995 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0]2F=E²‚(:º µÔèÔ‚Þë°" disk.nvme1n1.out‚ 17423.3671875 disk.rbd0.out‚ 0.62109375# proc.memory.rssMB‚ 3057.48046875 proc.cpu.threads‚17 disk.md0.in‚ 0.0078125 disk.sdb.in‚0 disk.sda2.out‚ 295.873046875) proc.memory.percent‚18.66137981414795 cpu‚25.479798665198896 disk.md0p1.out‚ 283.5078125 disk.sda1.in‚0 disk.md0.out‚ 283.5078125 disk.md0p1.in‚ 0.0078125 disk.rbd2.in‚0 disk.sdb.out‚ 295.873046875 disk.sda2.in‚ 0.0078125 disk.sda.in‚ 0.0078125 disk.dm-0.in‚9.109375) disk./.usagePercent‚83.38750884855774 disk.sda.out‚ 295.873046875 disk.rbd0.in‚0 disk.nvme0n1.in‚4.59375# disk.nvme0n1.out‚22722.62890625 disk.dm-0.out‚40145.99609375 disk.sdb2.in‚0 network.recv‚995319# disk./.usageGB‚2982.34122467041 disk.rbd2.out‚ 0.783203125 disk.sdb2.out‚ 295.873046875) proc.memory.availableMB‚ 1808.89453125 network.sent‚ 3.941906e+06 disk.sda1.out‚0 disk.nvme1n1.in‚4.515625 disk.sdb1.in‚0 disk.sdb1.out‚0$ memory_percent‚88.95938396453857nÍK@”³‚(:‰ ÄÔèÔ³¤¨® gpu.0.gpu‚71.0 gpu.0.memory‚13+ gpu.0.memoryAllocated‚47.54737854003906* gpu.0.memoryAllocatedBytes‚ 40842887168 gpu.0.temp‚45.0 gpu.0.powerWatts‚244.077' gpu.0.enforcedPowerLimitWatts‚400.0 gpu.0.powerPercent‚61.01925 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0㋺tU´‚(:Ê ÄÔèÔê𚯠disk.sda1.out‚0 network.sent‚ 4.007916e+06# disk.nvme0n1.out‚22818.73046875 disk.dm-0.in‚9.109375 disk.dm-0.out‚ 40244.140625 disk.md0p1.in‚ 0.0078125 disk.rbd2.out‚ 0.783203125 disk.sda2.in‚ 0.0078125 disk.sda.out‚297.41259765625) proc.memory.availableMB‚ 1808.15234375 disk.sdb2.in‚0 disk.rbd0.in‚0) disk./.usagePercent‚83.38843732848986 disk.sda1.in‚0 disk.md0.in‚ 0.0078125 disk.sdb.in‚0! disk.sda2.out‚297.41259765625 disk.nvme1n1.in‚4.515625) proc.memory.percent‚18.66137981414795 network.recv‚ 1.001296e+06% disk./.usageGB‚2982.3744316101074 disk.md0.out‚ 284.97265625 disk.sdb.out‚297.41259765625# proc.memory.rssMB‚ 3057.48046875 proc.cpu.threads‚17 disk.md0p1.out‚ 284.97265625 disk.rbd0.out‚ 0.62109375 disk.nvme0n1.in‚4.59375 disk.sdb1.out‚0# disk.nvme1n1.out‚17425.41015625 disk.sda.in‚ 0.0078125 cpu‚25.478871588155435 disk.sdb1.in‚0! disk.sdb2.out‚297.41259765625 disk.rbd2.in‚0# memory_percent‚88.9639139175415ƒ–a“µ‚(:ˆ ÓÔèÔ£ˆ‹° gpu.0.gpu‚97.0 gpu.0.memory‚17+ gpu.0.memoryAllocated‚47.54737854003906* gpu.0.memoryAllocatedBytes‚ 40842887168 gpu.0.temp‚45.0 gpu.0.powerWatts‚235.058' gpu.0.enforcedPowerLimitWatts‚400.0 gpu.0.powerPercent‚58.7645 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0ìE,G¶‚(:¼ ÓÔèÔψÿ° disk.rbd0.out‚ 0.62109375 disk.dm-0.out‚40483.37890625 disk.sdb1.in‚0 disk.md0.out‚ 286.26171875# disk.nvme0n1.out‚22967.73046875 disk.md0p1.out‚ 286.26171875 disk.sdb.out‚ 298.76953125 disk.sda2.in‚ 0.0078125 disk.sda2.out‚ 298.76953125$ memory_percent‚88.96093368530273# proc.memory.rssMB‚ 3057.48046875 disk.sdb2.out‚ 298.76953125 disk.rbd2.out‚ 0.783203125 disk.sdb.in‚0" disk.nvme1n1.out‚ 17515.6484375) proc.memory.percent‚18.66137981414795 disk.sda.out‚ 298.76953125 disk.sdb2.in‚0 network.sent‚ 4.01097e+06 disk.sda1.out‚0 disk.md0.in‚ 0.0078125 disk.nvme0n1.in‚4.59375 disk.sda.in‚ 0.0078125 proc.cpu.threads‚17 disk.sdb1.out‚0 network.recv‚ 1.004578e+06) disk./.usagePercent‚83.39454972360416% disk./.usageGB‚2982.5930404663086 disk.dm-0.in‚9.109375 disk.md0p1.in‚ 0.0078125 disk.rbd2.in‚0 disk.nvme1n1.in‚4.515625 disk.rbd0.in‚0 disk.sda1.in‚0' proc.memory.availableMB‚ 1808.640625 cpu‚25.47813678341575LÅ··‚(:’ âÔèÔ—›«° gpu.0.gpu‚96.0 gpu.0.memory‚17+ gpu.0.memoryAllocated‚47.54737854003906* gpu.0.memoryAllocatedBytes‚ 40842887168 gpu.0.temp‚45.0 gpu.0.powerWatts‚235.768' gpu.0.enforcedPowerLimitWatts‚400.0( gpu.0.powerPercent‚58.94200000000001 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0Tù§ÙD¸‚(:¹ âÔèÔàØ ± disk.rbd2.out‚ 0.783203125 disk.sdb.in‚0 disk.sda2.out‚ 299.9921875 disk.sdb1.out‚0 disk.sda.out‚ 299.9921875 disk.rbd0.out‚ 0.62109375 disk.md0p1.in‚ 0.0078125 disk.md0p1.out‚ 287.42578125 disk.sda2.in‚ 0.0078125) proc.memory.availableMB‚ 1808.22265625 network.sent‚ 4.016672e+06 disk.rbd0.in‚0! disk.nvme0n1.out‚ 23063.828125 disk.dm-0.in‚9.109375 disk.sda1.in‚0$ memory_percent‚88.96348476409912( disk./.usagePercent‚83.3945471637514$ disk./.usageGB‚2982.592948913574 disk.nvme1n1.in‚4.515625 disk.dm-0.out‚40650.26953125 disk.md0.out‚ 287.42578125 disk.sdb.out‚ 299.9921875# proc.memory.rssMB‚ 3057.48046875 cpu‚25.477354684426643 disk.sda.in‚ 0.0078125 disk.nvme0n1.in‚4.59375 disk.sdb2.in‚0 disk.sda1.out‚0 network.recv‚ 1.008406e+06# disk.nvme1n1.out‚17586.44140625) proc.memory.percent‚18.66137981414795 proc.cpu.threads‚17 disk.sdb1.in‚0 disk.sdb2.out‚ 299.9921875 disk.md0.in‚ 0.0078125 disk.rbd2.in‚0²Fý’¹‚(:‡ ñÔèÔ†×Я gpu.0.gpu‚98.0 gpu.0.memory‚18+ gpu.0.memoryAllocated‚47.54737854003906* gpu.0.memoryAllocatedBytes‚ 40842887168 gpu.0.temp‚45.0 gpu.0.powerWatts‚250.264' gpu.0.enforcedPowerLimitWatts‚400.0 gpu.0.powerPercent‚62.566 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0zÛ[MKº‚(:À ñÔèÔ«‡° disk.sda.in‚ 0.0078125 disk.nvme0n1.in‚4.59375 disk.dm-0.in‚9.109375 disk.sdb2.out‚301.7587890625 disk.sda1.in‚0 cpu‚25.476605518430695 network.sent‚ 4.022712e+06# disk.nvme0n1.out‚23090.65234375 disk.dm-0.out‚40792.5 disk.sdb1.out‚0( proc.memory.availableMB‚ 1808.3515625 disk.nvme1n1.in‚4.515625# disk.nvme1n1.out‚17701.84765625 disk.md0.out‚ 289.13671875 disk.sdb.out‚301.7587890625 proc.cpu.threads‚17) disk./.usagePercent‚83.39454663044873$ disk./.usageGB‚2982.592929840088 disk.sda2.out‚301.7587890625 disk.rbd0.out‚ 0.62109375 disk.sda1.out‚0$ memory_percent‚88.96269798278809 disk.sda2.in‚ 0.0078125 disk.rbd0.in‚0 disk.sdb1.in‚0 disk.md0.in‚ 0.0078125 disk.md0p1.in‚ 0.0078125 disk.rbd2.in‚0 disk.sdb.in‚0# proc.memory.rssMB‚ 3057.48046875 network.recv‚ 1.012593e+06) proc.memory.percent‚18.66137981414795 disk.rbd2.out‚ 0.783203125 disk.sda.out‚301.7587890625 disk.sdb2.in‚0 disk.md0p1.out‚ 289.13671875z0o†2»- _wandbruntime‚3111  _runtime‚3111 %ýmļ¾ trainer/global_step‚1749 % _timestamp‚1.7884883127739325e+09 _runtime‚ 3111.87638455 _step‚138 % lr-AdamW/pg1‚0.001017309021771196 % lr-AdamW/pg2‚0.001017309021771196ñ:˜eɽà % lr-AdamW/pg2‚0.001017309021771196 trainer/global_step‚1749 % _timestamp‚1.7884883127739325e+09 _runtime‚ 3111.87638455 _step‚138 % lr-AdamW/pg1‚0.001017309021771196Šæ©ÄY2¾- _wandbruntime‚3111  _runtime‚3111â]Lsó'¿íO _MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 5230731674624 A8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚3681 F=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚3681 SDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 3142752768 ZHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 4184769631744 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚23980 BDeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚387572670022144 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚6 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚19683 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 PCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚24973295 TGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚32719888 NFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚986 dQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚77700996476416 UADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚372289323694747 VBDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated‚387575211812864 ]IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚383387925818368 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚23681 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚11913878 H?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚1534 L?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚32700205 YJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 2516362752 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 IDeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 MDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚1534 UFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 3217031168 A8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚3681 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚25427968 ]JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚82931797576704 P@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 36985723397 YJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 2514370136 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚570 O?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 40290484224 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚3 SGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚7775194 YFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚82931699053056 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 963120128 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚69425664 RBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 40290484224 TGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚24973295 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 MDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚2147 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚23980 RCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 2541790720 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚25427968 KBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚2147 TDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 35913877504 `LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚368109877674256 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 H?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚2147 O@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 2541790720 L?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚24973289 WGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 35913877504 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚19689 TEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 1515847168 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚29097984 YEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚372291863492851 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚6 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 VGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 2516362752 `LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚383387925818368 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚2684 M@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚19689072 `MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚77700967378432 RCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 2539798104 ZHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 4179471448459 _step‚139 UHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚98523648 ^LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 4181985818595 VGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 3140738484 M=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 38610163200 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 577553920 TDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 40290484224 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 PCDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated‚32719888 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚567 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 % _timestamp‚1.7884883127756402e+09 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚19689 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚23681 cQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 5230801100288 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚2684 YEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚383387900390400 UADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚387572670022144 M@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚57693183o”_2ÀòO `LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚368109877674256 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 H?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚2147 O@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 2541790720 L?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚24973289 WGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 35913877504 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚19689 TEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 1515847168 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚29097984 YEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚372291863492851 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚6 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 VGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 2516362752 `LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚383387925818368 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚2684 M@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚19689072 `MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚77700967378432 RCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 2539798104 ZHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 4179471448459 _step‚139 UHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚98523648 ^LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 4181985818595 VGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 3140738484 M=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 38610163200 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 577553920 TDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 40290484224 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 PCDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated‚32719888 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚567 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 % _timestamp‚1.7884883127756402e+09 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚19689 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚23681 cQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 5230801100288 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚2684 YEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚383387900390400 UADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚387572670022144 M@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚57693183 _MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 5230731674624 A8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚3681 F=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚3681 SDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 3142752768 ZHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 4184769631744 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚23980 BDeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚387572670022144 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚6 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚19683 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 PCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚24973295 TGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚32719888 NFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚986 dQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚77700996476416 UADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚372289323694747 VBDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated‚387575211812864 ]IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚383387925818368 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚23681 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚11913878 H?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚1534 L?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚32700205 YJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 2516362752 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 IDeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 MDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚1534 UFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 3217031168 A8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚3681 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚25427968 ]JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚82931797576704 P@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 36985723397 YJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 2514370136 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚570 O?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 40290484224 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚3 SGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚7775194 YFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚82931699053056 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 963120128 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚69425664 RBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 40290484224 TGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚24973295 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 MDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚2147 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚23980 RCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 2541790720 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚25427968 KBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚2147 TDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 35913877504‹R T»2Á- _wandbruntime‚3113  _runtime‚3113GŠ¸ ý&Â÷M NADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚32718547 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 577553920 S?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚372497192490977 I@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚1534 G:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚57725441 REDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚24987185 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 TDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 37073453056 I@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚2147 trainer/global_step‚1749 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚7779560 RBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 35913877504 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 963120128 C6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚57705726 NADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚32738256 S?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚387788972188672 OBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚19700494 ZFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚368315367906784 M=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 40290484224 FDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 UEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 35913877504 XFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 4187125727744 LDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚330 K;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 38610163200 UCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 4187125727744 PADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 2541804032 REDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚32738256 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚25427968 ?6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚3681 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚2684 LDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚986 TEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 3140738484 _step‚140 PDeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚19699931 aODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 5233750221312 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚7779563 ?6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚3681 BDeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚19715 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚24987179 WCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚383601846460928 \JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 4189642103808 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 PADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 2539798312 XFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 4181824584193 % _timestamp‚1.7884883142866237e+09 N>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 38610163200 NADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚24987179 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚3 P@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 40290484224 ZFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚383601846460928 G:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚57705726 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚29097984 NADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚24987185 EDeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚57725441 YGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 4189642103808 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚19709 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚23681 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 YIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 37073453056 J=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚32718547 M>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 2541804032 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 WCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚387791513992704 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1309 WCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚372499732289289 N>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 36985723397 VGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 3217031168 F=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚2147 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚23980 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚6 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚11920931 UEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 34283657216 WHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 2514370344 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚19709 [GDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚383601871888896 WGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 37073453056 XIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 3217031168 T@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚387791513992704 ^JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚383601871888896 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚25427968 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚560 RCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 1515847168¸åžç'ÃüM I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚2684 LDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚986 TEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 3140738484 _step‚140 PDeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚19699931 aODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 5233750221312 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚7779563 ?6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚3681 BDeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚19715 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚24987179 WCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚383601846460928 \JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 4189642103808 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 PADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 2539798312 XFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 4181824584193 % _timestamp‚1.7884883142866237e+09 N>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 38610163200 NADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚24987179 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚3 P@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 40290484224 ZFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚383601846460928 G:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚57705726 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚29097984 NADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚24987185 EDeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚57725441 YGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 4189642103808 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚19709 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚23681 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 YIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 37073453056 J=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚32718547 M>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 2541804032 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 WCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚387791513992704 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1309 WCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚372499732289289 N>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 36985723397 VGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 3217031168 F=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚2147 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚23980 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚6 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚11920931 UEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 34283657216 WHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 2514370344 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚19709 [GDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚383601871888896 WGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 37073453056 XIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 3217031168 T@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚387791513992704 ^JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚383601871888896 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚25427968 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚560 RCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 1515847168 NADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚32718547 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 577553920 S?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚372497192490977 I@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚1534 G:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚57725441 REDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚24987185 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 TDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 37073453056 I@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚2147 trainer/global_step‚1749 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚7779560 RBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 35913877504 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 963120128 C6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚57705726 NADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚32738256 S?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚387788972188672 OBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚19700494 ZFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚368315367906784 M=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 40290484224 FDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 UEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 35913877504 XFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 4187125727744 LDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚330 K;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 38610163200 UCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 4187125727744 PADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 2541804032 REDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚32738256 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚25427968 ?6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚3681 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0ŒÎ’ÿd2Ä- _wandbruntime‚3113  _runtime‚3113³€K@[ÅÕ 1loss/FBt2/assignment_loss‚0.07518450395659472 .loss/FRt1/detection_loss‚2.211778804735102 _runtime‚3113.389865064 /loss/FRt1/assignment_loss‚3.410471834919669 2loss/SRqqt2/assignment_loss‚0.8678180375492148 1loss/FBt4/assignment_loss‚0.21633251889041907 0loss/SRqqt3/detection_loss‚2.417605224629523 _step‚141 2loss/SRqqt4/assignment_loss‚1.5622422097221715 .loss/FBt1/detection_loss‚3.170907155327175 % _timestamp‚1.7884883142887022e+09 0loss/FRt3/assignment_loss‚3.1901077733320347 2loss/SRqqt1/assignment_loss‚1.4461049903462182 1loss/FBt3/assignment_loss‚0.07302720030226434 /loss/FBt2/detection_loss‚0.6462650530360228 /loss/FBt3/detection_loss‚3.0680682102839154 /loss/FBt4/detection_loss‚2.5034845341806826 2loss/SRqqt3/assignment_loss‚1.7762849422197946 epoch‚2 trainer/global_step‚1749 /loss/FRt3/detection_loss‚2.8018813378670635 1loss/FBt1/assignment_loss‚0.20537639239742217 /loss/FRt2/detection_loss‚1.7669284633214932 0loss/SRqqt1/detection_loss‚2.311850006455806 %loss/total_loss‚42.84452419120415 /loss/FRt2/assignment_loss‚1.193067474537585 /loss/FRt4/detection_loss‚2.5152376732405495 0loss/SRqqt2/detection_loss‚1.114633099249904 0loss/SRqqt4/detection_loss‚1.673378188512288 0loss/FRt4/assignment_loss‚2.6264885621912337V°õ`ÆÚ 1loss/FBt1/assignment_loss‚0.20537639239742217 /loss/FRt2/detection_loss‚1.7669284633214932 0loss/SRqqt1/detection_loss‚2.311850006455806 %loss/total_loss‚42.84452419120415 /loss/FRt2/assignment_loss‚1.193067474537585 /loss/FRt4/detection_loss‚2.5152376732405495 0loss/SRqqt2/detection_loss‚1.114633099249904 0loss/SRqqt4/detection_loss‚1.673378188512288 0loss/FRt4/assignment_loss‚2.6264885621912337 1loss/FBt2/assignment_loss‚0.07518450395659472 .loss/FRt1/detection_loss‚2.211778804735102 _runtime‚3113.389865064 /loss/FRt1/assignment_loss‚3.410471834919669 2loss/SRqqt2/assignment_loss‚0.8678180375492148 1loss/FBt4/assignment_loss‚0.21633251889041907 0loss/SRqqt3/detection_loss‚2.417605224629523 _step‚141 2loss/SRqqt4/assignment_loss‚1.5622422097221715 .loss/FBt1/detection_loss‚3.170907155327175 % _timestamp‚1.7884883142887022e+09 0loss/FRt3/assignment_loss‚3.1901077733320347 2loss/SRqqt1/assignment_loss‚1.4461049903462182 1loss/FBt3/assignment_loss‚0.07302720030226434 /loss/FBt2/detection_loss‚0.6462650530360228 /loss/FBt3/detection_loss‚3.0680682102839154 /loss/FBt4/detection_loss‚2.5034845341806826 2loss/SRqqt3/assignment_loss‚1.7762849422197946 epoch‚2 trainer/global_step‚1749 /loss/FRt3/detection_loss‚2.8018813378670635פ`~KÇ‚(:À €ÕèÔÉЮ) disk./.usagePercent‚83.39455004358575 disk.rbd0.out‚ 0.62109375 disk.md0.out‚ 290.5234375 disk.rbd2.out‚ 0.783203125 proc.cpu.threads‚17 disk.nvme0n1.in‚4.59375 disk.sdb1.in‚0 disk.sdb2.in‚0 disk.md0p1.out‚ 290.5234375 disk.rbd2.in‚0 disk.sda2.in‚ 0.0078125( proc.memory.availableMB‚ 1807.9453125 disk.sdb2.out‚ 303.201171875) proc.memory.percent‚18.66137981414795 cpu‚25.475786902755843 network.sent‚ 4.028868e+06 disk.dm-0.out‚41076.36328125 disk.sdb1.out‚0 disk.sda1.out‚0 disk.sda2.out‚ 303.201171875 network.recv‚ 1.017283e+06 disk.nvme1n1.in‚4.515625 disk.dm-0.in‚9.109375 disk.md0.in‚ 0.0078125 disk.md0p1.in‚ 0.0078125 disk.sdb.in‚0# proc.memory.rssMB‚ 3057.48046875 disk.sdb.out‚ 303.201171875 disk.sda.in‚ 0.0078125 disk.rbd0.in‚0! disk.nvme0n1.out‚ 23213.734375 disk.sda1.in‚0$ memory_percent‚88.96517753601074% disk./.usageGB‚2982.5930519104004# disk.nvme1n1.out‚17862.62890625 disk.sda.out‚ 303.201171875Möä)”È‚(:‰ €ÕèÔˆÅØ® gpu.0.gpu‚98.0 gpu.0.memory‚18+ gpu.0.memoryAllocated‚47.54737854003906* gpu.0.memoryAllocatedBytes‚ 40842887168 gpu.0.temp‚45.0 gpu.0.powerWatts‚229.155' gpu.0.enforcedPowerLimitWatts‚400.0 gpu.0.powerPercent‚57.28875 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0:h͇‘É‚(:† ÕèÔŸÁ´® gpu.0.gpu‚32.0 gpu.0.memory‚9+ gpu.0.memoryAllocated‚47.54737854003906* gpu.0.memoryAllocatedBytes‚ 40842887168 gpu.0.temp‚41.0 gpu.0.powerWatts‚116.916' gpu.0.enforcedPowerLimitWatts‚400.0 gpu.0.powerPercent‚29.229 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0z‹gdEÊ‚(:º ÕèÔ„´ô® disk.sdb.out‚304.40869140625 proc.cpu.threads‚17) disk./.usagePercent‚83.38679816943223 disk.nvme0n1.in‚4.59375 disk.dm-0.in‚9.109375 disk.sda1.out‚0 disk.md0.in‚ 0.0078125 disk.nvme1n1.out‚17901.5! disk.sdb2.out‚304.40869140625 disk.md0.out‚ 291.6640625) proc.memory.percent‚18.66137981414795 disk.sda1.in‚0 disk.md0p1.in‚ 0.0078125 disk.rbd2.out‚ 0.783203125 disk.dm-0.out‚ 41237.9375% disk./.usageGB‚2982.3158073425293 disk.sdb1.in‚0 disk.rbd2.in‚0 disk.sdb.in‚0 cpu‚25.47494724010646 network.sent‚ 4.114869e+06 network.recv‚ 1.022017e+06 disk.sda.in‚ 0.0078125 disk.rbd0.in‚0 disk.rbd0.out‚ 0.62109375! disk.sda2.out‚304.40869140625$ memory_percent‚88.96269798278809( proc.memory.availableMB‚ 1808.3515625 disk.sda.out‚304.40869140625 disk.sdb1.out‚0 disk.sdb2.in‚0 disk.sda2.in‚ 0.0078125# proc.memory.rssMB‚ 3057.48046875 disk.nvme0n1.out‚ 23336.4375 disk.nvme1n1.in‚4.515625 disk.md0p1.out‚ 291.6640625¡,ÊJSË‚(:È žÕèÔ€çű! disk.sdb2.out‚306.09228515625 disk.md0p1.out‚ 293.296875 disk.rbd2.out‚ 0.783203125 disk.sda.out‚306.09228515625) proc.memory.percent‚18.66137981414795 cpu‚25.474391562041827) disk./.usagePercent‚83.38751748806085 disk.sdb2.in‚0 proc.cpu.threads‚17 disk.sdb.out‚306.09228515625 disk.nvme1n1.in‚4.515625 disk.dm-0.out‚ 41360.9140625 disk.sda1.in‚0 disk.md0.in‚ 0.0078125 disk.rbd2.in‚0# disk.nvme0n1.out‚23458.86328125! disk.sda2.out‚306.09228515625# disk.nvme1n1.out‚17902.05078125 disk.sda.in‚ 0.0078125$ memory_percent‚88.96338939666748 disk.rbd0.in‚0 disk.rbd0.out‚ 0.62109375 disk.sdb1.out‚0 disk.md0p1.in‚ 0.0078125 disk.sda2.in‚ 0.0078125# proc.memory.rssMB‚ 3057.48046875 network.recv‚ 1.026437e+06 disk.sdb1.in‚0 network.sent‚ 4.120836e+06% disk./.usageGB‚2982.3415336608887 disk.dm-0.in‚9.109375 disk.sda1.out‚0 disk.nvme0n1.in‚4.59375 disk.md0.out‚ 293.296875 disk.sdb.in‚0) proc.memory.availableMB‚ 1808.238281255Äï”Ì‚(:‰ žÕèÔ膋± gpu.0.gpu‚58.0 gpu.0.memory‚13+ gpu.0.memoryAllocated‚47.54737854003906* gpu.0.memoryAllocatedBytes‚ 40842887168 gpu.0.temp‚46.0 gpu.0.powerWatts‚306.715' gpu.0.enforcedPowerLimitWatts‚400.0 gpu.0.powerPercent‚76.67875 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0âÃQPžÍ‚(:“ ­ÕèÔ‚Þï® gpu.0.gpu‚98.0 gpu.0.memory‚18+ gpu.0.memoryAllocated‚47.54737854003906* gpu.0.memoryAllocatedBytes‚ 40842887168 gpu.0.temp‚45.0 gpu.0.powerWatts‚237.326' gpu.0.enforcedPowerLimitWatts‚400.0) gpu.0.powerPercent‚59.331500000000005 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0Üp1ãQ΂(:Æ ­ÕèÔ¡§î¯ disk.sda1.in‚0 disk.md0p1.in‚ 0.0078125 disk.md0p1.out‚ 295.12890625 disk.rbd2.out‚ 0.783203125 disk.dm-0.out‚ 41551.3515625 disk.sdb2.in‚0 disk.nvme1n1.in‚4.515625 disk.rbd0.in‚0 disk.dm-0.in‚9.109375$ memory_percent‚88.96420001983643# proc.memory.rssMB‚ 3057.48046875 cpu‚25.47344290935883 disk.md0.in‚ 0.0078125 disk.md0.out‚ 295.12890625 disk.sda2.in‚ 0.0078125 disk.sda.out‚307.99462890625) proc.memory.availableMB‚ 1808.10546875) disk./.usagePercent‚83.38843828843466 disk.sdb1.out‚0 disk.sdb.in‚0 disk.nvme1n1.out‚ 17902.71875 proc.cpu.threads‚17! disk.sdb2.out‚307.99462890625 disk.sdb.out‚307.99462890625) proc.memory.percent‚18.66137981414795 network.sent‚ 4.126643e+06 network.recv‚ 1.030247e+06! disk.sda2.out‚307.99462890625 disk.sda.in‚ 0.0078125 disk.nvme0n1.in‚4.59375 disk.rbd0.out‚ 0.62109375$ disk./.usageGB‚2982.374465942383 disk.sdb1.in‚0 disk.sda1.out‚0 disk.rbd2.in‚0" disk.nvme0n1.out‚ 23648.6328125ù¥ žœÏ‚(:‘ ¼ÕèÔ桾¯ gpu.0.gpu‚98.0 gpu.0.memory‚17+ gpu.0.memoryAllocated‚47.54737854003906* gpu.0.memoryAllocatedBytes‚ 40842887168 gpu.0.temp‚45.0 gpu.0.powerWatts‚250.62' gpu.0.enforcedPowerLimitWatts‚400.0( gpu.0.powerPercent‚62.65500000000001 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0üî6eЂ(:Ç ¼ÕèÔÙ’å¯ disk.sda1.out‚0) proc.memory.percent‚18.66137981414795 disk.sda¹Ú°Ûí2.in‚ 0.0078125 disk.dm-0.out‚41776.30859375 disk.sdb1.out‚0 proc.cpu.threads‚17 network.sent‚ 4.133035e+06% disk./.usageGB‚2981.5314292907715 disk.md0p1.in‚ 0.0078125 disk.rbd2.out‚ 0.783203125 disk.sdb.in‚0 disk.rbd0.out‚ 0.62109375 disk.sdb1.in‚0 disk.md0.in‚ 0.0078125 disk.sdb.out‚309.32470703125 disk.dm-0.in‚9.109375 disk.sda1.in‚0 disk.md0.out‚ 296.390625# proc.memory.rssMB‚ 3057.48046875 disk.rbd2.in‚0! disk.sda2.out‚309.32470703125 disk.rbd0.in‚0 network.recv‚ 1.034985e+06 disk.nvme1n1.in‚4.515625! disk.sdb2.out‚309.32470703125# memory_percent‚88.9657735824585 cpu‚25.472865970788614" disk.nvme1n1.out‚ 17902.8515625 disk.sda.in‚ 0.0078125 disk.nvme0n1.in‚4.59375) proc.memory.availableMB‚ 1807.84765625) disk./.usagePercent‚83.36486663081735 disk.md0p1.out‚ 296.390625 disk.sda.out‚309.32470703125# disk.nvme0n1.out‚23873.45703125 disk.sdb2.in‚0agà±2Ñ- _wandbruntime‚3194  _runtime‚3194;ó/ ÆÒÀ $ _timestamp‚1.788488395886379e+09 _runtime‚3194.987909365 _step‚142 & lr-AdamW/pg1‚0.0010070258590197092 & lr-AdamW/pg2‚0.0010070258590197092 trainer/global_step‚17990jCFËÓÅ $ _timestamp‚1.788488395886379e+09 _runtime‚3194.987909365 _step‚142 & lr-AdamW/pg1‚0.0010070258590197092 & lr-AdamW/pg2‚0.0010070258590197092 trainer/global_step‚1799ŽýþKn2Ô- _wandbruntime‚3194  _runtime‚3194ž5–ó'ÕíO [KDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 37073453056 % _timestamp‚1.7884883958872159e+09 IDeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 TDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 40290484224 KBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚2147 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚23980 P@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 38610163200 ZKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 3217031168 _step‚143 XIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 3217031168 RCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 2541790720 UADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚398387806176768 TGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚25667795 M=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 38610163200 VGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 2516362752 ^LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 4305118622720 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚19689 RCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 2539798104 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 UADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚382682790678255 PCDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated‚33640340 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚2684 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 577553920 VGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 3140738484 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚23681 WGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 35913877504 PCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚33620657 YEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚382685330476359 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚19689 ]JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚85285970238976 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 ]IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚394085229344768 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 H?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚2147 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚25427968 ZHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 4302602259968 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚12244746 YFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚85285871715328 `MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚79907483628032 BDeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚398387806176768 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚25427968 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚567 RBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 40290484224 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 L?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚25667789 SDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 3142752768 TDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 35913877504 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚69425664 _MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 5378388087296 WGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 34283657216 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚19683 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚23681 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚570 YEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚394085203916800 VGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 3142752768 M@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚20237673 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚6 trainer/global_step‚1799 WEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 4302602259968 F=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚3681 MDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚2147 `LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚378385660699056 G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚2684 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚23980 PCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚25667789 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 MDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚1534 QDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚20238243 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚12244179 VFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 37073453056 H?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚1534 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚25427968 NFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚330 UHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚98523648 ^LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 4299669777303 PCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚25667795 IDeviceStatsMonitor.on_train_batch_start/allocation.all.current‚19689 RCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 2539798104 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 UADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚382682790678255 PCDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated‚33640340 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚2684 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 577553920 VGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 3140738484 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚23681 WGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 35913877504 PCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚33620657 YEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚382685330476359 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚19689 ]JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚85285970238976 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 ]IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚394085229344768 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 H?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚2147 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚25427968 ZHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 4302602259968 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚12244746 YFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚85285871715328 `MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚79907483628032 BDeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚398387806176768 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚25427968 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚567 RBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 40290484224 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 L?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚25667789 SDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 3142752768 TDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 35913877504 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚69425664 _MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 5378388087296 WGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 34283657216 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚19683 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚23681 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚570 YEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚394085203916800 VGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 3142752768 M@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚20237673 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚6 trainer/global_step‚1799 WEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 4302602259968 F=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚3681 MDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚2147 `LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚378385660699056 G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚2684 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚23980 PCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚25667789 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 MDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚1534 QDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚20238243 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚12244179 VFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 37073453056 H?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚1534 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚25427968 NFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚330 UHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚98523648 ^LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 4299669777303 PCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚25667795 IDeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 TDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 40290484224 KBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚2147 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚23980 P@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 38610163200 ZKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 3217031168 _step‚143 XIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 3217031168 RCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 2541790720 UADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚398387806176768 TGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚25667795 M=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 38610163200 VGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 2516362752 ^LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 4305118622720`Õò’ׂ(:‡ ËÕèÔ¡Û¿¯ gpu.0.gpu‚0.0 gpu.0.memory‚0+ gpu.0.memoryAllocated‚47.54737854003906* gpu.0.memoryAllocatedBytes‚ 40842887168 gpu.0.temp‚40.0 gpu.0.powerWatts‚117.193' gpu.0.enforcedPowerLimitWatts‚400.0 gpu.0.powerPercent‚29.29825 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0A̸ALØ‚(:Á ËÕèÔÅÇý¯ disk.nvme0n1.out‚ 24091.34375 disk.sdb1.out‚0 disk.sdb2.in‚0 disk.sda1.in‚0 disk.md0p1.out‚ 297.96875' proc.memory.availableMB‚ 1807.828125 disk.dm-0.in‚9.109375! disk.sdb2.out‚310.96142578125 disk.sda1.out‚0$ memory_percent‚88.96589279174805 cpu‚25.472600975168238 disk.sda2.in‚ 0.0078125! disk.sda2.out‚310.96142578125 disk.sda.in‚ 0.0078125 disk.md0.in‚ 0.0078125( disk./.usagePercent‚83.3648385790974 disk.nvme1n1.in‚4.515625 disk.sda.out‚310.96142578125 disk.nvme0n1.in‚4.59375 disk.sdb1.in‚0 disk.sdb.in‚0 disk.rbd0.in‚0 disk.rbd2.in‚0# proc.memory.rssMB‚ 3057.48046875 network.recv‚ 1.038618e+06 disk.rbd2.out‚ 0.783203125 disk.sdb.out‚310.96142578125 disk.rbd0.out‚ 0.62109375 disk.dm-0.out‚41994.37890625 disk.md0p1.in‚ 0.0078125 proc.cpu.threads‚17% disk./.usageGB‚2981.5304260253906 disk.md0.out‚ 297.96875) proc.memory.percent‚18.66137981414795 network.sent‚ 4.138233e+06# disk.nvme1n1.out‚17903.03515625ßÇ42Ù- _wandbruntime‚3196  _runtime‚3196ä,ÈÏÖ#ÚøM TEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 3142752768 F=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚2147 KBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚1534 PADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 2541804032 N>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 38610163200 REDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚33658708 WDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚85332921866240 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 P@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 40290484224 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚29097984 S?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚398604108343296 OBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚20249136 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 VGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 3217031168 ?6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚3681 I@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚1534 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚19715 [GDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚394299175415296 F8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 RBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 40290484224 C6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚59320678 RBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 35913877504 TEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 2516376064 UEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 35913877504 ^KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚79951613953024 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 _runtime‚3196.501388004 J=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚33638999 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚2684 [HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚85333028765184 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 FDeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚59340393 WCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚382893199272797 XIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 3217031168 WHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 2516376064 ?6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚3681 TDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 37073453056 % _timestamp‚1.7884883973992357e+09 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚574 KBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚2147 I@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚2147 trainer/global_step‚1799 ^JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚394299175415296 NADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚25681685 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚19709 ]KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 5381307913216 TEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 3140738484 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚23980 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚7997863 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚571 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚77800960 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 UCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 4304958355968 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 106898944 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 M=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 40290484224 YIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 37073453056 WCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚394299149987328 NADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚25681679 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚12251273 bODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚79951643051008 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 577553920 SDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 3217031168 B9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚3681 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚25427968 UEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 34283657216 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚6 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚3 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 963120128 ZFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚378591150931584 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚23980 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚7997860 F=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚1534 _step‚144 T@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚398606650147328 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 N>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 36985723397 D;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚3681 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 M>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 2541804032 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚6 K>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚2024856×8;w(2 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1309 NADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚33638999 LDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚986 ^JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚378591176359552 YGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 4307474732032 XFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 4304958355968 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 NADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚33658708 K;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 38610163200Ýåɬ'ÛýM KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 UCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 4304958355968 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 106898944 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 M=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 40290484224 YIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 37073453056 WCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚394299149987328 NADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚25681679 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚12251273 bODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚79951643051008 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 577553920 SDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 3217031168 B9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚3681 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚25427968 UEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 34283657216 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚6 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚3 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 963120128 ZFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚378591150931584 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚23980 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚7997860 F=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚1534 _step‚144 T@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚398606650147328 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 N>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 36985723397 D;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚3681 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 M>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 2541804032 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚6 K>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚20248562 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1309 NADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚33638999 LDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚986 ^JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚378591176359552 YGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 4307474732032 XFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 4304958355968 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 NADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚33658708 K;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 38610163200 TEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 3142752768 F=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚2147 KBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚1534 PADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 2541804032 N>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 38610163200 REDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚33658708 WDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚85332921866240 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 P@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 40290484224 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚29097984 S?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚398604108343296 OBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚20249136 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 VGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 3217031168 ?6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚3681 I@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚1534 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚19715 [GDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚394299175415296 F8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 RBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 40290484224 C6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚59320678 RBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 35913877504 TEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 2516376064 UEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 35913877504 ^KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚79951613953024 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 _runtime‚3196.501388004 J=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚33638999 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚2684 [HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚85333028765184 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 FDeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚59340393 WCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚382893199272797 XIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 3217031168 WHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 2516376064 ?6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚3681 TDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 37073453056 % _timestamp‚1.7884883973992357e+09 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚574 KBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚2147 I@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚2147 trainer/global_step‚1799 ^JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚394299175415296 NADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚25681685 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚19709 ]KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 5381307913216 TEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 3140738484 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚23980 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚7997863 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚571 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚77800960 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0C„lë2Ü- _wandbruntime‚3196  _runtime‚3196k2IDXÝÒ 0loss/SRqqt1/detection_loss‚2.259281449019909 .loss/FBt3/detection_loss‚2.553791637604053 _runtime‚ 3196.50258533 2loss/SRqqt2/assignment_loss‚1.0583771270115014 .loss/FRt4/detection_loss‚2.267729576271359 epoch‚2 % _timestamp‚1.7884883974014351e+09 /loss/FRt2/detection_loss‚1.5317690307851386 2loss/SRqqt1/assignment_loss‚1.7594332641934696 1loss/SRqqt4/detection_loss‚1.7839126771639202 2loss/SRqqt3/assignment_loss‚1.7901898960654552 2loss/SRqqt4/assignment_loss‚1.5717890418388627 /loss/FBt2/detection_loss‚0.6842935164305535 .loss/FBt4/detection_loss‚2.675312345936185 _step‚145 1loss/FBt2/assignment_loss‚0.14873986357354455 0loss/SRqqt3/detection_loss‚2.226454467536547 &loss/total_loss‚41.296426777845944 /loss/FRt3/assignment_loss‚2.399942438571881 .loss/FRt3/detection_loss‚2.566227287818224 trainer/global_step‚1799 .loss/FRt1/detection_loss‚2.501390741388688 0loss/FRt1/assignment_loss‚3.3524336609347114 1loss/FBt3/assignment_loss‚0.30538664469853616 1loss/FBt4/assignment_loss‚0.23342340157125277 .loss/FBt1/detection_loss‚2.307587307233077 1loss/SRqqt2/detection_loss‚1.0833031977851724 0loss/FRt2/assignment_loss‚1.3960768617359938 /loss/FRt4/assignment_loss‚2.677212686231821 0loss/FBt1/assignment_loss‚0.1623686564460933ÇÃà¤]Þ× /loss/FRt4/assignment_loss‚2.677212686231821 0loss/FBt1/assignment_loss‚0.1623686564460933 0loss/SRqqt1/detection_loss‚2.259281449019909 .loss/FBt3/detection_loss‚2.553791637604053 _runtime‚ 3196.50258533 2loss/SRqqt2/assignment_loss‚1.0583771270115014 .loss/FRt4/detection_loss‚2.267729576271359 epoch‚2 % _timestamp‚1.7884883974014351e+09 /loss/FRt2/detection_loss‚1.5317690307851386 2loss/SRqqt1/assignment_loss‚1.7594332641934696 1loss/SRqqt4/detection_loss‚1.7839126771639202 2loss/SRqqt3/assignment_loss‚1.7901898960654552 2loss/SRqqt4/assignment_loss‚1.5717890418388627 /loss/FBt2/detection_loss‚0.6842935164305535 .loss/FBt4/detection_loss‚2.675312345936185 _step‚145 1loss/FBt2/assignment_loss‚0.14873986357354455 0loss/SRqqt3/detection_loss‚2.226454467536547 &loss/total_loss‚41.296426777845944 /loss/FRt3/assignment_loss‚2.399942438571881 .loss/FRt3/detection_loss‚2.566227287818224 trainer/global_step‚1799 .loss/FRt1/detection_loss‚2.501390741388688 0loss/FRt1/assignment_loss‚3.3524336609347114 1loss/FBt3/assignment_loss‚0.30538664469853616 1loss/FBt4/assignment_loss‚0.23342340157125277 .loss/FBt1/detection_loss‚2.307587307233077 1loss/SRqqt2/detection_loss‚1.0833031977851724 0loss/FRt2/assignment_loss‚1.3960768617359938‘•žÒ*“ß‚(:ˆ ÚÕèÔŒƒÖ¯ gpu.0.gpu‚98.0 gpu.0.memory‚23+ gpu.0.memoryAllocated‚47.54737854003906* gpu.0.memoryAllocatedBytes‚ 40842887168 gpu.0.temp‚45.0 gpu.0.powerWatts‚311.386' gpu.0.enforcedPowerLimitWatts‚400.0 gpu.0.powerPercent‚77.8465 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0CÌáÙSà‚(:È ÚÕèÔ‡éÕ° disk.md0.out‚ 299.2421875 disk.md0p1.in‚ 0.0078125# disk.nvme1n1.out‚17903.16796875 disk.rbd0.out‚ 0.62109375 disk.dm-0.out‚ 42038.5703125! disk.sdb2.out‚312.28857421875$ memory_percent‚88.96462917327881 proc.cpu.threads‚17 network.recv‚ 1.044252e+06 disk.sda.out‚312.28857421875 disk.sda1.out‚0# disk.nvme0n1.out‚24135.40234375 disk.dm-0.in‚9.109375 disk.sdb2.in‚0# proc.memory.rssMB‚ 3057.48046875! disk.sda2.out‚312.28857421875 network.sent‚ 4.226583e+06) disk./.usagePercent‚83.36483985902379 disk.md0.in‚ 0.0078125 disk.rbd0.in‚0 disk.sdb1.in‚0$ disk./.usageGB‚2981.530471801758 disk.sdb.out‚312.28857421875 disk.sda2.in‚ 0.0078125 disk.rbd2.in‚0 disk.rbd2.out‚ 0.783203125 disk.sdb.in‚0 disk.sda1.in‚0) proc.memory.availableMB‚ 1808.03515625) proc.memory.percent‚18.66137981414795 cpu‚25.47225299834577 disk.sdb1.out‚0 disk.md0p1.out‚ 299.2421875 disk.nvme1n1.in‚4.515625 disk.sda.in‚ 0.0078125 disk.nvme0n1.in‚4.59375 †-”á‚(:‰ éÕèÔ…ƒ§° gpu.0.gpu‚95.0 gpu.0.memory‚21+ gpu.0.memoryAllocated‚47.54737854003906* gpu.0.memoryAllocatedBytes‚ 40842887168 gpu.0.temp‚46.0 gpu.0.powerWatts‚306.715' gpu.0.enforcedPowerLimitWatts‚400.0 gpu.0.powerPercent‚76.67875 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0(ç?Hâ‚(:½ éÕèÔ–Êï± disk.nvme1n1.in‚4.515625 network.sent‚ 4.29206e+06 disk.sdb.out‚313.43017578125 disk.rbd0.in‚0! disk.nvme0n1.out‚ 24252.265625 disk.md0p1.in‚ 0.0078125 disk.dm-0.out‚ 42155.6796875 proc.cpu.threads‚17 network.recv‚ 1.049232e+06) disk./.usagePercent‚83.36483836577634 disk.sdb1.out‚0" proc.memory.rssMB‚ 3057.6015625 disk.md0.in‚ 0.0078125 disk.rbd2.out‚ 0.783203125 disk.sda2.in‚ 0.0078125 disk.sdb1.in‚0 disk.sda1.in‚0 disk.md0p1.out‚ 300.34375" disk.nvme1n1.out‚ 17903.4140625 disk.rbd0.out‚ 0.62109375 disk.dm-0.in‚9.109375 cpu‚25.48436303351632$ disk./.usageGB‚2981.530418395996 disk.rbd2.in‚0 disk.sdb.in‚0! disk.sda2.out‚313.43017578125 disk.sda.in‚ 0.0078125 disk.sda.out‚313.43017578125' proc.memory.availableMB‚ 1807.921875! disk.sdb2.out‚313.43017578125 disk.sda1.out‚0 disk.md0.out‚ 300.34375 disk.nvme0n1.in‚4.59375# memory_percent‚88.9653205871582* proc.memory.percent‚18.662118911743164 disk.sdb2.in‚04&Purity/*FRt*SRqqt2FBt/event_proportion‚0.06747731566429138 ?&Purity/*FRt*SRqqt3FBt/event_proportion‚0.004473286215215921 ("Purity/0FRt3SRqqt1FBt/event_purity‚0 :"Purity/*FRt*SRqqt*FBt/event_purity‚0.21076183021068573 :"Purity/0FRt1SRqqt2FBt/SRqqt_purity‚0.45765870809555054 7 Purity/1FRt1SRqqt*FBt/FBt_purity‚0.9867773652076721 @&Purity/3FRt*SRqqt1FBt/event_proportion‚0.0024571572430431843 'particle/f_score‚0.3524930775165558 +"Purity/0FRt4SRqqt0FBt/SRqqt_purity‚0.25 & Purity/1FRt2SRqqt2FBt/FBt_purity‚1 :"Purity/2FRt3SRqqt1FBt/SRqqt_purity‚0.36666667461395264 A&Purity/1FRt*SRqqt4FBt/event_proportion‚0.00012600806076079607 8 Purity/2FRt0SRqqt*FBt/FRt_purity‚0.23756299912929535 ,&Purity/3FRt4SRqqt3FBt/event_proportion‚0 jet/accuracy_4_of_6‚0 * Purity/1FRt*SRqqt4FBt/FBt_purity‚0.875 ("Purity/2FRt0SRqqt3FBt/event_purity‚0 ("Purity/*FRt1SRqqt4FBt/event_purity‚0 +"Purity/1FRt*SRqqt3FBt/SRqqt_purity‚0.75 +jet/accuracy_4_of_5‚0.01648745685815811 + Purity/0FRt0SRqqt4FBt/FBt_purity‚0.8125 9"Purity/*FRt1SRqqt3FBt/SRqqt_purity‚0.6190475821495056 ("Purity/4FRt1SRqqt0FBt/SRqqt_purity‚0 @&Purity/*FRt2SRqqt3FBt/event_proportion‚6.300403038039804e-05 7"Purity/*FRt1SRqqt1FBt/SRqqt_purity‚0.51287442445755 ("Purity/2FRt3SRqqt0FBt/event_purity‚0 7 Purity/3FRt1SRqqt1FBt/FRt_purity‚0.3717948794364929 @&Purity/4FRt0SRqqt*FBt/event_proportion‚6.300403038039804e-05 + Purity/4FRt*SRqqt0FBt/FRt_purity‚0.0625 +"Purity/4FRt*SRqqt0FBt/SRqqt_purity‚0.25 >&Purity/1FRt0SRqqt0FBt/event_proportion‚0.13142640888690948 :"Purity/2FRt0SRqqt0FBt/event_purity‚0.02127496339380741 :"Purity/3FRt*SRqqt*FBt/SRqqt_purity‚0.38419535756111145 8 Purity/3FRt*SRqqt0FBt/FRt_purity‚0.20079366862773895 ("Purity/3FRt0SRqqt0FBt/event_purity‚0 *jet/accuracy_1_of_4‚0.8894588351249695 9"Purity/1FRt3SRqqt2FBt/SRqqt_purity‚0.3333333432674408 ) Purity/4FRt2SRqqt0FBt/FRt_purity‚0.25 8 Purity/*FRt2SRqqt0FBt/FRt_purity‚0.25328224897384644 ,&Purity/*FRt2SRqqt4FBt/event_proportion‚0 @&Purity/0FRt*SRqqt3FBt/event_proportion‚0.0030871976632624865 7 Purity/2FRt*SRqqt1FBt/FBt_purity‚0.9905914068222046 9"Purity/2FRt2SRqqt1FBt/SRqqt_purity‚0.5270833373069763 ("Purity/0FRt4SRqqt*FBt/event_purity‚0 >&Purity/*FRt*SRqqt1FBt/event_proportion‚0.27079132199287415 ;"Purity/*FRt2SRqqt0FBt/event_purity‚0.016523730009794235 :"Purity/1FRt1SRqqt0FBt/SRqqt_purity‚0.32849568128585815 8 Purity/1FRt1SRqqt2FBt/FRt_purity‚0.34481292963027954 ,&Purity/3FRt1SRqqt4FBt/event_proportion‚0 7 Purity/*FRt4SRqqt*FBt/FRt_purity‚0.3055555522441864 ("Purity/*FRt3SRqqt*FBt/event_purity‚0 7 Purity/1FRt3SRqqt1FBt/FRt_purity‚0.2888888716697693 ,&Purity/1FRt3SRqqt3FBt/event_proportion‚0 8 Purity/2FRt2SRqqt1FBt/FRt_purity‚0.20744049549102783 ,&Purity/4FRt3SRqqt4FBt/event_proportion‚0 ("Purity/0FRt3SRqqt*FBt/event_purity‚0 :"Purity/1FRt*SRqqt0FBt/SRqqt_purity‚0.33100688457489014 :"Purity/0FRt*SRqqt*FBt/event_purity‚0.39606592059135437 ,&Purity/0FRt4SRqqt3FBt/event_proportion‚0 7 Purity/2FRt*SRqqt*FBt/FRt_purity‚0.2494080513715744 ,&Purity/3FRt4SRqqt4FBt/event_proportion‚0 :"Purity/0FRt3SRqqt0FBt/SRqqt_purity‚0.32956352829933167 & Purity/2FRt3SRqqt*FBt/FBt_purity‚1 jet/accuracy_4_of_7‚0 :"Purity/0FRt1SRqqt0FBt/event_purity‚0.32592734694480896 @&Purity/4FRt2SRqqt*FBt/event_proportion‚6.300403038039804e-05 particle/accuracy_2_of_3‚1 ,&Purity/1FRt2SRqqt4FBt/event_proportion‚0 & Purity/*FRt4SRqqt1FBt/FBt_purity‚1 jet/accuracy_1_of_7‚1 :"Purity/*FRt1SRqqt1FBt/event_purity‚0.09071654826402664 ?&Purity/1FRt1SRqqt2FBt/event_proportion‚0.006111391354352236 ,&Purity/2FRt*SRqqt4FBt/event_proportion‚0 trainer/global_step‚1811 7 Purity/*FRt1SRqqt1FBt/FBt_purity‚0.9930967688560486 ?&Purity/0FRt2SRqqt0FBt/event_proportion‚0.043283771723508835 7 Purity/*FRt0SRqqt1FBt/FBt_purity‚0.9986216425895691 ,&Purity/0FRt2SRqqt4FBt/event_proportion‚0 ,&Purity/2FRt3SRqqt4FBt/event_proportion‚0 *jet/accuracy_2_of_4‚0.3547448515892029 @&Purity/3FRt1SRqqt0FBt/event_proportion‚0.0025201612152159214 7 Purity/1FRt1SRqqt2FBt/FBt_purity‚0.9726190567016602 7 Purity/1FRt2SRqqt*FBt/FBt_purity‚0.9896313548088074 >&Purity/2FRt1SRqqt*FBt/event_proportion‚0.04158266261219978 ,&Purity/3FRt*SRqqt3FBt/event_proportion‚0 :"Purity/*FRt2SRqqt2FBt/SRqqt_purity‚0.42105263471603394 ("Purity/2FRt3SRqqt1FBt/event_purity‚0 7 Purity/1FRt1SRqqt3FBt/FBt_purity‚0.8333333730697632 7 Purity/*FRt1SRqqt*FBt/FRt_purity‚0.2747350335121155 8 Purity/*FRt2SRqqt1FBt/FRt_purity‚0.25289416313171387 >&Purity/0FRt*SRqqt1FBt/event_proportion‚0.14787046611309052 :"Purity/2FRt*SRqqt2FBt/event_purity‚0.03333333507180214 @&Purity/3FRt0SRqqt*FBt/event_proportion‚0.0027091733645647764 ,&Purity/4FRt1SRqqt1FBt/event_proportion‚0 particle/accuracy_1_of_5‚1 9"Purity/0FRt3SRqqt1FBt/SRqqt_purity‚0.3125000298023224 7 Purity/3FRt*SRqqt1FBt/FRt_purity‚0.3083333671092987 >&Purity/1FRt*SRqqt1FBt/event_proportion‚0.09929435700178146 *loss/val_total_loss‚44.310995154527745 :"Purity/0FRt*SRqqt3FBt/event_purity‚0.11728394776582718 9"Purity/0FRt2SRqqt1FBt/SRqqt_purity‚0.4510112702846527 7 Purity/1FRt4SRqqt*FBt/FRt_purity‚0.3333333432674408 ("Purity/3FRt0SRqqt2FBt/event_purity‚0 ("Purity/2FRt2SRqqt2FBt/event_purity‚0 9"Purity/0FRt2SRqqt0FBt/SRqqt_purity‚0.3387606143951416 6 Purity/1FRt3SRqqt0FBt/FRt_purity‚0.353532999753952 @&Purity/1FRt4SRqqt*FBt/event_proportion‚0.0003780241822823882 7 Purity/2FRt2SRqqt0FBt/FRt_purity‚0.2524426281452179 ("Purity/3FRt*SRqqt0FBt/event_purity‚0 ?&Purity/2FRt*SRqqt1FBt/event_proportion‚0.021169355139136314 9"Purity/2FRt*SRqqt1FBt/SRqqt_purity‚0.5178669095039368 ,&Purity/2FRt2SRqqt4FBt/event_proportion‚0 9"Purity/1FRt*SRqqt1FBt/event_purity‚0.1225278303027153 6 Purity/1FRt0SRqqt*FBt/FBt_purity‚0.991801381111145 8 Purity/*FRt0SRqqt*FBt/FRt_purity‚0.24821753799915314 :"Purity/*FRt0SRqqt3FBt/event_purity‚0.13214285671710968 ?&Purity/*FRt2SRqqt2FBt/event_proportion‚0.002079133177176118 ?&Purity/*FRt3SRqqt1FBt/event_proportion‚0.002142137149348855 8 Purity/1FRt0SRqqt0FBt/FRt_purity‚0.18158066272735596 7 Purity/2FRt1SRqqt*FBt/FRt_purity‚0.2553845942020416 9"Purity/0FRt2SRqqt*FBt/SRqqt_purity‚0.3570649325847626 ,&Purity/4FRt0SRqqt1FBt/event_proportion‚0 @&Purity/4FRt1SRqqt*FBt/event_proportion‚6.300403038039804e-05 7 Purity/0FRt2SRqqt2FBt/FBt_purity‚0.9807692170143127 *"Purity/0FRt1SRqqt3FBt/SRqqt_purity‚0.5 >&Purity/0FRt*SRqqt2FBt/event_proportion‚0.03944052383303642 A&Purity/0FRt*SRqqt4FBt/event_proportion‚0.00025201612152159214 7 Purity/0FRt0SRqqt1FBt/FBt_purity‚0.9985072016716003 :"Purity/0FRt2SRqqt*FBt/event_purity‚0.03772607445716858 & Purity/1FRt3SRqqt*FBt/FBt_purity‚1 7 Purity/2FRt2SRqqt*FBt/FBt_purity‚0.9821428656578064 ?&Purity/3FRt1SRqqt*FBt/event_proportion‚0.003717237850651145 jet/accuracy_6_of_6‚0 8 Purity/*FRt*SRqqt3FBt/FRt_purity‚0.44117647409439087 7 Purity/1FRt*SRqqt1FBt/FBt_purity‚0.9968535304069519 ,&Purity/2FRt4SRqqt1FBt/event_proportion‚0 8 Purity/3FRt2SRqqt*FBt/FRt_purity‚0.20023149251937866 ,&Purity/4FRt1SRqqt3FBt/event_proportion‚0 ("Purity/*FRt4SRqqt0FBt/event_purity‚0 8 Purity/2FRt4SRqqt*FBt/FRt_purity‚0.30000001192092896 6 Purity/3FRt3SRqqt*FBt/FRt_purity‚0.380952388048172 7 Purity/0FRt1SRqqt*FBt/FBt_purity‚0.9837377071380615 +jet/accuracy_3_of_6‚0.12280702590942383 ) Purity/3FRt1SRqqt2FBt/FRt_purity‚0.25 7 Purity/0FRt2SRqqt3FBt/FBt_purity‚0.6666666865348816 =&Purity/*FRt*SRqqt*FBt/event_proportion‚0.9591733813285828 8 Purity/1FRt2SRqqt0FBt/FRt_purity‚0.26159390807151794 ("Purity/1FRt2SRqqt1FBt/event_purity‚0 9"Purity/*FRt3SRqqt2FBt/SRqqt_purity‚0.3333333432674408 @&Purity/*FRt1SRqqt4FBt/event_proportion‚6.300403038039804e-05 >&Purity/0FRt*SRqqt0FBt/event_proportion‚0.21931703388690948 7 Purity/2FRt1SRqqt1FBt/FBt_purity‚0.9919354915618896 9"Purity/3FRt*SRqqt1FBt/SRqqt_purity‚0.6666666865348816 +"Purity/0FRt4SRqqt1FBt/SRqqt_purity‚0.25 & Purity/*FRt*SRqqt4FBt/FRt_purity‚0 ("Purity/1FRt1SRqqt2FBt/event_purity‚0 ?&Purity/2FRt0SRqqt0FBt/event_proportion‚0.024571573361754417 particle/accuracy_2_of_5‚1 7 Purity/2FRt3SRqqt1FBt/FRt_purity‚0.3499999940395355 +"Purity/3FRt4SRqqt0FBt/SRqqt_purity‚0.25 + Purity/0FRt3SRqqt*FBt/FBt_purity‚0.8125 8 Purity/1FRt*SRqqt*FBt/FRt_purity‚0.26818332076072693 @&Purity/1FRt*SRqqt3FBt/event_proportion‚0.0013230846961960196 ,&Purity/2FRt3SRqqt2FBt/event_proportion‚0 & Purity/3FRt2SRqqt1FBt/FBt_purity‚1 :"Purity/*FRt*SRqqt1FBt/event_purity‚0.39481866359710693 9"Purity/0FRt0SRqqt*FBt/event_purity‚0.8147090673446655 9"Purity/*FRt3SRqqt1FBt/SRqqt_purity‚0.3928571343421936 ?&Purity/0FRt0SRqqt3FBt/event_proportion‚0.002583165420219302 8 Purity/1FRt2SRqqt1FBt/FRt_purity‚0.26589861512184143 8 Purity/3FRt0SRqqt*FBt/FRt_purity‚0.24242424964904785 ,&Purity/4FRt2SRqqt1FBt/event_proportion‚0 ,&Purity/4FRt3SRqqt*FBt/event_proportion‚0 jet/accuracy_5_of_6‚0 9"Purity/*FRt*SRqqt2FBt/SRqqt_purity‚0.5245307683944702 :"Purity/0FRt*SRqqt*FBt/SRqqt_purity‚0.35810303688049316 9"Purity/0FRt*SRqqt1FBt/event_purity‚0.6384262442588806 A&Purity/1FRt1SRqqt3FBt/event_proportion‚0.00025201612152159214 9"Purity/0FRt2SRqqt2FBt/SRqqt_purity‚0.4423076808452606 & Purity/3FRt1SRqqt2FBt/FBt_purity‚1 & Purity/2FRt0SRqqt3FBt/FRt_purity‚0 8"Purity/1FRt2SRqqt0FBt/SRqqt_purity‚0.346381813287735 ?&Purity/2FRt1SRqqt1FBt/event_proportion‚0.007875503972172737 ?&Purity/2FRt2SRqqt1FBt/event_proportion‚0.003969253972172737 7 Purity/2FRt3SRqqt0FBt/FRt_purity‚0.3261904716491699 ("Purity/3FRt0SRqqt1FBt/event_purity‚0 ,&Purity/4FRt*SRqqt2FBt/event_proportion‚0 ,&Purity/4FRt*SRqqt4FBt/event_proportion‚0 @&Purity/4FRt0SRqqt0FBt/event_proportion‚6.300403038039804e-05 9"Purity/*FRt0SRqqt1FBt/event_purity‚0.6701472401618958 @&Purity/*FRt0SRqqt4FBt/event_proportion‚0.0003150201519019902 9"Purity/*FRt1SRqqt2FBt/SRqqt_purity‚0.5460367798805237 :"Purity/0FRt*SRqqt0FBt/SRqqt_purity‚0.32749179005622864 & Purity/3FRt0SRqqt1FBt/FBt_purity‚1 @&Purity/3FRt3SRqqt0FBt/event_proportion‚0.0005670362734235823 jet/accuracy_5_of_5‚0 particle/accuracy_1_of_6‚1 9"Purity/*FRt3SRqqt*FBt/SRqqt_purity‚0.3065526783466339 >&Purity/0FRt2SRqqt*FBt/event_proportion‚0.05361643061041832 7 Purity/1FRt0SRqqt1FBt/FRt_purity‚0.3694809377193451 ,&Purity/1FRt4SRqqt4FBt/event_proportion‚0 @&Purity/3FRt0SRqqt1FBt/event_proportion‚0.0011340725468471646 *jet/accuracy_1_of_5‚0.9376081228256226 9"Purity/0FRt3SRqqt*FBt/SRqqt_purity‚0.3293084502220154 ("Purity/1FRt4SRqqt0FBt/event_purity‚0 7 Purity/*FRt2SRqqt*FBt/FBt_purity‚0.9840930104255676 7 Purity/0FRt*SRqqt*FBt/FBt_purity‚0.9848901033401489 =&Purity/0FRt1SRqqt*FBt/event_proportion‚0.2305317521095276 9"Purity/2FRt0SRqqt2FBt/event_purity‚0.0357142873108387 ("Purity/3FRt*SRqqt*FBt/event_purity‚0 @&Purity/3FRt0SRqqt2FBt/event_proportion‚6.300403038039804e-05 9"Purity/3FRt1SRqqt0FBt/SRqqt_purity‚0.4416666030883789 +"Purity/3FRt*SRqqt2FBt/SRqqt_purity‚0.25 ;"Purity/1FRt*SRqqt2FBt/event_purity‚0.054287221282720566 ?&Purity/1FRt2SRqqt1FBt/event_proportion‚0.008946572430431843 ?&Purity/1FRt3SRqqt0FBt/event_proportion‚0.007938507944345474 ?&Purity/2FRt3SRqqt*FBt/event_proportion‚0.004032257944345474 & Purity/3FRt*SRqqt*FBt/FBt_purity‚1 7 Purity/*FRt*SRqqt4FBt/FBt_purity‚0.8333333134651184 +"Purity/0FRt4SRqqt*FBt/SRqqt_purity‚0.25 +"Purity/3FRt1SRqqt2FBt/SRqqt_purity‚0.25 ,&Purity/2FRt4SRqqt4FBt/event_proportion‚0 7 Purity/*FRt2SRqqt2FBt/FRt_purity‚0.3717948794364929 ("Purity/1FRt2SRqqt2FBt/event_purity‚0 7 Purity/*FRt3SRqqt1FBt/FBt_purity‚0.952380955219¾7ÁÁö`2688 ?&Purity/2FRt2SRqqt0FBt/event_proportion‚0.015435988083481789 particle/accuracy_4_of_5‚1 7 Purity/3FRt*SRqqt2FBt/FRt_purity‚0.2666666805744171 ("Purity/*FRt1SRqqt4FBt/SRqqt_purity‚1 % _timestamp‚1.7884884648363016e+09 9"Purity/*FRt*SRqqt*FBt/SRqqt_purity‚0.3680509924888611 ,&Purity/0FRt3SRqqt2FBt/event_proportion‚0 A&Purity/0FRt4SRqqt0FBt/event_proportion‚0.00044102821266278625 ("Purity/3FRt*SRqqt1FBt/event_purity‚0 ,&Purity/4FRt4SRqqt3FBt/event_proportion‚0 ("Purity/0FRt4SRqqt1FBt/event_purity‚0 8 Purity/1FRt1SRqqt0FBt/FRt_purity‚0.28188520669937134 :"Purity/1FRt2SRqqt1FBt/SRqqt_purity‚0.39193546772003174 @&Purity/3FRt*SRqqt2FBt/event_proportion‚0.0003150201519019902 & Purity/2FRt*SRqqt3FBt/FRt_purity‚0 >&Purity/*FRt1SRqqt0FBt/event_proportion‚0.33039313554763794 9"Purity/*FRt*SRqqt0FBt/SRqqt_purity‚0.3332124352455139 >&Purity/*FRt2SRqqt1FBt/event_proportion‚0.02255544438958168 8 Purity/*FRt3SRqqt*FBt/FRt_purity‚0.30743032693862915 7 Purity/0FRt1SRqqt2FBt/FBt_purity‚0.9648147821426392 @&Purity/0FRt4SRqqt*FBt/event_proportion‚0.0005040322430431843 =&Purity/1FRt1SRqqt*FBt/event_proportion‚0.1723790317773819 ,&Purity/3FRt*SRqqt4FBt/event_proportion‚0 ("Purity/3FRt2SRqqt0FBt/event_purity‚0 7 Purity/*FRt0SRqqt4FBt/FBt_purity‚0.8500000238418579 7 Purity/0FRt1SRqqt3FBt/FBt_purity‚0.8999999761581421 & Purity/4FRt0SRqqt0FBt/FRt_purity‚0 ("Purity/1FRt0SRqqt3FBt/event_purity‚0 >&Purity/1FRt1SRqqt1FBt/event_proportion‚0.03944052383303642 ,&Purity/4FRt0SRqqt4FBt/event_proportion‚0 particle/accuracy_2_of_7‚1 particle/accuracy_6_of_7‚1 ) Purity/*FRt1SRqqt4FBt/FBt_purity‚0.75 & Purity/1FRt3SRqqt2FBt/FBt_purity‚1 & Purity/2FRt2SRqqt2FBt/FBt_purity‚1 ,&Purity/0FRt1SRqqt4FBt/event_proportion‚0 7 Purity/0FRt2SRqqt*FBt/FBt_purity‚0.9685100317001343 @&Purity/0FRt3SRqqt1FBt/event_proportion‚0.0006300403038039804 >&Purity/2FRt1SRqqt0FBt/event_proportion‚0.03181703761219978 ("Purity/3FRt0SRqqt*FBt/event_purity‚0 ,&Purity/4FRt2SRqqt3FBt/event_proportion‚0 particle/accuracy_2_of_4‚1 7 Purity/*FRt0SRqqt3FBt/FRt_purity‚0.3333333432674408 >&Purity/1FRt2SRqqt0FBt/event_proportion‚0.04731602966785431 9"Purity/2FRt1SRqqt2FBt/SRqqt_purity‚0.7456140518188477 @&Purity/4FRt*SRqqt*FBt/event_proportion‚0.0001890120911411941 & Purity/3FRt*SRqqt2FBt/FBt_purity‚1 7 Purity/0FRt*SRqqt2FBt/FBt_purity‚0.9810502529144287 7 Purity/0FRt*SRqqt3FBt/FBt_purity‚0.8950618505477905 ?&Purity/0FRt2SRqqt1FBt/event_proportion‚0.009261592291295528 :"Purity/1FRt2SRqqt*FBt/SRqqt_purity‚0.36140623688697815 particle/accuracy_1_of_2‚1 7 Purity/*FRt2SRqqt2FBt/FBt_purity‚0.9912281036376953 7 Purity/*FRt2SRqqt3FBt/FBt_purity‚0.6666666865348816 >&Purity/*FRt3SRqqt0FBt/event_proportion‚0.01827116869390011 ;"Purity/2FRt0SRqqt1FBt/event_purity‚0.034259259700775146 ("Purity/2FRt3SRqqt*FBt/event_purity‚0 ,&Purity/4FRt*SRqqt1FBt/event_proportion‚0 ,&Purity/4FRt4SRqqt2FBt/event_proportion‚0 7 Purity/3FRt4SRqqt0FBt/FRt_purity‚0.1666666716337204 ("Purity/*FRt3SRqqt0FBt/event_purity‚0 :"Purity/1FRt0SRqqt*FBt/event_purity‚0.18423162400722504 ,&Purity/2FRt4SRqqt3FBt/event_proportion‚0 9"Purity/*FRt*SRqqt3FBt/SRqqt_purity‚0.6190475821495056 ) Purity/*FRt1SRqqt3FBt/FRt_purity‚0.75 :"Purity/*FRt0SRqqt*FBt/event_purity‚0.37596097588539124 :"Purity/1FRt3SRqqt0FBt/SRqqt_purity‚0.27901944518089294 ,&Purity/4FRt4SRqqt0FBt/event_proportion‚0 +jet/accuracy_2_of_5‚0.42225030064582825 jet/accuracy_1_of_6‚1 ("Purity/1FRt4SRqqt*FBt/event_purity‚0 & Purity/*FRt1SRqqt4FBt/FRt_purity‚0 9"Purity/0FRt*SRqqt1FBt/SRqqt_purity‚0.4590678811073303 9"Purity/0FRt*SRqqt2FBt/SRqqt_purity‚0.4588888883590698 ,&Purity/0FRt4SRqqt4FBt/event_proportion‚0 :"Purity/2FRt1SRqqt0FBt/SRqqt_purity‚0.37284016609191895 ("Purity/2FRt2SRqqt0FBt/event_purity‚0 @&Purity/3FRt*SRqqt0FBt/event_proportion‚0.0071824598126113415 & Purity/3FRt1SRqqt*FBt/FBt_purity‚1 +"Purity/1FRt1SRqqt3FBt/SRqqt_purity‚0.75 9"Purity/*FRt2SRqqt*FBt/event_purity‚0.0152859166264534 ("Purity/*FRt3SRqqt1FBt/event_purity‚0 >&Purity/0FRt0SRqqt0FBt/event_proportion‚0.04082661122083664 7 Purity/1FRt1SRqqt1FBt/FRt_purity‚0.2860495150089264 @&Purity/3FRt1SRqqt1FBt/event_proportion‚0.0009450604557059705 & Purity/4FRt1SRqqt0FBt/FRt_purity‚0 ?&Purity/0FRt3SRqqt*FBt/event_proportion‚0.006741431541740894 :"Purity/*FRt*SRqqt1FBt/SRqqt_purity‚0.48319631814956665 jet/accuracy_7_of_7‚0 particle/accuracy_1_of_7‚1 & Purity/*FRt0SRqqt4FBt/FRt_purity‚0 & Purity/1FRt3SRqqt2FBt/FRt_purity‚0 ( Purity/2FRt2SRqqt2FBt/FRt_purity‚0.5 >&Purity/0FRt*SRqqt*FBt/event_proportion‚0.40996724367141724 ("Purity/*FRt2SRqqt2FBt/event_purity‚0 & Purity/2FRt3SRqqt1FBt/FBt_purity‚1 particle/accuracy_5_of_7‚1 >&Purity/*FRt0SRqqt0FBt/event_proportion‚0.15757308900356293 9"Purity/*FRt1SRqqt0FBt/event_purity‚0.1723441779613495 9"Purity/2FRt1SRqqt*FBt/SRqqt_purity‚0.4197915196418762 ("Purity/2FRt2SRqqt*FBt/event_purity‚0 7 Purity/3FRt2SRqqt1FBt/FRt_purity‚0.1111111119389534 ("Purity/*FRt1SRqqt3FBt/event_purity‚0 @&Purity/*FRt4SRqqt1FBt/event_proportion‚6.300403038039804e-05 ?&Purity/0FRt1SRqqt2FBt/event_proportion‚0.008757560513913631 ("Purity/1FRt*SRqqt3FBt/event_purity‚0 7 Purity/2FRt0SRqqt2FBt/FRt_purity‚0.4791666865348816 @&Purity/3FRt2SRqqt*FBt/event_proportion‚0.0028351815417408943 ,&Purity/4FRt3SRqqt1FBt/event_proportion‚0 ,&Purity/4FRt4SRqqt4FBt/event_proportion‚0 *jet/accuracy_1_of_3‚0.8415170907974243 6 Purity/*FRt*SRqqt2FBt/FBt_purity‚0.982401430606842 7 Purity/*FRt*SRqqt3FBt/FBt_purity‚0.9038315415382385 9"Purity/*FRt2SRqqt1FBt/SRqqt_purity‚0.4442177712917328 ,&Purity/*FRt4SRqqt2FBt/event_proportion‚0 9"Purity/1FRt*SRqqt*FBt/SRqqt_purity‚0.3710376024246216 9"Purity/1FRt0SRqqt1FBt/event_purity‚0.2309117317199707 8 Purity/2FRt1SRqqt1FBt/FRt_purity‚0.32478880882263184 @&Purity/3FRt3SRqqt*FBt/event_proportion‚0.0005670362734235823 >&Purity/0FRt0SRqqt1FBt/event_proportion‚0.08606351166963577 7 Purity/1FRt*SRqqt1FBt/FRt_purity‚0.3244644105434418 ;"Purity/2FRt*SRqqt*FBt/event_purity‚0.008074735291302204 (particle/accuracy‚0.8473936915397644 & Purity/*FRt4SRqqt*FBt/FBt_purity‚1 particle/accuracy_3_of_6‚1 particle/accuracy_5_of_6‚1 ("Purity/4FRt2SRqqt0FBt/event_purity‚0 :"Purity/1FRt3SRqqt*FBt/SRqqt_purity‚0.29941970109939575 6 Purity/2FRt*SRqqt*FBt/FBt_purity‚0.988723635673523 7 Purity/2FRt3SRqqt*FBt/FRt_purity‚0.3312380909919739 0"Purity/*FRt4SRqqt*FBt/SRqqt_purity‚ 0.2109375 ("Purity/2FRt4SRqqt0FBt/event_purity‚0 particle/accuracy_4_of_6‚1 ("Purity/3FRt3SRqqt0FBt/event_purity‚0 +"Purity/4FRt*SRqqt*FBt/SRqqt_purity‚0.25 @&Purity/1FRt2SRqqt2FBt/event_proportion‚0.0010080644860863686 9"Purity/3FRt2SRqqt*FBt/SRqqt_purity‚0.3975694179534912 @&Purity/4FRt*SRqqt0FBt/event_proportion‚0.0001890120911411941 ("Purity/2FRt2SRqqt2FBt/SRqqt_purity‚0 =&Purity/1FRt1SRqqt0FBt/event_proportion‚0.1265120953321457 ;"Purity/2FRt*SRqqt0FBt/event_purity‚0.006667035166174173 9"Purity/2FRt*SRqqt2FBt/SRqqt_purity‚0.7105262875556946 8 Purity/3FRt0SRqqt0FBt/FRt_purity‚0.14705882966518402 ,&Purity/4FRt0SRqqt2FBt/event_proportion‚0 @&Purity/0FRt2SRqqt3FBt/event_proportion‚6.300403038039804e-05 8 Purity/1FRt*SRqqt0FBt/FRt_purity‚0.23848918080329895 ;"Purity/1FRt1SRqqt*FBt/event_purity‚0.014692626893520355 ?&Purity/2FRt3SRqqt0FBt/event_proportion‚0.003654233878478408 ("Purity/0FRt2SRqqt3FBt/SRqqt_purity‚1 8 Purity/*FRt*SRqqt2FBt/FRt_purity‚0.41129016876220703 8 Purity/1FRt2SRqqt*FBt/FRt_purity‚0.26468798518180847 8 Purity/3FRt0SRqqt1FBt/FRt_purity‚0.37820515036582947 7 Purity/1FRt0SRqqt1FBt/FBt_purity‚0.9988052845001221 :"Purity/1FRt0SRqqt2FBt/event_purity‚0.08207793533802032 9"Purity/1FRt1SRqqt1FBt/SRqqt_purity‚0.5535753965377808 7 Purity/2FRt0SRqqt2FBt/FBt_purity‚0.9940475821495056 particle/accuracy_3_of_3‚1 jet/accuracy_3_of_7‚0 & Purity/2FRt0SRqqt3FBt/FBt_purity‚1 ("Purity/1FRt3SRqqt2FBt/event_purity‚0 =&Purity/*FRt0SRqqt*FBt/event_proportion‚0.3558467626571655 9"Purity/*FRt0SRqqt2FBt/event_purity‚0.2371642142534256 @&Purity/1FRt3SRqqt1FBt/event_proportion‚0.0011340725468471646 7 Purity/3FRt2SRqqt0FBt/FRt_purity‚0.2077294886112213 ("Purity/0FRt0SRqqt4FBt/event_purity‚0 8 Purity/2FRt4SRqqt0FBt/FRt_purity‚0.30000001192092896 ("Purity/0FRt4SRqqt0FBt/event_purity‚0 ("Purity/3FRt3SRqqt*FBt/event_purity‚0 5 Purity/*FRt1SRqqt*FBt/FBt_purity‚0.98543381690979 ,&Purity/0FRt3SRqqt3FBt/event_proportion‚0 ?&Purity/2FRt0SRqqt1FBt/event_proportion‚0.008946572430431843 6 Purity/3FRt3SRqqt0FBt/FRt_purity‚0.380952388048172 ("Purity/4FRt1SRqqt*FBt/event_purity‚0 ("Purity/4FRt1SRqqt0FBt/event_purity‚0 6 Purity/*FRt0SRqqt*FBt/FBt_purity‚0.989166259765625 =&Purity/1FRt*SRqqt0FBt/event_proportion‚0.3135710656642914 8 Purity/1FRt0SRqqt*FBt/FRt_purity‚0.25107452273368835 >&Purity/1FRt2SRqqt*FBt/event_proportion‚0.05727066472172737 8 Purity/2FRt1SRqqt0FBt/FRt_purity‚0.23056648671627045 ("Purity/3FRt1SRqqt*FBt/event_purity‚0 @&Purity/*FRt4SRqqt*FBt/event_proportion‚0.0013230846961960196 @&Purity/1FRt1SRqqt4FBt/event_proportion‚6.300403038039804e-05 7 Purity/1FRt2SRqqt1FBt/FBt_purity‚0.9887096881866455 9"Purity/2FRt3SRqqt0FBt/SRqqt_purity‚0.2658369243144989 ,&Purity/3FRt2SRqqt2FBt/event_proportion‚0 -jet/accuracy_4_of_4‚0.0013440860202535987 +"Purity/2FRt4SRqqt*FBt/SRqqt_purity‚0.25 ("Purity/1FRt0SRqqt4FBt/event_purity‚0 7 Purity/*FRt0SRqqt2FBt/FRt_purity‚0.4503685235977173 =&Purity/*FRt2SRqqt*FBt/event_proportion‚0.1332535296678543 ,jet/accuracy_3_of_3‚0.017435600981116295 ("Purity/*FRt*SRqqt4FBt/event_purity‚0 +"Purity/3FRt4SRqqt*FBt/SRqqt_purity‚0.25 ("Purity/1FRt*SRqqt4FBt/event_purity‚0 ("Purity/0FRt1SRqqt3FBt/event_purity‚0 ("Purity/4FRt*SRqqt0FBt/event_purity‚0 :"Purity/1FRt0SRqqt0FBt/event_purity‚0.18158066272735596 A&Purity/3FRt1SRqqt2FBt/event_proportion‚0.00025201612152159214 A&Purity/3FRt4SRqqt0FBt/event_proportion‚0.00012600806076079607 +jet/accuracy_2_of_3‚0.22343683242797852 particle/accuracy_1_of_3‚1 & Purity/4FRt1SRqqt*FBt/FRt_purity‚0 @&Purity/*FRt1SRqqt3FBt/event_proportion‚0.0006930443341843784 9"Purity/*FRt2SRqqt0FBt/SRqqt_purity‚0.3487323522567749 ;"Purity/1FRt1SRqqt0FBt/event_purity‚0.013698057271540165 7 Purity/2FRt*SRqqt2FBt/FBt_purity‚0.9911110997200012 ,&Purity/2FRt0SRqqt4FBt/event_proportion‚0 ?&Purity/3FRt*SRqqt*FBt/event_proportion‚0.009954636916518211 ,&Purity/3FRt0SRqqt3FBt/event_proportion‚0 particle/accuracy_3_of_7‚1 7 Purity/*FRt1SRqqt2FBt/FRt_purity‚0.3343236446380615 @&Purity/2FRt4SRqqt*FBt/event_proportion‚0.0003150201519019902 & Purity/0FRt4SRqqt*FBt/FBt_purity‚1 & Purity/2FRt*SRqqt3FBt/FBt_purity‚1 *"Purity/0FRt*SRqqt3FBt/SRqqt_purity‚0.5 7 Purity/*FRt*SRqqt*FBt/FBt_purity‚0.9873675107955933 ;"Purity/*FRt2SRqqt1FBt/event_purity‚0.015157333575189114 :"Purity/1FRt*SRqqt0FBt/event_purity‚0.08152488619089127 ?&Purity/2FRt2SRqqt*FBt/event_proportion‚0.019468246027827263 particle/accuracy_2_of_2‚1 ("Purity/0FRt3SRqqt0FBt/event_purity‚0 7 Purity/3FRt4SRqqt*FBt/FRt_purity‚0.1666666716337204 :"Purity/0FRt1SRqqt0FBt/SRqqt_purity‚0.32592734694480896 + Purity/0FRt*SRqqt4FBt/FBt_purity‚0.8125 ("Purity/*FRt2SRqqt3FBt/SRqqt_purity‚1 :"Purity/*FRt1SRqqt*FBt/event_purity‚0.14813601970672607 >&Purity/*FRt2SRqqt0FBt/event_proportion‚0.10855595022439957 9"Purity/0FRt0SRqqt1FBt/event_purity‚0.9985072016716003 9"Purity/0FRt1SRqqt*FBt/SRqqt_purity‚0.3658779263496399 ,&Purity/3FRt2SRqqt4FBt/event_proportion‚0 ,&Purity/4FRt1SRqqt2FBt/event_proportion‚0 ("Purity/4FRt0SRqqt0FBt/event_purity‚0 8 Purity/*FRt3SRqqt1FBt/FRt_purity‚0.31111109256744385 :"Purity/0FRt1SRqqt2FBt/event_purity‚0.03453703969717026 particle/accuracy_1_of_4‚1 @&Purity/0FRt2SRqqt2FBt/event_proportion‚0.0010080644860863686 7 Purity/1FRt1SRqqt1FBt/FBt_purity‚0.9968053698539734 >&Purity/2FRt*SRqqt*FBt/event_proportion‚0.10244455933570862 7 Purity/*FRt0SRqqt2FBt/FBt_purity‚0.9863644242286682 6 Purity/*FRt0SRqqt3FBt/FBt_purity‚0.908333420753479 =&Purity/1FRt*SRqqt*FBt/event_proportion‚0.4366179406642914 ,&Purity/4FRt3SRqqt0FBt/event_proportion‚0 jet/accuracy_2_of_7‚0 7 Purity/*FRt1SRqqt2FBt/FBt_purity‚0.9696038961410522 8 Purity/*FRt3SRqqt0FBt/FRt_purity‚0.32808634638786316 5 Purity/*FRt1SRqqt3FBt/FBt_purity‚0.89682537317276 & Purity/4FRt0SRqqt*FBt/FRt_purity‚0 @&Purity/*FRt4SRqqt0FBt/event_proportion‚0.0012600806076079607 ?&Purity/0FRt3SRqqt0FBt/event_proportion‚0.006111391354352236 8 Purity/2FRt*SRqqt2FBt/FRt_purity‚0.44083335995674133 ,&Purity/2FRt3SRqqt3FBt/event_proportion‚0 ?&Purity/1FRt*SRqqt2FBt/event_proportion‚0.022303426638245583 :"Purity/1FRt1SRqqt1FBt/event_purity‚0.02143143303692341 7 Purity/2FRt0SRqqt*FBt/FBt_purity‚0.9967741966247559 particle/accuracy_1_of_1‚1 ("Purity/2FRt*SRqqt3FBt/event_purity‚0 ("Purity/4FRt2SRqqt*FBt/event_purity‚0 @&Purity/*FRt*SRqqt4FBt/event_proportion‚0.0003780241822823882 =&Purity/0FRt1SRqqt0FBt/event_proportion‚0.1694808453321457 >&Purity/0FRt1SRqqt1FBt/event_proportion‚0.05185231938958168 8 Purity/1FRt1SRqqt*FBt/FRt_purity‚0.28558334708213806 @&Purity/2FRt3SRqqt1FBt/event_proportion‚0.0003780241822823882 ,&Purity/4FRt3SRqqt3FBt/event_proportion‚0 particle/accuracy_6_of_6‚1 particle/accuracy_7_of_7‚0.5 :"Purity/0FRt2SRqqt0FBt/event_purity‚0.04135720804333687 9"Purity/3FRt*SRqqt0FBt/SRqqt_purity‚0.3423469364643097 +particle/sensitivity‚0.2660086750984192 ("Purity/1FRt1SRqqt4FBt/event_purity‚0 @&Purity/1FRt4SRqqt0FBt/event_proportion‚0.0003780241822823882 ,&Purity/*FRt4SRqqt4FBt/event_proportion‚0 ,&Purity/1FRt2SRqqt3FBt/event_proportion‚0 +jet/accuracy_2_of_2‚0.09548818320035934 ,&Purity/4FRt4SRqqt1FBt/event_proportion‚0 A&Purity/0FRt1SRqqt3FBt/event_proportion‚0.00044102821266278625 7 Purity/1FRt*SRqqt2FBt/FBt_purity‚0.9815362691879272 7 Purity/1FRt*SRqqt3FBt/FBt_purity‚0.9270833134651184 ,&Purity/4FRt4SRqqt*FBt/event_proportion‚0 *jet/accuracy_3_of_5‚0.0969638004899025 7 Purity/0FRt0SRqqt2FBt/FBt_purity‚0.9851524829864502 7 Purity/0FRt0SRqqt3FBt/FBt_purity‚0.9044445157051086 ,&Purity/2FRt2SRqqt3FBt/event_proportion‚0 0"Purity/*FRt4SRqqt0FBt/SRqqt_purity‚ 0.2109375 + Purity/4FRt*SRqqt*FBt/FRt_purity‚0.0625 *"Purity/4FRt2SRqqt*FBt/SRqqt_purity‚0.5 7 Purity/0FRt2SRqqt1FBt/FBt_purity‚0.9792627096176147 8 Purity/2FRt0SRqqt0FBt/FRt_purity‚0.18192803859710693 ("Purity/2FRt1SRqqt2FBt/event_purity‚0 9"Purity/3FRt1SRqqt1FBt/SRqqt_purity‚0.5769230723381042 9"Purity/3FRt2SRqqt1FBt/SRqqt_purity‚0.5833333134651184 @&Purity/3FRt0SRqqt0FBt/event_proportion‚0.0015120967291295528 *jet/accuracy_1_of_2‚0.7019760608673096 ,jet/accuracy_3_of_4‚0.050856780260801315 & Purity/1FRt0SRqqt4FBt/FBt_purity‚1 =&Purity/*FRt*SRqqt0FBt/event_proportion‚0.6160534024238586 7 Purity/*FRt3SRqqt*FBt/FBt_purity‚0.9545454382896423 8 Purity/*FRt*SRqqt1FBt/FRt_purity‚0.32187020778656006 ?&Purity/*FRt1SRqqt2FBt/event_proportion‚0.017011089250445366 9"Purity/0FRt*SRqqt2FBt/event_purity‚0.2675585150718689 >&Purity/0FRt0SRqqt*FBt/event_proportion‚0.11857359111309052 ?&Purity/2FRt1SRqqt2FBt/event_proportion‚0.001890120911411941 7 Purity/2FRt1SRqqt2FBt/FBt_purity‚0.9868420958518982 ("Purity/3FRt4SRqqt0FBt/event_purity‚0 7 Purity/3FRt0SRqqt2FBt/FRt_purity‚0.3333333432674408 >&Purity/*FRt0SRqqt1FBt/event_proportion‚0.14591734111309052 ("Purity/0FRt0SRqqt0FBt/event_purity‚1 ?&Purity/1FRt0SRqqt2FBt/event_proportion‚0.015120967291295528 :"Purity/*FRt1SRqqt*FBt/SRqqt_purity‚0.38096386194229126 9"Purity/0FRt1SRqqt*FBt/event_purity‚0.2772851586341858 7 Purity/1FRt*SRqqt2FBt/FRt_purity‚0.4339703321456909 ,&Purity/2FRt1SRqqt3FBt/event_proportion‚0 9"Purity/0FRt0SRqqt2FBt/event_purity‚0.3454820513725281 7 Purity/1FRt0SRqqt3FBt/FRt_purity‚0.3571428656578064 ,&Purity/2FRt4SRqqt2FBt/event_proportion‚0 & Purity/3FRt1SRqqt1FBt/FBt_purity‚1 8"Purity/2FRt1SRqqt1FBt/SRqqt_purity‚0.506950855255127 ,&Purity/4FRt*SRqqt3FBt/event_proportion‚0 7 Purity/*FRt*SRqqt0FBt/FRt_purity‚0.2315024435520172 ("Purity/1FRt2SRqqt0FBt/event_purity‚0 ,&Purity/1FRt4SRqqt1FBt/event_proportion‚0 ,&Purity/1FRt4SRqqt3FBt/event_proportion‚0 ,&Purity/3FRt4SRqqt2FBt/event_proportion‚0 ("Purity/*FRt4SRqqt1FBt/event_purity‚0 9"Purity/1FRt2SRqqt2FBt/SRqqt_purity‚0.4166666567325592 :"Purity/0FRt*SRqqt0FBt/event_purity‚0.25951263308525085 =&Purity/*FRt1SRqqt*FBt/event_proportion‚0.4482736885547638 ,&Purity/*FRt4SRqqt3FBt/event_proportion‚0 ,&Purity/1FRt3SRqqt4FBt/event_proportion‚0 ,&Purity/2FRt1SRqqt4FBt/event_proportion‚0 8 Purity/3FRt1SRqqt*FBt/FRt_purity‚0.24588480591773987 particle/accuracy_3_of_4‚1 9"Purity/0FRt1SRqqt1FBt/SRqqt_purity‚0.4776713252067566 9"Purity/1FRt3SRqqt1FBt/SRqqt_purity‚0.3962962329387665 9"Purity/1FRt4SRqqt0FBt/SRqqt_purity‚0.1666666716337204 ?&Purity/1FRt3SRqqt*FBt/event_proportion‚0.009135584346950054 7 Purity/2FRt0SRqqt1FBt/FRt_purity‚0.3043915033340454 ,&Purity/3FRt0SRqqt4FBt/event_proportion‚0 ("Purity/3FRt1SRqqt0FBt/event_purity‚0 ,&Purity/3FRt2SRqqt3FBt/event_proportion‚0 ,&Purity/3FRt3SRqqt2FBt/event_proportion‚0 ("Purity/3FRt4SRqqt*FBt/event_purity‚0 & Purity/1FRt0SRqqt4FBt/FRt_purity‚0 =&Purity/*FRt0SRqqt2FBt/event_proportion‚0.0483240932226181 7 Purity/2FRt1SRqqt2FBt/FRt_purity‚0.3464912474155426 ,&Purity/3FRt3SRqqt3FBt/event_proportion‚0 & Purity/3FRt0SRqqt2FBt/FBt_purity‚1 7 Purity/1FRt3SRqqt*FBt/FRt_purity‚0.3096902370452881 8 Purity/2FRt2SRqqt*FBt/FRt_purity‚0.24581310153007507 :"Purity/3FRt1SRqqt*FBt/SRqqt_purity‚0.42901232838630676 ("Purity/3FRt*SRqqt2FBt/event_purity‚0 :"Purity/*FRt*SRqqt2FBt/event_purity‚0.17671488225460052 8 Purity/*FRt0SRqqt0FBt/FRt_purity‚0.18090297281742096 ?&Purity/*FRt0SRqqt3FBt/event_proportion‚0.003717237850651145 7 Purity/2FRt1SRqqt*FBt/FBt_purity‚0.9910393953323364 *jet/accuracy_1_of_1‚0.4077473282814026 ("Purity/2FRt4SRqqt*FBt/event_purity‚0 ("Purity/0FRt2SRqqt2FBt/event_purity‚0 7 Purity/*FRt1SRqqt0FBt/FRt_purity‚0.2619290053844452 @&Purity/0FRt4SRqqt1FBt/event_proportion‚6.300403038039804e-05 ,&Purity/4FRt0SRqqt3FBt/event_proportion‚0 + Purity/0FRt3SRqqt1FBt/FBt_purity‚0.8125 9"Purity/3FRt3SRqqt*FBt/SRqqt_purity‚0.3095238208770752 & Purity/*FRt3SRqqt2FBt/FBt_purity‚1 ("Purity/1FRt1SRqqt3FBt/event_purity‚0 ("Purity/4FRt1SRqqt*FBt/SRqqt_purity‚0 @&Purity/1FRt0SRqqt3FBt/event_proportion‚0.0010710685746744275 @&Purity/2FRt2SRqqt2FBt/event_proportion‚6.300403038039804e-05 & Purity/3FRt2SRqqt*FBt/FBt_purity‚1 7 Purity/1FRt4SRqqt0FBt/FRt_purity‚0.3333333432674408 9"Purity/1FRt*SRqqt2FBt/SRqqt_purity‚0.5197644233703613 9"Purity/*FRt*SRqqt0FBt/event_purity‚0.1345396339893341 :"Purity/*FRt*SRqqt3FBt/event_purity‚0.09597700834274292 8"Purity/*FRt0SRqqt0FBt/event_purity‚0.153885617852211 ("Purity/3FRt1SRqqt1FBt/event_purity‚0 ("Purity/3FRt2SRqqt1FBt/event_purity‚0 ("Purity/3FRt1SRqqt2FBt/event_purity‚0 6 Purity/0FRt0SRqqt*FBt/FBt_purity‚0.987014651298523 & Purity/1FRt3SRqqt1FBt/FBt_purity‚1 >&Purity/2FRt*SRqqt0FBt/event_proportion‚0.07579384744167328 ?&Purity/2FRt0SRqqt2FBt/event_proportion‚0.003465221729129553 7 Purity/2FRt2SRqqt1FBt/FBt_purity‚0.9821428656578064 @&Purity/2FRt4SRqqt0FBt/event_proportion‚0.0003150201519019902 :"Purity/3FRt2SRqqt0FBt/SRqqt_purity‚0.37318843603134155 particle/accuracy_3_of_5‚1 @&Purity/*FRt3SRqqt2FBt/event_proportion‚6.300403038039804e-05 7 Purity/1FRt*SRqqt*FBt/FBt_purity‚0.9898868203163147 particle/accuracy_5_of_5‚1 +particle/specificity‚0.9550201892852783 *jet/accuracy_2_of_6‚0.5614035129547119 particle/accuracy_4_of_7‚1 ("Purity/*FRt*SRqqt4FBt/SRqqt_purity‚1 ("Purity/1FRt*SRqqt4FBt/SRqqt_purity‚1 ;"Purity/*FRt1SRqqt2FBt/event_purity‚0.022531770169734955 7 Purity/0FRt1SRqqt1FBt/FBt_purity‚0.9901546239852905 8"Purity/1FRt1SRqqt*FBt/SRqqt_purity‚0.391095906496048 7 Purity/2FRt*SRqqt1FBt/FRt_purity‚0.3154428005218506 ?&Purity/2FRt*SRqqt2FBt/event_proportion‚0.005418346729129553 ) Purity/1FRt1SRqqt4FBt/FBt_purity‚0.75 ?&Purity/0FRt0SRqqt2FBt/event_proportion‚0.029674898833036423 & Purity/3FRt0SRqqt*FBt/FBt_purity‚1 ("Purity/3FRt2SRqqt*FBt/event_purity‚0 ,&Purity/4FRt2SRqqt2FBt/event_proportion‚0 ("Purity/1FRt1SRqqt4FBt/SRqqt_purity‚1 7 Purity/*FRt2SRqqt1FBt/FBt_purity‚0.9857849478721619 7 Purity/0FRt*SRqqt1FBt/FBt_purity‚0.9939499497413635 :"Purity/0FRt1SRqqt1FBt/event_purity‚0.15680283308029175 ("Purity/1FRt3SRqqt1FBt/event_purity‚0 9"Purity/2FRt*SRqqt*FBt/SRqqt_purity‚0.3934762477874756 8 Purity/2FRt*SRqqt0FBt/FRt_purity‚0.22235623002052307 jet/accuracy_6_of_7‚0 ) Purity/1FRt1SRqqt3FBt/FRt_purity‚0.75 9"Purity/*FRt3SRqqt0FBt/SRqqt_purity‚0.2951071560382843 ,&Purity/*FRt3SRqqt4FBt/event_proportion‚0 ("Purity/1FRt3SRqqt*FBt/event_purity‚0 & Purity/3FRt*SRqqt1FBt/FBt_purity‚1 & Purity/*FRt3SRqqt2FBt/FRt_purity‚0 ("Purity/4FRt*SRqqt*FBt/event_purity‚0 7 Purity/*FRt0SRqqt1FBt/FRt_purity‚0.3585189878940582 ("Purity/1FRt3SRqqt0FBt/event_purity‚0 ,&Purity/4FRt1SRqqt4FBt/event_proportion‚0 ("Purity/*FRt4SRqqt*FBt/event_purity‚0 7 Purity/*FRt4SRqqt0FBt/FRt_purity‚0.3055555522441864 +"Purity/*FRt4SRqqt1FBt/SRqqt_purity‚0.25 :"Purity/1FRt*SRqqt*FBt/event_purity‚0.08903983235359192 9"Purity/1FRt1SRqqt2FBt/SRqqt_purity‚0.5439625978469849 ?&Purity/2FRt0SRqqt*FBt/event_proportion‚0.037046369165182114 ("Purity/2FRt1SRqqt0FBt/event_purity‚0 ("Purity/2FRt1SRqqt1FBt/event_purity‚0 9"Purity/2FRt2SRqqt0FBt/SRqqt_purity‚0.3643549680709839 & Purity/1FRt1SRqqt4FBt/FRt_purity‚0 =&Purity/*FRt1SRqqt1FBt/event_proportion‚0.1001134067773819 8 Purity/3FRt*SRqqt*FBt/FRt_purity‚0.23332907259464264 ) Purity/4FRt2SRqqt*FBt/FRt_purity‚0.25 >&Purity/1FRt0SRqqt*FBt/event_proportion‚0.19745463132858276 8 Purity/*FRt1SRqqt1FBt/FRt_purity‚0.30318138003349304 7 Purity/*FRt2SRqqt*FBt/FRt_purity‚0.2521216869354248 ("Purity/2FRt1SRqqt*FBt/event_purity‚0 9"Purity/2FRt2SRqqt*FBt/SRqqt_purity‚0.3977940082550049 ("Purity/*FRt3SRqqt2FBt/event_purity‚0 7 Purity/*FRt*SRqqt1FBt/FBt_purity‚0.9950977563858032 9"Purity/*FRt2SRqqt*FBt/SRqqt_purity‚0.3667086958885193 , Purity/1FRt*SRqqt3FBt/FRt_purity‚0.46875 @&Purity/2FRt0SRqqt3FBt/event_proportion‚6.300403038039804e-05 @&Purity/4FRt2SRqqt0FBt/event_proportion‚6.300403038039804e-05 ("Purity/*FRt2SRqqt3FBt/event_purity‚0 8"Purity/*FRt1SRqqt0FBt/SRqqt_purity‚0.331596702337265 A&Purity/0FRt0SRqqt4FBt/event_proportion‚0.00025201612152159214 ,&Purity/0FRt4SRqqt2FBt/event_proportion‚0 7 Purity/1FRt0SRqqt2FBt/FRt_purity‚0.4540017247200012 ,&Purity/4FRt3SRqqt2FBt/event_proportion‚0 particle/accuracy_4_of_4‚1 7validation_average_jet_accuracy‚0.38901591300964355 ("Purity/0FRt*SRqqt4FBt/event_purity‚0 @&Purity/1FRt3SRqqt2FBt/event_proportion‚6.300403038039804e-05 ,&Purity/1FRt4SRqqt2FBt/event_proportion‚0 8 Purity/3FRt1SRqqt0FBt/FRt_purity‚0.20000000298023224 @&Purity/3FRt2SRqqt0FBt/event_proportion‚0.0024571572430431843 particle/accuracy_2_of_6‚1 9"Purity/1FRt4SRqqt*FBt/SRqqt_purity‚0.1666666716337204 9"Purity/3FRt3SRqqt0FBt/SRqqt_purity‚0.3095238208770752 ?&Purity/1FRt0SRqqt1FBt/event_proportion‚0.049773186445236206 ,&Purity/3FRt3SRqqt4FBt/event_proportion‚0 ("Purity/4FRt0SRqqt*FBt/event_purity‚0 _runtime‚3263.940038366 ?&Purity/*FRt3SRqqt*FBt/event_proportion‚0.020476309582591057 ,&Purity/0FRt3SRqqt4FBt/event_proportion‚0 @&Purity/2FRt*SRqqt3FBt/event_proportion‚6.300403038039804e-05 ;"Purity/2FRt0SRqqt*FBt/event_purity‚0.023402776569128036 :"Purity/2FRt3SRqqt*FBt/SRqqt_purity‚0.28898411989212036 A&Purity/3FRt4SRqqt*FBt/event_proportion‚0.00012600806076079607 ,&Purity/4FRt2SRqqt4FBt/event_proportion‚0 *"Purity/4FRt2SRqqt0FBt/SRqqt_purity‚0.5 7 Purity/*FRt*SRqqt*FBt/FRt_purity‚0.2611018717288971 @&Purity/3FRt2SRqqt1FBt/event_proportion‚0.0003780241822823882 ,&Purity/3FRt4SRqqt1FBt/event_proportion‚0 +"Purity/2FRt4SRqqt0FBt/SRqqt_purity‚0.25 7 Purity/1FRt2SRqqt2FBt/FRt_purity‚0.3611111342906952 epoch‚2žÏoiüéç & Purity/*FRt4SRqqt*FBt/FBt_purity‚1 particle/accuracy_3_of_6‚1 particle/accuracy_5_of_6‚1 ("Purity/4FRt2SRqqt0FBt/event_purity‚0 :"Purity/1FRt3SRqqt*FBt/SRqqt_purity‚0.29941970109939575 6 Purity/2FRt*SRqqt*FBt/FBt_purity‚0.988723635673523 7 Purity/2FRt3SRqqt*FBt/FRt_purity‚0.3312380909919739 0"Purity/*FRt4SRqqt*FBt/SRqqt_purity‚ 0.2109375 ("Purity/2FRt4SRqqt0FBt/event_purity‚0 particle/accuracy_4_of_6‚1 ("Purity/3FRt3SRqqt0FBt/event_purity‚0 +"Purity/4FRt*SRqqt*FBt/SRqqt_purity‚0.25 @&Purity/1FRt2SRqqt2FBt/event_proportion‚0.0010080644860863686 9"Purity/3FRt2SRqqt*FBt/SRqqt_purity‚0.3975694179534912 @&Purity/4FRt*SRqqt0FBt/event_proportion‚0.0001890120911411941 ("Purity/2FRt2SRqqt2FBt/SRqqt_purity‚0 =&Purity/1FRt1SRqqt0FBt/event_proportion‚0.1265120953321457 ;"Purity/2FRt*SRqqt0FBt/event_purity‚0.006667035166174173 9"Purity/2FRt*SRqqt2FBt/SRqqt_purity‚0.7105262875556946 8 Purity/3FRt0SRqqt0FBt/FRt_purity‚0.14705882966518402 ,&Purity/4FRt0SRqqt2FBt/event_proportion‚0 @&Purity/0FRt2SRqqt3FBt/event_proportion‚6.300403038039804e-05 8 Purity/1FRt*SRqqt0FBt/FRt_purity‚0.23848918080329895 ;"Purity/1FRt1SRqqt*FBt/event_purity‚0.014692626893520355 ?&Purity/2FRt3SRqqt0FBt/event_proportion‚0.003654233878478408 ("Purity/0FRt2SRqqt3FBt/SRqqt_purity‚1 8 Purity/*FRt*SRqqt2FBt/FRt_purity‚0.41129016876220703 8 Purity/1FRt2SRqqt*FBt/FRt_purity‚0.26468798518180847 8 Purity/3FRt0SRqqt1FBt/FRt_purity‚0.37820515036582947 7 Purity/1FRt0SRqqt1FBt/FBt_purity‚0.9988052845001221 :"Purity/1FRt0SRqqt2FBt/event_purity‚0.08207793533802032 9"Purity/1FRt1SRqqt1FBt/SRqqt_purity‚0.5535753965377808 7 Purity/2FRt0SRqqt2FBt/FBt_purity‚0.9940475821495056 particle/accuracy_3_of_3‚1 jet/accuracy_3_of_7‚0 & Purity/2FRt0SRqqt3FBt/FBt_purity‚1 ("Purity/1FRt3SRqqt2FBt/event_purity‚0 =&Purity/*FRt0SRqqt*FBt/event_proportion‚0.3558467626571655 9"Purity/*FRt0SRqqt2FBt/event_purity‚0.2371642142534256 @&Purity/1FRt3SRqqt1FBt/event_proportion‚0.0011340725468471646 7 Purity/3FRt2SRqqt0FBt/FRt_purity‚0.2077294886112213 ("Purity/0FRt0SRqqt4FBt/event_purity‚0 8 Purity/2FRt4SRqqt0FBt/FRt_purity‚0.30000001192092896 ("Purity/0FRt4SRqqt0FBt/event_purity‚0 ("Purity/3FRt3SRqqt*FBt/event_purity‚0 5 Purity/*FRt1SRqqt*FBt/FBt_purity‚0.98543381690979 ,&Purity/0FRt3SRqqt3FBt/event_proportion‚0 ?&Purity/2FRt0SRqqt1FBt/event_proportion‚0.008946572430431843 6 Purity/3FRt3SRqqt0FBt/FRt_purity‚0.380952388048172 ("Purity/4FRt1SRqqt*FBt/event_purity‚0 ("Purity/4FRt1SRqqt0FBt/event_purity‚0 6 Purity/*FRt0SRqqt*FBt/FBt_purity‚0.989166259765625 =&Purity/1FRt*SRqqt0FBt/event_proportion‚0.3135710656642914 8 Purity/1FRt0SRqqt*FBt/FRt_purity‚0.25107452273368835 >&Purity/1FRt2SRqqt*FBt/event_proportion‚0.05727066472172737 8 Purity/2FRt1SRqqt0FBt/FRt_purity‚0.23056648671627045 ("Purity/3FRt1SRqqt*FBt/event_purity‚0 @&Purity/*FRt4SRqqt*FBt/event_proportion‚0.0013230846961960196 @&Purity/1FRt1SRqqt4FBt/event_proportion‚6.300403038039804e-05 7 Purity/1FRt2SRqqt1FBt/FBt_purity‚0.9887096881866455 9"Purity/2FRt3SRqqt0FBt/SRqqt_purity‚0.2658369243144989 ,&Purity/3FRt2SRqqt2FBt/event_proportion‚0 -jet/accuracy_4_of_4‚0.0013440860202535987 +"Purity/2FRt4SRqqt*FBt/SRqqt_purity‚0.25 ("Purity/1FRt0SRqqt4FBt/event_purity‚0 7 Purity/*FRt0SRqqt2FBt/FRt_purity‚0.4503685235977173 =&Purity/*FRt2SRqqt*FBt/event_proportion‚0.1332535296678543 ,jet/accuracy_3_of_3‚0.017435600981116295 ("Purity/*FRt*SRqqt4FBt/event_purity‚0 +"Purity/3FRt4SRqqt*FBt/SRqqt_purity‚0.25 ("Purity/1FRt*SRqqt4FBt/event_purity‚0 ("Purity/0FRt1SRqqt3FBt/event_purity‚0 ("Purity/4FRt*SRqqt0FBt/event_purity‚0 :"Purity/1FRt0SRqqt0FBt/event_purity‚0.18158066272735596 A&Purity/3FRt1SRqqt2FBt/event_proportion‚0.00025201612152159214 A&Purity/3FRt4SRqqt0FBt/event_proportion‚0.00012600806076079607 +jet/accuracy_2_of_3‚0.22343683242797852 particle/accuracy_1_of_3‚1 & Purity/4FRt1SRqqt*FBt/FRt_purity‚0 @&Purity/*FRt1SRqqt3FBt/event_proportion‚0.0006930443341843784 9"Purity/*FRt2SRqqt0FBt/SRqqt_purity‚0.3487323522567749 ;"Purity/1FRt1SRqqt0FBt/event_purity‚0.013698057271540165 7 Purity/2FRt*SRqqt2FBt/FBt_purity‚0.9911110997200012 ,&Purity/2FRt0SRqqt4FBt/event_proportion‚0 ?&Purity/3FRt*SRqqt*FBt/event_proportion‚0.009954636916518211 ,&Purity/3FRt0SRqqt3FBt/event_proportion‚0 particle/accuracy_3_of_7‚1 7 Purity/*FRt1SRqqt2FBt/FRt_purity‚0.3343236446380615 @&Purity/2FRt4SRqqt*FBt/event_proportion‚0.0003150201519019902 & Purity/0FRt4SRqqt*FBt/FBt_purity‚1 & Purity/2FRt*SRqqt3FBt/FBt_purity‚1 *"Purity/0FRt*SRqqt3FBt/SRqqt_purity‚0.5 7 Purity/*FRt*SRqqt*FBt/FBt_purity‚0.9873675107955933 ;"Purity/*FRt2SRqqt1FBt/event_purity‚0.015157333575189114 :"Purity/1FRt*SRqqt0FBt/event_purity‚0.08152488619089127 ?&Purity/2FRt2SRqqt*FBt/event_proportion‚0.019468246027827263 particle/accuracy_2_of_2‚1 ("Purity/0FRt3SRqqt0FBt/event_purity‚0 7 Purity/3FRt4SRqqt*FBt/FRt_purity‚0.1666666716337204 :"Purity/0FRt1SRqqt0FBt/SRqqt_purity‚0.32592734694480896 + Purity/0FRt*SRqqt4FBt/FBt_purity‚0.8125 ("Purity/*FRt2SRqqt3FBt/SRqqt_purity‚1 :"Purity/*FRt1SRqqt*FBt/event_purity‚0.14813601970672607 >&Purity/*FRt2SRqqt0FBt/event_proportion‚0.10855595022439957 9"Purity/0FRt0SRqqt1FBt/event_purity‚0.9985072016716003 9"Purity/0FRt1SRqqt*FBt/SRqqt_purity‚0.3658779263496399 ,&Purity/3FRt2SRqqt4FBt/event_proportion‚0 ,&Purity/4FRt1SRqqt2FBt/event_proportion‚0 ("Purity/4FRt0SRqqt0FBt/event_purity‚0 8 Purity/*FRt3SRqqt1FBt/FRt_purity‚0.31111109256744385 :"Purity/0FRt1SRqqt2FBt/event_purity‚0.03453703969717026 particle/accuracy_1_of_4‚1 @&Purity/0FRt2SRqqt2FBt/event_proportion‚0.0010080644860863686 7 Purity/1FRt1SRqqt1FBt/FBt_purity‚0.9968053698539734 >&Purity/2FRt*SRqqt*FBt/event_proportion‚0.10244455933570862 7 Purity/*FRt0SRqqt2FBt/FBt_purity‚0.9863644242286682 6 Purity/*FRt0SRqqt3FBt/FBt_purity‚0.908333420753479 =&Purity/1FRt*SRqqt*FBt/event_proportion‚0.4366179406642914 ,&Purity/4FRt3SRqqt0FBt/event_proportion‚0 jet/accuracy_2_of_7‚0 7 Purity/*FRt1SRqqt2FBt/FBt_purity‚0.9696038961410522 8 Purity/*FRt3SRqqt0FBt/FRt_purity‚0.32808634638786316 5 Purity/*FRt1SRqqt3FBt/FBt_purity‚0.89682537317276 & Purity/4FRt0SRqqt*FBt/FRt_purity‚0 @&Purity/*FRt4SRqqt0FBt/event_proportion‚0.0012600806076079607 ?&Purity/0FRt3SRqqt0FBt/event_proportion‚0.006111391354352236 8 Purity/2FRt*SRqqt2FBt/FRt_purity‚0.44083335995674133 ,&Purity/2FRt3SRqqt3FBt/event_proportion‚0 ?&Purity/1FRt*SRqqt2FBt/event_proportion‚0.022303426638245583 :"Purity/1FRt1SRqqt1FBt/event_purity‚0.02143143303692341 7 Purity/2FRt0SRqqt*FBt/FBt_purity‚0.9967741966247559 particle/accuracy_1_of_1‚1 ("Purity/2FRt*SRqqt3FBt/event_purity‚0 ("Purity/4FRt2SRqqt*FBt/event_purity‚0 @&Purity/*FRt*SRqqt4FBt/event_proportion‚0.0003780241822823882 =&Purity/0FRt1SRqqt0FBt/event_proportion‚0.1694808453321457 >&Purity/0FRt1SRqqt1FBt/event_proportion‚0.05185231938958168 8 Purity/1FRt1SRqqt*FBt/FRt_purity‚0.28558334708213806 @&Purity/2FRt3SRqqt1FBt/event_proportion‚0.0003780241822823882 ,&Purity/4FRt3SRqqt3FBt/event_proportion‚0 particle/accuracy_6_of_6‚1 particle/accuracy_7_of_7‚0.5 :"Purity/0FRt2SRqqt0FBt/event_purity‚0.04135720804333687 9"Purity/3FRt*SRqqt0FBt/SRqqt_purity‚0.3423469364643097 +particle/sensitivity‚0.2660086750984192 ("Purity/1FRt1SRqqt4FBt/event_purity‚0 @&Purity/1FRt4SRqqt0FBt/event_proportion‚0.0003780241822823882 ,&Purity/*FRt4SRqqt4FBt/event_proportion‚0 ,&Purity/1FRt2SRqqt3FBt/event_proportion‚0 +jet/accuracy_2_of_2‚0.09548818320035934 ,&Purity/4FRt4SRqqt1FBt/event_proportion‚0 A&Purity/0FRt1SRqqt3FBt/event_proportion‚0.00044ÍÎS£Ît102821266278625 7 Purity/1FRt*SRqqt2FBt/FBt_purity‚0.9815362691879272 7 Purity/1FRt*SRqqt3FBt/FBt_purity‚0.9270833134651184 ,&Purity/4FRt4SRqqt*FBt/event_proportion‚0 *jet/accuracy_3_of_5‚0.0969638004899025 7 Purity/0FRt0SRqqt2FBt/FBt_purity‚0.9851524829864502 7 Purity/0FRt0SRqqt3FBt/FBt_purity‚0.9044445157051086 ,&Purity/2FRt2SRqqt3FBt/event_proportion‚0 0"Purity/*FRt4SRqqt0FBt/SRqqt_purity‚ 0.2109375 + Purity/4FRt*SRqqt*FBt/FRt_purity‚0.0625 *"Purity/4FRt2SRqqt*FBt/SRqqt_purity‚0.5 7 Purity/0FRt2SRqqt1FBt/FBt_purity‚0.9792627096176147 8 Purity/2FRt0SRqqt0FBt/FRt_purity‚0.18192803859710693 ("Purity/2FRt1SRqqt2FBt/event_purity‚0 9"Purity/3FRt1SRqqt1FBt/SRqqt_purity‚0.5769230723381042 9"Purity/3FRt2SRqqt1FBt/SRqqt_purity‚0.5833333134651184 @&Purity/3FRt0SRqqt0FBt/event_proportion‚0.0015120967291295528 *jet/accuracy_1_of_2‚0.7019760608673096 ,jet/accuracy_3_of_4‚0.050856780260801315 & Purity/1FRt0SRqqt4FBt/FBt_purity‚1 =&Purity/*FRt*SRqqt0FBt/event_proportion‚0.6160534024238586 7 Purity/*FRt3SRqqt*FBt/FBt_purity‚0.9545454382896423 8 Purity/*FRt*SRqqt1FBt/FRt_purity‚0.32187020778656006 ?&Purity/*FRt1SRqqt2FBt/event_proportion‚0.017011089250445366 9"Purity/0FRt*SRqqt2FBt/event_purity‚0.2675585150718689 >&Purity/0FRt0SRqqt*FBt/event_proportion‚0.11857359111309052 ?&Purity/2FRt1SRqqt2FBt/event_proportion‚0.001890120911411941 7 Purity/2FRt1SRqqt2FBt/FBt_purity‚0.9868420958518982 ("Purity/3FRt4SRqqt0FBt/event_purity‚0 7 Purity/3FRt0SRqqt2FBt/FRt_purity‚0.3333333432674408 >&Purity/*FRt0SRqqt1FBt/event_proportion‚0.14591734111309052 ("Purity/0FRt0SRqqt0FBt/event_purity‚1 ?&Purity/1FRt0SRqqt2FBt/event_proportion‚0.015120967291295528 :"Purity/*FRt1SRqqt*FBt/SRqqt_purity‚0.38096386194229126 9"Purity/0FRt1SRqqt*FBt/event_purity‚0.2772851586341858 7 Purity/1FRt*SRqqt2FBt/FRt_purity‚0.4339703321456909 ,&Purity/2FRt1SRqqt3FBt/event_proportion‚0 9"Purity/0FRt0SRqqt2FBt/event_purity‚0.3454820513725281 7 Purity/1FRt0SRqqt3FBt/FRt_purity‚0.3571428656578064 ,&Purity/2FRt4SRqqt2FBt/event_proportion‚0 & Purity/3FRt1SRqqt1FBt/FBt_purity‚1 8"Purity/2FRt1SRqqt1FBt/SRqqt_purity‚0.506950855255127 ,&Purity/4FRt*SRqqt3FBt/event_proportion‚0 7 Purity/*FRt*SRqqt0FBt/FRt_purity‚0.2315024435520172 ("Purity/1FRt2SRqqt0FBt/event_purity‚0 ,&Purity/1FRt4SRqqt1FBt/event_proportion‚0 ,&Purity/1FRt4SRqqt3FBt/event_proportion‚0 ,&Purity/3FRt4SRqqt2FBt/event_proportion‚0 ("Purity/*FRt4SRqqt1FBt/event_purity‚0 9"Purity/1FRt2SRqqt2FBt/SRqqt_purity‚0.4166666567325592 :"Purity/0FRt*SRqqt0FBt/event_purity‚0.25951263308525085 =&Purity/*FRt1SRqqt*FBt/event_proportion‚0.4482736885547638 ,&Purity/*FRt4SRqqt3FBt/event_proportion‚0 ,&Purity/1FRt3SRqqt4FBt/event_proportion‚0 ,&Purity/2FRt1SRqqt4FBt/event_proportion‚0 8 Purity/3FRt1SRqqt*FBt/FRt_purity‚0.24588480591773987 particle/accuracy_3_of_4‚1 9"Purity/0FRt1SRqqt1FBt/SRqqt_purity‚0.4776713252067566 9"Purity/1FRt3SRqqt1FBt/SRqqt_purity‚0.3962962329387665 9"Purity/1FRt4SRqqt0FBt/SRqqt_purity‚0.1666666716337204 ?&Purity/1FRt3SRqqt*FBt/event_proportion‚0.009135584346950054 7 Purity/2FRt0SRqqt1FBt/FRt_purity‚0.3043915033340454 ,&Purity/3FRt0SRqqt4FBt/event_proportion‚0 ("Purity/3FRt1SRqqt0FBt/event_purity‚0 ,&Purity/3FRt2SRqqt3FBt/event_proportion‚0 ,&Purity/3FRt3SRqqt2FBt/event_proportion‚0 ("Purity/3FRt4SRqqt*FBt/event_purity‚0 & Purity/1FRt0SRqqt4FBt/FRt_purity‚0 =&Purity/*FRt0SRqqt2FBt/event_proportion‚0.0483240932226181 7 Purity/2FRt1SRqqt2FBt/FRt_purity‚0.3464912474155426 ,&Purity/3FRt3SRqqt3FBt/event_proportion‚0 & Purity/3FRt0SRqqt2FBt/FBt_purity‚1 7 Purity/1FRt3SRqqt*FBt/FRt_purity‚0.3096902370452881 8 Purity/2FRt2SRqqt*FBt/FRt_purity‚0.24581310153007507 :"Purity/3FRt1SRqqt*FBt/SRqqt_purity‚0.42901232838630676 ("Purity/3FRt*SRqqt2FBt/event_purity‚0 :"Purity/*FRt*SRqqt2FBt/event_purity‚0.17671488225460052 8 Purity/*FRt0SRqqt0FBt/FRt_purity‚0.18090297281742096 ?&Purity/*FRt0SRqqt3FBt/event_proportion‚0.003717237850651145 7 Purity/2FRt1SRqqt*FBt/FBt_purity‚0.9910393953323364 *jet/accuracy_1_of_1‚0.4077473282814026 ("Purity/2FRt4SRqqt*FBt/event_purity‚0 ("Purity/0FRt2SRqqt2FBt/event_purity‚0 7 Purity/*FRt1SRqqt0FBt/FRt_purity‚0.2619290053844452 @&Purity/0FRt4SRqqt1FBt/event_proportion‚6.300403038039804e-05 ,&Purity/4FRt0SRqqt3FBt/event_proportion‚0 + Purity/0FRt3SRqqt1FBt/FBt_purity‚0.8125 9"Purity/3FRt3SRqqt*FBt/SRqqt_purity‚0.3095238208770752 & Purity/*FRt3SRqqt2FBt/FBt_purity‚1 ("Purity/1FRt1SRqqt3FBt/event_purity‚0 ("Purity/4FRt1SRqqt*FBt/SRqqt_purity‚0 @&Purity/1FRt0SRqqt3FBt/event_proportion‚0.0010710685746744275 @&Purity/2FRt2SRqqt2FBt/event_proportion‚6.300403038039804e-05 & Purity/3FRt2SRqqt*FBt/FBt_purity‚1 7 Purity/1FRt4SRqqt0FBt/FRt_purity‚0.3333333432674408 9"Purity/1FRt*SRqqt2FBt/SRqqt_purity‚0.5197644233703613 9"Purity/*FRt*SRqqt0FBt/event_purity‚0.1345396339893341 :"Purity/*FRt*SRqqt3FBt/event_purity‚0.09597700834274292 8"Purity/*FRt0SRqqt0FBt/event_purity‚0.153885617852211 ("Purity/3FRt1SRqqt1FBt/event_purity‚0 ("Purity/3FRt2SRqqt1FBt/event_purity‚0 ("Purity/3FRt1SRqqt2FBt/event_purity‚0 6 Purity/0FRt0SRqqt*FBt/FBt_purity‚0.987014651298523 & Purity/1FRt3SRqqt1FBt/FBt_purity‚1 >&Purity/2FRt*SRqqt0FBt/event_proportion‚0.07579384744167328 ?&Purity/2FRt0SRqqt2FBt/event_proportion‚0.003465221729129553 7 Purity/2FRt2SRqqt1FBt/FBt_purity‚0.9821428656578064 @&Purity/2FRt4SRqqt0FBt/event_proportion‚0.0003150201519019902 :"Purity/3FRt2SRqqt0FBt/SRqqt_purity‚0.37318843603134155 particle/accuracy_3_of_5‚1 @&Purity/*FRt3SRqqt2FBt/event_proportion‚6.300403038039804e-05 7 Purity/1FRt*SRqqt*FBt/FBt_purity‚0.9898868203163147 particle/accuracy_5_of_5‚1 +particle/specificity‚0.9550201892852783 *jet/accuracy_2_of_6‚0.5614035129547119 particle/accuracy_4_of_7‚1 ("Purity/*FRt*SRqqt4FBt/SRqqt_purity‚1 ("Purity/1FRt*SRqqt4FBt/SRqqt_purity‚1 ;"Purity/*FRt1SRqqt2FBt/event_purity‚0.022531770169734955 7 Purity/0FRt1SRqqt1FBt/FBt_purity‚0.9901546239852905 8"Purity/1FRt1SRqqt*FBt/SRqqt_purity‚0.391095906496048 7 Purity/2FRt*SRqqt1FBt/FRt_purity‚0.3154428005218506 ?&Purity/2FRt*SRqqt2FBt/event_proportion‚0.005418346729129553 ) Purity/1FRt1SRqqt4FBt/FBt_purity‚0.75 ?&Purity/0FRt0SRqqt2FBt/event_proportion‚0.029674898833036423 & Purity/3FRt0SRqqt*FBt/FBt_purity‚1 ("Purity/3FRt2SRqqt*FBt/event_purity‚0 ,&Purity/4FRt2SRqqt2FBt/event_proportion‚0 ("Purity/1FRt1SRqqt4FBt/SRqqt_purity‚1 7 Purity/*FRt2SRqqt1FBt/FBt_purity‚0.9857849478721619 7 Purity/0FRt*SRqqt1FBt/FBt_purity‚0.9939499497413635 :"Purity/0FRt1SRqqt1FBt/event_purity‚0.15680283308029175 ("Purity/1FRt3SRqqt1FBt/event_purity‚0 9"Purity/2FRt*SRqqt*FBt/SRqqt_purity‚0.3934762477874756 8 Purity/2FRt*SRqqt0FBt/FRt_purity‚0.22235623002052307 jet/accuracy_6_of_7‚0 ) Purity/1FRt1SRqqt3FBt/FRt_purity‚0.75 9"Purity/*FRt3SRqqt0FBt/SRqqt_purity‚0.2951071560382843 ,&Purity/*FRt3SRqqt4FBt/event_proportion‚0 ("Purity/1FRt3SRqqt*FBt/event_purity‚0 & Purity/3FRt*SRqqt1FBt/FBt_purity‚1 & Purity/*FRt3SRqqt2FBt/FRt_purity‚0 ("Purity/4FRt*SRqqt*FBt/event_purity‚0 7 Purity/*FRt0SRqqt1FBt/FRt_purity‚0.3585189878940582 ("Purity/1FRt3SRqqt0FBt/event_purity‚0 ,&Purity/4FRt1SRqqt4FBt/event_proportion‚0 ("Purity/*FRt4SRqqt*FBt/event_purity‚0 7 Purity/*FRt4SRqqt0FBt/FRt_purity‚0.3055555522441864 +"Purity/*FRt4SRqqt1FBt/SRqqt_purity‚0.25 :"Purity/1FRt*SRqqt*FBt/event_purity‚0.08903983235359192 9"Purity/1FRt1SRqqt2FBt/SRqqt_purity‚0.5439625978469849 ?&Purity/2FRt0SRqqt*FBt/event_proportion‚0.037046369165182114 ("Purity/2FRt1SRqqt0FBt/event_purity‚0 ("Purity/2FRt1SRqqt1FBt/event_purity‚0 9"Purity/2FRt2SRqqt0FBt/SRqqt_purity‚0.3643549680709839 & Purity/1FRt1SRqqt4FBt/FRt_purity‚0 =&Purity/*FRt1SRqqt1FBt/event_proportion‚0.1001134067773819 8 Purity/3FRt*SRqqt*FBt/FRt_purity‚0.23332907259464264 ) Purity/4FRt2SRqqt*FBt/FRt_purity‚0.25 >&Purity/1FRt0SRqqt*FBt/event_proportion‚0.19745463132858276 8 Purity/*FRt1SRqqt1FBt/FRt_purity‚0.30318138003349304 7 Purity/*FRt2SRqqt*FBt/FRt_purity‚0.2521216869354248 ("Purity/2FRt1SRqqt*FBt/event_purity‚0 9"Purity/2FRt2SRqqt*FBt/SRqqt_purity‚0.3977940082550049 ("Purity/*FRt3SRqqt2FBt/event_purity‚0 7 Purity/*FRt*SRqqt1FBt/FBt_purity‚0.9950977563858032 9"Purity/*FRt2SRqqt*FBt/SRqqt_purity‚0.3667086958885193 , Purity/1FRt*SRqqt3FBt/FRt_purity‚0.46875 @&Purity/2FRt0SRqqt3FBt/event_proportion‚6.300403038039804e-05 @&Purity/4FRt2SRqqt0FBt/event_proportion‚6.300403038039804e-05 ("Purity/*FRt2SRqqt3FBt/event_purity‚0 8"Purity/*FRt1SRqqt0FBt/SRqqt_purity‚0.331596702337265 A&Purity/0FRt0SRqqt4FBt/event_proportion‚0.00025201612152159214 ,&Purity/0FRt4SRqqt2FBt/event_proportion‚0 7 Purity/1FRt0SRqqt2FBt/FRt_purity‚0.4540017247200012 ,&Purity/4FRt3SRqqt2FBt/event_proportion‚0 particle/accuracy_4_of_4‚1 7validation_average_jet_accuracy‚0.38901591300964355 ("Purity/0FRt*SRqqt4FBt/event_purity‚0 @&Purity/1FRt3SRqqt2FBt/event_proportion‚6.300403038039804e-05 ,&Purity/1FRt4SRqqt2FBt/event_proportion‚0 8 Purity/3FRt1SRqqt0FBt/FRt_purity‚0.20000000298023224 @&Purity/3FRt2SRqqt0FBt/event_proportion‚0.0024571572430431843 particle/accuracy_2_of_6‚1 9"Purity/1FRt4SRqqt*FBt/SRqqt_purity‚0.1666666716337204 9"Purity/3FRt3SRqqt0FBt/SRqqt_purity‚0.3095238208770752 ?&Purity/1FRt0SRqqt1FBt/event_proportion‚0.049773186445236206 ,&Purity/3FRt3SRqqt4FBt/event_proportion‚0 ("Purity/4FRt0SRqqt*FBt/event_purity‚0 _runtime‚3263.940038366 ?&Purity/*FRt3SRqqt*FBt/event_proportion‚0.020476309582591057 ,&Purity/0FRt3SRqqt4FBt/event_proportion‚0 @&Purity/2FRt*SRqqt3FBt/event_proportion‚6.300403038039804e-05 ;"Purity/2FRt0SRqqt*FBt/event_purity‚0.023402776569128036 :"Purity/2FRt3SRqqt*FBt/SRqqt_purity‚0.28898411989212036 A&Purity/3FRt4SRqqt*FBt/event_proportion‚0.00012600806076079607 ,&Purity/4FRt2SRqqt4FBt/event_proportion‚0 *"Purity/4FRt2SRqqt0FBt/SRqqt_purity‚0.5 7 Purity/*FRt*SRqqt*FBt/FRt_purity‚0.2611018717288971 @&Purity/3FRt2SRqqt1FBt/event_proportion‚0.0003780241822823882 ,&Purity/3FRt4SRqqt1FBt/event_proportion‚0 +"Purity/2FRt4SRqqt0FBt/SRqqt_purity‚0.25 7 Purity/1FRt2SRqqt2FBt/FRt_purity‚0.3611111342906952 epoch‚2 @&Purity/1FRt0SRqqt4FBt/event_proportion‚6.300403038039804e-05 ("Purity/2FRt2SRqqt1FBt/event_purity‚0 ("Purity/0FRt2SRqqt3FBt/event_purity‚0 ("Purity/1FRt2SRqqt*FBt/event_purity‚0 9"Purity/2FRt*SRqqt0FBt/SRqqt_purity‚0.3573751151561737 & Purity/0FRt4SRqqt1FBt/FBt_purity‚1 jet/accuracy_5_of_7‚0 & Purity/1FRt*SRqqt4FBt/FRt_purity‚0 ,&Purity/*FRt3SRqqt3FBt/event_proportion‚0 :"Purity/0FRt0SRqqt3FBt/event_purity‚0.18000000715255737 :"Purity/1FRt*SRqqt1FBt/SRqqt_purity‚0.49889713525772095 7 Purity/1FRt0SRqqt2FBt/FBt_purity‚0.9862310886383057 7 Purity/1FRt0SRqqt3FBt/FBt_purity‚0.9285714030265808 ;"Purity/2FRt*SRqqt1FBt/event_purity‚0.013922575861215591 & Purity/2FRt0SRqqt1FBt/FBt_purity‚1 ,&Purity/3FRt1SRqqt3FBt/event_proportion‚0 ,&Purity/3FRt3SRqqt1FBt/event_proportion‚0 @&Purity/4FRt1SRqqt0FBt/event_proportion‚6.300403038039804e-05 ("Purity/*FRt0SRqqt4FBt/event_purity‚0 _step‚146 ;"Purity/0FRt2SRqqt1FBt/event_purity‚0.030645160004496574 >&Purity/*FRt*SRqqt2FBt/event_proportion‚0.06747731566429138 ?&Purity/*FRt*SRqqt3FBt/event_proportion‚0.004473286215215921 ("Purity/0FRt3SRqqt1FBt/event_purity‚0 :"Purity/*FRt*SRqqt*FBt/event_purity‚0.21076183021068573 :"Purity/0FRt1SRqqt2FBt/SRqqt_purity‚0.45765870809555054 7 Purity/1FRt1SRqqt*FBt/FBt_purity‚0.9867773652076721 @&Purity/3FRt*SRqqt1FBt/event_proportion‚0.0024571572430431843 'particle/f_score‚0.3524930775165558 +"Purity/0FRt4SRqqt0FBt/SRqqt_purity‚0.25 & Purity/1FRt2SRqqt2FBt/FBt_purity‚1 :"Purity/2FRt3SRqqt1FBt/SRqqt_purity‚0.36666667461395264 A&Purity/1FRt*SRqqt4FBt/event_proportion‚0.00012600806076079607 8 Purity/2FRt0SRqqt*FBt/FRt_purity‚0.23756299912929535 ,&Purity/3FRt4SRqqt3FBt/event_proportion‚0 jet/accuracy_4_of_6‚0 * Purity/1FRt*SRqqt4FBt/FBt_purity‚0.875 ("Purity/2FRt0SRqqt3FBt/event_purity‚0 ("Purity/*FRt1SRqqt4FBt/event_purity‚0 +"Purity/1FRt*SRqqt3FBt/SRqqt_purity‚0.75 +jet/accuracy_4_of_5‚0.01648745685815811 + Purity/0FRt0SRqqt4FBt/FBt_purity‚0.8125 9"Purity/*FRt1SRqqt3FBt/SRqqt_purity‚0.6190475821495056 ("Purity/4FRt1SRqqt0FBt/SRqqt_purity‚0 @&Purity/*FRt2SRqqt3FBt/event_proportion‚6.300403038039804e-05 7"Purity/*FRt1SRqqt1FBt/SRqqt_purity‚0.51287442445755 ("Purity/2FRt3SRqqt0FBt/event_purity‚0 7 Purity/3FRt1SRqqt1FBt/FRt_purity‚0.3717948794364929 @&Purity/4FRt0SRqqt*FBt/event_proportion‚6.300403038039804e-05 + Purity/4FRt*SRqqt0FBt/FRt_purity‚0.0625 +"Purity/4FRt*SRqqt0FBt/SRqqt_purity‚0.25 >&Purity/1FRt0SRqqt0FBt/event_proportion‚0.13142640888690948 :"Purity/2FRt0SRqqt0FBt/event_purity‚0.02127496339380741 :"Purity/3FRt*SRqqt*FBt/SRqqt_purity‚0.38419535756111145 8 Purity/3FRt*SRqqt0FBt/FRt_purity‚0.20079366862773895 ("Purity/3FRt0SRqqt0FBt/event_purity‚0 *jet/accuracy_1_of_4‚0.8894588351249695 9"Purity/1FRt3SRqqt2FBt/SRqqt_purity‚0.3333333432674408 ) Purity/4FRt2SRqqt0FBt/FRt_purity‚0.25 8 Purity/*FRt2SRqqt0FBt/FRt_purity‚0.25328224897384644 ,&Purity/*FRt2SRqqt4FBt/event_proportion‚0 @&Purity/0FRt*SRqqt3FBt/event_proportion‚0.0030871976632624865 7 Purity/2FRt*SRqqt1FBt/FBt_purity‚0.9905914068222046 9"Purity/2FRt2SRqqt1FBt/SRqqt_purity‚0.5270833373069763 ("Purity/0FRt4SRqqt*FBt/event_purity‚0 >&Purity/*FRt*SRqqt1FBt/event_proportion‚0.27079132199287415 ;"Purity/*FRt2SRqqt0FBt/event_purity‚0.016523730009794235 :"Purity/1FRt1SRqqt0FBt/SRqqt_purity‚0.32849568128585815 8 Purity/1FRt1SRqqt2FBt/FRt_purity‚0.34481292963027954 ,&Purity/3FRt1SRqqt4FBt/event_proportion‚0 7 Purity/*FRt4SRqqt*FBt/FRt_purity‚0.3055555522441864 ("Purity/*FRt3SRqqt*FBt/event_purity‚0 7 Purity/1FRt3SRqqt1FBt/FRt_purity‚0.2888888716697693 ,&Purity/1FRt3SRqqt3FBt/event_proportion‚0 8 Purity/2FRt2SRqqt1FBt/FRt_purity‚0.20744049549102783 ,&Purity/4FRt3SRqqt4FBt/event_proportion‚0 ("Purity/0FRt3SRqqt*FBt/event_purity‚0 :"Purity/1FRt*SRqqt0FBt/SRqqt_purity‚0.33100688457489014 :"Purity/0FRt*SRqqt*FBt/event_purity‚0.39606592059135437 ,&Purity/0FRt4SRqqt3FBt/event_proportion‚0 7 Purity/2FRt*SRqqt*FBt/FRt_purity‚0.2494080513715744 ,&Purity/3FRt4SRqqt4FBt/event_proportion‚0 :"Purity/0FRt3SRqqt0FBt/SRqqt_purity‚0.32956352829933167 & Purity/2FRt3SRqqt*FBt/FBt_purity‚1 jet/accuracy_4_of_7‚0 :"Purity/0FRt1SRqqt0FBt/event_purity‚0.32592734694480896 @&Purity/4FRt2SRqqt*FBt/event_proportion‚6.300403038039804e-05 particle/accuracy_2_of_3‚1 ,&Purity/1FRt2SRqqt4FBt/event_proportion‚0 & Purity/*FRt4SRqqt1FBt/FBt_purity‚1 jet/accuracy_1_of_7‚1 :"Purity/*FRt1SRqqt1FBt/event_purity‚0.09071654826402664 ?&Purity/1FRt1SRqqt2FBt/event_proportion‚0.006111391354352236 ,&Purity/2FRt*SRqqt4FBt/event_proportion‚0 trainer/global_step‚1811 7 Purity/*FRt1SRqqt1FBt/FBt_purity‚0.9930967688560486 ?&Purity/0FRt2SRqqt0FBt/event_proportion‚0.043283771723508835 7 Purity/*FRt0SRqqt1FBt/FBt_purity‚0.9986216425895691 ,&Purity/0FRt2SRqqt4FBt/event_proportion‚0 ,&Purity/2FRt3SRqqt4FBt/event_proportion‚0 *jet/accuracy_2_of_4‚0.3547448515892029 @&Purity/3FRt1SRqqt0FBt/event_proportion‚0.0025201612152159214 7 Purity/1FRt1SRqqt2FBt/FBt_purity‚0.9726190567016602 7 Purity/1FRt2SRqqt*FBt/FBt_purity‚0.9896313548088074 >&Purity/2FRt1SRqqt*FBt/event_proportion‚0.04158266261219978 ,&Purity/3FRt*SRqqt3FBt/event_proportion‚0 :"Purity/*FRt2SRqqt2FBt/SRqqt_purity‚0.42105263471603394 ("Purity/2FRt3SRqqt1FBt/event_purity‚0 7 Purity/1FRt1SRqqt3FBt/FBt_purity‚0.8333333730697632 7 Purity/*FRt1SRqqt*FBt/FRt_purity‚0.2747350335121155 8 Purity/*FRt2SRqqt1FBt/FRt_purity‚0.25289416313171387 >&Purity/0FRt*SRqqt1FBt/event_proportion‚0.14787046611309052 :"Purity/2FRt*SRqqt2FBt/event_purity‚0.03333333507180214 @&Purity/3FRt0SRqqt*FBt/event_proportion‚0.0027091733645647764 ,&Purity/4FRt1SRqqt1FBt/event_proportion‚0 particle/accuracy_1_of_5‚1 9"Purity/0FRt3SRqqt1FBt/SRqqt_purity‚0.3125000298023224 7 Purity/3FRt*SRqqt1FBt/FRt_purity‚0.3083333671092987 >&Purity/1FRt*SRqqt1FBt/event_proportion‚0.09929435700178146 *loss/val_total_loss‚44.310995154527745 :"Purity/0FRt*SRqqt3FBt/event_purity‚0.11728394776582718 9"Purity/0FRt2SRqqt1FBt/SRqqt_purity‚0.4510112702846527 7 Purity/1FRt4SRqqt*FBt/FRt_purity‚0.3333333432674408 ("Purity/3FRt0SRqqt2FBt/event_purity‚0 ("Purity/2FRt2SRqqt2FBt/event_purity‚0 9"Purity/0FRt2SRqqt0FBt/SRqqt_purity‚0.3387606143951416 6 Purity/1FRt3SRqqt0FBt/FRt_purity‚0.353532999753952 @&Purity/1FRt4SRqqt*FBt/event_proportion‚0.0003780241822823882 7 Purity/2FRt2SRqqt0FBt/FRt_purity‚0.2524426281452179 ("Purity/3FRt*SRqqt0FBt/event_purity‚0 ?&Purity/2FRt*SRqqt1FBt/event_proportion‚0.021169355139136314 9"Purity/2FRt*SRqqt1FBt/SRqqt_purity‚0.5178669095039368 ,&Purity/2FRt2SRqqt4FBt/event_proportion‚0 9"Purity/1FRt*SRqqt1FBt/event_purity‚0.1225278303027153 6 Purity/1FRt0SRqqt*FBt/FBt_purity‚0.991801381111145 8 Purity/*FRt0SRqqt*FBt/FRt_purity‚0.24821753799915314 :"Purity/*FRt0SRqqt3FBt/event_purity‚0.13214285671710968 ?&Purity/*FRt2SRqqt2FBt/event_proportion‚0.002079133177176118 ?&Purity/*FRt3SRqqt1FBt/event_proportion‚0.002142137149348855 8 Purity/1FRt0SRqqt0FBt/FRt_purity‚0.18158066272735596 7 Purity/2FRt1SRqqt*FBt/FRt_purity‚0.2553845942020416 9"Purity/0FRt2SRqqt*FBt/SRqqt_purity‚0.3570649325847626 ,&Purity/4FRt0SRqqt1FBt/event_proportion‚0 @&Purity/4FRt1SRqqt*FBt/event_proportion‚6.300403038039804e-05 7 Purity/0FRt2SRqqt2FBt/FBt_purity‚0.9807692170143127 *"Purity/0FRt1SRqqt3FBt/SRqqt_purity‚0.5 >&Purity/0FRt*SRqqt2FBt/event_proportion‚0.03944052383303642 A&Purity/0FRt*SRqqt4FBt/event_proportion‚0.00025201612152159214 7 Purity/0FRt0SRqqt1FBt/FBt_purity‚0.9985072016716003 :"Purity/0FRt2SRqqt*FBt/event_purity‚0.03772607445716858 & Purity/1FRt3SRqqt*FBt/FBt_purity‚1 7 Purity/2FRt2SRqqt*FBt/FBt_purity‚0.9821428656578064 ?&Purity/3FRt1SRqqt*FBt/event_proportion‚0.003717237850651145 jet/accuracy_6_of_6‚0 8 Purity/*FRt*SRqqt3FBt/FRt_purity‚0.44117647409439087 7 Purity/1FRt*SRqqt1FBt/FBt_purity‚0.9968535304069519 ,&Purity/2FRt4SRqqt1FBt/event_proportion‚0 8 Purity/3FRt2SRqqt*FBt/FRt_purity‚0.20023149251937866 ,&Purity/4FRt1SRqqt3FBt/event_proportion‚0 ("Purity/*FRt4SRqqt0FBt/event_purity‚0 8 Purity/2FRt4SRqqt*FBt/FRt_purity‚0.30000001192092896 6 Purity/3FRt3SRqqt*FBt/FRt_purity‚0.380952388048172 7 Purity/0FRt1SRqqt*FBt/FBt_purity‚0.9837377071380615 +jet/accuracy_3_of_6‚0.12280702590942383 ) Purity/3FRt1SRqqt2FBt/FRt_purity‚0.25 7 Purity/0FRt2SRqqt3FBt/FBt_purity‚0.6666666865348816 =&Purity/*FRt*SRqqt*FBt/event_proportion‚0.9591733813285828 8 Purity/1FRt2SRqqt0FBt/FRt_purity‚0.26159390807151794 ("Purity/1FRt2SRqqt1FBt/event_purity‚0 9"Purity/*FRt3SRqqt2FBt/SRqqt_purity‚0.3333333432674408 @&Purity/*FRt1SRqqt4FBt/event_proportion‚6.300403038039804e-05 >&Purity/0FRt*SRqqt0FBt/event_proportion‚0.21931703388690948 7 Purity/2FRt1SRqqt1FBt/FBt_purity‚0.9919354915618896 9"Purity/3FRt*SRqqt1FBt/SRqqt_purity‚0.6666666865348816 +"Purity/0FRt4SRqqt1FBt/SRqqt_purity‚0.25 & Purity/*FRt*SRqqt4FBt/FRt_purity‚0 ("Purity/1FRt1SRqqt2FBt/event_purity‚0 ?&Purity/2FRt0SRqqt0FBt/event_proportion‚0.024571573361754417 particle/accuracy_2_of_5‚1 7 Purity/2FRt3SRqqt1FBt/FRt_purity‚0.3499999940395355 +"Purity/3FRt4SRqqt0FBt/SRqqt_purity‚0.25 + Purity/0FRt3SRqqt*FBt/FBt_purity‚0.8125 8 Purity/1FRt*SRqqt*FBt/FRt_purity‚0.26818332076072693 @&Purity/1FRt*SRqqt3FBt/event_proportion‚0.0013230846961960196 ,&Purity/2FRt3SRqqt2FBt/event_proportion‚0 & Purity/3FRt2SRqqt1FBt/FBt_purity‚1 :"Purity/*FRt*SRqqt1FBt/event_purity‚0.39481866359710693 9"Purity/0FRt0SRqqt*FBt/event_purity‚0.8147090673446655 9"Purity/*FRt3SRqqt1FBt/SRqqt_purity‚0.3928571343421936 ?&Purity/0FRt0SRqqt3FBt/event_proportion‚0.002583165420219302 8 Purity/1FRt2SRqqt1FBt/FRt_purity‚0.26589861512184143 8 Purity/3FRt0SRqqt*FBt/FRt_purity‚0.24242424964904785 ,&Purity/4FRt2SRqqt1FBt/event_proportion‚0 ,&Purity/4FRt3SRqqt*FBt/event_proportion‚0 jet/accuracy_5_of_6‚0 9"Purity/*FRt*SRqqt2FBt/SRqqt_purity‚0.5245307683944702 :"Purity/0FRt*SRqqt*FBt/SRqqt_purity‚0.35810303688049316 9"Purity/0FRt*SRqqt1FBt/event_purity‚0.6384262442588806 A&Purity/1FRt1SRqqt3FBt/event_proportion‚0.00025201612152159214 9"Purity/0FRt2SRqqt2FBt/SRqqt_purity‚0.4423076808452606 & Purity/3FRt1SRqqt2FBt/FBt_purity‚1 & Purity/2FRt0SRqqt3FBt/FRt_purity‚0 8"Purity/1FRt2SRqqt0FBt/SRqqt_purity‚0.346381813287735 ?&Purity/2FRt1SRqqt1FBt/event_proportion‚0.007875503972172737 ?&Purity/2FRt2SRqqt1FBt/event_proportion‚0.003969253972172737 7 Purity/2FRt3SRqqt0FBt/FRt_purity‚0.3261904716491699 ("Purity/3FRt0SRqqt1FBt/event_purity‚0 ,&Purity/4FRt*SRqqt2FBt/event_proportion‚0 ,&Purity/4FRt*SRqqt4FBt/event_proportion‚0 @&Purity/4FRt0SRqqt0FBt/event_proportion‚6.300403038039804e-05 9"Purity/*FRt0SRqqt1FBt/event_purity‚0.6701472401618958 @&Purity/*FRt0SRqqt4FBt/event_proportion‚0.0003150201519019902 9"Purity/*FRt1SRqqt2FBt/SRqqt_purity‚0.5460367798805237 :"Purity/0FRt*SRqqt0FBt/SRqqt_purity‚0.32749179005622864 & Purity/3FRt0SRqqt1FBt/FBt_purity‚1 @&Purity/3FRt3SRqqt0FBt/event_proportion‚0.0005670362734235823 jet/accuracy_5_of_5‚0 particle/accuracy_1_of_6‚1 9"Purity/*FRt3SRqqt*FBt/SRqqt_purity‚0.3065526783466339 >&Purity/0FRt2SRqqt*FBt/event_proportion‚0.05361643061041832 7 Purity/1FRt0SRqqt1FBt/FRt_purity‚0.3694809377193451 ,&Purity/1FRt4SRqqt4FBt/event_proportion‚0 @&Purity/3FRt0SRqqt1FBt/event_proportion‚0.0011340725468471646 *jet/accuracy_1_of_5‚0.9376081228256226 9"Purity/0FRt3SRqqt*FBt/SRqqt_purity‚0.3293084502220154 ("Purity/1FRt4SRqqt0FBt/event_purity‚0 7 Purity/*FRt2SRqqt*FBt/FBt_purity‚0.9840930104255676 7 Purity/0FRt*SRqqt*FBt/FBt_purity‚0.9848901033401489 =&Purity/0FRt1SRqqt*FBt/event_proportion‚0.2305317521095276 9"Purity/2FRt0SRqqt2FBt/event_purity‚0.0357142873108387 ("Purity/3FRt*SRqqt*FBt/event_purity‚0 @&Purity/3FRt0SRqqt2FBt/event_proportion‚6.300403038039804e-05 9"Purity/3FRt1SRqqt0FBt/SRqqt_purity‚0.4416666030883789 +"Purity/3FRt*SRqqt2FBt/SRqqt_purity‚0.25 ;"Purity/1FRt*SRqqt2FBt/event_purity‚0.054287221282720566 ?&Purity/1FRt2SRqqt1FBt/event_proportion‚0.008946572430431843 ?&Purity/1FRt3SRqqt0FBt/event_proportion‚0.007938507944345474 ?&Purity/2FRt3SRqqt*FBt/event_proportion‚0.004032257944345474 & Purity/3FRt*SRqqt*FBt/FBt_purity‚1 7 Purity/*FRt*SRqqt4FBt/FBt_purity‚0.8333333134651184 +"Purity/0FRt4SRqqt*FBt/SRqqt_purity‚0.25 +"Purity/3FRt1SRqqt2FBt/SRqqt_purity‚0.25 ,&Purity/2FRt4SRqqt4FBt/event_proportion‚0 7 Purity/*FRt2SRqqt2FBt/FRt_purity‚0.3717948794364929 ("Purity/1FRt2SRqqt2FBt/event_purity‚0 7 Purity/*FRt3SRqqt1FBt/FBt_purity‚0.9523809552192688 ?&Purity/2FRt2SRqqt0FBt/event_proportion‚0.015435988083481789 particle/accuracy_4_of_5‚1 7 Purity/3FRt*SRqqt2FBt/FRt_purity‚0.2666666805744171 ("Purity/*FRt1SRqqt4FBt/SRqqt_purity‚1 % _timestamp‚1.7884884648363016e+09 9"Purity/*FRt*SRqqt*FBt/SRqqt_purity‚0.3680509924888611 ,&Purity/0FRt3SRqqt2FBt/event_proportion‚0 A&Purity/0FRt4SRqqt0FBt/event_proportion‚0.00044102821266278625 ("Purity/3FRt*SRqqt1FBt/event_purity‚0 ,&Purity/4FRt4SRqqt3FBt/event_proportion‚0 ("Purity/0FRt4SRqqt1FBt/event_purity‚0 8 Purity/1FRt1SRqqt0FBt/FRt_purity‚0.28188520669937134 :"Purity/1FRt2SRqqt1FBt/SRqqt_purity‚0.39193546772003174 @&Purity/3FRt*SRqqt2FBt/event_proportion‚0.0003150201519019902 & Purity/2FRt*SRqqt3FBt/FRt_purity‚0 >&Purity/*FRt1SRqqt0FBt/event_proportion‚0.33039313554763794 9"Purity/*FRt*SRqqt0FBt/SRqqt_purity‚0.3332124352455139 >&Purity/*FRt2SRqqt1FBt/event_proportion‚0.02255544438958168 8 Purity/*FRt3SRqqt*FBt/FRt_purity‚0.30743032693862915 7 Purity/0FRt1SRqqt2FBt/FBt_purity‚0.9648147821426392 @&Purity/0FRt4SRqqt*FBt/event_proportion‚0.0005040322430431843 =&Purity/1FRt1SRqqt*FBt/event_proportion‚0.1723790317773819 ,&Purity/3FRt*SRqqt4FBt/event_proportion‚0 ("Purity/3FRt2SRqqt0FBt/event_purity‚0 7 Purity/*FRt0SRqqt4FBt/FBt_purity‚0.8500000238418579 7 Purity/0FRt1SRqqt3FBt/FBt_purity‚0.8999999761581421 & Purity/4FRt0SRqqt0FBt/FRt_purity‚0 ("Purity/1FRt0SRqqt3FBt/event_purity‚0 >&Purity/1FRt1SRqqt1FBt/event_proportion‚0.03944052383303642 ,&Purity/4FRt0SRqqt4FBt/event_proportion‚0 particle/accuracy_2_of_7‚1 particle/accuracy_6_of_7‚1 ) Purity/*FRt1SRqqt4FBt/FBt_purity‚0.75 & Purity/1FRt3SRqqt2FBt/FBt_purity‚1 & Purity/2FRt2SRqqt2FBt/FBt_purity‚1 ,&Purity/0FRt1SRqqt4FBt/event_proportion‚0 7 Purity/0FRt2SRqqt*FBt/FBt_purity‚0.9685100317001343 @&Purity/0FRt3SRqqt1FBt/event_proportion‚0.0006300403038039804 >&Purity/2FRt1SRqqt0FBt/event_proportion‚0.03181703761219978 ("Purity/3FRt0SRqqt*FBt/event_purity‚0 ,&Purity/4FRt2SRqqt3FBt/event_proportion‚0 particle/accuracy_2_of_4‚1 7 Purity/*FRt0SRqqt3FBt/FRt_purity‚0.3333333432674408 >&Purity/1FRt2SRqqt0FBt/event_proportion‚0.04731602966785431 9"Purity/2FRt1SRqqt2FBt/SRqqt_purity‚0.7456140518188477 @&Purity/4FRt*SRqqt*FBt/event_proportion‚0.0001890120911411941 & Purity/3FRt*SRqqt2FBt/FBt_purity‚1 7 Purity/0FRt*SRqqt2FBt/FBt_purity‚0.9810502529144287 7 Purity/0FRt*SRqqt3FBt/FBt_purity‚0.8950618505477905 ?&Purity/0FRt2SRqqt1FBt/event_proportion‚0.009261592291295528 :"Purity/1FRt2SRqqt*FBt/SRqqt_purity‚0.36140623688697815 particle/accuracy_1_of_2‚1 7 Purity/*FRt2SRqqt2FBt/FBt_purity‚0.9912281036376953 7 Purity/*FRt2SRqqt3FBt/FBt_purity‚0.6666666865348816 >&Purity/*FRt3SRqqt0FBt/event_proportion‚0.01827116869390011 ;"Purity/2FRt0SRqqt1FBt/event_purity‚0.034259259700775146 ("Purity/2FRt3SRqqt*FBt/event_purity‚0 ,&Purity/4FRt*SRqqt1FBt/event_proportion‚0 ,&Purity/4FRt4SRqqt2FBt/event_proportion‚0 7 Purity/3FRt4SRqqt0FBt/FRt_purity‚0.1666666716337204 ("Purity/*FRt3SRqqt0FBt/event_purity‚0 :"Purity/1FRt0SRqqt*FBt/event_purity‚0.18423162400722504 ,&Purity/2FRt4SRqqt3FBt/event_proportion‚0 9"Purity/*FRt*SRqqt3FBt/SRqqt_purity‚0.6190475821495056 ) Purity/*FRt1SRqqt3FBt/FRt_purity‚0.75 :"Purity/*FRt0SRqqt*FBt/event_purity‚0.37596097588539124 :"Purity/1FRt3SRqqt0FBt/SRqqt_purity‚0.27901944518089294 ,&Purity/4FRt4SRqqt0FBt/event_proportion‚0 +jet/accuracy_2_of_5‚0.42225030064582825 jet/accuracy_1_of_6‚1 ("Purity/1FRt4SRqqt*FBt/event_purity‚0 & Purity/*FRt1SRqqt4FBt/FRt_purity‚0 9"Purity/0FRt*SRqqt1FBt/SRqqt_purity‚0.4590678811073303 9"Purity/0FRt*SRqqt2FBt/SRqqt_purity‚0.4588888883590698 ,&Purity/0FRt4SRqqt4FBt/event_proportion‚0 :"Purity/2FRt1SRqqt0FBt/SRqqt_purity‚0.37284016609191895 ("Purity/2FRt2SRqqt0FBt/event_purity‚0 @&Purity/3FRt*SRqqt0FBt/event_proportion‚0.0071824598126113415 & Purity/3FRt1SRqqt*FBt/FBt_purity‚1 +"Purity/1FRt1SRqqt3FBt/SRqqt_purity‚0.75 9"Purity/*FRt2SRqqt*FBt/event_purity‚0.0152859166264534 ("Purity/*FRt3SRqqt1FBt/event_purity‚0 >&Purity/0FRt0SRqqt0FBt/event_proportion‚0.04082661122083664 7 Purity/1FRt1SRqqt1FBt/FRt_purity‚0.2860495150089264 @&Purity/3FRt1SRqqt1FBt/event_proportion‚0.0009450604557059705 & Purity/4FRt1SRqqt0FBt/FRt_purity‚0 ?&Purity/0FRt3SRqqt*FBt/event_proportion‚0.006741431541740894 :"Purity/*FRt*SRqqt1FBt/SRqqt_purity‚0.48319631814956665 jet/accuracy_7_of_7‚0 particle/accuracy_1_of_7‚1 & Purity/*FRt0SRqqt4FBt/FRt_purity‚0 & Purity/1FRt3SRqqt2FBt/FRt_purity‚0 ( Purity/2FRt2SRqqt2FBt/FRt_purity‚0.5 >&Purity/0FRt*SRqqt*FBt/event_proportion‚0.40996724367141724 ("Purity/*FRt2SRqqt2FBt/event_purity‚0 & Purity/2FRt3SRqqt1FBt/FBt_purity‚1 particle/accuracy_5_of_7‚1 >&Purity/*FRt0SRqqt0FBt/event_proportion‚0.15757308900356293 9"Purity/*FRt1SRqqt0FBt/event_purity‚0.1723441779613495 9"Purity/2FRt1SRqqt*FBt/SRqqt_purity‚0.4197915196418762 ("Purity/2FRt2SRqqt*FBt/event_purity‚0 7 Purity/3FRt2SRqqt1FBt/FRt_purity‚0.1111111119389534 ("Purity/*FRt1SRqqt3FBt/event_purity‚0 @&Purity/*FRt4SRqqt1FBt/event_proportion‚6.300403038039804e-05 ?&Purity/0FRt1SRqqt2FBt/event_proportion‚0.008757560513913631 ("Purity/1FRt*SRqqt3FBt/event_purity‚0 7 Purity/2FRt0SRqqt2FBt/FRt_purity‚0.4791666865348816 @&Purity/3FRt2SRqqt*FBt/event_proportion‚0.0028351815417408943 ,&Purity/4FRt3SRqqt1FBt/event_proportion‚0 ,&Purity/4FRt4SRqqt4FBt/event_proportion‚0 *jet/accuracy_1_of_3‚0.8415170907974243 6 Purity/*FRt*SRqqt2FBt/FBt_purity‚0.982401430606842 7 Purity/*FRt*SRqqt3FBt/FBt_purity‚0.9038315415382385 9"Purity/*FRt2SRqqt1FBt/SRqqt_purity‚0.4442177712917328 ,&Purity/*FRt4SRqqt2FBt/event_proportion‚0 9"Purity/1FRt*SRqqt*FBt/SRqqt_purity‚0.3710376024246216 9"Purity/1FRt0SRqqt1FBt/event_purity‚0.2309117317199707 8 Purity/2FRt1SRqqt1FBt/FRt_purity‚0.32478880882263184 @&Purity/3FRt3SRqqt*FBt/event_proportion‚0.0005670362734235823 >&Purity/0FRt0SRqqt1FBt/event_proportion‚0.08606351166963577 7 Purity/1FRt*SRqqt1FBt/FRt_purity‚0.3244644105434418 ;"Purity/2FRt*SRqqt*FBt/event_purity‚0.008074735291302204 (particle/accuracy‚0.8473936915397644’¼m]¤Kê‚(:À –ÖèÔÙÃù® disk.md0.out‚ 304.640625 network.recv‚ 1.061233e+06% disk./.usageGB‚2983.0506744384766 disk.rbd2.in‚0 disk.sdb.out‚317.9033203125# disk.nvme1n1.out‚18638.66796875) proc.memory.percent‚23.99296760559082 disk.nvme1n1.in‚4.515625 disk.sda.in‚ 0.0078125 disk.sda.out‚317.9033203125 disk.rbd0.out‚ 0.62109375 disk.sdb2.in‚0 cpu‚25.540699612821527 network.sent‚ 4.307492e+06 disk.rbd0.in‚0 disk.sdb1.in‚0 disk.sda1.out‚0 disk.md0p1.in‚ 0.0078125 disk.md0p1.out‚ 304.640625" proc.memory.rssMB‚ 3931.0078125# disk.nvme0n1.out‚24772.51953125 disk.dm-0.in‚9.109375 disk.md0.in‚ 0.0078125$ memory_percent‚83.21068286895752) disk./.usagePercent‚83.40734536100068) proc.memory.availableMB‚ 2750.76171875 disk.sdb2.out‚317.9033203125 disk.rbd2.out‚ 0.783203125 disk.sdb.in‚0 disk.dm-0.out‚ 43411.1875 proc.cpu.threads‚17 disk.sdb1.out‚0 disk.sda2.in‚ 0.0078125 disk.sda2.out‚317.9033203125 disk.nvme0n1.in‚4.59375 disk.sda1.in‚0]/‘ë‚(:† –ÖèÔŽˆê° gpu.0.gpu‚6.0 gpu.0.memory‚0+ gpu.0.memoryAllocated‚47.54737854003906* gpu.0.memoryAllocatedBytes‚ 40842887168 gpu.0.temp‚34.0 gpu.0.powerWatts‚87.119' gpu.0.enforcedPowerLimitWatts‚400.0 gpu.0.powerPercent‚21.77975 gpu.0.smClock‚1155 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0°XÐœžì‚(:“ ¥ÖèÔ¤Ö˜® gpu.0.gpu‚84.0 gpu.0.memory‚20+ gpu.0.memoryAllocated‚47.54737854003906* gpu.0.memoryAllocatedBytes‚ 40842887168 gpu.0.temp‚43.0 gpu.0.powerWatts‚122.818' gpu.0.enforcedPowerLimitWatts‚400.0) gpu.0.powerPercent‚30.704500000000003 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0 B`Q•í‚(:¾ ¥ÖèÔ££Û± disk.rbd2.out‚ 0.783203125 disk.sdb.in‚0 disk.sda.out‚319.55224609375 disk.nvme0n1.out‚ 25524.84375 disk.sda1.out‚0 disk.md0p1.out‚ 306.21875 network.sent‚ 4.412372e+06 disk.rbd0.in‚0 disk.dm-0.in‚9.109375 disk.md0p1.in‚ 0.0078125* proc.memory.percent‚18.656110763549805 network.recv‚ 1.066536e+06 disk.sda.in‚ 0.0078125 disk.sdb1.out‚0 disk.sda1.in‚0 disk.rbd2.in‚0 disk.nvme1n1.in‚4.515625# disk.nvme1n1.out‚21888.05859375 disk.sdb2.in‚0# memory_percent‚77.8696060180664% disk./.usageGB‚2983.5801582336426! disk.sdb2.out‚319.55224609375 proc.cpu.threads‚17 disk.sdb1.in‚0" proc.memory.rssMB‚ 3056.6171875 cpu‚25.53684473033949 disk.sdb.out‚319.55224609375! disk.sda2.out‚319.55224609375 disk.nvme0n1.in‚4.59375 disk.dm-0.out‚47412.90234375 disk.md0.in Ü’C´‚ 0.0078125 disk.md0.out‚ 306.21875& proc.memory.availableMB‚ 3625.84375 disk.sda2.in‚ 0.0078125 disk.rbd0.out‚ 0.62109375( disk./.usagePercent‚83.4221499495197^YÈ‹Iî‚(:¾ ´ÖèÔÌš÷® proc.cpu.threads‚17 network.sent‚ 4.417955e+06 disk.nvme1n1.in‚4.515625 disk.nvme0n1.in‚4.59375 disk.dm-0.in‚9.109375 disk.md0.in‚ 0.0078125 cpu‚25.535520068154206# disk./.usageGB‚2983.58016204834 disk.sda2.in‚ 0.0078125 disk.sda.in‚ 0.0078125 disk.sda.out‚320.7685546875 disk.rbd0.out‚ 0.62109375" disk.nvme0n1.out‚ 25524.8984375 disk.dm-0.out‚ 47481.921875 disk.rbd2.out‚ 0.783203125 disk.sdb2.out‚320.7685546875 disk.md0.out‚ 307.3828125 disk.md0p1.out‚ 307.3828125 disk.sda2.out‚320.7685546875) disk./.usagePercent‚83.42215005618023 disk.sdb1.out‚0 disk.sda1.out‚0 disk.rbd2.in‚0$ memory_percent‚77.86970138549805" disk.nvme1n1.out‚ 21957.0234375 disk.rbd0.in‚0 disk.sda1.in‚0 disk.md0p1.in‚ 0.0078125 disk.sdb.in‚0" proc.memory.rssMB‚ 3056.6171875* proc.memory.percent‚18.656110763549805 network.recv‚ 1.07094e+06 disk.sdb1.in‚0 disk.sdb2.in‚0 disk.sdb.out‚320.7685546875' proc.memory.availableMB‚ 3625.828125‹”|€‘ï‚(:† ´ÖèÔ›ðü¯ gpu.0.gpu‚98.0 gpu.0.memory‚23+ gpu.0.memoryAllocated‚47.54737854003906* gpu.0.memoryAllocatedBytes‚ 40842887168 gpu.0.temp‚48.0 gpu.0.powerWatts‚313.22' gpu.0.enforcedPowerLimitWatts‚400.0 gpu.0.powerPercent‚78.305 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0ÐøÂÔ”ð‚(:‰ ÃÖèÔðúð¯ gpu.0.gpu‚98.0 gpu.0.memory‚23+ gpu.0.memoryAllocated‚47.54737854003906* gpu.0.memoryAllocatedBytes‚ 40842887168 gpu.0.temp‚45.0 gpu.0.powerWatts‚236.709' gpu.0.enforcedPowerLimitWatts‚400.0 gpu.0.powerPercent‚59.17725 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0‰3±ŸSñ‚(:È ÃÖèÔ—‰µ± disk.md0.out‚ 308.99609375 disk.rbd0.in‚0 disk.dm-0.in‚9.109375 proc.cpu.threads‚17 disk.nvme0n1.in‚4.59375 disk.md0.in‚ 0.0078125 disk.sdb2.in‚0 disk.rbd2.in‚0( proc.memory.availableMB‚ 3626.0546875 disk.sda.out‚322.45361328125 disk.dm-0.out‚47579.30859375 cpu‚25.534123078633854 disk.sdb.out‚322.45361328125 disk.sda.in‚ 0.0078125$ memory_percent‚77.86831855773926! disk.nvme1n1.out‚ 22054.234375 disk.rbd0.out‚ 0.62109375# disk.nvme0n1.out‚25525.07421875 network.recv‚ 1.07371e+06 disk.sda1.in‚0 disk.sda1.out‚0 disk.rbd2.out‚ 0.783203125 disk.sdb.in‚0 network.sent‚ 4.420224e+06 disk.sdb1.out‚0 disk.md0p1.in‚ 0.0078125 disk.md0p1.out‚ 308.99609375 disk.sda2.in‚ 0.0078125! disk.sda2.out‚322.45361328125" proc.memory.rssMB‚ 3056.6171875* proc.memory.percent‚18.656110763549805) disk./.usagePercent‚83.42228231524007$ disk./.usageGB‚2983.584892272949 disk.sdb1.in‚0! disk.sdb2.out‚322.45361328125 disk.nvme1n1.in‚4.515625 ‡œò‚(:‘ ÒÖèÔûý° gpu.0.gpu‚97.0 gpu.0.memory‚17+ gpu.0.memoryAllocated‚47.54737854003906* gpu.0.memoryAllocatedBytes‚ 40842887168 gpu.0.temp‚45.0 gpu.0.powerWatts‚250.62' gpu.0.enforcedPowerLimitWatts‚400.0( gpu.0.powerPercent‚62.65500000000001 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0XÏèªJó‚(:¿ ÒÖèÔÔëر$ memory_percent‚77.86829471588135 disk.dm-0.in‚9.109375 disk.sdb2.in‚0" proc.memory.rssMB‚ 3056.6171875 disk.sdb2.out‚ 323.91796875 disk.rbd2.in‚0 disk.sda2.in‚ 0.0078125# disk.nvme1n1.out‚22214.99609375 disk.sda.in‚ 0.0078125 cpu‚25.532848143968845 network.sent‚ 4.428405e+06 network.recv‚ 1.078371e+06$ disk./.usageGB‚2983.584892272949 disk.md0p1.out‚ 310.41796875 proc.cpu.threads‚17# disk.nvme0n1.out‚25525.13671875 disk.dm-0.out‚ 47740.1328125 disk.sdb1.out‚0 disk.md0p1.in‚ 0.0078125 disk.sdb.out‚ 323.91796875 disk.sdb1.in‚0 disk.sda1.out‚0 disk.sdb.in‚0 disk.nvme1n1.in‚4.515625 disk.rbd0.in‚0) proc.memory.availableMB‚ 3626.05859375* proc.memory.percent‚18.656110763549805) disk./.usagePercent‚83.42228231524007 disk.nvme0n1.in‚4.59375 disk.sda1.in‚0 disk.rbd2.out‚ 0.783203125 disk.sda2.out‚ 323.91796875 disk.md0.in‚ 0.0078125 disk.md0.out‚ 310.41796875 disk.sda.out‚ 323.91796875 disk.rbd0.out‚ 0.62109375Žhí2ô- _wandbruntime‚3333  _runtime‚3333/æ‰ Åõ¿ % lr-AdamW/pg1‚0.000996376697171982 % lr-AdamW/pg2‚0.000996376697171982 trainer/global_step‚1849 % _timestamp‚1.7884885347163484e+09 _runtime‚3333.817777344 _step‚147y9êÊöÄ % lr-AdamW/pg1‚0.000996376697171982 % lr-AdamW/pg2‚0.000996376697171982 trainer/global_step‚1849 % _timestamp‚1.7884885347163484e+09 _runtime‚3333.817777344 _step‚147“mùñ§2÷- _wandbruntime‚3333  _runtime‚3333ßö‹ò'øìO UADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚397303145805812 ^LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 4453762875392 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚23981 M@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚20996678 P@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 38610163200 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚566 \HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚392857541549232 E8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚61476853 WGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 35913877504 YFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚87798332020224 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 H?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚1534 VBDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated‚413697644206080 PCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚26691509 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚6 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚19684 YIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 37073453056 KBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚1534 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 RCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 2541791232 YJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 2514370140 RBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 40290484224 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 TGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚34805028 ^LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 4448118626720 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 ZHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 4451246512128 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 A8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚3681 KBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚2147 PCDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated‚34805028 [IDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 4453762875392 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚12721836 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚12721273 P@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 36985723401 H?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚2147 RCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 2539798108 trainer/global_step‚1849 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 F=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚3681 SDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 3142753280 \HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚409243855902720 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 WGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 34283657216 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 `LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚392857566977200 ZHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 4445604256580 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚3 dQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚82240231412736 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 963120128 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 WEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 4451246512128 `LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚409243881330688 A8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚3681 O?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 40290484224 O@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 2541791232 L?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚34785344 TGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚26691515 TEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 1515847168 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 L?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚26691509 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚8275408 [KDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 37073453056 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 VGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 2516363264 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚25427968 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 D>DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 ZKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 3217031168 D;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚3681 YEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚409243855902720 SGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚8275405 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚563 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚25427968 G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚2684 VGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 3142753280 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 TDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 40290484224 UFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 3217031168 $ _timestamp‚1.788488534717426e+09 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚19690 ]IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚409243881330688 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 YEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚397305685603920 _step‚148 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚19684 YEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚413697644206080 UADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚413695102414848 cQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 5558199130624 TDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 35913877504 QDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚20997244 H?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚1309 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚69425152 _MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 5558129705472 BDeviceStatsMonitor.on_train_batch_start/allocation.all.current‚19690 XIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 3217031168 M@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚61496543 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚23682 UHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚98523136 IDeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚23682 R>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚413695102414848 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚23981 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚6 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚25427968 IDeviceStatsMonitor.on_train_batch_start/allocation.all.current‚19690 XIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 3217031168 M@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚61496543 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚23682 UHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚98523136 IDeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚23682 R>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚413695102414848 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚23981 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚6 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚25427968 IDeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 ZKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 3217031168 D;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚3681 YEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚409243855902720 SGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚8275405 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚563 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚25427968 G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚2684 VGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 3142753280 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 TDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 40290484224 UFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 3217031168 $ _timestamp‚1.788488534717426e+09 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚19690 ]IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚409243881330688 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 YEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚397305685603920 _step‚148”5ý´2ú- _wandbruntime‚3335  _runtime‚3335sk.ûøM REDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚34823396 % _timestamp‚1.7884885362571287e+09 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚6 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1309 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚12728333 bODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚82284361737728 LDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚330 S?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚397511014602042 B9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚3681 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚23981 RBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 35913877504 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚8279774 PADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 2539798316 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 ZFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚393063057209728 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 D;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚3681 TEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 2516376576 UCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 4453602608128 XFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 4453602608128 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 KBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚1534 ^JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚393063082637696 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚8279771 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚26705399 YGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 4456118984704 TEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 3142753280 K>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚61528801 G:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚61509085 [HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚87845503175680 EDeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 2541804544 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚19710 WCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚397513554400358 I@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚1534 S?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚413911404581376 WHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 2516376576 aODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 5561141437952 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 577553920 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 ^JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚409457827401216 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 ?6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚3681 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 J=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚34803686 [GDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚409457827401216 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚29097984 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚19716 NADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚34823396 PDeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚21007530 N>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 36985723401 \JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 4450471762662 _runtime‚3335.359135749 N>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 38610163200 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚23981 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚3 LDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚986 RCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 1515847168 ]KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 5561069928960 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 WGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 37073453056 K;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 38610163200 NADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚34803686 YIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 37073453056 REDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚26705405 üø'üýM LDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚330 S?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚397511014602042 B9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚3681 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚23981 RBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 35913877504 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚8279774 PADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 2539798316 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 ZFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚393063057209728 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 D;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚3681 TEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 2516376576 UCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 4453602608128 XFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 4453602608128 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 KBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚1534 ^JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚393063082637696 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚8279771 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚26705399 YGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 4456118984704 TEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 3142753280 K>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚61528801 G:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚61509085 [HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚87845503175680 EDeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 2541804544 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚19710 WCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚397513554400358 I@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚1534 S?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚413911404581376 WHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 2516376576 aODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 5561141437952 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 577553920 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 ^JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚409457827401216 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 ?6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚3681 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 J=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚34803686 [GDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚409457827401216 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚29097984 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚19716 NADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚34823396 PDeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚21007530 N>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 36985723401 \JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 4450471762662 _runtime‚3335.359135749 N>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 38610163200 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚23981 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚3 LDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚986 RCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 1515847168 ]KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 5561069928960 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 WGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 37073453056 K;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 38610163200 NADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚34803686 YIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 37073453056 REDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚26705405 REDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚34823396 % _timestamp‚1.7884885362571287e+09 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚6 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1309 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚12728333 bODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚82284361737728•ôô—®2ý- _wandbruntime‚3335  _runtime‚3335¨ZþÔ /loss/FBt1/detection_loss‚3.6226088133725254 /loss/FRt4/detection_loss‚2.5663394045084713 0loss/SRqqt4/detection_loss‚1.667885148343239 epoch‚3 _runtime‚3335.360100224 0loss/FRt1/assignment_loss‚3.1423702681293855 0loss/FRt2/assignment_loss‚1.4417196789925748 1loss/SRqqt4/assignment_loss‚1.693065516636458 1loss/FBt3/assignment_loss‚0.24566108741882173 0loss/SRqqt2/detection_loss‚1.120763001364789 /loss/FBt2/detection_loss‚0.7368668785255882 0loss/FRt4/assignment_loss‚2.7203123435378074 2loss/SRqqt2/assignment_loss‚0.9720042955097077 1loss/FBt4/assignment_loss‚0.19735090658234963 /loss/FRt2/detection_loss‚1.6308826266957277 trainer/global_step‚1849 2loss/SRqqt1/assignment_loss‚1.7578739012990678 1loss/FBt1/assignment_loss‚0.15230446060807232 0loss/SRqqt1/detection_loss‚2.511702682290758 0loss/SRqqt3/detection_loss‚2.419190753423251 % _timestamp‚1.7884885362588882e+09 .loss/FRt3/detection_loss‚2.764569724523104 &loss/total_loss‚44.365681901314865 /loss/FRt3/assignment_loss‚3.642745775442857 0loss/FBt2/assignment_loss‚0.1008882496063011 /loss/FBt3/detection_loss‚2.6054487228393555 _step‚150 2loss/SRqqt3/assignment_loss‚1.8076197934265321 .loss/FBt4/detection_loss‚2.599873666252409 .loss/FRt1/detection_loss‚2.245634201985712k ÀZ_ÿÙ /loss/FBt3/detection_loss‚2.6054487228393555 _step‚150 2loss/SRqqt3/assignment_loss‚1.8076197934265321 .loss/FBt4/detection_loss‚2.599873666252409 .loss/FRt1/detection_loss‚2.245634201985712 /loss/FBt1/detection_loss‚3.6226088133725254 /loss/FRt4/detection_loss‚2.5663394045084713 0loss/SRqqt4/detection_loss‚1.667885148343239 epoch‚3 _runtime‚3335.360100224 0loss/FRt1/assignment_loss‚3.1423702681293855 0loss/FRt2/assignment_loss‚1.4417196789925748 1loss/SRqqt4/assignment_loss‚1.693065516636458 1loss/FBt3/assignment_loss‚0.24566108741882173 0loss/SRqqt2/detection_loss‚1.120763001364789 /loss/FBt2/detection_loss‚0.7368668785255882 0loss/FRt4/assignment_loss‚2.7203123435378074 2loss/SRqqt2/assignment_loss‚0.9720042955097077 1loss/FBt4/assignment_loss‚0.19735090658234963 /loss/FRt2/detection_loss‚1.6308826266957277 trainer/global_step‚1849 2loss/SRqqt1/assignment_loss‚1.7578739012990678 1loss/FBt1/assignment_loss‚0.15230446060807232 0loss/SRqqt1/detection_loss‚2.511702682290758 0loss/SRqqt3/detection_loss‚2.419190753423251 % _timestamp‚1.7884885362588882e+09 .loss/FRt3/detection_loss‚2.764569724523104 &loss/total_loss‚44.365681901314865 /loss/FRt3/assignment_loss‚3.642745775442857 0loss/FBt2/assignment_loss‚0.1008882496063011–ÞìÈÚB€‚(:· áÖèÔþÀ³ disk.md0.in‚ 0.0078125 disk.md0p1.out‚ 311.703125* proc.memory.percent‚18.656110763549805 proc.cpu.threads‚17 disk.sda1.out‚0 disk.md0p1.in‚ 0.0078125 disk.sdb.out‚ 325.255859375% disk./.usageGB‚2983.4328536987305 disk.sda.out‚ 325.255859375 disk.rbd0.out‚ 0.62109375 disk.dm-0.in‚9.109375 disk.md0.out‚ 311.703125 disk.sdb.in‚0 disk.sda2.out‚ 325.255859375) disk./.usagePercent‚83.41803125307182 disk.sda.in‚ 0.0078125 disk.nvme0n1.out‚25525.25 disk.sda2.in‚ 0.0078125" proc.memory.rssMB‚ 3056.6171875 disk.rbd2.in‚0 disk.nvme0n1.in‚4.59375 disk.dm-0.out‚ 47840.328125 disk.sdb2.in‚0 disk.sdb2.out‚ 325.255859375 disk.sda1.in‚0 cpu‚25.531634531361302 disk.rbd0.in‚0 disk.sdb1.in‚0 disk.nvme1n1.in‚4.515625! disk.nvme1n1.out‚ 22315.078125 network.recv‚ 1.082466e+06 network.sent‚ 4.432006e+06$ memory_percent‚77.86862850189209) proc.memory.availableMB‚ 3626.00390625 disk.rbd2.out‚ 0.783203125 disk.sdb1.out‚0tœt’‚(:‡ áÖèÔîëµµ gpu.0.gpu‚97.0 gpu.0.memory‚17+ gpu.0.memoryAllocated‚47.54737854003906* gpu.0.memoryAllocatedBytes‚ 40842887168 gpu.0.temp‚45.0 gpu.0.powerWatts‚246.53' gpu.0.enforcedPowerLimitWatts‚400.0 gpu.0.powerPercent‚61.6325 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0M¬I6”‚‚(:‰ ðÖèÔ°Ÿ® gpu.0.gpu‚97.0 gpu.0.memory‚17+ gpu.0.memoryAllocated‚47.54737854003906* gpu.0.memoryAllocatedBytes‚ 40842887168 gpu.0.temp‚45.0 gpu.0.powerWatts‚229.155' gpu.0.enforcedPowerLimitWatts‚400.0 gpu.0.powerPercent‚57.28875 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0eTÝ«Gƒ‚(:¼ ðÖèÔŒ¸Ã±) disk./.usagePercent‚83.42143244412013 disk.sdb1.in‚0 disk.rbd2.out‚ 0.783203125 disk.sda2.out‚326.4580078125 disk.sda.in‚ 0.0078125 disk.rbd0.out‚ 0.62109375$ memory_percent‚77.86872386932373 proc.cpu.threads‚17 network.sent‚ 4.5215e+06 network.recv‚ 1.088352e+06 disk.md0p1.out‚ 312.84765625 disk.rbd2.in‚0 disk.dm-0.in‚9.109375 disk.md0p1.in‚ 0.0078125 cpu‚25.530343687265024 disk.sda2.in‚ 0.0078125 disk.nvme1n1.in‚4.515625 disk.sda.out‚326.4580078125 disk.sdb2.in‚0 disk.sdb.in‚0 disk.nvme1n1.out‚ 22438.03125 disk.nvme0n1.out‚ 25525.34375 disk.sdb1.out‚0 disk.sda1.in‚0 disk.sda1.out‚0 disk.md0.in‚ 0.0078125 disk.rbd0.in‚0" proc.memory.rssMB‚ 3056.6171875 disk.nvme0n1.in‚4.59375 disk.dm-0.out‚ 47963.375 disk.sdb2.out‚326.4580078125 disk.sdb.out‚326.4580078125) proc.memory.availableMB‚ 3625.98828125* proc.memory.percent‚18.656110763549805% disk./.usageGB‚2983.5544967651367 disk.md0.out‚ 312.84765625ÝB°§„‚(:… ÿÖèÔŠ‰‘° gpu.0.gpu‚0.0 gpu.0.memory‚0+ gpu.0.memoryAllocated‚47.54737854003906* gpu.0.memoryAllocatedBytes‚ 40842887168 gpu.0.temp‚44.0 gpu.0.powerWatts‚309.212' gpu.0.enforcedPowerLimitWatts‚400.0 gpu.0.powerPercent‚77.303 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚03ߎJ…‚(:¿ ÿÖèÔå´€±( disk./.usagePercent‚83.4301484228235 disk.rbd2.in‚0( proc.memory.availableMB‚ 3625.7421875 network.sent‚ 4.526234e+06 disk.dm-0.out‚ 48132.921875 disk.sda1.out‚0 disk.sdb.in‚0 disk.rbd0.in‚0" proc.memory.rssMB‚ 3056.6171875 cpu‚25.529262194968833 disk.dm-0.in‚9.109375 disk.nvme1n1.out‚22538.5* proc.memory.percent‚18.656110763549805 disk.md0.in‚ 0.0078125! disk.sda2.out‚328.28076171875 disk.sda.in‚ 0.0078125 network.recv‚ 1.091954e+06! disk.nvme0n1.out‚ 25594.421875 disk.md0.out‚ 314.59765625 disk.sdb.out‚328.28076171875 disk.rbd0.out‚ 0.62109375$ disk./.usageGB‚2983.866222381592 disk.sdb2.in‚0 disk.sda1.in‚0$ memory_percent‚77.87022590637207 disk.nvme0n1.in‚4.59375 disk.sdb1.in‚0! disk.sdb2.out‚328.28076171875 disk.md0p1.in‚ 0.0078125 disk.rbd2.out‚ 0.783203125 disk.sda.out‚328.28076171875 proc.cpu.threads‚17 disk.sdb1.out‚0 disk.md0p1.out‚ 314.59765625 disk.sda2.in‚ 0.0078125 disk.nvme1n1.in‚4.515625Ýýv”†‚(:‰ Ž×èÔÄôÒ¯ gpu.0.gpu‚97.0 gpu.0.memory‚18+ gpu.0.memoryAllocated‚47.54737854003906* gpu.0.memoryAllocatedBytes‚ 40842887168 gpu.0.temp‚41.0 gpu.0.powerWatts‚117.193' gpu.0.enforcedPowerLimitWatts‚400.0 gpu.0.powerPercent‚29.29825 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0ÕŠö‘E‡‚(:º Ž×èÔ•²â° disk.sdb2.in‚0 disk.sda.in‚ 0.0078125 disk.dm-0.out‚ 48323.6328125 disk.sda1.out‚0 disk.md0.in‚ 0.0078125 disk.sdb.out‚ 329.564453125 disk.nvme1n1.in‚4.515625$ memory_percent‚72.76358604431152( proc.memory.availableMB‚ 4462.4140625" proc.memory.rssMB‚ 3056.6171875 disk.md0.out‚ 315.8203125 disk.md0p1.in‚ 0.0078125 disk.sda.out‚ 329.564453125 disk.rbd2.in‚0 disk.sda2.in‚ 0.0078125 disk.rbd0.out‚ 0.62109375$ disk./.usageGB‚2984.085594177246 disk.dm-0.in‚9.109375 disk.sdb2.out‚ 329.564453125 disk.md0p1.out‚ 315.8203125" disk.nvme1n1.out‚ 22683.2578125* proc.memory.percent‚18.656110763549805 network.recv‚ 1.096285e+06 disk.sdb.in‚0 disk.rbd0.in‚0 proc.cpu.threads‚17 disk.nvme0n1.in‚4.59375 disk.sdb1.in‚0 cpu‚25.527965943121718 network.sent‚ 4.532295e+06) disk./.usagePercent‚83.43628215004424 disk.sda1.in‚0 disk.rbd2.out‚ 0.783203125 disk.sda2.out‚ 329.564453125 disk.nvme0n1.out‚ 25640.375 disk.sdb1.out‚0aáAžˆ‚(:“ ×èÔçƯ gpu.0.gpu‚98.0 gpu.0.memory‚17+ gpu.0.memoryAllocated‚47.54737854003906* gpu.0.memoryAllocatedBytes‚ 40842887168 gpu.0.temp‚45.0 gpu.0.powerWatts‚226.093' gpu.0.enforcedPowerLimitWatts‚400.0) gpu.0.powerPercent‚56.523250000000004 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0hP÷K‰‚(:À ×èÔç»ù¯ disk.sdb1.out‚0 disk.md0.in‚ 0.0078125 disk.rbd2.out‚ 0.783203125 disk.sda2.out‚331.0576171875 disk.sdb2.in‚0 disk.sda1.out‚0 network.sent‚ 4.538163e+06) disk./.usagePercent‚83.45379452949223 disk.sdb2.out‚331.0576171875 disk.md0.out‚ 317.23828125 disk.md0p1.in‚ 0.0078125 disk.sdb.out‚331.0576171875 disk.nvme0n1.in‚4.59375 disk.rbd0.in‚0 disk.rbd0.out‚ 0.62109375$ memory_percent‚75.16326904296875 disk.sda1.in‚0 network.recv‚ 1.100495e+06# proc.memory.availableMB‚4069.25* proc.memory.percent‚18.656110763549805 cpu‚25.526908891182654 disk.sda.out‚331.0576171875 disk.dm-0.out‚48579.04296875 disk.sdb1.in‚0 disk.md0p1.out‚ 317.23828125 disk.sdb.in‚0 disk.nvme1n1.in‚4.515625# disk.nvme1n1.out‚22800.38671875" proc.memory.rssMB‚ 3056.6171875 disk.rbd2.in‚0 disk.sda.in‚ 0.0078125 disk.sda2.in‚ 0.0078125% disk./.usageGB‚2984.7119216918945 disk.nvme0n1.out‚ 25778.65625 disk.dm-0.in‚9.109375 proc.cpu.threads‚17Øúu˜2Š- _wandbruntime‚3419  _runtime‚3419h„?œÇ‹Á & lr-AdamW/pg1‚0.0009853704275444354 & lr-AdamW/pg2‚0.0009853704275444354 trainer/global_step‚1899 % _timestamp‚1.7884886206321208e+09 _runtime‚3419.733670815 _step‚151o„Œ4ÌŒÆ & lr-AdamW/pg1‚0.0009853704275444354 & lr-AdamW/pg2‚0.0009853704275444354 trainer/global_step‚1899 % _timestamp‚1.7884886206321208e+09 _runtime‚3419.733670815 _step‚151—#;‚2- _wandbruntime‚3419  _runtime‚3419sÙ…ãt"ŽîO JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚578 YEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚407699152587428 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 YJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 2516363264 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚13048702 `MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚84446718564352 _runtime‚3419.735603274 IDeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚23682 UADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚424510238569472 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚3 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚29097984 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 D>DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 D;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚3681 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚19684 M=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 38610163200 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚25427968 ZHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 4569079140352 UADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚407696612789320 O@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 2541791232 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚23981 PCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚27386009 UFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 3217031168 H?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚1534 VGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 2516363264 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚19684 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 100620288 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚71522304 A8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚3681 `LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚403133350002000 VFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 37073453056 YEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚419941159429120 BDeviceStatsMonitor.on_train_batch_start/allocation.all.current‚19690 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 L?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚27386009 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚23981 L?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚35705796 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚6 TEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 1515847168 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚25427968 YJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 2514370140 RCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 2541791232 ^LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 4571595503616 TGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚27386015 NFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚330 RBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 40290484224 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 PCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚35705796 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 TDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 40290484224 O?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 40290484224 [KDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 37073453056 \HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚403133324574032 \HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚419941159429120 M@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚63111495 _MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 5705657380864 KBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚2147 TDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 35913877504 MDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚2147 E8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚63091805 PCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚27386015 PCDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated‚35725480 SDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 3142753280 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚25427968 KBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚1534 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚8493708 P@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 38610163200 QDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚21542410 NFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚986 ]JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚90152476565504 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 963120128 cQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 5705728903168 H?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚2147 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚19690 H?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚1309 P@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 36985723401 _step‚152 G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚2684 M@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚21541832 JDDeviceStatsMonitor.on_È++€train_batch_start/oversize_allocations.current‚0 YIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 37073453056 R>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚424510238569472 [IDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 4571595503616 YEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚424512780360704 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚2684 YFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚90152375945216 dQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚84446747662336 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚6 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚13048127 ZHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 4563288215288 F=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚3681 A8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚3681 WEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 4569079140352 ^LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 4565802585428 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 IDeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚23682 UADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚424510238569472 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚3 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚29097984 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 D>DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 D;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚3681 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚19684 M=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 38610163200 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚25427968 ZHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 4569079140352 UADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚407696612789320 O@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 2541791232 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚23981 PCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚27386009 UFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 3217031168 H?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚1534 VGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 2516363264 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚19684 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 100620288 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚71522304 A8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚3681 `LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚403133350002000 VFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 37073453056 YEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚419941159429120 BDeviceStatsMonitor.on_train_batch_start/allocation.all.current‚19690 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 L?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚27386009 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚23981 L?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚35705796 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚6 TEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 1515847168 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚25427968 YJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 2514370140 RCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 2541791232 ^LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 4571595503616 TGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚27386015 NFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚330 RBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 40290484224 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 PCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚35705796 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 TDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 40290484224 O?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 40290484224 [KDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 37073453056 \HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚403133324574032 \HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚419941159429120 M@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚63111495 _MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 5705657380864 KBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚2147 TDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 35913877504 MDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚2147 E8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚63091805 PCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚27386015 PCDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated‚35725480 SDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 3142753280 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚25427968 KBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚1534 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚8493708 P@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 38610163200 QDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚21542410 NFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚986 ]JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚90152476565504 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 963120128 cQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 5705728903168 H?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚2147 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚19690 H?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚1309 P@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 36985723401 _step‚152 G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚2684 M@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚21541832 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 YIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 37073453056 R>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚424510238569472 [IDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 4571595503616 YEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚424512780360704 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚2684 YFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚90152375945216 dQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚84446747662336 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚6 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚13048127 ZHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 4563288215288 F=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚3681 A8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚3681 WEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 4569079140352 ^LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 4565802585428 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 IDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 36985723401 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 _runtime‚3421.331764473 ZFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚420155105499648 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 100606976 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 XFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 4565641351022 bODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚84490877987328 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 UCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 4571435236352 ^JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚420155130927616 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚25427968 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚2684 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚19710 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 577553920 UEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 34283657216 C6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚63124037 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚23981 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚71508992 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 B9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚3681 NADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚27399905 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚27399899 PADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 2539798316 P@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 40290484224 XIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 3217031168 G:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚63124037 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚8498074 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚13055102 M>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 2541804544 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚580 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 PADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 2541804544 NADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚35724138 LDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚986 ^KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚84490848889344 WCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚420155105499648 WHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 2514370348 S?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚424726540736000 RBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 35913877504 VGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 3217031168 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚23682 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚577 ]KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 5708617595904 \JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 4568155721370 ?6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚3681 KBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚2147 FDeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚63143753 K>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚21552596 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 [GDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚420155130927616 TEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 2516376576 I@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚2147 UEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 35913877504 XFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 4571435236352 [HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚90199567092224 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 963120128 S?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚407904481585550 ^JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚403338865662496 ZFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚403338840234528 SDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 3217031168 QBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 3142753280 ?6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚3681 KBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚1534 _step‚153 NADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚35743848 N>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 38610163200 OBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚21553176 M=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 40290484224 RBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 40290484224 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚25427968 WCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚424729082540544 WDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚90199466485248 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 BDeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚63143753 K>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚21552596 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 [GDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚420155130927616 TEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 2516376576 I@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚2147 UEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 35913877504 XFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 4571435236352 [HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚90199567092224 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 963120128 S?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚407904481585550 ^JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚403338865662496 ZFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚403338840234528 SDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 3217031168 QBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 3142753280 ?6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚3681 KBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚1534 _step‚153 NADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚35743848 N>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 38610163200 OBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚21553176 M=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 40290484224 RBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 40290484224 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚25427968 WCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚424729082540544 WDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚90199466485248 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 BDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 36985723401 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 _runtime‚3421.331764473 ZFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚420155105499648 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 100606976 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 XFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 4565641351022 bODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚84490877987328 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 UCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 4571435236352 ^JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚420155130927616 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚25427968 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚2684 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚19710 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 577553920 UEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 34283657216 C6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚63124037 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚23981 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚71508992 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 B9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚3681 NADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚27399905 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚27399899 PADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 2539798316 P@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 40290484224 XIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 3217031168 G:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚63124037 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚8498074 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚13055102 M>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 2541804544 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚580 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 PADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 2541804544 NADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚35724138 LDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚986 ^KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚84490848889344 WCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚420155105499648 WHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 2514370348 S?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚424726540736000 RBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 35913877504 VGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 3217031168 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚23682 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚577 ]KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 5708617595904 \JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 4568155721370 ?6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚3681 KBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚2147 FDeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 ^LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 4683486544136 E8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚64706757 VGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 3140738488 F=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚3681 trainer/global_step‚1949 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 RCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 2539798108 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 _runtime‚3506.146008659 IDeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚23682 YEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚435327916515328 BDeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚435325374724096 YEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚430638462955520 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚23981 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚29097984 `MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚86653234813952 YEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚418092619570936 RBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 40290484224 ]IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚430638488383488 TDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 35913877504 M@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚64726447 MDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚2147 \HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚430638462955520 VGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 3142753280 IDeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚2684 TDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 40290484224 [KDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 37073453056 QDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚22086934 L?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚36626248 PCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚28080509 ]JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚92506414235136 UADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚418090079772828 A8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚3681 L?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚28080509 YJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 2516363264 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 M=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 38610163200 ^LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 4689428131840 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚3 SGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚8712005 YFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚92506301031936 A8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚3681 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 _step‚156 cQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 5853150323200 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚13374331 dQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚86653263911936 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 SDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 3142753280 NFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚986 KBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚2147 H?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚2147 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚19690 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚6 TGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚36645932 YIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 37073453056 RCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 2541791232 TGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚28080515 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚595 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 YJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 2514370140 H?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚1534 [IDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 4689428131840 P@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 38610163200 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚25427968 ZHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 4686911768576 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 UFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 3217031168 D;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚3681 UADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚435325374724096 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚841052160a1½ù'§óO R>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚435325374724096 YEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚430638462955520 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚23981 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚29097984 `MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚86653234813952 YEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚418092619570936 RBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 40290484224 ]IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚430638488383488 TDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 35913877504 M@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚64726447 MDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚2147 \HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚430638462955520 VGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 3142753280 IDeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚2684 TDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 40290484224 [KDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 37073453056 QDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚22086934 L?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚36626248 PCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚28080509 ]JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚92506414235136 UADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚418090079772828 A8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚3681 L?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚28080509 YJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 2516363264 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 M=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 38610163200 ^LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 4689428131840 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚3 SGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚8712005 YFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚92506301031936 A8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚3681 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 _step‚156 cQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 5853150323200 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚13374331 dQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚86653263911936 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 SDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 3142753280 NFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚986 KBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚2147 H?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚2147 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚19690 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚6 TGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚36645932 YIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 37073453056 RCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 2541791232 TGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚28080515 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚595 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 YJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 2514370140 H?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚1534 [IDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 4689428131840 P@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 38610163200 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚25427968 ZHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 4686911768576 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 UFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 3217031168 D;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚3681 UADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚435325374724096 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚84105216 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 577553920 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 XIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 3217031168 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚2684 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚6 H?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚1309 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚25427968 ZHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 4680972173996 NFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚330 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 963120128 _MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 5853066217984 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 PCDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated‚36645932 WGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 34283657216 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚8712008 PCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚28080515 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚13374926 O@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 2541791232 WEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 4686911768576 ZKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 3217031168 VBDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated‚435327916515328 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚25427968 `LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚430638488383488 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚23682 D>DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 ^LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 4683486544136 E8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚64706757 VGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 3140738488 F=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚3681 trainer/global_step‚1949 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 RCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 2539798108 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 _runtime‚3506.146008659 IDeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚23682 YEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚435327916515328 BDeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚64758705 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 WCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚418300488367374 PADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 2541804544 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 RBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 40290484224 M=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 40290484224 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚23981 REDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚28094405 S?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚418297948569058 PDeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚22097202 F=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚1534 C6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚64738989 S?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚435541676890624 N>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 38610163200 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 XFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 4683325309730 I@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚1534 ZFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚413614623259328 M>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 2541804544 T@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚435544218695168 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 LDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚330 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚29097984 WGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 37073453056 TDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 37073453056 UCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 4689267864576 [HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚92553477458944 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚3 WHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 2514370348 NADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚28094405 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚25427968 F8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 ?6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚3681 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚23682 OBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚22097797 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚595 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 P@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 40290484224 KBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚2147 _step‚157 NADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚28094399 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 KBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚1534 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 577553920 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 N>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 36985723401 D;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚3681 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚23981 XFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 4689267864576 TEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 3142753280 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚6 F=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚2147 % _timestamp‚1.7884887086155572e+09 _runtime‚3507.717669915 ZFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚430852409026048 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚75703296 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 REDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚36664300 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1309 WCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚435544218695168 YGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 4691784241152 TEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 2516376576 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0X™Ê­'ªýM VGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 3217031168 SDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 3217031168 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚28094399 NADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚36644590 TEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 3140738488 WHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 2516376576 G:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚64738989 E8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 ?6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚3681 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚23682 OBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚22097797 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚595 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 P@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 40290484224 KBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚2147 _step‚157 NADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚28094399 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 KBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚1534 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 577553920 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 N>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 36985723401 D;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚3681 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚23981 XFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 4689267864576 TEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 3142753280 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚6 F=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚2147 % _timestamp‚1.7884887086155572e+09 _runtime‚3507.717669915 ZFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚430852409026048 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚75703296 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 REDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚36664300 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1309 WCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚435544218695168 YGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 4691784241152 TEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 2516376576 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚19710 UEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 35913877504 XIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 3217031168 B9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚3681 J=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚36644590 [GDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚430852434454016 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚25427968 RBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 35913877504 QBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 3142753280 \JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 4691784241152 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚8716374 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 963120128 \JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 4685839680078 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 ^JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚430852434454016 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚592 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 104801280 ^JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚413614648687296 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚25427968 LDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚986 K>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚64758705 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 WCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚418300488367374 PADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 2541804544 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 RBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 40290484224 M=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 40290484224 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚23981 REDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚28094405 S?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚418297948569058 PDeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚22097202 F=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚1534 C6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚64738989 S?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚435541676890624 N>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 38610163200 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 XFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 4683325309730 I@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚1534 ZFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚413614623259328 M>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 2541804544 T@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚435544218695168 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 LDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚330 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚29097984 WGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 37073453056 TDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 37073453056 UCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 4689267864576 [HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚92553477458944 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚3 WHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 2514370348 NADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚28094405 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚25427968 F€¾íO TGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚13701316 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚25427968 \HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚441335766481920 M@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚66341399 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 102717440 TDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 40290484224 ]IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚441335791909888 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 YEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚441335766481920 H?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚1309 [KDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 37073453056 RCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 2539798108 P@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 36985723401 O@DeviceStatsMonitor.on_train_bÜ™æs#atch_start/active_bytes.all.current‚ 2541791232 ^LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 4807260760064 IDeviceStatsMonitor.on_train_batch_start/allocation.all.current‚19690 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 SDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 3142753280 QDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚22632213 YJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 2514370140 VFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 37073453056 A8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚3681 SGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚8930305 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚6 TEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 1515847168 cQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 6000353820672 BDeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚446140510878720 YFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚94860031264768 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚25427968 ZHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 4804744396800 RBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 40290484224 UFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 3217031168 H?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚1534 _runtime‚3590.920891005 dQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚88859780161536 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 L?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚28775009 G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚2684 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚23682 P@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 38610163200 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚19684 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚19690 `MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚88859751063552 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚73619456 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚23682 _MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 6000280201216 WGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 34283657216 _step‚160 L?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚37546700 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚13701905 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚29097984 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 `LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚441335791909888 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚589 ]JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚94860133982208 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 WEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 4804744396800 ZHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 4798656132704 trainer/global_step‚1999 NFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚986 MDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚2147 M@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚22631621 TDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 35913877504 PCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚37546700 UADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚428483546756336 D;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚3681 IDeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 YEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚428486086554444 VGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 3140738488 ZKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 3217031168 YEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚446143052669952 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 963120128 `LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚423684916051600FãÈø'¿òO WEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 4804744396800 ZHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 4798656132704 trainer/global_step‚1999 NFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚986 MDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚2147 M@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚22631621 TDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 35913877504 PCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚37546700 UADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚428483546756336 D;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚3681 IDeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 YEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚428486086554444 VGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 3140738488 ZKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 3217031168 YEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚446143052669952 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 963120128 `LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚423684916051600 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚13701316 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚25427968 \HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚441335766481920 M@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚66341399 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 102717440 TDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 40290484224 ]IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚441335791909888 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 YEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚441335766481920 H?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚1309 [KDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 37073453056 RCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 2539798108 P@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 36985723401 O@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 2541791232 ^LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 4807260760064 IDeviceStatsMonitor.on_train_batch_start/allocation.all.current‚19690 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 SDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 3142753280 QDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚22632213 YJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 2514370140 VFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 37073453056 A8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚3681 SGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚8930305 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚6 TEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 1515847168 cQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 6000353820672 BDeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚446140510878720 YFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚94860031264768 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚25427968 ZHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 4804744396800 RBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 40290484224 UFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 3217031168 H?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚1534 _runtime‚3590.920891005 dQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚88859780161536 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 L?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚28775009 G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚2684 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚23682 P@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 38610163200 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚19684 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚19690 `MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚88859751063552 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚73619456 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚23682 _MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 6000280201216 WGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 34283657216 _step‚160 L?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚37546700 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚13701905 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚29097984 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 `LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚441335791909888 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚589 ]JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚94860133982208 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 ï{^­2À- _wandbruntime‚3592  _runtime‚3592ÙíáÃþ&ÁøM FDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 36985723401 D;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚3681 PADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 2541804544 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 PADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 2539798316 WHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 2514370348 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 UEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 34283657216 J=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚37565042 ^JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚441549737980416 OBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚22643059 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚94577664 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚19710 UEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 35913877504 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 577553920 SDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 3217031168 KBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚2147 EDeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 2541804544 ^JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚423890431712096 NADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚37584752 T@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚446359354849792 [HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚94907188007424 C6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚66353941 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 \JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 4803523638786 _step‚161 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚19716 WCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚446359354849792 N>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 38610163200 LDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚986 TEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 3140738488 RBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 40290484224 G:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚66373657 K;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 38610163200 XFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 4807100492800 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚6 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚597 RCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 1515847168 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 963120128 [GDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚441549737980416 ZFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚441549712552448 TEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 3142753280 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 YIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 37073453056 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚23981 RBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 35913877504 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚2684 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 WCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚428693955350882 XFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 4801009268438 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚25427968 G:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚66353941 K>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚22642462 LDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚330 WDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚94907064331776 F=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚2147 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚28788899 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1309 ZFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚423890406284128 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 KBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚1534 YGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 4809616869376 K>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚66373657 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚23981 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 123675648 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚8934674 BDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 36985723401 D;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚3681 PADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 2541804544 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 PADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 2539798316 WHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 2514370348 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 UEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 34283657216 J=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚37565042 ^JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚441549737980416 OBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚22643059 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚94577664 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚19710 UEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 35913877504 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 577553920 SDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 3217031168 KBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚2147 EDeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 2541804544 ^JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚423890431712096 NADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚37584752 T@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚446359354849792 [HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚94907188007424 C6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚66353941 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 \JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 4803523638786 _step‚161 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚19716 WCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚446359354849792 N>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 38610163200 LDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚986 TEDeviceStatsMonitor.on_train_batch_end/requested_by kÁtes.small_pool.peak‚ 3140738488 RBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 40290484224 G:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚66373657 K;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 38610163200 XFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 4807100492800 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚6 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚597 RCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 1515847168 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 963120128 [GDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚441549737980416 ZFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚441549712552448 TEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 3142753280 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 YIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 37073453056 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚23981 RBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 35913877504 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚2684 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 WCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚428693955350882 XFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 4801009268438 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚25427968 G:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚66353941 K>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚22642462 LDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚330 WDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚94907064331776 F=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚2147 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚28788899 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1309 ZFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚423890406284128 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 KBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚1534 YGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 4809616869376 K>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚66373657 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚23981 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 123675648 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚8934674 B'úOË‚(:Ä þØèÔÄÈ“° disk.sda1.in‚0 disk.md0p1.out‚ 338.453125 disk.sdb.in‚0# disk.nvme1n1.out‚25148.99609375$ memory_percent‚93.08836460113525* proc.memory.percent‚18.656134605407715 network.sent‚ 4.927491e+06 network.recv‚ 1.174185e+06% disk./.usageGB‚2983.1929244995117 disk.sdb2.in‚0 disk.rbd2.in‚0 disk.rbd2.out‚1.125 disk.sda2.in‚ 0.0078125 cpu‚25.513209545164703 disk.md0.out‚ 338.453125 disk.sda.out‚353.1591796875 disk.sdb2.out‚353.1591796875 disk.md0p1.in‚ 0.0078125 disk.sdb.out‚353.1591796875 disk.nvme1n1.in‚4.515625 disk.sda.in‚ 0.0078125 disk.rbd0.out‚ 0.91259765625 disk.sdb1.out‚0 disk.md0.in‚ 0.0078125 disk.sda2.out‚353.1591796875 disk.nvme0n1.in‚ 4.6015625# proc.memory.rssMB‚ 3056.62109375 disk.sda1.out‚0 disk.dm-0.in‚ 9.1171875) disk./.usagePercent‚83.41132273224349 disk.rbd0.in‚0) proc.memory.availableMB‚ 1132.40234375 proc.cpu.threads‚17 disk.dm-0.out‚ 51243.953125 disk.sdb1.in‚0# disk.nvme0n1.out‚26094.95703125ÉÌr_žÌ‚(:“ ÙèÔºò̯ gpu.0.gpu‚98.0 gpu.0.memory‚18+ gpu.0.memoryAllocated‚47.54737854003906* gpu.0.memoryAllocatedBytes‚ 40842887168 gpu.0.temp‚46.0 gpu.0.powerWatts‚239.776' gpu.0.enforcedPowerLimitWatts‚400.0) gpu.0.powerPercent‚59.943999999999996 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0u†pxRÍ‚(:Ç ÙèÔĶ„° disk.md0.in‚ 0.0078125 disk.sda.out‚354.49658203125 proc.cpu.threads‚17 network.recv‚ 1.186358e+06( disk./.usagePercent‚83.4035461128465! disk.sda2.out‚354.49658203125 disk.nvme1n1.in‚4.515625 disk.nvme0n1.in‚ 4.6015625# disk.nvme0n1.out‚26095.02734375 disk.sdb1.in‚0! disk.sdb2.out‚354.49658203125 disk.md0p1.in‚ 0.0078125 disk.rbd2.out‚1.125) proc.memory.availableMB‚ 1132.63671875* proc.memory.percent‚18.656134605407715 network.sent‚ 4.94003e+06 disk.md0.out‚ 339.7421875 disk.sdb.out‚354.49658203125! disk.nvme1n1.out‚ 25286.859375$ disk./.usageGB‚2982.914794921875 disk.sdb2.in‚0 disk.sdb.in‚0 disk.sda2.in‚ 0.0078125 disk.sda.in‚ 0.0078125 disk.rbd0.in‚0 disk.dm-0.out‚51381.88671875 disk.sdb1.out‚0 disk.sda1.in‚0# proc.memory.rssMB‚ 3056.62109375 cpu‚25.512339729356643 disk.sda1.out‚0 disk.md0p1.out‚ 339.7421875 disk.rbd2.in‚0$ memory_percent‚93.08693408966064 disk.rbd0.out‚ 0.91259765625 disk.dm-0.in‚ 9.1171875½ôünœÎ‚(:‘ œÙèÔå‚௠gpu.0.gpu‚24.0 gpu.0.memory‚8+ gpu.0.memoryAllocated‚47.54737854003906* gpu.0.memoryAllocatedBytes‚ 40842887168 gpu.0.temp‚41.0 gpu.0.powerWatts‚117.81' gpu.0.enforcedPowerLimitWatts‚400.0) gpu.0.powerPercent‚29.452499999999997 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0`d¾QÏ‚(:Æ œÙèÔá×½° disk.rbd2.out‚1.125# proc.memory.rssMB‚ 3056.62109375 network.sent‚ 4.947031e+06 disk.sda2.out‚356.1337890625 disk.dm-0.out‚51500.33203125 disk.sdb2.in‚0 network.recv‚ 1.192068e+06) disk./.usagePercent‚83.40788644988012$ disk./.usageGB‚2983.070026397705 disk.sdb.in‚0 disk.sdb.out‚356.1337890625 disk.sda2.in‚ 0.0078125 disk.nvme1n1.in‚4.515625 disk.dm-0.in‚ 9.1171875 disk.sdb1.out‚0 disk.sdb2.out‚356.1337890625 disk.md0.in‚ 0.0078125 disk.md0.out‚ 341.32421875 proc.cpu.threads‚17# disk.nvme0n1.out‚26095.16796875 disk.rbd2.in‚0' proc.memory.availableMB‚ 1132.640625 disk.rbd0.out‚ 0.91259765625 disk.nvme0n1.in‚ 4.6015625 disk.md0p1.out‚ 341.32421875* proc.memory.percent‚18.656134605407715" disk.nvme1n1.out‚ 25405.1640625 disk.sda.out‚356.1337890625 disk.rbd0.in‚0 disk.sda1.out‚0$ memory_percent‚93.08691024780273 disk.md0p1.in‚ 0.0078125 cpu‚25.511604628556658 disk.sda.in‚ 0.0078125 disk.sdb1.in‚0 disk.sda1.in‚0ežóQ2Ð- _wandbruntime‚3673  _runtime‚3673PUÇïÆÑÀ _step‚163 & lr-AdamW/pg1‚0.0009503023112749418 & lr-AdamW/pg2‚0.0009503023112749418 trainer/global_step‚2049 $ _timestamp‚1.788488874657652e+09 _runtime‚3673.759008989‚ºqËÒÅ & lr-AdamW/pg1‚0.0009503023112749418 & lr-AdamW/pg2‚0.0009503023112749418 trainer/global_step‚2049 $ _timestamp‚1.788488874657652e+09 _runtime‚3673.759008989 _step‚163£éj2Ó- _wandbruntime‚3673  _runtime‚3673K{õÏô'ÔîO QDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚23176844 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚585 H?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚1309 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚9148608 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚23981 L?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚38467152 IDeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚456955647033344 YIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 37073453056 VGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 3142753280 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚582 VGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 3140738488 UFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 3217031168 G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚2684 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚23981 SGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚9148605 YFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚97214033463296 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 YEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚438879553537952 PCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚38467152 NFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚986 _MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 6147766150144 \HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚433960673648432 RBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 40290484224 D;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚3681 cQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 6147839769600 KBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚2147 IDeviceStatsMonitor.on_train_batch_start/allocation.all.current‚19690 ]JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚97214136180736 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 102717440 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 577553920 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚19690 PCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚29469509 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚14027654 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 963120128 A8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚3681 YJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 2514370140 VFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 37073453056 L?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚29469509 `LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚452033095436288 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚25427968 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 SDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 3142753280 \HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚452033070008320 YJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 2516363264 `MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚91066267313152 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 % _timestamp‚1.7884888746583595e+09 ^LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 4925093388288 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚2684 F=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚3681 M@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚67956351 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 PCDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated‚38486836 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚25427968 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 MDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚1534 YEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚456958188824576 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚3 BDeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚19684 UADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚456955647033344 M=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 38610163200 TEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 1515847168 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 MDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚2147 trainer/global_step‚2049 _runtime‚3673.762832887 YEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚452033070008320 TGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚29469515 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚6 NFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚330 UADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚438877013739844 `LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚433960699076400 PCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚29469515 [IDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 4925093388288 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 P@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 38610163200 P@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 36985723401 XIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 3217031168 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚6 O?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 40290484224 _step‚164 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚23682 O@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 2541791232 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 KBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚1534 WGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 35913877504 dQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚91066296411136 WGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 34283657216 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 H?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚2147 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 ]IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚4520330954362889<ÙaÕóO H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚23682 O@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 2541791232 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 KBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚1534 WGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 35913877504 dQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚91066296411136 WGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 34283657216 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 H?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚2147 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 ]IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚452033095436288 QDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚23176844 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚585 H?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚1309 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚9148608 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚23981 L?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚38467152 IDeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚456955647033344 YIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 37073453056 VGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 3142753280 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚582 VGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 3140738488 UFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 3217031168 G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚2684 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚23981 SGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚9148605 YFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚97214033463296 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 YEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚438879553537952 PCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚38467152 NFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚986 _MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 6147766150144 \HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚433960673648432 RBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 40290484224 D;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚3681 cQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 6147839769600 KBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚2147 IDeviceStatsMonitor.on_train_batch_start/allocation.all.current‚19690 ]JDeviceStatsMonitor.on_train_batch_Íô!˜start/inactive_split_bytes.all.allocated‚97214136180736 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 102717440 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 577553920 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚19690 PCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚29469509 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚14027654 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 963120128 A8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚3681 YJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 2514370140 VFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 37073453056 L?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚29469509 `LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚452033095436288 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚25427968 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 SDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 3142753280 \HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚452033070008320 YJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 2516363264 `MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚91066267313152 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 % _timestamp‚1.7884888746583595e+09 ^LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 4925093388288 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚2684 F=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚3681 M@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚67956351 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 PCDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated‚38486836 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚25427968 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 MDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚1534 YEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚456958188824576 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚3 BDeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚19684 UADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚456955647033344 M=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 38610163200 TEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 1515847168 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 MDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚2147 trainer/global_step‚2049 _runtime‚3673.762832887 YEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚452033070008320 TGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚29469515 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚6 NFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚330 UADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚438877013739844 `LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚433960699076400 PCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚29469515 [IDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 4925093388288 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 P@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 38610163200 P@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 36985723401 XIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 3217031168 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚6 O?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 40290484224 _step‚164¤äÌþ"Ö‚(:’ «ÙèÔ—Ꮀ gpu.0.gpu‚98.0 gpu.0.memory‚17+ gpu.0.memoryAllocated‚47.54737854003906* gpu.0.memoryAllocatedBytes‚ 40842887168 gpu.0.temp‚45.0 gpu.0.powerWatts‚241.048' gpu.0.enforcedPowerLimitWatts‚400.0( gpu.0.powerPercent‚60.26200000000001 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0è«ÃFׂ(:» «ÙèÔ׆ǰ network.recv‚ 1.195459e+06( disk./.usagePercent‚83.4087370676238 disk.sdb2.in‚0 disk.rbd2.in‚0! disk.nvme1n1.out‚ 25552.015625 disk.md0p1.in‚ 0.0078125% disk./.usageGB‚2983.1004486083984 disk.sdb.in‚0 disk.nvme1n1.in‚4.515625 disk.dm-0.out‚51647.25 disk.md0.in‚ 0.0078125 disk.rbd0.in‚0 disk.sdb1.in‚0$ proc.memory.availableMB‚1132.375! disk.sda2.out‚357.58056640625 disk.sda1.in‚0 disk.md0p1.out‚ 342.71484375# proc.memory.rssMB‚ 3056.62109375* proc.memory.percent‚18.656134605407715 cpu‚25.510813090948 proc.cpu.threads‚17 disk.rbd2.out‚1.125 disk.sdb.out‚357.58056640625 disk.sda.in‚ 0.0078125 disk.rbd0.out‚ 0.91259765625! disk.nvme0n1.out‚ 26095.234375 disk.dm-0.in‚ 9.1171875 disk.sdb1.out‚0 disk.nvme0n1.in‚ 4.6015625 disk.sda1.out‚0 disk.md0.out‚ 342.71484375$ memory_percent‚93.08853149414062 network.sent‚ 4.949995e+06 disk.sda2.in‚ 0.0078125 disk.sda.out‚357.58056640625! disk.sdb2.out‚357.58056640625›ËT2Ø- _wandbruntime‚3675  _runtime‚3675yV,þ&ÙøM GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚29483399 YGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 4927449497600 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚2684 REDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚14034119 TEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 3140738488 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 EDeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚67988609 NADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚29483399 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 ZFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚434166189308928 M>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 2541804544 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 577553920 TDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 37073453056 _step‚165 ZFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚452247016078848 G:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚67968893 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚568 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚23981 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 J=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚38485494 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚3 aODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 6150804740096 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 ^JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚434166214736896 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 F=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚1534 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚25427968 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚9152974 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 100606976 RCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 1515847168 BDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 K;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 38610163200 S?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚457171949199872 TEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 3142753280 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚6 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 I@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚2147 REDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚29483405 ?6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚3681 QBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 3142753280 WCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚457174491004416 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚25427968 \JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 4921207597494 I@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚1534 C6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚67968893 XFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 4918693227146 WGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 37073453056 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚19710 YIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 37073453056 % _timestamp‚1.7884888761664863e+09 K>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚23187090 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚14034684 [HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚97261231476224 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚71508992 N>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 36985723401 ?6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚3681 _runtime‚3675.268290261 G:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚67988609 NADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚38485494 LDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚330 WHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 2514370348 FDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 38610163200 PADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 2539798316 OBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚23187658 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚19716 NADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚38505204 UEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 35913877504 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚19710 LDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚986 D;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚3681 B9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚3681 NADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚29483405 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 WCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚452247016078848 UCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 4924933121024 ]KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 6150733231104 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 M=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 40290484224 FDeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 ^JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚452247041506816 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚09@/|'ÚýM WHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 2516376576 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 XFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 4924933121024 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚29097984 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 KBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚2147 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 ^JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚452247041506816 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚29483399 YGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 4927449497600 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚2684 REDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚14034119 TEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 3140738488 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 EDeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚67988609 NADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚29483399 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 ZFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚434166189308928 M>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 2541804544 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 577553920 TDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 37073453056 _step‚165 ZFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚452247016078848 G:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚67968893 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚568 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚23981 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 J=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚38485494 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚3 aODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 6150804740096 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 ^JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚434166214736896 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 F=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚1534 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚25427968 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚9152974 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 100606976 RCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 1515847168 BDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 K;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 38610163200 S?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚457171949199872 TEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 3142753280 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚6 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 I@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚2147 REDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚29483405 ?6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚3681 QBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 3142753280 WCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚457174491004416 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚25427968 \JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 4921207597494 I@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚1534 C6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚67968893 XFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 4918693227146 WGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 37073453056 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚19710 YIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 37073453056 % _timestamp‚1.7884888761664863e+09 K>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚23187090 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚14034684 [HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚97261231476224 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚71508992 N>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 36985723401 ?6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚3681 _runtime‚3675.268290261 G:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚67988609 NADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚38485494 LDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚330 WHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 2514370348 FDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 38610163200 PADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 2539798316 OBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚23187658 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚19716 NADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚38505204 UEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 35913877504 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚19710 LDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚986 D;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚3681 B9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚3681 NADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚29483405 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 WCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚452247016078848 UCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 4924933121024 ]KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 6150733231104 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 M=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 40290484224 FþQÕUÜÏ -loss/FBt4/assignment_loss‚0.1529646885644 0loss/SRqqt3/detection_loss‚2.344990541397686 .loss/FBt3/detection_loss‚3.038081407546997 0loss/FRt1/assignment_loss‚3.8991198640120657 1loss/FBt2/assignment_loss‚0.16684052519827483 .loss/FBt1/detection_loss‚3.154961171746254 _step‚166 0loss/FRt2/assignment_loss‚1.4179991287343643 .loss/FRt1/detection_loss‚2.266364500867693 /loss/FRt4/detection_loss‚2.5909322787076237 1loss/SRqqt2/detection_loss‚1.1989143370435789 /loss/FBt2/detection_loss‚0.6707049481190698 1loss/SRqqt1/assignment_loss‚1.782834814240535 0loss/FBt3/assignment_loss‚0.3988636963717315 1loss/SRqqt1/detection_loss‚2.4550112957755723 0loss/SRqqt4/detection_loss‚1.601612360527118 %loss/total_loss‚45.93544549386327 % _timestamp‚1.7884888761682427e+09 0loss/FRt4/assignment_loss‚2.5725634008646012 1loss/SRqqt2/assignment_loss‚1.245422277870214 0loss/FBt1/assignment_loss‚0.2638364169499255 trainer/global_step‚2049 /loss/FRt2/detection_loss‚1.7865419526310529 epoch‚3 2loss/SRqqt3/assignment_loss‚1.9665626193016348 .loss/FRt3/detection_loss‚2.809213919426078 /loss/FBt4/detection_loss‚3.0841989792310276 _runtime‚3675.269361036 0loss/FRt3/assignment_loss‚3.5438092760185698 2loss/SRqqt4/assignment_loss‚1.5231010927171964ƒã³öZÝÔ 2loss/SRqqt3/assignment_loss‚1.9665626193016348 .loss/FRt3/detection_loss‚2.809213919426078 /loss/FBt4/detection_loss‚3.0841989792310276 _runtime‚3675.269361036 0loss/FRt3/assignment_loss‚3.5438092760185698 2loss/SRqqt4/assignment_loss‚1.5231010927171964 -loss/FBt4/assignment_loss‚0.1529646885644 0loss/SRqqt3/detection_loss‚2.344990541397686 .loss/FBt3/detection_loss‚3.038081407546997 0loss/FRt1/assignment_loss‚3.8991198640120657 1loss/FBt2/assignment_loss‚0.16684052519827483 .loss/FBt1/detection_loss‚3.154961171746254 _step‚166 0loss/FRt2/assignment_loss‚1.4179991287343643 .loss/FRt1/detection_loss‚2.266364500867693 /loss/FRt4/detection_loss‚2.5909322787076237 1loss/SRqqt2/detection_loss‚1.1989143370435789 /loss/FBt2/detection_loss‚0.6707049481190698 1loss/SRqqt1/assignment_loss‚1.782834814240535 0loss/FBt3/assignment_loss‚0.3988636963717315 1loss/SRqqt1/detection_loss‚2.4550112957755723 0loss/SRqqt4/detection_loss‚1.601612360527118 %loss/total_loss‚45.93544549386327 % _timestamp‚1.7884888761682427e+09 0loss/FRt4/assignment_loss‚2.5725634008646012 1loss/SRqqt2/assignment_loss‚1.245422277870214 0loss/FBt1/assignment_loss‚0.2638364169499255 trainer/global_step‚2049 /loss/FRt2/detection_loss‚1.7865419526310529 epoch‚3¦…ÂöZžÞ‚(:“ ºÙèÔ¼ž° gpu.0.gpu‚91.0 gpu.0.memory‚16+ gpu.0.memoryAllocated‚47.54737854003906* gpu.0.memoryAllocatedBytes‚ 40842887168 gpu.0.temp‚45.0 gpu.0.powerWatts‚231.761' gpu.0.enforcedPowerLimitWatts‚400.0) gpu.0.powerPercent‚57.940250000000006 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0—ÎM Qß‚(:Æ ºÙèÔ¹•ð± disk.sdb2.in‚0 disk.md0.out‚ 344.04296875 disk.md0p1.out‚ 344.04296875 disk.rbd0.out‚ 0.91259765625* proc.memory.percent‚18.656134605407715) disk./.usagePercent‚83.41228737009601 disk.sda1.out‚0 disk.md0p1.in‚ 0.0078125# proc.memory.rssMB‚ 3056.62109375 proc.cpu.threads‚17 disk.sda.in‚ 0.0078125 disk.dm-0.in‚ 9.1171875) proc.memory.availableMB‚ 1130.11328125 disk.sda1.in‚0 disk.nvme1n1.in‚4.515625 network.sent‚ 5.044414e+06 disk.sdb1.in‚0 disk.rbd2.out‚1.125 disk.sdb.out‚358.94775390625! disk.sda2.out‚358.94775390625 disk.rbd0.in‚0 disk.nvme0n1.in‚ 4.6015625# disk.nvme0n1.out‚26095.38671875 disk.rbd2.in‚0 disk.nvme1n1.out‚25605.25 cpu‚25.510010133990754! disk.sdb2.out‚358.94775390625 disk.md0.in‚ 0.0078125 disk.sdb.in‚0 disk.sda2.in‚ 0.0078125 disk.sda.out‚358.94775390625 disk.dm-0.out‚51700.63671875# memory_percent‚93.1023359298706 network.recv‚ 1.215231e+06$ disk./.usageGB‚2983.227424621582 disk.sdb1.out‚0Gº »”à‚(:‰ ÉÙèÔ—×Á¯ gpu.0.gpu‚51.0 gpu.0.memory‚12+ gpu.0.memoryAllocated‚47.54737854003906* gpu.0.memoryAllocatedBytes‚ 40842887168 gpu.0.temp‚48.0 gpu.0.powerWatts‚325.629' gpu.0.enforcedPowerLimitWatts‚400.0 gpu.0.powerPercent‚81.40725 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0d+ðTUá‚(:Ê ÉÙèÔ᜗° disk.sdb.in‚0* proc.memory.percent‚18.656134605407715 disk.rbd0.in‚0 disk.md0p1.out‚ 345.3984375 disk.sda.in‚ 0.01171875% proc.memory.availableMB‚ 1131.9375 network.sent‚ 5.116709e+06 disk.dm-0.in‚ 9.1171875 network.recv‚ 1.236374e+06 disk.dm-0.out‚51899.75390625 disk.sdb2.in‚0( disk./.usagePercent‚83.4192764081238 disk.sda1.in‚0 disk.md0.out‚ 345.3984375 disk.md0p1.in‚ 0.01171875# disk.nvme0n1.out‚26095.54296875 disk.rbd2.out‚1.125 disk.sdb.out‚360.37158203125 disk.nvme1n1.in‚4.515625$ memory_percent‚93.09120178222656 proc.cpu.threads‚17 disk.sdb1.in‚0! disk.sdb2.out‚360.37158203125 disk.sda2.in‚ 0.01171875! disk.sda2.out‚360.37158203125" disk.nvme1n1.out‚ 25804.2109375# proc.memory.rssMB‚ 3056.62109375% disk./.usageGB‚2983.4773864746094 disk.rbd0.out‚ 0.91259765625 disk.nvme0n1.in‚ 4.6015625 disk.sda1.out‚0 disk.md0.in‚ 0.01171875 disk.sda.out‚360.37158203125 cpu‚25.509256768391104 disk.sdb1.out‚0 disk.rbd2.in‚0ñíX)7â‚(:‡ ØÙèÔèçù¯ gpu.0.gpu‚97.0 gpu.0.p8ã’[memory‚17+ gpu.0.memoryAllocated‚47.54737854003906* gpu.0.memoryAllocatedBytes‚ 40842887168 gpu.0.temp‚42.0 gpu.0.powerWatts‚131.79' gpu.0.enforcedPowerLimitWatts‚400.0 gpu.0.powerPercent‚32.9475 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0 µIâZã‚(:Ï ØÙèÔœÕé° network.sent‚ 5.127516e+06) disk./.usagePercent‚83.42276346090046 disk.sdb1.in‚0 disk.sdb2.in‚0 disk.md0.out‚ 347.05078125 disk.dm-0.out‚52092.43359375 disk.rbd0.in‚0 disk.nvme0n1.in‚ 4.6015625 disk.dm-0.in‚ 9.1171875 disk.md0.in‚ 0.01171875 disk.sdb.out‚362.07958984375 disk.sda2.in‚ 0.01171875* proc.memory.percent‚18.656134605407715 cpu‚25.508342747047426 disk.sda1.in‚0 disk.md0p1.in‚ 0.01171875 disk.md0p1.out‚ 347.05078125! disk.sda2.out‚362.07958984375 disk.sda.out‚362.07958984375) proc.memory.availableMB‚ 1131.03515625 disk.rbd2.out‚1.125 disk.sdb.in‚0 disk.rbd0.out‚ 0.91259765625$ memory_percent‚93.09670925140381 network.recv‚ 1.254606e+06% disk./.usageGB‚2983.6021003723145! disk.sdb2.out‚362.07958984375 disk.sda.in‚ 0.01171875# disk.nvme0n1.out‚26096.21484375# proc.memory.rssMB‚ 3056.62109375 proc.cpu.threads‚17 disk.sda1.out‚0 disk.nvme1n1.in‚4.515625 disk.sdb1.out‚0 disk.rbd2.in‚0 disk.nvme1n1.out‚ 25996.21875mÍg†’ä‚(:‡ çÙèÔ¯Ž° gpu.0.gpu‚91.0 gpu.0.memory‚16+ gpu.0.memoryAllocated‚47.54737854003906* gpu.0.memoryAllocatedBytes‚ 40842887168 gpu.0.temp‚45.0 gpu.0.powerWatts‚240.628' gpu.0.enforcedPowerLimitWatts‚400.0 gpu.0.powerPercent‚60.157 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0–ÙɦEå‚(:º çÙèÔƘþ°% disk./.usageGB‚2983.8521003723145 disk.sda.in‚ 0.01171875 disk.sda.out‚ 363.45703125# disk.nvme0n1.out‚26096.26953125 disk.dm-0.out‚52398.58203125 disk.md0p1.out‚ 348.3515625 disk.rbd2.in‚0 disk.rbd2.out‚1.125) disk./.usagePercent‚83.42975356553359 disk.rbd0.out‚ 0.91259765625 disk.md0.out‚ 348.3515625* proc.memory.percent‚18.656134605407715 disk.sdb1.in‚0 disk.sda2.out‚ 363.45703125 cpu‚25.507575721603605 disk.nvme1n1.out‚ 26302.3125 disk.sdb2.out‚ 363.45703125 disk.md0.in‚ 0.01171875 disk.sdb.in‚0# proc.memory.availableMB‚1131.75 network.sent‚ 5.135067e+06 disk.md0p1.in‚ 0.01171875 disk.sdb.out‚ 363.45703125# proc.memory.rssMB‚ 3056.62109375 proc.cpu.threads‚17 network.recv‚ 1.258131e+06 disk.sdb2.in‚0 disk.sda1.out‚0 disk.nvme0n1.in‚ 4.6015625 disk.sda1.in‚0 disk.sda2.in‚ 0.01171875 disk.nvme1n1.in‚4.515625 disk.rbd0.in‚0 disk.dm-0.in‚ 9.1171875 disk.sdb1.out‚0$ memory_percent‚93.09234619140625×+¸“æ‚(:ˆ öÙèÔŒ¸á¯ gpu.0.gpu‚26.0 gpu.0.memory‚6+ gpu.0.memoryAllocated‚47.54737854003906* gpu.0.memoryAllocatedBytes‚ 40842887168 gpu.0.temp‚47.0 gpu.0.powerWatts‚317.899' gpu.0.enforcedPowerLimitWatts‚400.0 gpu.0.powerPercent‚79.47475 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0ªòìªWç‚(:Ì öÙèÔ«™Ë° disk.sdb.out‚364.70068359375* proc.memory.percent‚18.656134605407715 cpu‚25.506822661606744) disk./.usagePercent‚83.42200691774613 disk.nvme0n1.in‚ 4.6015625 disk.dm-0.in‚ 9.1171875 disk.sda1.out‚0 disk.sda.in‚ 0.01171875% disk./.usageGB‚2983.5750427246094! disk.sdb2.out‚364.70068359375 disk.md0p1.in‚ 0.01171875 disk.rbd0.out‚ 0.97509765625$ memory_percent‚93.09229850769043 network.sent‚ 5.138842e+06 disk.sdb2.in‚0 disk.md0.out‚ 349.52734375 disk.rbd2.in‚0 proc.cpu.threads‚17 disk.dm-0.out‚ 52521.1875# disk.nvme1n1.out‚26424.80078125( proc.memory.availableMB‚ 1131.7578125# proc.memory.rssMB‚ 3056.62109375 network.recv‚ 1.26263e+06 disk.rbd2.out‚1.125 disk.sdb.in‚0 disk.sda2.in‚ 0.01171875 disk.rbd0.in‚0# disk.nvme0n1.out‚26096.38671875 disk.sdb1.in‚0 disk.md0.in‚ 0.01171875 disk.md0p1.out‚ 349.52734375! disk.sda2.out‚364.70068359375 disk.sda.out‚364.70068359375 disk.sdb1.out‚0 disk.sda1.in‚0 disk.nvme1n1.in‚4.515625å1.$2è- _wandbruntime‚3760  _runtime‚3760õP§,ÇéÁ _runtime‚3760.063424325 _step‚167 & lr-AdamW/pg1‚0.0009379623703628412 & lr-AdamW/pg2‚0.0009379623703628412 trainer/global_step‚2099 % _timestamp‚1.7884889609618902e+09ŽôuñÌêÆ _runtime‚3760.063424325 _step‚167 & lr-AdamW/pg1‚0.0009379623703628412 & lr-AdamW/pg2‚0.0009379623703628412 trainer/global_step‚2099 % _timestamp‚1.7884889609618902e+09§‘F·n2ë- _wandbruntime‚3760  _runtime‚3760‹’Sô'ìîO `LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚444236482101200 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 L?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚30164009 TGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚30164015 YFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚99567734781440 [IDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 5042926016512 WEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 5040409653248 VGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 3140738488 _runtime‚3760.065552355 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚582 `MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚93272783562752 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚23981 R>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚467770783187968 VGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 3142753280 H?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚1309 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 YJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 2514370140 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚19690 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚6 BDeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚2684 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 VGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 2516363264 `LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚462730398962688 NFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚986 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚25427968 SGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚9366905 D>DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 A8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚3681 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 trainer/global_step‚2099 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚2684 TEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 1515847168 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 963120128 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 O?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 40290484224 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚19684 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚23682 NFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚330 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚71522304 UADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚449270480723352 ^LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 5042926016512 M=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 38610163200 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚19684 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚23682 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 cQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 6295022740992 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 YEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚462730373534720 IDeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚467770783187968 VGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 3142753280 H?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚1309 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 YJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 2514370140 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚19690 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚6 BDeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚2684 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 VGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 2516363264 `LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚462730398962688 NFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚986 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚25427968 SGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚9366905 D>DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 A8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚3681 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 trainer/global_step‚2099 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚2684 TEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 1515847168 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 963120128 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 O?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 40290484224 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚19684 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚23682 NFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚330 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚71522304 UADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚449270480723352 ^LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 5042926016512 M=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 38610163200 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚19684 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚23682 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0¨ÝÔ|£2î- _wandbruntime‚3761  _runtime‚3761 ÓbïøM UEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 35913877504 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 QBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 3142753280 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 ?6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚3681 bODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚93316942985728 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1309 WCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚462944319605248 YGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 5045282125824 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚2684 YIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 37073453056 VGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 3217031168 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 NADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚39425656 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 NADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚30177899 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚30177899 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚3 aODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 6297977557504 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 577553920 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 ^JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚444441997761696 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚25427968 D;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚3681 NADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚39405946 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚578 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 _step‚169 WCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚467989627159040 ^JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚462944345033216 OBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚23733843 LDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚330 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 963120128 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 _runtime‚3761.627927843 EDeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 LDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚986 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 FDeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚69603561 G:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚69583845 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚23981 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚19716 C6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚69583845 PADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 2541804544 KBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚1534 I@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚1534 F=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚1534 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚19710 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚581 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚9371274 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 % _timestamp‚1.7884889625260828e+09 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚6 M>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 2541804544 N>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 38610163200 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚19710 K>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚23733262 TDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 37073453056 T@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚467989627159040 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 RBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 35913877504 TEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 2516376576 S?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚467987085354496 UEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 34283657216 F=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚2147 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚9371271 REDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚14361991 ^KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚93316913887744 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 [GDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚462944345033216 WHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ S1Yrì 2516376576 FDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 36985723401 I@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚2147 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚25427968 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 B9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚3681 WHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 2514370348 XFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 5036377185854 XIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 3217031168 REDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚30177905 WGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 37073453056 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚23981 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚71508992 M=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 40290484224 \JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 5045282125824 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚14362569 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 WCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚449480889317898›üÊ'ðýM REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚25427968 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 B9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚3681 WHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 2514370348 XFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 5036377185854 XIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 3217031168 REDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚30177905 WGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 37073453056 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚23981 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚71508992 M=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 40290484224 \JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 5045282125824 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚14362569 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 WCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚449480889317898 UEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 35913877504 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 QBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 3142753280 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 ?6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚3681 bODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚93316942985728 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1309 WCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚462944319605248 YGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 5045282125824 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚2684 YIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 37073453056 VGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 3217031168 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 NADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚39425656 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 NADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚30177899 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚30177899 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚3 aODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 6297977557504 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 577553920 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 ^JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚444441997761696 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚25427968 D;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚3681 NADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚39405946 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚578 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 _step‚169 WCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚467989627159040 ^JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚462944345033216 OBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚23733843 LDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚330 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 963120128 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 _runtime‚3761.627927843 EDeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 LDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚986 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 FDeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚69603561 G:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚69583845 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚23981 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚19716 C6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚69583845 PADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 2541804544 KBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚1534 I@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚1534 F=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚1534 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚19710 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚581 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚9371274 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 % _timestamp‚1.7884889625260828e+09 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚6 M>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 2541804544 N>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 38610163200 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚19710 K>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚23733262 TDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 37073453056 T@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚467989627159040 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 RBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 35913877504 TEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 2516376576 S?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚467987085354496 UEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 34283657216 F=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚2147 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚9371271 REDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚14361991 ^KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚93316913887744 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 [GDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚462944345033216 WHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 2516376576 FDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 36985723401 I@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚2147©E¤N€2ñ- _wandbruntime‚3761  _runtime‚3761‚$\\òÖ 2loss/SRqqt3/assignment_loss‚1.9504342904752667 /loss/FBt1/detection_loss‚2.6609223296767786 &loss/total_loss‚42.489756742808666 _runtime‚3761.629531631 0loss/FRt4/assignment_loss‚2.3827696968150396 /loss/FRt2/detection_loss‚1.5243523356879187 /loss/FRt4/detection_loss‚2.3436762434180065 1loss/SRqqt3/detection_loss‚2.4911749384859028 /loss/FBt2/detection_loss‚0.7345436883670297 /loss/FBt4/detection_loss‚2.2258694042330203 epoch‚3 % _timestamp‚1.7884889625283563e+09 /loss/SRqqt4/detection_loss‚1.60969920593657 trainer/global_step‚2099 _step‚170 1loss/SRqqt1/assignment_loss‚1.788511129777606 0loss/FBt3/assignment_loss‚0.1634931125355741 0loss/FRt3/assignment_loss‚3.7811921149492265 2loss/SRqqt4/assignment_loss‚1.5732862666053207 .loss/FRt1/detection_loss‚2.291979313051949 /loss/FBt3/detection_loss‚3.1544046218578634 0loss/FRt2/assignment_loss‚1.0533304509428358 2loss/SRqqt2/assignment_loss‚1.0717608561214285 0loss/FBt1/assignment_loss‚0.3607049808978397 1loss/SRqqt2/detection_loss‚0.8031149325804823 1loss/FBt2/assignment_loss‚0.11419390424628123 0loss/FRt1/assignment_loss‚3.3058206917097173 0loss/FBt4/assignment_loss‚0.2743365077158951 /loss/FRt3/detection_loss‚2.5358191460371016 0loss/SRqqt1/detection_loss‚2.294366580684011©¶ŠºaóÛ 0loss/FRt1/assignment_loss‚3.3058206917097173 0loss/FBt4/assignment_loss‚0.2743365077158951 /loss/FRt3/detection_loss‚2.5358191460371016 0loss/SRqqt1/detection_loss‚2.294366580684011 2loss/SRqqt3/assignment_loss‚1.9504342904752667 /loss/FBt1/detection_loss‚2.6609223296767786 &loss/total_loss‚42.489756742808666 _runtime‚3761.629531631 0loss/FRt4/assignment_loss‚2.3827696968150396 /loss/FRt2/detection_loss‚1.5243523356879187 /loss/FRt4/detection_loss‚2.3436762434180065 1loss/SRqqt3/detection_loss‚2.4911749384859028 /loss/FBt2/detection_loss‚0.7345436883670297 /loss/FBt4/detection_loss‚2.2258694042330203 epoch‚3 % _timestamp‚1.7884889625283563e+09 /loss/SRqqt4/detection_loss‚1.60969920593657 trainer/global_step‚2099 _step‚170 1loss/SRqqt1/assignment_loss‚1.788511129777606 0loss/FBt3/assignment_loss‚0.1634931125355741 0loss/FRt3/assignment_loss‚3.7811921149492265 2loss/SRqqt4/assignment_loss‚1.5732862666053207 .loss/FRt1/detection_loss‚2.291979313051949 /loss/FBt3/detection_loss‚3.1544046218578634 0loss/FRt2/assignment_loss‚1.0533304509428358 2loss/SRqqt2/assignment_loss‚1.0717608561214285 0loss/FBt1/assignment_loss‚0.3607049808978397 1loss/SRqqt2/detection_loss‚0.8031149325804823 1loss/FBt2/assignment_loss‚0.11419390424628123ªiÅ}æ’ô‚(:‡ …ÚèÔÝÐÁ° gpu.0.gpu‚75.0 gpu.0.memory‚17+ gpu.0.memoryAllocated‚47.54737854003906* gpu.0.memoryAllocatedBytes‚ 40842887168 gpu.0.temp‚42.0 gpu.0.powerWatts‚175.844' gpu.0.enforcedPowerLimitWatts‚400.0 gpu.0.powerPercent‚43.961 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0˜¯Á-[õ‚(:Ð …ÚèԘѱ) disk./.usagePercent‚83.43487220447011 disk.sda.out‚366.54833984375 disk.rbd0.in‚0 disk.nvme0n1.in‚ 62.8828125 disk.sda2.in‚ 0.01171875 disk.sda.in‚ 0.01171875 disk.dm-0.out‚ 52682.9453125 disk.md0.in‚ 0.01171875 disk.rbd2.out‚1.125$ memory_percent‚93.09554100036621! disk.nvme0n1.out‚ 26096.984375 disk.sdb1.in‚0 disk.sda1.in‚0 disk.sda1.out‚0 disk.md0p1.in‚ 0.01171875 disk.sdb.in‚0 disk.sdb.out‚366.54833984375" disk.nvme1n1.out‚ 26585.9609375 network.recv‚ 1.266676e+06$ disk./.usageGB‚2984.035167694092 disk.sdb1.out‚0 disk.sdb2.in‚0( proc.memory.availableMB‚ 1131.2265625* proc.memory.percent‚18.656134605407715 disk.rbd0.out‚ 0.97509765625! disk.sdb2.out‚366.54833984375 disk.md0.out‚ 351.29296875 proc.cpu.threads‚17 disk.rbd2.in‚0! disk.sda2.out‚366.54833984375 disk.dm-0.in‚ 113.9296875 disk.nvme1n1.in‚ 51.046875# proc.memory.rssMB‚ 3056.62109375 cpu‚25.505866075901963 disk.md0p1.out‚ 351.29296875 network.sent‚ 5.157551e+06œ3…sžö‚(:“ ”ÚèÔìäÙ¯ gpu.0.gpu‚96.0 gpu.0.memory‚16+ gpu.0.memoryAllocated‚47.54737854003906* gpu.0.memoryAllocatedBytes‚ 40842887168 gpu.0.temp‚45.0 gpu.0.powerWatts‚230.204' gpu.0.enforcedPowerLimitWatts‚400.0) gpu.0.powerPercent‚57.550999999999995 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0¤úkÓY÷‚(:Î ”ÚèÔ̇ô¯ disk.sdb.out‚368.27392578125 disk.md0.out‚ 352.95703125# disk.nvme0n1.out‚26166.01171875 disk.dm-0.in‚ 113.9296875$ memory_percent‚93.09406280517578 disk.nvme1n1.in‚ 51.046875 disk.sda.out‚368.27392578125 disk.rbd0.in‚0 network.sent‚ 5.237046e+06$ disk./.usageGB‚2984.102226257324 disk.rbd2.out‚1.125 disk.sda2.in‚ 0.01171875 disk.rbd0.out‚ 0.9990234375 disk.dm-0.out‚ 52874.7109375 disk.sdb1.in‚0 disk.sdb1.out‚0# disk.nvme1n1.out‚26708.69921875( disk./.usagePercent‚83.4367471899643 disk.md0p1.in‚ 0.01171875 disk.rbd2.in‚0! disk.sda2.out‚368.27392578125 disk.sda1.in‚0& proc.memory.availableMB‚ 1131.46875* proc.memory.percent‚18.656134605407715 network.recv‚ 1.281036e+06 disk.nvme0n1.in‚ 62.8828125! disk.sdb2.out‚368.27392578125 disk.sda1.out‚0# proc.memory.rssMB‚ 3056.62109375 cpu‚25.50513573998685 disk.sda.in‚ 0.01171875 proc.cpu.threads‚17 disk.sdb2.in‚0 disk.md0.in‚ 0.01171875 disk.md0p1.out‚ 352.95703125 disk.sdb.in‚0UHA„’ø‚(:‡ £ÚèÔÈ—è¯ gpu.0.gpu‚0.0 gpu.0.memory‚0+ gpu.0.memoryAllocated‚47.54737854003906* gpu.0.memoryAllocatedBytes‚ 40842887168 gpu.0.temp‚40.0 gpu.0.powerWatts‚116.583' gpu.0.enforcedPowerLimitWatts‚400.0 gpu.0.powerPercent‚29.14575 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0ð«\ù‚(:Ñ £ÚèÔÔŠƒ²) proc.memory.availableMB‚ 1131.20703125 disk.sdb1.out‚0) disk./.usagePercent‚83.44024800194958 disk.md0p1.out‚ 354.43359375 disk.sdb.in‚0 disk.dm-0.out‚ 53043.171875% disk./.usageGB‚2984.2274322509766 disk.sda2.in‚ 0.01171875 disk.rbd0.out‚ 0.9990234375 disk.sdb1.in‚0 disk.md0.out‚ 354.43359375 network.recv‚ 1.284585e+06 disk.rbd0.in‚0 disk.md0.in‚ 0.01171875! disk.sda2.out‚369.81201171875 disk.sda.out‚369.81201171875 disk.dm-0.in‚ 113.9296875 disk.md0p1.in‚ 0.01171875 disk.nvme1n1.in‚ 51.046875 disk.nvme0n1.in‚ 62.8828125" disk.nvme0n1.out‚ 26211.9921875# proc.memory.rssMB‚ 3056.62109375 cpu‚25.504300364758837 network.sent‚ 5.242362e+06 disk.rbd2.in‚0 disk.rbd2.out‚1.125 disk.sdb2.in‚0 disk.sda1.in‚0 disk.sda1.out‚0$ memory_percent‚93.09566020965576 proc.cpu.threads‚17" disk.nvme1n1.out‚ 26831.1796875* proc.memory.percent‚18.656134605407715 disk.sdb.out‚369.81201171875 disk.sda.in‚ 0.01171875! disk.sdb2.out‚369.81201171875Õ+±+žú‚(:“ ²ÚèÔÍ…å¯ gpu.0.gpu‚98.0 gpu.0.memory‚17+ gpu.0.memoryAllocated‚47.54737854003906* gpu.0.memoryAllocatedBytes‚ 40842887168 gpu.0.temp‚45.0 gpu.0.powerWatts‚215.013' gpu.0.enforcedPowerLimitWatts‚400.0) gpu.0.powerPercent‚53.753249999999994 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0ù"”\Gû‚(:¼ ²ÚèÔùÖþ¯ network.sent‚ 5.246199e+06 disk.sdb.out‚ 371.3671875 cpu‚25.503529654556377 disk.sda1.in‚0 disk.sda1.out‚0 disk.md0p1.in‚ 0.01171875 disk.sda.out‚ 371.3671875) proc.memory.availableMB‚ 1131.12890625 disk.rbd2.out‚1.125 disk.dm-0.in‚ 113.9296875 disk.dm-0.out‚53285.48828125 disk.sdb1.in‚0 disk.nvme0n1.out‚26350$ memory_percent‚93.09613704681396 network.recv‚ 1.289296e+06 disk.md0.in‚ 0.01171875 disk.md0.out‚ 355.91796875 disk.rbd0.in‚0 disk.rbd0.out‚ 1.06201171875# proc.memory.rssMB‚ 3056.62109375 proc.cpu.threads‚17$ disk./.usageGB‚2984.352039337158 disk.sda2.out‚ 371.3671875 disk.nvme1n1.in‚ 51.046875* proc.memory.percent‚18.656134605407715) disk./.usagePercent‚83.44373206823134 disk.sdb1.out‚0 disk.sdb2.in‚0 disk.sdb2.out‚ 371.3671875 disk.rbd2.in‚0 disk.sdb.in‚0 disk.sda2.in‚ 0.01171875 disk.nvme0n1.in‚ 62.8828125 disk.md0p1.out‚ 355.91796875# disk.nvme1n1.out‚26935.48828125 disk.sda.in‚ 0.01171875xŸuaü‚(:’ ÁÚèÔû¼Æ¯ gpu.0.gpu‚96.0 gpu.0.memory‚17+ gpu.0.memoryAllocated‚47.54737854003906* gpu.0.memoryAllocatedBytes‚ 40842887168 gpu.0.temp‚45.0 gpu.0.powerWatts‚226.141' gpu.0.enforcedPowerLimitWatts‚400.0( gpu.0.powerPercent‚56.53524999999999 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0}.4;ý‚(:° ÁÚèÔ…åš° disk.md0.out‚357.5625 disk.sda2.in‚ 0.01171875 disk.nvme1n1.in‚ 51.046875 network.sent‚ 5.254266e+06 disk.nvme0n1.in‚ 62.8828125 disk.sda1.out‚0 disk.sdb.in‚0 cpu‚25.502798435086497 proc.cpu.threads‚17) disk./.usagePercent‚83.44373153492867 disk.sda1.in‚0 disk.md0p1.out‚357.5625* proc.memory.percent‚18.656134605407715 disk.dm-0.in‚ 113.9296875 disk.sda2.out‚373.0625" disk.nvme1n1.out‚ 27015.0546875 disk.rbd0.in‚0 disk.sda.out‚373.0625 disk.md0.in‚ 0.01171875$ memory_percent‚93.09689998626709# proc.memory.rssMB‚ 3056.62109375 disk.rbd2.out‚1.125 disk.sda.in‚ 0.01171875) proc.memory.availableMB‚ 1131.00390625 disk.sdb1.in‚0 disk.rbd2.in‚0 disk.rbd0.out‚ 1.06201171875 network.recv‚ 1.293725e+06# disk.nvme0n1.out‚26418.92578125 disk.dm-0.out‚53433.98046875 disk.md0p1.in‚ 0.01171875 disk.sdb.out‚373.0625$ disk./.usageGB‚2984.352020263672 disk.sdb1.out‚0 disk.sdb2.in‚0 disk.sdb2.out‚373.0625:6~ñ’þ‚(:‡ ÐÚèÔâ®®° gpu.0.gpu‚25.0 gpu.0.memory‚6+ gpu.0.memoryAllocated‚47.54737854003906* gpu.0.memoryAllocatedBytes‚ 40842887168 gpu.0.temp‚47.0 gpu.0.powerWatts‚316.342' gpu.0.enforcedPowerLimitWatts‚400.0 gpu.0.powerPercent‚79.0855 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0:YåæHÿ‚(:½ ÐÚèÔ…®¤± disk.dm-0.in‚ 113.9296875 disk.sdb2.in‚0 disk.sda1.in‚0 disk.md0.out‚ 358.9921875 disk.rbd2.out‚1.125 disk.sdb.in‚0 disk.md0.in‚ 0.01171875 disk.sda2.out‚ 374.564453125" disk.nvme1n1.out‚ 27137.5859375# proc.memory.rssMB‚ 3056.62109375* proc.memory.percent‚18.656134605407715 network.recv‚ 1.298329e+06 disk.nvme0n1.in‚ 62.8828125# disk.nvme0n1.out‚26602.69921875 disk.sdb1.in‚0 disk.sdb2.out‚ 374.564453125 disk.rbd0.out‚1.125 disk.dm-0.out‚53740.28515625 disk.sda1.out‚0 disk.sdb1.out‚0 disk.md0p1.in‚ 0.01171875 disk.sdb.out‚ 374.564453125 disk.nvme1n1.in‚ 51.046875 cpu‚25.50206128115236$ disk./.usageGB‚2984.352020263672 disk.sda.in‚ 0.01171875 disk.rbd0.in‚0 disk.rbd2.in‚0 disk.sda2.in‚ 0.01171875% proc.memory.availableMB‚ 1131.6875 proc.cpu.threads‚17 disk.sda.out‚ 374.564453125 disk.md0p1.out‚ 358.9921875$ memory_percent‚93.09272766113281 network.sent‚ 5.260483e+06) disk./.usagePercent‚83.44373153492867âºÈç2€- _wandbruntime‚3846  _runtime‚3846D{ùÎÇÁ trainer/global_step‚2149 % _timestamp‚1.7884890477510943e+09 _runtime‚3846.852560777 _step‚171 & lr-AdamW/pg1‚0.0009253140936090879 & lr-AdamW/pg2‚0.0009253140936090879¦SJÉÌ‚Æ & lr-AdamW/pg1‚0.0009253140936090879 & lr-AdamW/pg2‚0.0009253140936090879 trainer/global_step‚2149 % _timestamp‚1.7884890477510943e+09 _runtime‚3846.852560777 _step‚171«–ÍQ­2ƒ- _wandbruntime‚3846  _runtime‚3846ŒáÄî(„ïO R>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚478585919342592 VGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 2516363264 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚584 trainer/global_step‚2149 P@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 38610163200 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 RBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 40290484224 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 ^LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 5160758644736 cQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 6442942030336 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 TGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚40327740 M@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚24270139 dQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚95479328910336 YEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚459666487504968 YIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 37073453056 KBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚2147 H?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚2147 ]IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚473427702489088 TDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 35913877504 PCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚30858509 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚14684934 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 WGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 34283657216 YJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 2514370140 ZHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 5151708008828 M=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 38610163200 ZFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚101922168223232 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 H?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚1534 _step‚172 `LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚473427702489088 XIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 3217031168 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚23682 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚23981 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 963120128 _runtime‚ 3846.85807294 IDeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚2684 NFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚330 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 577553920 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 MDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚1534 P@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 36985723401 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 \HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚473427677061120 H?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚1309 YEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚473427677061120 RCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 2541791232 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚19684 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚587 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 UADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚478585919342592 _MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 6442868410880 VGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 3140738488 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 MDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚2147 PCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚30858515 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚23682 SGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚9585205 `LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚454512265126000 O?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 40290484224 [IDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 5160758644736 SDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 3142753280 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚25427968 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 M@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚71186255 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 TDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 40290484224 WEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 5158242281472 YJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 2516363264 PCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚40308056 ^JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚101922270940672 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 UADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚459663947706860 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚6 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚73619456 ^LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 5154222378968 UFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 3217031168 D;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚3681 KBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚1534 WGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 35913877504 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.smºZ£ÙÍ all_pool.allocated‚14685518 \HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚454512239698032 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚23981 NFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚986 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 ZKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 3217031168 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 VBDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated‚478588461133824 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚19690 TGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚30858515 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚3 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚29097984 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚25427968 D>DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 [KDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 37073453056 A8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚3681 % _timestamp‚1.7884890477525475e+09 L?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚30858509 E8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚71166565 YEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚478588461133824 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚9585208 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚25427968 PCDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated‚40327740 L?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚40308056 VGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 3142753280 `MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚95479299812352 BDeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚23682 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚23981 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 963120128 _runtime‚ 3846.85807294 IDeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚2684 NFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚330 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 577553920 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 MDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚1534 P@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 36985723401 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 \HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚473427677061120 H?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚1309 YEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚473427677061120 RCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 2541791232 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚19684 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚587 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 UADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚478585919342592 _MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 6442868410880 VGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 3140738488 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 MDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚2147 PCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚30858515 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚23682 SGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚9585205 `LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚454512265126000 O?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 40290484224 [IDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 5160758644736 SDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 3142753280 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚25427968 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 M@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚71186255 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 TDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 40290484224 WEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 5158242281472 YJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 2516363264 PCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚40308056 ^JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚101922270940672 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 UADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚459663947706860 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚6 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚73619456 ^LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 5154222378968 UFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 3217031168 D;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚3681 KBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚1534 WGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 35913877504 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚14685518 \HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚454512239698032 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚23981 NFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚986 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 ZKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 3217031168 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 VBDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated‚478588461133824 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚19690 TGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚30858515 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚3 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚29097984 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚25427968 D>DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 [KDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 37073453056 A8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚3681 % _timestamp‚1.7884890477525475e+09 L?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚30858509 E8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚71166565 YEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚478588461133824 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚9585208 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚25427968 PCDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated‚40327740 L?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚40308056 VGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 3142753280 `MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚95479299812352 BDeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚478585919342592 VGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 2516363264 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚584 trainer/global_step‚2149 P@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 38610163200 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 RBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 40290484224 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 ^LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 5160758644736 cQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 6442942030336 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 TGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚40327740 M@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚24270139 dQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚95479328910336 YEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚459666487504968 YIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 37073453056 KBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚2147 H?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚2147 ]IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚473427702489088 TDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 35913877504 PCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚30858509 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚14684934 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0¬j2ºŒ2†- _wandbruntime‚3848  _runtime‚3848ÃÅ,ÿ&‡ùM ^JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚473641648559616 REDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚30872405 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚9589574 ZFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚454717755358528 F=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚2147 M>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 2541804544 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚577 LDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚986 TEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 3140738488 N>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 38610163200 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚23981 BDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 36985723401 K>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚24281055 FDeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 NADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚40326398 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚71508992 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 WHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 2516376576 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚19710 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1309 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 100606976 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 SDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 3217031168 QBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 3142753280 ?6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚3681 FDeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 WGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 37073453056 PDeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚71218513 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚14692058 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚2684 RCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 1515847168 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 VGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 3217031168 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚19716 S?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚478802221509120 XFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 5160598377472 REDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚14691484 F=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚1534 RBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 35913877504 aODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 6445896669696 M=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 40290484224 TDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 37073453056 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 577553920 YIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 37073453056 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚23981 ]KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 6445825160704 \JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 5156575514910 XFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 5154061144562 LDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚330 K;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 38610163200 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 963120128 E+Ýå%ˆþM J=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚40326398 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚6 \HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚101969355905024 [GDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚473641648559616 WCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚473641623131648 YGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 5163114754048 PADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 2541804544 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚25427968 bODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚95523459235328 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 KBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚2147 _runtime‚3848.452585945 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚6 NADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚30872399 ^JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚454717780786496 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 NADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚40326398 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚71508992 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 WHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 2516376576 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚19710 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1309 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 100606976 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 SDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 3217031168 QBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 3142753280 ?6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚3681 FDeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 WGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 37073453056 PDeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚71218513 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚14692058 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚2684 RCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 1515847168 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 VGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 3217031168 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚19716 S?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚478802221509120 XFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 5160598377472 REDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚14691484 F=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚1534 RBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 35913877504 aODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 6445896669696 M=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 40290484224 TDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 37073453056 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 577553920 YIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 37073453056 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚23981 ]KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 6445825160704 \JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 5156575514910 XFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 5154061144562 LDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚330 K;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 38610163200 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 963120128 EDeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 2541804544 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚577 LDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚986 TEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 3140738488 N>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 38610163200 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚23981 BDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 36985723401 K>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚24281055 FDeviceStatsMonitor.on_train_batch_start/allocation.all.current‚19690 M@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚24816117 H?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚1309 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚9803508 cQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 6590259545600 YEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚489403597288448 QDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚24816701 D>DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 % _timestamp‚1.7884891335128338e+09 G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚2684 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚19684 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚25427968 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚29097984 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚73619456 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚2684 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 ZHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 5269391967536 YIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 37073453056 P@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 38610163200 L?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚31553009 TGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚31553015 VBDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated‚489403597288448 ]IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚484125006015488 ^LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 5278591272960 YJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 2516363264 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚581 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 IDeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚23682 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 M=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 38610163200 O@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 2541791232 SGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚9803505 WGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 34283657216 YJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 2514370140 [KDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 37073453056 R>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚489401055497216 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 O?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 40290484224 PCDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated‚41248192 VGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 3142753280 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚6 dQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚97685845159936 UFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 3217031168 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚584 UADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚489401055497216 RCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 2541791232 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚15013193 NFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚986 TEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 1515847168 MDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚2147 YEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚484124980587520 _MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 6590185926144 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 KBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚2147 KBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚1534 _step‚176 ZHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 5276074909696 NFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚330 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 F=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚3681 trainer/global_step‚2199 PCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚31553015 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚6 L?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚41228508 [IDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 5278591272960 P@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 36985723401 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚23981 TGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚41248192 D;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚3681 PCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚31553009 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚23682 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 102717440 A8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚3681 M@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚72801207 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 MDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚1534 WEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 5276074909696 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚23981 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 _runtime‚3932.614520632 RCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 2539798108 ^LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 5271906337676 XIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 3217031168 H?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚2147 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚15012612 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 SDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 3142753280 PCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚41228508 `LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚464788048150800 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚25427968 TDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 40290484224 WGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 35913877504 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 \HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚464788022722832 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 H?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚1534 E8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚72781517 `LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚484125006015488 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚3 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 UADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚470057414690368 YEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚470059954488476 VGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 3140738488 RBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 40290484224 VGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 2516363264 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚19684 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 A8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚3681 BDeviceStatsMonitor.on_train_batch_start/allocation.all.current‚19690 M@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚24816117 H?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚1309 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚9803508 cQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 6590259545600 YEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚489403597288448 QDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚24816701 D>DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 % _timestamp‚1.7884891335128338e+09 G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚2684 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚19684 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚25427968 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚29097984 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚73619456 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚2684 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 ZHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 5269391967536 YIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 37073453056 P@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 38610163200 L?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚31553009 TGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚31553015 VBDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated‚489403597288448 ]IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚484125006015488 ^LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 5278591272960 YJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 2516363264 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚581 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 IDeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚23682 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 M=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 38610163200 O@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 2541791232 SGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚9803505 WGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 34283657216 YJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 2514370140 [KDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 37073453056 R>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚489401055497216 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 O?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 40290484224 PCDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated‚41248192 VGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 3142753280 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚6 dQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚97685845159936 UFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 3217031168 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚584 UADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚489401055497216 RCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 2541791232 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚15013193 NFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚986 TEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 1515847168 MDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚2147 YEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚484124980587520 _MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 6590185926144 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 KBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚2147 KBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚1534 _step‚176 ZHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 5276074909696 NFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚330 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 F=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚3681 trainer/global_step‚2199 PCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚31553015 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚6 L?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚41228508 [IDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 5278591272960 P@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 36985723401 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚23981 TGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚41248192 D;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚3681 PCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚31553009 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚23682 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 102717440 A8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚3681 M@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚72801207 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 MDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚1534 WEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 5276074909696 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚23981 í£œ‹ÇLFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 _runtime‚3932.614520632 RCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 2539798108 ^LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 5271906337676 XIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 3217031168 H?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚2147°¡ì._2ž- _wandbruntime‚3934  _runtime‚3934¤D­'ŸúM J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚31566899 NADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚41246850 ?6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚3681 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚19710 FDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 36985723401 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚23981 T@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚489619899468288 PADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 2541804544 ZFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚484338926658048 ?6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚3681 TEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 3140738488 XIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 3217031168 F=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚2147 S?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚489617357663744 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚23682 OBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚24827500 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 I@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚2147 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 C6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚72813749 TEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 3142753280 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚2684 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 _step‚177 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚589 bODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚97729975484928 aODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 6593199933952 RBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 40290484224 M=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 40290484224 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚9807871 LDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚986 SDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 3217031168 M>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 2541804544 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚23981 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚6 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 F=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚1534 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1309 WCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚489619899468288 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚81994752 WGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 37073453056 TDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 37073453056 K>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚72833465 XDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚104323064326144 RCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 1515847168 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 \JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 5274259473618 \JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 5280947382272 G:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚72813749 ^JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚464993563811296 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚25427968 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 XFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 5271745103270 P8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 trainer/global_step‚2199 _runtime‚3934.141622699 QBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 3142753280 UEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 35913877504 YIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 37073453056 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 NADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚31566905 NADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚41266560 RBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 35913877504 \HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚104323175418880 WCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚470267823284914 REDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚41266560 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚592 K>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚24826908 UEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 34283657216 VGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 3217031168 G:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚72833465 WCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚484338926658048 ]KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 6593117939200 B9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚3681 KBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚1534 WHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 2516376576 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚9807874 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 PADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 2539798316 [GDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚484338952086016 ^JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚484338952086016 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚25427968 NADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚31566899 LDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚330 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚6 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 111092736 WHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 2514370348 I@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚1534 BDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 FDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 38610163200 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 % _timestamp‚1.7884891350397308e+09 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚3 REDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚15019037 ^KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚97729946386944 UCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 5278431005696 S?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚470265283486598 YGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 5280947382272 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 577553920 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0æ€y}' ÿM P8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 trainer/global_step‚2199 _runtime‚3934.141622699 QBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 3142753280 UEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 35913877504 YIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 37073453056 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 NADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚31566905 NADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚41266560 RBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 35913877504 \HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚104323175418880 WCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚470267823284914 REDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚41266560 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚592 K>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚24826908 UEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 34283657216 VGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 3217031168 G:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚72833465 WCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚484338926658048 ]KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 6593117939200 B9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚3681 KBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚1534 WHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 2516376576 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚9807874 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 PADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 2539798316 [GDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚484338952086016 ^JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚484338952086016 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚25427968 NADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚31566899 LDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚330 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚6 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 111092736 WHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 2514370348 I@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚1534 BDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 FDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 38610163200 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 % _timestamp‚1.7884891350397308e+09 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚3 REDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚15019037 ^KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚97729946386944 UCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 5278431005696 S?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚470265283486598 YGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 5280947382272 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 577553920 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚31566899 NADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚41246850 ?6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚3681 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚19710 FDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 36985723401 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚23981 T@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚489619899468288 PADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 2541804544 ZFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚484338926658048 ?6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚3681 TEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 3140738488 XIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 3217031168 F=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚2147 S?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚489617357663744 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚23682 OBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚24827500 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 I@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚2147 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 C6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚72813749 TEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 3142753280 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚2684 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 _step‚177 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚589 bODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚97729975484928 aODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 6593199933952 RBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 40290484224 M=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 40290484224 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚9807871 LDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚986 SDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 3217031168 M>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 2541804544 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚23981 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚6 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 F=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚1534 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1309 WCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚489619899468288 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚81994752 WGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 37073453056 TDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 37073453056 K>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚72833465 XDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚104323064326144 RCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 1515847168 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 \JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 5274259473618 \JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 5280947382272 G:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚72813749 ^JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚464993563811296 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚25427968 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 XFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 5271745103270±‚"–2¡- _wandbruntime‚3934  _runtime‚3934\ê)[¢Õ 1loss/FBt3/assignment_loss‚0.29207402281463146 trainer/global_step‚2199 % _timestamp‚1.7884891350422235e+09 _step‚178 2loss/SRqqt2/assignment_loss‚1.1460111005115323 2loss/SRqqt4/assignment_loss‚1.4899920360870003 /loss/FRt1/detection_loss‚2.1292770591520127 1loss/SRqqt2/detection_loss‚0.9295025829757962 /loss/FBt1/detection_loss‚2.9985793895191617 1loss/SRqqt3/assignment_loss‚1.870636074206768 1loss/FBt2/assignment_loss‚0.13737882345584257 -loss/FRt3/detection_loss‚2.72809699177742 _runtime‚3934.143369876 0loss/FRt3/assignment_loss‚3.3452091821482486 0loss/FBt1/assignment_loss‚0.2291681444928852 /loss/FBt4/detection_loss‚3.1646181808577643 $loss/total_loss‚44.7464823889881 0loss/FRt2/assignment_loss‚1.4355687156762227 2loss/SRqqt1/assignment_loss‚1.8435269295292742 1loss/FBt4/assignment_loss‚0.13435755871695518 .loss/FBt3/detection_loss‚3.407612842321396 0loss/FRt4/assignment_loss‚3.0542912090217675 1loss/SRqqt1/detection_loss‚2.0390970286201027 0loss/SRqqt3/detection_loss‚2.311917537297958 .loss/FRt4/detection_loss‚2.470057026370541 1loss/SRqqt4/detection_loss‚1.6255845014444168 epoch‚3 /loss/FRt2/detection_loss‚1.5198739014824134 /loss/FBt2/detection_loss‚0.7384480837928621 0loss/FRt1/assignment_loss‚3.7056034667151314(§‰H`£Ú 1loss/SRqqt1/detection_loss‚2.0390970286201027 0loss/SRqqt3/detection_loss‚2.311917537297958 .loss/FRt4/detection_loss‚2.470057026370541 1loss/SRqqt4/detection_loss‚1.6255845014444168 epoch‚3 /loss/FRt2/detection_loss‚1.5198739014824134 /loss/FBt2/detection_loss‚0.7384480837928621 0loss/FRt1/assignment_loss‚3.7056034667151314 1loss/FBt3/assignment_loss‚0.29207402281463146 trainer/global_step‚2199 % _timestamp‚1.7884891350422235e+09 _step‚178 2loss/SRqqt2/assignment_loss‚1.1460111005115323 2loss/SRqqt4/assignment_loss‚1.4899920360870003 /loss/FRt1/detection_loss‚2.1292770591520127 1loss/SRqqt2/detection_loss‚0.9295025829757962 /loss/FBt1/detection_loss‚2.9985793895191617 1loss/SRqqt3/assignment_loss‚1.870636074206768 1loss/FBt2/assignment_loss‚0.13737882345584257 -loss/FRt3/detection_loss‚2.72809699177742 _runtime‚3934.143369876 0loss/FRt3/assignment_loss‚3.3452091821482486 0loss/FBt1/assignment_loss‚0.2291681444928852 /loss/FBt4/detection_loss‚3.1646181808577643 $loss/total_loss‚44.7464823889881 0loss/FRt2/assignment_loss‚1.4355687156762227 2loss/SRqqt1/assignment_loss‚1.8435269295292742 1loss/FBt4/assignment_loss‚0.13435755871695518 .loss/FBt3/detection_loss‚3.407612842321396 0loss/FRt4/assignment_loss‚3.0542912090217675²Nà”¤‚(:‰ ¹ÛèÔ“¸á¯ gpu.0.gpu‚96.0 gpu.0.memory‚23+ gpu.0.memoryAllocated‚47.54737854003906* gpu.0.memoryAllocatedBytes‚ 40842887168 gpu.0.temp‚48.0 gpu.0.powerWatts‚294.637' gpu.0.enforcedPowerLimitWatts‚400.0 gpu.0.powerPercent‚73.65925 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0µB¥‚(:· ¹ÛèÔù­Å±) disk./.usagePercent‚83.43600824579774 disk.sda.in‚ 0.01171875$ memory_percent‚93.09308528900146 proc.memory.rssMB‚3056.625 disk.rbd2.out‚1.125 disk.sdb.out‚385.49267578125 disk.sda.out‚385.49267578125 disk.rbd0.in‚0 disk.sdb1.out‚0 disk.sdb2.in‚0! disk.nvme0n1.out‚ 27125.484375! disk.sdb2.out‚385.49267578125 disk.md0p1.in‚ 0.01171875 disk.md0p1.out‚369.5 disk.rbd2.in‚0 network.recv‚ 1.328894e+06 disk.md0.out‚369.5) proc.memory.availableMB‚ 1131.62890625* proc.memory.percent‚18.656158447265625$ disk./.usageGB‚2984.075798034668 disk.dm-0.out‚55118.81640625 disk.sda1.in‚0 disk.nvme1n1.in‚ 55.046875# disk.nvme1n1.out‚27993.33203125 disk.sda1.out‚0 disk.md0.in‚ 0.01171875 cpu‚25.49677604852013 network.sent‚ 5.391329e+06 disk.rbd0.out‚1.125 disk.dm-0.in‚ 117.9296875 disk.sdb1.in‚0 disk.sdb.in‚0 disk.sda2.in‚ 0.01171875! disk.sda2.out‚385.49267578125 proc.cpu.threads‚17 disk.nvme0n1.in‚ 62.8828125ì‰Ð¯”¦‚(:‰ ÈÛèÔ­ýò¯ gpu.0.gpu‚38.0 gpu.0.memory‚10+ gpu.0.memoryAllocated‚47.54737854003906* gpu.0.memoryAllocatedBytes‚ 40842887168 gpu.0.temp‚41.0 gpu.0.powerWatts‚117.193' gpu.0.enforcedPowerLimitWatts‚400.0 gpu.0.powerPercent‚29.29825 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0î5ÇXK§‚(:À ÈÛèÔ¦Èΰ proc.cpu.threads‚17 disk.sda1.out‚0 disk.sdb.out‚386.87353515625 disk.nvme1n1.in‚ 55.046875 disk.sda.out‚386.87353515625) proc.memory.availableMB‚ 1131.38671875 disk.md0.in‚ 0.01171875 disk.rbd2.in‚0 disk.sdb.in‚0 disk.sdb2.in‚0! disk.sdb2.out‚386.87353515625) disk./.usagePercent‚83.43489065674218% disk./.usageGB‚2984.0358276367188 disk.nvme0n1.in‚ 62.8828125 disk.md0p1.in‚ 0.01171875 disk.rbd2.out‚1.125 disk.sda2.in‚ 0.01171875# disk.nvme1n1.out‚27996.00390625 disk.dm-0.out‚55160.75 disk.rbd0.out‚1.125# memory_percent‚93.0945634841919 cpu‚25.496173155361078 network.recv‚ 1.335608e+06# disk.nvme0n1.out‚27164.74609375 disk.sdb1.in‚0 disk.sdb1.out‚0 disk.md0.out‚ 370.8359375! disk.sda2.out‚386.87353515625 disk.sda.in‚ 0.01171875 disk.sda1.in‚0 disk.dm-0.in‚ 117.9296875 network.sent‚ 5.465207e+06 disk.md0p1.out‚ 370.8359375 proc.memory.rssMB‚3056.625* proc.memory.percent‚18.656158447265625 disk.rbd0.in‚0¿‘̨‚(:’ ×ÛèÔùøï gpu.0.gpu‚97.0 gpu.0.memory‚16+ gpu.0.memoryAllocated‚47.54737854003906* gpu.0.memoryAllocatedBytes‚ 40842887168 gpu.0.temp‚45.0 gpu.0.powerWatts‚241.909' gpu.0.enforcedPowerLimitWatts‚400.0( gpu.0.powerPercent‚60.47724999999999 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0MçlÌP©‚(:Å ×ÛèÔþø °* proc.memory.percent‚18.656158447265625 network.recv‚ 1.339231e+06) disk./.usagePercent‚83.43676638886006 disk.sdb.in‚0 disk.sdb.out‚388.93408203125 network.sent‚ 5.472972e+06 disk.rbd0.out‚1.125 disk.dm-0.out‚55350.10546875! disk.sdb2.out‚388.93408203125 disk.md0p1.in‚ 0.01171875 disk.md0p1.out‚ 372.8359375 disk.sda2.in‚ 0.01171875 disk.nvme1n1.in‚ 55.046875 disk.nvme0n1.in‚ 62.8828125 disk.dm-0.in‚ 117.9296875 disk.sdb2.in‚0 disk.sda1.in‚0 disk.md0.out‚ 372.8359375 disk.rbd2.in‚0! disk.sda2.out‚388.93408203125$ disk./.usageGB‚2984.102912902832 disk.sdb1.out‚0 disk.md0.in‚ 0.01171875 disk.sda.out‚388.93408203125 disk.rbd2.out‚1.125$ memory_percent‚93.09303760528564 cpu‚25.495317146819485# disk.nvme1n1.out‚28118.48828125 disk.sdb1.in‚0) proc.memory.availableMB‚ 1131.63671875 proc.memory.rssMB‚3056.625 proc.cpu.threads‚17 disk.sda.in‚ 0.01171875" disk.nvme0n1.out‚ 27231.6171875 disk.rbd0.in‚0 disk.sda1.out‚0)ÕX”ª‚(:‰ æÛèÔºµ° gpu.0.gpu‚95.0 gpu.0.memory‚16+ gpu.0.memoryAllocated‚47.54737854003906* gpu.0.memoryAllocatedBytes‚ 40842887168 gpu.0.temp‚45.0 gpu.0.powerWatts‚182.031' gpu.0.enforcedPowerLimitWatts‚400.0 gpu.0.powerPercent‚45.50775 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0òaÎ H«‚(:½ æÛèÔ²ñ± disk.sdb1.in‚0 disk.sdb1.out‚0 disk.md0p1.in‚ 0.01171875 disk.sda2.in‚ 0.01171875 network.sent‚ 5.4766e+06 disk.nvme0n1.in‚ 62.8828125 disk.md0p1.out‚374.0625* proc.memory.percent‚18.656158447265625 cpu‚25.494638715592718 network.recv‚ 1.343453e+06 disk.sda.out‚390.22802734375 disk.sda1.in‚0 disk.sda1.out‚0 disk.nvme1n1.in‚ 55.046875 disk.rbd0.in‚0 disk.dm-0.out‚ 55520.6015625! disk.sda2.out‚390.22802734375 disk.sda.in‚ 0.01171875 disk.rbd0.out‚1.125 disk.rbd2.out‚1.125 disk.sdb.in‚0) disk./.usagePercent‚83.44026656088216$ disk./.usageGB‚2984.228096008301 disk.dm-0.in‚ 117.9296875 disk.sdb2.in‚0 disk.md0.out‚374.0625) proc.memory.availableMB‚ 1131.14453125! disk.sdb2.out‚390.22802734375 disk.md0.in‚ 0.01171875 disk.sdb.out‚390.22802734375 proc.memory.rssMB‚3056.625 disk.rbd2.in‚0# disk.nvme1n1.out‚28241.00390625$ memory_percent‚93.09604167938232 proc.cpu.threads‚17# disk.nvme0n1.out‚27279.59765625©‘¬‚(:† õÛèÔˆ¿á¯ gpu.0.gpu‚23.0 gpu.0.memory‚7+ gpu.0.memoryAllocated‚47.54737854003906* gpu.0.memoryAllocatedBytes‚ 40842887168 gpu.0.temp‚47.0 gpu.0.powerWatts‚309.212' gpu.0.enforcedPowerLimitWatts‚400.0 gpu.0.powerPercent‚77.303 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0Ô…ÁÞN­‚(:à õÛèÔ㊱" disk.nvme1n1.out‚ 28363.4453125 disk.sda.out‚391.61083984375 disk.rbd0.in‚0 cpu‚25.494038348202427 network.recv‚ 1.348019e+06( disk./.usagePercent‚83.4437555335484! disk.sda2.out‚391.61083984375 disk.nvme1n1.in‚ 55.046875 disk.nvme0n1.out‚ 27417.46875! disk.sdb2.out‚391.61083984375 disk.sdb2.in‚0 disk.sda1.out‚0 disk.md0p1.in‚ 0.01171875 disk.sdb.in‚0 disk.sda2.in‚ 0.01171875 disk.dm-0.out‚ 55780.9140625 disk.sdb1.out‚0 disk.sda1.in‚0 network.sent‚ 5.484771e+06) proc.memory.availableMB‚ 1131.63671875 disk.md0.out‚ 375.40234375 disk.nvme0n1.in‚ 62.8828125 disk.dm-0.in‚ 117.9296875 disk.md0.in‚ 0.01171875 disk.sdb.out‚391.61083984375 disk.rbd0.out‚1.125 disk.sdb1.in‚0* proc.memory.percent‚18.656158447265625% disk./.usageGB‚2984.3528785705566 disk.rbd2.in‚0 disk.sda.in‚ 0.01171875$ memory_percent‚93.09303760528564 proc.cpu.threads‚17 disk.md0p1.out‚ 375.40234375 proc.memory.rssMB‚3056.625 disk.rbd2.out‚1.125E9k2®- _wandbruntime‚4018  _runtime‚4018÷†PSƯÀ $ _timestamp‚1.788489219526436e+09 _runtime‚4018.628893338 _step‚179 & lr-AdamW/pg1‚0.0008991350229570849 & lr-AdamW/pg2‚0.0008991350229570849 trainer/global_step‚2249C+å<Ë°Å trainer/global_step‚2249 $ _timestamp‚1.788489219526436e+09 _runtime‚4018.628893338 _step‚179 & lr-AdamW/pg1‚0.0008991350229570849 & lr-AdamW/pg2‚0.0008991350229570849³˜5 H2±- _wandbruntime‚4018  _runtime‚4018Àš)²ðO PCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚32247509 H?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚1309 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚15340112 \HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚494822284113920 YJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 2516363264 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 E8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚74396469 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 TDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 35913877504 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚2684 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚23682 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚572 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚3 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 IDeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 `LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚475063831175600 XIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 3217031168 A8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚3681 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚6 TGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚42168644 PCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚42148960 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 VGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 3140738488 G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚2684 L?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚42148960 VBDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated‚500218733443072 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚6 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 YEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚494822284113920 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚19684 ^JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚106630289718784 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 YEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚480453421471984 D;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚3681 UADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚500216191651840 O?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 40290484224 ]IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚494822309541888 dQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚99892361409536 P@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 36985723401 WGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 34283657216 H?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚1534 R>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚500216191651840 cQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 6737928309248 ZKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 3217031168 `LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚494822309541888 TDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 40290484224 UFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 3217031168 _runtime‚ 4018.62934458 M=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 38610163200 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 ZHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 5387075926244 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚23981 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚19684 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚71522304 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 RBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 40290484224 KBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚2147 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚23682 UADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚480450881673876 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚29097984 [KDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 37073453056 VGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 3142753280 IDeviceStatsMonitor.on_train_batch_start/allocation.all.current‚19690 \HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚475063805747632 trainer/global_step‚2249 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚25427968 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚569 YJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 2514370140 WEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 5393907537920 WGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 35913877504 M@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚25361917 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 H?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚2147 RCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 2541791232 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 100620288 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 963120128 RCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 2539798108 PCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚32247515 NFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚330?YFëû'³õO O@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 2541791232 YEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚500218733443072 PCDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated‚42168644 TEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 1515847168 BDeviceStatsMonitor.on_train_batch_start/allocation.all.current‚19690 \HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚475063805747632 trainer/global_step‚2249 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚25427968 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚569 YJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 2514370140 WEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 5393907537920 WGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 35913877504 M@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚25361917 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 H?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚2147 RCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 2541791232 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 100620288 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 963120128 RCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 2539798108 PCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚32247515 NFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚330 PCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚32247509 H?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚1309 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚15340112 \HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚494822284113920 YJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 2516363264 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 E8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚74396469 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 TDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 35913877504 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚2684 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚23682 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚572 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚3 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 IDeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 `LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚475063831175600 XIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 3217031168 A8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚3681 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚6 TGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚42168644 PCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚42148960 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 VGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 3140738488 G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚2684 L?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚42148960 VBDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated‚500218733443072 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚6 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 YEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚494822284113920 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚19684 ^JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚106630289718784 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 YEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚480453421471984 D;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚3681 UADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚500216191651840 O?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 40290484224 ]IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚494822309541888 dQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚99892361409536 P@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 36985723401 WGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 34283657216 H?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚1534 R>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚500216191651840 cQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 6737928309248 ZKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 3217031168 `LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚494822309541888 TDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 40290484224 UFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 3217031168 _runtime‚ 4018.62934458 M=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 38610163200 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 ZHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 5387075926244 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚23981 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚19684 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚71522304 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 RBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 40290484224 KBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚2147 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚23682 UADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚480450881673876 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚29097984 [KDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 37073453056 VGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 3142753280 IDeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚25372813 RCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 1515847168 PADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 2539798316 WHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 2514370348 RBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 35913877504 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 963120128 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚23981 % _timestamp‚1.7884892211285553e+09 _runtime‚4020.231098474 ZFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚495036230184448 REDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚42187012 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚10026174 I@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚2147 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚25427968 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚19710 N>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 38610163200 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚23981 FDeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚74448417 FDeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 aODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 6740879192576 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 F=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚1534 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚569 TDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 37073453056 SDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 3217031168 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚6 QBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 3142753280 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚3 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 WCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚480661290268422 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚32261399 TEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 2516376576 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 N>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 36985723401 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 NADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚42187012 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚566 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 LDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚986 ]KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 6740807683584 UEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 34283657216 YIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 37073453056 UEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 35913877504 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 100606976 ^JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚475269346836096 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚25427968 [GDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚495036255612416 ^JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚495036255612416 G:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚74448417 WHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 2516376576 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚2684 \HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚106677370927104 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚71508992 NADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚32261405 NADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚42167302 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 ZFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚475269321408128 \JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 5391943432326 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚19716 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚15347208 J=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚42167302 M>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 2541804544 PDeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚74448417 FDeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 aODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 6740879192576 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 F=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚1534 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚569 TDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 37073453056 SDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 3217031168 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚6 QBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 3142753280 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚3 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 WCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚480661290268422 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚32261399 TEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 2516376576 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 N>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 36985723401 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 NADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚42187012 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚566 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 LDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚986 ]KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 6740807683584 UEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 34283657216 YIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 37073453056 UEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 35913877504 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 100606976 ^JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚475269346836096 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚25427968 [GDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚495036255612416 ^JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚495036255612416 G:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚74448417 WHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 2516376576 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚2684 \HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚106677370927104 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚71508992 NADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚32261405 NADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚42167302 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 ZFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚475269321408128 \JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 5391943432326 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚19716 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚15347208 J=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚42167302 M>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 2541804544 PDeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚25372813 RCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 1515847168 PADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 2539798316 WHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 2514370348 RBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 35913877504 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 963120128 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚23981 % _timestamp‚1.7884892211285553e+09 _runtime‚4020.231098474 ZFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚495036230184448 REDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚42187012 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚10026174 I@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚2147 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚25427968 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚19710 N>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 38610163200 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚23981 FVi‘‚(:† ÀÜèÔÕèþ¯ gpu.0.gpu‚91.0 gpu.0.memory‚16+ gpu.0.memoryAllocated‚47.54737854003906* gpu.0.memoryAllocatedBytes‚ 40842887168 gpu.0.temp‚45.0 gpu.0.powerWatts‚313.22' gpu.0.enforcedPowerLimitWatts‚400.0 gpu.0.powerPercent‚78.305 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0ßVâLÂ(:Á ÀÜèÔŽ§Ú° network.recv‚ 1.373848e+06# disk.nvme0n1.out‚27799.75390625 disk.dm-0.in‚ 117.9296875 disk.md0.out‚ 383.5546875 proc.memory.rssMB‚3056.625 cpu‚25.490558711589358 proc.cpu.threads‚17 disk.rbd0.out‚1.125 disk.dm-0.out‚ 56776.03125 disk.sda1.in‚0 disk.md0.in‚ 0.01171875 disk.md0p1.in‚ 0.01171875 disk.sda.in‚ 0.01171875) disk./.usagePercent‚83.43764452502103 disk.sdb1.in‚0 disk.md0p1.out‚ 383.5546875 disk.rbd2.in‚0! disk.sda2.out‚400.05029296875 disk.sda.out‚400.05029296875 disk.rbd0.in‚0 disk.nvme0n1.in‚ 62.8828125! disk.sdb2.out‚400.05029296875 disk.nvme1n1.in‚ 55.046875 disk.sdb1.out‚0 disk.sdb.in‚0 disk.sdb.out‚400.05029296875$ memory_percent‚93.09334754943848( proc.memory.availableMB‚ 1131.5859375 disk.sda1.out‚0# disk.nvme1n1.out‚28976.27734375* proc.memory.percent‚18.656158447265625 network.sent‚ 5.653834e+06# disk./.usageGB‚2984.13431930542 disk.sdb2.in‚0 disk.rbd2.out‚1.125 disk.sda2.in‚ 0.01171875,Ä`?žÄ‚(:“ ÏÜèÔÐׯ gpu.0.gpu‚97.0 gpu.0.memory‚17+ gpu.0.memoryAllocated‚47.54737854003906* gpu.0.memoryAllocatedBytes‚ 40842887168 gpu.0.temp‚45.0 gpu.0.powerWatts‚221.409' gpu.0.enforcedPowerLimitWatts‚400.0) gpu.0.powerPercent‚55.352250000000005 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0Bc§5IÅ‚(:¾ ÏÜèÔ‹—Ž°$ memory_percent‚93.09337139129639* proc.memory.percent‚18.656158447265625! disk.sda2.out‚401.39892578125 disk.sda.out‚401.39892578125 network.sent‚ 5.65702e+06 disk.nvme0n1.in‚ 63.02734375 disk.dm-0.in‚ 118.08984375 disk.dm-0.out‚ 57015.2421875! disk.sdb2.out‚401.39892578125 disk.md0p1.in‚ 0.01171875% disk./.usageGB‚2984.3534202575684 disk.sdb1.out‚0 disk.sda1.out‚0 disk.md0.in‚ 0.01171875 disk.rbd2.out‚1.125 disk.sda2.in‚ 0.01171875 disk.rbd0.in‚0 proc.cpu.threads‚17 disk.nvme1n1.in‚55.0625 network.recv‚ 1.377397e+06 disk.sdb2.in‚0 disk.sda1.in‚0 disk.md0.out‚ 384.84375! disk.nvme1n1.out‚ 29098.765625 disk.sda.in‚ 0.01171875 disk.sdb1.in‚0 disk.md0p1.out‚ 384.84375) disk./.usagePercent‚83.44377067934397 disk.rbd2.in‚0 disk.sdb.out‚401.39892578125) proc.memory.availableMB‚ 1131.58203125 proc.memory.rssMB‚3056.625 cpu‚25.489872469562375" disk.nvme0n1.out‚ 27916.4765625 disk.sdb.in‚0 disk.rbd0.out‚1.125z e2Æ- _wandbruntime‚4104  _runtime‚4104ýé³2ÇÇÁ & lr-AdamW/pg1‚0.0008856260867791042 & lr-AdamW/pg2‚0.0008856260867791042 trainer/global_step‚2299 % _timestamp‚1.7884893051089127e+09 _runtime‚4104.211137634 _step‚183«ËÍiÌÈÆ & lr-AdamW/pg1‚0.0008856260867791042 & lr-AdamW/pg2‚0.0008856260867791042 trainer/global_step‚2299 % _timestamp‚1.7884893051089127e+09 _runtime‚4104.211137634 _step‚183·ŸŒšÞ2É- _wandbruntime‚4104  _runtime‚4104¹_~ú'ÊôO GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚6 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚25427968 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚6 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚10240105 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚29097984 WGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 34283657216 A8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚3681 VGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 2516363264 WGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 35913877504 QDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚25908358 cQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 6885589288960 H?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚2147 TDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 35913877504 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚573 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 NFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚986 BDeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 ]IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚505519613068288 A8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚3681 L?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚43069412 \HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚505519587640320 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚576 VBDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated‚511033869597696 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 H?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚1534 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚23981 P@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 38610163200 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚19690 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 100620288 ZFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚108984366327808 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 ZKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 3217031168 MDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚2147 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 % _timestamp‚1.7884893051104953e+09 [IDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 5514256529408 M@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚76031111 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚15667677 YEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚490846888455492 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 577553920 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 P@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 36985723401 E8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚76011421 PCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚32942015 TGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚32942015 ^JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚108984466948096 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚23682 YJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 2516363264 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 ZHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 5504759884952 PCDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated‚43089096 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚10240108 aMDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚102098848561152 RBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 40290484224 KBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚1534 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 O@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 2541791232 R>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚511031327806464 IDeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚2684 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚2684 NFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚330 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 `LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚485339614200400 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚25427968 M=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 38610163200 YEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚511033869597696 TEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 1515847168 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 963120128 [KDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 37073453056 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 RCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 2541791232 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚23682 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚3 RCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 2539798108 YIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 37073453056 XIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 3217031168 trainer/global_step‚2299 L?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚32942009 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚19684 ZHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 5511740166144 VGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 3142753280 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚23981 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚25427968 _MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 6885517766656 UADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚490844348657384 _step‚184 VGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 3140738488 UFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 3217031168 WEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 5511740166144 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 F=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚3681°t¬&ËùO XIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 3217031168 trainer/global_step‚2299 L?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚32942009 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚19684 ZHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 5511740166144 VGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 3142753280 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚23981 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚25427968 _MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 6885517766656 UADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚490844348657384 _step‚184 VGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 3140738488 UFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 3217031168 WEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 5511740166144 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 F=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚3681 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚6 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚25427968 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚6 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚10240105 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚29097984 WGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 34283657216 A8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚3681 VGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 2516363264 WGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 35913877504 QDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚25908358 cQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 6885589288960 H?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚2147 TDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 35913877504 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚573 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 NFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚986 BDeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 ]IDeviceStat+UÙsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚505519613068288 A8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚3681 L?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚43069412 \HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚505519587640320 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚576 VBDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated‚511033869597696 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 H?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚1534 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚23981 P@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 38610163200 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚19690 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 100620288 ZFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚108984366327808 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 ZKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 3217031168 MDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚2147 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 % _timestamp‚1.7884893051104953e+09 [IDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 5514256529408 M@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚76031111 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚15667677 YEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚490846888455492 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 577553920 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 P@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 36985723401 E8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚76011421 PCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚32942015 TGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚32942015 ^JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚108984466948096 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚23682 YJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 2516363264 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 ZHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 5504759884952 PCDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated‚43089096 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚10240108 aMDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚102098848561152 RBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 40290484224 KBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚1534 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 O@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 2541791232 R>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚511031327806464 IDeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚2684 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚2684 NFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚330 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 `LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚485339614200400 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚25427968 M=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 38610163200 YEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚511033869597696 TEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 1515847168 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 963120128 [KDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 37073453056 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 RCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 2541791232 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚23682 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚3 RCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 2539798108 YIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 37073453056¸ÓQ2Ì- _wandbruntime‚4105  _runtime‚4105g‘q›'ÍýM UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚25427968 XFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 5507113020686 ZFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚505733533710848 REDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚15674248 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 100606976 FDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 36985723401 ^JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚485545129860896 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚25427968 D;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚3681 T@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚511250171777536 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚25427968 ]KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 6888467432448 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 S?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚491052217453614 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚23981 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚19710 WHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 2514370348 P@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 40290484224 VGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 3217031168 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 NADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚32955905 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚32955899 M>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 2541804544 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 WCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚491054757251930 KBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚2147 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 WCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚511250171777536 aODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 6888538941440 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 G:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚76063369 K>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚25918719 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚578 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 BDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 38610163200 M=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 40290484224 ?6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚3681 UCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 5514096262144 QBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 3142753280 K>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚76063369 REDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚32955905 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚19716 REDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚43107464 WCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚505733533710848 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚71508992 UEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 34283657216 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 XIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 3217031168 % _timestamp‚1.7884893068788602e+09 _step‚185 NADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚32955899 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚581 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1309 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚3 XDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚109031446318592 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 B9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚3681 NADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚43087754 OBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚25919300 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚6 WHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 2516376576 TEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 3142753280 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚2684 TDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 37073453056 C6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚76043653 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚23981 S?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚511247629972992 \JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 5516612638720 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚6 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 NADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚43107464 ^JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚505733559138816_åé'΂N E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 NADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚43107464 ^JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚505733559138816 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚25427968 XFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 5507113020686 ZFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚505733533710848 REDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚15674248 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 100606976 FDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 36985723401 ^JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚485545129860896 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚25427968 D;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚3681 T@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚511250171777536 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚25427968 ]KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 6888467432448 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 S?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚491052217453614 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚23981 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚19710 WHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 2514370348 P@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 40290484224 VGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 3217031168 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 NADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚32955905 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚32955899 M>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 2541804544 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 WCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚491054757251930 KBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚2147 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 WCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚511250171777536 aODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 6888538941440 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 G:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚76063369 K>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚25918719 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚578 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 BDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 38610163200 M=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 40290484224 ?6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚3681 UCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 5514096262144 QBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 3142753280 K>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚76063369 REDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚32955905 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚19716 REDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚43107464 WCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚505733533710848 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚71508992 UEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 34283657216 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 XIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 3217031168 % _timestamp‚1.7884893068788602e+09 _step‚185 NADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚32955899 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚581 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1309 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚3 XDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚109031446318592 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 B9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚3681 NADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚43087754 OBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚25919300 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚6 WHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 2516376576 TEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 3142753280 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚2684 TDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 37073453056 C6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚76043653 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚23981 S?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚511247629972992 \JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 5516612638720 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚6¹§iÈY2Ï- _wandbruntime‚4105  _runtime‚4105`›TXZÐÔ 1loss/SRqqt2/detection_loss‚0.9982413320539106 /loss/FBt1/detection_loss‚3.5559846437894382 /loss/FBt2/detection_loss‚0.6708529423822921 .loss/FBt4/detection_loss‚2.425150850239922 epoch‚3 /loss/FRt4/assignment_loss‚2.410084404209827 /loss/FRt2/detection_loss‚1.5218735687208897 2loss/SRqqt3/assignment_loss‚1.8688931087652842 2loss/SRqqt4/assignment_loss‚1.5034187562138277 _step‚186 1loss/FBt1/assignment_loss‚0.20269044201771835 _runtime‚4105.981801292 0loss/FRt2/assignment_loss‚1.2595117417248813 .loss/FRt1/detection_loss‚2.125768793050689 1loss/SRqqt3/detection_loss‚2.5236231098572413 0loss/SRqqt4/detection_loss‚1.590612874087626 .loss/FBt3/detection_loss‚2.701011163847787 %loss/total_loss‚43.10393176818721 0loss/FRt3/assignment_loss‚3.0538622336140993 2loss/SRqqt1/assignment_loss‚1.8991836477880892 0loss/FBt3/assignment_loss‚0.3463351514171033 /loss/FRt3/detection_loss‚3.0112858352989984 1loss/SRqqt1/detection_loss‚2.1862737212492074 trainer/global_step‚2299 2loss/SRqqt2/assignment_loss‚1.1166353017102415 0loss/FBt2/assignment_loss‚0.1111110664645491 1loss/FBt4/assignment_loss‚0.19804556597869927 % _timestamp‚1.7884893068806632e+09 /loss/FRt1/assignment_loss‚3.448004688277389 .loss/FRt4/detection_loss‚2.375476825427502#€_ÑÙ 1loss/SRqqt1/detection_loss‚2.1862737212492074 trainer/global_step‚2299 2loss/SRqqt2/assignment_loss‚1.1166353017102415 0loss/FBt2/assignment_loss‚0.1111110664645491 1loss/FBt4/assignment_loss‚0.19804556597869927 % _timestamp‚1.7884893068806632e+09 /loss/FRt1/assignment_loss‚3.448004688277389 .loss/FRt4/detection_loss‚2.375476825427502 1loss/SRqqt2/detection_loss‚0.9982413320539106 /loss/FBt1/detection_loss‚3.5559846437894382 /loss/FBt2/detection_loss‚0.6708529423822921 .loss/FBt4/detection_loss‚2.425150850239922 epoch‚3 /loss/FRt4/assignment_loss‚2.410084404209827 /loss/FRt2/detection_loss‚1.5218735687208897 2loss/SRqqt3/assignment_loss‚1.8688931087652842 2loss/SRqqt4/assignment_loss‚1.5034187562138277 _step‚186 1loss/FBt1/assignment_loss‚0.20269044201771835 _runtime‚4105.981801292 0loss/FRt2/assignment_loss‚1.2595117417248813 .loss/FRt1/detection_loss‚2.125768793050689 1loss/SRqqt3/detection_loss‚2.5236231098572413 0loss/SRqqt4/detection_loss‚1.590612874087626 .loss/FBt3/detection_loss‚2.701011163847787 %loss/total_loss‚43.10393176818721 0loss/FRt3/assignment_loss‚3.0538622336140993 2loss/SRqqt1/assignment_loss‚1.8991836477880892 0loss/FBt3/assignment_loss‚0.3463351514171033 /loss/FRt3/detection_loss‚3.0112858352989984º…â$™Ò‚(:’ ÞÜèÔȚð gpu.0.gpu‚96.0 gpu.0.memory‚23+ gpu.0.memoryAllocated‚47.54737854003906* gpu.0.memoryAllocatedBytes‚ 40842887168 gpu.0.temp‚45.0 gpu.0.powerWatts‚242.89' gpu.0.enforcedPowerLimitWatts‚400.0) gpu.0.powerPercent‚60.722500000000004 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0ÒY;PÓ‚(:Å ÞÜèԆ쀱% disk./.usageGB‚2984.3529777526855 disk.rbd0.in‚0 disk.nvme1n1.in‚55.0625$ memory_percent‚93.09611320495605 network.sent‚ 5.675765e+06 network.recv‚ 1.381784e+06" disk.nvme0n1.out‚ 28031.5390625 disk.sda1.in‚0 disk.sdb.out‚403.10107421875 disk.dm-0.in‚ 118.08984375 disk.md0.in‚ 0.01171875 disk.md0.out‚ 386.4765625 disk.sdb.in‚0 disk.sda.in‚ 0.01171875 disk.rbd0.out‚1.125 disk.nvme0n1.in‚ 63.02734375 disk.dm-0.out‚57252.84765625 disk.md0p1.in‚ 0.01171875# disk.nvme1n1.out‚29221.30859375 disk.sdb1.out‚0 disk.sdb2.in‚0 disk.sda1.out‚0! disk.sda2.out‚403.10107421875 disk.sda.out‚403.10107421875( proc.memory.availableMB‚ 1131.1328125 proc.cpu.threads‚17) disk./.usagePercent‚83.44375830672223! disk.sdb2.out‚403.10107421875 disk.md0p1.out‚ 386.4765625 disk.rbd2.out‚1.125 disk.sda2.in‚ 0.01171875 disk.sdb1.in‚0 disk.rbd2.in‚0 proc.memory.rssMB‚3056.625* proc.memory.percent‚18.656158447265625 cpu‚25.489232538603495ž2Ÿ ”Ô‚(:‰ íÜèԜӔ® gpu.0.gpu‚42.0 gpu.0.memory‚11+ gpu.0.memoryAllocated‚47.54737854003906* gpu.0.memoryAllocatedBytes‚ 40842887168 gpu.0.temp‚46.0 gpu.0.powerWatts‚312.595' gpu.0.enforcedPowerLimitWatts‚400.0 gpu.0.powerPercent‚78.14875 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0×0AGHÕ‚(:½ íÜèÔÐíû° network.sent‚ 5.751251e+06 disk.md0.out‚ 388.3984375 disk.sda2.in‚ 0.01171875# disk.nvme1n1.out‚29343.75390625 disk.nvme0n1.in‚ 63.02734375* proc.memory.percent‚18.656158447265625 disk.sdb1.in‚0 disk.sdb1.out‚0 disk.sdb.out‚ 405.091796875 disk.rbd0.out‚1.125# disk.nvme0n1.out‚28075.61328125 proc.memory.rssMB‚3056.625 disk.dm-0.in‚ 118.08984375 disk.sda1.out‚0 disk.md0p1.in‚ 0.01171875 disk.md0p1.out‚ 388.3984375 disk.rbd0.in‚0$ memory_percent‚93.09182167053223( proc.memory.availableMB‚ 1131.8359375 cpu‚25.488609378329578) disk./.usagePercent‚83.44379553124796 disk.dm-0.out‚ 57419.3671875 disk.sdb2.in‚0 disk.sda1.in‚0 disk.sda2.out‚ 405.091796875 disk.nvme1n1.in‚55.0625 disk.sda.out‚ 405.091796875% disk./.usageGB‚2984.3543090820312 disk.md0.in‚ 0.01171875 disk.rbd2.in‚0 disk.rbd2.out‚1.125 proc.cpu.threads‚17 disk.sdb2.out‚ 405.091796875 disk.sdb.in‚0 network.recv‚ 1.387494e+06 disk.sda.in‚ 0.01171875úI®§âÖ‚(:‡ üÜèÔ°æή gpu.0.gpu‚98.0 gpu.0.memory‚18+ gpu.0.memoryAllocated‚47.54737854003906* gpu.0.memoryAllocatedBytes‚ 40842887168 gpu.0.temp‚45.0 gpu.0.powerWatts‚225.476' gpu.0.enforcedPoÉ “Ê°werLimitWatts‚400.0 gpu.0.powerPercent‚56.369 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0¯ƒ˜Fׂ(:» üÜèÔèÍž¯) disk./.usagePercent‚83.43604675024984 disk.sda.out‚406.60107421875 disk.md0.out‚ 389.828125 disk.md0p1.in‚ 0.01171875 disk.md0p1.out‚ 389.828125& proc.memory.availableMB‚ 1131.34375 network.sent‚ 5.755236e+06 disk.rbd0.out‚1.125 disk.sdb2.in‚0 disk.sdb.out‚406.60107421875 proc.memory.rssMB‚3056.625 proc.cpu.threads‚17 disk.sda2.in‚ 0.01171875 disk.nvme1n1.in‚55.0625 disk.nvme0n1.in‚ 63.02734375 disk.sda1.out‚0! disk.sda2.out‚406.60107421875 disk.rbd2.out‚1.125! disk.nvme1n1.out‚ 29345.140625" disk.nvme0n1.out‚ 28192.4140625 disk.sdb1.in‚0 disk.sda1.in‚0* proc.memory.percent‚18.656158447265625 cpu‚25.48783210679333 network.recv‚ 1.392114e+06 disk.dm-0.in‚ 118.08984375 disk.sdb1.out‚0! disk.sdb2.out‚406.60107421875 disk.sdb.in‚0# memory_percent‚93.0948257446289 disk.sda.in‚ 0.01171875 disk.dm-0.out‚ 57537.5546875 disk.md0.in‚ 0.01171875$ disk./.usageGB‚2984.077175140381 disk.rbd0.in‚0 disk.rbd2.in‚0³Îì.’Ø‚(:‡ ‹ÝèÔ¿àÄ® gpu.0.gpu‚71.0 gpu.0.memory‚13+ gpu.0.memoryAllocated‚47.54737854003906* gpu.0.memoryAllocatedBytes‚ 40842887168 gpu.0.temp‚45.0 gpu.0.powerWatts‚235.39' gpu.0.enforcedPowerLimitWatts‚400.0 gpu.0.powerPercent‚58.8475 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0ž²HÙ‚(:½ ‹ÝèÔâþ…¯) disk./.usagePercent‚83.43492926785481 disk.rbd0.in‚0 disk.nvme1n1.in‚55.0625 disk.nvme1n1.out‚ 29467.6875 disk.sdb.in‚0" disk.nvme0n1.out‚ 28231.6015625 disk.sdb2.out‚407.9306640625 disk.md0.in‚ 0.01171875 disk.md0p1.out‚ 391.09765625 proc.cpu.threads‚17 network.sent‚ 5.760203e+06 disk.rbd0.out‚1.125 disk.dm-0.out‚ 57699.2890625 disk.sda1.in‚0$ memory_percent‚93.09332370758057) proc.memory.availableMB‚ 1131.58984375 network.recv‚ 1.395874e+06 disk.sda.in‚ 0.01171875 disk.sda1.out‚0 disk.md0.out‚ 391.09765625 disk.rbd2.in‚0 disk.sdb1.in‚0 disk.sdb1.out‚0 disk.rbd2.out‚1.125 disk.sdb.out‚407.9306640625$ disk./.usageGB‚2984.037208557129 disk.sda.out‚407.9306640625 disk.nvme0n1.in‚ 63.02734375 disk.sdb2.in‚0* proc.memory.percent‚18.656158447265625 cpu‚25.48722027520452 disk.dm-0.in‚ 118.08984375 disk.sda2.out‚407.9306640625 proc.memory.rssMB‚3056.625 disk.md0p1.in‚ 0.01171875 disk.sda2.in‚ 0.01171875æ°•‰”Ú‚(:‰ šÝèÔª•Ñ¯ gpu.0.gpu‚99.0 gpu.0.memory‚23+ gpu.0.memoryAllocated‚47.54737854003906* gpu.0.memoryAllocatedBytes‚ 40842887168 gpu.0.temp‚46.0 gpu.0.powerWatts‚322.847' gpu.0.enforcedPowerLimitWatts‚400.0 gpu.0.powerPercent‚80.71175 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0¦žØHÛ‚(:½ šÝèÔ ˜¤° disk.sdb1.in‚0 disk.md0.in‚ 0.01171875 disk.nvme1n1.in‚55.0625 disk.sda.out‚ 409.494140625$ memory_percent‚93.09353828430176 cpu‚25.48665420187327# disk.nvme1n1.out‚29590.23046875 disk.sda1.in‚0 disk.sda1.out‚0 disk.rbd2.in‚0 disk.sdb.out‚ 409.494140625 disk.dm-0.in‚ 118.08984375* proc.memory.percent‚18.656158447265625 network.sent‚ 5.766268e+06) disk./.usagePercent‚83.43680510663323 disk.md0p1.in‚ 0.01171875 disk.sdb.in‚0 disk.rbd0.out‚1.125 disk.nvme0n1.in‚ 63.02734375( proc.memory.availableMB‚ 1131.5546875 network.recv‚ 1.40049e+06% disk./.usageGB‚2984.1042976379395 disk.sdb1.out‚0 disk.sda.in‚ 0.01171875 disk.rbd0.in‚0 disk.md0.out‚ 392.59765625# disk.nvme0n1.out‚28298.55859375 disk.dm-0.out‚ 57888.7890625 proc.memory.rssMB‚3056.625 disk.sda2.out‚ 409.494140625 disk.sdb2.in‚0 disk.sdb2.out‚ 409.494140625 disk.rbd2.out‚1.125 disk.sda2.in‚ 0.01171875 proc.cpu.threads‚17 disk.md0p1.out‚ 392.59765625Õ¼’Ü‚(:‡ ©ÝèÔÀ‡‹° gpu.0.gpu‚98.0 gpu.0.memory‚18+ gpu.0.memoryAllocated‚47.54737854003906* gpu.0.memoryAllocatedBytes‚ 40842887168 gpu.0.temp‚42.0 gpu.0.powerWatts‚130.232' gpu.0.enforcedPowerLimitWatts‚400.0 gpu.0.powerPercent‚32.558 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0?kÁÈDÝ‚(:¹ ©ÝèԎ׬° disk.sda1.in‚0 disk.sda1.out‚0 network.sent‚ 5.77424e+06 disk.nvme1n1.in‚55.0625 disk.dm-0.in‚ 118.08984375 cpu‚25.485988315353662) disk./.usagePercent‚83.44030527865533 disk.md0p1.in‚ 0.01171875 disk.sda.out‚411.46337890625 disk.rbd0.in‚0 disk.sdb1.in‚0 proc.cpu.threads‚17 disk.rbd2.in‚0! disk.sda2.out‚411.46337890625 disk.nvme0n1.in‚ 63.02734375! disk.nvme0n1.out‚ 28346.453125 disk.sdb1.out‚0! disk.sdb2.out‚411.46337890625 network.recv‚ 1.40463e+06 disk.md0.out‚ 394.515625 disk.md0p1.out‚ 394.515625 disk.rbd2.out‚1.125 disk.rbd0.out‚1.125 disk.sdb.in‚0 disk.sda2.in‚ 0.01171875) proc.memory.availableMB‚ 1131.79296875 disk.md0.in‚ 0.01171875 disk.sda.in‚ 0.01171875 disk.dm-0.out‚ 58059.125$ memory_percent‚93.09208393096924 proc.memory.rssMB‚3056.625* proc.memory.percent‚18.656158447265625 disk.sdb2.in‚0$ disk./.usageGB‚2984.229480743408 disk.sdb.out‚411.46337890625! disk.nvme1n1.out‚ 29712.671875ø%~2Þ- _wandbruntime‚4190  _runtime‚4190g·úÆßÀ & lr-AdamW/pg2‚0.0008718525119503188 trainer/global_step‚2349 $ _timestamp‚1.788489391069773e+09 _runtime‚4190.171358229 _step‚187 & lr-AdamW/pg1‚0.0008718525119503188òüËàÅ $ _timestamp‚1.788489391069773e+09 _runtime‚4190.171358229 _step‚187 & lr-AdamW/pg1‚0.0008718525119503188 & lr-AdamW/pg2‚0.0008718525119503188 trainer/global_step‚2349»ÛA·2á- _wandbruntime‚4190  _runtime‚4190N™ ©ú'âôO IDeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚23682 ]IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚516216916594688 TDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 35913877504 VGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 2516363264 `LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚516216916594688 \HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚516216891166720 RCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 2539798108 P@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 36985723401 MDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚2147 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚19690 QDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚26454046 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 577553920 KBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚1534 ^LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 5632089157632 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚10458408 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 102717440 ZKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 3217031168 NFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚986 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚19684 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚25427968 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚15995638 aMDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚104305364810752 YJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 2514370140 G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚2684 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚2684 D>DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 \HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚495615371797232 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚19690 eQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚104305393908736 % _timestamp‚1.7884893910713613e+09 ZHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 5622443843660 MDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚1534 E8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚77626373 RCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 2541791232 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚582 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 A8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚3681 A8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚3681 YEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚521849005752320 IDeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚521846463961088 O?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 40290484224 ^JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚111338606499840 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 PCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚33636515 PCDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated‚44009548 TEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 1515847168 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 YEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚516216891166720 PCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚43989864 M@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚26453464 NFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚330 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚579 WGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 34283657216 UFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 3217031168 WEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 5629572794368 H?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚1309 VFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 37073453056 L?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚33636509 [IDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 5632089157632 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚25427968 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 KBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚2147 H?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚21478.c†ÿ'ãùO L?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚43989864 VBDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated‚521849005752320 WGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 35913877504 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚23981 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚15995059 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 TGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚33636515 YEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚501240355439000 UADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚501237815640892 `LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚495615397225200 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 YIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 37073453056 D;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚3681 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚23682 ]IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚516216916594688 TDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 35913877504 VGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 2516363264 `LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚516216916594688 \HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚516216891166720 RCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 2539798108 P@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 36985723401 MDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚2147 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚19690 QDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚26454046 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 577553920 KBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚1534 ^LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 5632089157632 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚10458408 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 102717440 ZKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 3217031168 NFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚986 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚19684 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚25427968 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚15995638 aMDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚104305364810752 YJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 2514370140 G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚2684 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚2684 D>DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 \HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚495615371797232 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚19690 eQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚104305393908736 % _timestamp‚1.7884893910713613e+09 ZHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 5622443843660 MDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚1534 E8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚77626373 RCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 2541791232 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚582 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 A8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚3681 A8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚3681 YEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚521849005752320 IDeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚521846463961088 O?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 40290484224 ^JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚111338606499840 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 PCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚33636515 PCDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated‚44009548 TEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 1515847168 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 YEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚516216891166720 PCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚43989864 M@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚26453464 NFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚330 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚579 WGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 34283657216 UFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 3217031168 WEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 5629572794368 H?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚1309 VFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 37073453056 L?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚33636509 [IDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 5632089157632 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚25427968 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 KBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚2147 H?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚2147 IDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 36985723401 KBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚2147 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 963120128 BDeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 2541804544 REDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚44027916 NADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚44008206 ?6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚3681 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 B9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚3681 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.cuMkrrent‚6 WHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 2516376576 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚3 UEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 35913877504 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 REDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚33650405 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 TEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 3140738488 VGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 3217031168 ^JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚516430862665216 K>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚77678321 OBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚26464982 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 RBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 35913877504 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 WCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚501448224235438 P@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 40290484224 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 C6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚77658605 K;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 38610163200 WCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚516430837237248 N>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 38610163200 G:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚77658605 LDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚986 PDeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚26464399 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 WHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 2514370348 F=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚2147 $ _timestamp‚1.788489392665651e+09 ^JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚495820912885696 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚33650399 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚25427968 YGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 5634445266944 TEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 2516376576 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚19710 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚580 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 PADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 2539798316 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚23682 UCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 5631928890368 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚23981 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 577553920 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚6 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚2684t\Na'æ‚N G:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚77678321 S?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚522062766127616 ZFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚516430837237248 ZFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚495820887457728 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚23981 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚583 _KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚104349495135744 FDeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 2541804544 REDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚44027916 NADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚44008206 ?6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚3681 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 B9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚3681 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚6 WHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 2516376576 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚3 UEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 35913877504 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 REDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚33650405 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 TEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 3140738488 VGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 3217031168 ^JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚516430862665216 K>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚77678321 OBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚26464982 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 RBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 35913877504 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 WCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚501448224235438 P@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 40290484224 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 C6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚77658605 K;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 38610163200 WCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚516430837237248 N>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 38610163200 G:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚77658605 LDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚986 PDeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚26464399 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 WHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 2514370348 F=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚2147 $ _timestamp‚1.788489392665651e+09 ^JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚495820912885696 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚33650399 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚25427968 YGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 5634445266944 TEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 2516376576 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚19710 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚580 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 PADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 2539798316 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚23682 UCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 5631928890368 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚23981 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 577553920 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚6 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚2684 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚16002208 ]KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 7036076682240 NADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚44027916 T@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚522065307932160 QBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 3142753280 REDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚16001628 aODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 7036150288384 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 ?6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚3681 I@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚2147 J=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚44008206 \JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 5634445266944 REDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚10462771 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚19716 EDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 36985723401 KBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚2147 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 963120128 BDeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚532661600115712 YJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 2516363264 L?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚44910316 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚19684 NFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚330 cQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 7180173566976 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 D>DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 YEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚532664141906944 ZHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 5747405422592 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚16321886 TEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 1515847168 ZHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 5740127802368 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 VBDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated‚532664141906944 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚23981 UFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 3217031168 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚6 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚10676708 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 XIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 3217031168 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 577553920 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 H?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚2147 H?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚1534 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚25427968 YEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚526914194693120 [IDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 5749921785856 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚577 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚25427968 TGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚44930000 A8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚3681 `LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚505891180250000 ^LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 5742642172508 F=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚3681 G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚2684 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚580 eQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚106511910158336 _runtime‚4275.291917622 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚19690 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚23682 UADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚511631282624400 YJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 2514370140 PCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚34331015 JDDeviceStatsMonitor.on_train_batch_sèÏüMÓtart/oversize_allocations.current‚0 KBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚1534 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 _step‚192 \HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚526914194693120 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 \HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚505891154822032 % _timestamp‚1.7884894761872683e+09 ^JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚113692083725312 YEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚511633822422508 TDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 40290484224 RBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 40290484224 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚6 QDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚26999171 ZFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚113691962133504 P@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 36985723401 H?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚1309 O@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 2541791232 PCDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated‚44930000 SDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 3142753280 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚25427968 IDeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚23682 IDeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚2684 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚580 eQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚106511910158336 _runtime‚4275.291917622 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚19690 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚23682 UADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚511631282624400 YJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 2514370140 PCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚34331015 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 KBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚1534 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 _step‚192 \HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚526914194693120 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 \HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚505891154822032 % _timestamp‚1.7884894761872683e+09 ^JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚113692083725312 YEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚511633822422508 TDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 40290484224 RBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 40290484224 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚6 QDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚26999171 ZFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚113691962133504 P@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 36985723401 H?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚1309 O@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 2541791232 PCDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated‚44930000 SDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 3142753280 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚25427968 IDeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚23682 IDeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚532661600115712 YJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 2516363264 L?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚44910316 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚19684 NFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚330 cQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 7180173566976 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 D>DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0ÀãcØJ2ü- _wandbruntime‚4276  _runtime‚4276T–r'ýþM TEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 3142753280 WCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚527128140763648 NADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚44928658 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚23682 K>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚27009392 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 S?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚511839151420630 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚6 YIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 37073453056 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚23981 ^JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚506096695910496 D;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚3681 LDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚330 _KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚106556011385344 cODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚106556040483328 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 ZFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚506096670482528 UEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 34283657216 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚10681074 FDeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 2541804544 \JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 5752277895168 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚577 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚25427968 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚2684 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 102704128 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚29097984 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 N>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 38610163200 K>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚79293273 XDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚113739040005120 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 WHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 2514370348 WCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚532880444086784 PADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 2541804544 WCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚511841691218946 \HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚113739142709248 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 N>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 36985723401 XFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 5742480938102 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 I@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚1534 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1309 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚6 YGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 5752277895168 S?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚532877902282240 P@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 40290484224 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 _runtime‚4276.801079296 TEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 3140738488 C6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚79273557 RCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 1515847168 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 NADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚34344899 REDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚44948368 B9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚3681 F=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚2147 PDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 QBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 3142753280 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚16328898 REDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚16328321 XIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 3217031168 XFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 5749761518592 REDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚34344905 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚34344899 UEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 35913877504 WHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 2516376576 [GDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚527128166191616 RBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 35913877504 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚3 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 577553920 WGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 37073453056 G:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚79293273 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚19710 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 M=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 40290484224 UCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 5749761518592 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚25427968 OBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚27009972 REDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚10681071 LDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚986 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 aODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 7183102225920 ]KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 7183028619776 PADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 2539798316 B8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 F8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 FDeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚27009392 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 S?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚511839151420630 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚6 YIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 37073453056 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚23981 ^JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚506096695910496 D;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚3681 LDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚330 _KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚106556011385344 cODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚106556040483328 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 ZFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚506096670482528 UEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 34283657216 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚10681074 FDeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 2541804544 \JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 5752277895168 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚577 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚25427968 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚2684 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 102704128 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚29097984 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 N>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 38610163200 K>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚79293273 XDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚113739040005120 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 WHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 2514370348 WCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚532880444086784 PADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 2541804544 WCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚511841691218946 \HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚113739142709248 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 N>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 36985723401 XFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 5742480938102 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 I@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚1534 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1309 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚6 YGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 5752277895168 S?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚532877902282240 P@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 40290484224 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 _runtime‚4276.801079296 TEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 3140738488 C6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚79273557 RCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 1515847168 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 NADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚34344899 REDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚44948368 B9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚3681 F=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚2147 PDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 QBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 3142753280 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚16328898 REDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚16328321 XIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 3217031168 XFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 5749761518592 REDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚34344905 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚34344899 UEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 35913877504 WHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 2516376576 [GDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚527128166191616 RBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 35913877504 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚3 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 577553920 WGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 37073453056Á—A2ÿ- _wandbruntime‚4276  _runtime‚4276WÌK¬]€× /loss/FRt4/assignment_loss‚2.524166872104009 2loss/FBt1/assignment_loss‚0.051358905770549815 1loss/FBt2/assignment_loss‚0.08793562895541872 -loss/FRt4/detection_loss‚2.75764217807187 /loss/FBt4/detection_loss‚1.9586550508226668 1loss/SRqqt3/detection_loss‚2.3956202758896734 /loss/FBt3/detection_loss‚2.7990688869827673 trainer/global_step‚2399 % _timestamp‚1.7884894777009096e+09 /loss/FRt1/detection_loss‚2.2282467455940043 0loss/FRt2/assignment_loss‚1.4780856300425786 0loss/SRqqt2/assignment_loss‚1.00750530573229 2loss/SRqqt4/assignment_loss‚1.7080780035980485 _runtime‚ 4276.80204819 0loss/FRt1/assignment_loss‚3.4542560089141765 0loss/FRt3/assignment_loss‚3.2075457766458584 1loss/SRqqt1/detection_loss‚2.2065554563678913 /loss/FBt1/detection_loss‚3.0466682485171726 epoch‚3 1loss/SRqqt4/detection_loss‚1.5720935294075924 _step‚194 2loss/SRqqt3/assignment_loss‚1.6008545675825687 0loss/FBt4/assignment_loss‚0.1089412301017936 2loss/SRqqt1/assignment_loss‚1.7301236306553456 1loss/FBt3/assignment_loss‚0.08259892532958857 0loss/SRqqt2/detection_loss‚0.910895825647716 &loss/total_loss‚41.573229889096204 .loss/FRt2/detection_loss‚1.572169461557942 /loss/FRt3/detection_loss‚2.5898599535613864 0loss/FBt2/detection_loss‚0.49430379124330076–êpRbÜ /loss/FBt1/detection_loss‚3.0466682485171726 epoch‚3 1loss/SRqqt4/detection_loss‚1.5720935294075924 _step‚194 2loss/SRqqt3/assignment_loss‚1.6008545675825687 0loss/FBt4/assignment_loss‚0.1089412301017936 2loss/SRqqt1/assignment_loss‚1.7301236306553456 1loss/FBt3/assignment_loss‚0.08259892532958857 0loss/SRqqt2/detection_loss‚0.910895825647716 &loss/total_loss‚41.573229889096204 .loss/FRt2/detection_loss‚1.572169461557942 /loss/FRt3/detection_loss‚2.5898599535613864 0loss/FBt2/detection_loss‚0.49430379124330076 /loss/FRt4/assignment_loss‚2.524166872104009 2loss/FBt1/assignment_loss‚0.051358905770549815 1loss/FBt2/assignment_loss‚0.08793562895541872 -loss/FRt4/detection_loss‚2.75764217807187 /loss/FBt4/detection_loss‚1.9586550508226668 1loss/SRqqt3/detection_loss‚2.3956202758896734 /loss/FBt3/detection_loss‚2.7990688869827673 trainer/global_step‚2399 % _timestamp‚1.7884894777009096e+09 /loss/FRt1/detection_loss‚2.2282467455940043 0loss/FRt2/assignment_loss‚1.4780856300425786 0loss/SRqqt2/assignment_loss‚1.00750530573229 2loss/SRqqt4/assignment_loss‚1.7080780035980485 _runtime‚ 4276.80204819 0loss/FRt1/assignment_loss‚3.4542560089141765 0loss/FRt3/assignment_loss‚3.2075457766458584 1loss/SRqqt1/detection_loss‚2.2065554563678913Â;£e)ž‚‚(:“ ’ÞèÔø®’° gpu.0.gpu‚98.0 gpu.0.memory‚17+ gpu.0.memoryAllocated‚47.54737854003906* gpu.0.memoryAllocatedBytes‚ 40842887168 gpu.0.temp‚45.0 gpu.0.powerWatts‚224.584' gpu.0.enforcedPowerLimitWatts‚400.0) gpu.0.powerPercent‚56.145999999999994 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0ˆ±#Eƒ‚(:º ’ÞèÔöŒ¾± disk.sda.in‚ 0.01171875 disk.dm-0.in‚ 118.08984375 disk.sdb1.in‚0 disk.sda1.out‚0 proc.memory.rssMB‚3056.625 cpu‚25.48156408992591 network.sent‚ 5.893544e+06 disk.nvme0n1.in‚ 63.02734375 disk.md0.in‚ 0.01171875 disk.rbd2.in‚0 disk.sdb.out‚ 422.373046875 proc.cpu.threads‚17% disk./.usageGB‚2984.3545455932617# disk.nvme1n1.out‚30570.38671875# memory_percent‚93.0943250656128 disk.sda1.in‚0 disk.md0.out‚ 404.99609375 disk.nvme1n1.in‚55.0625 disk.rbd0.out‚1.125 disk.dm-0.out‚59437.01171875 disk.sdb2.in‚0 disk.sdb2.out‚ 422.373046875) proc.memory.availableMB‚ 1131.42578125) disk./.usagePercent‚83.44380214420094 disk.sda.out‚ 422.373046875 disk.md0p1.out‚ 404.99609375 disk.rbd2.out‚1.125 disk.sdb.in‚0 disk.rbd0.in‚0 disk.nvme0n1.out‚ 28866.625 disk.sdb1.out‚0 disk.md0p1.in‚ 0.01171875 disk.sda2.in‚ 0.01171875 disk.sda2.out‚ 422.373046875* proc.memory.percent‚18.656158447265625 network.recv‚ 1.436465e+06Ïòè“„‚(:ˆ ¡ÞèÔ¨òˆ° gpu.0.gpu‚43.0 gpu.0.memory‚9+ gpu.0.memoryAllocated‚47.54737854003906* gpu.0.memoryAllocatedBytes‚ 40842887168 gpu.0.temp‚45.0 gpu.0.powerWatts‚317.567' gpu.0.enforcedPowerLimitWatts‚400.0 gpu.0.powerPercent‚79.39175 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0VQtG…‚(:¼ ¡ÞèÔܽ¦±% disk./.usageGB‚2984.3545455932617 disk.nvme0n1.in‚ 63.02734375 disk.md0.in‚ 0.01171875 disk.md0.out‚ 406.6484375 disk.sdb.in‚0" disk.nvme0n1.out‚ 28937.4453125 disk.dm-0.in‚ 118.08984375 disk.sdb2.out‚ 424.095703125 disk.sda1.out‚0 disk.sdb.out‚ 424.095703125 disk.sda.in‚ 0.01171875 disk.rbd0.out‚1.125) proc.memory.availableMB‚ 1131.19140625 disk.dm-0.out‚59630.26953125 disk.sdb1.in‚0 disk.rbd2.out‚1.125 disk.sda.out‚ 424.095703125 cpu‚25.48301593765559 proc.cpu.threads‚17 network.recv‚ 1.443282e+06 disk.sdb1.out‚0 proc.memory.rssMB‚3056.625 disk.md0p1.in‚ 0.01171875 disk.sda2.out‚ 424.095703125* proc.memory.percent‚18.656158447265625 network.sent‚ 5.980183e+06 disk.nvme1n1.in‚55.0625# disk.nvme1n1.out‚30692.82421875 disk.sdb2.in‚0 disk.md0p1.out‚ 406.6484375 disk.rbd0.in‚0# memory_percent‚93.0957555770874) disk./.usagePercent‚83.44380214420094 disk.sda1.in‚0 disk.rbd2.in‚0 disk.sda2.in‚ 0.01171875<ÄiW‘†‚(:† °ÞèÔÊ£Ù® gpu.0.gpu‚0.0 gpu.0.memory‚0+ gpu.0.memoryAllocated‚47.54737854003906* gpu.0.memoryAllocatedBytes‚ 40842887168 gpu.0.temp‚39.0 gpu.0.powerWatts‚115.966' gpu.0.enforcedPowerLimitWatts‚400.0 gpu.0.powerPercent‚28.9915 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0¶C­A‡‚(:¶ °ÞèÔæÙ–¯ disk.sda2.out‚ 425.3359375* proc.memory.percent‚18.651962280273438 proc.cpu.threads‚17" disk.nvme0n1.out‚ 29098.7578125 disk.dm-0.in‚ 118.08984375 disk.sdb2.out‚ 425.3359375 disk.md0p1.in‚ 0.01171875 disk.sdb.out‚ 425.3359375 cpu‚25.501111313300726 disk.nvme0n1.in‚ 63.02734375 disk.dm-0.out‚59839.07421875 disk.sdb1.out‚0 disk.sda1.out‚0 disk.rbd2.out‚1.125 network.recv‚ 1.447931e+06 disk.md0.in‚ 0.01171875) disk./.usagePercent‚83.44380353078786$ disk./.usageGB‚2984.354595184326 disk.sdb1.in‚0 disk.md0.out‚ 407.83984375 disk.sdb.in‚0$ memory_percent‚93.09499263763428 disk.sdb2.in‚0 disk.nvme1n1.in‚55.0625 proc.memory.rssMB‚ 3055.9375 network.sent‚ 5.9864e+06 disk.rbd0.in‚0 disk.rbd0.out‚1.125 disk.sda1.in‚0 disk.md0p1.out‚ 407.83984375 disk.rbd2.in‚0) proc.memory.availableMB‚ 1131.31640625 disk.sda.in‚ 0.01171875 disk.sda2.in‚ 0.01171875# disk.nvme1n1.out‚30753.69140625 disk.sda.out‚ 425.33593759l3'’ˆ‚(:‡ ¿ÞèÔ칇¯ gpu.0.gpu‚91.0 gpu.0.memory‚19+ gpu.0.memoryAllocated‚47.54737854003906* gpu.0.memoryAllocatedBytes‚ 40842887168 gpu.0.temp‚46.0 gpu.0.powerWatts‚240.384' gpu.0.enforcedPowerLimitWatts‚400.0 gpu.0.powerPercent‚60.096 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0‚gZG‰‚(:¼ ¿ÞèÔÉßí¯) disk./.usagePercent‚83.43604899012101 disk.md0.out‚ 409.62890625 disk.sdb2.out‚ 427.19140625 disk.md0.in‚ 0.01171875" disk.nvme1n1.out‚ 30815.7265625 disk.sda.out‚ 427.19140625% disk./.usageGB‚2984.0772552490234 disk.sda1.in‚0 disk.md0p1.out‚ 409.62890625 disk.rbd2.in‚0 disk.dm-0.out‚59953.21484375 network.recv‚ 1.451545e+06 disk.sdb.out‚ 427.19140625 disk.sdb2.in‚0 proc.memory.rssMB‚ 3055.9375 proc.cpu.threads‚17 disk.sdb.in‚0 disk.sda2.out‚ 427.19140625$ memory_percent‚93.09284687042236* proc.memory.percent‚18.651962280273438 disk.sdb1.out‚0 network.sent‚ 5.991902e+06 disk.sda1.out‚0 disk.rbd2.out‚1.125 disk.nvme0n1.in‚ 63.02734375 disk.dm-0.in‚ 118.08984375) proc.memory.availableMB‚ 1131.66796875 disk.nvme1n1.in‚55.0625 disk.rbd0.in‚0 disk.rbd0.out‚1.125# disk.nvme0n1.out‚29137.48828125 disk.sdb1.in‚0 disk.md0p1.in‚ 0.01171875 disk.sda2.in‚ 0.01171875 disk.sda.in‚ 0.01171875 cpu‚25.51709827688866˜ð<“Š‚(:ˆ ÎÞèÔ¢›® gpu.0.gpu‚89.0 gpu.0.memory‚19+ gpu.0.memoryAllocated‚47.54737854003906* gpu.0.memoryAllocatedBytes‚ 40842887168 gpu.0.temp‚47.0 gpu.0.powerWatts‚316.618' gpu.0.enforcedPowerLimitWatts‚400.0 gpu.0.powerPercent‚79.1545 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0á 0_R‹‚(:Ç ÎÞèÔ’ìÝ® disk.nvme1n1.in‚55.0625 disk.sda.out‚428.5380859375 cpu‚25.531968836932357 disk.rbd0.out‚1.125 proc.cpu.threads‚17 network.sent‚ 5.997578e+06# disk.nvme0n1.out‚29137.59765625 disk.sda1.in‚0$ memory_percent‚93.11575889587402* proc.memory.percent‚18.662476539611816 disk.sdb1.out‚0 disk.md0p1.in‚ 0.01171875" disk.nvme1n1.out‚ 31005.4921875( proc.memory.availableMB‚ 1127.9140625 disk.sdb1.in‚0 disk.nvme0n1.in‚ 63.02734375# proc.memory.rssMB‚ 3057.66015625 network.recv‚ 1.455732e+06$ disk./.usageGB‚2984.102752685547 disk.sdb2.in‚0 disk.sdb.out‚428.5380859375 disk.sda2.out‚428.5380859375 disk.rbd0.in‚0) disk./.usagePercent‚83.43676190911772 disk.dm-0.in‚ 118.08984375 disk.dm-0.out‚60143.08984375 disk.md0.in‚ 0.01171875 disk.rbd2.in‚0 disk.sdb.in‚0 disk.sda2.in‚ 0.01171875 disk.sda.in‚ 0.01171875 disk.sda1.out‚0 disk.sdb2.out‚428.5380859375 disk.md0.out‚ 410.91796875 disk.md0p1.out‚ 410.91796875 disk.rbd2.out‚1.125†Ô,ƒ2Œ- _wandbruntime‚4352  _runtime‚4352­²ëGìPè¦ particle/accuracy_2_of_7‚1 9"Purity/*FRt1SRqqt3FBt/SRqqt_purity‚0.8333333134651184 7 Purity/1FRt0SRqqt2FBt/FBt_purity‚0.9828657507896423 9"Purity/*FRt*SRqqt*FBt/SRqqt_purity‚0.3158181309700012 >&Purity/*FRt*SRqqt1FBt/event_proportion‚0.27079132199287415 9"Purity/1FRt*SRqqt0FBt/SRqqt_purity‚0.2611111104488373 ,&Purity/3FRt0SRqqt4FBt/event_proportion‚0 ,&Purity/4FRt0SRqqt3FBt/event_proportion‚0 8 Purity/2FRt3SRqqt1FBt/FRt_purity‚0.30000001192092896 ("Purity/0FRt1SRqqt3FBt/event_purity‚0 @&Purity/0FRt2SRqqt2FBt/event_proportion‚0.0010080644860863686 9"Purity/3FRt2SRqqt*FBt/SRqqt_purity‚0.3107638955116272 ,&Purity/4FRt3SRqqt1FBt/event_proportion‚0 :"Purity/0FRt3SRqqt*FBt/SRqqt_purity‚0.21994520723819733 ("Purity/2FRt*SRqqt3FBt/event_purity‚0 9"Purity/*FRt1SRqqt0FBt/event_purity‚0.1466256082057953 @&Purity/*FRt1SRqqt4FBt/event_proportion‚6.300403038039804e-05 :"Purity/*FRt2SRqqt*FBt/SRqqt_purity‚0.29357510805130005 ?&Purity/2FRt2SRqqt*FBt/event_proportion‚0.019468246027827263 8 Purity/3FRt1SRqqt0FBt/FRt_purity‚0.30972224473953247 @&Purity/4FRt*SRqqt0FBt/event_proportion‚0.0001890120911411941 ("Purity/1FRt*SRqqt3FBt/SRqqt_purity‚1 & Purity/4FRt0SRqqt*FBt/FRt_purity‚0 7 Purity/1FRt0SRqqt*FBt/FBt_purity‚0.9899824261665344 ;"Purity/2FRt*SRqqt1FBt/event_purity‚0.035914383828639984 ("Purity/2FRt1SRqqt0FBt/event_purity‚0 ?&Purity/2FRt3SRqqt*FBt/event_proportion‚0.004032257944345474 ,&Purity/3FRt*SRqqt3FBt/event_proportion‚0 @&Purity/3FRt0SRqqt1FBt/event_proportion‚0.0011340725468471646 ,jet/accuracy_4_of_5‚0.008216156624257565 particle/accuracy_1_of_7‚1 =&Purity/*FRt*SRqqt0FBt/event_proportion‚0.6160534024238586 8 Purity/*FRt3SRqqt*FBt/FRt_purity‚0.32461172342300415 7 Purity/1FRt2SRqqt1FBt/FBt_purity‚0.9919354915618896 7 Purity/2FRt*SRqqt2FBt/FRt_purity‚0.4747222363948822 @&Purity/3FRt*SRqqt0FBt/event_proportion‚0.0071824598126113415 particle/accuracy_5_of_5‚1 6 Purity/*FRt4SRqqt0FBt/FRt_purity‚0.277777761220932 * Purity/1FRt*SRqqt4FBt/FBt_purity‚0.875 ,&Purity/*FRt4SRqqt4FBt/event_proportion‚0 ?&Purity/3FRt*SRqqt*FBt/event_proportion‚0.009954636916518211 ("Purity/2FRt0SRqqt3FBt/event_purity‚0 ("Purity/4FRt*SRqqt*FBt/event_purity‚0 =&Purity/0FRt1SRqqt0FBt/event_proportion‚0.1694808453321457 7 Purity/0FRt1SRqqt1FBt/FBt_purity‚0.9955925345420837 9"Purity/1FRt1SRqqt*FBt/SRqqt_purity‚0.3469739556312561 >&Purity/1FRt2SRqqt*FBt/event_proportion‚0.05727066472172737 8 Purity/1FRt2SRqqt1FBt/FRt_purity‚0.25944700837135315 7 Purity/1FRt3SRqqt*FBt/FRt_purity‚0.3181643784046173 7 Purity/1FRt4SRqqt0FBt/FRt_purity‚0.1666666716337204 :"Purity/*FRt0SRqqt0FBt/event_purity‚0.19419921934604645 7 Purity/*FRt2SRqqt2FBt/FRt_purity‚0.3974359333515167 ("Purity/0FRt4SRqqt0FBt/event_purity‚0 ("Purity/1FRt*SRqqt4FBt/event_purity‚0 ,&Purity/4FRt4SRqqt4FBt/event_proportion‚0 7 Purity/*FRt0SRqqt0FBt/FRt_purity‚0.2335006296634674 9"Purity/0FRt*SRqqt*FBt/SRqqt_purity‚0.3162253499031067 ,&Purity/0FRt4SRqqt4FBt/event_proportion‚0 7 Purity/1FRt*SRqqt1FBt/FBt_purity‚0.9966728091239929 8 Purity/2FRt0SRqqt*FBt/FRt_purity‚0.30303895473480225 +particle/specificity‚0.9471937417984009 & Purity/0FRt4SRqqt*FBt/FBt_purity‚1 ("Purity/0FRt*SRqqt4FBt/event_purity‚0 & Purity/3FRt2SRqqt*FBt/FBt_purity‚1 & Purity/2FRt3SRqqt1FBt/FBt_purity‚1 +"Purity/1FRt4SRqqt*FBt/SRqqt_purity‚0.25 :"Purity/2FRt*SRqqt*FBt/SRqqt_purity‚0.31391891837120056 :"Purity/2FRt1SRqqt*FBt/SRqqt_purity‚0.34586408734321594 ("Purity/2FRt2SRqqt0FBt/event_purity‚0 @&Purity/3FRt0SRqqt*FBt/event_proportion‚0.0027091733645647764 ,&Purity/3FRt1SRqqt3FBt/event_proportion‚0 ,&Purity/4FRt1SRqqt1FBt/event_proportion‚0 ,&Purity/4FRt4SRqqt0FBt/event_proportion‚0 7validation_average_jet_accuracy‚0.38158681988716125 7 Purity/2FRt2SRqqt0FBt/FRt_purity‚0.3471948504447937 ,&Purity/2FRt3SRqqt3FBt/event_proportion‚0 ?&Purity/*FRt3SRqqt*FBt/event_proportion‚0.020476309582591057 9"Purity/2FRt*SRqqt0FBt/SRqqt_purity‚0.2679072618484497 jet/accuracy_5_of_5‚0 ?&Purity/0FRt2SRqqt1FBt/event_proportion‚0.009261592291295528 , Purity/1FRt*SRqqt3FBt/FRt_purity‚0.59375 & Purity/*FRt2SRqqt2FBt/FBt_purity‚1 ("Purity/3FRt1SRqqt2FBt/event_purity‚0 ("Purity/1FRt4SRqqt0FBt/event_purity‚0 6 Purity/*FRt1SRqqt0FBt/FRt_purity‚0.319568008184433 ("Purity/1FRt1SRqqt4FBt/SRqqt_purity‚1 :"Purity/*FRt*SRqqt2FBt/event_purity‚0.17083577811717987 9"Purity/*FRt*SRqqt*FBt/event_purity‚0.2130253165960312 9"Purity/1FRt*SRqqt2FBt/event_purity‚0.0713355615735054 :"Purity/2FRt*SRqqt1FBt/SRqqt_purity‚0.46686428785324097 ,&Purity/4FRt3SRqqt4FBt/event_proportion‚0 particle/accuracy_3_of_4‚1 7 Purity/3FRt3SRqqt*FBt/FRt_purity‚0.3809523582458496 ("Purity/0FRt2SRqqt3FBt/event_purity‚0 7 Purity/*FRt*SRqqt0FBt/FRt_purity‚0.2907537817955017 7 Purity/*FRt2SRqqt*FBt/FBt_purity‚0.9909133911132812 >&Purity/0FRt1SRqqt1FBt/event_proportion‚0.05185231938958168 ("Purity/3FRt2SRqqt*FBt/event_purity‚0 @&Purity/3FRt2SRqqt1FBt/event_proportion‚0.0003780241822823882 ("Purity/0FRt3SRqqt*FBt/event_purity‚0 ;"Purity/*FRt2SRqqt*FBt/event_purity‚0.010293247178196907 7 Purity/2FRt1SRqqt0FBt/FRt_purity‚0.2812124192714691 @&Purity/2FRt3SRqqt1FBt/event_proportion‚0.0003780241822823882 ,&Purity/4FRt3SRqqt2FBt/event_proportion‚0 *"Purity/3FRt1SRqqt2FBt/SRqqt_purity‚0.5 =&Purity/*FRt1SRqqt1FBt/event_proportion‚0.1001134067773819 >&Purity/0FRt*SRqqt2FBt/event_proportion‚0.03944052383303642 ?&Purity/1FRt1SRqqt2FBt/event_proportion‚0.006111391354352236 @&Purity/*FRt4SRqqt1FBt/event_proportion‚6.300403038039804e-05 9"Purity/*FRt*SRqqt2FBt/SRqqt_purity‚0.5536362528800964 :"Purity/*FRt*SRqqt0FBt/SRqqt_purity‚0.26327845454216003 :"Purity/*FRt3SRqqt1FBt/SRqqt_purity‚0.31878307461738586 9"Purity/1FRt*SRqqt2FBt/SRqqt_purity‚0.5547106862068176 8 Purity/1FRt1SRqqt0FBt/FRt_purity‚0.34041136503219604 7 Purity/3FRt0SRqqt0FBt/FRt_purity‚0.1666666716337204 epoch‚3 7 Purity/0FRt0SRqqt1FBt/FBt_purity‚0.9990224242210388 :"Purity/0FRt2SRqqt0FBt/SRqqt_purity‚0.24866622686386108 7 Purity/1FRt1SRqqt1FBt/FBt_purity‚0.9967661499977112 7 Purity/2FRt0SRqqt2FBt/FRt_purity‚0.5202381014823914 & Purity/3FRt0SRqqt1FBt/FBt_purity‚1 ("Purity/2FRt4SRqqt0FBt/event_purity‚0 >&Purity/*FRt2SRqqt0FBt/event_proportion‚0.10855595022439957 9"Purity/1FRt0SRqqt1FBt/event_purity‚0.2820163071155548 :"Purity/*FRt3SRqqt0FBt/SRqqt_purity‚0.21077412366867065 :"Purity/0FRt2SRqqt1FBt/event_purity‚0.03870967775583267 & Purity/2FRt1SRqqt1FBt/FBt_purity‚1 9"Purity/3FRt*SRqqt*FBt/SRqqt_purity‚0.3228106200695038 0"Purity/*FRt4SRqqt*FBt/SRqqt_purity‚ 0.2421875 & Purity/0FRt3SRqqt1FBt/FBt_purity‚1 7 Purity/*FRt*SRqqt1FBt/FBt_purity‚0.9972519278526306 ;"Purity/*FRt2SRqqt0FBt/event_purity‚0.010600889101624489 :"Purity/0FRt*SRqqt3FBt/event_purity‚0.16049382090568542 >&Purity/2FRt*SRqqt0FBt/event_proportion‚0.07579384744167328 ("Purity/0FRt3SRqqt0FBt/event_purity‚0 7 Purity/0FRt2SRqqt3FBt/FBt_purity‚0.6666666865348816 @&Purity/*FRt4SRqqt*FBt/event_proportion‚0.0013230846961960196 >&Purity/0FRt2SRqqt*FBt/event_proportion‚0.05361643061041832 ?&Purity/0FRt2SRqqt0FBt/event_proportion‚0.043283771723508835 ?&Purity/0FRt3SRqqt*FBt/event_proportion‚0.006741431541740894 ?&Purity/1FRt0SRqqt1FBt/event_proportion‚0.049773186445236206 ,&Purity/1FRt4SRqqt3FBt/event_proportion‚0 ,&Purity/2FRt*SRqqt4FBt/event_proportion‚0 jet/accuracy_6_of_7‚0 7 Purity/*FRt1SRqqt1FBt/FBt_purity‚0.9963663220405579 ("Purity/*FRt4SRqqt0FBt/event_purity‚0 particle/accuracy_4_of_6‚1 & Purity/1FRt3SRqqt2FBt/FRt_purity‚1 ("Purity/4FRt0SRqqt0FBt/event_purity‚0 & Purity/4FRt2SRqqt0FBt/FRt_purity‚0 7 Purity/*FRt0SRqqt3FBt/FRt_purity‚0.5666666626930237 8 Purity/*FRt2SRqqt*FBt/FRt_purity‚0.32429733872413635 ("Purity/1FRt3SRqqt*FBt/event_purity‚0 9"Purity/2FRt0SRqqt2FBt/event_purity‚0.0535714291036129 -"Purity/0FRt4SRqqt*FBt/SRqqt_purity‚0.3125 7 Purity/0FRt*SRqqt3FBt/FBt_purity‚0.9032922387123108 A&Purity/1FRt*SRqqt4FBt/event_proportion‚0.00012600806076079607 :"Purity/1FRt0SRqqt2FBt/event_purity‚0.10211760550737381 @&Purity/1FRt4SRqqt0FBt/event_proportion‚0.0003780241822823882 @&Purity/3FRt1SRqqt0FBt/event_proportion‚0.0025201612152159214 ("Purity/2FRt4SRqqt*FBt/event_purity‚0 jet/accuracy_4_of_6‚0 & Purity/2FRt2SRqqt1FBt/FBt_purity‚1 ,&Purity/2FRt4SRqqt3FBt/event_proportion‚0 ,&Purity/3FRt4SRqqt2FBt/event_proportion‚0 ,&Purity/3FRt4SRqqt3FBt/event_proportion‚0 + Purity/0FRt*SRqqt4FBt/FBt_purity‚0.8125 ( Purity/*FRt1SRqqt3FBt/FRt_purity‚0.5 :"Purity/*FRt1SRqqt2FBt/event_purity‚0.05198215693235397 :"Purity/0FRt*SRqqt0FBt/event_purity‚0.22062766551971436 :"Purity/1FRt0SRqqt0FBt/event_purity‚0.22988179326057434 9"Purity/*FRt3SRqqt*FBt/SRqqt_purity‚0.2152818888425827 >&Purity/0FRt0SRqqt0FBt/event_proportion‚0.04082661122083664 ?&Purity/0FRt1SRqqt2FBt/event_proportion‚0.008757560513913631 :"Purity/1FRt*SRqqt0FBt/event_purity‚0.10019713640213013 @&Purity/2FRt*SRqqt3FBt/event_proportion‚6.300403038039804e-05 ,&Purity/2FRt1SRqqt3FBt/event_proportion‚0 7 Purity/2FRt3SRqqt0FBt/FRt_purity‚0.4046536684036255 @&Purity/4FRt*SRqqt*FBt/event_proportion‚0.0001890120911411941 >&Purity/0FRt*SRqqt*FBt/event_proportion‚0.40996724367141724 7 Purity/*FRt0SRqqt1FBt/FBt_purity‚0.9987428784370422 9"Purity/0FRt*SRqqt2FBt/SRqqt_purity‚0.5051851868629456 9"Purity/0FRt0SRqqt3FBt/event_purity‚0.2266666740179062 7 Purity/1FRt*SRqqt0FBt/FRt_purity‚0.2937387228012085 9"Purity/1FRt1SRqqt1FBt/SRqqt_purity‚0.5478251576423645 & Purity/1FRt3SRqqt*FBt/FBt_purity‚1 7 Purity/2FRt0SRqqt2FBt/FBt_purity‚0.9940475821495056 @&Purity/*FRt*SRqqt4FBt/event_proportion‚0.0003780241822823882 9"Purity/*FRt1SRqqt0FBt/SRqqt_purity‚0.2733638882637024 @&Purity/2FRt2SRqqt2FBt/event_proportion‚6.300403038039804e-05 7 Purity/3FRt1SRqqt1FBt/FRt_purity‚0.2820512652397156 +jet/accuracy_2_of_3‚0.22665588557720184 particle/accuracy_2_of_2‚1 jet/accuracy_1_of_7‚1 9"Purity/*FRt1SRqqt2FBt/SRqqt_purity‚0.5963643789291382 >&Purity/*FRt2SRqqt1FBt/event_proportion‚0.02255544438958168 ?&Purity/2FRt*SRqqt2FBt/event_proportion‚0.005418346729129553 ,&Purity/3FRt3SRqqt3FBt/event_proportion‚0 ,&Purity/4FRt*SRqqt2FBt/event_proportion‚0 ,&Purity/4FRt1SRqqt2FBt/event_proportion‚0 particle/accuracy_1_of_3‚1 ( Purity/1FRt1SRqqt3FBt/FRt_purity‚0.5 :"Purity/*FRt*SRqqt1FBt/event_purity‚0.41315343976020813 :"Purity/*FRt*SRqqt3FBt/event_purity‚0.12068965286016464 7 Purity/*FRt*SRqqt3FBt/FRt_purity‚0.5882353186607361 ?&Purity/0FRt3SRqqt0FBt/event_proportion‚0.006111391354352236 @&Purity/0FRt4SRqqt*FBt/event_proportion‚0.0005040322430431843 ,&Purity/1FRt2SRqqt4FBt/event_proportion‚0 ("Purity/1FRt3SRqqt1FBt/event_purity‚0 7 Purity/2FRt0SRqqt*FBt/FBt_purity‚0.9967741966247559 :"Purity/0FRt*SRqqt2FBt/event_purity‚0.24558140337467194 9"Purity/2FRt1SRqqt0FBt/SRqqt_purity‚0.2741154730319977 9"Purity/3FRt*SRqqt1FBt/SRqqt_purity‚0.6555556058883667 8 Purity/3FRt2SRqqt*FBt/FRt_purity‚0.31481483578681946 ("Purity/3FRt0SRqqt0FBt/event_purity‚0 ,&Purity/4FRt2SRqqt1FBt/event_proportion‚0 jet/accuracy_2_of_7‚0 & Purity/1FRt3SRqqt2FBt/FBt_purity‚1 8 Purity/1FRt2SRqqt0FBt/FRt_purity‚0.33670440316200256 particle/accuracy_1_of_4‚1 7 Purity/*FRt0SRqqt4FBt/FBt_purity‚0.8500000238418579 & Purity/*FRt4SRqqt1FBt/FBt_purity‚1 ("Purity/1FRt*SRqqt4FBt/SRqqt_purity‚1 & Purity/*FRt3SRqqt2FBt/FBt_purity‚1 9"Purity/0FRt0SRqqt1FBt/event_purity‚0.9990224242210388 +jet/accuracy_3_of_4‚0.06460268050432205 7 Purity/*FRt1SRqqt3FBt/FBt_purity‚0.8730158805847168 :"Purity/0FRt0SRqqt2FBt/event_purity‚0.29995962977409363 ,&Purity/2FRt4SRqqt2FBt/event_proportion‚0 ("Purity/3FRt*SRqqt*FBt/event_purity‚0 & Purity/3FRt1SRqqt1FBt/FBt_purity‚1 *jet/accuracy_1_of_3‚0.8125315308570862 ("Purity/*FRt4SRqqt*FBt/event_purity‚0 ("Purity/0FRt2SRqqt2FBt/event_purity‚0 =&Purity/*FRt*SRqqt*FBt/event_proportion‚0.9591733813285828 7 Purity/0FRt*SRqqt1FBt/FBt_purity‚0.9969891309738159 +jet/accuracy_2_of_2‚0.09841278940439224 (particle/f_score‚0.36029088497161865 9"Purity/0FRt4SRqqt0FBt/SRqqt_purity‚0.3214285671710968 7 Purity/0FRt0SRqqt3FBt/FBt_purity‚0.9177777767181396 8 Purity/1FRt0SRqqt*FBt/FRt_purity‚0.31120651960372925 ?&Purity/2FRt*SRqqt1FBt/event_proportion‚0.021169355139136314 :"Purity/2FRt0SRqqt*FBt/event_purity‚0.03291160240769386 ,&Purity/4FRt*SRqqt3FBt/event_proportion‚0 +jet/accuracy_3_of_5‚0.08627788722515106 7 Purity/1FRt1SRqqt3FBt/FBt_purity‚0.8333333730697632 7 Purity/*FRt0SRqqt1FBt/FRt_purity‚0.4371524155139923 =&Purity/*FRt1SRqqt*FBt/event_proportion‚0.4482736885547638 ?&Purity/*FRt2SRqqt2FBt/event_proportion‚0.002079133177176118 & Purity/*FRt3SRqqt*FBt/FBt_purity‚1 =&Purity/0FRt1SRqqt*FBt/event_proportion‚0.2305317521095276 @&Purity/4FRt2SRqqt*FBt/event_proportion‚6.300403038039804e-05 :"Purity/1FRt3SRqqt*FBt/SRqqt_purity‚0.23582948744297028 7 Purity/2FRt2SRqqt1FBt/FRt_purity‚0.3422618806362152 ("Purity/0FRt4SRqqt*FBt/event_purity‚0 particle/accuracy_3_of_6‚1 :"Purity/0FRt2SRqqt2FBt/SRqqt_purity‚0.32692307233810425 7 Purity/3FRt*SRqqt2FBt/FRt_purity‚0.2666666805744171 7 Purity/*FRt*SRqqt3FBt/FBt_purity‚0.9114943146705627 :"Purity/0FRt2SRqqt1FBt/SRqqt_purity‚0.46776753664016724 7 Purity/0FRt2SRqqt1FBt/FBt_purity‚0.9792627096176147 @&Purity/1FRt0SRqqt4FBt/event_proportion‚6.300403038039804e-05 A&Purity/1FRt1SRqqt3FBt/event_proportion‚0.00025201612152159214 :"Purity/2FRt2SRqqt0FBt/SRqqt_purity‚0.28153133392333984 :"Purity/2FRt4SRqqt*FBt/SRqqt_purity‚0.15000000596046448 @&Purity/0FRt4SRqqt1FBt/event_proportion‚6.300403038039804e-05 ?&Purity/1FRt3SRqqt*FBt/event_proportion‚0.009135584346950054 & Purity/3FRt*SRqqt*FBt/FBt_purity‚1 particle/accuracy_2_of_4‚1 7 Purity/*FRt*SRqqt4FBt/FBt_purity‚0.8333333134651184 7 Purity/3FRt4SRqqt*FBt/FRt_purity‚0.3333333432674408 +"Purity/1FRt4SRqqt0FBt/SRqqt_purity‚0.25 ("Purity/1FRt1SRqqt4FBt/event_purity‚0 >&Purity/1FRt0SRqqt*FBt/event_proportion‚0.19745463132858276 :"Purity/2FRt*SRqqt0FBt/event_purity‚0.00605451874434948 ("Purity/*FRt2SRqqt3FBt/event_purity‚0 ("Purity/*FRt3SRqqt*FBt/event_purity‚0 A&Purity/0FRt0SRqqt4FBt/event_proportion‚0.00025201612152159214 @&Purity/1FRt4SRqqt*FBt/event_proportion‚0.0003780241822823882 ("Purity/3FRt*SRqqt1FBt/event_purity‚0 ,&Purity/4FRt3SRqqt3FBt/event_proportion‚0 ,jet/accuracy_3_of_3‚0.025819260627031326 particle/accuracy_1_of_2‚1 ("Purity/0FRt0SRqqt4FBt/event_purity‚0 9"Purity/*FRt*SRqqt1FBt/SRqqt_purity‚0.4903438985347748 7 Purity/*FRt1SRqqt1FBt/FRt_purity‚0.3371106684207916 ,&Purity/0FRt3SRqqt3FBt/event_proportion‚0 >&Purity/1FRt0SRqqt0FBt/event_proportion‚0.13142640888690948 ,&Purity/1FRt2SRqqt3FBt/event_proportion‚0 9"Purity/1FRt3SRqqt1FBt/SRqqt_purity‚0.3888889253139496 7 Purity/2FRt*SRqqt*FBt/FRt_purity‚0.3109205961227417 ,&Purity/3FRt3SRqqt1FBt/event_proportion‚0 ?&Purity/2FRt0SRqqt0FBt/event_proportion‚0.024571573361754417 7 Purity/2FRt1SRqqt1FBt/FRt_purity‚0.3477534353733063 ,&Purity/4FRt*SRqqt1FBt/event_proportion‚0 ,&Purity/4FRt2SRqqt2FBt/event_proportion‚0 *jet/accuracy_1_of_4‚0.8885867595672607 particle/accuracy_2_of_3‚1 + Purity/0FRt0SRqqt4FBt/FBt_purity‚0.8125 particle/accuracy_1_of_6‚1 8 Purity/*FRt*SRqqt1FBt/FRt_purity‚0.37970566749572754 =&Purity/1FRt*SRqqt*FBt/event_proportion‚0.4366179406642914 jet/accuracy_5_of_7‚0 & Purity/3FRt*SRqqt2FBt/FBt_purity‚1 ) Purity/4FRt*SRqqt*FBt/FRt_purity‚0.25 ("Purity/4FRt0SRqqt*FBt/event_purity‚0 9"Purity/*FRt*SRqqt0FBt/event_purity‚0.1301674097776413 ;"Purity/1FRt1SRqqt1FBt/event_purity‚0.049583740532398224 7 Purity/1FRt1SRqqt1FBt/FRt_purity‚0.3359280526638031 7 Purity/3FRt0SRqqt1FBt/FRt_purity‚0.5320513248443604 *jet/accuracy_1_of_1‚0.3986526429653168 ) Purity/*FRt1SRqqt4FBt/FBt_purity‚0.75 ) Purity/1FRt1SRqqt4FBt/FBt_purity‚0.75 & Purity/*FRt3SRqqt2FBt/FRt_purity‚1 7 Purity/*FRt0SRqqt3FBt/FBt_purity‚0.9192460775375366 @&Purity/0FRt*SRqqt3FBt/event_proportion‚0.0030871976632624865 ,&Purity/4FRt3SRqqt0FBt/event_proportion‚0 ,particle/sensitivity‚0.28248053789138794 particle/accuracy_6_of_7‚1 =&Purity/*FRt0SRqqt*FBt/event_proportion‚0.3558467626571655 7 Purity/1FRt2SRqqt*FBt/FBt_purity‚0.9919354915618896 ("Purity/1FRt3SRqqt0FBt/event_purity‚0 ,&Purity/1FRt3SRqqt3FBt/event_proportion‚0 9"Purity/2FRt*SRqqt2FBt/event_purity‚0.0277777798473835 ?&Purity/2FRt1SRqqt2FBt/event_proportion‚0.001890120911411941 ("Purity/2FRt2SRqqt1FBt/event_purity‚0 @&Purity/3FRt2SRqqt*FBt/event_proportion‚0.0028351815417408943 :"Purity/1FRt*SRqqt*FBt/event_purity‚0.11193739622831345 7 Purity/1FRt0SRqqt1FBt/FBt_purity‚0.9977647066116333 ?&Purity/1FRt0SRqqt2FBt/event_proportion‚0.015120967291295528 @&Purity/1FRt3SRqqt1FBt/event_proportion‚0.0011340725468471646 ,&Purity/3FRt3SRqqt4FBt/event_proportion‚0 ( Purity/2FRt4SRqqt0FBt/FRt_purity‚0.5 ("Purity/2FRt3SRqqt1FBt/event_purity‚0 ( Purity/4FRt1SRqqt0FBt/FRt_purity‚0.5 =&Purity/*FRt0SRqqt2FBt/event_proportion‚0.0483240932226181 9"Purity/0FRt1SRqqt1FBt/SRqqt_purity‚0.5083388686180115 :"Purity/2FRt0SRqqt1FBt/event_purity‚0.07240740954875946 @&Purity/3FRt1SRqqt1FBt/event_proportion‚0.0009450604557059705 9"Purity/0FRt3SRqqt1FBt/SRqqt_purity‚0.2916666567325592 ,&Purity/0FRt1SRqqt4FBt/event_proportion‚0 >&Purity/1FRt2SRqqt0FBt/event_proportion‚0.04731602966785431 *"Purity/3FRt*SRqqt2FBt/SRqqt_purity‚0.5 ( Purity/2FRt*SRqqt3FBt/FRt_purity‚0.5 ("Purity/*FRt*SRqqt4FBt/SRqqt_purity‚1 ("Purity/3FRt3SRqqt0FBt/event_purity‚0 ("Purity/*FRt3SRqqt2FBt/SRqqt_purity‚0 & Purity/4FRt0SRqqt0FBt/FRt_purity‚0 ;"Purity/*FRt2SRqqt1FBt/event_purity‚0.009449332021176815 9"Purity/2FRt*SRqqt2FBt/SRqqt_purity‚0.7105262875556946 ,&Purity/4FRt0SRqqt2FBt/event_proportion‚0 +"Purity/*FRt4SRqqt1FBt/SRqqt_purity‚0.25 7 Purity/3FRt2SRqqt1FBt/FRt_purity‚0.1666666716337204 7 Purity/1FRt4SRqqt*FBt/FRt_purity‚0.1666666716337204 7 Purity/*FRt3SRqqt0FBt/FRt_purity‚0.3374015986919403 @&Purity/3FRt3SRqqt0FBt/event_proportion‚0.0005670362734235823 6 Purity/*FRt4SRqqt*FBt/FRt_purity‚0.277777761220932 & Purity/1FRt2SRqqt2FBt/FBt_purity‚1 particle/accuracy_5_of_7‚1 :"Purity/*FRt1SRqqt*FBt/event_purity‚0.13473019003868103 9"Purity/0FRt*SRqqt1FBt/SRqqt_purity‚0.4831486642360687 6 Purity/3FRt1SRqqt*FBt/FRt_purity‚0.299382746219635 A&Purity/3FRt4SRqqt*FBt/event_proportion‚0.00012600806076079607 @&Purity/4FRt0SRqqt*FBt/event_proportion‚6.300403038039804e-05 ("Purity/3FRt*SRqqt2FBt/event_purity‚0 ("Purity/*FRt3SRqqt2FBt/event_purity‚0 7 Purity/*FRt2SRqqt3FBt/FBt_purity‚0.6666666865348816 ;"Purity/2FRt0SRqqt0FBt/event_purity‚0.016274018213152885 8 Purity/3FRt*SRqqt0FBt/FRt_purity‚0.29407408833503723 ("Purity/3FRt2SRqqt0FBt/event_purity‚0 ("Purity/3FRt4SRqqt0FBt/event_purity‚0 ,&Purity/*FRt4SRqqt2FBt/event_proportion‚0 ,&Purity/3FRt2SRqqt3FBt/event_proportion‚0 particle/accuracy_4_of_5‚1 *"Purity/4FRt*SRqqt0FBt/SRqqt_purity‚0.5 ("Purity/4FRt1SRqqt*FBt/SRqqt_purity‚0 @&Purity/*FRt4SRqqt0FBt/event_proportion‚0.0012600806076079607 >&Purity/0FRt*SRqqt0FBt/event_proportion‚0.21931703388690948 9"Purity/*FRt0SRqqt*FBt/event_purity‚0.4011985659599304 @&Purity/1FRt2SRqqt2FBt/event_proportion‚0.0010080644860863686 :"Purity/2FRt3SRqqt0FBt/SRqqt_purity‚0.20479798316955566 ,jet/accuracy_4_of_4‚0.006121249403804541 ("Purity/*FRt2SRqqt3FBt/SRqqt_purity‚1 ("Purity/3FRt1SRqqt1FBt/event_purity‚0 7 Purity/1FRt0SRqqt3FBt/FRt_purity‚0.5714285969734192 =&Purity/1FRt1SRqqt*FBt/event_proportion‚0.1723790317773819 >&Purity/2FRt1SRqqt0FBt/event_proportion‚0.03181703761219978 particle/accuracy_2_of_5‚1 9"Purity/*FRt1SRqqt1FBt/SRqqt_purity‚0.5276229977607727 7 Purity/*FRt2SRqqt1FBt/FRt_purity‚0.2893287241458893 ,&Purity/*FRt2SRqqt4FBt/event_proportion‚0 9"Purity/0FRt2SRqqt*FBt/SRqqt_purity‚0.2833922505378723 9"Purity/2FRt1SRqqt1FBt/SRqqt_purity‚0.5572196841239929 ,&Purity/3FRt*SRqqt4FBt/event_proportion‚0 ,&Purity/3FRt0SRqqt3FBt/event_proportion‚ÜEªƒB0 ("Purity/4FRt1SRqqt0FBt/event_purity‚0 ,&Purity/0FRt4SRqqt3FBt/event_proportion‚0 & Purity/2FRt*SRqqt1FBt/FBt_purity‚1 7 Purity/1FRt2SRqqt2FBt/FRt_purity‚0.3888889253139496 ("Purity/4FRt2SRqqt0FBt/SRqqt_purity‚1 ,&Purity/2FRt1SRqqt4FBt/event_proportion‚0 ,&Purity/2FRt2SRqqt3FBt/event_proportion‚0 9"Purity/3FRt*SRqqt0FBt/SRqqt_purity‚0.2448979765176773 +"Purity/0FRt4SRqqt1FBt/SRqqt_purity‚0.25 ,"Purity/3FRt4SRqqt*FBt/SRqqt_purity‚0.125 particle/accuracy_4_of_7‚1 7 Purity/*FRt1SRqqt*FBt/FRt_purity‚0.3260553777217865 9"Purity/1FRt1SRqqt2FBt/SRqqt_purity‚0.6064625978469849 7 Purity/2FRt*SRqqt1FBt/FRt_purity‚0.3820081055164337 particle/accuracy_3_of_5‚1 9"Purity/*FRt2SRqqt1FBt/SRqqt_purity‚0.4407080411911011 9"Purity/1FRt*SRqqt*FBt/SRqqt_purity‚0.3174333870410919 *jet/accuracy_1_of_2‚0.6832196712493896 ("Purity/*FRt4SRqqt1FBt/event_purity‚0 9"Purity/0FRt*SRqqt3FBt/SRqqt_purity‚0.6000000238418579 7 Purity/3FRt3SRqqt0FBt/FRt_purity‚0.3809523582458496 9"Purity/0FRt*SRqqt1FBt/event_purity‚0.6448374390602112 =&Purity/1FRt*SRqqt0FBt/event_proportion‚0.3135710656642914 ,&Purity/4FRt3SRqqt*FBt/event_proportion‚0 & Purity/1FRt*SRqqt4FBt/FRt_purity‚0 >&Purity/*FRt0SRqqt0FBt/event_proportion‚0.15757308900356293 6 Purity/*FRt2SRqqt1FBt/FBt_purity‚0.990708589553833 9"Purity/*FRt0SRqqt1FBt/event_purity‚0.6912403106689453 ("Purity/0FRt0SRqqt0FBt/event_purity‚1 ;"Purity/1FRt1SRqqt2FBt/event_purity‚0.013095238246023655 ?&Purity/1FRt3SRqqt0FBt/event_proportion‚0.007938507944345474 ?&Purity/3FRt1SRqqt*FBt/event_proportion‚0.003717237850651145 +jet/accuracy_2_of_4‚0.33582329750061035 jet/accuracy_5_of_6‚0 ("Purity/1FRt0SRqqt4FBt/event_purity‚0 >&Purity/*FRt0SRqqt1FBt/event_proportion‚0.14591734111309052 ,&Purity/*FRt3SRqqt4FBt/event_proportion‚0 9"Purity/0FRt1SRqqt0FBt/SRqqt_purity‚0.2787102460861206 6 Purity/1FRt*SRqqt2FBt/FRt_purity‚0.529728353023529 7 Purity/1FRt0SRqqt3FBt/FBt_purity‚0.9285714030265808 7 Purity/1FRt1SRqqt*FBt/FRt_purity‚0.3402889370918274 :"Purity/2FRt2SRqqt1FBt/SRqqt_purity‚0.42351189255714417 ,&Purity/2FRt2SRqqt4FBt/event_proportion‚0 8 Purity/*FRt*SRqqt*FBt/FRt_purity‚0.32088181376457214 9"Purity/0FRt0SRqqt*FBt/event_purity‚0.8074281811714172 :"Purity/1FRt2SRqqt0FBt/SRqqt_purity‚0.26039260625839233 7 Purity/3FRt0SRqqt*FBt/FRt_purity‚0.3131313621997833 ,&Purity/3FRt4SRqqt4FBt/event_proportion‚0 *jet/accuracy_1_of_6‚0.9824562072753906 particle/accuracy_5_of_6‚1 :"Purity/2FRt3SRqqt1FBt/SRqqt_purity‚0.06666667014360428 :"Purity/*FRt1SRqqt*FBt/SRqqt_purity‚0.34360548853874207 ?&Purity/*FRt*SRqqt3FBt/event_proportion‚0.004473286215215921 >&Purity/0FRt0SRqqt1FBt/event_proportion‚0.08606351166963577 :"Purity/0FRt1SRqqt1FBt/event_purity‚0.17680126428604126 ,&Purity/0FRt2SRqqt4FBt/event_proportion‚0 9"Purity/1FRt1SRqqt0FBt/SRqqt_purity‚0.2679435610771179 8 Purity/2FRt1SRqqt*FBt/FRt_purity‚0.29837173223495483 ("Purity/0FRt3SRqqt1FBt/event_purity‚0 >&Purity/*FRt*SRqqt2FBt/event_proportion‚0.06747731566429138 :"Purity/*FRt1SRqqt1FBt/event_purity‚0.10965678840875626 ("Purity/2FRt1SRqqt1FBt/event_purity‚0 ?&Purity/2FRt3SRqqt0FBt/event_proportion‚0.003654233878478408 ,&Purity/4FRt1SRqqt3FBt/event_proportion‚0 ,&Purity/4FRt4SRqqt2FBt/event_proportion‚0 ("Purity/4FRt1SRqqt*FBt/event_purity‚0 % _timestamp‚1.7884895536083257e+09 ,&Purity/0FRt3SRqqt4FBt/event_proportion‚0 A&Purity/3FRt1SRqqt2FBt/event_proportion‚0.00025201612152159214 ("Purity/0FRt4SRqqt1FBt/event_purity‚0 & Purity/3FRt2SRqqt1FBt/FBt_purity‚1 & Purity/2FRt3SRqqt*FBt/FBt_purity‚1 & Purity/1FRt0SRqqt4FBt/FBt_purity‚1 & Purity/2FRt*SRqqt3FBt/FBt_purity‚1 ,&Purity/*FRt3SRqqt3FBt/event_proportion‚0 7 Purity/0FRt1SRqqt2FBt/FBt_purity‚0.9587036967277527 7 Purity/1FRt*SRqqt*FBt/FBt_purity‚0.9893227815628052 8 Purity/1FRt3SRqqt0FBt/FRt_purity‚0.33275729417800903 ?&Purity/2FRt2SRqqt1FBt/event_proportion‚0.003969253972172737 @&Purity/3FRt0SRqqt2FBt/event_proportion‚6.300403038039804e-05 & Purity/0FRt4SRqqt1FBt/FBt_purity‚1 jet/accuracy_6_of_6‚0 ,&Purity/2FRt3SRqqt4FBt/event_proportion‚0 9"Purity/3FRt1SRqqt0FBt/SRqqt_purity‚0.2750000059604645 9"Purity/*FRt*SRqqt3FBt/SRqqt_purity‚0.8333333134651184 ("Purity/*FRt1SRqqt4FBt/event_purity‚0 ;"Purity/0FRt2SRqqt*FBt/event_purity‚0.025378217920660973 7 Purity/2FRt2SRqqt*FBt/FRt_purity‚0.3362894654273987 7 Purity/1FRt*SRqqt2FBt/FBt_purity‚0.9802407622337341 7 Purity/*FRt0SRqqt*FBt/FRt_purity‚0.3112078607082367 7 Purity/0FRt1SRqqt*FBt/FBt_purity‚0.9848227500915527 ,&Purity/0FRt4SRqqt2FBt/event_proportion‚0 ,&Purity/1FRt4SRqqt2FBt/event_proportion‚0 8 Purity/2FRt0SRqqt0FBt/FRt_purity‚0.24463026225566864 @&Purity/2FRt0SRqqt3FBt/event_proportion‚6.300403038039804e-05 ("Purity/3FRt0SRqqt*FBt/event_purity‚0 7 Purity/1FRt0SRqqt1FBt/FRt_purity‚0.4388604760169983 ,&Purity/2FRt3SRqqt2FBt/event_proportion‚0 ,&Purity/3FRt2SRqqt2FBt/event_proportion‚0 9"Purity/3FRt1SRqqt1FBt/SRqqt_purity‚0.5769230723381042 ("Purity/*FRt0SRqqt4FBt/event_purity‚0 ("Purity/3FRt4SRqqt*FBt/event_purity‚0 8 Purity/2FRt3SRqqt*FBt/FRt_purity‚0.40047621726989746 ,&Purity/2FRt4SRqqt1FBt/event_proportion‚0 @&Purity/3FRt3SRqqt*FBt/event_proportion‚0.0005670362734235823 ,&Purity/4FRt4SRqqt1FBt/event_proportion‚0 7 Purity/*FRt0SRqqt2FBt/FBt_purity‚0.9810138940811157 >&Purity/0FRt*SRqqt1FBt/event_proportion‚0.14787046611309052 A&Purity/0FRt4SRqqt0FBt/event_proportion‚0.00044102821266278625 8 Purity/1FRt*SRqqt*FBt/FRt_purity‚0.32564759254455566 8 Purity/1FRt1SRqqt2FBt/FRt_purity‚0.37253400683403015 :"Purity/1FRt3SRqqt0FBt/SRqqt_purity‚0.22320792078971863 & Purity/2FRt0SRqqt1FBt/FBt_purity‚1 ("Purity/*FRt*SRqqt4FBt/event_purity‚0 7 Purity/*FRt*SRqqt2FBt/FRt_purity‚0.4793015420436859 @&Purity/4FRt1SRqqt*FBt/event_proportion‚6.300403038039804e-05 particle/accuracy_6_of_6‚1 7 Purity/3FRt0SRqqt2FBt/FRt_purity‚0.3333333432674408 8"Purity/3FRt3SRqqt0FBt/SRqqt_purity‚0.261904776096344 & Purity/2FRt2SRqqt2FBt/FBt_purity‚1 8 Purity/2FRt1SRqqt2FBt/FRt_purity‚0.37061405181884766 & Purity/*FRt1SRqqt4FBt/FRt_purity‚0 trainer/global_step‚2415 4 Purity/*FRt0SRqqt*FBt/FBt_purity‚0.9881631731987 8 Purity/*FRt1SRqqt2FBt/FRt_purity‚0.34955543279647827 8 Purity/*FRt3SRqqt1FBt/FRt_purity‚0.42222222685813904 9"Purity/0FRt1SRqqt*FBt/SRqqt_purity‚0.3411489427089691 & Purity/1FRt3SRqqt1FBt/FBt_purity‚1 >&Purity/2FRt*SRqqt*FBt/event_proportion‚0.10244455933570862 7 Purity/3FRt2SRqqt0FBt/FRt_purity‚0.3212560713291168 particle/accuracy_1_of_1‚1 & Purity/*FRt*SRqqt4FBt/FRt_purity‚0 ("Purity/4FRt2SRqqt*FBt/event_purity‚0 @&Purity/3FRt0SRqqt0FBt/event_proportion‚0.0015120967291295528 & Purity/1FRt1SRqqt4FBt/FRt_purity‚0 ?&Purity/*FRt1SRqqt2FBt/event_proportion‚0.017011089250445366 >&Purity/1FRt1SRqqt1FBt/event_proportion‚0.03944052383303642 & Purity/2FRt2SRqqt*FBt/FBt_purity‚1 jet/accuracy_4_of_7‚0 ("Purity/3FRt3SRqqt*FBt/event_purity‚0 ?&Purity/*FRt0SRqqt3FBt/event_proportion‚0.003717237850651145 >&Purity/*FRt3SRqqt0FBt/event_proportion‚0.01827116869390011 7 Purity/2FRt1SRqqt2FBt/FBt_purity‚0.9868420958518982 ("Purity/3FRt1SRqqt*FBt/event_purity‚0 9"Purity/1FRt2SRqqt2FBt/SRqqt_purity‚0.4236111342906952 ("Purity/3FRt0SRqqt2FBt/event_purity‚0 ("Purity/4FRt*SRqqt0FBt/event_purity‚0 7 Purity/*FRt*SRqqt2FBt/FBt_purity‚0.9786669015884399 8"Purity/1FRt2SRqqt1FBt/SRqqt_purity‚0.422465443611145 ("Purity/2FRt2SRqqt*FBt/event_purity‚0 7 Purity/3FRt*SRqqt1FBt/FRt_purity‚0.3492424488067627 9"Purity/3FRt2SRqqt0FBt/SRqqt_purity‚0.2771739363670349 ,"Purity/3FRt4SRqqt0FBt/SRqqt_purity‚0.125 jet/accuracy_3_of_7‚0 ( Purity/2FRt2SRqqt2FBt/FRt_purity‚0.5 7 Purity/*FRt0SRqqt2FBt/FRt_purity‚0.5379140973091125 7 Purity/*FRt1SRqqt*FBt/FBt_purity‚0.9868172407150269 7 Purity/0FRt0SRqqt2FBt/FBt_purity‚0.9787583947181702 7 Purity/1FRt1SRqqt2FBt/FBt_purity‚0.9755951762199402 ,&Purity/4FRt2SRqqt4FBt/event_proportion‚0 *jet/accuracy_1_of_5‚0.9322202205657959 & Purity/3FRt0SRqqt2FBt/FBt_purity‚1 ("Purity/0FRt2SRqqt3FBt/SRqqt_purity‚1 9"Purity/1FRt*SRqqt1FBt/SRqqt_purity‚0.5013907551765442 ("Purity/2FRt3SRqqt0FBt/event_purity‚0 ,&Purity/4FRt1SRqqt4FBt/event_proportion‚0 9"Purity/*FRt2SRqqt2FBt/SRqqt_purity‚0.3684210479259491 7 Purity/*FRt*SRqqt*FBt/FBt_purity‚0.9879723787307739 8 Purity/*FRt2SRqqt0FBt/FRt_purity‚0.33266615867614746 9"Purity/1FRt2SRqqt*FBt/SRqqt_purity‚0.2935531437397003 ,&Purity/1FRt4SRqqt4FBt/event_proportion‚0 ("Purity/1FRt2SRqqt2FBt/event_purity‚0 ( Purity/2FRt0SRqqt3FBt/FRt_purity‚0.5 ("Purity/4FRt1SRqqt0FBt/SRqqt_purity‚0 ("Purity/4FRt2SRqqt0FBt/event_purity‚0 ,&Purity/0FRt3SRqqt2FBt/event_proportion‚0 7 Purity/2FRt1SRqqt*FBt/FBt_purity‚0.9964157938957214 ("Purity/2FRt3SRqqt*FBt/event_purity‚0 ,&Purity/4FRt0SRqqt1FBt/event_proportion‚0 & Purity/*FRt0SRqqt4FBt/FRt_purity‚0 & Purity/0FRt3SRqqt*FBt/FBt_purity‚1 @&Purity/0FRt2SRqqt3FBt/event_proportion‚6.300403038039804e-05 @&Purity/0FRt3SRqqt1FBt/event_proportion‚0.0006300403038039804 9"Purity/1FRt0SRqqt*FBt/event_purity‚0.2316482663154602 ("Purity/3FRt*SRqqt0FBt/event_purity‚0 ("Purity/3FRt2SRqqt1FBt/event_purity‚0 _step‚195 7 Purity/*FRt1SRqqt2FBt/FBt_purity‚0.9676783680915833 A&Purity/0FRt*SRqqt4FBt/event_proportion‚0.00025201612152159214 >&Purity/0FRt0SRqqt*FBt/event_proportion‚0.11857359111309052 7 Purity/0FRt0SRqqt*FBt/FBt_purity‚0.9862682223320007 7 Purity/1FRt1SRqqt*FBt/FBt_purity‚0.9871369004249573 & Purity/3FRt0SRqqt*FBt/FBt_purity‚1 ,&Purity/4FRt2SRqqt3FBt/event_proportion‚0 +jet/accuracy_2_of_5‚0.45241019129753113 ("Purity/*FRt2SRqqt2FBt/event_purity‚0 & Purity/4FRt2SRqqt*FBt/FRt_purity‚0 ?&Purity/1FRt*SRqqt2FBt/event_proportion‚0.022303426638245583 @&Purity/1FRt*SRqqt3FBt/event_proportion‚0.0013230846961960196 7 Purity/2FRt*SRqqt0FBt/FRt_purity‚0.2840317189693451 particle/accuracy_1_of_5‚1 ( Purity/4FRt1SRqqt*FBt/FRt_purity‚0.5 _runtime‚4352.713515829 7 Purity/3FRt4SRqqt0FBt/FRt_purity‚0.3333333432674408 ("Purity/1FRt3SRqqt2FBt/SRqqt_purity‚0 @&Purity/1FRt1SRqqt4FBt/event_proportion‚6.300403038039804e-05 particle/accuracy_2_of_6‚1 ) Purity/3FRt1SRqqt2FBt/FRt_purity‚0.25 *"Purity/2FRt2SRqqt2FBt/SRqqt_purity‚0.5 ,&Purity/*FRt4SRqqt3FBt/event_proportion‚0 :"Purity/0FRt1SRqqt*FBt/event_purity‚0.24842491745948792 9"Purity/0FRt1SRqqt0FBt/event_purity‚0.2787102460861206 =&Purity/1FRt1SRqqt0FBt/event_proportion‚0.1265120953321457 jet/accuracy_7_of_7‚0 ,&Purity/3FRt4SRqqt1FBt/event_proportion‚0 & Purity/2FRt0SRqqt3FBt/FBt_purity‚1 ("Purity/1FRt1SRqqt3FBt/SRqqt_purity‚1 ("Purity/4FRt2SRqqt*FBt/SRqqt_purity‚1 :"Purity/1FRt1SRqqt0FBt/event_purity‚0.00961779523640871 ("Purity/1FRt3SRqqt2FBt/event_purity‚0 ) Purity/4FRt*SRqqt0FBt/FRt_purity‚0.25 @&Purity/*FRt0SRqqt4FBt/event_proportion‚0.0003150201519019902 ("Purity/1FRt2SRqqt0FBt/event_purity‚0 & Purity/3FRt*SRqqt1FBt/FBt_purity‚1 @&Purity/3FRt*SRqqt2FBt/event_proportion‚0.0003150201519019902 ,&Purity/3FRt1SRqqt4FBt/event_proportion‚0 8"Purity/3FRt3SRqqt*FBt/SRqqt_purity‚0.261904776096344 ("Purity/2FRt2SRqqt2FBt/event_purity‚0 @&Purity/*FRt1SRqqt3FBt/event_proportion‚0.0006930443341843784 ?&Purity/0FRt0SRqqt3FBt/event_proportion‚0.002583165420219302 7 Purity/0FRt2SRqqt*FBt/FBt_purity‚0.9738863706588745 8"Purity/3FRt1SRqqt*FBt/SRqqt_purity‚0.395061731338501 & Purity/3FRt1SRqqt2FBt/FBt_purity‚1 ("Purity/*FRt1SRqqt3FBt/event_purity‚0 9"Purity/0FRt1SRqqt3FBt/SRqqt_purity‚0.6000000238418579 9"Purity/*FRt0SRqqt3FBt/event_purity‚0.1577381044626236 >&Purity/*FRt1SRqqt0FBt/event_proportion‚0.33039313554763794 7 Purity/0FRt*SRqqt2FBt/FBt_purity‚0.9755595922470093 A&Purity/0FRt1SRqqt3FBt/event_proportion‚0.00044102821266278625 :"Purity/2FRt2SRqqt*FBt/SRqqt_purity‚0.31785011291503906 ("Purity/*FRt1SRqqt4FBt/SRqqt_purity‚1 & Purity/*FRt3SRqqt1FBt/FBt_purity‚1 8 Purity/1FRt2SRqqt*FBt/FRt_purity‚0.32678866386413574 8 Purity/1FRt3SRqqt1FBt/FRt_purity‚0.42222222685813904 @&Purity/2FRt4SRqqt0FBt/event_proportion‚0.0003150201519019902 ("Purity/3FRt0SRqqt1FBt/event_purity‚0 (particle/accuracy‚0.8433614373207092 @&Purity/4FRt0SRqqt0FBt/event_proportion‚6.300403038039804e-05 9"Purity/3FRt2SRqqt1FBt/SRqqt_purity‚0.5833333134651184 7 Purity/0FRt*SRqqt*FBt/FBt_purity‚0.9856035709381104 @&Purity/*FRt2SRqqt3FBt/event_proportion‚6.300403038039804e-05 @&Purity/1FRt0SRqqt3FBt/event_proportion‚0.0010710685746744275 <"Purity/1FRt2SRqqt*FBt/event_purity‚0.0011520737316459417 ;"Purity/1FRt2SRqqt1FBt/event_purity‚0.006451612804085016 ?&Purity/2FRt2SRqqt0FBt/event_proportion‚0.015435988083481789 ,&Purity/3FRt2SRqqt4FBt/event_proportion‚0 @&Purity/*FRt3SRqqt2FBt/event_proportion‚6.300403038039804e-05 9"Purity/2FRt3SRqqt*FBt/SRqqt_purity‚0.1895555555820465 & Purity/3FRt1SRqqt*FBt/FBt_purity‚1 ("Purity/3FRt1SRqqt0FBt/event_purity‚0 & Purity/0FRt2SRqqt2FBt/FBt_purity‚1 8 Purity/1FRt0SRqqt0FBt/FRt_purity‚0.22988179326057434 ?&Purity/*FRt3SRqqt1FBt/event_proportion‚0.002142137149348855 :"Purity/1FRt*SRqqt1FBt/event_purity‚0.15802635252475739 @&Purity/4FRt1SRqqt0FBt/event_proportion‚6.300403038039804e-05 ."Purity/1FRt*SRqqt3FBt/event_purity‚0.03125 >&Purity/1FRt*SRqqt1FBt/event_proportion‚0.09929435700178146 8 Purity/2FRt0SRqqt1FBt/FRt_purity‚0.39974868297576904 >&Purity/2FRt1SRqqt*FBt/event_proportion‚0.04158266261219978 ,&Purity/4FRt*SRqqt4FBt/event_proportion‚0 @&Purity/4FRt2SRqqt0FBt/event_proportion‚6.300403038039804e-05 & Purity/*FRt4SRqqt*FBt/FBt_purity‚1 +jet/accuracy_2_of_6‚0.48245611786842346 7 Purity/1FRt*SRqqt3FBt/FBt_purity‚0.9270833134651184 7 Purity/1FRt0SRqqt2FBt/FRt_purity‚0.5750864148139954 ?&Purity/2FRt0SRqqt2FBt/event_proportion‚0.003465221729129553 ,&Purity/4FRt0SRqqt4FBt/event_proportion‚0 +jet/accuracy_3_of_6‚0.05263157933950424 particle/accuracy_7_of_7‚1 *"Purity/4FRt*SRqqt*FBt/SRqqt_purity‚0.5 *loss/val_total_loss‚42.274803792452765 9"Purity/0FRt1SRqqt2FBt/SRqqt_purity‚0.5247354507446289 ;"Purity/0FRt2SRqqt0FBt/event_purity‚0.028327105566859245 :"Purity/1FRt1SRqqt*FBt/event_purity‚0.01820864900946617 7 Purity/2FRt*SRqqt*FBt/FBt_purity‚0.9961161017417908 9"Purity/2FRt1SRqqt2FBt/SRqqt_purity‚0.7280701398849487 :"Purity/2FRt4SRqqt0FBt/SRqqt_purity‚0.15000000596046448 9"Purity/*FRt2SRqqt0FBt/SRqqt_purity‚0.2606409788131714 ?&Purity/0FRt0SRqqt2FBt/event_proportion‚0.029674898833036423 7 Purity/3FRt*SRqqt*FBt/FRt_purity‚0.3173195719718933 @&Purity/3FRt*SRqqt1FBt/event_proportion‚0.0024571572430431843 ,&Purity/4FRt4SRqqt3FBt/event_proportion‚0 :"Purity/0FRt3SRqqt0FBt/SRqqt_purity‚0.21746031939983368 ("Purity/*FRt3SRqqt0FBt/event_purity‚0 @&Purity/2FRt4SRqqt*FBt/event_proportion‚0.0003150201519019902 particle/accuracy_3_of_3‚1 particle/accuracy_3_of_7‚1 8"Purity/0FRt*SRqqt*FBt/event_purity‚0.375698983669281 7 Purity/2FRt*SRqqt2FBt/FBt_purity‚0.9911110997200012 ?&Purity/2FRt0SRqqt1FBt/event_proportion‚0.008946572430431843 ,&Purity/4FRt4SRqqt*FBt/event_proportion‚0 0"Purity/*FRt4SRqqt0FBt/SRqqt_purity‚ 0.2421875 :"Purity/0FRt1SRqqt2FBt/event_purity‚0.08481481671333313 9"Purity/1FRt0SRqqt3FBt/event_purity‚0.0357142873108387 ("Purity/2FRt1SRqqt2FBt/event_purity‚0 ("Purity/1FRt4SRqqt*FBt/event_purity‚0 7 Purity/0FRt1SRqqt3FBt/FBt_purity‚0.8666666150093079 ?&Purity/1FRt2SRqqt1FBt/event_proportion‚0.008946572430431843 @&Purity/1FRt3SRqqt2FBt/event_proportion‚6.300403038039804e-05 ,&Purity/1FRt4SRqqt1FBt/event_proportion‚0 ;"Purity/2FRt*SRqqt*FBt/event_purity‚0.013046753592789173 ("Purity/2FRt1SRqqt*FBt/event_purity‚0 ,&Purity/2FRt4SRqqt4FBt/event_proportion‚0 ,&Purity/3FRt3SRqqt2FBt/event_proportion‚0 ?&Purity/2FRt1SRqqt1FBt/event_proportion‚0.007875503972172737 particle/accuracy_4_of_4‚1 & Purity/1FRt0SRqqt4FBt/FRt_purity‚0 ("Purity/1FRt1SRqqt3FBt/event_purity‚0 :"Purity/0FRt*SRqqt0FBt/SRqqt_purity‚0.26524558663368225 ,&Purity/2FRt0SRqqt4FBt/event_proportion‚0 ?&Purity/2FRt0SRqqt*FBt/event_proportion‚0.037046369165182114 @&Purity/3FRt2SRqqt0FBt/event_proportion‚0.0024571572430431843 A&Purity/3FRt4SRqqt0FBt/event_proportion‚0.00012600806076079607 :"Purity/*FRt0SRqqt2FBt/event_purity‚0.22115261852741241 =&Purity/*FRt2SRqqt*FBt/event_proportion‚0.1332535296678543 ("Purity/*FRt3SRqqt1FBt/event_purity‚0 7 Purity/1FRt*SRqqt1FBt/FRt_purity‚0.3811166286468506 ,&Purity/1FRt3SRqqt4FBt/event_proportion‚0 ( Purity/2FRt4SRqqt*FBt/FRt_purity‚0.5Ë (o=Ží¦ =&Purity/0FRt1SRqqt0FBt/event_proportion‚0.1694808453321457 7 Purity/0FRt1SRqqt1FBt/FBt_purity‚0.9955925345420837 9"Purity/1FRt1SRqqt*FBt/SRqqt_purity‚0.3469739556312561 >&Purity/1FRt2SRqqt*FBt/event_proportion‚0.05727066472172737 8 Purity/1FRt2SRqqt1FBt/FRt_purity‚0.25944700837135315 7 Purity/1FRt3SRqqt*FBt/FRt_purity‚0.3181643784046173 7 Purity/1FRt4SRqqt0FBt/FRt_purity‚0.1666666716337204 :"Purity/*FRt0SRqqt0FBt/event_purity‚0.19419921934604645 7 Purity/*FRt2SRqqt2FBt/FRt_purity‚0.3974359333515167 ("Purity/0FRt4SRqqt0FBt/event_purity‚0 ("Purity/1FRt*SRqqt4FBt/event_purity‚0 ,&Purity/4FRt4SRqqt4FBt/event_proportion‚0 7 Purity/*FRt0SRqqt0FBt/FRt_purity‚0.2335006296634674 9"Purity/0FRt*SRqqt*FBt/SRqqt_purity‚0.3162253499031067 ,&Purity/0FRt4SRqqt4FBt/event_proportion‚0 7 Purity/1FRt*SRqqt1FBt/FBt_purity‚0.9966728091239929 8 Purity/2FRt0SRqqt*FBt/FRt_purity‚0.30303895473480225 +particle/specificity‚0.9471937417984009 & Purity/0FRt4SRqqt*FBt/FBt_purity‚1 ("Purity/0FRt*SRqqt4FBt/event_purity‚0 & Purity/3FRt2SRqqt*FBt/FBt_purity‚1 & Purity/2FRt3SRqqt1FBt/FBt_purity‚1 +"Purity/1FRt4SRqqt*FBt/SRqqt_purity‚0.25 :"Purity/2FRt*SRqqt*FBt/SRqqt_purity‚0.31391891837120056 :"Purity/2FRt1SRqqt*FBt/SRqqt_purity‚0.34586408734321594 ("Purity/2FRt2SRqqt0FBt/event_purity‚0 @&Purity/3FRt0SRqqt*FBt/event_proportion‚0.0027091733645647764 ,&Purity/3FRt1SRqqt3FBt/event_proportion‚0 ,&Purity/4FRt1SRqqt1FBt/event_proportion‚0 ,&Purity/4FRt4SRqqt0FBt/event_proportion‚0 7validation_average_jet_accuracy‚0.38158681988716125 7 Purity/2FRt2SRqqt0FBt/FRt_purity‚0.3471948504447937 ,&Purity/2FRt3SRqqt3FBt/event_proportion‚0 ?&Purity/*FRt3SRqqt*FBt/event_proportion‚0.020476309582591057 9"Purity/2FRt*SRqqt0FBt/SRqqt_purity‚0.2679072618484497 jet/accuracy_5_of_5‚0 ?&Purity/0FRt2SRqqt1FBt/event_proportion‚0.009261592291295528 , Purity/1FRt*SRqqt3FBt/FRt_purity‚0.59375 & Purity/*FRt2SRqqt2FBt/FBt_purity‚1 ("Purity/3FRt1SRqqt2FBt/event_purity‚0 ("Purity/1FRt4SRqqt0FBt/event_purity‚0 6 Purity/*FRt1SRqqt0FBt/FRt_purity‚0.319568008184433 ("Purity/1FRt1SRqqt4FBt/SRqqt_purity‚1 :"Purity/*FRt*SRqqt2FBt/event_purity‚0.17083577811717987 9"Purity/*FRt*SRqqt*FBt/event_purity‚0.2130253165960312 9"Purity/1FRt*SRqqt2FBt/event_purity‚0.0713355615735054 :"Purity/2FRt*SRqqt1FBt/SRqqt_purity‚0.46686428785324097 ,&Purity/4FRt3SRqqt4FBt/event_proportion‚0 particle/accuracy_3_of_4‚1 7 Purity/3FRt3SRqqt*FBt/FRt_purity‚0.3809523582458496 ("Purity/0FRt2SRqqt3FBt/event_purity‚0 7 Purity/*FRt*SRqqt0FBt/FRt_purity‚0.2907537817955017 7 Purity/*FRt2SRqqt*FBt/FBt_purity‚0.9909133911132812 >&Purity/0FRt1SRqqt1FBt/event_proportion‚0.05185231938958168 ("Purity/3FRt2SRqqt*FBt/event_purity‚0 @&Purity/3FRt2SRqqt1FBt/event_proportion‚0.0003780241822823882 ("Purity/0FRt3SRqqt*FBt/event_purity‚0 ;"Purity/*FRt2SRqqt*FBt/event_purity‚0.010293247178196907 7 Purity/2FRt1SRqqt0FBt/FRt_purity‚0.2812124192714691 @&Purity/2FRt3SRqqt1FBt/event_proportion‚0.0003780241822823882 ,&Purity/4FRt3SRqqt2FBt/event_proportion‚0 *"Purity/3FRt1SRqqt2FBt/SRqqt_purity‚0.5 =&Purity/*FRt1SRqqt1FBt/event_proportion‚0.1001134067773819 >&Purity/0FRt*SRqqt2FBt/event_proportion‚0.03944052383303642 ?&Purity/1FRt1SRqqt2FBt/event_proportion‚0.006111391354352236 @&Purity/*FRt4SRqqt1FBt/event_proportion‚6.300403038039804e-05 9"Purity/*FRt*SRqqt2FBt/SRqqt_purity‚0.5536362528800964 :"Purity/*FRt*SRqqt0FBt/SRqqt_purity‚0.26327845454216003 :"Purity/*FRt3SRqqt1FBt/SRqqt_purity‚0.31878307461738586 9"Purity/1FRt*SRqqt2FBt/SRqqt_purity‚0.5547106862068176 8 Purity/1FRt1SRqqt0FBt/FRt_purity‚0.34041136503219604 7 Purity/3FRt0SRqqt0FBt/FRt_purity‚0.1666666716337204 epoch‚3 7 Purity/0FRt0SRqqt1FBt/FBt_purity‚0.9990224242210388 :"Purity/0FRt2SRqqt0FBt/SRqqt_purity‚0.24866622686386108 7 Purity/1FRt1SRqqt1FBt/FBt_purity‚0.9967661499977112 7 Purity/2FRt0SRqqt2FBt/FRt_purity‚0.5202381014823914 & Purity/3FRt0SRqqt1FBt/FBt_purity‚1 ("Purity/2FRt4SRqqt0FBt/event_purity‚0 >&Purity/*FRt2SRqqt0FBt/event_proportion‚0.10855595022439957 9"Purity/1FRt0SRqqt1FBt/event_purity‚0.2820163071155548 :"Purity/*FRt3SRqqt0FBt/SRqqt_purity‚0.21077412366867065 :"Purity/0FRt2SRqqt1FBt/event_purity‚0.03870967775583267 & Purity/2FRt1SRqqt1FBt/FBt_purity‚1 9"Purity/3FRt*SRqqt*FBt/SRqqt_purity‚0.3228106200695038 0"Purity/*FRt4SRqqt*FBt/SRqqt_purity‚ 0.2421875 & Purity/0FRt3SRqqt1FBt/FBt_purity‚1 7 Purity/*FRt*SRqqt1FBt/FBt_purity‚0.9972519278526306 ;"Purity/*FRt2SRqqt0FBt/event_purity‚0.010600889101624489 :"Purity/0FRt*SRqqt3FBt/event_purity‚0.16049382090568542 >&Purity/2FRt*SRqqt0FBt/event_proportion‚0.07579384744167328 ("Purity/0FRt3SRqqt0FBt/event_purity‚0 7 Purity/0FRt2SRqqt3FBt/FBt_purity‚0.6666666865348816 @&Purity/*FRt4SRqqt*FBt/event_proportion‚0.0013230846961960196 >&Purity/0FRt2SRqqt*FBt/event_proportion‚0.05361643061041832 ?&Purity/0FRt2SRqqt0FBt/event_proportion‚0.043283771723508835 ?&Purity/0FRt3SRqqt*FBt/event_proportion‚0.006741431541740894 ?&Purity/1FRt0SRqqt1FBt/event_proportion‚0.049773186445236206 ,&Purity/1FRt4SRqqt3FBt/event_proportion‚0 ,&Purity/2FRt*SRqqt4FBt/event_proportion‚0 jet/accuracy_6_of_7‚0 7 Purity/*FRt1SRqqt1FBt/FBt_purity‚0.9963663220405579 ("Purity/*FRt4SRqqt0FBt/event_purity‚0 particle/accuracy_4_of_6‚1 & Purity/1FRt3SRqqt2FBt/FRt_purity‚1 ("Purity/4FRt0SRqqt0FBt/event_purity‚0 & Purity/4FRt2SRqqt0FBt/FRt_purity‚0 7 Purity/*FRt0SRqqt3FBt/FRt_purity‚0.5666666626930237 8 Purity/*FRt2SRqqt*FBt/FRt_purity‚0.32429733872413635 ("Purity/1FRt3SRqqt*FBt/event_purity‚0 9"Purity/2FRt0SRqqt2FBt/event_purity‚0.0535714291036129 -"Purity/0FRt4SRqqt*FBt/SRqqt_purity‚0.3125 7 Purity/0FRt*SRqqt3FBt/FBt_purity‚0.9032922387123108 A&Purity/1FRt*SRqqt4FBt/event_proportion‚0.00012600806076079607 :"Purity/1FRt0SRqqt2FBt/event_purity‚0.10211760550737381 @&Purity/1FRt4SRqqt0FBt/event_proportion‚0.0003780241822823882 @&Purity/3FRt1SRqqt0FBt/event_proportion‚0.0025201612152159214 ("Purity/2FRt4SRqqt*FBt/event_purity‚0 jet/accuracy_4_of_6‚0 & Purity/2FRt2SRqqt1FBt/FBt_purity‚1 ,&Purity/2FRt4SRqqt3FBt/event_proportion‚0 ,&Purity/3FRt4SRqqt2FBt/event_proportion‚0 ,&Purity/3FRt4SRqqt3FBt/event_proportion‚0 + Purity/0FRt*SRqqt4FBt/FBt_purity‚0.8125 ( Purity/*FRt1SRqqt3FBt/FRt_purity‚0.5 :"Purity/*FRt1SRqqt2FBt/event_purity‚0.05198215693235397 :"Purity/0FRt*SRqqt0FBt/event_purity‚0.22062766551971436 :"Purity/1FRt0SRqqt0FBt/event_purity‚0.22988179326057434 9"Purity/*FRt3SRqqt*FBt/SRqqt_purity‚0.2152818888425827 >&Purity/0FRt0SRqqt0FBt/event_proportion‚0.04082661122083664 ?&Purity/0FRt1SRqqt2FBt/event_proportion‚0.008757560513913631 :"Purity/1FRt*SRqqt0FBt/event_purity‚0.10019713640213013 @&Purity/2FRt*SRqqt3FBt/event_proportion‚6.300403038039804e-05 ,&Purity/2FRt1SRqqt3FBt/event_proportion‚0 7 Purity/2FRt3SRqqt0FBt/FRt_purity‚0.4046536684036255 @&Purity/4FRt*SRqqt*FBt/event_proportion‚0.0001890120911411941 >&Purity/0FRt*SRqqt*FBt/event_proportion‚0.40996724367141724 7 Purity/*FRt0SRqqt1FBt/FBt_purity‚0.9987428784370422 9"Purity/0FRt*SRqqt2FBt/SRqqt_purity‚0.5051851868629456 9"Purity/0FRt0SRqqt3FBt/event_purity‚0.2266666740179062 7 Purity/1FRt*SRqqt0FBt/FRt_purity‚0.2937387228012085 9"Purity/1FRt1SRqqt1FBt/SRqqt_purity‚0.5478251576423645 & Purity/1FRt3SRqqt*FBt/FBt_purity‚1 7 Purity/2FRt0SRqqt2FBt/FBt_purity‚0.9940475821495056 @&Purity/*FRt*SRqqt4FBt/event_proportion‚0.0003780241822823882 9"Purity/*FRt1SRqqt0FBt/SRqqt_purity‚0.2733638882637024 @&Purity/2FRt2SRqqt2FBt/event_proportion‚6.300403038039804e-05 7 Purity/3FRt1SRqqt1FBt/FRt_purity‚0.2820512652397156 +jet/accuracy_2_of_3‚0.22665588557720184 particle/accuracy_2_of_2‚1 jet/accuracy_1_of_7‚1 9"Purity/*FRt1SRqqt2FBt/SRqqt_purity‚0.5963643789291382 >&Purity/*FRt2SRqqt1FBt/event_proportion‚0.02255544438958168 ?&Purity/2FRt*SRqqt2FBt/event_proportion‚0.005418346729129553 ,&Purity/3FRt3SRqqt3FBt/event_proportion‚0 ,&Purity/4FRt*SRqqt2FBt/event_proportion‚0 ,&Purity/4FRt1SRqqt2FBt/event_proportion‚0 particle/accuracy_1_of_3‚1 ( Purity/1FRt1SRqqt3FBt/FRt_purity‚0.5 :"Purity/*FRt*SRqqt1FBt/event_purity‚0.41315343976020813 :"Purity/*FRt*SRqqt3FBt/event_purity‚0.12068965286016464 7 Purity/*FRt*SRqqt3FBt/FRt_purity‚0.5882353186607361 ?&Purity/0FRt3SRqqt0FBt/event_proportion‚0.006111391354352236 @&Purity/0FRt4SRqqt*FBt/event_proportion‚0.0005040322430431843 ,&Purity/1FRt2SRqqt4FBt/event_proportion‚0 ("Purity/1FRt3SRqqt1FBt/event_purity‚0 7 Purity/2FRt0SRqqt*FBt/FBt_purity‚0.9967741966247559 :"Purity/0FRt*SRqqt2FBt/event_purity‚0.24558140337467194 9"Purity/2FRt1SRqqt0FBt/SRqqt_purity‚0.2741154730319977 9"Purity/3FRt*SRqqt1FBt/SRqqt_purity‚0.6555556058883667 8 Purity/3FRt2SRqqt*FBt/FRt_purity‚0.31481483578681946 ("Purity/3FRt0SRqqt0FBt/event_purity‚0 ,&Purity/4FRt2SRqqt1FBt/event_proportion‚0 jet/accuracy_2_of_7‚0 & Purity/1FRt3SRqqt2FBt/FBt_purity‚1 8 Purity/1FRt2SRqqt0FBt/FRt_purity‚0.33670440316200256 particle/accuracy_1_of_4‚1 7 Purity/*FRt0SRqqt4FBt/FBt_purity‚0.8500000238418579 & Purity/*FRt4SRqqt1FBt/FBt_purity‚1 ("Purity/1FRt*SRqqt4FBt/SRqqt_purity‚1 & Purity/*FRt3SRqqt2FBt/FBt_purity‚1 9"Purity/0FRt0SRqqt1FBt/event_purity‚0.9990224242210388 +jet/accuracy_3_of_4‚0.06460268050432205 7 Purity/*FRt1SRqqt3FBt/FBt_purity‚0.8730158805847168 :"Purity/0FRt0SRqqt2FBt/event_purity‚0.29995962977409363 ,&Purity/2FRt4SRqqt2FBt/event_proportion‚0 ("Purity/3FRt*SRqqt*FBt/event_purity‚0 & Purity/3FRt1SRqqt1FBt/FBt_purity‚1 *jet/accuracy_1_of_3‚0.8125315308570862 ("Purity/*FRt4SRqqt*FBt/event_purity‚0 ("Purity/0FRt2SRqqt2FBt/event_purity‚0 =&Purity/*FRt*SRqqt*FBt/event_proportion‚0.9591733813285828 7 Purity/0FRt*SRqqt1FBt/FBt_purity‚0.9969891309738159 +jet/accuracy_2_of_2‚0.09841278940439224 (particle/f_score‚0.36029088497161865 9"Purity/0FRt4SRqqt0FBt/SRqqt_purity‚0.3214285671710968 7 Purity/0FRt0SRqqt3FBt/FBt_purity‚0.9177777767181396 8 Purity/1FRt0SRqqt*FBt/FRt_purity‚0.31120651960372925 ?&Purity/2FRt*SRqqt1FBt/event_proportion‚0.021169355139136314 :"Purity/2FRt0SRqqt*FBt/event_purity‚0.03291160240769386 ,&Purity/4FRt*SRqqt3FBt/event_proportion‚0 +jet/accuracy_3_of_5‚0.08627788722515106 7 Purity/1FRt1SRqqt3FBt/FBt_purity‚0.8333333730697632 7 Purity/*FRt0SRqqt1FBt/FRt_purity‚0.4371524155139923 =&Purity/*FRt1SRqqt*FBt/event_proportion‚0.4482736885547638 ?&Purity/*FRt2SRqqt2FBt/event_proportion‚0.002079133177176118 & Purity/*FRt3SRqqt*FBt/FBt_purity‚1 =&Purity/0FRt1SRqqt*FBt/event_proportion‚0.2305317521095276 @&Purity/4FRt2SRqqt*FBt/event_proportion‚6.300403038039804e-05 :"Purity/1FRt3SRqqt*FBt/SRqqt_purity‚0.23582948744297028 7 Purity/2FRt2SRqqt1FBt/FRt_purity‚0.3422618806362152 ("Purity/0FRt4SRqqt*FBt/event_purity‚0 particle/accuracy_3_of_6‚1 :"Purity/0FRt2SRqqt2FBt/SRqqt_purity‚0.32692307233810425 7 Purity/3FRt*SRqqt2FBt/FRt_purity‚0.2666666805744171 7 Purity/*FRt*SRqqt3FBt/FBt_purity‚0.9114943146705627 :"Purity/0FRt2SRqqt1FBt/SRqqt_purity‚0.46776753664016724 7 Purity/0FRt2SRqqt1FBt/FBt_purity‚0.9792627096176147 @&Purity/1FRt0SRqqt4FBt/event_proportion‚6.300403038039804e-05 A&Purity/1FRt1SRqqt3FBt/event_proportion‚0.00025201612152159214 :"Purity/2FRt2SRqqt0FBt/SRqqt_purity‚0.28153133392333984 :"Purity/2FRt4SRqqt*FBt/SRqqt_purity‚0.15000000596046448 @&Purity/0FRt4SRqqt1FBt/event_proportion‚6.300403038039804e-05 ?&Purity/1FRt3SRqqt*FBt/event_proportion‚0.009135584346950054 & Purity/3FRt*SRqqt*FBt/FBt_purity‚1 particle/accuracy_2_of_4‚1 7 Purity/*FRt*SRqqt4FBt/FBt_purity‚0.8333333134651184 7 Purity/3FRt4SRqqt*FBt/FRt_purity‚0.3333333432674408 +"Purity/1FRt4SRqqt0FBt/SRqqt_purity‚0.25 ("Purity/1FRt1SRqqt4FBt/event_purity‚0 >&Purity/1FRt0SRqqt*FBt/event_proportion‚0.19745463132858276 :"Purity/2FRt*SRqqt0FBt/event_purity‚0.00605451874434948 ("Purity/*FRt2SRqqt3FBt/event_purity‚0 ("Purity/*FRt3SRqqt*FBt/event_purity‚0 A&Purity/0FRt0SRqqt4FBt/event_proportion‚0.00025201612152159214 @&Purity/1FRt4SRqqt*FBt/event_proportion‚0.0003780241822823882 ("Purity/3FRt*SRqqt1FBt/event_purity‚0 ,&Purity/4FRt3SRqqt3FBt/event_proportion‚0 ,jet/accuracy_3_of_3‚0.025819260627031326 particle/accuracy_1_of_2‚1 ("Purity/0FRt0SRqqt4FBt/event_purity‚0 9"Purity/*FRt*SRqqt1FBt/SRqqt_purity‚0.4903438985347748 7 Purity/*FRt1SRqqt1FBt/FRt_purity‚0.3371106684207916 ,&Purity/0FRt3SRqqt3FBt/event_proportion‚0 >&Purity/1FRt0SRqqt0FBt/event_proportion‚0.13142640888690948 ,&Purity/1FRt2SRqqt3FBt/event_proportion‚0 9"Purity/1FRt3SRqqt1FBt/SRqqt_purity‚0.3888889253139496 7 Purity/2FRt*SRqqt*FBt/FRt_purity‚0.3109205961227417 ,&Purity/3FRt3SRqqt1FBt/event_proportion‚0 ?&Purity/2FRt0SRqqt0FBt/event_proportion‚0.024571573361754417 7 Purity/2FRt1SRqqt1FBt/FRt_purity‚0.3477534353733063 ,&Purity/4FRt*SRqqt1FBt/event_proportion‚0 ,&Purity/4FRt2SRqqt2FBt/event_proportion‚0 *jet/accuracy_1_of_4‚0.8885867595672607 particle/accuracy_2_of_3‚1 + Purity/0FRt0SRqqt4FBt/FBt_purity‚0.8125 particle/accuracy_1_of_6‚1 8 Purity/*FRt*SRqqt1FBt/FRt_purity‚0.37970566749572754 =&Purity/1FRt*SRqqt*FBt/event_proportion‚0.4366179406642914 jet/accuracy_5_of_7‚0 & Purity/3FRt*SRqqt2FBt/FBt_purity‚1 ) Purity/4FRt*SRqqt*FBt/FRt_purity‚0.25 ("Purity/4FRt0SRqqt*FBt/event_purity‚0 9"Purity/*FRt*SRqqt0FBt/event_purity‚0.1301674097776413 ;"Purity/1FRt1SRqqt1FBt/event_purity‚0.049583740532398224 7 Purity/1FRt1SRqqt1FBt/FRt_purity‚0.3359280526638031 7 Purity/3FRt0SRqqt1FBt/FRt_purity‚0.5320513248443604 *jet/accuracy_1_of_1‚0.3986526429653168 ) Purity/*FRt1SRqqt4FBt/FBt_purity‚0.75 ) Purity/1FRt1SRqqt4FBt/FBt_purity‚0.75 & Purity/*FRt3SRqqt2FBt/FRt_purity‚1 7 Purity/*FRt0SRqqt3FBt/FBt_purity‚0.9192460775375366 @&Purity/0FRt*SRqqt3FBt/event_proportion‚0.0030871976632624865 ,&Purity/4FRt3SRqqt0FBt/event_proportion‚0 ,particle/sensitivity‚0.28248053789138794 particle/accuracy_6_of_7‚1 =&Purity/*FRt0SRqqt*FBt/event_proportion‚0.3558467626571655 7 Purity/1FRt2SRqqt*FBt/FBt_purity‚0.9919354915618896 ("Purity/1FRt3SRqqt0FBt/event_purity‚0 ,&Purity/1FRt3SRqqt3FBt/event_proportion‚0 9"Purity/2FRt*SRqqt2FBt/event_purity‚0.0277777798473835 ?&Purity/2FRt1SRqqt2FBt/event_proportion‚0.001890120911411941 ("Purity/2FRt2SRqqt1FBt/event_purity‚0 @&Purity/3FRt2SRqqt*FBt/event_proportion‚0.0028351815417408943 :"Purity/1FRt*SRqqt*FBt/event_purity‚0.11193739622831345 7 Purity/1FRt0SRqqt1FBt/FBt_purity‚0.9977647066116333 ?&Purity/1FRt0SRqqt2FBt/event_proportion‚0.015120967291295528 @&Purity/1FRt3SRqqt1FBt/event_proportion‚0.0011340725468471646 ,&Purity/3FRt3SRqqt4FBt/event_proportion‚0 ( Purity/2FRt4SRqqt0FBt/FRt_purity‚0.5 ("Purity/2FRt3SRqqt1FBt/event_purity‚0 ( Purity/4FRt1SRqqt0FBt/FRt_purity‚0.5 =&Purity/*FRt0SRqqt2FBt/event_proportion‚0.0483240932226181 9"Purity/0FRt1SRqqt1FBt/SRqqt_purity‚0.5083388686180115 :"Purity/2FRt0SRqqt1FBt/event_purity‚0.07240740954875946 @&Purity/3FRt1SRqqt1FBt/event_proportion‚0.0009450604557059705 9"Purity/0FRt3SRqqt1FBt/SRqqt_purity‚0.2916666567325592 ,&Purity/0FRt1SRqqt4FBt/event_proportion‚0 >&Purity/1FRt2SRqqt0FBt/event_proportion‚0.04731602966785431 *"Purity/3FRt*SRqqt2FBt/S