:W&Bá¾Pê¹ÆM e4z99wkeª; 9 /0.29.0+7872fd9d9f86ca5c50617e19cb168122b7f959480.70.0œ¯”ø‚" 40o936230hcv e4z99wkeŠÔ e4z99wkejuju11-uc-san-diegoSPAtop"— _wandb‚{}  batch_size‚512  dataset_limit‚0.2  dropout‚0.1487882170160095 epochs‚10  hidden_dim‚64 & learning_rate‚0.001304478809897408  num_attention_heads‚8  num_branch_encoder_layers‚7  num_embedding_layers‚14  num_encoder_layers‚4Bjjtttt_blocks_e4z99wkebs2s2ev0zj*jj-spatop-sweepagent-tttt-pairwise-2-dvk5pŠ »·éÔ°Š´Êš3 (H¨¸hp"3.12.10*0.29.0j linux-x86_643ä 97 e4z99wke&  _runtime‚0  _wandb.runtime‚0 ‚š§jÒä -Linux-6.8.0-138-generic-x86_64-with-glibc2.36CPython 3.12.10 º·éÔ—Ö›*-of*=/data/spatop/options_files/sweep_tttt_jj/config_e4z99wke.json*-l* /scratch/logs*-n*jjtttt_blocks_e4z99wke2-m spanet.trainRjacksonjulian2014@gmail.comZ /scratch/logsb*jj-spatop-sweepagent-tttt-pairwise-2-dvk5pr/opt/conda/bin/pythonº xhra6ob3rj5qcgx6i8i5dhue1of4kza7…¹Ù®ÉÒÀ€ˆ€’NVIDIA A100-SXM4-80GB˜¢ /€€€¥È.€€‹žÖ$ª€ äÓ½ÂS NVIDIA A100-SXM4-80GB€€€€À€6"Ampere*(GPU-8cb16037-0025-0eda-9425-82f6843a99c3Ê13.2º xhra6ob3rj5qcgx6i8i5dhue1of4kza7µª½Ç2  requirements.txtôÃW e4z99wkeZF (H¨¸ (H¨¸ (hpð"3.12.10*0.29.0j linux-x86_64oV$Y e4z99wkeZH (H¨¸ (H¨¸ (8hpð"3.12.10*0.29.0j linux-x86_64ï@*  e4z99wkeb trainer/global_step2:Š#ô\  e4z99wkeZK (H¨¸ (H¨¸(8hpð"3.12.10*0.29.0j linux-x86_64™¤²/  e4z99wkeb*"trainer/global_step2VòýµJ  e4z99wkej9 ¼·éÔ¨½å *LOCAL_RANK: 0 - CUDA_VISIBLE_DEVICES: [0] l•å“  e4z99wkej  ¼·éÔèß¡„î â”â”â”â”┳â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”┳â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”┳â”â”â”â”â”â”â”â”┳â”â”â”â”â”â”â”┳â”â”â”â”â”â”â”┓ ┃ ┃ Name ┃ Type ┃ Params ┃ Mode ┃ FLOPs ┃ ┡â”â”â”╇â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”╇â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”╇â”â”â”â”â”â”â”â”╇â”â”â”â”â”â”â”╇â”â”â”â”â”â”â”┩ │ 0 │ embedding │ MultiInputVectorEmbed… │ 220 K │ train │ 0 │ │ 1 │ encoder │ JetEncoderWithPairwise │ 332 K │ train │ 0 │ │ 2 │ branch_decoders │ ModuleList │ 13.6 M │ train │ 0 │ │ 3 │ regression_decoder │ RegressionDecoder │ 0 │ train │ 0 │ │ 4 │ classification_decoder │ ClassificationDecoder │ 0 │ train │ 0 │ │ │ other params │ n/a │ 48 │ n/a │ n/a │ └───┴────────────────────────┴────────────────────────┴────────┴───────┴───────┘ x“¶A\ e4z99wkejÊ ¼·éÔøý®…·Trainable params: 14.1 M Non-trainable params: 86 Total params: 14.1 M Total estimated model params size (MB): 56.496 Modules in train mode: 5866 Modules in eval mode: 0 Total FLOPs: 0 7üáËq e4z99wkej` ¼·éÔ¸èÊ…Pwandb: WARNING Config item 'hidden_dim' was locked by 'sweep' (ignored update). .+6Ô{ e4z99wkejj ¼·éÔøöÚ…Zwandb: WARNING Config item 'num_embedding_layers' was locked by 'sweep' (ignored update). ¼7Éjy e4z99wkejh ¼·éÔð´ã…Xwandb: WARNING Config item 'num_encoder_layers' was locked by 'sweep' (ignored update). 3YW€ e4z99wkejo ¼·éÔ¨·ê…_wandb: WARNING Config item 'num_branch_encoder_layers' was locked by 'sweep' (ignored update). û]øÇz e4z99wkeji ¼·éÔˆƒñ…Ywandb: WARNING Config item 'num_attention_heads' was locked by 'sweep' (ignored update). wƒ«µt e4z99wkejc ¼·éÔðòø…Swandb: WARNING Config item 'dataset_limit' was locked by 'sweep' (ignored update). ¢¦¹œq e4z99wkej` ¼·éÔ¸Æÿ…Pwandb: WARNING Config item 'batch_size' was locked by 'sweep' (ignored update). WùôNt e4z99wkejc ¼·éÔøò…†Swandb: WARNING Config item 'learning_rate' was locked by 'sweep' (ignored update). üNn e4z99wkej] ¼·éÔ¸ŸŒ†Mwandb: WARNING Config item 'dropout' was locked by 'sweep' (ignored update). |°ùm e4z99wkej\ ¼·éÔ¨³•†Lwandb: WARNING Config item 'epochs' was locked by 'sweep' (ignored update). 'eËÙ e4z99wke*Ç  transformer_dim‚64  transformer_dim_scale‚2.0  initial_embedding_dim‚64  position_embedding_dim‚64 ! num_branch_embedding_layers‚3  num_jet_embedding_layers‚0  num_jet_encoder_layers‚2  num_detector_layers‚2  num_regression_layers‚3  num_classification_layers‚3  split_symmetric_attention‚1 ! transformer_activation‚"gelu"  skip_connections‚1 ( "initial_embedding_skip_connections‚1  linear_block_type‚"GRU"  transformer_type‚"Gated"  linear_activation‚"gelu"  normalization‚ "LayerNorm"  masking‚ "Filling" linear_prelu_activation‚true  use_pairwise_interactions‚1  num_pairwise_features‚4  pairwise_embedding_dim‚8  pairwise_input_source‚"all"  pairwise_block_embeddings‚1  pairwise_cross_type‚true  pairwise_feature_set‚""  pairwise_eq2to2‚false  use_moe‚false  num_experts‚8  num_experts_per_tok‚2  moe_loss_scale‚0.01 F event_info_file‚2"/data/spatop/event_files/tttt/tttt_hadronic.yaml" = training_file‚+"/data/spatop/tttt_15M/h5/tttt_training.h5"  validation_file‚""  testing_file‚""  normalize_features‚1  limit_to_num_jets‚0  balance_particles‚0  balance_jets‚0 ! balance_classifications‚false  partial_events‚1  dataset_randomization‚0  train_validation_split‚0.95  limit_val_batches‚1.0  num_dataloader_workers‚0  mask_sequence_vectors‚1  combine_pair_loss‚"min"  optimizer‚"AdamW"  focal_gamma‚0.0  combinatorial_scale‚0.0 # learning_rate_warmup_epochs‚1.0  learning_rate_cycles‚1  assignment_loss_scale‚1.0  detection_loss_scale‚1.0  kl_loss_scale‚0.0  regression_loss_scale‚0.0 ! classification_loss_scale‚0.0 ! use_mass_classification‚false  mass_classes‚[]  balance_losses‚true  l2_penalty‚0.0002  gradient_clip‚10.0 num_gpu‚1  verbose_output‚false  usable_gpus‚""  trial_time‚"" $ trial_output_dir‚"./test_output" fI{ë e4z99wkejÙ ¼·éÔ×ÍŒÈ/opt/conda/lib/python3.12/site-packages/pytorch_lightning/utilities/_pytree.py:21: `isinstance(treespec, LeafSpec)` is deprecated, use `isinstance(treespec, TreeSpec) and treespec.is_leaf()` instead. ²›õN e4z99wkej¼ ¼·éÔØ•ÛŒ«/opt/conda/lib/python3.12/site-packages/pytorch_lightning/trainer/connectors/data_connector.py:434: The 'val_dataloader' does not have many workers which may be a bottleneck. Consider increasing the value of the `num_workers` argument` to `num_workers=3` in the `DataLoader` to improve performance. 7T¼ë e4z99wkejÙ À·éÔ€·À§È/opt/conda/lib/python3.12/site-packages/pytorch_lightning/utilities/_pytree.py:21: `isinstance(treespec, LeafSpec)` is deprecated, use `isinstance(treespec, TreeSpec) and treespec.is_leaf()` instead. }ZBP e4z99wkej¾ À·éÔè…Ó§­/opt/conda/lib/python3.12/site-packages/pytorch_lightning/trainer/connectors/data_connector.py:434: The 'train_dataloader' does not have many workers which may be a bottleneck. Consider increasing the value of the `num_workers` argument` to `num_workers=3` in the `DataLoader` to improve performance. ±]µT’‚(:ˆ Ê·éÔŽúź gpu.0.gpu‚15.0 gpu.0.memory‚2, gpu.0.memoryAllocated‚16.713943481445312* gpu.0.memoryAllocatedBytes‚ 14357168128 gpu.0.temp‚34.0 gpu.0.powerWatts‚169.583' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚33.9166 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0Ók¯J‚(:À Ê·éÔ…ìýÃ% disk./.usageGB‚1172.6421508789062 disk.sda2.in‚0 disk.sda1.in‚0 disk.sda.in‚0* proc.memory.percent‚17.832398414611816 disk.sda2.out‚0 disk.sda3.in‚0 disk.sda3.out‚ 0.54296875 disk.nvme0c0n1.in‚ 157.84375 disk.dm-0.in‚ 157.84375# proc.memory.rssMB‚ 2921.66015625 proc.cpu.threads‚17# disk.nvme0c0n1.out‚ 595.62109375! disk.nvme0n1.out‚ 595.62109375 disk.dm-1.in‚0 disk.sda1.out‚0 disk.sda.out‚ 0.54296875$ memory_percent‚89.61329460144043( proc.memory.availableMB‚ 1701.7578125 cpu‚26.26077614319403 network.recv‚69073 disk.nvme0n1.in‚ 157.84375 disk.dm-0.out‚ 595.62109375 disk.dm-1.out‚ 0.54296875 disk.drbd1260.in‚0 disk.drbd1260.out‚0 network.sent‚54727) disk./.usagePercent‚78.69699865778323 â×á ‚(:“ Ù·éÔÌĺ gpu.0.gpu‚45.0 gpu.0.memory‚4, gpu.0.memoryAllocated‚16.716384887695312* gpu.0.memoryAllocatedBytes‚ 14359265280 gpu.0.temp‚35.0 gpu.0.powerWatts‚163.148' gpu.0.enforcedPowerLimitWatts‚500.0) gpu.0.powerPercent‚32.629599999999996 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0H× US!‚(:É Ù·éÔÀ»¨½ disk.drbd1260.in‚0$ memory_percent‚89.61691856384277# proc.memory.rssMB‚ 2921.69140625* proc.memory.percent‚17.832589149475098 network.sent‚58441 disk.nvme0c0n1.in‚ 157.84375$ disk.nvme0c0n1.out‚ 789.650390625 disk.nvme0n1.in‚ 157.84375 disk.dm-1.out‚ 0.9140625 disk.drbd1260.out‚ 0.00390625 disk.sda1.out‚0( proc.memory.availableMB‚ 1701.1640625 network.recv‚73129$ disk./.usageGB‚1172.642234802246 disk.sda3.in‚0" disk.nvme0n1.out‚ 789.650390625 disk.dm-0.out‚ 789.650390625 disk.dm-1.in‚0 disk.sda1.in‚0 disk.sda.in‚0 disk.sda2.in‚0 disk.sda3.out‚ 0.9140625 disk.sda.out‚ 0.9140625 cpu‚26.140530114025236 proc.cpu.threads‚17) disk./.usagePercent‚78.69700428994896 disk.sda2.out‚0 disk.dm-0.in‚ 157.84375¥œ"‚(:† è·éÔ»÷ì¸ gpu.0.gpu‚51.0 gpu.0.memory‚6, gpu.0.memoryAllocated‚16.716384887695312* gpu.0.memoryAllocatedBytes‚ 14359265280 gpu.0.temp‚34.0 gpu.0.powerWatts‚157.71' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚31.542 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0¶­Ý¬V#‚(:Ì è·éÔ²Û¹¾ disk.nvme0n1.in‚ 157.84375 disk.dm-0.out‚1094.087890625 disk.dm-1.in‚0 disk.drbd1260.out‚ 0.00390625% disk.nvme0c0n1.out‚1094.087890625) disk./.usagePercent‚78.70584295006861% disk./.usageGB‚1172.7739372253418 disk.sda1.in‚0 disk.sda1.out‚0 disk.sda3.in‚0" proc.memory.rssMB‚ 2921.6953125* proc.memory.percent‚17.832612991333008 network.recv‚78486 disk.sda.in‚0 disk.sda3.out‚ 1.5234375# disk.nvme0n1.out‚1094.087890625$ memory_percent‚89.61901664733887( proc.memory.availableMB‚ 1700.8203125 cpu‚26.035749090475015 proc.cpu.threads‚17 disk.sda.out‚ 1.5234375 disk.sda2.out‚0 disk.nvme0c0n1.in‚ 157.84375 disk.dm-0.in‚ 157.84375 disk.dm-1.out‚ 1.5234375 disk.drbd1260.in‚0 network.sent‚67241 disk.sda2.in‚03c€Ï3$b/ lr-AdamW/pg1"trainer/global_step2:JP˜‰7&3%b/ lr-AdamW/pg2"trainer/global_step2:JP”Î “-&) _wandbruntime‚59  _runtime‚59a3íW¿'º _runtime‚ 59.71661149 _step‚0 & lr-AdamW/pg1‚0.0001058269233194917 & lr-AdamW/pg2‚0.0001058269233194917 trainer/global_step‚49 % _timestamp‚1.7885009836340501e+09BJ¾Á(¼ _step‚0 & lr-AdamW/pg1‚0.0001058269233194917 & lr-AdamW/pg2‚0.0001058269233194917 trainer/global_step‚49 % _timestamp‚1.7885009836340501e+09 _runtime‚ 59.71661149\_»n)bj GDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak"trainer/global_step2:JPãé7e*ba >DeviceStatsMonitor.on_train_batch_start/allocation.all.current"trainer/global_step2:JPBµ9v+br ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current"trainer/global_step2:JPëá|_,b[ 8DeviceStatsMonitor.on_train_batch_start/segment.all.peak"trainer/global_step2:JP?ÛÉ=e-ba >DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak"trainer/global_step2:JPí"®ìp.bl IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated"trainer/global_step2:JPÅ3Vm/bi FDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak"trainer/global_step2:JPˆØùÒv0br ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current"trainer/global_step2:JP¢ïÕc1b_ DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak"trainer/global_step2:JPš)Äøj4bf CDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current"trainer/global_step2:JPžo5bk HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed"trainer/global_step2:JP¸yèžo6bk HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed"trainer/global_step2:JPJM+Ìf7bb ?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed"trainer/global_step2:JP…·¸i8be BDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated"trainer/global_step2:JP»*ljn9bj GDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current"trainer/global_step2:JPöëìs:bo LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated"trainer/global_step2:JPÛ/ãih;bd ADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed"trainer/global_step2:JP´û‘÷qbd ADeviceStatsMonitor.on_train_batch_start/active.small_pool.current"trainer/global_step2:JPt~¬Lh?bd ADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak"trainer/global_step2:JP£•×l@bh EDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated"trainer/global_step2:JP{›íçlAbh EDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current"trainer/global_step2:JPßwõnBbj GDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed"trainer/global_step2:JP‚Ç9úmCbi FDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak"trainer/global_step2:JPÝà·ücDb_ DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed"trainer/global_step2:JP?‚OÛx`bt QDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated"trainer/global_step2:JP‡@C'jabf CDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated"trainer/global_step2:JPù!ýànbbj GDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed"trainer/global_step2:JP ãAicbe BDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current"trainer/global_step2:JP‹îž^kdbg DDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated"trainer/global_step2:JP=Ögebc @DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current"trainer/global_step2:JP|€N`dfb` =DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak"trainer/global_step2:JPfH½}rgbn KDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated"trainer/global_step2:JPµßÝehba >DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak"trainer/global_step2:JPF=àèoibk HDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed"trainer/global_step2:JPƒù‹^jbZ 7DeviceStatsMonitor.on_train_batch_start/active.all.peak"trainer/global_step2:JPhÂikbe BDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current"trainer/global_step2:JPê0\ mlbi FDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak"trainer/global_step2:JPEeýsgmbc @DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak"trainer/global_step2:JPjôŠöqnbm JDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current"trainer/global_step2:JP¿ axobt QDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated"trainer/global_step2:JP+pÎägpbc @DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak"trainer/global_step2:JP@Iljrqbn KDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated"trainer/global_step2:JP'jrÞirbe BDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current"trainer/global_step2:JP ¬msbi FDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak"trainer/global_step2:JP¥²¯ùotbk HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed"trainer/global_step2:JP*¨”subo LDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak"trainer/global_step2:JPÇñ ¤avb] :DeviceStatsMonitor.on_train_batch_start/active.all.current"trainer/global_step2:JPœV"¢fwbb ?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed"trainer/global_step2:JPvŠ½ºnxbj GDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated"trainer/global_step2:JPHâ<_yb[ 8DeviceStatsMonitor.on_train_batch_start/num_device_alloc"trainer/global_step2:JP:­ãòizbe BDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed"trainer/global_step2:JP¥Åùnn{bj GDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak"trainer/global_step2:JPUFRIf|bb ?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed"trainer/global_step2:JP'¤Jh}bd ADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed"trainer/global_step2:JPtžÂk~bg DDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated"trainer/global_step2:JPÔ09ônbj GDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed"trainer/global_step2:JP™dóDm€bh EDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak"trainer/global_step2:JP¶á'‚eb` =DeviceStatsMonitor.on_train_batch_start/segment.all.allocated"trainer/global_step2:JPN·¬.r‚bm JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated"trainer/global_step2:JP6¹M’bƒb] :DeviceStatsMonitor.on_train_batch_start/num_oom_rejections"trainer/global_step2:JP1¢±æt„bo LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated"trainer/global_step2:JP—R,o…bj GDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak"trainer/global_step2:JP¦=Šs†bn KDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated"trainer/global_step2:JPÞRÔžm‡bh EDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed"trainer/global_step2:JPq*mˆbh EDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed"trainer/global_step2:JP‚É4åc‰b^ ;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak"trainer/global_step2:JPÐ Ê}kŠbf CDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed"trainer/global_step2:JPW\u‹bp MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed"trainer/global_step2:JP!¾•ósŒbn KDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated"trainer/global_step2:JP»ŒêKtbo LDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak"trainer/global_step2:JP1k”}kŽbf CDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current"trainer/global_step2:JP]--.ibd ADeviceStatsMonitor.on_train_batch_start/active.large_pool.current"trainer/global_step2:JP›Ü¯lbg DDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak"trainer/global_step2:JPâç“Ah‘bc @DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated"trainer/global_step2:JPi-+&k’bf CDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed"trainer/global_step2:JP.š­´i“bd ADeviceStatsMonitor.on_train_batch_start/oversize_segments.current"trainer/global_step2:JPæâeh”bc @DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed"trainer/global_step2:JPN_Ȇl•bg DDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated"trainer/global_step2:JPéZ¼h–bc @DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed"trainer/global_step2:JP[‚˜j—be BDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak"trainer/global_step2:JPÄÎ' l˜bg DDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated"trainer/global_step2:JP$P½ g™bb ?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak"trainer/global_step2:JPô\+lšbg DDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak"trainer/global_step2:JP[ jo›bj GDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak"trainer/global_step2:JP bDÂmœbh EDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current"trainer/global_step2:JPO`°¹hbc @DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed"trainer/global_step2:JP¾7Þqžbl IDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current"trainer/global_step2:JPê™(ítŸbo LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated"trainer/global_step2:JP0•¨wq bl IDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current"trainer/global_step2:JPp3_nu¡bp MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed"trainer/global_step2:JP©ÜUšj¢be BDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current"trainer/global_step2:JPk†3g£bb ?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak"trainer/global_step2:JP¶;™!.¤) _wandbruntime‚59  _runtime‚59ŽŽÄÊæ¥O TDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 13805551616 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 SEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 384795648 E=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚894 \JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚ 4064065282048 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 ^LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚ 3866960901904 WGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 12589930496 [KDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 13207863296 WEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚ 3953513202176 UEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 48186472448 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚30635 OCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚1008573 ]MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 50647683072 VKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚582054 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 315220480 QCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 243625384 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚2 TDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 13035530240 L@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚2256944 OCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚1223575 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚609 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚3689 ODDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚884672 O?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 13805551616 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 586243584 WGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 13035530240 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 trainer/global_step‚49 K@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚884064 YIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 13207863296 \LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 48417055744 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚2 _MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚ 4013107379200 RBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 13805551616 G?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚609 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 586243584 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚24796 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚24903680 @8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚894 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚30390 [IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚ 3953530241536 MFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚33 ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 285316096 BDeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚3689 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 247622656 ZHDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚ 3866943862544 ZHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚ 3953513202176 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 TBDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated‚ 4001947297280 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 230583296 ^LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚ 3953530241536 SADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚ 3914859735236 XJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 226586024 C;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚894 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚24794 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 WEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚ 3915103360620 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚24794 RGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚582052 TFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 597688320 HDeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚ 4001699674624 cQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚ 4013132282880 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 JBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚609 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚285 N@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 247622656 M=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 13567452160 VKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚302618 D>DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 XHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 47915872692 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚30635 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚608 OFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚1275 P@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 13567452160 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 230583296 aQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 50932999168 P@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 13118517701 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 597688320 JBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚285 _step‚1 VFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 13207863296 XHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 48186472448 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 152061440 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚24796 K?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚1008573 SGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚1008575 @8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚894 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 UGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 582234445 K?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚1223575 SADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚ 4001699674624šߟš'¦”O F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 OCDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated‚1248369 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚30390 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 _runtime‚ 59.716891888 WEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚ 4001947297280 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 XFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚ 4063755062272 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 597688320 HDeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚ 4001699674624 cQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚ 4013132282880 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 JBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚609 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚285 N@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 247622656 M=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 13567452160 VKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚302618 D>DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 XHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 47915872692 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚30635 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚608 OFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚1275 P@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 13567452160 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 230583296 aQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 50932999168 P@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 13118517701 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 597688320 JBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚285 _step‚1 VFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 13207863296 XHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 48186472448 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 152061440 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚24796 K?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚1008573 SGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚1008575 @8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚894 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 UGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 582234445 K?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚1223575 SADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚ 4001699674624 TDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 13805551616 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 SEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 384795648 E=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚894 \JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚ 4064065282048 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 ^LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚ 3866960901904 WGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 12589930496 [KDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 13207863296 WEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚ 3953513202176 UEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 48186472448 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚30635 OCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚1008573 ]MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 50647683072 VKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚582054 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 315220480 QCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 243625384 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚2 TDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 13035530240 L@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚2256944 OCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚1223575 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚609 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚3689 ODDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚884672 O?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 13805551616 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 586243584 WGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 13035530240 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 trainer/global_step‚49 K@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚884064 YIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 13207863296 \LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 48417055744 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚2 _MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚ 4013107379200 RBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 13805551616 G?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚609 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 586243584 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚24796 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚24903680 @8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚894 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚30390 [IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚ 3953530241536 MFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚33 ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 285316096 BDeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚3689 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 247622656 ZHDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚ 3866943862544 ZHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚ 3953513202176 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 TBDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated‚ 4001947297280 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 230583296 ^LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚ 3953530241536 SADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚ 3914859735236 XJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 226586024 C;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚894 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚24794 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 WEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚ 3915103360620 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚24794 RGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚582052 TFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 597688320 HDeviceStatsMonitor.on_train_batch_end/active_bytes.all.current"trainer/global_step2:JPºt1Ãi¯bd ADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current"trainer/global_step2:JP+ Ûf°ba >DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated"trainer/global_step2:JPž̈k±bf CDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current"trainer/global_step2:JP‹2e²b` =DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak"trainer/global_step2:JP^æìtq³bl IDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated"trainer/global_step2:JPðwòžk´bf CDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak"trainer/global_step2:JPktZ"sµbn KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed"trainer/global_step2:JP·©Œ°m¶bh EDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed"trainer/global_step2:JP\¨Rf·ba >DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed"trainer/global_step2:JPˆŒë/j¸be BDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak"trainer/global_step2:JPuK}m¹bh EDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated"trainer/global_step2:JP)öDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak"trainer/global_step2:JP“šÁ{eÏb` =DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak"trainer/global_step2:JP’—«"gÐbb ?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed"trainer/global_step2:JPP³ý¢^ÑbY 6DeviceStatsMonitor.on_train_batch_end/active.all.freed"trainer/global_step2:JP1[*kÒbf CDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed"trainer/global_step2:JPêÎGibÓb] :DeviceStatsMonitor.on_train_batch_end/allocation.all.freed"trainer/global_step2:JP~rYÌdÔb_ DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak"trainer/global_step2:JPÕ‰p%bãb] :DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams"trainer/global_step2:JPuðŽ'häbc @DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current"trainer/global_step2:JPpˆiæfåba >DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed"trainer/global_step2:JPg%+¼kæbf CDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed"trainer/global_step2:JP¹Sêlçbg DDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak"trainer/global_step2:JP€dÖheèb` =DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed"trainer/global_step2:JPyqû pébk HDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current"trainer/global_step2:JPº€…@jêbe BDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak"trainer/global_step2:JP<²Ãmëbh EDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current"trainer/global_step2:JPcùÚpìbk HDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current"trainer/global_step2:JP"ª²híbc @DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated"trainer/global_step2:JPñWV>sîbn KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed"trainer/global_step2:JPfQ¼wïbr ODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated"trainer/global_step2:JPÛW ]pðbk HDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current"trainer/global_step2:JP‚'>!eñb` =DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak"trainer/global_step2:JPë©\Édòb_ €jilóbg DDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak"trainer/global_step2:JPlç üiôbd ADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current"trainer/global_step2:JP–úä.jõbe BDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated"trainer/global_step2:JPäú!/möbh EDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak"trainer/global_step2:JP¤ÙF0o÷bj GDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current"trainer/global_step2:JPØ+Þüjøbe BDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current"trainer/global_step2:JPÉÕêÞgùbb ?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak"trainer/global_step2:JP8LL½dúb_ DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed"trainer/global_step2:JPº‡Ô£iþbd ADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated"trainer/global_step2:JPåô±^aÿb\ 9DeviceStatsMonitor.on_train_batch_end/segment.all.current"trainer/global_step2:JPÚE"dw€br ODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated"trainer/global_step2:JPbWh _bZ 7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries"trainer/global_step2:JP;r‚bm JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated"trainer/global_step2:JP¡å¸eƒb` =DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak"trainer/global_step2:JP;¬l„bg DDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak"trainer/global_step2:JPäv&ýa…b\ 9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak"trainer/global_step2:JP–íýŽp†bk HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated"trainer/global_step2:JPFµo'g‡bb ?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current"trainer/global_step2:JPØtZšgˆbb ?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed"trainer/global_step2:JP>°±n‰bi FDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed"trainer/global_step2:JP)³qŠbl IDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated"trainer/global_step2:JPIõÞj‹be BDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated"trainer/global_step2:JPáÞ¬ßrŒbm JDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak"trainer/global_step2:JPª®i^bY 6DeviceStatsMonitor.on_train_batch_end/num_device_alloc"trainer/global_step2:JP›.æ_ŽbZ 7DeviceStatsMonitor.on_train_batch_end/segment.all.freed"trainer/global_step2:JPC¤énbi FDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed"trainer/global_step2:JP¦òKürbm JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated"trainer/global_step2:JPâL5Òm‘bh EDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current"trainer/global_step2:JP[«¬}V’bQ .DeviceStatsMonitor.on_train_batch_end/num_ooms"trainer/global_step2:JPÁ jo“bj GDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current"trainer/global_step2:JP|0MÌk”bf CDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current"trainer/global_step2:JP#ï÷2l•bg DDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated"trainer/global_step2:JPÇÂÖq–bl IDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated"trainer/global_step2:JP±a#j—be BDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated"trainer/global_step2:JPùš h˜bc @DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current"trainer/global_step2:JPqŠýAm™bh EDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed"trainer/global_step2:JP–¼U`šb[ 8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections"trainer/global_step2:JP6ôu±f›ba >DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed"trainer/global_step2:JPwøIkiœbd ADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated"trainer/global_step2:JPø†ËÂkbf CDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated"trainer/global_step2:JP°± pžbk HDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current"trainer/global_step2:JP SšrŸbm JDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak"trainer/global_step2:JP\0Gce b` =DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed"trainer/global_step2:JP®ŸCi¡bd ADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated"trainer/global_step2:JP%Оm¢bh EDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak"trainer/global_step2:JP…7d£b_ 6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚894 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 VFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 49143943680 \JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚ 3943807636448 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17039360 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 WGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 13207863296 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚24820 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 YIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 13207863296 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚609 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚285 PEDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚593545 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 MADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚1028532 UCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚ 4081415990784 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚17039360 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 152061440 I=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚1028530 MADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚1272980 UEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 13035530240 FDeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 247635968 OADeviceStatsMonitor.on_train_batch_end/allocated_bytS†Â?es.all.current‚ 247635968 _step‚2 J>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚2301512 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1297 TIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚308633 QCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 384795648 ]KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚ 4092702239232 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 PBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 586244608 QEDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚1028532 MADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚1248160 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚2 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 306012160 % _timestamp‚1.7885009847959006e+09 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚30637 NDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 13118517709 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚285 Q?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚ 4081168354816 B6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚2276690 SCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 49143943680 F:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚2276690 _runtime‚ 60.875440548 FDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 13567453184 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚609 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 UCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚ 4032024411136 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚33 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚17039360 RBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 13035530240 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 230596608 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 230596608 R@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚ 4081415990784 [KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 51653662208 aODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚ 4092727142912 VHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 226586232 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚609 E6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚894 trainer/global_step‚49 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚30392 I>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚901592 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚894÷f¤&¦žM OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 243625592 RBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 13805551616 SEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 586244608 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚586 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 B6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚894 trainer/global_step‚49 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚30392 I>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚901592 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚894 _ODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 51934770688 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 ZJDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 49094506318 M=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 13805551616 TDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 13207863296 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚30637 ZHDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚ 4144661913600 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 Q?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚ 3992658517174 VFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 48867920086 WIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 597688320 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚285 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 315220480 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚894 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 VFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 49143943680 \JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚ 3943807636448 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17039360 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 WGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 13207863296 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚24820 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 YIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 13207863296 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚609 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚285 PEDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚593545 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 MADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚1028532 UCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚ 4081415990784 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚17039360 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 152061440 I=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚1028530 MADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚1272980 UEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 13035530240 FDeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 247635968 OADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 247635968 _step‚2 J>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚2301512 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1297 TIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚308633 QCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 384795648 ]KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚ 4092702239232 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 PBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 586244608 QEDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚1028532 MADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚1248160 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚2 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 306012160 % _timestamp‚1.7885009847959006e+09 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚30637 NDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 13118517709 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚285 Q?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚ 4081168354816 B6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚2276690 SCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 49143943680 F:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚2276690 _runtime‚ 60.875440548 FDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 13567453184 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚609 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 UCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚ 4032024411136 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚33 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚17039360 RBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 13035530240 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 230596608 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 230596608 R@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚ 4081415990784 [KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 51653662208 aODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚ 4092727142912 VHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 226586232 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚609 E loss/SRqqt1/assignment_loss"trainer/global_step2:JPê-jC¨b> loss/SRqqt2/assignment_loss"trainer/global_step2:JPþj‹…A©b< loss/FBt3/assignment_loss"trainer/global_step2:JPdÜîaAªb< loss/FRt2/assignment_loss"trainer/global_step2:JPo‚pC«b> loss/SRqqt4/assignment_loss"trainer/global_step2:JP âÈßA¬b< loss/FBt2/assignment_loss"trainer/global_step2:JP±Jœ@­b; loss/FBt3/detection_loss"trainer/global_step2:JPÓd×÷-®b( epoch"trainer/global_step2:JP¸ƒÑlB¯b= loss/SRqqt4/detection_loss"trainer/global_step2:JPn 7ÏA°b< loss/FRt1/assignment_loss"trainer/global_step2:JP˜¦“A±b< loss/FBt4/assignment_loss"trainer/global_step2:JP;»i@²b; loss/FRt3/detection_loss"trainer/global_step2:JP­5(B³b= loss/SRqqt2/detection_loss"trainer/global_step2:JP.š†ê@´b; loss/FBt1/detection_loss"trainer/global_step2:JPniže@µb; loss/FBt4/detection_loss"trainer/global_step2:JPdµ ¢A¶b< loss/FRt4/assignment_loss"trainer/global_step2:JPÅS>D@·b; loss/FRt2/detection_loss"trainer/global_step2:JPÇjÆŽ@¸b; loss/FRt1/detection_loss"trainer/global_step2:JP‡™Þ@¹b; loss/FRt4/detection_loss"trainer/global_step2:JPˆ1°{Aºb< loss/FRt3/assignment_loss"trainer/global_step2:JP5{B»b= loss/SRqqt3/detection_loss"trainer/global_step2:JPy±H@¼b; loss/FBt2/detection_loss"trainer/global_step2:JPÖð}ñ7½b2 loss/total_loss"trainer/global_step2:JPn‡ÄC¾b> loss/SRqqt3/assignment_loss"trainer/global_step2:JP¼J«\A¿b< loss/FBt1/assignment_loss"trainer/global_step2:JP $ZÉBÀb= loss/SRqqt1/detection_loss"trainer/global_step2:JP(N³‚.Á) _wandbruntime‚60  _runtime‚60>2š*LÂÆ /loss/FRt4/assignment_loss‚7.216680661687311 /loss/FRt2/detection_loss‚3.8920772838923665 _runtime‚ 60.878085721 .loss/FRt1/detection_loss‚5.071290171083069 .loss/FRt4/detection_loss‚3.039144439218601 trainer/global_step‚49 _step‚3 /loss/FRt3/assignment_loss‚7.442895107491072 1loss/SRqqt3/detection_loss‚1.7566061612218618 /loss/FBt2/detection_loss‚2.7758675060855844 &loss/total_loss‚122.82008488349805 1loss/SRqqt3/assignment_loss‚5.558706547663762 0loss/FBt1/assignment_loss‚2.6831808657873246 0loss/SRqqt1/detection_loss‚4.852581491209056 1loss/SRqqt1/assignment_loss‚5.466019499791812 1loss/SRqqt2/assignment_loss‚5.413882115992104 0loss/FBt3/assignment_loss‚2.5955700036641716 0loss/FRt2/assignment_loss‚7.3475263648562965 1loss/SRqqt4/assignment_loss‚5.325008274218358 0loss/FBt2/assignment_loss‚2.6509449630975723 /loss/FBt3/detection_loss‚4.7251141935184195 epoch‚0 /loss/SRqqt4/detection_loss‚2.78544235502908 % _timestamp‚1.7885009847981668e+09 /loss/FRt1/assignment_loss‚7.324054624353137 0loss/FBt4/assignment_loss‚2.7451224327087402 /loss/FRt3/detection_loss‚3.8368875961317572 1loss/SRqqt2/detection_loss‚3.9750432784600958 .loss/FBt1/detection_loss‚15.85005247592926 .loss/FBt4/detection_loss‚8.490386470407248gg¯PÃÊ &loss/total_loss‚122.82008488349805 1loss/SRqqt3/assignment_loss‚5.558706547663762 0loss/FBt1/assignment_loss‚2.6831808657873246 0loss/SRqqt1/detection_loss‚4.852581491209056 1loss/SRqqt1/assignment_loss‚5.466019499791812 1loss/SRqqt2/assignment_loss‚5.413882115992104 0loss/FBt3/assignment_loss‚2.5955700036641716 0loss/FRt2/assignment_loss‚7.3475263648562965 1loss/SRqqt4/assignment_loss‚5.325008274218358 0loss/FBt2/assignment_loss‚2.6509449630975723 /loss/FBt3/detection_loss‚4.7251141935184195 epoch‚0 /loss/SRqqt4/detection_loss‚2.78544235502908 % _timestamp‚1.7885009847981668e+09 /loss/FRt1/assignment_loss‚7.324054624353137 0loss/FBt4/assignment_loss‚2.7451224327087402 /loss/FRt3/detection_loss‚3.8368875961317572 1loss/SRqqt2/detection_loss‚3.9750432784600958 .loss/FBt1/detection_loss‚15.85005247592926 .loss/FBt4/detection_loss‚8.490386470407248 /loss/FRt4/assignment_loss‚7.216680661687311 /loss/FRt2/detection_loss‚3.8920772838923665 _runtime‚ 60.878085721 .loss/FRt1/detection_loss‚5.071290171083069 .loss/FRt4/detection_loss‚3.039144439218601 trainer/global_step‚49 _step‚3 /loss/FRt3/assignment_loss‚7.442895107491072 1loss/SRqqt3/detection_loss‚1.7566061612218618 /loss/FBt2/detection_loss‚2.7758675060855844Ô0Û]“Ä‚(:ˆ †¸éÔ¤ÔšÀ gpu.0.gpu‚24.0 gpu.0.memory‚2, gpu.0.memoryAllocated‚16.716384887695312* gpu.0.memoryAllocatedBytes‚ 14359265280 gpu.0.temp‚31.0 gpu.0.powerWatts‚114.461' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚22.8922 gpu.0.smClock‚1275 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0wlz}WÅ‚(:Ì †¸éÔ¶üÈÇ& disk.nvme0c0n1.out‚1553.2529296875( disk./.usagePercent‚78.7203747057347 disk.nvme0n1.in‚ 159.41796875 disk.sda.in‚0" disk.nvme0c0n1.in‚ 159.41796875! proc.memory.rssMB‚ 2921.734375) proc.memory.percent‚17.83285140991211 network.sent‚118863$ disk.nvme0n1.out‚1553.2529296875 disk.dm-1.in‚0 disk.sda1.out‚0 disk.sda2.in‚0 disk.sda3.out‚2.40625& proc.memory.availableMB‚ 1700.34375! disk.dm-0.out‚1553.2529296875 disk.drbd1260.in‚0 disk.sda1.in‚0 disk.sda3.in‚0# memory_percent‚89.6219253540039 cpu‚25.91666323778243 proc.cpu.threads‚17 network.recv‚116932% disk./.usageGB‚1172.9904708862305 disk.dm-0.in‚ 159.41796875 disk.dm-1.out‚2.40625 disk.drbd1260.out‚ 0.0078125 disk.sda.out‚2.40625 disk.sda2.out‚07K‹“Æ‚(:ˆ •¸éԣЙº gpu.0.gpu‚10.0 gpu.0.memory‚1, gpu.0.memoryAllocated‚16.716384887695312* gpu.0.memoryAllocatedBytes‚ 14359265280 gpu.0.temp‚31.0 gpu.0.powerWatts‚109.358' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚21.8716 gpu.0.smClock‚1275 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0¸X^Ç‚(:Ó •¸éÔçü²¼ disk.sda3.in‚0 disk.sda3.out‚ 2.71484375! disk.nvme0c0n1.in‚ 310.7890625 disk.drbd1260.in‚0 disk.sda1.in‚0 disk.sda.in‚0 disk.sda.out‚ 2.71484375 disk.sda2.in‚0& disk.nvme0c0n1.out‚1867.1748046875$ memory_percent‚89.62249755859375! proc.memory.rssMB‚ 2921.734375 disk.drbd1260.out‚ 0.01171875 disk.nvme0n1.in‚ 310.7890625$ disk.nvme0n1.out‚1867.1748046875! disk.dm-0.out‚1867.1748046875 disk.dm-1.in‚0 disk.sda2.out‚0) proc.memory.percent‚17.83285140991211 cpu‚25.868249977310956 network.recv‚122532 disk.sda1.out‚0# proc.memory.availableMB‚1700.25 proc.cpu.threads‚17 disk.dm-1.out‚ 2.71484375 network.sent‚159843) disk./.usagePercent‚78.73770948789318% disk./.usageGB‚1173.2487716674805 disk.dm-0.in‚ 310.7890625™¿€%žÈ‚(:“ ¤¸éÔߙຠgpu.0.gpu‚56.0 gpu.0.memory‚6, gpu.0.memoryAllocated‚16.716384887695312* gpu.0.memoryAllocatedBytes‚ 14359265280 gpu.0.temp‚35.0 gpu.0.powerWatts‚171.699' gpu.0.enforcedPowerLimitWatts‚500.0) gpu.0.powerPercent‚34.339800000000004 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0,ZÀdÉ‚(:Ù ¤¸éÔ´çµ¼% disk./.usageGB‚1173.1377334594727$ disk.nvme0n1.out‚2166.0537109375 disk.dm-1.in‚0 disk.sda1.in‚0 disk.sda1.out‚0) proc.memory.availableMB‚ 1700.48828125! proc.memory.rssMB‚ 2921.734375) proc.memory.percent‚17.83285140991211 network.sent‚163623 network.recv‚126565 disk.sda3.in‚0& disk.nvme0c0n1.out‚2166.0537109375 disk.nvme0n1.in‚ 310.7890625 disk.dm-1.out‚ 3.14453125 disk.drbd1260.in‚0 disk.drbd1260.out‚ 0.01171875) disk./.usagePercent‚78.73025762058651! disk.nvme0c0n1.in‚ 310.7890625 disk.dm-0.in‚ 310.7890625 disk.sda.in‚0 disk.sda.out‚ 3.14453125 disk.sda2.in‚0 disk.sda2.out‚0$ memory_percent‚89.62104320526123 disk.sda3.out‚ 3.14453125! disk.dm-0.out‚2166.0537109375 cpu‚25.836895126606695 proc.cpu.threads‚17eRžÊ‚(:“ ³¸éÔØžˆ» gpu.0.gpu‚57.0 gpu.0.memory‚7, gpu.0.memoryAllocated‚16.716384887695312* gpu.0.memoryAllocatedBytes‚ 14359265280 gpu.0.temp‚34.0 gpu.0.powerWatts‚151.663' gpu.0.enforcedPowerLimitWatts‚500.0) gpu.0.powerPercent‚30.332600000000003 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0mQ¾½aË‚(:Ö ³¸éÔ©Š‘¾ disk.drbd1260.in‚0 disk.sda.out‚ 3.5234375# memory_percent‚89.6212100982666) proc.memory.percent‚17.83292293548584! disk.nvme0c0n1.in‚ 310.7890625 disk.nvme0n1.in‚ 310.7890625 disk.drbd1260.out‚ 0.01953125 proc.cpu.threads‚17 network.sent‚171941& disk.nvme0c0n1.out‚2353.6826171875 disk.dm-0.in‚ 310.7890625 disk.dm-1.in‚0 disk.dm-1.out‚ 3.5234375( proc.memory.availableMB‚ 1700.4609375# proc.memory.rssMB‚ 2921.74609375 cpu‚25.810618449003872% disk./.usageGB‚1173.3134994506836 disk.sda2.in‚0 disk.sda2.out‚0 disk.sda3.in‚0 disk.sda3.out‚ 3.5234375 disk.sda1.in‚0 disk.sda1.out‚0 disk.sda.in‚0 network.recv‚131512) disk./.usagePercent‚78.74205342373418$ disk.nvme0n1.out‚2353.6826171875! disk.dm-0.out‚2353.6826171875FPŒ6žÌ‚(:“ ¸éÔüúÞ¸ gpu.0.gpu‚40.0 gpu.0.memory‚3, gpu.0.memoryAllocated‚16.716384887695312* gpu.0.memoryAllocatedBytes‚ 14359265280 gpu.0.temp‚32.0 gpu.0.powerWatts‚130.922' gpu.0.enforcedPowerLimitWatts‚500.0) gpu.0.powerPercent‚26.184400000000004 gpu.0.smClock‚1335 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0Øÿî$dÍ‚(:Ù ¸éÔ¿Œñ¼ disk.sda2.in‚0 disk.dm-1.in‚0 disk.drbd1260.in‚0# proc.memory.rssMB‚ 2921.74609375 disk.sda1.in‚0 disk.sda1.out‚0( proc.memory.availableMB‚ 1634.8671875 proc.cpu.threads‚17& disk.nvme0c0n1.out‚2635.9951171875 disk.nvme0n1.in‚ 310.7890625$ disk.nvme0n1.out‚2635.9951171875! disk.dm-0.out‚2635.9951171875 disk.sda2.out‚0 disk.sda3.in‚0 disk.sda3.out‚ 4.07421875! disk.nvme0c0n1.in‚ 310.7890625 network.sent‚176892 disk.dm-1.out‚ 4.07421875 disk.drbd1260.out‚ 0.01953125 disk.sda.in‚0 disk.sda.out‚ 4.07421875$ memory_percent‚90.02156257629395) proc.memory.percent‚17.83292293548584 cpu‚25.79367133372694 disk.dm-0.in‚ 310.7890625 network.recv‚136971) disk./.usagePercent‚78.78239278691194% disk./.usageGB‚1173.9145851135254üŒy0Î+ _wandbruntime‚143  _runtime‚143uÉXÄϾ _runtime‚ 143.669703516 _step‚4 ' lr-AdamW/pg1‚0.00021381357976795265 ' lr-AdamW/pg2‚0.00021381357976795265 trainer/global_step‚99 % _timestamp‚1.7885010675902774e+09¦ÈРtrainer/global_step‚99 % _timestamp‚1.7885010675902774e+09 _runtime‚ 143.669703516 _step‚4 ' lr-AdamW/pg1‚0.00021381357976795265 ' lr-AdamW/pg2‚0.000213813579767952659Ð`0Ñ+ _wandbruntime‚143  _runtime‚143®}wÇÒ–O XHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 95545414400 YIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 13207863296 WEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚ 7975409135104 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 BDeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚ 7975161512448 [KDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 13207863296 YIDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 96318445568 \LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 96318445568 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 VKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚605282 \LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 95772000424 @8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚894 D8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚4462600 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚30637 QCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 243625384 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚24796 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚30392 M=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 13567453184 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 SADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚ 7804826004144 VFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 13207863296 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚24794 \JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚ 8093902992896 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 597688320 TFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 597688320 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 247622656 TBDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated‚ 7975409135104 WGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 13035530240 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚2 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 586244608 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚30637 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 597688320 ?9DeviceStatsMonitor.on_train_batch_start/΢ yÕsegment.all.freed‚0 PDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚1761986 H?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚1308 SGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚2480971 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 152061440 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 E=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚894 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚609 G?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚609 _runtime‚ 143.673659188 HDeviceStatsMonitor.on_train_batch_start/allocation.all.current‚24796 G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚3689 OCDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated‚2480971 [IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚ 7879090689536 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚1156704 ^MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 100744489472 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 315220480 @8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚894 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚24794 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚568 G?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚285 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚17039360 XHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 96087862272 ZHDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚ 7709280589744 XJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 226586024 trainer/global_step‚99 HDeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 K?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚2456177 OCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚2456177 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 308122624 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚285 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚2 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 230583296 ZHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚ 7879073650176 OFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚1286 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚24903680 WEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚ 7805069629528Ï»wæ 'ÓšO UEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 96087862272 P@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 13567453184 XFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚ 8093594870272 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚24796 G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚3689 OCDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated‚2480971 [IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚ 7879090689536 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚1156704 ^MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 100744489472 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 315220480 @8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚894 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚24794 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚568 G?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚285 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚17039360 XHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 96087862272 ZHDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚ 7709280589744 XJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 226586024 trainer/global_step‚99 HDeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 K?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚2456177 OCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚2456177 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 308122624 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚285 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚2 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 230583296 ZHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚ 7879073650176 OFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚1286 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚24903680 WEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚ 7805069629528 XHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 95545414400 YIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 13207863296 WEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚ 7975409135104 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 BDeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚ 7975161512448 [KDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 13207863296 YIDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 96318445568 \LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 96318445568 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 VKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚605282 \LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 95772000424 @8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚894 D8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚4462600 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚30637 QCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 243625384 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚24796 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚30392 M=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 13567453184 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 SADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚ 7804826004144 VFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 13207863296 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚24794 \JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚ 8093902992896 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 597688320 TFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 597688320 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 247622656 TBDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated‚ 7975409135104 WGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 13035530240 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚2 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 586244608 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚30637 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 597688320 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 PDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚1761986 H?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚1308 SGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚2480971 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 152061440 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 E=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚894 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚609 G?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚609 _runtime‚ 143.673659188 HDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 13118517709 RBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 13805551616 R@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚ 8054877828608 F:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚4531964 J>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚4531964 WIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 597688320 YGDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚ 7957601898496 XFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚ 7957584859136 MADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚2026380 MADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚2505582 VFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 97045333504 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 UCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚ 7957584859136 QEDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚2026382 \KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 101742268416 L>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 247635968 UCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚ 7882868411674 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚609 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚609 $ _timestamp‚1.788501069073654e+09 _step‚6 \JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚ 7786144363648 I=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚2480762 N>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 13567453184 VDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚ 8174187509248 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚285 \JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚ 7957601898496 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚604 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚2 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 M=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 13805551616 B6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚4507142 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 PEDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚610759 YIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 13207863296 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 230596608 NBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚1779558 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚30637 PBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 586244608 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚3689 trainer/global_step‚99 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 Q?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚ 8054630192640 Q?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚ 7882624786082 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 aODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚ 8072470144512 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 P@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 13805551616 N6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚894 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚285 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚2 K;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 13567453184 ZJDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 97275930112 F6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚894 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 XFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚ 7786127324288 WGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 13207863296 UEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 13035530240 F:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚4507142 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 318595072 VFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 96497461794 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚285 SEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 586244608 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚1168197 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚1168195 ]KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚ 8072445240832 UCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚ 8054877828608 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚17039360 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚30392 MDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚1286 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 315220480 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 RBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 13035530240 SCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 97045333504 ZHDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚ 8174506104320 ZJDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 96724048026 J>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚1778954 FDeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 aODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚ 8072470144512 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 P@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 13805551616 N6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚894 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚285 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚2 K;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 13567453184 ZJDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 97275930112 F6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚894 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 XFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚ 7786127324288 WGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 13207863296 UEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 13035530240 F:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚4507142 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 318595072 VFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 96497461794 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚285 SEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 586244608 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚1168197 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚1168195 ]KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚ 8072445240832 UCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚ 8054877828608 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚17039360 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚30392 MDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚1286 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 315220480 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 RBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 13035530240 SCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 97045333504 ZHDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚ 8174506104320 ZJDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 96724048026 J>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚1778954 FDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 13118517709 RBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 13805551616 R@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚ 8054877828608 F:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚4531964 J>DeviceStatsMonitor.on_traiÀ·k n_batch_end/allocation.all.allocated‚4531964 WIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 597688320 YGDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚ 7957601898496 XFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚ 7957584859136 MADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚2026380 MADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚2505582 VFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 97045333504 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 UCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚ 7957584859136 QEDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚2026382 \KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 101742268416 L>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 247635968 UCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚ 7882868411674 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚609 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚609 $ _timestamp‚1.788501069073654e+09 _step‚6 \JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚ 7786144363648 I=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚2480762 N>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 13567453184 VDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚ 8174187509248 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚285 \JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚ 7957601898496 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚604 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚2 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 M=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 13805551616 B6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚4507142 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 PEDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚610759 YIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 13207863296 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 230596608CE¸0×+ _wandbruntime‚145  _runtime‚145K>–—KØÅ /loss/FRt3/assignment_loss‚7.392929553985596 0loss/FBt1/assignment_loss‚0.6163432548161257 .loss/FRt2/detection_loss‚2.892421890406505 .loss/FBt2/detection_loss‚6.611329047652703 0loss/SRqqt3/assignment_loss‚5.12969918086611 .loss/FRt1/detection_loss‚4.163432263114801 1loss/SRqqt1/detection_loss‚3.1789992232935145 1loss/SRqqt3/detection_loss‚1.6142200779403297 /loss/FBt3/detection_loss‚1.6851594484222114 .loss/FBt4/detection_loss‚7.905463024610427 % _timestamp‚1.7885010690761223e+09 _runtime‚ 145.154645533 0loss/FRt1/assignment_loss‚7.1887463287070945 /loss/FRt2/assignment_loss‚7.322418830705725 /loss/FRt4/assignment_loss‚7.175548668911582 1loss/SRqqt2/assignment_loss‚4.476698059661715 1loss/SRqqt4/assignment_loss‚4.447720380311601 &loss/total_loss‚105.54068892927287 epoch‚0 trainer/global_step‚99 0loss/SRqqt2/detection_loss‚2.975715851168121 0loss/SRqqt1/assignment_loss‚4.23455669026856 /loss/FBt1/detection_loss‚14.399989671327852 .loss/FRt3/detection_loss‚3.714880782901571 1loss/SRqqt4/detection_loss‚3.4775676450618866 _step‚7 0loss/FBt2/assignment_loss‚0.8804479027166963 0loss/FBt3/assignment_loss‚0.3382680777782448 /loss/FBt4/assignment_loss‚0.566688310718391 /loss/FRt4/detection_loss‚3.1514447639255145§ŠOÙÉ % _timestamp‚1.7885010690761223e+09 _runtime‚ 145.154645533 0loss/FRt1/assignment_loss‚7.1887463287070945 /loss/FRt2/assignment_loss‚7.322418830705725 /loss/FRt4/assignment_loss‚7.175548668911582 1loss/SRqqt2/assignment_loss‚4.476698059661715 1loss/SRqqt4/assignment_loss‚4.447720380311601 &loss/total_loss‚105.54068892927287 epoch‚0 trainer/global_step‚99 0loss/SRqqt2/detection_loss‚2.975715851168121 0loss/SRqqt1/assignment_loss‚4.23455669026856 /loss/FBt1/detection_loss‚14.399989671327852 .loss/FRt3/detection_loss‚3.714880782901571 1loss/SRqqt4/detection_loss‚3.4775676450618866 _step‚7 0loss/FBt2/assignment_loss‚0.8804479027166963 0loss/FBt3/assignment_loss‚0.3382680777782448 /loss/FBt4/assignment_loss‚0.566688310718391 /loss/FRt4/detection_loss‚3.1514447639255145 /loss/FRt3/assignment_loss‚7.392929553985596 0loss/FBt1/assignment_loss‚0.6163432548161257 .loss/FRt2/detection_loss‚2.892421890406505 .loss/FBt2/detection_loss‚6.611329047652703 0loss/SRqqt3/assignment_loss‚5.12969918086611 .loss/FRt1/detection_loss‚4.163432263114801 1loss/SRqqt1/detection_loss‚3.1789992232935145 1loss/SRqqt3/detection_loss‚1.6142200779403297 /loss/FBt3/detection_loss‚1.6851594484222114 .loss/FBt4/detection_loss‚7.905463024610427ÑzÍF‘Ú‚(:† ѸéÔ çκ gpu.0.gpu‚0.0 gpu.0.memory‚0, gpu.0.memoryAllocated‚16.716384887695312* gpu.0.memoryAllocatedBytes‚ 14359265280 gpu.0.temp‚31.0 gpu.0.powerWatts‚93.674' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚18.7348 gpu.0.smClock‚1275 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0-"ø¹_Û‚(:Ô ѸéÔë ½ disk.dm-1.in‚0 disk.dm-1.out‚ 4.41015625 network.recv‚140990) disk./.usagePercent‚78.79239525728224 disk.sda1.in‚0 disk.sda1.out‚0 disk.drbd1260.in‚0 disk.drbd1260.out‚ 0.0234375$ memory_percent‚93.03343296051025# proc.memory.rssMB‚ 2921.74609375 network.sent‚182846% disk./.usageGB‚1174.0636291503906 disk.sda.in‚0 disk.sda3.out‚ 4.41015625# disk.nvme0c0n1.out‚ 2853.0078125 proc.cpu.threads‚17 disk.sda2.in‚0 disk.sda3.in‚0! disk.nvme0n1.out‚ 2853.0078125) proc.memory.availableMB‚ 1141.40234375) proc.memory.percent‚17.83292293548584 cpu‚25.766207015725712 disk.sda.out‚ 4.41015625 disk.sda2.out‚0" disk.nvme0c0n1.in‚ 447.99609375 disk.nvme0n1.in‚ 447.99609375 disk.dm-0.in‚ 447.99609375 disk.dm-0.out‚ 2853.0078125ýå‘Z’Ü‚(:‡ à¸éÔ’…¹ gpu.0.gpu‚8.0 gpu.0.memory‚1, gpu.0.memoryAllocated‚16.716384887695312* gpu.0.memoryAllocatedBytes‚ 14359265280 gpu.0.temp‚34.0 gpu.0.powerWatts‚165.218' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚33.0436 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0Û8pbÝ‚(:× à¸éÔñ»í» network.recv‚146791) disk./.usagePercent‚78.80955569830991 disk.sda2.in‚0" disk.nvme0n1.out‚ 3243.11328125 disk.dm-1.in‚0 disk.sda1.out‚0# proc.memory.rssMB‚ 2921.75390625) proc.memory.percent‚17.83297061920166 network.sent‚231545 disk.sda3.in‚0 disk.sda3.out‚ 4.83984375" disk.nvme0c0n1.in‚ 447.99609375 disk.nvme0n1.in‚ 447.99609375 disk.dm-0.out‚ 3243.11328125 disk.dm-1.out‚ 4.83984375 disk.drbd1260.in‚0% disk./.usageGB‚1174.3193321228027 disk.sda2.out‚0 disk.dm-0.in‚ 447.99609375 disk.drbd1260.out‚ 0.0234375 disk.sda1.in‚0 disk.sda.in‚0 disk.sda.out‚ 4.83984375$ memory_percent‚93.03534030914307$ disk.nvme0c0n1.out‚ 3243.11328125) proc.memory.availableMB‚ 1141.08984375 cpu‚25.750067657530245 proc.cpu.threads‚17r>äYÞ‚(:’ ï¸éÔÒä÷» gpu.0.gpu‚43.0 gpu.0.memory‚4, gpu.0.memoryAllocated‚16.716384887695312* gpu.0.memoryAllocatedBytes‚ 14359265280 gpu.0.temp‚34.0 gpu.0.powerWatts‚168.28' gpu.0.enforcedPowerLimitWatts‚500.0) gpu.0.powerPercent‚33.656000000000006 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0Þþiß‚(:Þ ï¸éÔØìú¿ disk.drbd1260.out‚ 0.02734375 disk.sda1.in‚0$ memory_percent‚93.03534030914307) proc.memory.availableMB‚ 1141.08984375 disk.sda3.in‚0 disk.sda.in‚0 disk.sda.out‚ 5.1953125) proc.memory.percent‚17.83297061920166 cpu‚25.730396269531173 network.recv‚152140% disk./.usageGB‚1174.3183135986328 disk.sda2.in‚0 disk.drbd1260.in‚0 disk.sda1.out‚0# proc.memory.rssMB‚ 2921.75390625 proc.cpu.threads‚17 network.sent‚238042) disk./.usagePercent‚78.80948734429826 disk.sda2.out‚0 disk.sda3.out‚ 5.1953125" disk.nvme0c0n1.in‚ 447.99609375' disk.nvme0c0n1.out‚3797.42041015625% disk.nvme0n1.out‚3797.42041015625 disk.dm-1.in‚0 disk.nvme0n1.in‚ 447.99609375 disk.dm-0.in‚ 447.99609375" disk.dm-0.out‚3797.42041015625 disk.dm-1.out‚ 5.1953125¾=â“à‚(:ˆ þ¸éÔè·Ã¹ gpu.0.gpu‚25.0 gpu.0.memory‚2, gpu.0.memoryAllocated‚16.716384887695312* gpu.0.memoryAllocatedBytes‚ 14359265280 gpu.0.temp‚31.0 gpu.0.powerWatts‚103.887' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚20.7774 gpu.0.smClock‚1275 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0*©âká‚(:à þ¸éÔÚû¾¼ network.recv‚157572( disk./.usagePercent‚78.8136776756187 disk.dm-0.in‚ 447.99609375" disk.dm-0.out‚4053.63916015625 disk.dm-1.in‚0 disk.dm-1.out‚ 5.59765625 disk.sda.out‚ 5.59765625 disk.nvme0n1.in‚ 447.99609375% disk.nvme0n1.out‚4053.63916015625 disk.sda1.in‚0 disk.sda2.in‚0 disk.sda2.out‚0" disk.nvme0c0n1.in‚ 447.99609375$ memory_percent‚93.03386211395264) proc.memory.availableMB‚ 1141.33203125 disk.drbd1260.out‚ 0.02734375 disk.sda.in‚0 cpu‚25.713805067378654 proc.cpu.threads‚17% disk./.usageGB‚1174.3807525634766 disk.drbd1260.in‚0 disk.sda1.out‚0 disk.sda3.in‚0 disk.sda3.out‚ 5.59765625' disk.nvme0c0n1.out‚4053.63916015625# proc.memory.rssMB‚ 2921.75390625) proc.memory.percent‚17.83297061920166 network.sent‚244958À€ÍÊ“â‚(:ˆ ¹éÔØÑ×¹ gpu.0.gpu‚23.0 gpu.0.memory‚2, gpu.0.memoryAllocated‚16.716384887695312* gpu.0.memoryAllocatedBytes‚ 14359265280 gpu.0.temp‚31.0 gpu.0.powerWatts‚106.014' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚21.2028 gpu.0.smClock‚1275 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0 ùÇfã‚(:Û ¹éÔ´Åݽ disk.nvme0n1.in‚ 594.26171875 disk.sda1.in‚0 disk.sda1.out‚0' proc.memory.availableMB‚ 1141.078125 network.recv‚161606 disk.sda3.in‚0% disk.nvme0n1.out‚4213.55517578125" disk.dm-0.out‚4213.55517578125 disk.drbd1260.in‚0 disk.sda.in‚0 disk.sda.out‚ 5.96484375) proc.memory.percent‚17.83297061920166 network.sent‚250420' disk.nvme0c0n1.out‚4213.55517578125 disk.dm-1.in‚0 disk.drbd1260.out‚0.03125# memory_percent‚93.0354118347168# proc.memory.rssMB‚ 2921.75390625 cpu‚25.698169980000962 proc.cpu.threads‚17) disk./.usagePercent‚78.85156448661846% disk./.usageGB‚1174.9452934265137 disk.sda2.in‚0 disk.sda2.out‚0 disk.sda3.out‚ 5.96484375" disk.nvme0c0n1.in‚ 594.26171875 disk.dm-0.in‚ 594.26171875 disk.dm-1.out‚ 5.96484375X•’ä‚(:‡ œ¹éÔµ·˜¹ gpu.0.gpu‚0.0 gpu.0.memory‚0, gpu.0.memoryAllocated‚16.716384887695312* gpu.0.memoryAllocatedBytes‚ 14359265280 gpu.0.temp‚32.0 gpu.0.powerWatts‚113.006' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚22.6012 gpu.0.smClock‚1275 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0T6ÀÏcå‚(:Ø œ¹éÔ”) proc.memory.percent‚17.83299446105957 cpu‚25.68298302049351% disk.nvme0n1.out‚4662.21533203125 network.recv‚165711) disk./.usagePercent‚78.85824167512652 disk.drbd1260.out‚0.03125 disk.sda1.in‚0 disk.sda1.out‚0 disk.sda.in‚0 disk.sda.out‚ 6.6953125 disk.sda3.in‚0 disk.sda3.out‚ 6.6953125" disk.nvme0c0n1.in‚ 594.26171875 disk.dm-1.in‚0 disk.dm-1.out‚ 6.6953125$ memory_percent‚93.03393363952637( proc.memory.availableMB‚ 1141.3203125" proc.memory.rssMB‚ 2921.7578125 disk.dm-0.in‚ 594.26171875 network.sent‚256130% disk./.usageGB‚1175.0447883605957 disk.sda2.in‚0 disk.sda2.out‚0' disk.nvme0c0n1.out‚4662.21533203125 disk.nvme0n1.in‚ 594.26171875 proc.cpu.threads‚17" disk.dm-0.out‚4662.21533203125 disk.drbd1260.in‚0\¡U±0æ+ _wandbruntime‚232  _runtime‚232nÃç½ _step‚8 & lr-AdamW/pg1‚0.0003218002362164136 & lr-AdamW/pg2‚0.0003218002362164136 trainer/global_step‚149 % _timestamp‚1.7885011568314016e+09 _runtime‚ 232.911878852û=ëÇèÁ _runtime‚ 232.911878852 _step‚8 & lr-AdamW/pg1‚0.0003218002362164136 & lr-AdamW/pg2‚0.0003218002362164136 trainer/global_step‚149 % _timestamp‚1.7885011568314016e+09ŒÀa‰0é+ _wandbruntime‚232  _runtime‚232ÇõҴ'ê®O XEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚11804634098176 @8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚894 YHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 143174956108 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 597688320 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚24794 [KDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 13207863296 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 597688320 SEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 384795648 P@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 13118517709 @8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚894 % _timestamp‚1.7885011568356268e+09 Q>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚11948623350272 ZIDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 144219835392 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 586244608 L@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚6717848 ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 293704704 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 _runtime‚ 232.916758428 WGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 13035530240 SGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚1731352 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚30637 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚565 YFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚12123494790144 WGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 12589930496 JBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚609 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚2 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 UGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 582234453 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚30392 L@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚2638532 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 C;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚894 OCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚3688779 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚3689 OCDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated‚3713573 _LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚11804651137536 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 152061440 [HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚11551617316944 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚285 _step‚9 D8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚6693052 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚30392 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 318608384 VFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 13207863296 P@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 13567453184 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚2 HDeviceStatsMonitor.on_train_batch_start/allocation.all.current‚24796 ]JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚12123813398528 ]LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 143401542132 TADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚11948623350272 [HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚11804634098176 MFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚33 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚24796 TFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 597688320 JBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚285 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚24794 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 D>DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 XJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 226586024 E=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚894 G?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚609 bQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 151195112448 TADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚11694792273052 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 QCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 243625384 SGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚3713573 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 315220480 BDeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚3689 OCDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated‚3713573 _LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚11804651137536 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 152061440 [HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚11551617316944 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚285 _step‚9 D8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚6693052 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚30392 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 318608384 VFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 13207863296 P@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 13567453184 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚2 HDeviceStatsMonitor.on_train_batch_start/allocation.all.current‚24796 ]JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚12123813398528 ]LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 143401542132 TADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚11948623350272 [HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚11804634098176 MFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚33 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚24796 TFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 597688320 JBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚285 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚24794 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 D>DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 XJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 226586024 E=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚894 G?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚609 bQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 151195112448 TADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚11694792273052 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 QCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 243625384 SGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚3713573 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 315220480 BDeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚11948623350272 ZIDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 144219835392 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 586244608 L@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚6717848 ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 293704704 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 _runtime‚ 232.916758428 WGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 13035530240 SGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚1731352 E?DeviceStatsMonitor.óE: ¨on_train_batch_start/oversize_segments.freed‚0 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚30637 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚565 YFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚12123494790144 WGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 12589930496 JBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚609 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚2 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 UGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 582234453 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚30392 L@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚2638532 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 C;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚894 OCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚3688779 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 Ôþ×J0ì+ _wandbruntime‚235  _runtime‚235ã[ß¾½&í·M ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚1742845 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 310206464 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚609 L>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 247635968 B6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚6737594 N>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 13567453184 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚560 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 315220480 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚24820 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚2 PBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 586244608 MADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚3024232 MADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚3738184 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 M=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 13805551616 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚285 K;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 13567453184 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 230596608 F:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚6737594 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚30637 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚3689 `ODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 152190844416 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚24822 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 WGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 13207863296 F:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚6762416 MADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚3713364 [MDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚ 285302784 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚24820 S@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚12028339666432 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 230596608 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 582234453 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 UEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 12589930496 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚609 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17039360 YFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚11883145307136 QEDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚3738184 ^KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚12052188242432 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚285 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 ZGDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚11883162346496 QEDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚3024232 PEDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚912891 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚894 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1312 R?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚12028092030464 F8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 I=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚3713364 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚33 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24903680 N>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 13118517709 ]JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚11628481090848 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚894 UEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 13035530240 VHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 226586232 RBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 13805551616 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚609 $ _timestamp‚1.788501159468249e+09 _runtime‚ 235.55251348 VCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚11883145307136 J>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚6762416 trainer/global_step‚149 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚2 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 bODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚12052213146112 WIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 597688320 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 VCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚11772834680582 SEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 586244608 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚30392 J>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚2655736 MDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚1290 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 597688320 MADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚3024230 XGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 145177319936 E6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚894 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 243625592 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚894 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚285 [JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 144353589734 WFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 144127003502~¹wÛÁ&î»M H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚562 WDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚12204093784064 TDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 13207863296 _step‚10 ]JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚11883162346496 F8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 I=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚3713364 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚33 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24903680 N>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 13118517709 ]JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚11628481090848 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚894 UEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 13035530240 VHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 226586232 RBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 13805551616 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚609 $ _timestamp‚1.788501159468249e+09 _runtime‚ 235.55251348 VCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚11883145307136 J>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚6762416 trainer/global_step‚149 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚2 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 bODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚12052213146112 WIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 597688320 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 VCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚11772834680582 SEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 586244608 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚30392 J>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚2655736 MDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚1290 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 597688320 MADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚3024230 XGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 145177319936 E6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚894 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 243625592 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚894 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚285 [JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 144353589734 WFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 144127003502 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚1742845 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 310206464 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚609 L>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 247635968 B6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚6737594 N>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 13567453184 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚560 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 315220480 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚24820 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚2 PBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 586244608 MADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚3024232 MADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚3738184 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 M=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 13805551616 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚285 K;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 13567453184 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 230596608 F:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚6737594 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚30637 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚3689 `ODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 152190844416 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚24822 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 WGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 13207863296 F:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚6762416 MADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚3713364 [MDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚ 285302784 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚24820 S@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚12028339666432 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 230596608 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 582234453 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 UEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 12589930496 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚609 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17039360 YFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚11883145307136 QEDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚3738184 ^KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚12052188242432 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚285 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 ZGDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚11883162346496 QEDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚3024232 PEDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚912891 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚894 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1312 R?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚12028092030464 FDeviceStatsMonitor.on_train_batch_start/allocation.all.current‚24796 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚3689 JBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚609 [HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚15393954044144 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 230583296 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚596 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚24903680 _LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚15393971083504 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚30637 XEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚15730194546176 OCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚4002123 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 TFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 597688320 E=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚894 SGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚2306002 OFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚1290 ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 295801856 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 O?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 13805551616 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 UBDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated‚15922332810752 XEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚15922332810752 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚30392 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 [KDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 13207863296 YIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 13207863296 SGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚4002125 SGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚4946175 SEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 384795648 @8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚894 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 @8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚894 _LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚15730211585536 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 315220480 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 597688320 Q>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚15922085188096 M=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 13567453184 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚30637 UGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 582234453 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚30392 WGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 13035530240 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚24794 `MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚15952336384000 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚609 ]LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 192121225216 L@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚3515626 H?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚1312 YFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚16153332406272 BDeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 PDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚3516224 ]JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚16153653111808 ^MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 200996022272 ]LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 191031083840 G?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚609 trainer/global_step‚199 _runtime‚ 330.170610672 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚17039360 bQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 201291824128 MFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚33 D8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚8923504 OCDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated‚4946175 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚24794 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 586244608 YHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 191890641920 OCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚4921381 ZIDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 192121225216 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 247622656 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚1210220 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 597688320 K?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚4002123 G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚3689 \IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚15730211585536 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 586244608 TDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 13805551616 OCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚4002125 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚2 WGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 12589930496 XJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 226586024b²”v»'…µO QCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 243625384 TADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚15584758541960 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚285 L@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚8948300 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 230583296 G?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚285 K?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚4921381 VEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 191890641920 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 152061440 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 HDeviceStatsMonitor.on_train_batch_start/allocation.all.current‚24796 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚3689 JBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚609 [HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚15393954044144 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 230583296 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚596 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚24903680 _LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚15393971083504 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚30637 XEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚15730194546176 OCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚4002123 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 TFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 597688320 E=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚894 SGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚2306002 OFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚1290 ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 295801856 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 O?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 13805551616 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 UBDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated‚15922332810752 XEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚15922332810752 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚30392 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 [KDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 13207863296 YIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 13207863296 SGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚4002125 SGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚4946175 SEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 384795648 @8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚894 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 @8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚894 _LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚15730211585536 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 315220480 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 597688320 Q>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚15922085188096 M=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 13567453184 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚30637 UGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 582234453 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚30392 WGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 13035530240 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚24794 `MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚15952336384000 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚609 ]LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 192121225216 L@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚3515626 H?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚1312 YFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚16153332406272 BDeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 PDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚3516224 ]JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚16153653111808 ^MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 200996022272 ]LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 191031083840 G?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚609 trainer/global_step‚199 _runtime‚ 330.170610672 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚17039360 bQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 201291824128 MFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚33 D8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚8923504 OCDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated‚4946175 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚24794 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 586244608 YHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 191890641920 OCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚4921381 ZIDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 192121225216 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 247622656 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚1210220 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 597688320 K?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚4002123 G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚3689 \IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚15730211585536 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 586244608 TDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 13805551616 OCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚4002125 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚2 WGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 12589930496 XJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 226586024 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 SGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚1209624 ò¬0†+ _wandbruntime‚331  _runtime‚331%ŠÁ&‡»M ]JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚15808722794496 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚2317497 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 RBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 13035530240 NBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚3533762 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24903680 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚2 `ODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 202284332544 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 B6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚8968046 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚24820 N>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 13118517709 TDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 13207863296 trainer/global_step‚199 SEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 586244608 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 VHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 226586232 S@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚16001801504256 UEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 13035530240 J>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚8992868 \KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 202007418368 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 MADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚4945966 BDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 13567453184 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 230596608 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚2317495 [HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚16234240480256 WFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 191756545210 WFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 192848113152 FDeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚1215703 QCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 384795648 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 243625592 RBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 13805551616 WGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 13207863296 R?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚16001553868288 VCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚15808705755136 YFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚15808705755136 [MDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚ 276914176 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17039360 QEDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚4970786 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚564 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 582234453 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚894 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚894 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚285 MADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚4022082 MDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚1290 UEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 12589930496 ODeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 J>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚3533198 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚2 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 230596608 MADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚4970786 L>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 247635968 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚30637 WIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 597688320 MADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚4022080 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 301817856 WDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚16233938662400 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚609 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚285 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚24820 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚894 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 F:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚8992868 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚24822…]Kˆ¿M GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 VHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 226586232 S@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚16001801504256 UEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 13035530240 J>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚8992868 \KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 202007418368 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 MADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚4945966 BDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 13567453184 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 230596608 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚2317495 [HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚16234240480256 WFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 191756545210 WFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 192848113152 FDeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚1215703 QCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 384795648 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 243625592 RBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 13805551616 WGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 13207863296 R?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚16001553868288 VCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚15808705755136 YFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚15808705755136 [MDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚ 276914176 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17039360 QEDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚4970786 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚564 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 582234453 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚894 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚894 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚285 MADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚4022082 MDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚1290 UEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 12589930496 ODeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 J>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚3533198 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚2 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 230596608 MADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚4970786 L>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 247635968 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚30637 WIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 597688320 MADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚4022080 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 301817856 WDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚16233938662400 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚609 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚285 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚24820 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚894 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 F:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚8992868 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚24822 ]JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚15808722794496 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚2317497 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 RBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 13035530240 NBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚3533762 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24903680 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚2 `ODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 202284332544 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 B6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚8968046 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚24820 N>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 13118517709 TDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 13207863296 trainer/global_step‚199 SEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 586244608Û“2”0‰+ _wandbruntime‚331  _runtime‚331eï•ÒJŠÄ 2loss/SRqqt4/assignment_loss‚3.3524170909609112 -loss/FBt2/detection_loss‚5.53390878879458 trainer/global_step‚199 /loss/FRt3/assignment_loss‚7.293433040700933 -loss/FRt1/detection_loss‚3.77730299477224 0loss/SRqqt3/detection_loss‚4.937845734081098 1loss/SRqqt2/assignment_loss‚3.399207196584562 1loss/SRqqt3/assignment_loss‚3.448597197021757 /loss/FRt2/detection_loss‚2.8698480434996902 .loss/FBt4/detection_loss‚6.145065985461499 _runtime‚ 331.583868851 _step‚15 /loss/FRt4/assignment_loss‚7.184665947817685 /loss/FRt4/detection_loss‚3.1352143805013615 0loss/FBt1/assignment_loss‚0.7558132680831477 -loss/FRt3/detection_loss‚3.08094641894743 0loss/SRqqt2/detection_loss‚2.186550709763466 epoch‚0 % _timestamp‚1.7885012555047274e+09 1loss/SRqqt4/detection_loss‚1.5596714736802304 -loss/FBt3/detection_loss‚2.16068624837512 0loss/FBt3/assignment_loss‚0.2528268960552208 0loss/FBt4/assignment_loss‚0.4626078090625518 0loss/SRqqt1/detection_loss‚3.226589150085885 /loss/FBt1/detection_loss‚13.946439984440804 /loss/FRt2/assignment_loss‚7.436230735958747 2loss/SRqqt1/assignment_loss‚3.2090208299698366 0loss/FBt2/assignment_loss‚0.8491652290523052 %loss/total_loss‚97.45170655406055 /loss/FRt1/assignment_loss‚7.247651400389494KªåN‹È /loss/FRt4/assignment_loss‚7.184665947817685 /loss/FRt4/detection_loss‚3.1352143805013615 0loss/FBt1/assignment_loss‚0.7558132680831477 -loss/FRt3/detection_loss‚3.08094641894743 0loss/SRqqt2/detection_loss‚2.186550709763466 epoch‚0 % _timestamp‚1.7885012555047274e+09 1loss/SRqqt4/detection_loss‚1.5596714736802304 -loss/FBt3/detection_loss‚2.16068624837512 0loss/FBt3/assignment_loss‚0.2528268960552208 0loss/FBt4/assignment_loss‚0.4626078090625518 0loss/SRqqt1/detection_loss‚3.226589150085885 /loss/FBt1/detection_loss‚13.946439984440804 /loss/FRt2/assignment_loss‚7.436230735958747 2loss/SRqqt1/assignment_loss‚3.2090208299698366 0loss/FBt2/assignment_loss‚0.8491652290523052 %loss/total_loss‚97.45170655406055 /loss/FRt1/assignment_loss‚7.247651400389494 2loss/SRqqt4/assignment_loss‚3.3524170909609112 -loss/FBt2/detection_loss‚5.53390878879458 trainer/global_step‚199 /loss/FRt3/assignment_loss‚7.293433040700933 -loss/FRt1/detection_loss‚3.77730299477224 0loss/SRqqt3/detection_loss‚4.937845734081098 1loss/SRqqt2/assignment_loss‚3.399207196584562 1loss/SRqqt3/assignment_loss‚3.448597197021757 /loss/FRt2/detection_loss‚2.8698480434996902 .loss/FBt4/detection_loss‚6.145065985461499 _runtime‚ 331.583868851 _step‚15Ü0Gú“Œ‚(:ˆ ”ºéÔדºº gpu.0.gpu‚26.0 gpu.0.memory‚2, gpu.0.memoryAllocated‚16.716384887695312* gpu.0.memoryAllocatedBytes‚ 14359265280 gpu.0.temp‚34.0 gpu.0.powerWatts‚164.013' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚32.8026 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0Ø[ßrZ‚(:Ï ”ºéÔÔÁ̼ proc.cpu.threads‚17 disk.dm-0.in‚ 895.98828125 network.sent‚346713 network.recv‚202102 disk.sda2.out‚0 disk.sda3.in‚0 disk.nvme0n1.in‚ 895.98828125# disk.nvme0n1.out‚7864.583984375 disk.sda1.in‚0% disk./.usageGB‚1175.7467231750488 disk.sda2.in‚0 disk.sda3.out‚11.28125% disk.nvme0c0n1.out‚7864.583984375 disk.dm-1.in‚0 disk.dm-1.out‚11.28125 disk.drbd1260.out‚ 0.05078125 disk.sda1.out‚0' disk./.usagePercent‚78.905349109487" disk.nvme0c0n1.in‚ 895.98828125 disk.drbd1260.in‚0 disk.sda.out‚11.28125$ memory_percent‚93.02830696105957( proc.memory.availableMB‚ 1142.2421875! proc.memory.rssMB‚ 2921.765625) proc.memory.percent‚17.83304214477539 disk.dm-0.out‚7864.583984375 disk.sda.in‚0 cpu‚25.665174534316470¼ú’Ž‚(:‡ £ºéÔ„÷€¹ gpu.0.gpu‚1.0 gpu.0.memory‚0, gpu.0.memoryAllocated‚16.716384887695312* gpu.0.memoryAllocatedBytes‚ 14359265280 gpu.0.temp‚32.0 gpu.0.powerWatts‚115.966' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚23.1932 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0°\Xa‚(:Ö £ºéÔ͸º disk.drbd1260.in‚0 disk.sda1.out‚0 disk.sda2.out‚0 disk.sda3.out‚ 11.734375" disk.nvme0c0n1.in‚ 895.98828125& disk.nvme0c0n1.out‚8120.9697265625' disk./.usagePercent‚78.905349109487! disk.dm-0.out‚8120.9697265625 disk.sda1.in‚0) proc.memory.availableMB‚ 1142.21484375! proc.memory.rssMB‚ 2921.765625) proc.memory.percent‚17.83304214477539 proc.cpu.threads‚17 network.sent‚396593 disk.drbd1260.out‚ 0.0546875 disk.sda2.in‚0$ memory_percent‚93.02847385406494 cpu‚25.667344671120517% disk./.usageGB‚1175.7467231750488 disk.nvme0n1.in‚ 895.98828125$ disk.nvme0n1.out‚8120.9697265625 disk.dm-1.in‚0 disk.dm-1.out‚ 11.734375 disk.sda.in‚0 disk.sda.out‚ 11.734375 disk.sda3.in‚0 network.recv‚208264 disk.dm-0.in‚ 895.98828125»~’‚(:‡ ²ºéÔØì¹ gpu.0.gpu‚6.0 gpu.0.memory‚1, gpu.0.memoryAllocated‚16.716384887695312* gpu.0.memoryAllocatedBytes‚ 14359265280 gpu.0.temp‚32.0 gpu.0.powerWatts‚119.703' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚23.9406 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0¤×Á[h‘‚(:Ý ²ºéÔ—£Â»$ disk.nvme0n1.out‚8282.1533203125 disk.sda.out‚ 12.09765625! proc.memory.rssMB‚ 2921.765625) disk./.usagePercent‚78.90534936549454$ disk./.usageGB‚1175.746726989746! disk.dm-0.out‚8282.1533203125 disk.dm-1.in‚0 disk.drbd1260.in‚0 disk.sda1.out‚0$ memory_percent‚93.02332401275635) proc.memory.availableMB‚ 1143.05859375 disk.sda2.out‚0 disk.nvme0n1.in‚ 895.98828125 disk.dm-0.in‚ 895.98828125 disk.dm-1.out‚ 12.09765625 disk.drbd1260.out‚ 0.0546875 disk.sda1.in‚0 cpu‚25.664233729340033 network.sent‚403148 disk.sda3.in‚0 disk.sda3.out‚ 12.09765625& disk.nvme0c0n1.out‚8282.1533203125 disk.sda.in‚0) proc.memory.percent‚17.83304214477539 proc.cpu.threads‚17 network.recv‚213445 disk.sda2.in‚0" disk.nvme0c0n1.in‚ 895.98828125€j¦“’‚(:ˆ Áºéԩι gpu.0.gpu‚25.0 gpu.0.memory‚2, gpu.0.memoryAllocated‚16.716384887695312* gpu.0.memoryAllocatedBytes‚ 14359265280 gpu.0.temp‚33.0 gpu.0.powerWatts‚172.101' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚34.4202 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0¬½k“‚(:à ÁºéÔ‡Ô¨½ disk.dm-0.in‚ 895.98828125 disk.dm-1.out‚ 12.73046875$ memory_percent‚93.02327632904053) proc.memory.availableMB‚ 1143.06640625! proc.memory.rssMB‚ 2921.765625 proc.cpu.threads‚17 network.sent‚410415 disk.sda2.in‚0 disk.sda2.out‚0' disk.nvme0c0n1.out‚8851.09423828125" disk.dm-0.out‚8851.09423828125 disk.dm-1.in‚0 disk.drbd1260.in‚0 disk.drbd1260.out‚ 0.05859375 disk.sda3.in‚0 disk.nvme0n1.in‚ 895.98828125 disk.sda1.out‚0 disk.sda.in‚0 disk.sda.out‚ 12.73046875) proc.memory.percent‚17.83304214477539 cpu‚25.66524301009409 network.recv‚217404 disk.sda3.out‚ 12.73046875" disk.nvme0c0n1.in‚ 895.98828125% disk.nvme0n1.out‚8851.09423828125 disk.sda1.in‚0( disk./.usagePercent‚78.9037759431891% disk./.usageGB‚1175.7232818603516š¡l“”‚(:ˆ кéÔÜÛ…¹ gpu.0.gpu‚56.0 gpu.0.memory‚6, gpu.0.memoryAllocated‚16.716384887695312* gpu.0.memoryAllocatedBytes‚ 14359265280 gpu.0.temp‚34.0 gpu.0.powerWatts‚160.302' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚32.0604 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0’”¡¢[•‚(:Ð кéÔãü» network.sent‚414063) disk./.usagePercent‚78.90377466315142 disk.sda1.in‚0 disk.sda1.out‚0 disk.sda2.out‚0% disk.nvme0n1.out‚9107.30517578125" disk.dm-0.out‚9107.30517578125 network.recv‚221629 disk.drbd1260.in‚0 disk.drbd1260.out‚ 0.05859375 disk.sda.in‚0 disk.sda.out‚13.25 disk.sda2.in‚0 disk.nvme0n1.in‚ 1024.0859375 disk.dm-0.in‚ 1024.0859375% disk./.usageGB‚1175.7232627868652' disk.nvme0c0n1.out‚9107.30517578125 disk.dm-1.in‚0 disk.dm-1.out‚13.25$ memory_percent‚93.02332401275635) proc.memory.percent‚17.83304214477539 cpu‚25.661578838145086 disk.sda3.in‚0 disk.sda3.out‚13.25" disk.nvme0c0n1.in‚ 1024.0859375) proc.memory.availableMB‚ 1143.05859375! proc.memory.rssMB‚ 2921.765625 proc.cpu.threads‚17ªyŸ“–‚(:ˆ ߺéÔ꾡¹ gpu.0.gpu‚12.0 gpu.0.memory‚2, gpu.0.memoryAllocated‚16.716384887695312* gpu.0.memoryAllocatedBytes‚ 14359265280 gpu.0.temp‚33.0 gpu.0.powerWatts‚114.748' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚22.9496 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0aúÄÆP—‚(:Å ߺéÔ¬º»» disk.nvme0n1.out‚ 9531.953125 disk.dm-1.out‚13.78125$ memory_percent‚93.02334785461426! proc.memory.rssMB‚ 2921.765625 disk.nvme0n1.in‚ 1024.0859375 disk.sda.out‚13.78125 disk.sda3.out‚13.78125" disk.nvme0c0n1.out‚ 9531.953125 disk.dm-0.in‚ 1024.0859375 disk.dm-0.out‚ 9531.953125 disk.drbd1260.in‚0 disk.sda1.in‚0 network.recv‚226230 disk.sda2.out‚0 disk.dm-1.in‚0 disk.drbd1260.out‚0.0625) proc.memory.percent‚17.83304214477539 proc.cpu.threads‚17% disk./.usageGB‚1175.7467765808105 disk.sda.in‚0 disk.sda2.in‚0 disk.sda3.in‚0" disk.nvme0c0n1.in‚ 1024.0859375 disk.sda1.out‚0( proc.memory.availableMB‚ 1143.0546875 cpu‚25.662335314502418 network.sent‚420299) disk./.usagePercent‚78.90535269359249Ž%k0˜+ _wandbruntime‚428  _runtime‚428—·¡LÄ™¾ _step‚16 & lr-AdamW/pg1‚0.0005377735491133354 & lr-AdamW/pg2‚0.0005377735491133354 trainer/global_step‚249 % _timestamp‚1.7885013519810603e+09 _runtime‚ 428.059806637ŠØÁȚ & lr-AdamW/pg2‚0.0005377735491133354 trainer/global_step‚249 % _timestamp‚1.7885013519810603e+09 _runtime‚ 428.059806637 _step‚16 & lr-AdamW/pg1‚0.0005377735491133354§“®Ð0›+ _wandbruntime‚428  _runtime‚428™U˺'œ´O _LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚19655772033536 ]JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚20183470422528 bQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 251366133248 IDeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚19895547025920 \IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚19655772033536 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚24903680 ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 281121792 VFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 13207863296 N@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 247622656 XEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚19655754994176 TDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 13035530240 SGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚6178777 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚595 OFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚1290 YFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚20183164397056 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 230583296 [HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚19236290771344 [KDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 13207863296 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚30392 K?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚4999973 OCDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated‚6178777 ]LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 240022615040 XEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚19895794648576 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 306025472 UGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 582234453 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 _runtime‚ 428.061138697 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚30637 ZIDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 240022615040 ]LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 238660625548 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚24796 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 247622656 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚24796 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 _LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚19236307810704 RBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 13805551616 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚17039360 OCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚4999973 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚2 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 152061440 @8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚894 _step‚17 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚597 L@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚4392218 MFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚33 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 315220480 BDeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 WGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 13035530240 M@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚11178752 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚17039360 SGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚4999975 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 QCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 243625384 JBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚609 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚285 [HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚19655754994176 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚2880654 ^MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 251085011456 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚2 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚24794 JBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚285 `MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚19932079385600 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 P@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 13118517709 E=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚894 E8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚11153956 G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚3689 VEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 239792031744 dQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚19932104289280 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 WGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 12589930496 C;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚894 K?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚6153983 TADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚19895547025920—Ï9û¢ ¸O JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 _LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚19236307810704 RBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 13805551616 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚17039360 OCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚4999973 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚2 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 152061440 @8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚894 _step‚17 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚597 L@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚4392218 MFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚33 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 315220480 BDeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 WGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 13035530240 M@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚11178752 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚17039360 SGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚4999975 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 QCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 243625384 JBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚609 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚285 [HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚19655754994176 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚2880654 ^MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 251085011456 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚2 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚24794 JBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚285 `MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚19932079385600 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 P@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 13118517709 E=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚894 E8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚11153956 G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚3689 VEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 239792031744 dQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚19932104289280 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 WGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 12589930496 C;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚894 K?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚6153983 TADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚19895547025920 _LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚19655772033536 ]JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚20183470422528 bQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 251366133248 IDeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚19895547025920 \IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚19655772033536 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚24903680 ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 281121792 VFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 13207863296 N@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 247622656 XEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚19655754994176 TDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 13035530240 SGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚6178777 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚595 OFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚1290 YFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚20183164397056 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 230583296 [HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚19236290771344 [KDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 13207863296 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚30392 K?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚4999973 OCDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated‚6178777 ]LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 240022615040 XEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚19895794648576 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 306025472 UGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 582234453 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 _runtime‚ 428.061138697 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚30637 ZIDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 240022615040 ]LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 238660625548 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚24796 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 247622656 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚24796 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2³BÇ0ž+ _wandbruntime‚428  _runtime‚428!é!æÃ&Ÿ½M L>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 247635968 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 230596608 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 230596608 K>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚11223320 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 MADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚6203388 SEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 586244608 YFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚19313137505888 M=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 13805551616 RBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 13035530240 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 582234453 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 NBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚4410218 `ODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 252368603136 trainer/global_step‚249 RBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 13805551616 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 XGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 240980099584 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚24820 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚2892145 MDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚1290 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 # _timestamp‚1.78850135290534e+09 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 597688320 ZGDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚19734283242496 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24903680 YFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚19734266203136 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 315220480 VHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 226586232 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚3689 J>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚4409662 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚2 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 _runtime‚ 428.984658567 I=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚6178568 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚30392 R?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚19552523592806 N>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 13118517709 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚285 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 WFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 239386086918 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 597688320 OADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 247635968 [HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚20264067752448 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 WFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 240749502976 MADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚5019930 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 243625592 WGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 13207863296 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚894 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚2 VCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚19734266203136 G:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚11198498 QEDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚5019932 QEDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚6203388 MADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚6178568 VCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚19975263342080 \KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 252089591808 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚894 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚894 O6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚894 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 K;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 13567453184 FDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 13567453184 QCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 384795648 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 P@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 13805551616 YIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 13207863296‰(ÞcÇ& ÁM `ODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 252368603136 trainer/global_step‚249 RBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 13805551616 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 XGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 240980099584 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚24820 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚2892145 MDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚1290 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 # _timestamp‚1.78850135290534e+09 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 597688320 ZGDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚19734283242496 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24903680 YFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚19734266203136 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 315220480 VHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 226586232 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚3689 J>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚4409662 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚2 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 _runtime‚ 428.984658567 I=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚6178568 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚30392 R?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚19552523592806 N>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 13118517709 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚285 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 WFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 239386086918 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 597688320 OADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 247635968 [HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚20264067752448 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 WFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 240749502976 MADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚5019930 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 243625592 WGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 13207863296 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚894 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚2 VCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚19734266203136 G:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚11198498 QEDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚5019932 QEDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚6203388 MADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚6178568 VCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚19975263342080 \KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 252089591808 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚894 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚894 O6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚894 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 K;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 13567453184 FDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 13567453184 QCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 384795648 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 P@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 13805551616 YIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 13207863296 L>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 247635968 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 230596608 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 230596608 K>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚11223320 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 MADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚6203388 SEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 586244608 YFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚19313137505888 M=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 13805551616 RBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 13035530240 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 582234453 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 NBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚4410218ø)œ0¡+ _wandbruntime‚428  _runtime‚428=±À¨S¢Í .loss/FBt4/detection_loss‚4.305701452588278 1loss/SRqqt1/assignment_loss‚2.340945966541767 /loss/FRt1/detection_loss‚2.5263302070753917 .loss/FRt4/detection_loss‚2.362491740128108 /loss/FBt1/detection_loss‚7.1631212226442385 epoch‚0 _step‚19 0loss/FBt2/assignment_loss‚0.6556369901442883 /loss/FRt3/detection_loss‚2.7420340526316846 1loss/SRqqt3/detection_loss‚2.9562144835343522 trainer/global_step‚249 /loss/FBt3/detection_loss‚2.0302898083201786 1loss/SRqqt2/assignment_loss‚3.500431387048019 2loss/SRqqt4/assignment_loss‚2.8153859700343404 0loss/FBt1/assignment_loss‚0.4365071391126745 /loss/FRt2/detection_loss‚3.6246141986477944 1loss/SRqqt4/detection_loss‚1.9940172317022704 /loss/FBt2/detection_loss‚4.5296618143535605 /loss/FRt1/assignment_loss‚6.592791807083857 /loss/FRt2/assignment_loss‚7.338838634036836 /loss/FRt4/assignment_loss‚7.116440929142775 0loss/FBt4/assignment_loss‚0.4530364184924226 1loss/SRqqt1/detection_loss‚2.8836833874766645 2loss/SRqqt3/assignment_loss‚3.4034320209803206 0loss/FBt3/assignment_loss‚0.3334561665099803 %loss/total_loss‚80.64658263395827 % _timestamp‚1.7885013529076676e+09 /loss/FRt3/assignment_loss‚5.713579695139613 0loss/SRqqt2/detection_loss‚2.827939910588876 _runtime‚ 428.986281354ì3jW£Ñ %loss/total_loss‚80.64658263395827 % _timestamp‚1.7885013529076676e+09 /loss/FRt3/assignment_loss‚5.713579695139613 0loss/SRqqt2/detection_loss‚2.827939910588876 _runtime‚ 428.986281354 .loss/FBt4/detection_loss‚4.305701452588278 1loss/SRqqt1/assignment_loss‚2.340945966541767 /loss/FRt1/detection_loss‚2.5263302070753917 .loss/FRt4/detection_loss‚2.362491740128108 /loss/FBt1/detection_loss‚7.1631212226442385 epoch‚0 _step‚19 0loss/FBt2/assignment_loss‚0.6556369901442883 /loss/FRt3/detection_loss‚2.7420340526316846 1loss/SRqqt3/detection_loss‚2.9562144835343522 trainer/global_step‚249 /loss/FBt3/detection_loss‚2.0302898083201786 1loss/SRqqt2/assignment_loss‚3.500431387048019 2loss/SRqqt4/assignment_loss‚2.8153859700343404 0loss/FBt1/assignment_loss‚0.4365071391126745 /loss/FRt2/detection_loss‚3.6246141986477944 1loss/SRqqt4/detection_loss‚1.9940172317022704 /loss/FBt2/detection_loss‚4.5296618143535605 /loss/FRt1/assignment_loss‚6.592791807083857 /loss/FRt2/assignment_loss‚7.338838634036836 /loss/FRt4/assignment_loss‚7.116440929142775 0loss/FBt4/assignment_loss‚0.4530364184924226 1loss/SRqqt1/detection_loss‚2.8836833874766645 2loss/SRqqt3/assignment_loss‚3.4034320209803206 0loss/FBt3/assignment_loss‚0.3334561665099803TÔhu’¤‚(:‡ îºéÔ’¸–¹ gpu.0.gpu‚1.0 gpu.0.memory‚0, gpu.0.memoryAllocated‚16.716384887695312* gpu.0.memoryAllocatedBytes‚ 14359265280 gpu.0.temp‚32.0 gpu.0.powerWatts‚114.479' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚22.8958 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0+:©T¥‚(:É îºéÔ€‡¸½) proc.memory.availableMB‚ 1142.96484375 network.sent‚426998 network.recv‚231345) disk./.usagePercent‚78.88001972404928 disk.nvme0n1.out‚ 9883.3125 disk.dm-1.out‚ 14.16796875 disk.drbd1260.out‚0.0625 disk.sda1.in‚0$ memory_percent‚93.02389621734619 disk.sda2.in‚0 disk.sda2.out‚0 disk.dm-1.in‚0 cpu‚25.659661532423875$ disk./.usageGB‚1175.369297027588" disk.nvme0c0n1.in‚ 1024.0859375 disk.nvme0n1.in‚ 1024.0859375 disk.drbd1260.in‚0 disk.sda1.out‚0# proc.memory.rssMB‚ 2921.76953125( proc.memory.percent‚17.8330659866333 proc.cpu.threads‚17 disk.sda3.in‚0 disk.sda3.out‚ 14.16796875 disk.nvme0c0n1.out‚ 9883.3125 disk.dm-0.in‚ 1024.0859375 disk.dm-0.out‚ 9883.3125 disk.sda.in‚0 disk.sda.out‚ 14.16796875RàÜB“¦‚(:ˆ ýºéÔ਎» gpu.0.gpu‚30.0 gpu.0.memory‚3, gpu.0.memoryAllocated‚16.716384887695312* gpu.0.memoryAllocatedBytes‚ 14359265280 gpu.0.temp‚32.0 gpu.0.powerWatts‚115.537' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚23.1074 gpu.0.smClock‚1275 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0+šÄj§‚(:ß ýºéÔ¬‚°½ network.sent‚477876 disk.dm-0.in‚ 1024.0859375! disk.dm-0.out‚10139.771484375 disk.dm-1.in‚0 disk.sda1.in‚0 disk.sda1.out‚0 disk.sda3.in‚0$ disk.nvme0n1.out‚10139.771484375 network.recv‚236528) disk./.usagePercent‚78.88001870001914 disk.dm-1.out‚ 14.72265625 disk.drbd1260.in‚0 disk.sda.in‚0 disk.sda2.in‚0" disk.nvme0c0n1.in‚ 1024.0859375$ memory_percent‚93.02353858947754 disk.drbd1260.out‚ 0.06640625 disk.sda.out‚ 14.72265625 disk.sda2.out‚0& disk.nvme0c0n1.out‚10139.771484375 disk.nvme0n1.in‚ 1024.0859375( proc.memory.availableMB‚ 1143.0234375# proc.memory.rssMB‚ 2921.76953125( proc.memory.percent‚17.8330659866333% disk./.usageGB‚1175.3692817687988 disk.sda3.out‚ 14.72265625 cpu‚25.660379020193464 proc.cpu.threads‚17d04Š‘¨‚(:† Œ»éÔ±Ù¹ gpu.0.gpu‚0.0 gpu.0.memory‚0, gpu.0.memoryAllocated‚16.716384887695312* gpu.0.memoryAllocatedBytes‚ 14359265280 gpu.0.temp‚31.0 gpu.0.powerWatts‚93.674' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚18.7348 gpu.0.smClock‚1275 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0ó¦Á‰©‚(:á Œ»éÔÔ¡» network.sent‚481524 disk.sda3.out‚ 15.17578125! disk.dm-0.out‚10531.994140625 disk.drbd1260.in‚0$ memory_percent‚93.02816390991211# proc.memory.rssMB‚ 2921.86328125* proc.memory.percent‚17.833638191223145$ disk./.usageGB‚1173.814624786377# disk.nvme0c0n1.in‚ 1178.16015625& disk.nvme0c0n1.out‚10531.994140625 disk.dm¿ë-1.in‚0 disk.drbd1260.out‚ 0.06640625 disk.sda.in‚0' proc.memory.availableMB‚ 1142.265625 cpu‚25.65932279525897 network.recv‚240486 disk.sda2.in‚0 disk.sda2.out‚0 disk.sda3.in‚0 disk.dm-0.in‚ 1178.16015625 disk.dm-1.out‚ 15.17578125 disk.sda.out‚ 15.17578125 proc.cpu.threads‚17) disk./.usagePercent‚78.77568436548471! disk.nvme0n1.in‚ 1178.16015625$ disk.nvme0n1.out‚10531.994140625 disk.sda1.in‚0 disk.sda1.out‚0ü­Ì¨žª‚(:“ ›»éÔ„ì—¹ gpu.0.gpu‚48.0 gpu.0.memory‚4, gpu.0.memoryAllocated‚16.716384887695312* gpu.0.memoryAllocatedBytes‚ 14359265280 gpu.0.temp‚34.0 gpu.0.powerWatts‚156.547' gpu.0.enforcedPowerLimitWatts‚500.0) gpu.0.powerPercent‚31.309399999999997 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0Ÿê`/k«‚(:à ›»éÔªé»& disk.nvme0n1.out‚11192.11865234375 disk.dm-0.in‚ 1178.16015625 disk.sda1.in‚0 cpu‚25.65947065869589 network.sent‚487796) disk./.usagePercent‚78.77565185252786 disk.sda2.in‚0# disk.nvme0c0n1.in‚ 1178.16015625# disk.dm-0.out‚11192.11865234375 disk.dm-1.in‚0 disk.drbd1260.in‚0 disk.sda3.in‚0 disk.dm-1.out‚ 15.578125 disk.sda1.out‚0 disk.sda.out‚ 15.578125& proc.memory.availableMB‚ 1142.21875* proc.memory.percent‚17.833638191223145 proc.cpu.threads‚17 network.recv‚245519% disk./.usageGB‚1173.8141403198242 disk.sda2.out‚0 disk.sda3.out‚ 15.578125 disk.drbd1260.out‚ 0.0703125 disk.sda.in‚0$ memory_percent‚93.02845001220703# proc.memory.rssMB‚ 2921.86328125( disk.nvme0c0n1.out‚11192.11865234375! disk.nvme0n1.in‚ 1178.16015625h·]Âo¬‚(:ä ª»éÔöÄλ disk.sda.in‚0 disk.sda2.out‚0 disk.drbd1260.out‚ 0.0703125# proc.memory.rssMB‚ 2921.86328125 cpu‚25.65784956536624 proc.cpu.threads‚17% disk./.usageGB‚1173.8141403198242 disk.sda1.in‚0 disk.sda3.in‚0( disk.nvme0c0n1.out‚11519.42724609375! disk.nvme0n1.in‚ 1178.16015625& disk.nvme0n1.out‚11519.42724609375 disk.dm-1.out‚ 15.8828125' proc.memory.availableMB‚ 1142.703125 network.sent‚496273 network.recv‚250516 disk.sda.out‚ 15.8828125 disk.sda2.in‚0 disk.sda3.out‚ 15.8828125 disk.dm-0.in‚ 1178.16015625# disk.dm-0.out‚11519.42724609375 disk.dm-1.in‚0) disk./.usagePercent‚78.77565185252786 disk.sda1.out‚0# disk.nvme0c0n1.in‚ 1178.16015625 disk.drbd1260.in‚0$ memory_percent‚93.02549362182617* proc.memory.percent‚17.833638191223145ÔóÝ­‚(:’ ª»éԙ» gpu.0.gpu‚7.0 gpu.0.memory‚1, gpu.0.memoryAllocated‚16.716384887695312* gpu.0.memoryAllocatedBytes‚ 14359265280 gpu.0.temp‚33.0 gpu.0.powerWatts‚119.415' gpu.0.enforcedPowerLimitWatts‚500.0) gpu.0.powerPercent‚23.883000000000003 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0[ÞIž®‚(:“ ¹»éÔ¶“íº gpu.0.gpu‚30.0 gpu.0.memory‚3, gpu.0.memoryAllocated‚16.716384887695312* gpu.0.memoryAllocatedBytes‚ 14359265280 gpu.0.temp‚34.0 gpu.0.powerWatts‚139.899' gpu.0.enforcedPowerLimitWatts‚500.0) gpu.0.powerPercent‚27.979799999999997 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0*/^d¯‚(:Ù ¹»éÔ¢‡‹»% disk.nvme0n1.out‚11871.5302734375 proc.cpu.threads‚17$ disk./.usageGB‚1173.837631225586 disk.sda.out‚16.4375 disk.sda2.in‚0' disk.nvme0c0n1.out‚11871.5302734375" disk.dm-0.out‚11871.5302734375# proc.memory.rssMB‚ 2921.86328125* proc.memory.percent‚17.833638191223145 cpu‚25.658137058293676 network.recv‚254643 disk.sda1.in‚0 disk.sda.in‚0 disk.sda2.out‚0 disk.sda3.out‚16.4375# disk.nvme0c0n1.in‚ 1178.16015625! disk.nvme0n1.in‚ 1178.16015625 disk.dm-0.in‚ 1178.16015625( disk./.usagePercent‚78.7772283469237 disk.drbd1260.in‚0 disk.drbd1260.out‚ 0.07421875 disk.sda1.out‚0 disk.dm-1.in‚0 disk.dm-1.out‚16.4375$ memory_percent‚93.02706718444824( proc.memory.availableMB‚ 1142.4453125 network.sent‚499934 disk.sda3.in‚0¹G½0°+ _wandbruntime‚516  _runtime‚516¾šÄ±¾ _runtime‚ 516.870753293 _step‚20 & lr-AdamW/pg1‚0.0006457602055617963 & lr-AdamW/pg2‚0.0006457602055617963 trainer/global_step‚299 % _timestamp‚1.7885014407919762e+09@ŒT@Ȳ _step‚20 & lr-AdamW/pg1‚0.0006457602055617963 & lr-AdamW/pg2‚0.0006457602055617963 trainer/global_step‚299 % _timestamp‚1.7885014407919762e+09 _runtime‚ 516.870753293Z–0³+ _wandbruntime‚516  _runtime‚516¤ì” »'´µO SGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚3455302 MFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚33 `MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚23911822387200 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 152061440 XEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚23364934705160 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 G?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚285 \IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚23581332481536 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚3689 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚3689 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 JBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚285 OCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚5997825 ]JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚24213276906496 dQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚23911847290880 ^MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 301158979584 TADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚23364691079776 _LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚23581332481536 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 230583296 H?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚1317 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚2 ZIDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 287924004864 YHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 287693421568 M@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚13409204 SEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 384795648 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚24903680 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 IDeviceStatsMonitor.on_train_batch_start/allocation.all.current‚24796 IDeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚23869008863744 WGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 12589930496 YHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 286063581232 TDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 13805551616 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚24794 UBDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated‚23869256486400 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚2 @8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚894 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 K?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚7386585 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 230583296 WGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 13035530240 ]LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 287924004864 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚24794 E=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚894 _step‚21 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚576 C;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚894 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚30392 N@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 247622656 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 [HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚23078627498544 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚17039360 OCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚7386585 @8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚894 bQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 301429615616 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 597688320 % _timestamp‚1.7885014407929091e+09 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚609 TFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 597688320 P@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 13567453184 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚17039360 XEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚23869256486400 SGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚7411379 BDeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 QCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 243625384 ]LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 286290167256 RBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 13805551616 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 YIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 13207863296 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚30392 SGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚1813115 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 XJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 226586024 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚30637 K?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚5997823 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚3455304 ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 270636032 _LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚23078644537904 JBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚609 TDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 13035530240 L@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚5268417 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 UGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 582234453 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚24796 TADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚23869008863744 PDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚5268993 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚574 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 E8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚13384408 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 586244608 O?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 13805551616 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚285*«W¿'µ¹O D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚24796 TADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚23869008863744 PDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚5268993 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚574 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 E8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚13384408 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 586244608 O?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 13805551616 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚285 SGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚3455302 MFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚33 `MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚23911822387200 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 152061440 XEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚23364934705160 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 G?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚285 \IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚23581332481536 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚3689 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚3689 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 JBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚285 OCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚5997825 ]JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚24213276906496 dQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚23911847290880 ^MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 301158979584 TADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚23364691079776 _LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚23581332481536 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 230583296 H?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚1317 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚2 ZIDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 287924004864 YHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 287693421568 M@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚13409204 SEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 384795648 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚24903680 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 IDeviceStatsMonitor.on_train_batch_start/allocation.all.current‚24796 IDeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚23869008863744 WGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 12589930496 YHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 286063581232 TDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 13805551616 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚24794 UBDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated‚23869256486400 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚2 @8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚894 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 K?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚7386585 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 230583296 WGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 13035530240 ]LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 287924004864 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚24794 E=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚894 _step‚21 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚576 C;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚894 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚30392 N@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 247622656 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 [HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚23078627498544 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚17039360 OCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚7386585 @8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚894 bQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 301429615616 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 597688320 % _timestamp‚1.7885014407929091e+09 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚609 TFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 597688320 P@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 13567453184 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚17039360 XEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚23869256486400 SGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚7411379 BDeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 QCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 243625384 ]LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 286290167256 RBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 13805551616 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 YIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 13207863296 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚30392 SGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚1813115 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 XJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 226586024 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚30637 K?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚5997823 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚3455304 ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 270636032 _LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚23078644537904 JBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚609 TDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 13035530240 L@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚5268417 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 UGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 582234453hÊŸ±0¶+ _wandbruntime‚517  _runtime‚517}97ÿ·¿M NBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚5286348 QCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 384795648 VCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚23442733487306 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 597688320 FDeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 trainer/global_step‚299 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚24820 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 582234453 _runtime‚ 517.799600649 L>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 247635968 TCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 288650892800 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚33 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 I=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚7411170 R?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚23948477543936 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚894 MADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚7435990 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 R?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚23442489861714 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚582 WIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 597688320 XGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 288881489408 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚30637 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚1819551 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚1818969 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 TDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 13207863296 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚2 G:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚13453772 PBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 586244608 UEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 13035530240 [JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 288881489408 WFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 288650892800 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 ]JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚23155491272448 MADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚6017780 MDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚1295 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 YFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚23155474233088 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 [JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 287242214858 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚285 OADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 247635968 N>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 13567453184 ^KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚23991417247232 RBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 13805551616 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1317 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚609 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚3466795 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚2 \KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 302170015232 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 FDeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 C6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚13428950 ODevicÝÃØÆ eStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚609 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 I=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚6017780 ]JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚23659843690496 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚24820 J>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚5285764 [HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚24293884886016 `ODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 302442735104 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 230596608 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 230596608 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚30392 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 243625592 P@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 13805551616 M=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 13805551616 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚894 MADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚6017782 ZGDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚23659843690496 VCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚23659826651136 QEDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚6017782 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚3466797 N>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 13118517709 RBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 13035530240 SEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 586244608 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 S@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚23948725179904 VCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚23948725179904 K>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚13453772 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚584 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚609 _step‚22Äå¥É&¸ÃM QCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 384795648 VCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚23442733487306 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 597688320 FDeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 trainer/global_step‚299 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚24820 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 582234453 _runtime‚ 517.799600649 L>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 247635968 TCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 288650892800 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚33 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 I=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚7411170 R?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚23948477543936 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚894 MADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚7435990 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 R?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚23442489861714 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚582 WIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 597688320 XGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 288881489408 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚30637 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚1819551 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚1818969 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 TDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 13207863296 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚2 G:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚13453772 PBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 586244608 UEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 13035530240 [JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 288881489408 WFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 288650892800 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 ]JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚23155491272448 MADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚6017780 MDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚1295 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 YFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚23155474233088 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 [JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 287242214858 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚285 OADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 247635968 N>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 13567453184 ^KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚23991417247232 RBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 13805551616 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1317 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚609 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚3466795 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚2 \KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 302170015232 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 FDeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 C6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚13428950 ODeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚609 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 I=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚6017780 ]JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚23659843690496 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚24820 J>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚5285764 [HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚24293884886016 `ODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 302442735104 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 230596608 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 230596608 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚30392 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 243625592 P@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 13805551616 M=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 13805551616 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚894 MADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚6017782 ZGDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚23659843690496 VCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚23659826651136 QEDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚6017782 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚3466797 N>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 13118517709 RBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 13035530240 SEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 586244608 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 S@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚23948725179904 VCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚23948725179904 K>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚13453772 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚584 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚609 _step‚22 NBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚5286348¸««‰0¹+ _wandbruntime‚517  _runtime‚517ÞËOÍRºÌ epoch‚0 0loss/FBt2/assignment_loss‚0.7770235795906046 1loss/FBt3/assignment_loss‚0.27917224549491654 /loss/FRt1/detection_loss‚2.6320474314689637 /loss/FRt3/assignment_loss‚5.577964809205797 2loss/SRqqt1/assignment_loss‚2.4125512990736424 0loss/FBt1/assignment_loss‚0.5347408998358013 .loss/FBt3/detection_loss‚2.935786113427812 .loss/FRt3/detection_loss‚2.761798387826091 /loss/FBt4/detection_loss‚2.9742819291573985 -loss/FRt2/detection_loss‚2.75460030913353 /loss/FRt2/assignment_loss‚6.328515119552613 0loss/FRt4/assignment_loss‚5.7870404618851685 trainer/global_step‚299 $ _timestamp‚1.788501441722963e+09 1loss/FBt4/assignment_loss‚0.16873448864932056 0loss/SRqqt2/detection_loss‚4.037517582140271 1loss/SRqqt4/detection_loss‚1.2787662902504113 _runtime‚ 517.801496912 2loss/SRqqt2/assignment_loss‚3.2856997724563355 2loss/SRqqt4/assignment_loss‚2.8073913034485893 /loss/FRt4/detection_loss‚2.9067209049108182 .loss/FBt1/detection_loss‚6.526779465578698 2loss/SRqqt3/assignment_loss‚2.9250378637086776 /loss/FBt2/detection_loss‚3.7669475004076958 %loss/total_loss‚74.87965006664001 _step‚23 /loss/FRt1/assignment_loss‚4.896579651832581 0loss/SRqqt1/detection_loss‚2.037823204819421 0loss/SRqqt3/detection_loss‚4.486129452784856P%áƒV»Ð 2loss/SRqqt4/assignment_loss‚2.8073913034485893 /loss/FRt4/detection_loss‚2.9067209049108182 .loss/FBt1/detection_loss‚6.526779465578698 2loss/SRqqt3/assignment_loss‚2.9250378637086776 /loss/FBt2/detection_loss‚3.7669475004076958 %loss/total_loss‚74.87965006664001 _step‚23 /loss/FRt1/assignment_loss‚4.896579651832581 0loss/SRqqt1/detection_loss‚2.037823204819421 0loss/SRqqt3/detection_loss‚4.486129452784856 epoch‚0 0loss/FBt2/assignment_loss‚0.7770235795906046 1loss/FBt3/assignment_loss‚0.27917224549491654 /loss/FRt1/detection_loss‚2.6320474314689637 /loss/FRt3/assignment_loss‚5.577964809205797 2loss/SRqqt1/assignment_loss‚2.4125512990736424 0loss/FBt1/assignment_loss‚0.5347408998358013 .loss/FBt3/detection_loss‚2.935786113427812 .loss/FRt3/detection_loss‚2.761798387826091 /loss/FBt4/detection_loss‚2.9742819291573985 -loss/FRt2/detection_loss‚2.75460030913353 /loss/FRt2/assignment_loss‚6.328515119552613 0loss/FRt4/assignment_loss‚5.7870404618851685 trainer/global_step‚299 $ _timestamp‚1.788501441722963e+09 1loss/FBt4/assignment_loss‚0.16873448864932056 0loss/SRqqt2/detection_loss‚4.037517582140271 1loss/SRqqt4/detection_loss‚1.2787662902504113 _runtime‚ 517.801496912 2loss/SRqqt2/assignment_loss‚3.2856997724563355¤Tàß¼‚(:… È»éÔ¢ÄÖ» gpu.0.gpu‚0.0 gpu.0.memory‚0, gpu.0.memoryAllocated‚16.716384887695312* gpu.0.memoryAllocatedBytes‚ 14359265280 gpu.0.temp‚31.0 gpu.0.powerWatts‚93.345' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚18.669 gpu.0.smClock‚1275 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0m"§]½‚(:Ò È»éÔîÅè½ disk.sda3.in‚0' disk.nvme0c0n1.out‚12030.6201171875# memory_percent‚93.0269718170166( proc.memory.availableMB‚ 1142.4609375# proc.memory.rssMB‚ 2921.86328125* proc.memory.percent‚17.833638191223145 proc.cpu.threads‚17( disk./.usagePercent‚78.7772283469237$ disk./.usageGB‚1173.837631225586 disk.dm-1.in‚0 disk.sda.in‚0# disk.nvme0c0n1.in‚ 1319.27734375 network.recv‚258549 disk.drbd1260.in‚0 disk.sda1.in‚0 disk.sda3.out‚16.75% disk.nvme0n1.out‚12030.6201171875 disk.dm-0.in‚ 1319.27734375 cpu‚25.656362002397664 disk.sda.out‚16.75 disk.sda2.out‚0 network.sent‚505357 disk.dm-1.out‚16.75 disk.drbd1260.out‚ 0.07421875 disk.sda1.out‚0! disk.nvme0n1.in‚ 1319.27734375" disk.dm-0.out‚12030.6201171875 disk.sda2.in‚0OI”hž¾‚(:“ ×»éÔЦþ¹ gpu.0.gpu‚48.0 gpu.0.memory‚4, gpu.0.memoryAllocated‚16.716384887695312* gpu.0.memoryAllocatedBytes‚ 14359265280 gpu.0.temp‚34.0 gpu.0.powerWatts‚169.167' gpu.0.enforcedPowerLimitWatts‚500.0) gpu.0.powerPercent‚33.833400000000005 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0úôp¿‚(:å ×»éÔ÷Á» disk.drbd1260.out‚0.078125$ memory_percent‚93.02976131439209# proc.memory.rssMB‚ 2921.86328125 cpu‚25.658106372276762$ disk./.usageGB‚1173.039852142334 disk.sda1.in‚0 disk.sda.in‚0' disk.nvme0c0n1.out‚12398.1337890625" disk.dm-0.out‚12398.1337890625* proc.memory.percent‚17.833638191223145 proc.cpu.threads‚17 disk.sda3.out‚ 17.16015625! disk.nvme0n1.in‚ 1319.27734375% disk.nvme0n1.out‚12398.1337890625 disk.dm-0.in‚ 1319.27734375) proc.memory.availableMB‚ 1142.00390625 network.sent‚555387 network.recv‚264738 disk.sda1.out‚0 disk.sda.out‚ 17.16015625 disk.sda2.in‚0 disk.sda2.out‚0# disk.nvme0c0n1.in‚ 1319.27734375 disk.dm-1.in‚0) disk./.usagePercent‚78.72368872326544 disk.sda3.in‚0 disk.dm-1.out‚ 17.16015625 disk.drbd1260.in‚0Ðô¶R‘À‚(:† æ»éÔã̪¹ gpu.0.gpu‚48.0 gpu.0.memory‚6, gpu.0.memoryAllocated‚16.716384887695312* gpu.0.memoryAllocatedBytes‚ 14359265280 gpu.0.temp‚34.0 gpu.0.powerWatts‚118.51' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚23.702 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0ë«‘#lÁ‚(:á æ»éÔÞغ network.recv‚270033) disk./.usagePercent‚78.73071766612554 disk.sda2.out‚0' disk.nvme0c0n1.out‚12792.9970703125% disk.nvme0n1.out‚12792.9970703125 disk.dm-0.in‚ 1319.27734375 disk.dm-1.in‚0 disk.sda1.out‚0 disk.sda2.in‚0 disk.sda3.in‚0 disk.sda3.out‚ 17.6171875! disk.nvme0n1.in‚ 1319.27734375 disk.dm-1.out‚ 17.6171875 disk.sda1.in‚0 disk.sda.in‚0$ memory_percent‚93.02821159362793$ disk./.usageGB‚1173.144588470459" disk.dm-0.out‚12792.9970703125 disk.drbd1260.in‚0 disk.drbd1260.out‚0.078125# proc.memory.rssMB‚ 2921.86328125* proc.memory.percent‚17.833638191223145 proc.cpu.threads‚17 network.sent‚563849# disk.nvme0c0n1.in‚ 1319.27734375 disk.sda.out‚ 17.6171875( proc.memory.availableMB‚ 1142.2578125 cpu‚25.660053210992846¬¬èžÂ‚(:“ õ»éÔðëä¼ gpu.0.gpu‚44.0 gpu.0.memory‚3, gpu.0.memoryAllocated‚16.716384887695312* gpu.0.memoryAllocatedBytes‚ 14359265280 gpu.0.temp‚35.0 gpu.0.powerWatts‚161.123' gpu.0.enforcedPowerLimitWatts‚500.0) gpu.0.powerPercent‚32.224599999999995 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0ƒ¦&'[Â(:Ð õ»éÔï«Æ¾$ disk.nvme0n1.out‚13056.755859375 network.recv‚273811) disk./.usagePercent‚78.72756544535645% disk./.usageGB‚1173.0976181030273& disk.nvme0c0n1.out‚13056.755859375 disk.drbd1260.out‚ 0.08203125 disk.sda1.out‚0" proc.memory.rssMB‚ 2921.8671875 disk.sda3.in‚0 disk.dm-0.in‚ 1319.27734375 disk.dm-1.out‚18.25 disk.drbd1260.in‚0 disk.sda.in‚0$ memory_percent‚93.02811622619629* proc.memory.percent‚17.833662033081055 cpu‚25.65826207544017! disk.nvme0n1.in‚ 1319.27734375 network.sent‚566998 disk.sda2.out‚0 disk.sda3.out‚18.25# disk.nvme0c0n1.in‚ 1319.27734375 disk.dm-1.in‚0( proc.memory.availableMB‚ 1142.2734375 proc.cpu.threads‚17 disk.sda2.in‚0! disk.dm-0.out‚13056.755859375 disk.sda1.in‚0 disk.sda.out‚18.25T ö0Ä+ _wandbruntime‚576  _runtime‚576´6 [Äž trainer/global_step‚349 % _timestamp‚1.7885015002123756e+09 _runtime‚ 576.291373318 _step‚24 & lr-AdamW/pg1‚0.0007537468620102572 & lr-AdamW/pg2‚0.0007537468620102572Õº/’ÈÆ & lr-AdamW/pg1‚0.0007537468620102572 & lr-AdamW/pg2‚0.0007537468620102572 trainer/global_step‚349 % _timestamp‚1.7885015002123756e+09 _runtime‚ 576.291373318 _step‚24û‚M0Ç+ _wandbruntime‚576  _runtime‚576$ópº'È´O GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚2 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 YIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 13207863296 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚609 SGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚4029952 PDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚6146836 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚4029954 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 JBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚285 SGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚6995675 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚2 YFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚28242929018368 SEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 384795648 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 C;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚894 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 230583296 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 316511232 [KDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 13207863296 TFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 597688320 M=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 13567453184 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚24796 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚24903680 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 597688320 TADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚27842470701568 IDeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚3689 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 586244608 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚587 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚24796 ]LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 335825394688 bQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 351655237120 ]LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 333919708964 UBDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated‚27842718324224 IDeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚27842470701568 TDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 13035530240 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚3689 SGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚2116295 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 @8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚894 K?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚8619187 VEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 335594811392 XEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚27842718324224 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚24794 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚30392 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 trainer/global_step‚349 P@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 13567453184 O?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 13805551616 VFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 13207863296 _runtime‚ 576.293054826 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚17039360 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚2116882 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 XEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚27506875890176 SGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚8643981 TADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚27254657348684 _LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚26920981265104 UGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 582234453 E8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚15614860 OCDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated‚8643981 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 247622656 M@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚15639656 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚30392 D>DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚30637 MFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚33 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 315220480 QCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 243625384 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚17039360 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 586244608 $ _timestamp‚1.788501500214003e+09 YHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 335594811392 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚589 OFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚1299 [HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚26920964225744 _step‚25 OCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚6995675 \IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚27506892929536 ^MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 351363629568 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 P@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 13118517709 TDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 13805551616 G?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚609 BDeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚27842470701568 TDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 13035530240 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚3689 SGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚2116295 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 @8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚894 K?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚8619187 VEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 335594811392 XEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚27842718324224 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚24794 LBDeviceStatsMonitor.on_train_batch_start/al3>“A!location.small_pool.peak‚30392 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 trainer/global_step‚349 P@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 13567453184 O?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 13805551616 VFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 13207863296 _runtime‚ 576.293054826 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚17039360 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚2116882 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 XEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚27506875890176 SGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚8643981 TADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚27254657348684 _LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚26920981265104 UGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 582234453 E8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚15614860 OCDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated‚8643981 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 247622656 M@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚15639656 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚30392 D>DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚30637 MFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚33 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 315220480 QCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 243625384 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚17039360 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 586244608 $ _timestamp‚1.788501500214003e+09 YHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 335594811392 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚589 OFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚1299 [HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚26920964225744 _step‚25 OCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚6995675 \IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚27506892929536 ^MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 351363629568 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 P@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 13118517709 TDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 13805551616 G?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚609 BDeviceStatsMonitor.on_train_batch_start/allocation.all.current‚24796 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚24903680 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 597688320 TADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚27842470701568 IDeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚3689 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 586244608 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚587 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚24796 ]LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 335825394688 bQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 351655237120/Ó¡0Ê+ _wandbruntime‚577  _runtime‚577£‘Ó^Å&Ë¿M E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 N>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 13118517709 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚565 [JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 334871756566 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 RBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 13035530240 XGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 336782879232 G:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚15659402 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚24820 [HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚28323832238592 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 YFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚26997810960288 SEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 586244608 K>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚15684224 QEDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚7015632 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚894 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚609 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 243625592 ]JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚26997827999648 WGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 13207863296 % _timestamp‚1.7885015013533978e+09 [JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 336782879232 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 152061440 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚563 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 315220480 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚609 K;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 13567453184 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17039360 \KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 352376463360 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 ]JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚27585404138496 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚2 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚24820 TCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 336552282624 QCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 384795648 S@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚27922187017728 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 230596608 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚894 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚285 trainer/global_step‚349 F8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 VCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚27585387099136 R?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚27921939381760 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚2122966 WFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 334645170334 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚17039360 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 L>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 247635968 [MDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚ 270622720 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚894 _runtime‚ 577.433655981 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚33 MADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚7015632 I=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚8643772 MADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚8643772 J>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚6163848 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚2122403 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚17039360 ZGDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚27585404138496 YFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚27585387099136 UEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 13035530240 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 VCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚27332699756214 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚3689 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 295526400 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚30637 VHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 226586232 PBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 586244608 bODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚27971185152512 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24903680 VCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚27922187017728 N>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 13567453184 NBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚6164413 G:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚15684224 MADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚7015630 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚4041447 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 `ODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 352647086080 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 R?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚27332456130622 YIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 13207863296 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 ^KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚27971160248832 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚285 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚285 F6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚894 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚285 trainer/global_step‚349 F8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 VCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚27585387099136 R?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚27921939381760 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚2122966 WFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 334645170334 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚17039360 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 L>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 247635968 [MDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚ 270622720 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚894 _runtime‚ 577.433655981 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚33 MADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚7015632 I=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚8643772 MADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚8643772 J>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚6163848 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚2122403 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚17039360 ZGDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚27585404138496 YFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚27585387099136 UEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 13035530240 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 VCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚27332699756214 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚3689 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 295526400 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚30637 VHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 226586232 PBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 586244608 bODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚27971185152512 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24903680 VCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚27922187017728 N>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 13567453184 NBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚6164413 G:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚15684224 MADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚7015630 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚4041447 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 `ODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 352647086080 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 R?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚27332456130622 YIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 13207863296 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 ^KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚27971160248832 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚285 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚285 FDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 13118517709 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚565 [JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 334871756566 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 RBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 13035530240 XGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 336782879232 G:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚15659402 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚24820 [HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚28323832238592 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 YFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚26997810960288 SEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 586244608 K>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚15684224 QEDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚7015632 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚894 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚609 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 243625592 ]JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚26997827999648 WGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 13207863296 % _timestamp‚1.7885015013533978e+09 [JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 336782879232 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 152061440 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚563 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 315220480 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚609 K;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 13567453184 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17039360 \KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 352376463360 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 ]JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚27585404138496 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚2 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚24820 TCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 336552282624Uå¿Â0Í+ _wandbruntime‚577  _runtime‚577›èdEPÎÊ 0loss/FBt4/assignment_loss‚0.6239430849665099 0loss/SRqqt3/detection_loss‚5.116161033511162 % _timestamp‚1.7885015013563156e+09 .loss/FRt4/detection_loss‚2.307750057736668 .loss/FBt1/detection_loss‚8.895129884066789 trainer/global_step‚349 0loss/SRqqt2/detection_loss‚2.826794095661329 epoch‚0 /loss/FRt2/assignment_loss‚5.975606827025718 1loss/SRqqt1/assignment_loss‚2.538085589479459 2loss/SRqqt4/assignment_loss‚2.7191114404622247 0loss/FBt2/assignment_loss‚0.8365241092733413 /loss/FRt2/detection_loss‚2.6663501389483186 _runtime‚ 577.43508286 1loss/FBt3/assignment_loss‚0.34530159467325083 /loss/FRt1/detection_loss‚2.5063688629110095 .loss/FBt2/detection_loss‚3.199187855972179 /loss/FRt1/assignment_loss‚5.168373991066302 .loss/FRt3/assignment_loss‚4.00853311427882 /loss/FRt4/assignment_loss‚5.062842115349726 0loss/FBt1/assignment_loss‚0.6201413698853804 1loss/SRqqt2/assignment_loss‚3.125597497691279 1loss/SRqqt1/detection_loss‚1.7364346250695617 1loss/SRqqt4/detection_loss‚0.9702188394963741 /loss/FBt3/detection_loss‚1.4262538757075125 %loss/total_loss‚74.06466576145974 _step‚27 2loss/SRqqt3/assignment_loss‚3.5518617033958435 /loss/FRt3/detection_loss‚3.0194706287182553 /loss/FBt4/detection_loss‚4.8186234261127225dª¿JTÏÎ /loss/FRt1/detection_loss‚2.5063688629110095 .loss/FBt2/detection_loss‚3.199187855972179 /loss/FRt1/assignment_loss‚5.168373991066302 .loss/FRt3/assignment_loss‚4.00853311427882 /loss/FRt4/assignment_loss‚5.062842115349726 0loss/FBt1/assignment_loss‚0.6201413698853804 1loss/SRqqt2/assignment_loss‚3.125597497691279 1loss/SRqqt1/detection_loss‚1.7364346250695617 1loss/SRqqt4/detection_loss‚0.9702188394963741 /loss/FBt3/detection_loss‚1.4262538757075125 %loss/total_loss‚74.06466576145974 _step‚27 2loss/SRqqt3/assignment_loss‚3.5518617033958435 /loss/FRt3/detection_loss‚3.0194706287182553 /loss/FBt4/detection_loss‚4.8186234261127225 0loss/FBt4/assignment_loss‚0.6239430849665099 0loss/SRqqt3/detection_loss‚5.116161033511162 % _timestamp‚1.7885015013563156e+09 .loss/FRt4/detection_loss‚2.307750057736668 .loss/FBt1/detection_loss‚8.895129884066789 trainer/global_step‚349 0loss/SRqqt2/detection_loss‚2.826794095661329 epoch‚0 /loss/FRt2/assignment_loss‚5.975606827025718 1loss/SRqqt1/assignment_loss‚2.538085589479459 2loss/SRqqt4/assignment_loss‚2.7191114404622247 0loss/FBt2/assignment_loss‚0.8365241092733413 /loss/FRt2/detection_loss‚2.6663501389483186 _runtime‚ 577.43508286 1loss/FBt3/assignment_loss‚0.34530159467325083I[F“Ђ(:ˆ „¼éÔۘ˹ gpu.0.gpu‚29.0 gpu.0.memory‚4, gpu.0.memoryAllocated‚16.716384887695312* gpu.0.memoryAllocatedBytes‚ 14359265280 gpu.0.temp‚35.0 gpu.0.powerWatts‚116.629' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚23.3258 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0ŸE/O_Ñ‚(:Ô „¼éÔ‚äñº disk.drbd1260.in‚0 disk.drbd1260.out‚ 0.08203125 disk.sda.out‚ 18.69140625( proc.memory.availableMB‚ 1142.2890625" proc.memory.rssMB‚ 2921.8671875 proc.cpu.threads‚17 disk.sda1.in‚0 disk.sda2.in‚0 disk.sda3.out‚ 18.69140625 disk.nvme0c0n1.in‚1469.25 cpu‚25.656001163198376) disk./.usagePercent‚78.71944591040048% disk./.usageGB‚1172.9766311645508 disk.sda1.out‚0 disk.sda.in‚0' disk.nvme0c0n1.out‚13346.7021484375 disk.nvme0n1.in‚1469.25% disk.nvme0n1.out‚13346.7021484375* proc.memory.percent‚17.833662033081055 network.sent‚572233 network.recv‚277486 disk.dm-1.out‚ 18.69140625 disk.sda2.out‚0 disk.sda3.in‚0 disk.dm-0.in‚1469.25" disk.dm-0.out‚13346.7021484375$ memory_percent‚93.02802085876465 disk.dm-1.in‚0„õÒ‚(:ˆ “¼éÔǵ ¹ gpu.0.gpu‚24.0 gpu.0.memory‚2, gpu.0.memoryAllocated‚16.716384887695312* gpu.0.memoryAllocatedBytes‚ 14359265280 gpu.0.temp‚32.0 gpu.0.powerWatts‚110.446' gpu.0.enforcedPowerLimitWatts‚500 h»ž.0 gpu.0.powerPercent‚22.0892 gpu.0.smClock‚1275 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0îÀKÓ‚(:À “¼éÔûȃ»* proc.memory.percent‚17.833662033081055 cpu‚25.653532684736064( disk./.usagePercent‚78.7348946850542$ disk./.usageGB‚1173.206829071045" disk.dm-0.out‚13617.6513671875 disk.drbd1260.in‚0 disk.sda2.in‚0 disk.sda3.in‚0 disk.nvme0c0n1.in‚1469.25 disk.nvme0n1.in‚1469.25 network.sent‚622148 disk.dm-1.out‚19.25 disk.drbd1260.out‚ 0.0859375' disk.nvme0c0n1.out‚13617.6513671875$ memory_percent‚93.02799701690674 proc.cpu.threads‚17 disk.dm-0.in‚1469.25 disk.dm-1.in‚0 disk.sda1.in‚0 disk.sda.in‚0 disk.sda3.out‚19.25% disk.nvme0n1.out‚13617.6513671875 network.recv‚283416 disk.sda1.out‚0 disk.sda.out‚19.25 disk.sda2.out‚0) proc.memory.availableMB‚ 1142.29296875" proc.memory.rssMB‚ 2921.8671875û‹À'“Ô‚(:ˆ ¢¼éÔŽÑѸ gpu.0.gpu‚44.0 gpu.0.memory‚4, gpu.0.memoryAllocated‚16.716384887695312* gpu.0.memoryAllocatedBytes‚ 14359265280 gpu.0.temp‚35.0 gpu.0.powerWatts‚159.056' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚31.8112 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0I QÕ‚(:Æ ¢¼éÔ¿£º disk.sda1.in‚0% disk./.usageGB‚1173.1145973205566' disk.nvme0c0n1.out‚13762.3193359375 disk.dm-0.in‚1469.25" disk.dm-0.out‚13762.3193359375 disk.dm-1.out‚19.625$ memory_percent‚93.02728176116943) proc.memory.availableMB‚ 1142.41015625# proc.memory.rssMB‚ 2921.87109375 disk.sda.in‚0 disk.sda2.in‚0 disk.sda1.out‚0 cpu‚25.650116819902973 network.sent‚628662 disk.sda2.out‚0 disk.sda3.in‚0 disk.sda3.out‚19.625 disk.dm-1.in‚0 disk.drbd1260.out‚ 0.0859375* proc.memory.percent‚17.833685874938965 proc.cpu.threads‚17 network.recv‚288491) disk./.usagePercent‚78.72870493489134 disk.sda.out‚19.625 disk.nvme0c0n1.in‚1469.25 disk.nvme0n1.in‚1469.25% disk.nvme0n1.out‚13762.3193359375 disk.drbd1260.in‚07F©˜žÖ‚(:“ ±¼éÔ·Æ—» gpu.0.gpu‚51.0 gpu.0.memory‚6, gpu.0.memoryAllocated‚16.716384887695312* gpu.0.memoryAllocatedBytes‚ 14359265280 gpu.0.temp‚35.0 gpu.0.powerWatts‚159.876' gpu.0.enforcedPowerLimitWatts‚500.0) gpu.0.powerPercent‚31.975200000000005 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0Íøè_ׂ(:Ô ±¼éԚ϶¾ disk.nvme0n1.in‚1469.25 disk.dm-1.in‚0 disk.drbd1260.out‚ 0.08984375 disk.sda.in‚0 disk.sda2.in‚0& disk.nvme0n1.out‚14194.60595703125$ memory_percent‚93.02420616149902( proc.memory.availableMB‚ 1142.9140625# proc.memory.rssMB‚ 2921.87109375( disk./.usagePercent‚78.7329761645926 disk.sda1.out‚0 cpu‚25.649048748942366 proc.cpu.threads‚17 disk.dm-0.in‚1469.25 disk.drbd1260.in‚0 disk.sda1.in‚0 disk.sda3.in‚0 disk.sda3.out‚ 19.9453125 disk.nvme0c0n1.in‚1469.25( disk.nvme0c0n1.out‚14194.60595703125* proc.memory.percent‚17.833685874938965 network.sent‚633976 network.recv‚292435% disk./.usageGB‚1173.1782417297363# disk.dm-0.out‚14194.60595703125 disk.dm-1.out‚ 19.9453125 disk.sda.out‚ 19.9453125 disk.sda2.out‚0‚*£…‘Ø‚(:† À¼éԲ鱹 gpu.0.gpu‚0.0 gpu.0.memory‚0, gpu.0.memoryAllocated‚16.716384887695312* gpu.0.memoryAllocatedBytes‚ 14359265280 gpu.0.temp‚31.0 gpu.0.powerWatts‚93.674' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚18.7348 gpu.0.smClock‚1275 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0CÑvÙ‚(:ë À¼éÔ×âຠdisk.dm-0.in‚ 1605.35546875 disk.drbd1260.in‚0 disk.sda1.in‚0 disk.sda.in‚0 disk.sda2.in‚0 disk.sda2.out‚0# disk.nvme0c0n1.in‚ 1605.35546875( disk.nvme0c0n1.out‚14626.46923828125 network.sent‚639196# disk.dm-0.out‚14626.46923828125 disk.dm-1.out‚ 20.45703125 disk.drbd1260.out‚ 0.08984375 disk.sda1.out‚0 disk.sda3.in‚0& disk.nvme0n1.out‚14626.46923828125$ memory_percent‚93.02575588226318 network.recv‚296007! disk.nvme0n1.in‚ 1605.35546875) proc.memory.availableMB‚ 1142.66015625# proc.memory.rssMB‚ 2921.87109375 disk.sda.out‚ 20.45703125) disk./.usagePercent‚78.74046208090132% disk./.usageGB‚1173.2897872924805 disk.dm-1.in‚0 disk.sda3.out‚ 20.45703125* proc.memory.percent‚17.833685874938965 cpu‚25.646824551450155 proc.cpu.threads‚17T?âA0Ú+ _wandbruntime‚645  _runtime‚645 –ÄÛ¾ & lr-AdamW/pg2‚0.0008617335184587182 trainer/global_step‚399 % _timestamp‚1.7885015697263887e+09 _runtime‚ 645.806156021 _step‚28 & lr-AdamW/pg1‚0.0008617335184587182X½°®ÈÜ _step‚28 & lr-AdamW/pg1‚0.0008617335184587182 & lr-AdamW/pg2‚0.0008617335184587182 trainer/global_step‚399 % _timestamp‚1.7885015697263887e+09 _runtime‚ 645.806156021. ü„0Ý+ _wandbruntime‚645  _runtime‚645Z™€¹'Þ³O G?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚285 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚2 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 OCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚7993523 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 152061440 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 D>DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 ]LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 383726784512 `MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚31871308390400 BDeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚3689 TDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 13035530240 IDeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚31815932539392 ZIDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 383726784512 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚24796 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 @8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚894 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚30392 WGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 13035530240 XEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚31144867242976 TFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 597688320 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚17039360 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 586244608 OCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚9851789 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚4604604 TADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚31144623617592 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 247622656 TDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 13805551616 [KDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 13207863296 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 597688320 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚30637 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚24794 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚598 SGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚2419143 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚609 G?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚609 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚285 K?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚7993523 XEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚31432436338176 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚596 ]LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 381549250672 YIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 13207863296 N@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 247622656 [HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚31432436338176 dQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚31871333294080 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 JBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚609 trainer/global_step‚399 UBDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated‚31816180162048 \IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚31432453377536 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 586244608 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 230583296 L@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚7023745 SEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 384795648 WGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 12589930496 VFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 13207863296 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚17039360 UGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 582234453 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 597688320 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 P@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 13567453184 M@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚17870108 K?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚9851789 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚30392 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚2419739 YFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚32272807087616 YHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 381322664648 E8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚17845312 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚2 MFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚33 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚24903680 _runtime‚ 645.8096373 ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 289510400 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 E=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚894 % _timestamp‚1.7885015697294729e+09 YHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 383496201216 OFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚1299 QCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 243625384 VEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 383496201216 XJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 226586024 OCDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated‚9876583 SGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚4604602 _LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚30763317992304 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 TADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚31815932539392 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 314414080 ^MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 401498697216 [HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚30763300952944 C;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚894 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 _step‚29 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚3689 _LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚31432453377536 bQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 401788207616A=äM½'ß·O WKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚4604604 TADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚31144623617592 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 247622656 TDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 13805551616 [KDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 13207863296 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 597688320 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚30637 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚24794 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚598 SGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚2419143 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚609 G?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚609 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚285 K?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚7993523 XEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚31432436338176 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚596 ]LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 381549250672 YIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 13207863296 N@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 247622656 [HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚31432436338176 dQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚31871333294080 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 JBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚609 trainer/global_step‚399 UBDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated‚31816180162048 \IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚31432453377536 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 586244608 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 230583296 L@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚7023745 SEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 384795648 WGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 12589930496 VFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 13207863296 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚17039360 UGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 582234453 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 597688320 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 P@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 13567453184 M@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚17870108 K?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚9851789 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚30392 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚2419739 YFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚32272807087616 YHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 381322664648 E8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚17845312 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚2 MFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚33 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚24903680 _runtime‚ 645.8096373 ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 289510400 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 E=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚894 % _timestamp‚1.7885015697294729e+09 YHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 383496201216 OFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚1299 QCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 243625384 VEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 383496201216 XJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 226586024 OCDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated‚9876583 SGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚4604602 _LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚30763317992304 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 TADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚31815932539392 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 314414080 ^MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 401498697216 [HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚30763300952944 C;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚894 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 _step‚29 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚3689 _LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚31432453377536 bQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 401788207616 G?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚285 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚2 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 OCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚7993523 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 152061440 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 D>DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 ]LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 383726784512 `MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚31871308390400 BDeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚3689 TDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 13035530240 IDeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚31815932539392 ZIDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 383726784512 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚24796 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 @8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚894 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚30392 WGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 13035530240 XEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚31144867242976 TFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 597688320 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚17039360 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 586244608 OCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚9851789Öâî—0à+ _wandbruntime‚647  _runtime‚647¢ªìT-á¿M OADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 247635968 \KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 402500754432 C6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚17889854 J>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚7041132 trainer/global_step‚399 ODeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 13567453184 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 230596608 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚285 VCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚31510947547136 RBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 13035530240 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚33 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚2425638 R?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚31222422399530 VCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚31222666025122 [JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 382501298274 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 I=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚9876374 K;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 13567453184 YFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚31510947547136 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 VHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 226586232 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚2 QEDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚8013482 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚3689 [MDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚ 285302784 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚285 TCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 384453672448 VCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚31895648855552 [JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 384684269056 MDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚1299 UEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 12589930496 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚30637 ZGDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚31510964586496 FDeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚17914676 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚24820 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚30392 ]JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚30840164726848 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚17039360 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚24820 NBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚7041735 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚894 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚2425037 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚894 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 582234453 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚285 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 PBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 586244608 R?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚31895401219584 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 EDeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚17039360 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 310206464 M=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 13805551616 WGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 13207863296 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚894 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚609 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 315220480 MADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚8013482 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚603 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24903680 _step‚30 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚601 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 YFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚30840147687488 YIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 13207863296 _runtime‚ 647.860978905 G:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚17914676 N>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 13118517709 L>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 247635968 QCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 384795648 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚894 BDeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚17039360 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 310206464 M=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 13805551616 WGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 13207863296 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚894 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚609 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 315220480 MADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚8013482 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚603 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24903680 _step‚30 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚601 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 YFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚30840147687488 YIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 13207863296 _runtime‚ 647.860978905 G:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚17914676 N>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 13118517709 L>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 247635968 QCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 384795648 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚894 BDeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚7041132 trainer/global_step‚399 ODeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 13567453184 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 230596608 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚285 VCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚31510947547136 RBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 13035530240 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚33 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚2425638 R?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚31222422399530 VCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚31222666025122 [JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 382501298274 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 I=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚9876374 K;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 13567453184 YFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚31510947547136 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 VHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 226586232 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚2 QEDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚8013482 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚3689 [MDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚ 285302784 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚285 TCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 384453672448 VCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚31895648855552 [JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 384684269056 MDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚1299 UEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 12589930496 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚30637 ZGDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚31510964586496 FDeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚17914676 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚24820 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚30392 ]JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚30840164726848 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚17039360 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚24820 NBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚7041735 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚894 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚2425037 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚894 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 582234453 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚285 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 PBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 586244608 R?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚31895401219584 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 EDeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚35789394377216 TDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 13805551616 L?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚11084391 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 H?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚1321 YFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚36302650177536 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚24903680 OCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚8991375 VEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 431397591040 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚2 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚2721766 SEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 384795648 UGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 582234453 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 XEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚35357996786176 TADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚35789394377216 SGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚2721197 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 O?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 13805551616 ]LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 431628174336 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚24794 YHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 431397591040 JBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚285 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 230583296 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 BDeviceStatsMonitor.on_train_batch_start/allocation.all.current‚24796 SGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚8991375 P@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 13118517709 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 ZIDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 431628174336 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚3689 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚571 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 597688320 E8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚20075764 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚30392 PDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚7901020 ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 287413248 YIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 13207863296 TDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 13035530240 ]LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 429178792380 % _timestamp‚1.7885016609277163e+09 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚569 bQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 451886198784 P@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 13567453184 ]JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚36302962494464 RBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 13805551616 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚609 UBDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated‚35789641999872 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 230583296 ^MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 451598785536 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 WGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 12589930496 [HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚35357996786176 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚5179254 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 152061440 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 597688320 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚24796 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚30392 JBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚609 G?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚609 trainer/global_step‚449 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚30637 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 586244608 MFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚33 dQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚35851076295680 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 E=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚894 _runtime‚ 737.008195543 L@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚7900449 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 TGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚11109185 D>DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 C;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚894 K?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚8991373 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 586244608 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚285 M@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚20100560 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚30637 OCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚8991373 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 QCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 243625384 TADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚35034589886500 YHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 428952206356 _step‚33 G>DeviceStatsMonitor.on_train_batch_start/active.large_poD¿Šÿ›ol.peak‚3689 N@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 247622656 PCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚11084391 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 XEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚35034833511884 XJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 226586024 TFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 597688320 PCDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated‚11109185 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚24794 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚17039360 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 315220480 _LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚34605654719504 G?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚285 \IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚35358013825536 XEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚35789641999872 WGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 13035530240 SGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚5179252 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1'sÃ'÷½O LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚285 M@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚20100560 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚30637 OCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚8991373 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 QCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 243625384 TADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚35034589886500 YHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 428952206356 _step‚33 G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚3689 N@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 247622656 PCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚11084391 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 XEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚35034833511884 XJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 226586024 TFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 597688320 PCDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated‚11109185 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚24794 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚17039360 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 315220480 _LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚34605654719504 G?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚285 \IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚35358013825536 XEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚35789641999872 WGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 13035530240 SGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚5179252 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 M=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 13567453184 IDeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚35789394377216 TDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 13805551616 L?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚11084391 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 H?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚1321 YFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚36302650177536 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚24903680 OCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚8991375 VEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 431397591040 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚2 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚2721766 SEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 384795648 UGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 582234453 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 XEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚35357996786176 TADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚35789394377216 SGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚2721197 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 O?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 13805551616 ]LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 431628174336 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚24794 YHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 431397591040 JBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚285 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 230583296 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 BDeviceStatsMonitor.on_train_batch_start/allocation.all.current‚24796 SGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚8991375 P@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 13118517709 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 ZIDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 431628174336 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚3689 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚571 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 597688320 E8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚20075764 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚30392 PDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚7901020 ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 287413248 YIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 13207863296 TDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 13035530240 ]LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 429178792380 % _timestamp‚1.7885016609277163e+09 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚569 bQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 451886198784 P@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 13567453184 ]JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚36302962494464 RBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 13805551616 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚609 UBDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated‚35789641999872 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 230583296 ^MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 451598785536 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 WGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 12589930496 [HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚35357996786176 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚5179254 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 152061440 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 597688320 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚24796 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚30392 JBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚609 G?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚609 trainer/global_step‚449 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚30637 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 586244608 MFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚33 dQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚35851076295680 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 E=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚894 _runtime‚ 737.008195543 L@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚7900449 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 TGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚11109185 D>DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 C;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚894 K?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚8991373 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 586244608!þ90ø+ _wandbruntime‚739  _runtime‚739¦ÿîÀÈ&ùÂM =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚30637 K;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 13567453184 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚30637 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚3689 ^KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚35930646252032 [JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 432585658880 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚609 NADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚11133796 VCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚35869110693376 OADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 247635968 YFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚35436507995136 NBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚7918435 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚894 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚609 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚285 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 243625592 YFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚34682484414688 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚24822 I=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚9011330 ODeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚7917850 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚5190747 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚583 QCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 384795648 PBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 586244608 ]JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚35436525034496 WFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 432355062272 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚30392 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 P@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 13805551616 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 230596608 TCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 432355062272 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 230596608 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚585 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 315220480 EDeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚20145128 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 ]JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚34682501454048 M=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 13805551616 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 SEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 586244608 BDeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚2 bODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚35930671155712 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24903680 WFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 429904253750 WGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 13207863296 [HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚36383550157824 [MDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚ 276914176 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚24820 FDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 UEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 12589930496 G:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚20145128 C6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚20120306 N>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 13567453184 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚24820 [JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 430130839982 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 582234453 XGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 432585658880 RBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 13035530240 QEDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚9011332 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚33 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 301817856 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 S@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚35869110693376 VCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚35436507995136 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 152061440 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 YIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 13207863296 WIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 597688320 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚894 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17039360 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚2727688 WDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚36383248339968 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 N>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 13118517709 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 597688320 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚894 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚17039360 L>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 247635968 `ODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 452879002112 VCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚35112632294030 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚609 MADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚9011332 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 597688320 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚894 R?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚35868863057408 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1321 R?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚35112388668438‚cŸ»Ì&úÆM I=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚9011330 ODeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚7917850 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚5190747 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚583 QCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 384795648 PBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 586244608 ]JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚35436525034496 WFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 432355062272 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚30392 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 P@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 13805551616 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 230596608 TCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 432355062272 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 230596608 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚585 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 315220480 EDeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚20145128 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 ]JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚34682501454048 M=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 13805551616 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 SEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 586244608 BDeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚2 bODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚35930671155712 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24903680 WFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 429904253750 WGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 13207863296 [HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚36383550157824 [MDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚ 276914176 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚24820 FDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 UEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 12589930496 G:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚20145128 C6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚20120306 N>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 13567453184 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚24820 [JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 430130839982 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 582234453 XGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 432585658880 RBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 13035530240 QEDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚9011332 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚33 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 301817856 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 S@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚35869110693376 VCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚35436507995136 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 152061440 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 YIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 13207863296 WIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 597688320 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚894 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17039360 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚2727688 WDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚36383248339968 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 N>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 13118517709 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 597688320 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚894 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚17039360 L>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 247635968 `ODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 452879002112 VCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚35112632294030 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚609 MADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚9011332 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 597688320 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚894 R?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚35868863057408 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1321 R?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚35112388668438 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚30637 K;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 13567453184 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚30637 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚3689 ^KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚35930646252032 [JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 432585658880 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚609 NADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚11133796 VCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚35869110693376 OADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 247635968 YFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚35436507995136 NBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚7918435 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚894 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚609 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚285 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 243625592 YFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚34682484414688 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚24822"ªã²³0û+ _wandbruntime‚739  _runtime‚739þèßF}üÌ trainer/global_step‚449 /loss/FRt2/assignment_loss‚5.995434462134518 epoch‚0 _runtime‚ 739.418919445 1loss/FBt1/assignment_loss‚0.46761072282097305 0loss/SRqqt2/detection_loss‚2.685257188173441 /loss/FBt4/detection_loss‚3.7188888606817825 % _timestamp‚1.7885016633392448e+09 0loss/FBt2/assignment_loss‚0.5230387076735497 0loss/FRt4/assignment_loss‚4.2944724281628925 2loss/SRqqt4/assignment_loss‚2.5340347736455593 /loss/FRt3/detection_loss‚2.1579563681797316 /loss/FRt4/detection_loss‚1.9735973097383976 0loss/SRqqt3/detection_loss‚3.485754958788554 2loss/SRqqt2/assignment_loss‚2.5477073431015014 0loss/SRqqt3/assignment_loss‚3.05985271135966 0loss/FBt4/assignment_loss‚0.6209922826761296 .loss/FRt1/detection_loss‚2.747825977660842 -loss/FBt1/detection_loss‚5.88603845834732 /loss/FBt3/detection_loss‚0.8497243867440648 _step‚35 0loss/FRt3/assignment_loss‚3.5239082683799086 1loss/FBt3/assignment_loss‚0.25688737832244873 .loss/FRt2/detection_loss‚2.909690301809738 0loss/SRqqt1/detection_loss‚2.067252208611795 /loss/FRt1/assignment_l ¿åÕoss‚5.699453329635879 2loss/SRqqt1/assignment_loss‚2.7635248195646063 1loss/SRqqt4/detection_loss‚0.9137513618066103 .loss/FBt2/detection_loss‚3.746693529188633 %loss/total_loss‚65.42934813720854¦4rVýÐ 0loss/FBt2/assignment_loss‚0.5230387076735497 0loss/FRt4/assignment_loss‚4.2944724281628925 2loss/SRqqt4/assignment_loss‚2.5340347736455593 /loss/FRt3/detection_loss‚2.1579563681797316 /loss/FRt4/detection_loss‚1.9735973097383976 0loss/SRqqt3/detection_loss‚3.485754958788554 2loss/SRqqt2/assignment_loss‚2.5477073431015014 0loss/SRqqt3/assignment_loss‚3.05985271135966 0loss/FBt4/assignment_loss‚0.6209922826761296 .loss/FRt1/detection_loss‚2.747825977660842 -loss/FBt1/detection_loss‚5.88603845834732 /loss/FBt3/detection_loss‚0.8497243867440648 _step‚35 0loss/FRt3/assignment_loss‚3.5239082683799086 1loss/FBt3/assignment_loss‚0.25688737832244873 .loss/FRt2/detection_loss‚2.909690301809738 0loss/SRqqt1/detection_loss‚2.067252208611795 /loss/FRt1/assignment_loss‚5.699453329635879 2loss/SRqqt1/assignment_loss‚2.7635248195646063 1loss/SRqqt4/detection_loss‚0.9137513618066103 .loss/FBt2/detection_loss‚3.746693529188633 %loss/total_loss‚65.42934813720854 trainer/global_step‚449 /loss/FRt2/assignment_loss‚5.995434462134518 epoch‚0 _runtime‚ 739.418919445 1loss/FBt1/assignment_loss‚0.46761072282097305 0loss/SRqqt2/detection_loss‚2.685257188173441 /loss/FBt4/detection_loss‚3.7188888606817825 % _timestamp‚1.7885016633392448e+09#¿`”‘þ‚(:† ©½éÔ‘Žƒ» gpu.0.gpu‚1.0 gpu.0.memory‚0, gpu.0.memoryAllocated‚16.716384887695312* gpu.0.memoryAllocatedBytes‚ 14359265280 gpu.0.temp‚31.0 gpu.0.powerWatts‚93.386' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚18.6772 gpu.0.smClock‚1275 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0vˆ²uÿ‚(:ê ©½éÔèâ«¿ disk.dm-1.out‚ 23.74609375 disk.drbd1260.out‚ 0.10546875 disk.sda1.out‚0 disk.sda.in‚0 disk.sda2.in‚0 disk.dm-0.in‚ 1788.20703125 disk.sda.out‚ 23.74609375$ memory_percent‚93.02666187286377) proc.memory.availableMB‚ 1142.51171875* proc.memory.percent‚17.833685874938965 network.sent‚722632) disk./.usagePercent‚78.79204887908841 disk.drbd1260.in‚0# proc.memory.rssMB‚ 2921.87109375 cpu‚25.63282476831433 proc.cpu.threads‚17% disk./.usageGB‚1174.0584678649902 disk.sda3.in‚0 disk.sda3.out‚ 23.74609375# disk.nvme0c0n1.in‚ 1788.20703125( disk.nvme0c0n1.out‚16862.78564453125& disk.nvme0n1.out‚16862.78564453125 disk.dm-1.in‚0 disk.sda1.in‚0 network.recv‚326929 disk.sda2.out‚0! disk.nvme0n1.in‚ 1788.20703125# disk.dm-0.out‚16862.78564453125å…{3“€‚(:ˆ ¸½éÔ™½±¹ gpu.0.gpu‚33.0 gpu.0.memory‚3, gpu.0.memoryAllocated‚16.716384887695312* gpu.0.memoryAllocatedBytes‚ 14359265280 gpu.0.temp‚32.0 gpu.0.powerWatts‚114.479' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚22.8958 gpu.0.smClock‚1275 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0 ¡ s‚(:è ¸½éÔ…‘ê» disk.sda3.in‚0 disk.sda3.out‚ 24.1953125( disk.nvme0c0n1.out‚17263.24658203125 disk.dm-0.in‚ 1788.20703125 disk.sda1.in‚0 disk.sda1.out‚0# proc.memory.rssMB‚ 2921.87109375* proc.memory.percent‚17.833685874938965 network.recv‚332098! disk.nvme0n1.in‚ 1788.20703125# disk.dm-0.out‚17263.24658203125 disk.dm-1.out‚ 24.1953125 disk.drbd1260.in‚0 disk.sda.out‚ 24.1953125 disk.sda2.in‚0 disk.sda2.out‚0 network.sent‚772026) disk./.usagePercent‚78.80875823484075% disk./.usageGB‚1174.3074493408203# disk.nvme0c0n1.in‚ 1788.20703125& disk.nvme0n1.out‚17263.24658203125 disk.sda.in‚0$ memory_percent‚93.02666187286377) proc.memory.availableMB‚ 1142.51171875 disk.dm-1.in‚0 disk.drbd1260.out‚ 0.10546875 cpu‚25.631528799698003 proc.cpu.threads‚17ȳ\“‚‚(:ˆ ǽéÔ‚¯°¹ gpu.0.gpu‚24.0 gpu.0.memory‚2, gpu.0.memoryAllocated‚16.716384887695312* gpu.0.memoryAllocatedBytes‚ 14359265280 gpu.0.temp‚31.0 gpu.0.powerWatts‚111.122' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚22.2244 gpu.0.smClock‚1275 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0õ-rƒ‚(:ç ǽéÔåÛ» disk.sda3.in‚0# disk.nvme0c0n1.in‚ 1946.89453125( disk.nvme0c0n1.out‚17520.50146484375 disk.dm-1.in‚0 disk.drbd1260.out‚0.109375 disk.sda.in‚0 network.recv‚336405! disk.nvme0n1.in‚ 1946.89453125# disk.dm-0.out‚17520.50146484375 disk.drbd1260.in‚0 disk.sda1.in‚0 disk.sda1.out‚0 disk.dm-0.in‚ 1946.89453125) disk./.usagePercent‚78.80875823484075 disk.sda2.out‚0& disk.nvme0n1.out‚17520.50146484375 disk.dm-1.out‚ 24.51171875$ memory_percent‚93.02816390991211' proc.memory.availableMB‚ 1142.265625* proc.memory.percent‚17.833685874938965% disk./.usageGB‚1174.3074493408203 disk.sda3.out‚ 24.51171875 disk.sda.out‚ 24.51171875# proc.memory.rssMB‚ 2921.87109375 cpu‚25.629791813402942 proc.cpu.threads‚17 network.sent‚777389 disk.sda2.in‚0y"ÂÌ“„‚(:ˆ Ö½éÔÔé×¹ gpu.0.gpu‚33.0 gpu.0.memory‚2, gpu.0.memoryAllocated‚16.716384887695312* gpu.0.memoryAllocatedBytes‚ 14359265280 gpu.0.temp‚31.0 gpu.0.powerWatts‚115.521' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚23.1042 gpu.0.smClock‚1275 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0ž¡m…‚(:â Ö½éÔÌøš»' disk./.usagePercent‚78.724070430499$ disk./.usageGB‚1173.045539855957 disk.sda1.out‚0 disk.dm-0.in‚ 1946.89453125 disk.drbd1260.in‚0 disk.drbd1260.out‚0.109375# proc.memory.rssMB‚ 2921.87109375 disk.sda2.out‚0 disk.sda3.in‚0! disk.nvme0n1.in‚ 1946.89453125# disk.dm-0.out‚17969.13427734375$ memory_percent‚93.02785396575928 disk.sda1.in‚0 disk.sda.in‚0 disk.sda.out‚ 24.8984375 disk.sda3.out‚ 24.8984375( disk.nvme0c0n1.out‚17969.13427734375 disk.dm-1.in‚0 disk.dm-1.out‚ 24.8984375) proc.memory.availableMB‚ 1142.31640625 network.recv‚341333 disk.sda2.in‚0# disk.nvme0c0n1.in‚ 1946.89453125& disk.nvme0n1.out‚17969.13427734375* proc.memory.percent‚17.833685874938965 cpu‚25.62866201615989 proc.cpu.threads‚17 network.sent‚783575’3+’†‚(:‡ å½éÔ½˜®¹ gpu.0.gpu‚62.0 gpu.0.memory‚5, gpu.0.memoryAllocated‚16.716384887695312* gpu.0.memoryAllocatedBytes‚ 14359265280 gpu.0.temp‚33.0 gpu.0.powerWatts‚120.525' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚24.105 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0‘¹pím‡‚(:â å½éÔ—œ–»# disk.nvme0n1.out‚18554.22265625$ memory_percent‚93.02632808685303 disk.dm-0.in‚ 1946.89453125 disk.dm-1.out‚ 25.22265625 disk.sda2.in‚0 disk.sda2.out‚0 disk.sda3.in‚0# disk.nvme0c0n1.in‚ 1946.89453125% disk.nvme0c0n1.out‚18554.22265625 proc.cpu.threads‚17% disk./.usageGB‚1173.7042655944824 disk.dm-0.out‚18554.22265625 disk.dm-1.in‚0 disk.sda.in‚0 cpu‚25.626413133755086 network.sent‚788600 network.recv‚345007 disk.sda1.out‚0 disk.sda3.out‚ 25.22265625) proc.memory.availableMB‚ 1142.56640625# proc.memory.rssMB‚ 2921.87109375* proc.memory.percent‚17.833685874938965 disk.sda.out‚ 25.22265625) disk./.usagePercent‚78.76827806751913 disk.drbd1260.in‚0 disk.drbd1260.out‚ 0.11328125 disk.sda1.in‚0! disk.nvme0n1.in‚ 1946.89453125;À:žˆ‚(:“ ô½éÔ–Ú»º gpu.0.gpu‚24.0 gpu.0.memory‚3, gpu.0.memoryAllocated‚16.716384887695312* gpu.0.memoryAllocatedBytes‚ 14359265280 gpu.0.temp‚34.0 gpu.0.powerWatts‚170.028' gpu.0.enforcedPowerLimitWatts‚500.0) gpu.0.powerPercent‚34.005599999999994 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0'Ÿ§ñl‰‚(:á ô½éÔ¹ñ³») disk./.usagePercent‚78.80327301741211$ disk./.usageGB‚1174.225715637207# disk.nvme0c0n1.in‚ 1946.89453125! disk.nvme0n1.in‚ 1946.89453125# disk.nvme0n1.out‚18907.69140625 disk.dm-0.in‚ 1946.89453125 network.sent‚793727 disk.sda2.in‚0 disk.sda2.out‚0% disk.nvme0c0n1.out‚18907.69140625 disk.dm-0.out‚18907.69140625 disk.sda1.out‚0 cpu‚25.625946967382152 disk.sda3.out‚ 25.69921875 disk.drbd1260.out‚ 0.11328125 disk.sda.in‚0 disk.sda.out‚ 25.69921875$ memory_percent‚93.02785396575928) proc.memory.availableMB‚ 1142.31640625# proc.memory.rssMB‚ 2921.87109375* proc.memory.percent‚17.833685874938965 network.recv‚348432 disk.sda3.in‚0 disk.dm-1.in‚0 disk.dm-1.out‚ 25.69921875 disk.drbd1260.in‚0 disk.sda1.in‚0 proc.cpu.threads‚17Æî0Š+ _wandbruntime‚828  _runtime‚828Çrÿ;‹¸ $ lr-AdamW/pg1‚0.00107770683135564 $ lr-AdamW/pg2‚0.00107770683135564 trainer/global_step‚499 $ _timestamp‚1.788501752035592e+09 _runtime‚ 828.11431242 _step‚36-OÒZÂŒ¼ trainer/global_step‚499 $ _timestamp‚1.788501752035592e+09 _runtime‚ 828.11431242 _step‚36 $ lr-AdamW/pg1‚0.00107770683135564 $ lr-AdamW/pg2‚0.00107770683135564$¼7ðÇ0+ _wandbruntime‚828  _runtime‚828”.¤¿'Ž¹O [HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚39283557234176 O?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 13805551616 M@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚22331012 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚2 bQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 501987121152 _LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚38447991446704 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 597688320 C;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚894 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 UBDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated‚39763103837696 Q>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚39762856215040 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚609 K?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚9989223 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 TFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 597688320 PDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚8779997 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 YHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 476581748064 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 SEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 384795648 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚24796 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚30637 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 TDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 13805551616 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚24794 PCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚12316993 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 ^MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 501701805056 M=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 13567453184 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 247622656 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 230583296 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 586244608 P@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 13118517709 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚30637 OCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚9989225 \IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚39283574273536 MFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚33 YFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚40332496198656 _LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚39283574273536 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚3689 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚5753904 SGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚5753902 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 QCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 243625384 JBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚285 E8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚22306216 UGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 582234453 @8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚894 TADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚39762856215040 VFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 13207863296 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚30392 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚24903680 TGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚12341787 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 152061440 ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 285316096 [KDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 13207863296 JBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚609 IDeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚30392 ]LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 479529564160 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 315220480 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 WGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 12589930496 RBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 13805551616 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 trainer/global_step‚499 TADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚38924556155408 ]LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 476808334088 YIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 13207863296 G?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚609 L?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚12316993 VEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 479298980864 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 586244608 XEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚39763103837696 SGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚3025497 OFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚1299 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 230583296 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚17039360 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚24796 IDeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚3689 XEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚39283557234176 SGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚9989225 H?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚1321 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 @8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚894 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 % _timestamp‚1.7885017520368855e+09 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚2 BDeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 E=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚894 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 YHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 479298980864 N@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 247622656 L@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚8779399 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚3026093 ]JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚40332806418432 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 597688320Û‹4YÃ'½O RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 586244608 XEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚39763103837696 SGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚3025497 OFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚1299 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 230583296 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚17039360 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚24796 IDeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚3689 XEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚39283557234176 SGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚9989225 H?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚1321 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 @8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚894 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 % _timestamp‚1.7885017520368855e+09 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚2 BDeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 E=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚894 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 YHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 479298980864 N@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 247622656 L@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚8779399 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚3026093 ]JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚40332806418432 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 597688320 [HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚39283557234176 O?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 13805551616 M@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚22331012 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚2 bQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 501987121152 _LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚38447991446704 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 597688320 C;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚894 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 UBDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated‚39763103837696 Q>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚39762856215040 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚609 K?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚9989223 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 TFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 597688320 PDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚8779997 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 YHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 476581748064 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 SEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 384795648 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚24796 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚30637 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 TDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 13805551616 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚24794 PCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚12316993 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 ^MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 501701805056 M=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 13567453184 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 247622656 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 230583296 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 586244608 P@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 13118517709 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚30637 OCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚9989225 \IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚39283574273536 MFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚33 YFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚40332496198656 _LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚39283574273536 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚3689 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚5753904 SGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚5753902 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 QCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 243625384 JBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚285 E8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚22306216 UGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 582234453 @8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚894 TADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚39762856215040 VFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 13207863296 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚30392 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚24903680 TGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚12341787 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 152061440 ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 285316096 [KDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 13207863296 JBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚609 IDeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚30392 ]LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 479529564160 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 315220480 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 WGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 12589930496 RBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 13805551616 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 trainer/global_step‚499 TADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚38924556155408 ]LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 476808334088 YIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 13207863296 G?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚609 L?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚12316993 VEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 479298980864%vw0+ _wandbruntime‚830  _runtime‚830ä* ‘ÇM >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚894 BDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 13567453184 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚10009180 _step‚38 R?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚39842324895232 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1321 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚598 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 597688320 SEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 586244608 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚5765397 WDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚40413097017344 YFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚38524821141888 VHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 226586232 P@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 13805551616 XGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 480487048704 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 R?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚39002354937346 WIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 597688320 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚2 ]JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚39362085482496 REDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚10009182 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚285 trainer/global_step‚499 C6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚22350758 MDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚1299 M=DevicÁä²í>eStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 13805551616 K;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 13567453184 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 243625592 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚30637 VCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚39362068443136 S@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚39842572531200 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 230596608 ]JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚38524838181248 QCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 384795648 [MDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚ 276914176 \KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 502707763712 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 [JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 480487048704 [HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚40413398835200 bODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚39910414157312 G:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚22375580 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 152061440 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 UEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 12589930496 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚894 N>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 13118517709 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚609 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚3689 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚2 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚285 WGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 13207863296 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚17039360 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚5765395 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚17039360 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚609 NADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚12366398 NBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚8797656 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 VCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚39002598562938 YFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚39362068443136 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚894 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 K>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚22375580 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 % _timestamp‚1.7885017540935743e+09 RBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 13035530240 G:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚22350758 [JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 477760381690 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 NADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚10009182 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚24820 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚3031661 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 301817856 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 315220480 WFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 477533795458 ODeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 247635968 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚24822 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 _runtime‚ 830.175478953 EDeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚8797056 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚3032259 FDeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚24820 ^KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚39910389253632 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚30637 WFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 480256452096 F6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚894 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 K>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚22375580 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 % _timestamp‚1.7885017540935743e+09 RBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 13035530240 G:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚22350758 [JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 477760381690 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 NADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚10009182 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚24820 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚3031661 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 301817856 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 315220480 WFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 477533795458 ODeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 247635968 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚24822 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 _runtime‚ 830.175478953 EDeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚8797056 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚3032259 FDeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚24820 ^KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚39910389253632 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚30637 WFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 480256452096 F6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚894 BDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 13567453184 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚10009180 _step‚38 R?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚39842324895232 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1321 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚598 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 597688320 SEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 586244608 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚5765397 WDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚40413097017344 YFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚38524821141888 VHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 226586232 P@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 13805551616 XGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 480487048704 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 R?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚39002354937346 WIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 597688320 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚2 ]JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚39362085482496 REDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚10009182 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚285 trainer/global_step‚499 C6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚22350758 MDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚1299 M=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 13805551616 K;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 13567453184 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 243625592 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚30637 VCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚39362068443136 S@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚39842572531200 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 230596608 ]JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚38524838181248 QCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 384795648 [MDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚ 276914176 \KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 502707763712 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 [JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 480487048704 [HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚40413398835200 bODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚39910414157312 G:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚22375580 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 152061440 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 UEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 12589930496 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚894 N>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 13118517709 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚609&¬kžÌ0“+ _wandbruntime‚830  _runtime‚830ðÁíðP”Ê 0loss/FRt4/assignment_loss‚3.3263119150252236 1loss/FBt3/assignment_loss‚0.26069482924383963 /loss/FRt4/detection_loss‚2.0053071755997456 /loss/FBt3/detection_loss‚0.8154137319188131 /loss/FBt4/detection_loss‚2.8273050122790866 trainer/global_step‚499 _step‚39 /loss/FRt3/assignment_loss‚3.524463790384206 1loss/SRqqt3/assignment_loss‚3.544288396835327 1loss/SRqqt1/detection_loss‚2.7243820336811684 /loss/FBt2/assignment_loss‚0.591894854126232 /loss/FBt1/detection_loss‚3.6768959118769717 .loss/FBt2/detection_loss‚4.233271139008658 /loss/FRt1/assignment_loss‚5.963734922274737 0loss/FBt4/assignment_loss‚0.6832705042220537 /loss/FRt2/assignment_loss‚6.054049110412597 2loss/SRqqt1/assignment_loss‚2.5473673023721752 -loss/FRt2/detection_loss‚2.94586483331827 epoch‚0 # _timestamp‚1.78850175410125e+09 1loss/SRqqt2/assignment_loss‚2.776634014610733 2loss/SRqqt4/assignment_loss‚2.2416709314176546 /loss/FRt1/detection_loss‚2.5154792901495813 0loss/SRqqt3/detection_loss‚3.819370651245117 1loss/SRqqt4/detection_loss‚0.8924027011958078 _runtime‚ 830.181380582 0loss/FBt1/assignment_loss‚0.4172557637702994 /loss/FRt3/detection_loss‚2.2122567135163327 0loss/SRqqt2/detection_loss‚2.714588719287089 %loss/total_loss‚63.31417424777172òÛ T•Î /loss/FRt3/assignment_loss‚3.524463790384206 1loss/SRqqt3/assignment_loss‚3.544288396835327 1loss/SRqqt1/detection_loss‚2.7243820336811684 /loss/FBt2/assignment_loss‚0.591894854126232 /loss/FBt1/detection_loss‚3.6768959118769717 .loss/FBt2/detection_loss‚4.233271139008658 /loss/FRt1/assignment_loss‚5.963734922274737 0loss/FBt4/assignment_loss‚0.6832705042220537 /loss/FRt2/assignment_loss‚6.054049110412597 2loss/SRqqt1/assignment_loss‚2.5473673023721752 -loss/FRt2/detection_loss‚2.94586483331827 epoch‚0 # _timestamp‚1.78850175410125e+09 1loss/SRqqt2/assignment_loss‚2.776634014610733 2loss/SRqqt4/assignment_loss‚2.2416709314176546 /loss/FRt1/detection_loss‚2.5154792901495813 0loss/SRqqt3/detection_loss‚3.819370651245117 1loss/SRqqt4/detection_loss‚0.8924027011958078 _runtime‚ 830.181380582 0loss/FBt1/assignment_loss‚0.4172557637702994 /loss/FRt3/detection_loss‚2.2122567135163327 0loss/SRqqt2/detection_loss‚2.714588719287089 %loss/total_loss‚63.31417424777172 0loss/FRt4/assignment_loss‚3.3263119150252236 1loss/FBt3/assignment_loss‚0.26069482924383963 /loss/FRt4/detection_loss‚2.0053071755997456 /loss/FBt3/detection_loss‚0.8154137319188131 /loss/FBt4/detection_loss‚2.8273050122790866 trainer/global_step‚499 _step‚39'"ù­’“–‚(:ˆ ƒ¾éÔí𤻠gpu.0.gpu‚41.0 gpu.0.memory‚3, gpu.0.memoryAllocated‚16.716384887695312* gpu.0.memoryAllocatedBytes‚ 14359265280 gpu.0.temp‚33.0 gpu.0.powerWatts‚115.012' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚23.0024 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0gt1ío—‚(:ä ƒ¾éÔÄÞ¾ disk.sda.out‚ 26.4765625 disk.sda3.in‚0# disk.dm-0.out‚19069.07470703125 disk.sda1.in‚0* proc.memory.percent‚17.833685874938965 network.recv‚353120% disk./.usageGB‚1174.2258071899414 disk.sda.in‚0 disk.sda2.in‚0& disk.nvme0n1.out‚19069.07470703125 disk.dm-0.in‚ 2122.7421875 disk.drbd1260.in‚0 disk.drbd1260.out‚ 0.1171875 disk.sda2.out‚0 disk.sda3.out‚ 26.4765625( disk.nvme0c0n1.out‚19069.07470703125 disk.dm-1.in‚0 disk.sda1.out‚0) proc.memory.availableMB‚ 1142.48828125# proc.memory.rssMB‚ 2921.87109375 cpu‚25.625060252199976 network.sent‚799666) disk./.usagePercent‚78.80327916159294" disk.nvme0c0n1.in‚ 2122.7421875 disk.nvme0n1.in‚ 2122.7421875 disk.dm-1.out‚ 26.4765625$ memory_percent‚93.02680492401123 proc.cpu.threads‚17Ö-“˜‚(:ˆ ’¾éÔ£Êú¸ gpu.0.gpu‚48.0 gpu.0.memory‚6, gpu.0.memoryAllocated‚16.716384887695312* gpu.0.memoryAllocatedBytes‚ 14359265280 gpu.0.temp‚34.0 gpu.0.powerWatts‚159.776' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚31.9552 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0Ìþ]Gp™‚(:å ’¾éÔç¨Åº proc.cpu.threads‚17) disk./.usagePercent‚78.80327916159294 disk.dm-0.in‚ 2122.7421875 disk.dm-1.out‚ 26.94140625 disk.sda2.out‚0$ memory_percent‚93.02854537963867 network.recv‚359360% disk./.usageGB‚1174.2258071899414" disk.nvme0c0n1.in‚ 2122.7421875( disk.nvme0c0n1.out‚19254.34033203125 disk.nvme0n1.in‚ 2122.7421875& disk.nvme0n1.out‚19254.34033203125 disk.sda1.out‚0 disk.sda.in‚0 disk.sda1.in‚0 disk.sda2.in‚0 disk.sda3.out‚ 26.94140625' proc.memory.availableMB‚ 1142.203125# proc.memory.rssMB‚ 2921.87109375* proc.memory.percent‚17.833685874938965 cpu‚25.621808129901588 disk.dm-1.in‚0 disk.drbd1260.in‚0 disk.drbd1260.out‚ 0.1171875 network.sent‚849555# disk.dm-0.out‚19254.34033203125 disk.sda.out‚ 26.94140625 disk.sda3.in‚0C¾¹œ’š‚(:‡ ¡¾éÔžœú¸ gpu.0.gpu‚1.0 gpu.0.memory‚0, gpu.0.memoryAllocated‚16.716384887695312* gpu.0.memoryAllocatedBytes‚ 14359265280 gpu.0.temp‚32.0 gpu.0.powerWatts‚114.748' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚22.9496 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0æªh>|›‚(:ñ ¡¾éÔå¦÷º disk.dm-1.out‚64.21875 disk.sda1.out‚0 disk.drbd1260.in‚0 disk.sda2.in‚0" disk.nvme0c0n1.in‚ 2122.7421875( disk.nvme0c0n1.out‚19704.68603515625 disk.nvme0n1.in‚ 2122.7421875& disk.nvme0n1.out‚19704.68603515625$ memory_percent‚93.02701950073242* proc.memory.percent‚17.833685874938965' proc.memory.availableMB‚ 1142.453125# proc.memory.rssMB‚ 2921.87109375 cpu‚25.61788619308364 proc.cpu.threads‚17# disk.dm-0.out‚19704.68603515625 disk.drbd1260.out‚ 0.12109375 disk.sda1.in‚0 disk.sda.in‚0.203125 disk.sda.out‚64.21875 disk.sda2.out‚0 disk.sda3.in‚0.203125 disk.sda3.out‚64.21875 network.sent‚853870 network.recv‚361999) disk./.usagePercent‚78.80328300170595% disk./.usageGB‚1174.2258644104004 disk.dm-0.in‚ 2122.7421875 disk.dm-1.in‚0.203125_ŒÚØœ‚(:… °¾éÔ¿±º gpu.0.gpu‚0.0 gpu.0.memory‚0, gpu.0.memoryAllocated‚16.716384887695312* gpu.0.memoryAllocatedBytes‚ 14359265280 gpu.0.temp‚31.0 gpu.0.powerWatts‚93.455' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚18.691 gpu.0.smClock‚1275 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0%J¼1”‚(:‰ °¾éԪ떼 network.recv‚365652 disk.sda3.in‚ 0.69140625" disk.nvme0c0n1.in‚ 2122.7421875' proc.memory.availableMB‚ 1142.453125 cpu‚25.614040737641268 proc.cpu.threads‚17 disk.sda2.out‚0 disk.sda3.out‚ 101.7265625( disk.nvme0c0n1.out‚19936.96728515625 disk.dm-0.in‚ 2122.7421875 disk.drbd1260.in‚0 disk.sda1.out‚0$ memory_percent‚93.02701950073242* proc.memory.percent‚17.833685874938965 network.sent‚858890) disk./.usagePercent‚78.80171981970187 disk.nvme0n1.in‚ 2122.7421875# disk.dm-0.out‚19936.96728515625 disk.dm-1.out‚ 101.7265625 disk.drbd1260.out‚ 0.12109375 disk.sda1.in‚ 0.1015625# proc.memory.rssMB‚ 2921.87109375% disk./.usageGB‚1174.2025718688965 disk.sda2.in‚0& disk.nvme0n1.out‚19936.96728515625 disk.dm-1.in‚ 0.69140625 disk.sda.in‚ 0.79296875 disk.sda.out‚ 101.7265625ɛ‚(:“ ¿¾éÔ‘ºú¸ gpu.0.gpu‚32.0 gpu.0.memory‚3, gpu.0.memoryAllocated‚16.716384887695312* gpu.0.memoryAllocatedBytes‚ 14359265280 gpu.0.temp‚32.0 gpu.0.powerWatts‚113.374' gpu.0.enforcedPowerLimitWatts‚500.0) gpu.0.powerPercent‚22.674799999999998 gpu.0.smClock‚1275 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0.eÍm~Ÿ‚(:ó ¿¾éÔíÇ•» network.sent‚864746) disk./.usagePercent‚78.80329196196965 disk.sda3.in‚ 0.69140625! disk.nvme0c0n1.in‚ 2270.609375 disk.nvme0n1.in‚ 2270.609375 disk.dm-1.out‚ 123.28125 disk.sda1.in‚ 0.1015625 disk.sda1.out‚0 network.recv‚370119% disk./.usageGB‚1174.2259979248047 disk.sda3.out‚ 123.28125 disk.sda.in‚ 0.79296875 disk.sda.out‚ 123.28125$ memory_percent‚93.03011894226074# proc.memory.rssMB‚ 2921.87109375 disk.sda2.in‚0% disk.nvme0c0n1.out‚20266.55859375 disk.dm-0.out‚20266.55859375 disk.dm-1.in‚ 0.69140625 disk.drbd1260.out‚0.125 proc.cpu.threads‚17 disk.sda2.out‚0# disk.nvme0n1.out‚20266.55859375 disk.dm-0.in‚ 2270.609375 disk.drbd1260.in‚0( proc.memory.availableMB‚ 1141.9453125* proc.memory.percent‚17.833685874938965 cpu‚25.610444521285945S”c“ ‚(:ˆ ξéÔ­¶ƒ¹ gpu.0.gpu‚32.0 gpu.0.memory‚3, gpu.0.memoryAllocated‚16.716384887695312* gpu.0.memoryAllocatedBytes‚ 14359265280 gpu.0.temp‚31.0 gpu.0.powerWatts‚108.044' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚21.6088 gpu.0.smClock‚1275 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0A¾Ú¡‚(:ô ξéÔ’•Ðº cpu‚25.607194344339305 proc.cpu.threads‚17) disk./.usagePercent‚78.79224370082197% disk./.usageGB‚1174.0613708496094 disk.drbd1260.out‚0.125 disk.sda.in‚0.796875 disk.sda2.in‚0 disk.nvme0n1.in‚ 2270.609375 disk.dm-0.in‚ 2270.609375$ memory_percent‚93.03007125854492 network.recv‚374940 disk.sda1.in‚ 0.1015625 disk.sda1.out‚0 disk.sda3.in‚ 0.6953125! disk.nvme0c0n1.in‚ 2270.609375 disk.dm-1.out‚ 161.28515625* proc.memory.percent‚17.833685874938965 disk.drbd1260.in‚0 disk.sda2.out‚0 disk.dm-0.out‚ 20685.3828125 disk.dm-1.in‚ 0.6953125' proc.memory.availableMB‚ 1141.953125 network.sent‚870961 disk.sda.out‚ 161.28515625 disk.sda3.out‚ 161.28515625$ disk.nvme0c0n1.out‚ 20685.3828125" disk.nvme0n1.out‚ 20685.3828125# proc.memory.rssMB‚ 2921.87109375(Cö0¢+ _wandbruntime‚919  _runtime‚919ì-‰Â£¼ _step‚40 % lr-AdamW/pg1‚0.001185693487804101 % lr-AdamW/pg2‚0.001185693487804101 trainer/global_step‚549 % _timestamp‚1.7885018436208696e+09 _runtime‚ 919.6996021520ÈƤÀ _runtime‚ 919.699602152 _step‚40 % lr-AdamW/pg1‚0.001185693487804101 % lr-AdamW/pg2‚0.001185693487804101 trainer/global_step‚549 % _timestamp‚1.7885018436208696e+09(RuèÂ0¥+ _wandbruntime‚919  _runtime‚919éLÂä6¦½O H?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚1321 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 % _timestamp‚1.7885018436221373e+09 E8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚24536668 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚6328554 SGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚3329038 bQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 552162902528 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 586244608 TGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚13574389 VFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 13207863296 L?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚13549595 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 230583296 XEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚43736565675520 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 JBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚609 trainer/global_step‚549 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚614 SEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 384795648 [KDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 13207863296 TFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 597688320 IDeviceStatsMonitor.on_train_batch_start/allocation.all.current‚24796 @8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚894 G?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚609 MFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚33 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 306025472 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 \IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚43209134721536 M@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚24561464 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 PCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚10987073 L@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚9657590 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚3329652 ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 281121792 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 UBDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated‚43736565675520 [HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚43209117682176 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚2 P@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 13118517709 YHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 524211289772 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 L?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚10987073 `MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚43810537395200 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 UGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 582234453 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 597688320 M=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 13567453184 OFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚1299 @8DeviceÞ…ÝStatsMonitor.on_train_batch_start/num_device_alloc‚894 TADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚42814522424316 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 230583296 RBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 13805551616 _runtime‚ 919.701230489 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚30392 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 315220480 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 WGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 12589930496 O?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 13805551616 E=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚894 ]JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚44362725201408 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 _LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚43209134721536 SGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚6328552 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 C;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚894 XEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚43209117682176 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 586244608 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 247622656 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚17039360 YHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 527200370688 TGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚10987075 ^MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 551881780736 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 PCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚13549595 YFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚44362419175936 [HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚42290311134544 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚30637 PCDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated‚13574389 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 152061440 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚2 D>DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 IDeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚43736318052864 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚24794 WGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 13035530240 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚3689 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚616 ZIDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 527430953984 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚30637 dQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚43810562298880 G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚3689 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚30392 XEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚42814766049700 N@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 247622656 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚17039360 ]LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 527430953984 JBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚285 P@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 13567453184 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚24903680 G?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚285 PDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚9658206 QCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 243625384 _LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚42290328173904 TDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 13805551616 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 _step‚41 TADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚43736318052864 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 YIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 13207863296 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 597688320 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚609 XJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 226586024 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚285 ]LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 524437875796)W£ZÇ'§ÁO LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 C;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚894 XEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚43209117682176 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 586244608 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 247622656 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚17039360 YHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 527200370688 TGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚10987075 ^MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 551881780736 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 PCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚13549595 YFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚44362419175936 [HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚42290311134544 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚30637 PCDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated‚13574389 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 152061440 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚2 D>DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 IDeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚43736318052864 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚24794 WGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 13035530240 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚3689 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚616 ZIDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 527430953984 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚30637 dQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚43810562298880 G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚3689 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚30392 XEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚42814766049700 N@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 247622656 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚17039360 ]LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 527430953984 JBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚285 P@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 13567453184 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚24903680 G?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚285 PDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚9658206 QCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 243625384 _LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚42290328173904 TDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 13805551616 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 _step‚41 TADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚43736318052864 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 YIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 13207863296 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 597688320 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚609 XJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 226586024 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚285 ]LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 524437875796 H?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚1321 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 % _timestamp‚1.7885018436221373e+09 E8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚24536668 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚6328554 SGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚3329038 bQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 552162902528 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 586244608 TGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚13574389 VFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 13207863296 L?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚13549595 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 230583296 XEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚43736565675520 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 JBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚609 trainer/global_step‚549 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚614 SEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 384795648 [KDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 13207863296 TFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 597688320 IDeviceStatsMonitor.on_train_batch_start/allocation.all.current‚24796 @8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚894 G?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚609 MFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚33 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 306025472 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 \IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚43209134721536 M@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚24561464 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 PCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚10987073 L@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚9657590 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚3329652 ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 281121792 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 UBDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated‚43736565675520 [HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚43209117682176 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚2 P@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 13118517709 YHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 524211289772 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 L?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚10987073 `MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚43810537395200 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 UGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 582234453 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 597688320 M=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 13567453184 OFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚1299 @8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚894 TADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚42814522424316 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 230583296 RBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 13805551616 _runtime‚ 919.701230489 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚30392 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 315220480 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 WGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 12589930496 O?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 13805551616 E=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚894 ]JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚44362725201408 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 _LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚43209134721536 SGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚6328552)ö‰Ð«0¨+ _wandbruntime‚921  _runtime‚921:€uûÍ&©ÇM C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚17039360 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17039360 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 303915008 ^KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚43890132255232 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 582234453 WIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 597688320 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 G:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚24606032 K;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 13567453184 VCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚43287628891136 VCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚42892564831846 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚3334969 YFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚42367157869088 P@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 13805551616 RBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 13035530240 N>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 13567453184 NADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚13574180 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚33 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 SEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 586244608 [HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚44443324610048 [JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 525389923398 OADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 247635968 REDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚11007032 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚30637 NADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚11007032 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚894 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚17039360 REDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚13599000 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚894 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 243625592 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 YIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 13207863296 trainer/global_step‚549 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚579 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24903680 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 UEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 12589930496 bODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚43890157158912 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 152061440 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 L>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 247635968 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚3689 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚285 % _timestamp‚1.7885018453215797e+09 FDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 13118517709 ]JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚42367174908448 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 S@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚43816034369024 [JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 528388438528 WFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 528157841920 QCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 384795648 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 315220480 VHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 226586232 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚894 EDeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚24606032 MDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚1299 WDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚44443020695040 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚609 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚11007030 WFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 525163337166 TDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 13207863296 J=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚13574180 YFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚43287628891136 J>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚9675014 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1321 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚3335548 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 NADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚13599000 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚30637 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚24820 BDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 13567453184 NADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚13574180 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚33 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 SEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 586244608 [HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚44443324610048 [JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 525389923398 OADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 247635968 REDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚11007032 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚30637 NADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚11007032 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚894 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚17039360 REDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚13599000 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚894 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 243625592 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 YIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 13207863296 trainer/global_step‚549 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚579 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24903680 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 UEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 12589930496 bODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚43890157158912 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 152061440 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 L>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 247635968 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚3689 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚285 % _timestamp‚1.7885018453215797e+09 FDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 13118517709 ]JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚42367174908448 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 S@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚43816034369024 [JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 528388438528 WFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 528157841920 QCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 384795648 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 315220480 VHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 226586232 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚894 EDeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚24606032 MDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚1299 WDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚44443020695040 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚609 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚11007030 WFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 525163337166 TDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 13207863296 J=DeviceStatsMonitor.™)B³E on_train_batch_end/active.small_pool.freed‚13574180 YFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚43287628891136 J>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚9675014 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1321 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚3335548 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 NADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚13599000 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚30637 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚24820 BDeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 G:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚24606032 K;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 13567453184 VCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚43287628891136 VCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚42892564831846 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚3334969 YFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚42367157869088 P@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 13805551616*Y”[0«+ _wandbruntime‚921  _runtime‚921êtãIP¬Ê 0loss/FBt4/assignment_loss‚0.6935400780283999 /loss/FRt4/detection_loss‚1.9860715099743433 % _timestamp‚1.7885018453238897e+09 0loss/FBt2/assignment_loss‚0.8312142654807152 /loss/FRt2/detection_loss‚3.2415572237968444 1loss/SRqqt2/detection_loss‚2.7641507464426534 1loss/SRqqt3/detection_loss‚3.3950982163934147 1loss/SRqqt4/detection_loss‚0.8919598130929854 .loss/FRt2/assignment_loss‚5.79477216720581 -loss/FRt4/assignment_loss‚3.4102213132003 2loss/SRqqt2/assignment_loss‚2.4601767046584024 0loss/SRqqt1/detection_loss‚2.355855628083914 /loss/FBt1/detection_loss‚3.8350963780754492 /loss/FBt3/detection_loss‚0.8111953255008598 _step‚43 0loss/FRt3/assignment_loss‚3.3846028666747245 1loss/SRqqt4/assignment_loss‚2.072786698117852 1loss/FBt1/assignment_loss‚0.27404076979325365 /loss/FRt3/detection_loss‚2.2030735167625703 1loss/SRqqt1/assignment_loss‚2.363883963953226 .loss/FRt1/detection_loss‚2.412515808770686 epoch‚0 /loss/FRt1/assignment_loss‚4.927595291608646 .loss/FBt2/detection_loss‚3.313699550098843 .loss/FBt4/detection_loss‚2.339482386782765 /loss/FBt3/assignment_loss‚0.266635348480526 %loss/total_loss‚58.97390375834749 trainer/global_step‚549 2loss/SRqqt3/assignment_loss‚2.9446781873703003 _runtime‚ 921.402374685Üû€—T­Î epoch‚0 /loss/FRt1/assignment_loss‚4.927595291608646 .loss/FBt2/detection_loss‚3.313699550098843 .loss/FBt4/detection_loss‚2.339482386782765 /loss/FBt3/assignment_loss‚0.266635348480526 %loss/total_loss‚58.97390375834749 trainer/global_step‚549 2loss/SRqqt3/assignment_loss‚2.9446781873703003 _runtime‚ 921.402374685 0loss/FBt4/assignment_loss‚0.6935400780283999 /loss/FRt4/detection_loss‚1.9860715099743433 % _timestamp‚1.7885018453238897e+09 0loss/FBt2/assignment_loss‚0.8312142654807152 /loss/FRt2/detection_loss‚3.2415572237968444 1loss/SRqqt2/detection_loss‚2.7641507464426534 1loss/SRqqt3/detection_loss‚3.3950982163934147 1loss/SRqqt4/detection_loss‚0.8919598130929854 .loss/FRt2/assignment_loss‚5.79477216720581 -loss/FRt4/assignment_loss‚3.4102213132003 2loss/SRqqt2/assignment_loss‚2.4601767046584024 0loss/SRqqt1/detection_loss‚2.355855628083914 /loss/FBt1/detection_loss‚3.8350963780754492 /loss/FBt3/detection_loss‚0.8111953255008598 _step‚43 0loss/FRt3/assignment_loss‚3.3846028666747245 1loss/SRqqt4/assignment_loss‚2.072786698117852 1loss/FBt1/assignment_loss‚0.27404076979325365 /loss/FRt3/detection_loss‚2.2030735167625703 1loss/SRqqt1/assignment_loss‚2.363883963953226 .loss/FRt1/detection_loss‚2.412515808770686+Ž™Œ“®‚(:ˆ ݾéÔíÀ«º gpu.0.gpu‚32.0 gpu.0.memory‚3, gpu.0.memoryAllocated‚16.716384887695312* gpu.0.memoryAllocatedBytes‚ 14359265280 gpu.0.temp‚31.0 gpu.0.powerWatts‚115.966' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚23.1932 gpu.0.smClock‚1275 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0 k ‡¯‚(:ü ݾéÔ±» disk.sda.in‚0.796875$ memory_percent‚93.03004741668701( disk./.usagePercent‚78.7872500178591$ disk./.usageGB‚1173.986961364746 disk.sda3.in‚ 0.6953125% disk.nvme0c0n1.out‚20919.05078125 disk.drbd1260.in‚0 disk.drbd1260.out‚ 0.12890625 disk.sda1.out‚0 disk.sda.out‚ 161.69140625 disk.sda2.in‚0 disk.sda3.out‚ 161.69140625! disk.nvme0c0n1.in‚ 2270.609375# disk.nvme0n1.out‚20919.05078125 disk.dm-0.in‚ 2270.609375# proc.memory.rssMB‚ 2921.87109375* proc.memory.percent‚17.833685874938965 network.recv‚378320 disk.sda2.out‚0 disk.dm-0.out‚20919.05078125 disk.sda1.in‚ 0.1015625) proc.memory.availableMB‚ 1141.95703125 cpu‚25.603993317232327 proc.cpu.threads‚17 network.sent‚876110 disk.nvme0n1.in‚ 2270.609375 disk.dm-1.in‚ 0.6953125 disk.dm-1.out‚ 161.691406251]vPž°‚(:“ ì¾éÔÔĺ gpu.0.gpu‚11.0 gpu.0.memory‚1, gpu.0.memoryAllocated‚16.716384887695312* gpu.0.memoryAllocatedBytes‚ 14359265280 gpu.0.temp‚31.0 gpu.0.powerWatts‚109.885' gpu.0.enforcedPowerLimitWatts‚500.0) gpu.0.powerPercent‚21.977000000000004 gpu.0.smClock‚1275 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0ø¡¬€±‚(:õ ì¾éÔø¦€¼ disk.sda1.in‚ 0.1015625 disk.sda1.out‚0 disk.sda3.out‚ 162.27734375 disk.nvme0n1.in‚ 2270.609375 network.recv‚383518) disk./.usagePercent‚78.78725385797209$ disk./.usageGB‚1173.987018585205 disk.dm-0.in‚ 2270.609375 disk.dm-1.in‚ 0.6953125 disk.sda2.in‚0" disk.nvme0n1.out‚ 21271.5390625# proc.memory.rssMB‚ 2921.87109375 disk.drbd1260.out‚ 0.12890625 disk.sda.in‚0.796875 disk.sda3.in‚ 0.6953125$ disk.nvme0c0n1.out‚ 21271.5390625& proc.memory.availableMB‚ 1142.34375 disk.sda.out‚ 162.27734375 disk.sda2.out‚0! disk.nvme0c0n1.in‚ 2270.609375# memory_percent‚93.0276870727539* proc.memory.percent‚17.833685874938965 cpu‚25.60110861420936 proc.cpu.threads‚17 network.sent‚925075 disk.dm-0.out‚ 21271.5390625 disk.dm-1.out‚ 162.27734375 disk.drbd1260.in‚0Íi÷'ž²‚(:“ û¾éÔ¨®ý¸ gpu.0.gpu‚45.0 gpu.0.memory‚5, gpu.0.memoryAllocated‚16.716384887695312* gpu.0.memoryAllocatedBytes‚ 14359265280 gpu.0.temp‚34.0 gpu.0.powerWatts‚166.564' gpu.0.enforcedPowerLimitWatts‚500.0) gpu.0.powerPercent‚33.312799999999996 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0)V|j{³‚(:ð û¾éÔÛû£» proc.cpu.threads‚17 network.sent‚932762 network.recv‚387580 disk.sda3.in‚ 0.6953125 disk.sda3.out‚ 162.64453125% disk.nvme0c0n1.out‚21760.21484375 disk.dm-0.in‚2323.875 disk.dm-0.out‚21760.21484375 disk.dm-1.in‚ 0.6953125 disk.sda2.in‚0 disk.sda2.out‚0 disk.nvme0n1.in‚2323.875 disk.dm-1.out‚ 162.64453125 disk.sda.in‚0.796875 disk.sda.out‚ 162.64453125$ memory_percent‚93.02921295166016& proc.memory.availableMB‚ 1142.09375# disk.nvme0n1.out‚21760.21484375 disk.drbd1260.in‚0 disk.sda1.in‚ 0.1015625 disk.sda1.out‚0* proc.memory.percent‚17.833685874938965 cpu‚25.59765518775422) disk./.usagePercent‚78.79765544408828% disk./.usageGB‚1174.1420097351074 disk.nvme0c0n1.in‚2323.875 disk.drbd1260.out‚ 0.1328125# proc.memory.rssMB‚ 2921.87109375Áåê’´‚(:‡ Š¿éÔØïй gpu.0.gpu‚31.0 gpu.0.memory‚3, gpu.0.memoryAllocated‚16.716384887695312* gpu.0.memoryAllocatedBytes‚ 14359265280 gpu.0.temp‚34.0 gpu.0.powerWatts‚170.155' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚34.031 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0VyWðµ‚(:‚ Š¿éÔòÀ®¼) proc.memory.availableMB‚ 1142.08984375 cpu‚25.59543307726298) disk./.usagePercent‚78.80964146083387 disk.sda2.out‚0 disk.sda3.in‚ 0.6953125 disk.dm-0.in‚ 2420.37109375 disk.dm-0.out‚22092.16796875* proc.memory.percent‚17.833685874938965 proc.cpu.threads‚17 network.sent‚937214% disk./.usageGB‚1174.3206100463867 disk.sda2.in‚0 disk.sda3.out‚ 162.95703125 disk.dm-1.out‚ 162.95703125 disk.drbd1260.in‚0 disk.drbd1260.out‚ 0.1328125 disk.sda1.out‚0# disk.nvme0c0n1.in‚ 2420.37109375 disk.dm-1.in‚ 0.6953125 disk.sda.in‚0.796875 disk.sda.out‚ 162.95703125# proc.memory.rssMB‚ 2921.87109375 network.recv‚392635% disk.nvme0c0n1.out‚22095.41796875! disk.nvme0n1.in‚ 2420.37109375# disk.nvme0n1.out‚22095.41796875 disk.sda1.in‚ 0.1015625$ memory_percent‚93.02923679351807ÎÙMÔž¶‚(:“ ™¿éÔù넹 gpu.0.gpu‚19.0 gpu.0.memory‚2, gpu.0.memoryAllocated‚16.716384887695312* gpu.0.memoryAllocatedBytes‚ 14359265280 gpu.0.temp‚34.0 gpu.0.powerWatts‚133.457' gpu.0.enforcedPowerLimitWatts‚500.0) gpu.0.powerPercent‚26.691399999999998 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0íx}7·‚(:„ ™¿éÔ®ó¼ network.sent‚942125 network.recv‚396175% disk./.usageGB‚1174.3206100463867 disk.sda2.in‚0 disk.sda2.out‚0& disk.nvme0n1.out‚22312.74560546875 disk.sda1.in‚ 0.1015625$ memory_percent‚93.02773475646973 disk.sda3.out‚ 163.765625 disk.dm-0.in‚ 2420.37109375 disk.dm-1.out‚ 163.765625 disk.drbd1260.out‚0.140625 disk.sda.in‚0.796875# proc.memory.rssMB‚ 2921.87109375 cpu‚25.592579214618304 proc.cpu.threads‚17 disk.dm-1.in‚ 0.6953125 disk.sda.out‚ 163.765625( proc.memory.availableMB‚ 1142.3359375* proc.memory.percent‚17.833685874938965) disk./.usagePercent‚78.80964146083387 disk.sda3.in‚ 0.6953125# disk.nvme0c0n1.in‚ 2420.37109375( disk.nvme0c0n1.out‚22312.74560546875! disk.nvme0n1.in‚ 2420.37109375# disk.dm-0.out‚22312.74560546875 disk.drbd1260.in‚0 disk.sda1.out‚0ßæ|²‘¸‚(:† ¨¿éÔ¼¯Òº gpu.0.gpu‚29.0 gpu.0.memory‚2, gpu.0.memoryAllocated‚16.716384887695312* gpu.0.memoryAllocatedBytes‚ 14359265280 gpu.0.temp‚33.0 gpu.0.powerWatts‚142.34' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚28.468 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0pà•¹‚(:Š ¨¿éÔ˜ÓϾ$ memory_percent‚93.02773475646973) disk./.usagePercent‚78.80806650248323( disk.nvme0c0n1.out‚22594.19873046875! disk.nvme0n1.in‚ 2420.37109375# proc.memory.rssMB‚ 2921.87109375* proc.memory.percent‚17.833685874938965 disk.sda2.out‚0 disk.sda3.out‚ 164.40234375 disk.dm-0.in‚ 2420.37109375# disk.dm-0.out‚22594.19873046875 disk.drbd1260.out‚0.140625( proc.memory.availableMB‚ 1142.3359375 cpu‚25.590458452574786 proc.cpu.threads‚17 network.recv‚399999% disk./.usageGB‚1174.2971420288086# disk.nvme0c0n1.in‚ 2420.37109375& disk.nvme0n1.out‚22594.19873046875 disk.dm-1.out‚ 164.40234375 disk.drbd1260.in‚0 disk.sda1.in‚ 0.1015625 disk.sda1.out‚0 network.sent‚947308 disk.sda2.in‚0 disk.sda3.in‚ 0.6953125 disk.dm-1.in‚ 0.6953125 disk.sda.in‚0.796875 disk.sda.out‚ 164.40234375º_2º- _wandbruntime‚1009  _runtime‚1009h1Õ»¼ _step‚44 % lr-AdamW/pg1‚0.001293680144252562 % lr-AdamW/pg2‚0.001293680144252562 trainer/global_step‚599 % _timestamp‚1.7885019335691626e+09 _runtime‚ 1009.64879234ļ·Æ¼À % _timestamp‚1.7885019335691626e+09 _runtime‚ 1009.64879234 _step‚44 % lr-AdamW/pg1‚0.001293680144252562 % lr-AdamW/pg2‚0.001293680144252562 trainer/global_step‚599,Ažt2½- _wandbruntime‚1009  _runtime‚1009Üù_ôÆ'¾ÀO ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 PCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚11984925 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 586244608 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚30392 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚590 TDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 13805551616 UGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 582234453 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚17039360 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 YIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 13207863296 VFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 13207863296 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚24796 H?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚1321 ]JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚48392598710784 L?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚11984923 dQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚47790305300480 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 315220480 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 QCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 243625384 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 G?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚285 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 152061440 _LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚47134695169536 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 IDeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚3689 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 230583296 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 230583296 PCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚11984923 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚24794 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 XEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚46704732318608 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 597688320 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 ]LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 572067417504 RBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 13805551616 VEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 575101760512 IDeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 P@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 13118517709 ZIDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 575332343808 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚30637 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 @8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚894 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚2 \IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚47134695169536 SGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚6903202 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚588 % _timestamp‚1.7885019335724423e+09 XEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚47710027513344 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚17039360 bQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 602293410304 O?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 13805551616 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚24794 M@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚10535516 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚6903204 SGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚3632314 JBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚609 TADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚47709779890688 YHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 575101760512 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚609 trainer/global_step‚599 MFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚33 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚24903680 WGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 12589930496 _runtime‚1009.652376322 @8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚894 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 247622656 [HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚47134678130176 WGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 13035530240 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 310219776 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 586244608 YFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚48392288491008 BDeviceStatsMonitor.on_train_batch_start/allocation.all.current‚24796 TGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚14806991 G?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚609 N@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 247622656 Q>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚47709779890688 P@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 13567453184 E8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚26767120 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚30392 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 TGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚11984925 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚3632902 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 [HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚46132647861744 [KDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 13207863296 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 YHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 571840831480 TFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 597688320 C;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚894 JBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚285 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚2 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 597688320íµº1Ø¿ÄO F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 YIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 13207863296 VFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 13207863296 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚24796 H?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚1321 ]JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚48392598710784 L?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚11984923 dQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚47790305300480 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 315220480 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 QCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 243625384 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 G?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚285 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 152061440 _LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚47134695169536 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 IDeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚3689 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 230583296 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 230583296 PCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚11984923 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚24794 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 XEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚46704732318608 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 597688320 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 ]LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 572067417504 RBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 13805551616 VEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 575101760512 IDeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 P@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 13118517709 ZIDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 575332343808 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚30637 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 @8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚894 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚2 \IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚47134695169536 SGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚6903202 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚588 % _timestamp‚1.7885019335724423e+09 XEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚47710027513344 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚17039360 bQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 602293410304 O?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 13805551616 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚24794 M@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚10535516 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚6903204 SGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚3632314 JBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚609 TADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚47709779890688 YHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 575101760512 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚609 trainer/global_step‚599 MFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚33 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚24903680 WGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 12589930496 _runtime‚1009.652376322 @8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚894 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 247622656 [HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚47134678130176 WGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 13035530240 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 310219776 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 586244608 YFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚48392288491008 BDeviceStatsMonitor.on_train_batch_start/allo¿¾*òcation.all.current‚24796 TGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚14806991 G?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚609 N@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 247622656 Q>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚47709779890688 P@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 13567453184 E8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚26767120 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚30392 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 TGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚11984925 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚3632902 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 [HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚46132647861744 [KDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 13207863296 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 YHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 571840831480 TFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 597688320 C;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚894 JBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚285 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚2 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 597688320 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 PCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚11984925 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 586244608 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚30392 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚590 TDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 13805551616 UGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 582234453 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚17039360-mŸ$ 2À- _wandbruntime‚1011  _runtime‚1011«³cÐ&ÁÊM UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚17039360 REDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚12004882 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚17039360 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚894 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚30637 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚3639082 UEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 12589930496 WIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 597688320 VCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚47789496206848 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚6914697 MDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚1299 N>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 13118517709 NADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚12004882 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 597688320 R?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚47789248570880 XGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 576289828352 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚3689 C6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚26811662 VCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚46782531100754 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚609 N>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 13567453184 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 243625592 K>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚26836484 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚2 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚33 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 RBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 13805551616 _runtime‚1011.937295771 J=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚14806782 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚2 SEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 586244608 F6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚894 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚24820 BDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 WGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 13207863296 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 ]JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚46209511635648 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚609 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 230596608 `ODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 603294864384 trainer/global_step‚599 EDeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚10553198 WFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 572792878874 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 582234453 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚894 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚609 UEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 13035530240 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚24820 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚581 [HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚48473195024896 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 % _timestamp‚1.7885019358552637e+09 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚24822 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17039360 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 316497920 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 M=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 13805551616 S@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚47789496206848 FDeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 247635968 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚12004880 NADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚12004880 REDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚14831602 VHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 226586232 P@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 13805551616 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚285 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 ^KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚47869875256832 _step‚46ÕµLõÔ&ÂÎM S@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚47789496206848 FDeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 247635968 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚12004880 NADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚12004880 REDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚14831602 VHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 226586232 P@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 13805551616 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚285 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 ^KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚47869875256832 _step‚46 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚17039360 REDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚12004882 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚17039360 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚894 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚30637 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚3639082 UEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 12589930496 WIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 597688320 VCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚47789496206848 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚6914697 MDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚1299 N>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 13118517709 NADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚12004882 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 597688320 R?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚47789248570880 XGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 576289828352 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚3689 C6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚26811662 VCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚46782531100754 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚609 N>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 13567453184 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 243625592 K>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚26836484 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚2 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚33 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 RBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 13805551616 _runtime‚1011.937295771 J=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚14806782 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚2 SEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 586244608 F6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚894 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚24820 BDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 WGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 13207863296 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 ]JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚46209511635648 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚609 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 230596608 `ODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 603294864384 trainer/global_step‚599 EDeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚10553198 WFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 572792878874 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 582234453 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚894 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚609 UEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 13035530240 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚24820 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚581 [HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚48473195024896 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 % _timestamp‚1.7885019358552637e+09 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚24822 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17039360 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 316497920 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 M=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 13805551616.è½F2Ã- _wandbruntime‚1011  _runtime‚1011S;‹[VÄÐ 0loss/FRt4/assignment_loss‚3.1437766734233574 2loss/SRqqt2/assignment_loss‚2.5376731244000523 1loss/FBt1/assignment_loss‚0.47632301577767877 1loss/SRqqt1/detection_loss‚2.3891866687518446 2loss/SRqqt4/assignment_loss‚1.8210129209610957 /loss/FRt3/detection_loss‚1.9907927325602328 1loss/SRqqt3/detection_loss‚3.2306069493293763 /loss/FBt1/detection_loss‚2.8177992900212607 %loss/total_loss‚57.32019398644044 0loss/FRt1/assignment_loss‚4.7406585995356245 /loss/FBt2/assignment_loss‚0.304132486313723 0loss/FBt4/assignment_loss‚0.4290979040176937 /loss/SRqqt4/detection_loss‚1.01135612277618 .loss/FBt4/detection_loss‚3.008175674606772 % _timestamp‚1.7885019358622992e+09 /loss/FRt4/detection_loss‚1.9291775182258983 0loss/SRqqt2/detection_loss‚2.808634877204895 _step‚47 .loss/FRt2/detection_loss‚3.031730236678288 1loss/SRqqt1/assignment_loss‚2.732331024510647 2loss/SRqqt3/assignment_loss‚3.1907000064849855 /loss/FRt2/assignment_loss‚5.856630715830573 0loss/FRt3/assignment_loss‚2.4231137305163264 /loss/FBt3/detection_loss‚0.7150862395244142 trainer/global_step‚599 _runtime‚1011.943494368 1loss/FBt3/assignment_loss‚0.25644022178337833 /loss/FRt1/detection_loss‚2.3676633818944297 .loss/FBt2/detection_loss‚4.108093871311708 epoch‚0­&¡ZÅÔ 0loss/FRt1/assignment_loss‚4.7406585995356245 /loss/FBt2/assignment_loss‚0.304132486313723 0loss/FBt4/assignment_loss‚0.4290979040176937 /loss/SRqqt4/detection_loss‚1.01135612277618 .loss/FBt4/detection_loss‚3.008175674606772 % _timestamp‚1.7885019358622992e+09 /loss/FRt4/detection_loss‚1.9291775182258983 0loss/SRqqt2/detection_loss‚2.808634877204895 _step‚47 .loss/FRt2/detection_loss‚3.031730236678288 1loss/SRqqt1/assignment_loss‚2.732331024510647 2loss/SRqqt3/assignment_loss‚3.1907000064849855 /loss/FRt2/assignment_loss‚5.856630715830573 0loss/FRt3/assignment_loss‚2.4231137305163264 /loss/FBt3/detection_loss‚0.7150862395244142 trainer/global_step‚599 _runtime‚1011.943494368 1loss/FBt3/assignment_loss‚0.25644022178337833 /loss/FRt1/detection_loss‚2.3676633818944297 .loss/FBt2/detection_loss‚4.108093871311708 epoch‚0 0loss/FRt4/assignment_loss‚3.1437766734233574 2loss/SRqqt2/assignment_loss‚2.5376731244000523 1loss/FBt1/assignment_loss‚0.47632301577767877 1loss/SRqqt1/detection_loss‚2.3891866687518446 2loss/SRqqt4/assignment_loss‚1.8210129209610957 /loss/FRt3/detection_loss‚1.9907927325602328 1loss/SRqqt3/detection_loss‚3.2306069493293763 /loss/FBt1/detection_loss‚2.8177992900212607 %loss/total_loss‚57.32019398644044/6zž’Æ‚(:‡ ·¿éÔùÔçº gpu.0.gpu‚8.0 gpu.0.memory‚0, gpu.0.memoryAllocated‚16.716384887695312* gpu.0.memoryAllocatedBytes‚ 14359265280 gpu.0.temp‚33.0 gpu.0.powerWatts‚130.302' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚26.0604 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0z–ã,ŽÇ‚(:ƒ ·¿éÔ¼Žä¼( disk.nvme0c0n1.out‚22931.77490234375! disk.nvme0n1.in‚ 2420.37109375 disk.dm-1.out‚ 164.859375 disk.drbd1260.out‚0.140625 disk.sda.in‚0.796875( proc.memory.availableMB‚ 1142.2734375 network.recv‚404324) disk./.usagePercent‚78.80859771811679 disk.sda2.out‚0 disk.drbd1260.in‚0* proc.memory.percent‚17.833685874938965# disk.nvme0c0n1.in‚ 2420.37109375 disk.sda1.in‚ 0.1015625 disk.sda1.out‚0 disk.sda.out‚ 164.859375$ memory_percent‚93.02811622619629 cpu‚25.58787513959462 proc.cpu.threads‚17 network.sent‚953297 disk.sda2.in‚0 disk.sda3.in‚ 0.6953125& disk.nvme0n1.out‚22931.77490234375 disk.dm-0.in‚ 2420.37109375# disk.dm-0.out‚22931.77490234375 disk.dm-1.in‚ 0.6953125# proc.memory.rssMB‚ 2921.87109375% disk./.usageGB‚1174.3050575256348 disk.sda3.out‚ 164.859375`_Eý‘È‚(:† Æ¿éÔªÏÖ¸ gpu.0.gpu‚0.0 gpu.0.memory‚0, gpu.0.memoryAllocated‚16.716384887695312* gpu.0.memoryAllocatedBytes‚ 14359265280 gpu.0.temp‚32.0 gpu.0.powerWatts‚114.355' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚22.871 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0Û§ð•É‚(:Š Æ¿éÔÉ÷›» disk.sda1.in‚ 0.1015625 disk.sda1.out‚0' proc.memory.availableMB‚ 980.2734375# proc.memory.rssMB‚ 3083.61328125 network.recv‚410936) disk./.usagePercent‚78.81017267646743$ disk./.usageGB‚1174.328525543213( disk.nvme0c0n1.out‚23164.01318359375# disk.dm-0.out‚23164.01318359375 disk.drbd1260.in‚0 disk.sda.out‚ 165.20703125* proc.memory.percent‚18.820881843566895 network.sent‚ 1.005228e+06 disk.sda3.out‚ 165.20703125 disk.dm-1.in‚ 0.6953125 proc.cpu.threads‚17 disk.sda2.out‚0 disk.sda3.in‚ 0.6953125 disk.nvme0n1.in‚ 2581.609375 disk.dm-0.in‚ 2581.609375 disk.dm-1.out‚ 165.20703125 disk.sda.in‚0.796875$ memory_percent‚94.01688575744629 cpu‚25.661113244950464 disk.sda2.in‚0! disk.nvme0c0n1.in‚ 2581.609375& disk.nvme0n1.out‚23164.01318359375 disk.drbd1260.out‚ 0.14453125Pšl²“Ê‚(:ˆ Õ¿éÔö´¶º gpu.0.gpu‚72.0 gpu.0.memory‚4, gpu.0.memoryAllocated‚16.716384887695312* gpu.0.memoryAllocatedBytes‚ 14359265280 gpu.0.temp‚34.0 gpu.0.powerWatts‚174.387' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚34.8774 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0 /ÜŽË‚(:ƒ Õ¿éÔ‹üû» network.sent‚ 1.010387e+06% disk./.usageGB‚1174.3284225463867 disk.sda3.out‚ 165.55078125 disk.drbd1260.in‚0 disk.sda1.in‚ 0.1015625 disk.sda1.out‚0 disk.sda.in‚0.796875 network.recv‚414511 disk.sda2.in‚0 disk.nvme0n1.in‚ 2581.609375$ disk.nvme0n1.out‚23565.904296875 disk.dm-0.in‚ 2581.609375! disk.dm-0.out‚23565.904296875 disk.dm-1.out‚ 165.55078125& proc.memory.availableMB‚ 1049.90625 disk.sda3.in‚ 0.6953125! disk.nvme0c0n1.in‚ 2581.609375& disk.nvme0c0n1.out‚23565.904296875 disk.drbd1260.out‚ 0.14453125 disk.sda.out‚ 165.55078125* proc.memory.percent‚18.386220932006836 cpu‚25.722212258084127 proc.cpu.threads‚17) disk./.usagePercent‚78.81016576426401 disk.sda2.out‚0 disk.dm-1.in‚ 0.6953125$ memory_percent‚93.59188079833984" proc.memory.rssMB‚ 3012.3984375SX°“Ì‚(:ˆ ä¿éÔÕ¥é¸ gpu.0.gpu‚32.0 gpu.0.memory‚1, gpu.0.memoryAllocated‚16.716384887695312* gpu.0.memoryAllocatedBytes‚ 14359265280 gpu.0.temp‚33.0 gpu.0.powerWatts‚113.801' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚22.7602 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0TÄÍ‚(:‚ ä¿éÔàí®½ disk.sda3.in‚ 0.6953125& disk.nvme0c0n1.out‚23929.423828125 disk.dm-1.in‚ 0.6953125 network.recv‚418383 disk.sda3.out‚ 165.9921875 disk.drbd1260.in‚0 disk.drbd1260.out‚ 0.1484375 disk.sda1.out‚0 disk.sda.out‚ 165.9921875$ memory_percent‚94.02086734771729( proc.memory.availableMB‚ 979.62109375 disk.dm-0.in‚ 2581.609375 disk.sda2.in‚0! disk.nvme0c0n1.in‚ 2581.609375 disk.nvme0n1.in‚ 2581.609375 disk.sda.in‚0.796875# proc.memory.rssMB‚ 3083.86328125 proc.cpu.threads‚17 disk.sda2.out‚0 disk.dm-1.out‚ 165.9921875 disk.sda1.in‚ 0.1015625* proc.memory.percent‚18.822407722473145 cpu‚25.816019596145242$ disk.nvme0n1.out‚23929.423828125! disk.dm-0.out‚23929.423828125 network.sent‚ 1.013573e+06) disk./.usagePercent‚78.74669893645049% disk./.usageGB‚1173.3827209472656Òz,NÎbI &Purity/3FRt0SRqqt1FBt/event_proportion"trainer/global_step2:JPïŽfJÏbE "Purity/3FRt1SRqqt1FBt/SRqqt_purity"trainer/global_step2:JP9ñÎJÐbE "Purity/*FRt0SRqqt*FBt/event_purity"trainer/global_step2:JPšÈÜcHÑbC Purity/*FRt1SRqqt0FBt/FRt_purity"trainer/global_step2:JPLø©»NÒbI &Purity/1FRt1SRqqt4FBt/event_proportion"trainer/global_step2:JP„n;úNÓbI &Purity/1FRt2SRqqt3FBt/event_proportion"trainer/global_step2:JP–ŸJÔbE "Purity/2FRt2SRqqt*FBt/SRqqt_purity"trainer/global_step2:JPÓ;ä³JÕbE "Purity/3FRt*SRqqt0FBt/event_purity"trainer/global_step2:JPuôÑ·NÖbI &Purity/3FRt*SRqqt1FBt/event_proportion"trainer/global_step2:JP0êäïJ×bE "Purity/*FRt4SRqqt1FBt/SRqqt_purity"trainer/global_step2:JP4<;JØbE "Purity/*FRt3SRqqt*FBt/event_purity"trainer/global_step2:JP®ÛNÙbI &Purity/0FRt0SRqqt2FBt/event_proportion"trainer/global_step2:JPºqîENÚbI &Purity/3FRt2SRqqt0FBt/event_proportion"trainer/global_step2:JPl¸(HÛbC Purity/*FRt2SRqqt2FBt/FRt_purity"trainer/global_step2:JPÖ¤§dJÜbE "Purity/0FRt3SRqqt1FBt/event_purity"trainer/global_step2:JPÿ´ÃSJÝbE "Purity/1FRt*SRqqt4FBt/event_purity"trainer/global_step2:JPZ¢O HÞbC Purity/1FRt0SRqqt4FBt/FRt_purity"trainer/global_step2:JPc$|ÇJßbE "Purity/1FRt4SRqqt0FBt/SRqqt_purity"trainer/global_step2:JPt"HàbC Purity/*FRt3SRqqt*FBt/FBt_purity"trainer/global_step2:JPhyÆHábC Purity/*FRt3SRqqt1FBt/FBt_purity"trainer/global_step2:JP÷A£–NâbI &Purity/0FRt2SRqqt3FBt/event_proportion"trainer/global_step2:JPö|<¿HãbC Purity/3FRt1SRqqt1FBt/FRt_purity"trainer/global_step2:JPð†¯#NäbI &Purity/3FRt2SRqqt*FBt/event_proportion"trainer/global_step2:JP ô:1NåbI &Purity/3FRt4SRqqt*FBt/event_proportion"trainer/global_step2:JPìòi`NæbI &Purity/4FRt2SRqqt*FBt/event_proportion"trainer/global_step2:JP2!ÄJçbE "Purity/0FRt2SRqqt2FBt/SRqqt_purity"trainer/global_step2:JP‘‹Å6@èb; particle/accuracy_1_of_7"trainer/global_step2:JPø0 JébE "Purity/3FRt*SRqqt2FBt/SRqqt_purity"trainer/global_step2:JPÓ© JêbE "Purity/0FRt2SRqqt3FBt/SRqqt_purity"trainer/global_step2:JP•Cô  ëbC Purity/*FRt0SRqqt0FBt/FRtíÐøÇ(_purity"trainer/global_step2:JPww¿NìbI &Purity/*FRt1SRqqt4FBt/event_proportion"trainer/global_step2:JPáŠyüNíbI &Purity/0FRt*SRqqt3FBt/event_proportion"trainer/global_step2:JP¡íÉ HîbC Purity/1FRt2SRqqt2FBt/FRt_purity"trainer/global_step2:JPÉdTÝJïbE "Purity/1FRt2SRqqt2FBt/SRqqt_purity"trainer/global_step2:JPozT\JðbE "Purity/4FRt2SRqqt0FBt/SRqqt_purity"trainer/global_step2:JP™‡L_NñbI &Purity/0FRt*SRqqt1FBt/event_proportion"trainer/global_step2:JP¹t¥ØJòbE "Purity/2FRt3SRqqt0FBt/event_purity"trainer/global_step2:JP/#SgNóbI &Purity/3FRt4SRqqt2FBt/event_proportion"trainer/global_step2:JP”üú†NôbI &Purity/4FRt2SRqqt3FBt/event_proportion"trainer/global_step2:JPpËï @õb; particle/accuracy_6_of_6"trainer/global_step2:JPîã"­HöbC Purity/*FRt*SRqqt4FBt/FRt_purity"trainer/global_step2:JP%Ë,8N÷bI &Purity/1FRt*SRqqt0FBt/event_proportion"trainer/global_step2:JP¹p—DNøbI &Purity/3FRt4SRqqt0FBt/event_proportion"trainer/global_step2:JPæù*HùbC Purity/3FRt*SRqqt2FBt/FRt_purity"trainer/global_step2:JP¾-\JúbE "Purity/1FRt3SRqqt2FBt/event_purity"trainer/global_step2:JPÉEfJûbE "Purity/1FRt0SRqqt2FBt/event_purity"trainer/global_step2:JPésá—JübE "Purity/1FRt1SRqqt0FBt/SRqqt_purity"trainer/global_step2:JPˆü¹NýbI &Purity/2FRt4SRqqt3FBt/event_proportion"trainer/global_step2:JPŒ|H—JþbE "Purity/1FRt0SRqqt3FBt/event_purity"trainer/global_step2:JPœâZ,JÿbE "Purity/*FRt1SRqqt*FBt/SRqqt_purity"trainer/global_step2:JPÆÙßÿN€bI &Purity/*FRt2SRqqt0FBt/event_proportion"trainer/global_step2:JPrÅjJbE "Purity/1FRt*SRqqt*FBt/event_purity"trainer/global_step2:JPÐÖƒTJ‚bE "Purity/2FRt*SRqqt0FBt/SRqqt_purity"trainer/global_step2:JPØ¡‡JƒbE "Purity/3FRt2SRqqt0FBt/SRqqt_purity"trainer/global_step2:JP®LnšJ„bE "Purity/0FRt4SRqqt0FBt/event_purity"trainer/global_step2:JPæ“H…bC Purity/1FRt4SRqqt*FBt/FRt_purity"trainer/global_step2:JPH­™N†bI &Purity/1FRt*SRqqt3FBt/event_proportion"trainer/global_step2:JPãUŸN‡bI &Purity/3FRt*SRqqt0FBt/event_proportion"trainer/global_step2:JPn ðJˆbE "Purity/2FRt2SRqqt2FBt/event_purity"trainer/global_step2:JPOÚõ|N‰bI &Purity/*FRt*SRqqt0FBt/event_proportion"trainer/global_step2:JP9¥àµJŠbE "Purity/1FRt1SRqqt0FBt/event_purity"trainer/global_step2:JP•‰LòN‹bI &Purity/4FRt1SRqqt1FBt/event_proportion"trainer/global_step2:JPX7UíHŒbC Purity/*FRt0SRqqt4FBt/FRt_purity"trainer/global_step2:JP ™ç‚JbE "Purity/0FRt1SRqqt3FBt/event_purity"trainer/global_step2:JP»ÝïÆJŽbE "Purity/*FRt*SRqqt2FBt/SRqqt_purity"trainer/global_step2:JPÓ†~öJbE "Purity/*FRt2SRqqt1FBt/event_purity"trainer/global_step2:JPŒÌÐJbE "Purity/*FRt3SRqqt*FBt/SRqqt_purity"trainer/global_step2:JPrL‰¸N‘bI &Purity/4FRt1SRqqt3FBt/event_proportion"trainer/global_step2:JP‚»óJ’bE "Purity/*FRt3SRqqt2FBt/SRqqt_purity"trainer/global_step2:JPÀ+ÍH“bC Purity/*FRt*SRqqt0FBt/FRt_purity"trainer/global_step2:JPH€H”bC Purity/1FRt2SRqqt*FBt/FRt_purity"trainer/global_step2:JPëSYWN•bI &Purity/2FRt0SRqqt3FBt/event_proportion"trainer/global_step2:JP(‚ÖJ–bE "Purity/2FRt2SRqqt*FBt/event_purity"trainer/global_step2:JPj•£N—bI &Purity/4FRt4SRqqt2FBt/event_proportion"trainer/global_step2:JPI<ÎJ˜bE "Purity/*FRt*SRqqt1FBt/event_purity"trainer/global_step2:JPîëîH™bC Purity/*FRt3SRqqt1FBt/FRt_purity"trainer/global_step2:JP°Uî…JšbE "Purity/2FRt*SRqqt0FBt/event_purity"trainer/global_step2:JPP#ñJ›bE "Purity/2FRt1SRqqt2FBt/SRqqt_purity"trainer/global_step2:JPßy­áHœbC Purity/3FRt1SRqqt*FBt/FBt_purity"trainer/global_step2:JP°3D&HbC Purity/3FRt1SRqqt1FBt/FBt_purity"trainer/global_step2:JP fÈJžbE "Purity/3FRt2SRqqt0FBt/event_purity"trainer/global_step2:JP-&ˆJŸbE "Purity/0FRt4SRqqt0FBt/SRqqt_purity"trainer/global_step2:JP »:kJ bE "Purity/1FRt3SRqqt1FBt/SRqqt_purity"trainer/global_step2:JPÑkN¡bI &Purity/1FRt4SRqqt1FBt/event_proportion"trainer/global_step2:JPG ´¸J¢bE "Purity/2FRt2SRqqt1FBt/event_purity"trainer/global_step2:JP·¦N£bI &Purity/3FRt0SRqqt4FBt/event_proportion"trainer/global_step2:JPûßC¦N¤bI &Purity/4FRt3SRqqt0FBt/event_proportion"trainer/global_step2:JPº‚$@¥b; particle/accuracy_5_of_5"trainer/global_step2:JPî*îX;¦b6 jet/accuracy_3_of_7"trainer/global_step2:JPõH§bC Purity/3FRt*SRqqt2FBt/FBt_purity"trainer/global_step2:JPØØ£J¨bE "Purity/1FRt*SRqqt2FBt/SRqqt_purity"trainer/global_step2:JP¤bGN©bI &Purity/3FRt*SRqqt3FBt/event_proportion"trainer/global_step2:JP¨pAŒJªbE "Purity/2FRt3SRqqt0FBt/SRqqt_purity"trainer/global_step2:JPÄY˜ÎH«bC Purity/1FRt2SRqqt2FBt/FBt_purity"trainer/global_step2:JP~9‹N¬bI &Purity/0FRt1SRqqt4FBt/event_proportion"trainer/global_step2:JP!$ÔûH­bC Purity/2FRt2SRqqt*FBt/FBt_purity"trainer/global_step2:JPßÕ?SH®bC Purity/2FRt2SRqqt1FBt/FBt_purity"trainer/global_step2:JP/PÚH¯bC Purity/3FRt*SRqqt*FBt/FRt_purity"trainer/global_step2:JP “Á;°b6 jet/accuracy_6_of_6"trainer/global_step2:JP÷§0ÙJ±bE "Purity/3FRt*SRqqt0FBt/SRqqt_purity"trainer/global_step2:JP3³Er@²b; particle/accuracy_3_of_7"trainer/global_step2:JPY·H³bC Purity/4FRt1SRqqt0FBt/FRt_purity"trainer/global_step2:JPwð§ÇJ´bE "Purity/*FRt1SRqqt*FBt/event_purity"trainer/global_step2:JPâ¤ùJµbE "Purity/1FRt*SRqqt*FBt/SRqqt_purity"trainer/global_step2:JPœ®fŽN¶bI &Purity/2FRt1SRqqt3FBt/event_proportion"trainer/global_step2:JP£a©N·bI &Purity/3FRt*SRqqt*FBt/event_proportion"trainer/global_step2:JPfµ¥”<¸b7 particle/specificity"trainer/global_step2:JP“ý{-;¹b6 jet/accuracy_4_of_6"trainer/global_step2:JPŠ4e@JºbE "Purity/4FRt2SRqqt0FBt/event_purity"trainer/global_step2:JP“—,_H»bC Purity/1FRt*SRqqt*FBt/FRt_purity"trainer/global_step2:JP)…;§N¼bI &Purity/2FRt0SRqqt0FBt/event_proportion"trainer/global_step2:JP%ÏÜ"H½bC Purity/*FRt2SRqqt2FBt/FBt_purity"trainer/global_step2:JP¶fʼnN¾bI &Purity/0FRt*SRqqt4FBt/event_proportion"trainer/global_step2:JPui”àN¿bI &Purity/2FRt1SRqqt*FBt/event_proportion"trainer/global_step2:JPÕHß4;Àb6 jet/accuracy_7_of_7"trainer/global_step2:JP·\‘—HÁbC Purity/1FRt*SRqqt4FBt/FBt_purity"trainer/global_step2:JP#Ô&JÂbE "Purity/1FRt4SRqqt0FBt/event_purity"trainer/global_step2:JPB¢<þHÃbC Purity/1FRt1SRqqt4FBt/FRt_purity"trainer/global_step2:JPýuylNÄbI &Purity/2FRt4SRqqt2FBt/event_proportion"trainer/global_step2:JPÉq´JÅbE "Purity/3FRt0SRqqt0FBt/event_purity"trainer/global_step2:JPê'NÆbI &Purity/3FRt0SRqqt3FBt/event_proportion"trainer/global_step2:JPµõ;Çb6 jet/accuracy_1_of_3"trainer/global_step2:JP³R(|;Èb6 jet/accuracy_3_of_3"trainer/global_step2:JPX¥±JÉbE "Purity/3FRt2SRqqt1FBt/event_purity"trainer/global_step2:JP÷ÊÐHÊbC Purity/*FRt1SRqqt4FBt/FRt_purity"trainer/global_step2:JPÄÛÕ$JËbE "Purity/4FRt0SRqqt*FBt/event_purity"trainer/global_step2:JPfsÈÃNÌbI &Purity/*FRt*SRqqt1FBt/event_proportion"trainer/global_step2:JP/ b¢NÍbI &Purity/*FRt0SRqqt*FBt/event_proportion"trainer/global_step2:JPy#&cNÎbI &Purity/*FRt2SRqqt3FBt/event_proportion"trainer/global_step2:JPà|ó¢JÏbE "Purity/1FRt2SRqqt1FBt/event_purity"trainer/global_step2:JPþ˜ç¹HÐbC Purity/2FRt*SRqqt0FBt/FRt_purity"trainer/global_step2:JPMËÌ5NÑbI &Purity/3FRt3SRqqt*FBt/event_proportion"trainer/global_step2:JP¤ªîNÒbI &Purity/4FRt*SRqqt3FBt/event_proportion"trainer/global_step2:JPãi}+@Ób; particle/accuracy_4_of_7"trainer/global_step2:JPºÛ HÔbC Purity/*FRt2SRqqt*FBt/FRt_purity"trainer/global_step2:JP‰^¸JNÕbI &Purity/0FRt0SRqqt*FBt/event_proportion"trainer/global_step2:JP.ôü¿HÖbC Purity/1FRt*SRqqt2FBt/FBt_purity"trainer/global_step2:JP‚¦a¦;×b6 jet/accuracy_1_of_2"trainer/global_step2:JPÄ+WìHØbC Purity/2FRt4SRqqt*FBt/FRt_purity"trainer/global_step2:JPº;ã•JÙbE "Purity/*FRt*SRqqt4FBt/SRqqt_purity"trainer/global_step2:JP97UéJÚbE "Purity/3FRt3SRqqt0FBt/SRqqt_purity"trainer/global_step2:JP·±EsNÛbI &Purity/*FRt3SRqqt0FBt/event_proportion"trainer/global_step2:JPNm“nJÜbE "Purity/0FRt*SRqqt2FBt/event_purity"trainer/global_step2:JP8LÕHÝbC Purity/1FRt0SRqqt0FBt/FRt_purity"trainer/global_step2:JPý&ÌNÞbI &Purity/2FRt*SRqqt0FBt/event_proportion"trainer/global_step2:JPr£”ËHßbC Purity/*FRt4SRqqt*FBt/FRt_purity"trainer/global_step2:JP»ƒc÷NàbI &Purity/*FRt0SRqqt3FBt/event_proportion"trainer/global_step2:JPûPËNábI &Purity/0FRt0SRqqt4FBt/event_proportion"trainer/global_step2:JPS…í©HâbC Purity/1FRt1SRqqt1FBt/FRt_purity"trainer/global_step2:JP%É´JãbE "Purity/0FRt3SRqqt*FBt/event_purity"trainer/global_step2:JPÎÛA¡;äb6 jet/accuracy_2_of_6"trainer/global_step2:JPCàHÙ;åb6 jet/accuracy_5_of_6"trainer/global_step2:JPš5siHæbC Purity/1FRt3SRqqt*FBt/FRt_purity"trainer/global_step2:JP#|·-NçbI &Purity/1FRt3SRqqt3FBt/event_proportion"trainer/global_step2:JPz‡HðJèbE "Purity/2FRt2SRqqt0FBt/event_purity"trainer/global_step2:JPœ»›HébC Purity/2FRt2SRqqt0FBt/FRt_purity"trainer/global_step2:JPÆÛ½ZHêbC Purity/3FRt0SRqqt1FBt/FRt_purity"trainer/global_step2:JPaæ)ËNëbI &Purity/4FRt*SRqqt2FBt/event_proportion"trainer/global_step2:JPÓ11ÜNìbI &Purity/4FRt4SRqqt*FBt/event_proportion"trainer/global_step2:JP‡Ìèˆ;íb6 jet/accuracy_4_of_7"trainer/global_step2:JPgõp:JîbE "Purity/*FRt0SRqqt3FBt/event_purity"trainer/global_step2:JPEÉŸ NïbI &Purity/0FRt3SRqqt*FBt/event_proportion"trainer/global_step2:JP4\8!NðbI &Purity/0FRt3SRqqt1FBt/event_proportion"trainer/global_step2:JPð*5 JñbE "Purity/1FRt1SRqqt*FBt/event_purity"trainer/global_step2:JP?ê£gNòbI &Purity/4FRt1SRqqt0FBt/event_proportion"trainer/global_step2:JPŒ7ãNóbI &Purity/0FRt1SRqqt*FBt/event_proportion"trainer/global_step2:JPQJ¦JôbE "Purity/1FRt*SRqqt1FBt/SRqqt_purity"trainer/global_step2:JP,7Ó°NõbI &Purity/2FRt*SRqqt*FBt/event_proportion"trainer/global_step2:JPE(ëANöbI &Purity/2FRt1SRqqt2FBt/event_proportion"trainer/global_step2:JP¶ÅwÏN÷bI &Purity/2FRt3SRqqt3FBt/event_proportion"trainer/global_step2:JP…rECNøbI &Purity/3FRt0SRqqt*FBt/event_proportion"trainer/global_step2:JPGdÔ NùbI &Purity/3FRt4SRqqt3FBt/event_proportion"trainer/global_step2:JP>7ݬ;úb6 jet/accuracy_2_of_7"trainer/global_step2:JP8“RHûbC Purity/*FRt*SRqqt*FBt/FBt_purity"trainer/global_step2:JPÓ¾}$HübC Purity/*FRt*SRqqt1FBt/FBt_purity"trainer/global_step2:JPºaËPJýbE "Purity/*FRt*SRqqt3FBt/event_purity"trainer/global_step2:JPч2‰JþbE "Purity/1FRt1SRqqt1FBt/SRqqt_purity"trainer/global_step2:JPkû3=NÿbI &Purity/1FRt3SRqqt2FBt/event_proportion"trainer/global_step2:JP< À³J€bE "Purity/2FRt*SRqqt1FBt/event_purity"trainer/global_step2:JPNÌòJbE "Purity/2FRt1SRqqt1FBt/SRqqt_purity"trainer/global_step2:JPã³.ž@‚b; particle/accuracy_1_of_4"trainer/global_step2:JP¥¿ NƒbI &Purity/0FRt4SRqqt1FBt/event_proportion"trainer/global_step2:JP+N„bI &Purity/1FRt0SRqqt*FBt/event_proportion"trainer/global_step2:JP°ÆåJH…bC Purity/3FRt1SRqqt0FBt/FRt_purity"trainer/global_step2:JP@¢kN†bI &Purity/3FRt1SRqqt2FBt/event_proportion"trainer/global_step2:JP6«Ü”J‡bE "Purity/2FRt3SRqqt1FBt/SRqqt_purity"trainer/global_step2:JPÇá“[@ˆb; particle/accuracy_5_of_7"trainer/global_step2:JPk›lÂJ‰bE "Purity/3FRt1SRqqt2FBt/event_purity"trainer/global_step2:JPrÀ0¸HŠbC Purity/1FRt3SRqqt2FBt/FBt_purity"trainer/global_step2:JPµØ¦J‹bE "Purity/0FRt1SRqqt*FBt/event_purity"trainer/global_step2:JPÛ²ÛJŒbE "Purity/1FRt0SRqqt*FBt/event_purity"trainer/global_step2:JP3QX;b6 jet/accuracy_2_of_4"trainer/global_step2:JP ðwHŽbC Purity/0FRt3SRqqt*FBt/FBt_purity"trainer/global_step2:JPOS°HbC Purity/0FRt3SRqqt1FBt/FBt_purity"trainer/global_step2:JPÒÆ¥»HbC Purity/*FRt1SRqqt1FBt/FRt_purity"trainer/global_step2:JP~&ÐNJ‘bE "Purity/*FRt3SRqqt1FBt/SRqqt_purity"trainer/global_step2:JP׬—H’bC Purity/0FRt0SRqqt*FBt/FBt_purity"trainer/global_step2:JP¸æ~ÅH“bC Purity/0FRt0SRqqt1FBt/FBt_purity"trainer/global_step2:JP=9^J”bE "Purity/0FRt1SRqqt1FBt/SRqqt_purity"trainer/global_step2:JP0¬ÛJ•bE "Purity/1FRt3SRqqt*FBt/SRqqt_purity"trainer/global_step2:JPÅ×ÏJN–bI &Purity/1FRt3SRqqt1FBt/event_proportion"trainer/global_step2:JPуJ—bE "Purity/4FRt1SRqqt*FBt/SRqqt_purity"trainer/global_step2:JP_‡íWN˜bI &Purity/2FRt3SRqqt4FBt/event_proportion"trainer/global_step2:JPFH°gJ™bE "Purity/*FRt3SRqqt0FBt/event_purity"trainer/global_step2:JPy“V¸HšbC Purity/2FRt0SRqqt2FBt/FRt_purity"trainer/global_step2:JPïÉô>J›bE "Purity/4FRt1SRqqt0FBt/SRqqt_purity"trainer/global_step2:JPØC8JœbE "Purity/*FRt1SRqqt1FBt/SRqqt_purity"trainer/global_step2:JPƒž"JbE "Purity/0FRt2SRqqt1FBt/event_purity"trainer/global_step2:JP$œÕ±NžbI &Purity/0FRt2SRqqt4FBt/event_proportion"trainer/global_step2:JP‹â+HŸbC Purity/1FRt0SRqqt*FBt/FBt_purity"trainer/global_step2:JP úë²H bC Purity/1FRt0SRqqt1FBt/FBt_purity"trainer/global_step2:JPU¯áN¡bI &Purity/1FRt1SRqqt0FBt/event_proportion"trainer/global_step2:JPÄ8ÜN¢bI &Purity/2FRt1SRqqt0FBt/event_proportion"trainer/global_step2:JPïßJN£bI &Purity/2FRt4SRqqt0FBt/event_proportion"trainer/global_step2:JPHš©\J¤bE "Purity/1FRt*SRqqt0FBt/SRqqt_purity"trainer/global_step2:JPÈÙH¥bC Purity/1FRt*SRqqt3FBt/FBt_purity"trainer/global_step2:JP3åfîN¦bI &Purity/1FRt0SRqqt2FBt/event_proportion"trainer/global_step2:JP |šgJ§bE "Purity/3FRt1SRqqt0FBt/event_purity"trainer/global_step2:JP«£³MH¨bC Purity/3FRt2SRqqt*FBt/FRt_purity"trainer/global_step2:JPÖ/+H©bC Purity/*FRt0SRqqt1FBt/FRt_purity"trainer/global_step2:JPQ=NªbI &Purity/*FRt3SRqqt2FBt/event_proportion"trainer/global_step2:JPÿ°ÐN«bI &Purity/*FRt3SRqqt4FBt/event_proportion"trainer/global_step2:JP¤d0ÝH¬bC Purity/2FRt*SRqqt*FBt/FBt_purity"trainer/global_step2:JPË.ÙH­bC Purity/2FRt*SRqqt1FBt/FBt_purity"trainer/global_step2:JP^ÑDG@®b; particle/accuracy_2_of_7"trainer/global_step2:JPP¶úáJ¯bE "Purity/2FRt*SRqqt3FBt/event_purity"trainer/global_step2:JPdŠËH°bC Purity/2FRt0SRqqt3FBt/FRt_purity"trainer/global_step2:JP Å /J±bE "Purity/*FRt2SRqqt0FBt/SRqqt_purity"trainer/global_step2:JPŵºMJ²bE "Purity/0FRt*SRqqt1FBt/event_purity"trainer/global_step2:JP¡¥ I@³b; particle/accuracy_2_of_6"trainer/global_step2:JPêƒ@´b; particle/accuracy_5_of_6"trainer/global_step2:JPµK|,HµbC Purity/0FRt2SRqqt*FBt/FBt_purity"trainer/global_step2:JPKº—„H¶bC Purity/0FRt2SRqqt1FBt/FBt_purity"trainer/global_step2:JPUÝ*œH·bC Purity/2FRt1SRqqt2FBt/FRt_purity"trainer/global_step2:JPºTŒ;¸b6 jet/accuracy_1_of_5"trainer/global_step2:JPãê™ÞJ¹bE "Purity/0FRt*SRqqt*FBt/event_purity"trainer/global_step2:JPÞÅfÂNºbI &Purity/0FRt0SRqqt0FBt/event_proportion"trainer/global_step2:JP’#JãbE "Purity/3FRt*SRqqt2FBt/event_purity"trainer/global_step2:JP8 ¶†NäbI &Purity/2FRt*SRqqt3FBt/event_proportion"trainer/global_step2:JPõ£`>GåbB validation_average_jet_accuracy"trainer/global_step2:JPí¹¹S;æb6 jet/accuracy_3_of_6"trainer/global_step2:JPý'yšHçbC Purity/2FRt3SRqqt1FBt/FRt_purity"trainer/global_step2:JPQËzyJèbE "Purity/0FRt*SRqqt0FBt/event_purity"trainer/global_step2:JP‰ %ßJébE "Purity/0FRt0SRqqt1FBt/event_purity"trainer/global_step2:JPv)ãàNêbI &Purity/1FRt1SRqqt1FBt/event_proportion"trainer/global_step2:JP]þ/;JëbE "Purity/2FRt1SRqqt*FBt/event_purity"trainer/global_step2:JPˆ‘ÙNìbI &Purity/3FRt1SRqqt*FBt/event_proportion"trainer/global_step2:JP>ÓäHíbC Purity/3FRt2SRqqt0FBt/FRt_purity"trainer/global_step2:JP*XaHîbC Purity/*FRt4SRqqt*FBt/FBt_purity"trainer/global_step2:JPy`±¦HïbC Purity/*FRt4SRqqt1FBt/FBt_purity"trainer/global_step2:JPQÑeJðbE "Purity/*FRt0SRqqt2FBt/event_purity"trainer/global_step2:JPöõ7¦HñbC Purity/*FRt2SRqqt*FBt/FBt_purity"trainer/global_step2:JPÜHòbC Purity/*FRt2SRqqt1FBt/FBt_purity"trainer/global_step2:JPdiOJóbE "Purity/1FRt2SRqqt0FBt/SRqqt_purity"trainer/global_step2:JPJ$LNôbI &Purity/1FRt4SRqqt3FBt/event_proportion"trainer/global_step2:JPˆuQNõbI &Purity/4FRt4SRqqt0FBt/event_proportion"trainer/global_step2:JP}ÒÞÒJöbE "Purity/3FRt4SRqqt0FBt/SRqqt_purity"trainer/global_step2:JPwyÍuJ÷bE "Purity/0FRt3SRqqt1FBt/SRqqt_purity"trainer/global_step2:JPqšÎëJøbE "Purity/0FRt2SRqqt0FBt/event_purity"trainer/global_step2:JP'~‰:JùbE "Purity/3FRt1SRqqt1FBt/event_purity"trainer/global_step2:JP~°·ËHúbC Purity/0FRt2SRqqt3FBt/FBt_purity"trainer/global_step2:JPþÁœNûbI &Purity/0FRt1SRqqt3FBt/event_proportion"trainer/global_step2:JPGÌ5©NübI &Purity/2FRt4SRqqt*FBt/event_proportion"trainer/global_step2:JPŸkB JýbE "Purity/*FRt4SRqqt1FBt/event_purity"trainer/global_step2:JP)ãì‚JþbE "Purity/0FRt4SRqqt*FBt/SRqqt_purity"trainer/global_step2:JP´¿øXJÿbE "Purity/1FRt2SRqqt2FBt/event_purity"trainer/global_step2:JP»H­iH€bC Purity/*FRt3SRqqt2FBt/FRt_purity"trainer/global_step2:JP•ÖÒJbE "Purity/*FRt2SRqqt*FBt/SRqqt_purity"trainer/global_step2:JPý¯vON‚bI &Purity/4FRt1SRqqt2FBt/event_proportion"trainer/global_step2:JPÌæÞ‡JƒbE "Purity/2FRt4SRqqt0FBt/SRqqt_purity"trainer/global_step2:JP¸>KÐJ„bE "Purity/1FRt4SRqqt*FBt/event_purity"trainer/global_step2:JPH9H…bC Purity/2FRt*SRqqt3FBt/FBt_purity"trainer/global_step2:JP»¹øH†bC Purity/1FRt*SRqqt*FBt/FBt_purity"trainer/global_step2:JPÔóÒH‡bC Purity/1FRt*SRqqt1FBt/FBt_purity"trainer/global_step2:JP„Ó@'JˆbE "Purity/1FRt2SRqqt*FBt/event_purity"trainer/global_step2:JP»i·èN‰bI &Purity/4FRt2SRqqt0FBt/event_proportion"trainer/global_step2:JPÊ}cK@Šb; particle/accuracy_4_of_5"trainer/global_step2:JPîÄ…SJ‹bE "Purity/1FRt3SRqqt2FBt/SRqqt_purity"trainer/global_step2:JP¬"oHŒbC Purity/1FRt0SRqqt3FBt/FBt_purity"trainer/global_step2:JP^é!ÇNbI &Purity/3FRt1SRqqt1FBt/event_proportion"trainer/global_step2:JPaå~ŽNŽbI &Purity/4FRt0SRqqt0FBt/event_proportion"trainer/global_step2:JPqÕzÉNbI &Purity/4FRt3SRqqt3FBt/event_proportion"trainer/global_step2:JPGåJbE "Purity/0FRt0SRqqt4FBt/event_purity"trainer/global_step2:JPiñfQJ‘bE "Purity/2FRt2SRqqt2FBt/SRqqt_purity"trainer/global_step2:JP«¶%*J’bE "Purity/*FRt*SRqqt0FBt/event_purity"trainer/global_step2:JPõ9þN“bI &Purity/2FRt3SRqqt1FBt/event_proportion"trainer/global_step2:JPj ê“N”bI &Purity/3FRt2SRqqt1FBt/event_proportion"trainer/global_step2:JPø°å¸J•bE "Purity/0FRt3SRqqt0FBt/SRqqt_purity"trainer/global_step2:JPoHÖJ–bE "Purity/3FRt4SRqqt*FBt/event_purity"trainer/global_step2:JPƒÊ¢J—bE "Purity/1FRt1SRqqt4FBt/SRqqt_purity"trainer/global_step2:JP²f~ÿH˜bC Purity/*FRt0SRqqt3FBt/FRt_purity"trainer/global_step2:JP£IÙüN™bI &Purity/*FRt2SRqqt*FBt/event_proportion"trainer/global_step2:JPBpNšbI &Purity/0FRt*SRqqt2FBt/event_proportion"trainer/global_step2:JP'õÔòJ›bE "Purity/2FRt*SRqqt*FBt/event_purity"trainer/global_step2:JPÅRåHœbC Purity/2FRt0SRqqt1FBt/FRt_purity"trainer/global_step2:JPð’tŽJbE "Purity/*FRt4SRqqt0FBt/SRqqt_purity"trainer/global_step2:JPsé‰NžbI &Purity/*FRt0SRqqt1FBt/event_proportion"trainer/global_step2:JPî^Ù@Ÿb; particle/accuracy_1_of_2"trainer/global_step2:JPY’@ßb; particle/accuracy_7_of_7"trainer/global_step2:JP%sVhHàbC Purity/1FRt0SRqqt*FBt/FRt_purity"trainer/global_step2:JPhâ(!JábE "Purity/0FRt0SRqqt*FBt/event_purity"trainer/global_step2:JP£Ê NâbI &Purity/0FRt4SRqqt2FBt/event_proportion"trainer/global_step2:JPÚoNãbI &Purity/1FRt1SRqqt2FBt/event_proportion"trainer/global_step2:JPÏ#NäbI &Purity/2FRt4SRqqt1FBt/event_proportion"trainer/global_step2:JP¡sNåbI &Purity/3FRt3SRqqt4FBt/event_proportion"trainer/global_step2:JPMëlHæbC Purity/*FRt4SRqqt0FBt/FRt_purity"trainer/global_step2:JPZD›†HçbC Purity/3FRt2SRqqt*FBt/FBt_purity"trainer/global_step2:JPÑëËJèbE "Purity/*FRt*SRqqt*FBt/SRqqt_purity"trainer/global_step2:JPÁÀEŽJébE "Purity/*FRt1SRqqt2FBt/event_purity"trainer/global_step2:JP_ŸNêbI &Purity/*FRt1SRqqt2FBt/event_proportion"trainer/global_step2:JPT|sHëbC Purity/*FRt1SRqqt2FBt/FRt_purity"trainer/global_step2:JPmM¶aHìbC Purity/*FRt2SRqqt0FBt/FRt_purity"trainer/global_step2:JP–«ÔJíbE "Purity/2FRt*SRqqt2FBt/event_purity"trainer/global_step2:JP㺂NîbI &Purity/4FRt2SRqqt4FBt/event_proportion"trainer/global_step2:JP§iî;ïb6 jet/accuracy_1_of_4"trainer/global_step2:JP¶ÒÝ)NðbI &Purity/*FRt*SRqqt4FBt/event_proportion"trainer/global_step2:JP¤–½¦HñbC Purity/*FRt1SRqqt2FBt/FBt_purity"trainer/global_step2:JPÌ«GSJòbE "Purity/2FRt0SRqqt1FBt/event_purity"trainer/global_step2:JPY!NóbI &Purity/4FRt*SRqqt4FBt/event_proportion"trainer/global_step2:JPÍWt~JôbE "Purity/*FRt4SRqqt*FBt/SRqqt_purity"trainer/global_step2:JPÓÄ#[HõbC Purity/2FRt4SRqqt0FBt/FRt_purity"trainer/global_step2:JPÙÍ6ÒHöbC Purity/1FRt0SRqqt4FBt/FBt_purity"trainer/global_step2:JPaEH÷bC Purity/2FRt1SRqqt*FBt/FBt_purity"trainer/global_step2:JPÀÐHøbC Purity/2FRt1SRqqt1FBt/FBt_purity"trainer/global_step2:JP¯¦LNùbI &Purity/4FRt0SRqqt4FBt/event_proportion"trainer/global_step2:JPAû6HúbC Purity/3FRt2SRqqt1FBt/FBt_purity"trainer/global_step2:JP!ºYuHûbC Purity/0FRt2SRqqt2FBt/FBt_purity"trainer/global_step2:JPÀ¥Ê;üb6 jet/accuracy_5_of_7"trainer/global_step2:JPHñcJýbE "Purity/1FRt1SRqqt3FBt/SRqqt_purity"trainer/global_step2:JPÇ9± JþbE "Purity/0FRt*SRqqt1FBt/SRqqt_purity"trainer/global_step2:JPFSÊ9NÿbI &Purity/0FRt1SRqqt1FBt/event_proportion"trainer/global_step2:JPL‘*OH€bC Purity/1FRt3SRqqt1FBt/FRt_purity"trainer/global_step2:JPG…•¼;bC Purity/3FRt0SRqqt*FBt/FRt_purity"trainer/global_stepÕõ p2:JP_©¡²N‚bI &Purity/3FRt4SRqqt4FBt/event_proportion"trainer/global_step2:JP²D~HƒbC Purity/0FRt*SRqqt4FBt/FBt_purity"trainer/global_step2:JPHvÑH„bC Purity/0FRt4SRqqt*FBt/FBt_purity"trainer/global_step2:JPyŸH…bC Purity/0FRt4SRqqt1FBt/FBt_purity"trainer/global_step2:JPlxJ†bE "Purity/0FRt0SRqqt2FBt/event_purity"trainer/global_step2:JPüD:H‡bC Purity/1FRt1SRqqt*FBt/FRt_purity"trainer/global_step2:JPÞ+ÙNˆbI &Purity/1FRt3SRqqt*FBt/event_proportion"trainer/global_step2:JPºZæ@‰b; particle/accuracy_2_of_4"trainer/global_step2:JPâ÷šHŠbC Purity/1FRt*SRqqt4FBt/FRt_purity"trainer/global_step2:JPžyaFH‹bC Purity/1FRt1SRqqt4FBt/FBt_purity"trainer/global_step2:JP‘ýk^HŒbC Purity/3FRt3SRqqt0FBt/FRt_purity"trainer/global_step2:JP×wŒJbE "Purity/4FRt*SRqqt*FBt/event_purity"trainer/global_step2:JPIøyHŽbC Purity/*FRt1SRqqt*FBt/FRt_purity"trainer/global_step2:JPåÉ»NbI &Purity/1FRt4SRqqt2FBt/event_proportion"trainer/global_step2:JPÀVœRHbC Purity/2FRt0SRqqt*FBt/FBt_purity"trainer/global_step2:JP¯u•H‘bC Purity/2FRt0SRqqt1FBt/FBt_purity"trainer/global_step2:JPŸ¹åõ;’b6 jet/accuracy_4_of_5"trainer/global_step2:JPËÌ<J“bE "Purity/3FRt1SRqqt2FBt/SRqqt_purity"trainer/global_step2:JP€ íH”bC Purity/2FRt*SRqqt3FBt/FRt_purity"trainer/global_step2:JPÕeÍJ•bE "Purity/4FRt2SRqqt*FBt/event_purity"trainer/global_step2:JPú6JýN–bI &Purity/0FRt3SRqqt2FBt/event_proportion"trainer/global_step2:JP£œß®H—bC Purity/1FRt0SRqqt2FBt/FRt_purity"trainer/global_step2:JP꺓èN˜bI &Purity/1FRt2SRqqt2FBt/event_proportion"trainer/global_step2:JP‹gÞH™bC Purity/2FRt3SRqqt0FBt/FRt_purity"trainer/global_step2:JPFð‹NšbI &Purity/3FRt*SRqqt4FBt/event_proportion"trainer/global_step2:JP-™ãN›bI &Purity/4FRt0SRqqt1FBt/event_proportion"trainer/global_step2:JP—1pHœbC Purity/*FRt1SRqqt4FBt/FBt_purity"trainer/global_step2:JP6ÿJbE "Purity/3FRt3SRqqt*FBt/event_purity"trainer/global_step2:JP’F,.HžbC Purity/1FRt0SRqqt3FBt/FRt_purity"trainer/global_step2:JPîOHŸbC Purity/3FRt0SRqqt2FBt/FBt_purity"trainer/global_step2:JPߥü¯J bE "Purity/*FRt2SRqqt3FBt/event_purity"trainer/global_step2:JP øœ­J¡bE "Purity/*FRt1SRqqt1FBt/event_purity"trainer/global_step2:JPw GoH¢bC Purity/0FRt*SRqqt2FBt/FBt_purity"trainer/global_step2:JPË\átJ£bE "Purity/0FRt2SRqqt1FBt/SRqqt_purity"trainer/global_step2:JP]OH¤bC Purity/1FRt1SRqqt2FBt/FBt_purity"trainer/global_step2:JPûd% J¥bE "Purity/2FRt*SRqqt*FBt/SRqqt_purity"trainer/global_step2:JP¡ú7H¦bC Purity/2FRt*SRqqt2FBt/FRt_purity"trainer/global_step2:JPÞü‡N§bI &Purity/4FRt2SRqqt1FBt/event_proportion"trainer/global_step2:JP€ˆaÂJ¨bE "Purity/*FRt*SRqqt*FBt/event_purity"trainer/global_step2:JP®ÁBwH©bC Purity/*FRt*SRqqt2FBt/FRt_purity"trainer/global_step2:JPˆ€GHªbC Purity/*FRt0SRqqt*FBt/FRt_purity"trainer/global_step2:JPÓ¦âÎJ«bE "Purity/*FRt1SRqqt0FBt/event_purity"trainer/global_step2:JP˨¢¬H¬bC Purity/0FRt1SRqqt2FBt/FBt_purity"trainer/global_step2:JP¶6N­bI &Purity/0FRt2SRqqt0FBt/event_proportion"trainer/global_step2:JP‘|%|N®bI &Purity/0FRt4SRqqt0FBt/event_proportion"trainer/global_step2:JP)Æ–£;¯b6 jet/accuracy_5_of_5"trainer/global_step2:JPwFJ°bE "Purity/*FRt3SRqqt1FBt/event_purity"trainer/global_step2:JPݦØýJ±bE "Purity/0FRt0SRqqt0FBt/event_purity"trainer/global_step2:JP¨NY§J²bE "Purity/0FRt1SRqqt1FBt/event_purity"trainer/global_step2:JP Ð¥JJ³bE "Purity/3FRt0SRqqt*FBt/event_purity"trainer/global_step2:JPR3 ~N´bI &Purity/3FRt4SRqqt1FBt/event_proportion"trainer/global_step2:JPElä;NµbI &Purity/4FRt0SRqqt2FBt/event_proportion"trainer/global_step2:JPIgb—;¶b6 jet/accuracy_2_of_5"trainer/global_step2:JP³gåH·bC Purity/*FRt0SRqqt4FBt/FBt_purity"trainer/global_step2:JPþ£nlJ¸bE "Purity/*FRt0SRqqt0FBt/event_purity"trainer/global_step2:JPyÃÐN¹bI &Purity/*FRt2SRqqt4FBt/event_proportion"trainer/global_step2:JP¥JºbE "Purity/1FRt*SRqqt3FBt/event_purity"trainer/global_step2:JP]eàIJ»bE "Purity/1FRt1SRqqt1FBt/event_purity"trainer/global_step2:JP)qÓÑJ¼bE "Purity/2FRt*SRqqt1FBt/SRqqt_purity"trainer/global_step2:JPj2ßJ½bE "Purity/2FRt1SRqqt1FBt/event_purity"trainer/global_step2:JPúI¬J¾bE "Purity/3FRt*SRqqt*FBt/SRqqt_purity"trainer/global_step2:JPÝ«-_H¿bC Purity/3FRt2SRqqt1FBt/FRt_purity"trainer/global_step2:JP`ñl ·HãbC Purity/4FRt2SRqqt0FBt/FRt_purity"trainer/global_step2:JPæ ð*JäbE "Purity/*FRt*SRqqt0FBt/SRqqt_purity"trainer/global_step2:JP™á¾NåbI &Purity/1FRt0SRqqt1FBt/event_proportion"trainer/global_step2:JP×^×IJæbE "Purity/2FRt0SRqqt*FBt/event_purity"trainer/global_step2:JP[o¡HçbC Purity/2FRt0SRqqt0FBt/FRt_purity"trainer/global_step2:JPÀ\JèbE "Purity/2FRt1SRqqt2FBt/event_purity"trainer/global_step2:JPž–“@éb; particle/accuracy_3_of_4"trainer/global_step2:JPÏ©ZœJêbE "Purity/0FRt3SRqqt0FBt/event_purity"trainer/global_step2:JPŠäÀJëbE "Purity/3FRt2SRqqt1FBt/SRqqt_purity"trainer/global_step2:JP·$· HìbC Purity/0FRt*SRqqt3FBt/FBt_purity"trainer/global_step2:JPS§vNíbI &Purity/1FRt2SRqqt1FBt/event_proportion"trainer/global_step2:JPaß ØJîbE "Purity/1FRt2SRqqt1FBt/SRqqt_purity"trainer/global_step2:JP©o);ïb6 jet/accuracy_1_of_6"trainer/global_step2:JPåFÑ›HðbC Purity/2FRt3SRqqt*FBt/FBt_purity"trainer/global_step2:JPŠ 8\HñbC Purity/2FRt3SRqqt1FBt/FBt_purity"trainer/global_step2:JP’óbNHòbC Purity/1FRt1SRqqt3FBt/FBt_purity"trainer/global_step2:JPnõÑ.JóbE "Purity/1FRt*SRqqt2FBt/event_purity"trainer/global_step2:JP­û¼¹JôbE "Purity/1FRt2SRqqt0FBt/event_purity"trainer/global_step2:JP3bÍHõbC Purity/3FRt*SRqqt1FBt/FRt_purity"trainer/global_step2:JPŸkH=JöbE "Purity/2FRt4SRqqt*FBt/SRqqt_purity"trainer/global_step2:JPSj1yJ÷bE "Purity/3FRt4SRqqt0FBt/event_purity"trainer/global_step2:JP^*¯ÕHøbC Purity/*FRt1SRqqt3FBt/FBt_purity"trainer/global_step2:JPKl1HùbC Purity/*FRt3SRqqt2FBt/FBt_purity"trainer/global_step2:JP':GRNúbI &Purity/*FRt4SRqqt0FBt/event_proportion"trainer/global_step2:JP‹9@šJûbE "Purity/0FRt*SRqqt0FBt/SRqqt_purity"trainer/global_step2:JP;Ge3@üb; particle/accuracy_1_of_3"trainer/global_step2:JP>×XJýbE "Purity/0FRt*SRqqt3FBt/SRqqt_purity"trainer/global_step2:JP ¬çžJþbE "Purity/*FRt0SRqqt1FBt/event_purity"trainer/global_step2:JPp¼NÿbI &Purity/*FRt0SRqqt2FBt/event_proportion"trainer/global_step2:JPçØ>N€ bI &Purity/*FRt2SRqqt2FBt/event_proportion"trainer/global_step2:JP¨_vN bI &Purity/*FRt4SRqqt4FBt/event_proportion"trainer/global_step2:JPò£nQJ‚ bE "Purity/0FRt2SRqqt0FBt/SRqqt_purity"trainer/global_step2:JP[cÄBHƒ bC Purity/2FRt1SRqqt0FBt/FRt_purity"trainer/global_step2:JPß÷D@„ b; particle/accuracy_4_of_6"trainer/global_step2:JP¹ÀëH… bC Purity/*FRt0SRqqt3FBt/FBt_purity"trainer/global_step2:JPµ®r N† bI &Purity/1FRt1SRqqt*FBt/event_proportion"trainer/global_step2:JPÇËÜyN‡ bI &Purity/4FRt*SRqqt0FBt/event_proportion"trainer/global_step2:JP DËxJˆ bE "Purity/3FRt4SRqqt*FBt/SRqqt_purity"trainer/global_step2:JP ÆšH‰ bC Purity/4FRt0SRqqt*FBt/FRt_purity"trainer/global_step2:JPÄ,sNŠ bI &Purity/*FRt1SRqqt*FBt/event_proportion"trainer/global_step2:JP=[ÞÒH‹ bC Purity/1FRt*SRqqt1FBt/FRt_purity"trainer/global_step2:JP{¶|NŒ bI &Purity/4FRt3SRqqt*FBt/event_proportion"trainer/global_step2:JPlâbS@ b; particle/accuracy_1_of_6"trainer/global_step2:JPîQ˜qHŽ bC Purity/4FRt*SRqqt*FBt/FRt_purity"trainer/global_step2:JPM©•ÑJ bE "Purity/1FRt2SRqqt*FBt/SRqqt_purity"trainer/global_step2:JPã;4J bE "Purity/1FRt3SRqqt1FBt/event_purity"trainer/global_step2:JPy‚;­N‘ bI &Purity/2FRt*SRqqt1FBt/event_proportion"trainer/global_step2:JPõUµçJ’ bE "Purity/2FRt2SRqqt1FBt/SRqqt_purity"trainer/global_step2:JP#ÔÑN“ bI &Purity/2FRt3SRqqt*FBt/event_proportion"trainer/global_step2:JP'ß1‚J” bE "Purity/1FRt0SRqqt4FBt/event_purity"trainer/global_step2:JP]kq®J• bE "Purity/0FRt2SRqqt3FBt/event_purity"trainer/global_step2:JPõX;ˆJ– bE "Purity/*FRt2SRqqt*FBt/event_purity"trainer/global_step2:JPbÌq3J— bE "Purity/1FRt4SRqqt*FBt/SRqqt_purity"trainer/global_step2:JPOþ'H˜ bC Purity/2FRt2SRqqt2FBt/FRt_purity"trainer/global_step2:JPÛʦN™ bI &Purity/2FRt0SRqqt4FBt/event_proportion"trainer/global_step2:JP –Jš bE "Purity/2FRt1SRqqt0FBt/SRqqt_purity"trainer/global_step2:JPPÏnN› bI &Purity/4FRt0SRqqt3FBt/event_proportion"trainer/global_step2:JP‘òÆNœ bI &Purity/4FRt3SRqqt4FBt/event_proportion"trainer/global_step2:JPy‹`J bE "Purity/0FRt4SRqqt*FBt/event_purity"trainer/global_step2:JP“Ÿò;ž b6 jet/accuracy_6_of_7"trainer/global_step2:JPZo'õHŸ bC Purity/0FRt1SRqqt3FBt/FBt_purity"trainer/global_step2:JPÊgÚîJ  bE "Purity/4FRt*SRqqt0FBt/SRqqt_purity"trainer/global_step2:JPÒ´WµJ¡ bE "Purity/*FRt*SRqqt1FBt/SRqqt_purity"trainer/global_step2:JP —Á­H¢ bC Purity/*FRt2SRqqt1FBt/FRt_purity"trainer/global_step2:JPH/ ÀN£ bI &Purity/*FRt4SRqqt3FBt/event_proportion"trainer/global_step2:JPm¶ÉJ¤ bE "Purity/0FRt*SRqqt2FBt/SRqqt_purity"trainer/global_step2:JP¬ÇPªJ¥ bE "Purity/1FRt3SRqqt0FBt/event_purity"trainer/global_step2:JP¯ÄN¦ bI &Purity/1FRt4SRqqt0FBt/event_proportion"trainer/global_step2:JPI›ÌÆN§ bI &Purity/2FRt0SRqqt*FBt/event_proportion"trainer/global_step2:JPÁäà N¨ bI &Purity/1FRt0SRqqt3FBt/event_proportion"trainer/global_step2:JP/èyˆJ© bE "Purity/*FRt2SRqqt1FBt/SRqqt_purity"trainer/global_step2:JPÈà “Nª bI &Purity/0FRt2SRqqt2FBt/event_proportion"trainer/global_step2:JP–]v†J« bE "Purity/2FRt0SRqqt0FBt/event_purity"trainer/global_step2:JPåsHJ¬ bE "Purity/2FRt1SRqqt*FBt/SRqqt_purity"trainer/global_step2:JP“7‚ãH­ bC Purity/3FRt1SRqqt2FBt/FRt_purity"trainer/global_step2:JPô¸¡ƒJ® bE "Purity/*FRt1SRqqt4FBt/event_purity"trainer/global_step2:JPñÙH¯ bC Purity/*FRt*SRqqt1FBt/FRt_purity"trainer/global_step2:JP=–¹N° bI &Purity/1FRt0SRqqt4FBt/event_proportion"trainer/global_step2:JPO.¸J± bE "Purity/1FRt3SRqqt*FBt/event_purity"trainer/global_step2:JPfDPN² bI &Purity/2FRt2SRqqt*FBt/event_proportion"trainer/global_step2:JP€ÙÃN³ bI &Purity/4FRt4SRqqt1FBt/event_proportion"trainer/global_step2:JP¢gìH´ bC Purity/3FRt4SRqqt0FBt/FRt_purity"trainer/global_step2:JP|ÆN¼ bI &Purity/0FRt*SRqqt*FBt/event_proportion"trainer/global_step2:JPcfº‹N½ bI &Purity/1FRt2SRqqt*FBt/event_proportion"trainer/global_step2:JPä¿ÁH¾ bC Purity/2FRt1SRqqt2FBt/FBt_purity"trainer/global_step2:JPÄ™N¿ bI &Purity/4FRt2SRqqt2FBt/event_proportion"trainer/global_step2:JPï—¯NÀ bI &Purity/4FRt3SRqqt1FBt/event_proportion"trainer/global_step2:JP¸QßHÁ bC Purity/*FRt1SRqqt*FBt/FBt_purity"trainer/global_step2:JPFîºwH bC Purity/*FRt1SRqqt1FBt/FBt_purity"trainer/global_step2:JPþ=îHà bC Purity/2FRt0SRqqt*FBt/FRt_purity"trainer/global_step2:JPø3RNÄ bI &Purity/4FRt4SRqqt4FBt/event_proportion"trainer/global_step2:JPçñ£JÅ bE "Purity/1FRt1SRqqt3FBt/event_purity"trainer/global_step2:JP'ÚJÑJÆ bE "Purity/4FRt1SRqqt*FBt/event_purity"trainer/global_step2:JPÊz¼¢HÇ bC Purity/0FRt*SRqqt*FBt/FBt_purity"trainer/global_step2:JP)³HÈ bC Purity/0FRt*SRqqt1FBt/FBt_purity"trainer/global_step2:JPå}ž+HÉ bC Purity/1FRt1SRqqt*FBt/FBt_purity"trainer/global_step2:JPŒuƒHÊ bC Purity/1FRt1SRqqt1FBt/FBt_purity"trainer/global_step2:JPÄs<üJË bE "Purity/3FRt1SRqqt0FBt/SRqqt_purity"trainer/global_step2:JPrÎ@Ì b; particle/accuracy_3_of_3"trainer/global_step2:JP£õ«JÍ bE "Purity/*FRt3SRqqt0FBt/SRqqt_purity"trainer/global_step2:JP°à“žNÎ bI &Purity/0FRt*SRqqt0FBt/event_proportion"trainer/global_step2:JPÑI¨ÜJÏ bE "Purity/1FRt1SRqqt*FBt/SRqqt_purity"trainer/global_step2:JPbýÏ¥NÐ bI &Purity/2FRt*SRqqt4FBt/event_proportion"trainer/global_step2:JPzó9HÑ bC Purity/3FRt0SRqqt*FBt/FBt_purity"trainer/global_step2:JP„Ò¾HÒ bC Purity/3FRt0SRqqt1FBt/FBt_purity"trainer/global_step2:JPhÏ €NÓ bI &Purity/0FRt1SRqqt0FBt/event_proportion"trainer/global_step2:JPû•QJHÔ bC Purity/2FRt1SRqqt*FBt/FRt_purity"trainer/global_step2:JPBºmJí bE "Purity/4FRt1SRqqt0FBt/event_purity"trainer/global_step2:JPxÆùLHî bC Purity/0FRt1SRqqt*FBt/FBt_purity"trainer/global_step2:JPŒ‹Hï bC Purity/0FRt1SRqqt1FBt/FBt_purity"trainer/global_step2:JPe~bNð bI &Purity/1FRt*SRqqt1FBt/event_proportion"trainer/global_step2:JPc^Nñ bI &Purity/2FRt0SRqqt2FBt/event_proportion"trainer/global_step2:JP!¨@ò b; particle/accuracy_1_of_1"trainer/global_step2:JPÛC@ó b; particle/accuracy_3_of_5"trainer/global_step2:JPÑ©?VJô bE "Purity/0FRt1SRqqt2FBt/SRqqt_purity"trainer/global_step2:JPZï‰Jõ bE "Purity/2FRt2SRqqt0FBt/SRqqt_purity"trainer/global_step2:JP»¥ñ Hö bC Purity/1FRt0SRqqt1FBt/FRt_purity"trainer/global_step2:JP°«ùN÷ bI &Purity/2FRt2SRqqt2FBt/event_proportion"trainer/global_step2:JP.bUoNø bI &Purity/4FRt1SRqqt*FBt/event_proportion"trainer/global_step2:JPÈf÷;ù b6 jet/accuracy_3_of_4"trainer/global_step2:JP2mJú bE "Purity/*FRt1SRqqt2FBt/SRqqt_purity"trainer/global_step2:JP ŠrNû bI &Purity/1FRt3SRqqt0FBt/event_proportion"trainer/global_step2:JP ÚxJü bE "Purity/2FRt*SRqqt2FBt/SRqqt_purity"trainer/global_step2:JPF™Ný bI &Purity/2FRt1SRqqt4FBt/event_proportion"trainer/global_step2:JP¼EfNþ bI &Purity/3FRt1SRqqt4FBt/event_proportion"trainer/global_step2:JP–}p<ÿ b7 particle/sensitivity"trainer/global_step2:JPñlZbH€ bC Purity/*FRt0SRqqt*FBt/FBt_purity"trainer/global_step2:JPž&³¥H bC Purity/*FRt0SRqqt1FBt/FBt_purity"trainer/global_step2:JP³«þN‚ bI &Purity/0FRt2SRqqt*FBt/event_proportion"trainer/global_step2:JP0ëFHƒ bC Purity/2FRt*SRqqt1FBt/FRt_purity"trainer/global_step2:JPý©?‚N„ bI &Purity/3FRt3SRqqt1FBt/event_proportion"trainer/global_step2:JP;¡;… b6 jet/accuracy_1_of_1"trainer/global_step2:JPøI‡œJ† bE "Purity/*FRt0SRqqt4FBt/event_purity"trainer/global_step2:JP÷(Ñ J‡ bE "Purity/1FRt1SRqqt4FBt/event_purity"trainer/global_step2:JP¼û;ˆ b6 jet/accuracy_1_of_7"trainer/global_step2:JPÛ§ÊN‰ bI &Purity/*FRt*SRqqt3FBt/event_proportion"trainer/global_step2:JP øŸ¥HŠ bC Purity/1FRt*SRqqt2FBt/FRt_purity"trainer/global_step2:JPÞ‘r^N‹ bI &Purity/3FRt1SRqqt3FBt/event_proportion"trainer/global_step2:JP‚›ÙNŒ bI &Purity/4FRt*SRqqt*FBt/event_proportion"trainer/global_step2:JP‹óH bC Purity/1FRt*SRqqt3FBt/FRt_purity"trainer/global_step2:JPœz`HŽ bC Purity/1FRt1SRqqt0FBt/FRt_purity"trainer/global_step2:JPçPˆN bI &Purity/4FRt*SRqqt1FBt/event_proportion"trainer/global_step2:JP1ºO»H bC Purity/3FRt3SRqqt*FBt/FRt_purity"trainer/global_step2:JPcI˜N‘ bI &Purity/1FRt3SRqqt4FBt/event_proportion"trainer/global_step2:JP3ÿ‚H’ bC Purity/2FRt2SRqqt1FBt/FRt_purity"trainer/global_step2:JPÉ]û,H“ bC Purity/3FRt0SRqqt0FBt/FRt_purity"trainer/global_step2:JPJkÑ…;” b6 jet/accuracy_4_of_4"trainer/global_step2:JPãÂß‚H• bC Purity/2FRt0SRqqt3FBt/FBt_purity"trainer/global_step2:JP¦ÑŠŠJ– bE "Purity/3FRt3SRqqt0FBt/event_purity"trainer/global_step2:JP-HÛ;— b6 loss/val_total_loss"trainer/global_step2:JPéå,$J˜ bE "Purity/0FRt*SRqqt3FBt/event_purity"trainer/global_step2:JPË:‚N™ bI &Purity/1FRt*SRqqt2FBt/event_proportion"trainer/global_step2:JPZ­ìÆNš bI &Purity/2FRt*SRqqt2FBt/event_proportion"trainer/global_step2:JPL&âzN› bI &Purity/2FRt1SRqqt1FBt/event_proportion"trainer/global_step2:JP…*öHHœ bC Purity/2FRt3SRqqt*FBt/FRt_purity"trainer/global_step2:JPŒd†þ2 - _wandbruntime‚1071  _runtime‚1071êmífÖ&ž ÿ£ >&Purity/*FRt*SRqqt2FBt/event_proportion‚0.06747731566429138 >&Purity/*FRt0SRqqt0FBt/event_proportion‚0.15757308900356293 @&Purity/*FRt1SRqqt3FBt/event_proportion‚0.0006930443341843784 >&Purity/0FRt0SRqqt1FBt/event_proportion‚0.08606351166963577 9"Purity/0FRt0SRqqt3FBt/event_purity‚0.1599999964237213 :"Purity/1FRt3SRqqt0FBt/SRqqt_purity‚0.22186385095119476 ("Purity/0FRt*SRqqt4FBt/event_purity‚0 =&Purity/*FRt1SRqqt1FBt/event_proportion‚0.1001134067773819 >&Purity/*FRt2SRqqt1FBt/event_proportion‚0.02255544438958168 8 Purity/3FRt1SRqqt*FBt/FRt_purity‚0.16152264177799225 ("Purity/2FRt4SRqqt*FBt/event_purity‚0 ("Purity/2FRt0SRqqt3FBt/event_purity‚0 & Purity/2FRt2SRqqt2FBt/FBt_purity‚1 & Purity/1FRt2SRqqt*FBt/FBt_purity‚1 & Purity/1FRt2SRqqt1FBt/FBt_purity‚1 @&Purity/3FRt0SRqqt0FBt/event_proportion‚0.0015120967291295528 ?&Purity/*FRt3SRqqt1FBt/event_proportion‚0.002142137149348855 ("Purity/0FRt2SRqqt2FBt/event_purity‚0 ("Purity/3FRt*SRqqt2FBt/event_purity‚0 @&Purity/2FRt*SRqqt3FBt/event_proportion‚6.300403038039804e-05 7validation_average_jet_accuracy‚0.38774991035461426 +jet/accuracy_3_of_6‚0.05263157933950424 8 Purity/2FRt3SRqqt1FBt/FRt_purity‚0.15000000596046448 :"Purity/0FRt*SRqqt0FBt/event_purity‚0.33333590626716614 ("Purity/0FRt0SRqqt1FBt/event_purity‚1 >&Purity/1FRt1SRqqt1FBt/event_proportion‚0.03944052383303642 ("Purity/2FRt1SRqqt*FBt/event_purity‚0 ?&Purity/3FRt1SRqqt*FBt/event_proportion‚0.003717237850651145 8 Purity/3FRt2SRqqt0FBt/FRt_purity‚0.07004830986261368 & Purity/*FRt4SRqqt*FBt/FBt_purity‚1 & Purity/*FRt4SRqqt1FBt/FBt_purity‚1 9"Purity/*FRt0SRqqt2FBt/event_purity‚0.2601272463798523 & Purity/*FRt2SRqqt*FBt/FBt_purity‚1 & Purity/*FRt2SRqqt1FBt/FBt_purity‚1 9"Purity/1FRt2SRqqt0FBt/SRqqt_purity‚0.3070523738861084 ,&Purity/1FRt4SRqqt3FBt/event_proportion‚0 ,&Purity/4FRt4SRqqt0FBt/event_proportion‚0 ("Purity/3FRt4SRqqt0FBt/SRqqt_purity‚0 9"Purity/0FRt3SRqqt1FBt/SRqqt_purity‚0.3333333134651184 :"Purity/0FRt2SRqqt0FBt/event_purity‚0.03554140031337738 ("Purity/3FRt1SRqqt1FBt/event_purity‚0 & Purity/0FRt2SRqqt3FBt/FBt_purity‚1 A&Purity/0FRt1SRqqt3FBt/event_proportion‚0.00044102821266278625 @&Purity/2FRt4SRqqt*FBt/event_proportion‚0.0003150201519019902 ("Purity/*FRt4SRqqt1FBt/event_purity‚0 ."Purity/0FRt4SRqqt*FBt/SRqqt_purity‚0.03125 ("Purity/1FRt2SRqqt2FBt/event_purity‚0 & Purity/*FRt3SRqqt2FBt/FRt_purity‚1 9"Purity/*FRt2SRqqt*FBt/SRqqt_purity‚0.3108208477497101 ,&Purity/4FRt1SRqqt2FBt/event_proportion‚0 :"Purity/2FRt4SRqqt0FBt/SRqqt_purity‚0.05000000074505806 ("Purity/1FRt4SRqqt*FBt/event_purity‚0 & Purity/2FRt*SRqqt3FBt/FBt_purity‚1 6 Purity/1FRt*SRqqt*FBt/FBt_purity‚0.994266927242279 & Purity/1FRt*SRqqt1FBt/FBt_purity‚1 ("Purity/1FRt2SRqqt*FBt/event_purity‚0 @&Purity/4FRt2SRqqt0FBt/event_proportion‚6.300403038039804e-05 particle/accuracy_4_of_5‚1 9"Purity/1FRt3SRqqt2FBt/SRqqt_purity‚0.3333333432674408 7 Purity/1FRt0SRqqt3FBt/FBt_purity‚0.9523809552192688 @&Purity/3FRt1SRqqt1FBt/event_proportion‚0.0009450604557059705 @&Purity/4FRt0SRqqt0FBt/event_proportion‚6.300403038039804e-05 ,&Purity/4FRt3SRqqt3FBt/event_proportion‚0 ("Purity/0FRt0SRqqt4FBt/event_purity‚0 *"Purity/2FRt2SRqqt2FBt/SRqqt_purity‚0.5 % _timestamp‚1.7885019949111621e+09 :"Purity/*FRt*SRqqt0FBt/event_purity‚0.13784803450107574 @&Purity/2FRt3SRqqt1FBt/event_proportion‚0.0003780241822823882 @&Purity/3FRt2SRqqt1FBt/event_proportion‚0.0003780241822823882 :"Purity/0FRt3SRqqt0FBt/SRqqt_purity‚0.22013887763023376 ("Purity/3FRt4SRqqt*FBt/event_purity‚0 ("Purity/1FRt1SRqqt4FBt/SRqqt_purity‚1 8 Purity/*FRt0SRqqt3FBt/FRt_purity‚0.30000001192092896 =&Purity/*FRt2SRqqt*FBt/event_proportion‚0.1332535296678543 >&Purity/0FRt*SRqqt2FBt/event_proportion‚0.03944052383303642 <"Purity/2FRt*SRqqt*FBt/event_purity‚0.0011243795743212104 8 Purity/2FRt0SRqqt1FBt/FRt_purity‚0.25723543763160706 0"Purity/*FRt4SRqqt0FBt/SRqqt_purity‚ 0.1015625 >&Purity/*FRt0SRqqt1FBt/event_proportion‚0.14591734111309052 particle/accuracy_1_of_2‚1 * Purity/0FRt0SRqqt4FBt/FBt_purity‚0.875 ("Purity/3FRt0SRqqt2FBt/event_purity‚0 @&Purity/3FRt*SRqqt2FBt/event_proportion‚0.0003150201519019902 * Purity/*FRt*SRqqt4FBt/FBt_purity‚0.875 9"Purity/3FRt3SRqqt*FBt/SRqqt_purity‚0.3095238208770752 7 Purity/*FRt*SRqqt*FBt/FRt_purity‚0.1630123406648636 7 Purity/*FRt0SRqqt2FBt/FRt_purity‚0.3921659588813782 ,&Purity/0FRt3SRqqt4FBt/event_proportion‚0 ,&Purity/0FRt4SRqqt3FBt/event_proportion‚0 6 Purity/1FRt2SRqqt0FBt/FRt_purity‚0.152561217546463 7 Purity/0FRt0SRqqt2FBt/FBt_purity‚0.9885537028312683 ,&Purity/2FRt2SRqqt4FBt/event_proportion‚0 ("Purity/4FRt*SRqqt*FBt/SRqqt_purity‚1 ("Purity/4FRt2SRqqt*FBt/SRqqt_purity‚1 _step‚48 9"Purity/*FRt1SRqqt0FBt/SRqqt_purity‚0.4326682388782501 @&Purity/1FRt4SRqqt*FBt/event_proportion‚0.0003780241822823882 7 Purity/2FRt1SRqqt1FBt/FRt_purity‚0.3018241226673126 ,&Purity/2FRt2SRqqt3FBt/event_proportion‚0 ,&Purity/4FRt3SRqqt2FBt/event_proportion‚0 ("Purity/1FRt*SRqqt4FBt/SRqqt_purity‚1 ("Purity/4FRt0SRqqt0FBt/event_purity‚0 9"Purity/0FRt1SRqqt0FBt/SRqqt_purity‚0.4214657247066498 & Purity/1FRt3SRqqt*FBt/FBt_purity‚1 & Purity/1FRt3SRqqt1FBt/FBt_purity‚1 8 Purity/3FRt*SRqqt0FBt/FRt_purity‚0.09428572654724121 ,&Purity/3FRt3SRqqt3FBt/event_proportion‚0 9"Purity/*FRt*SRqqt3FBt/SRqqt_purity‚0.5952380895614624 =&Purity/*FRt*SRqqt*FBt/event_proportion‚0.9591733813285828 7 Purity/*FRt*SRqqt2FBt/FBt_purity‚0.9873462319374084 ?&Purity/0FRt0SRqqt3FBt/event_proportion‚0.002583165420219302 ?&Purity/0FRt1SRqqt2FBt/event_proportion‚0.008757560513913631 ,&Purity/0FRt3SRqqt3FBt/event_proportion‚0 +jet/accuracy_2_of_3‚0.21224664151668549 particle/accuracy_2_of_3‚1 8 Purity/1FRt1SRqqt2FBt/FRt_purity‚0.33877548575401306 A&Purity/1FRt1SRqqt3FBt/event_proportion‚0.00025201612152159214 7 Purity/2FRt*SRqqt2FBt/FBt_purity‚0.9911110997200012 *"Purity/3FRt*SRqqt1FBt/SRqqt_purity‚0.5 @&Purity/*FRt4SRqqt*FBt/event_proportion‚0.0013230846961960196 ;"Purity/0FRt2SRqqt*FBt/event_purity‚0.033947091549634933 ("Purity/0FRt4SRqqt1FBt/SRqqt_purity‚0 & Purity/3FRt0SRqqt2FBt/FRt_purity‚0 ) Purity/4FRt1SRqqt*FBt/FRt_purity‚0.25 7 Purity/1FRt0SRqqt2FBt/FBt_purity‚0.9862310886383057 >&Purity/*FRt1SRqqt0FBt/event_proportion‚0.33039313554763794 particle/accuracy_6_of_7‚1 8 Purity/1FRt*SRqqt0FBt/FRt_purity‚0.11462289839982986 ("Purity/*FRt2SRqqt2FBt/event_purity‚0 & Purity/1FRt1SRqqt3FBt/FRt_purity‚0 ,&Purity/*FRt4SRqqt2FBt/event_proportion‚0 7 Purity/2FRt*SRqqt*FBt/FRt_purity‚0.1508835405111313 ?&Purity/2FRt2SRqqt0FBt/event_proportion‚0.015435988083481789 ("Purity/2FRt3SRqqt*FBt/event_purity‚0 :"Purity/3FRt1SRqqt*FBt/SRqqt_purity‚0.48148152232170105 ("Purity/3FRt2SRqqt*FBt/event_purity‚0 & Purity/*FRt1SRqqt3FBt/FRt_purity‚0 @&Purity/*FRt4SRqqt1FBt/event_proportion‚6.300403038039804e-05 ;"Purity/0FRt1SRqqt2FBt/event_purity‚0.025317460298538208 =&Purity/1FRt*SRqqt*FBt/event_proportion‚0.4366179406642914 ?&Purity/2FRt2SRqqt1FBt/event_proportion‚0.003969253972172737 ("Purity/3FRt*SRqqt*FBt/event_purity‚0 @&Purity/3FRt1SRqqt0FBt/event_proportion‚0.0025201612152159214 particle/accuracy_2_of_5‚1 particle/accuracy_7_of_7‚1 8 Purity/1FRt0SRqqt*FBt/FRt_purity‚0.17214414477348328 9"Purity/0FRt0SRqqt*FBt/event_purity‚0.8201134204864502 ,&Purity/0FRt4SRqqt2FBt/event_proportion‚0 ?&Purity/1FRt1SRqqt2FBt/event_proportion‚0.006111391354352236 ,&Purity/2FRt4SRqqt1FBt/event_proportion‚0 ,&Purity/3FRt3SRqqt4FBt/event_proportion‚0 ) Purity/*FRt4SRqqt0FBt/FRt_purity‚0.25 & Purity/3FRt2SRqqt*FBt/FBt_purity‚1 9"Purity/*FRt*SRqqt*FBt/SRqqt_purity‚0.3787279725074768 ;"Purity/*FRt1SRqqt2FBt/event_purity‚0.020707787945866585 ?&Purity/*FRt1SRqqt2FBt/event_proportion‚0.017011089250445366 7 Purity/*FRt1SRqqt2FBt/FRt_purity‚0.2981066405773163 8 Purity/*FRt2SRqqt0FBt/FRt_purity‚0.12830613553524017 ("Purity/2FRt*SRqqt2FBt/event_purity‚0 ,&Purity/4FRt2SRqqt4FBt/event_proportion‚0 *jet/accuracy_1_of_4‚0.8258363604545593 @&Purity/*FRt*SRqqt4FBt/event_proportion‚0.0003780241822823882 7 Purity/*FRt1SRqqt2FBt/FBt_purity‚0.9832815527915955 ("Purity/2FRt0SRqqt1FBt/event_purity‚0 ,&Purity/4FRt*SRqqt4FBt/event_proportion‚0 ."Purity/*FRt4SRqqt*FBt/SRqqt_purity‚0.09375 8 Purity/2FRt4SRqqt0FBt/FRt_purity‚0.20000000298023224 & Purity/1FRt0SRqqt4FBt/FBt_purity‚1 trainer/global_step‚603 7 Purity/2FRt1SRqqt*FBt/FBt_purity‚0.9964157938957214 & Purity/2FRt1SRqqt1FBt/FBt_purity‚1 ,&Purity/4FRt0SRqqt4FBt/event_proportion‚0 & Purity/3FRt2SRqqt1FBt/FBt_purity‚1 & Purity/0FRt2SRqqt2FBt/FBt_purity‚1 jet/accuracy_5_of_7‚0 +"Purity/1FRt1SRqqt3FBt/SRqqt_purity‚0.75 9"Purity/0FRt*SRqqt1FBt/SRqqt_purity‚0.4040747582912445 >&Purity/0FRt1SRqqt1FBt/event_proportion‚0.05185231938958168 8 Purity/1FRt3SRqqt1FBt/FRt_purity‚0.30000001192092896 8 Purity/3FRt0SRqqt*FBt/FRt_purity‚0.15656568109989166 ,&Purity/3FRt4SRqqt4FBt/event_proportion‚0 * Purity/0FRt*SRqqt4FBt/FBt_purity‚0.875 & Purity/0FRt4SRqqt*FBt/FBt_purity‚1 & Purity/0FRt4SRqqt1FBt/FBt_purity‚1 :"Purity/0FRt0SRqqt2FBt/event_purity‚0.35592129826545715 8 Purity/1FRt1SRqqt*FBt/FRt_purity‚0.16569927334785461 ?&Purity/1FRt3SRqqt*FBt/event_proportion‚0.009135584346950054 particle/accuracy_2_of_4‚1 & Purity/1FRt*SRqqt4FBt/FRt_purity‚1 ) Purity/1FRt1SRqqt4FBt/FBt_purity‚0.75 7 Purity/3FRt3SRqqt0FBt/FRt_purity‚0.1190476268529892 ("Purity/4FRt*SRqqt*FBt/event_purity‚0 8 Purity/*FRt1SRqqt*FBt/FRt_purity‚0.16205354034900665 ,&Purity/1FRt4SRqqt2FBt/event_proportion‚0 7 Purity/2FRt0SRqqtªL70k*FBt/FBt_purity‚0.9967741966247559 & Purity/2FRt0SRqqt1FBt/FBt_purity‚1 -jet/accuracy_4_of_5‚0.0021505376789718866 +"Purity/3FRt1SRqqt2FBt/SRqqt_purity‚0.25 ( Purity/2FRt*SRqqt3FBt/FRt_purity‚0.5 ("Purity/4FRt2SRqqt*FBt/event_purity‚0 ,&Purity/0FRt3SRqqt2FBt/event_proportion‚0 8 Purity/1FRt0SRqqt2FBt/FRt_purity‚0.45895007252693176 @&Purity/1FRt2SRqqt2FBt/event_proportion‚0.0010080644860863686 8 Purity/2FRt3SRqqt0FBt/FRt_purity‚0.10005411505699158 ,&Purity/3FRt*SRqqt4FBt/event_proportion‚0 ,&Purity/4FRt0SRqqt1FBt/event_proportion‚0 ) Purity/*FRt1SRqqt4FBt/FBt_purity‚0.75 ("Purity/3FRt3SRqqt*FBt/event_purity‚0 7 Purity/1FRt0SRqqt3FBt/FRt_purity‚0.2857142984867096 & Purity/3FRt0SRqqt2FBt/FBt_purity‚1 ("Purity/*FRt2SRqqt3FBt/event_purity‚0 :"Purity/*FRt1SRqqt1FBt/event_purity‚0.07901166379451752 7 Purity/0FRt*SRqqt2FBt/FBt_purity‚0.9866310358047485 9"Purity/0FRt2SRqqt1FBt/SRqqt_purity‚0.3831029236316681 7 Purity/1FRt1SRqqt2FBt/FBt_purity‚0.9836309552192688 :"Purity/2FRt*SRqqt*FBt/SRqqt_purity‚0.37650057673454285 7 Purity/2FRt*SRqqt2FBt/FRt_purity‚0.2499999850988388 ,&Purity/4FRt2SRqqt1FBt/event_proportion‚0 :"Purity/*FRt*SRqqt*FBt/event_purity‚0.21480116248130798 7 Purity/*FRt*SRqqt2FBt/FRt_purity‚0.3559475839138031 8 Purity/*FRt0SRqqt*FBt/FRt_purity‚0.16779378056526184 :"Purity/*FRt1SRqqt0FBt/event_purity‚0.21873195469379425 7 Purity/0FRt1SRqqt2FBt/FBt_purity‚0.9829629063606262 ?&Purity/0FRt2SRqqt0FBt/event_proportion‚0.043283771723508835 A&Purity/0FRt4SRqqt0FBt/event_proportion‚0.00044102821266278625 jet/accuracy_5_of_5‚0 ("Purity/*FRt3SRqqt1FBt/event_purity‚0 ("Purity/0FRt0SRqqt0FBt/event_purity‚1 9"Purity/0FRt1SRqqt1FBt/event_purity‚0.1262608766555786 ("Purity/3FRt0SRqqt*FBt/event_purity‚0 ,&Purity/3FRt4SRqqt1FBt/event_proportion‚0 ,&Purity/4FRt0SRqqt2FBt/event_proportion‚0 *jet/accuracy_2_of_5‚0.3725011944770813 7 Purity/*FRt0SRqqt4FBt/FBt_purity‚0.8999999761581421 :"Purity/*FRt0SRqqt0FBt/event_purity‚0.06902338564395905 ,&Purity/*FRt2SRqqt4FBt/event_proportion‚0 -"Purity/1FRt*SRqqt3FBt/event_purity‚0.0625 :"Purity/1FRt1SRqqt1FBt/event_purity‚0.03787361457943916 9"Purity/2FRt*SRqqt1FBt/SRqqt_purity‚0.4233380854129791 ("Purity/2FRt1SRqqt1FBt/event_purity‚0 :"Purity/3FRt*SRqqt*FBt/SRqqt_purity‚0.39738646149635315 7 Purity/3FRt2SRqqt1FBt/FRt_purity‚0.2222222238779068 7 Purity/*FRt*SRqqt3FBt/FRt_purity‚0.2647058963775635 9"Purity/0FRt1SRqqt0FBt/event_purity‚0.4214657247066498 :"Purity/2FRt3SRqqt*FBt/SRqqt_purity‚0.25685715675354004 ,&Purity/2FRt3SRqqt2FBt/event_proportion‚0 ("Purity/3FRt1SRqqt*FBt/event_purity‚0 9"Purity/3FRt2SRqqt*FBt/SRqqt_purity‚0.2986111044883728 ,&Purity/3FRt2SRqqt2FBt/event_proportion‚0 particle/accuracy_4_of_4‚1 7 Purity/*FRt0SRqqt2FBt/FBt_purity‚0.9885528087615967 (particle/f_score‚0.39225271344184875 ("Purity/2FRt3SRqqt1FBt/event_purity‚0 ?&Purity/*FRt3SRqqt*FBt/event_proportion‚0.020476309582591057 9"Purity/1FRt*SRqqt1FBt/event_purity‚0.1456620991230011 @&Purity/3FRt3SRqqt0FBt/event_proportion‚0.0005670362734235823 ("Purity/*FRt4SRqqt0FBt/event_purity‚0 :"Purity/0FRt2SRqqt*FBt/SRqqt_purity‚0.30127444863319397 ,&Purity/1FRt2SRqqt4FBt/event_proportion‚0 ?&Purity/2FRt3SRqqt0FBt/event_proportion‚0.003654233878478408 @&Purity/4FRt0SRqqt*FBt/event_proportion‚6.300403038039804e-05 ("Purity/*FRt*SRqqt4FBt/event_purity‚0 ("Purity/*FRt4SRqqt*FBt/event_purity‚0 ("Purity/*FRt1SRqqt3FBt/event_purity‚0 :"Purity/1FRt0SRqqt1FBt/event_purity‚0.26097288727760315 9"Purity/1FRt1SRqqt2FBt/SRqqt_purity‚0.5383502840995789 7 Purity/1FRt4SRqqt0FBt/FRt_purity‚0.3333333432674408 9"Purity/*FRt*SRqqt2FBt/event_purity‚0.1929907351732254 8 Purity/*FRt3SRqqt*FBt/FRt_purity‚0.14518451690673828 ("Purity/*FRt3SRqqt2FBt/event_purity‚0 ,&Purity/3FRt2SRqqt4FBt/event_proportion‚0 :"Purity/1FRt0SRqqt0FBt/event_purity‚0.08189694583415985 >&Purity/1FRt0SRqqt0FBt/event_proportion‚0.13142640888690948 8 Purity/1FRt2SRqqt1FBt/FRt_purity‚0.26075270771980286 ,&Purity/4FRt1SRqqt4FBt/event_proportion‚0 ("Purity/2FRt4SRqqt0FBt/event_purity‚0 7 Purity/3FRt4SRqqt*FBt/FRt_purity‚0.3333333432674408 & Purity/4FRt2SRqqt0FBt/FRt_purity‚0 9"Purity/*FRt*SRqqt0FBt/SRqqt_purity‚0.3667586147785187 ?&Purity/1FRt0SRqqt1FBt/event_proportion‚0.049773186445236206 <"Purity/2FRt0SRqqt*FBt/event_purity‚0.0032583903521299362 8 Purity/2FRt0SRqqt0FBt/FRt_purity‚0.09273655712604523 ("Purity/2FRt1SRqqt2FBt/event_purity‚0 particle/accuracy_3_of_4‚1 ("Purity/0FRt3SRqqt0FBt/event_purity‚0 9"Purity/3FRt2SRqqt1FBt/SRqqt_purity‚0.0833333358168602 7 Purity/0FRt*SRqqt3FBt/FBt_purity‚0.9176955819129944 ?&Purity/1FRt2SRqqt1FBt/event_proportion‚0.008946572430431843 9"Purity/1FRt2SRqqt1FBt/SRqqt_purity‚0.3076421022415161 *jet/accuracy_1_of_6‚0.8684210777282715 & Purity/2FRt3SRqqt*FBt/FBt_purity‚1 & Purity/2FRt3SRqqt1FBt/FBt_purity‚1 7 Purity/1FRt1SRqqt3FBt/FBt_purity‚0.9166666865348816 :"Purity/1FRt*SRqqt2FBt/event_purity‚0.08447099477052689 ("Purity/1FRt2SRqqt0FBt/event_purity‚0 8 Purity/3FRt*SRqqt1FBt/FRt_purity‚0.27828285098075867 :"Purity/2FRt4SRqqt*FBt/SRqqt_purity‚0.05000000074505806 ("Purity/3FRt4SRqqt0FBt/event_purity‚0 6 Purity/*FRt1SRqqt3FBt/FBt_purity‚0.920634925365448 & Purity/*FRt3SRqqt2FBt/FBt_purity‚1 @&Purity/*FRt4SRqqt0FBt/event_proportion‚0.0012600806076079607 :"Purity/0FRt*SRqqt0FBt/SRqqt_purity‚0.36749303340911865 particle/accuracy_1_of_3‚1 :"Purity/0FRt*SRqqt3FBt/SRqqt_purity‚0.36666667461395264 9"Purity/*FRt0SRqqt1FBt/event_purity‚0.6806540489196777 =&Purity/*FRt0SRqqt2FBt/event_proportion‚0.0483240932226181 ?&Purity/*FRt2SRqqt2FBt/event_proportion‚0.002079133177176118 ,&Purity/*FRt4SRqqt4FBt/event_proportion‚0 :"Purity/0FRt2SRqqt0FBt/SRqqt_purity‚0.29334983229637146 8 Purity/2FRt1SRqqt0FBt/FRt_purity‚0.11505943536758423 particle/accuracy_4_of_6‚1 7 Purity/*FRt0SRqqt3FBt/FBt_purity‚0.9236112236976624 =&Purity/1FRt1SRqqt*FBt/event_proportion‚0.1723790317773819 @&Purity/4FRt*SRqqt0FBt/event_proportion‚0.0001890120911411941 ("Purity/3FRt4SRqqt*FBt/SRqqt_purity‚0 & Purity/4FRt0SRqqt*FBt/FRt_purity‚0 =&Purity/*FRt1SRqqt*FBt/event_proportion‚0.4482736885547638 6 Purity/1FRt*SRqqt1FBt/FRt_purity‚0.288621723651886 ,&Purity/4FRt3SRqqt*FBt/event_proportion‚0 particle/accuracy_1_of_6‚1 * Purity/4FRt*SRqqt*FBt/FRt_purity‚0.125 :"Purity/1FRt2SRqqt*FBt/SRqqt_purity‚0.30995163321495056 ("Purity/1FRt3SRqqt1FBt/event_purity‚0 ?&Purity/2FRt*SRqqt1FBt/event_proportion‚0.021169355139136314 9"Purity/2FRt2SRqqt1FBt/SRqqt_purity‚0.3619047999382019 ?&Purity/2FRt3SRqqt*FBt/event_proportion‚0.004032257944345474 ("Purity/1FRt0SRqqt4FBt/event_purity‚0 ("Purity/0FRt2SRqqt3FBt/event_purity‚0 epoch‚0 ;"Purity/*FRt2SRqqt*FBt/event_purity‚0.014175290241837502 +"Purity/1FRt4SRqqt*FBt/SRqqt_purity‚0.25 ( Purity/2FRt2SRqqt2FBt/FRt_purity‚0.5 ,&Purity/2FRt0SRqqt4FBt/event_proportion‚0 9"Purity/2FRt1SRqqt0FBt/SRqqt_purity‚0.4260632395744324 ,&Purity/4FRt0SRqqt3FBt/event_proportion‚0 ,&Purity/4FRt3SRqqt4FBt/event_proportion‚0 ("Purity/0FRt4SRqqt*FBt/event_purity‚0 jet/accuracy_6_of_7‚0 7 Purity/0FRt1SRqqt3FBt/FBt_purity‚0.8999999761581421 ("Purity/4FRt*SRqqt0FBt/SRqqt_purity‚1 9"Purity/*FRt*SRqqt1FBt/SRqqt_purity‚0.4152052402496338 8 Purity/*FRt2SRqqt1FBt/FRt_purity‚0.24810664355754852 ,&Purity/*FRt4SRqqt3FBt/event_proportion‚0 :"Purity/0FRt*SRqqt2FBt/SRqqt_purity‚0.42287036776542664 ("Purity/1FRt3SRqqt0FBt/event_purity‚0 @&Purity/1FRt4SRqqt0FBt/event_proportion‚0.0003780241822823882 ?&Purity/2FRt0SRqqt*FBt/event_proportion‚0.037046369165182114 @&Purity/1FRt0SRqqt3FBt/event_proportion‚0.0010710685746744275 9"Purity/*FRt2SRqqt1FBt/SRqqt_purity‚0.3412233889102936 @&Purity/0FRt2SRqqt2FBt/event_proportion‚0.0010080644860863686 ;"Purity/2FRt0SRqqt0FBt/event_purity‚0.004201680887490511 9"Purity/2FRt1SRqqt*FBt/SRqqt_purity‚0.4491351246833801 ) Purity/3FRt1SRqqt2FBt/FRt_purity‚0.25 ("Purity/*FRt1SRqqt4FBt/event_purity‚0 7 Purity/*FRt*SRqqt1FBt/FRt_purity‚0.2896887958049774 @&Purity/1FRt0SRqqt4FBt/event_proportion‚6.300403038039804e-05 ("Purity/1FRt3SRqqt*FBt/event_purity‚0 ?&Purity/2FRt2SRqqt*FBt/event_proportion‚0.019468246027827263 ,&Purity/4FRt4SRqqt1FBt/event_proportion‚0 7 Purity/3FRt4SRqqt0FBt/FRt_purity‚0.3333333432674408 & Purity/4FRt2SRqqt*FBt/FRt_purity‚0 ;"Purity/*FRt2SRqqt0FBt/event_purity‚0.014578606933355331 7 Purity/*FRt3SRqqt0FBt/FRt_purity‚0.1253616362810135 9"Purity/0FRt1SRqqt*FBt/SRqqt_purity‚0.4262346029281616 >&Purity/1FRt2SRqqt0FBt/event_proportion‚0.04731602966785431 ,&Purity/4FRt4SRqqt3FBt/event_proportion‚0 (particle/accuracy‚0.8451045155525208 >&Purity/0FRt*SRqqt*FBt/event_proportion‚0.40996724367141724 >&Purity/1FRt2SRqqt*FBt/event_proportion‚0.05727066472172737 7 Purity/2FRt1SRqqt2FBt/FBt_purity‚0.9868420958518982 ,&Purity/4FRt2SRqqt2FBt/event_proportion‚0 ,&Purity/4FRt3SRqqt1FBt/event_proportion‚0 7 Purity/*FRt1SRqqt*FBt/FBt_purity‚0.9923127293586731 7 Purity/*FRt1SRqqt1FBt/FBt_purity‚0.9982787370681763 8 Purity/2FRt0SRqqt*FBt/FRt_purity‚0.15170440077781677 ,&Purity/4FRt4SRqqt4FBt/event_proportion‚0 ("Purity/1FRt1SRqqt3FBt/event_purity‚0 ("Purity/4FRt1SRqqt*FBt/event_purity‚0 7 Purity/0FRt*SRqqt*FBt/FBt_purity‚0.9912183880805969 7 Purity/0FRt*SRqqt1FBt/FBt_purity‚0.9987989664077759 7 Purity/1FRt1SRqqt*FBt/FBt_purity‚0.9941508173942566 & Purity/1FRt1SRqqt1FBt/FBt_purity‚1 9"Purity/3FRt1SRqqt0FBt/SRqqt_purity‚0.5416666269302368 particle/accuracy_3_of_3‚1 9"Purity/*FRt3SRqqt0FBt/SRqqt_purity‚0.2284957617521286 >&Purity/0FRt*SRqqt0FBt/event_proportion‚0.21931703388690948 :"Purity/1FRt1SRqqt*FBt/SRqqt_purity‚0.45970261096954346 ,&Purity/2FRt*SRqqt4FBt/event_proportion‚0 & Purity/3FRt0SRqqt*FBt/FBt_purity‚1 & Purity/3FRt0SRqqt1FBt/FBt_purity‚1 =&Purity/0FRt1SRqqt0FBt/event_proportion‚0.1694808453321457 7 Purity/2FRt1SRqqt*FBt/FRt_purity‚0.1587657630443573 ("Purity/3FRt*SRqqt1FBt/event_purity‚0 ,&Purity/3FRt2SRqqt3FBt/event_proportion‚0 +jet/accuracy_2_of_2‚0.09339653700590134 particle/accuracy_3_of_6‚1 ?&Purity/0FRt2SRqqt1FBt/event_proportion‚0.009261592291295528 :"Purity/1FRt*SRqqt0FBt/event_purity‚0.03717515990138054 ,&Purity/1FRt4SRqqt4FBt/event_proportion‚0 7 Purity/2FRt0SRqqt2FBt/FBt_purity‚0.9940475821495056 :"Purity/0FRt3SRqqt*FBt/SRqqt_purity‚0.22051207721233368 ("Purity/0FRt4SRqqt1FBt/event_purity‚0 ("Purity/*FRt2SRqqt3FBt/SRqqt_purity‚0 ?&Purity/2FRt0SRqqt1FBt/event_proportion‚0.008946572430431843 +"Purity/1FRt*SRqqt3FBt/SRqqt_purity‚0.75 * Purity/4FRt*SRqqt0FBt/FRt_purity‚0.125 9"Purity/0FRt*SRqqt*FBt/SRqqt_purity‚0.3767129182815552 @&Purity/0FRt4SRqqt*FBt/event_proportion‚0.0005040322430431843 A&Purity/1FRt*SRqqt4FBt/event_proportion‚0.00012600806076079607 @&Purity/3FRt0SRqqt2FBt/event_proportion‚6.300403038039804e-05 +jet/accuracy_3_of_5‚0.08260243386030197 particle/accuracy_2_of_2‚1 & Purity/4FRt0SRqqt0FBt/FRt_purity‚0 :"Purity/*FRt2SRqqt2FBt/SRqqt_purity‚0.29385966062545776 7"Purity/*FRt1SRqqt3FBt/SRqqt_purity‚0.69047611951828 & Purity/1FRt3SRqqt2FBt/FRt_purity‚1 ("Purity/4FRt1SRqqt0FBt/event_purity‚0 7 Purity/0FRt1SRqqt*FBt/FBt_purity‚0.9909148812294006 7 Purity/0FRt1SRqqt1FBt/FBt_purity‚0.9969950318336487 >&Purity/1FRt*SRqqt1FBt/event_proportion‚0.09929435700178146 ?&Purity/2FRt0SRqqt2FBt/event_proportion‚0.003465221729129553 particle/accuracy_1_of_1‚1 particle/accuracy_3_of_5‚1 :"Purity/0FRt1SRqqt2FBt/SRqqt_purity‚0.44603174924850464 9"Purity/2FRt2SRqqt0FBt/SRqqt_purity‚0.3426436483860016 8 Purity/1FRt0SRqqt1FBt/FRt_purity‚0.31786155700683594 @&Purity/2FRt2SRqqt2FBt/event_proportion‚6.300403038039804e-05 @&Purity/4FRt1SRqqt*FBt/event_proportion‚6.300403038039804e-05 (jet/accuracy_3_of_4‚0.04249257594347 9"Purity/*FRt1SRqqt2FBt/SRqqt_purity‚0.5134053826332092 ?&Purity/1FRt3SRqqt0FBt/event_proportion‚0.007938507944345474 9"Purity/2FRt*SRqqt2FBt/SRqqt_purity‚0.5789473652839661 ,&Purity/2FRt1SRqqt4FBt/event_proportion‚0 ,&Purity/3FRt1SRqqt4FBt/event_proportion‚0 ,particle/sensitivity‚0.32016095519065857 7 Purity/*FRt0SRqqt*FBt/FBt_purity‚0.9920475482940674 & Purity/*FRt0SRqqt1FBt/FBt_purity‚1 >&Purity/0FRt2SRqqt*FBt/event_proportion‚0.05361643061041832 8 Purity/2FRt*SRqqt1FBt/FRt_purity‚0.28566113114356995 ,&Purity/3FRt3SRqqt1FBt/event_proportion‚0 +jet/accuracy_1_of_1‚0.42880913615226746 ("Purity/*FRt0SRqqt4FBt/event_purity‚0 ("Purity/1FRt1SRqqt4FBt/event_purity‚0 jet/accuracy_1_of_7‚1 ?&Purity/*FRt*SRqqt3FBt/event_proportion‚0.004473286215215921 7 Purity/1FRt*SRqqt2FBt/FRt_purity‚0.4272156357765198 ,&Purity/3FRt1SRqqt3FBt/event_proportion‚0 @&Purity/4FRt*SRqqt*FBt/event_proportion‚0.0001890120911411941 ) Purity/1FRt*SRqqt3FBt/FRt_purity‚0.25 7 Purity/1FRt1SRqqt0FBt/FRt_purity‚0.1297631561756134 ,&Purity/4FRt*SRqqt1FBt/event_proportion‚0 7 Purity/3FRt3SRqqt*FBt/FRt_purity‚0.1190476268529892 ,&Purity/1FRt3SRqqt4FBt/event_proportion‚0 8 Purity/2FRt2SRqqt1FBt/FRt_purity‚0.23273809254169464 8 Purity/3FRt0SRqqt0FBt/FRt_purity‚0.04575163498520851 ,jet/accuracy_4_of_4‚0.002061113715171814 & Purity/2FRt0SRqqt3FBt/FBt_purity‚1 ("Purity/3FRt3SRqqt0FBt/event_purity‚0 *loss/val_total_loss‚57.634858344452304 :"Purity/0FRt*SRqqt3FBt/event_purity‚0.12345678359270096 ?&Purity/1FRt*SRqqt2FBt/event_proportion‚0.022303426638245583 ?&Purity/2FRt*SRqqt2FBt/event_proportion‚0.005418346729129553 ?&Purity/2FRt1SRqqt1FBt/event_proportion‚0.007875503972172737 8 Purity/2FRt3SRqqt*FBt/FRt_purity‚0.11204762756824493 @&Purity/3FRt0SRqqt1FBt/event_proportion‚0.0011340725468471646 9"Purity/3FRt1SRqqt1FBt/SRqqt_purity‚0.5769230723381042 9"Purity/*FRt0SRqqt*FBt/event_purity‚0.3466196358203888 8 Purity/*FRt1SRqqt0FBt/FRt_purity‚0.12200312316417694 @&Purity/1FRt1SRqqt4FBt/event_proportion‚6.300403038039804e-05 ,&Purity/1FRt2SRqqt3FBt/event_proportion‚0 9"Purity/2FRt2SRqqt*FBt/SRqqt_purity‚0.3458143472671509 ("Purity/3FRt*SRqqt0FBt/event_purity‚0 @&Purity/3FRt*SRqqt1FBt/event_proportion‚0.0024571572430431843 ("Purity/*FRt4SRqqt1FBt/SRqqt_purity‚0 ("Purity/*FRt3SRqqt*FBt/event_purity‚0 ?&Purity/0FRt0SRqqt2FBt/event_proportion‚0.029674898833036423 @&Purity/3FRt2SRqqt0FBt/event_proportion‚0.0024571572430431843 7 Purity/*FRt2SRqqt2FBt/FRt_purity‚0.3205128312110901 ("Purity/0FRt3SRqqt1FBt/event_purity‚0 ("Purity/1FRt*SRqqt4FBt/event_purity‚0 & Purity/1FRt0SRqqt4FBt/FRt_purity‚1 +"Purity/1FRt4SRqqt0FBt/SRqqt_purity‚0.25 & Purity/*FRt3SRqqt*FBt/FBt_purity‚1 & Purity/*FRt3SRqqt1FBt/FBt_purity‚1 @&Purity/0FRt2SRqqt3FBt/event_proportion‚6.300403038039804e-05 8 Purity/3FRt1SRqqt1FBt/FRt_purity‚0.26923075318336487 @&Purity/3FRt2SRqqt*FBt/event_proportion‚0.0028351815417408943 A&Purity/3FRt4SRqqt*FBt/event_proportion‚0.00012600806076079607 @&Purity/4FRt2SRqqt*FBt/event_proportion‚6.300403038039804e-05 9"Purity/0FRt2SRqqt2FBt/SRqqt_purity‚0.3076923191547394 particle/accuracy_1_of_7‚1 +"Purity/3FRt*SRqqt2FBt/SRqqt_purity‚0.25 ("Purity/0FRt2SRqqt3FBt/SRqqt_purity‚0 8 Purity/*FRt0SRqqt0FBt/FRt_purity‚0.08425780385732651 @&Purity/*FRt1SRqqt4FBt/event_proportion‚6.300403038039804e-05 @&Purity/0FRt*SRqqt3FBt/event_proportion‚0.0030871976632624865 7 Purity/1FRt2SRqqt2FBt/FRt_purity‚0.3055555522441864 9"Purity/1FRt2SRqqt2FBt/SRqqt_purity‚0.1944444626569748 ("Purity/4FRt2SRqqt0FBt/SRqqt_purity‚1 >&Purity/0FRt*SRqqt1FBt/event_proportion‚0.14787046611309052 ("Purity/2FRt3SRqqt0FBt/event_purity‚0 ,&Purity/3FRt4SRqqt2FBt/event_proportion‚0 ,&Purity/4FRt2SRqqt3FBt/event_proportion‚0 particle/accuracy_6_of_6‚1 & Purity/*FRt*SRqqt4FBt/FRt_purity‚1 =&Purity/1FRt*SRqqt0FBt/event_proportion‚0.3135710656642914 A&Purity/3FRt4SRqqt0FBt/event_proportion‚0.00012600806076079607 8 Purity/3FRt*SRqqt2FBt/FRt_purity‚0.20000000298023224 ("Purity/1FRt3SRqqt2FBt/event_purity‚0 :"Purity/1FRt0SRqqt2FBt/event_purity‚0.12427285313606262 9"Purity/1FRt1SRqqt0FBt/SRqqt_purity‚0.4451372027397156 ,&Purity/2FRt4SRqqt3FBt/event_proportion‚0 9"Purity/1FRt0SRqqt3FBt/event_purity‚0.0714285746216774 9"Purity/*FRt1SRqqt*FBt/SRqqt_purity‚0.4429338872432709 >&Purity/*FRt2SRqqt0FBt/event_proportion‚0.10855595022439957 :"Purity/1FRt*SRqqt*FBt/event_purity‚0.06522779166698456 :"Purity/2FRt*SRqqt0FBt/SRqqt_purity‚0.35901257395744324 9"Purity/3FRt2SRqqt0FBt/SRqqt_purity‚0.3278985321521759 ("Purity/0FRt4SRqqt0FBt/event_purity‚0 7 Purity/1FRt4SRqqt*FBt/FRt_purity‚0.3333333432674408 @&Purity/1FRt*SRqqt3FBt/event_proportion‚0.0013230846961960196 @&Purity/3FRt*SRqqt0FBt/event_proportion‚0.0071824598126113415 ("Purity/2FRt2SRqqt2FBt/event_purity‚0 =&Purity/*FRt*SRqqt0FBt/event_proportion‚0.6160534024238586 :"Purity/1FRt1SRqqt0FBt/event_purity‚0.00696146534755826 ,&Purity/4FRt1SRqqt1FBt/event_proportion‚0 & Purity/*FRt0SRqqt4FBt/FRt_purity‚1 ("Purity/0FRt1SRqqt3FBt/event_purity‚0 9"Purity/*FRt*SRqqt2FBt/SRqqt_purity‚0.4694756269454956 ;"Purity/*FRt2SRqqt1FBt/event_purity‚0.014706171117722988 :"Purity/*FRt3SRqqt*FBt/SRqqt_purity‚0.22978773713111877 ,&Purity/4FRt1SRqqt3FBt/event_proportion‚0 9"Purity/*FRt3SRqqt2FBt/SRqqt_purity‚0.3333333432674408 8 Purity/*FRt*SRqqt0FBt/FRt_purity‚0.10999345779418945 6 Purity/1FRt2SRqqt*FBt/FRt_purity‚0.173463374376297 @&Purity/2FRt0SRqqt3FBt/event_proportion‚6.300403038039804e-05 ("Purity/2FRt2SRqqt*FBt/event_purity‚0 ,&Purity/4FRt4SRqqt2FBt/event_proportion‚0 9"Purity/*FRt*SRqqt1FBt/event_purity‚0.3965000808238983 7 Purity/*FRt3SRqqt1FBt/FRt_purity‚0.2888888716697693 :"Purity/2FRt*SRqqt0FBt/event_purity‚0.00136542075779289 9"Purity/2FRt1SRqqt2FBt/SRqqt_purity‚0.5964912176132202 & Purity/3FRt1SRqqt*FBt/FBt_purity‚1 & Purity/3FRt1SRqqt1FBt/FBt_purity‚1 ("Purity/3FRt2SRqqt0FBt/event_purity‚0 9"Purity/0FRt4SRqqt0FBt/SRqqt_purity‚0.0357142873108387 :"Purity/1FRt3SRqqt1FBt/SRqqt_purity‚0.19629628956317902 ,&Purity/1FRt4SRqqt1FBt/event_proportion‚0 ("Purity/2FRt2SRqqt1FBt/event_purity‚0 ,&Purity/3FRt0SRqqt4FBt/event_proportion‚0 ,&Purity/4FRt3SRqqt0FBt/event_proportion‚0 particle/accuracy_5_of_5‚1 jet/accuracy_3_of_7‚0 & Purity/3FRt*SRqqt2FBt/FBt_purity‚1 :"Purity/1FRt*SRqqt2FBt/SRqqt_purity‚0.45879417657852173 ,&Purity/3FRt*SRqqt3FBt/event_proportion‚0 9"Purity/2FRt3SRqqt0FBt/SRqqt_purity‚0.2354617565870285 & Purity/1FRt2SRqqt2FBt/FBt_purity‚1 ,&Purity/0FRt1SRqqt4FBt/event_proportion‚0 & Purity/2FRt2SRqqt*FBt/FBt_purity‚1 & Purity/2FRt2SRqqt1FBt/FBt_purity‚1 8 Purity/3FRt*SRqqt*FBt/FRt_purity‚0.14176055788993835 jet/accuracy_6_of_6‚0 9"Purity/3FRt*SRqqt0FBt/SRqqt_purity‚0.3791525065898895 particle/accuracy_3_of_7‚1 ) Purity/4FRt1SRqqt0FBt/FRt_purity‚0.25 :"Purity/*FRt1SRqqt*FBt/event_purity‚0.18009153008460999 :"Purity/1FRt*SRqqt*FBt/SRqqt_purity‚0.38307252526283264 ,&Purity/2FRt1SRqqt3FBt/event_proportion‚0 ?&Purity/3FRt*SRqqt*FBt/event_proportion‚0.009954636916518211 *particle/specificity‚0.942259669303894 jet/accuracy_4_of_6‚0 ("Purity/4FRt2SRqqt0FBt/event_purity‚0 8 Purity/1FRt*SRqqt*FBt/FRt_purity‚0.17014014720916748 ?&Purity/2FRt0SRqqt0FBt/event_proportion‚0.024571573361754417 & Purity/*FRt2SRqqt2FBt/FBt_purity‚1 A&Purity/0FRt*SRqqt4FBt/event_proportion‚0.00025201612152159214 >&Purity/2FRt1SRqqt*FBt/event_proportion‚0.04158266261219978 jet/accuracy_7_of_7‚0 * Purity/1FRt*SRqqt4FBt/FBt_purity‚0.875 ("Purity/1FRt4SRqqt0FBt/event_purity‚0 & Purity/1FRt1SRqqt4FBt/FRt_purity‚1 ,&Purity/2FRt4SRqqt2FBt/event_proportion‚0 ("Purity/3FRt0SRqqt0FBt/event_purity‚0 ,&Purity/3FRt0SRqqt3FBt/event_proportion‚0 *jet/accuracy_1_of_3‚0.7843191623687744 ,jet/accuracy_3_of_3‚0.022122491151094437 ("Purity/3FRt2SRqqt1FBt/event_purity‚0 & Purity/*FRt1SRqqt4FBt/FRt_purity‚1 ("Purity/4FRt0SRqqt*FBt/event_purity‚0 >&Purity/*FRt*SRqqt1FBt/event_proportion‚0.27079132199287415 =&Purity/*FRt0SRqqt*FBt/event_proportion‚0.3558467626571655 @&Purity/*FRt2SRqqt3FBt/event_proportion‚6.300403038039804e-05 ("Purity/1FRt2SRqqt1FBt/event_purity‚0 8 Purity/2FRt*SRqqt0FBt/FRt_purity‚0.10540957003831863 @&Purity/3FRt3SRqqt*FBt/event_proportion‚0.0005670362734235823 ,&Purity/4FRt*SRqqt3FBt/event_proportion‚0 particle/accuracy_4_of_7‚1 7 Purity/*FRt2SRqqt*FBt/FRt_purity‚0.1517876237630844 >&Purity/0FRt0SRqqt*FBt/event_proportion‚0.11857359111309052 7 Purity/1FRt*SRqqt2FBt/FBt_purity‚0.9868507385253906 *jet/accuracy_1_of_2‚0.6850609183311462 8 Purity/2FRt4SRqqt*FBt/FRt_purity‚0.20000000298023224 ("Purity/*FRt*SRqqt4FBt/SRqqt_purity‚1 9"Purity/3FRt3SRqqt0FBt/SRqqt_purity‚0.3095238208770752 >&Purity/*FRt3SRqqt0FBt/event_proportion‚0.01827116869390011 9"Purity/0FRt*SRqqt2FBt/event_purity‚0.2762887179851532 8 Purity/1FRt0SRqqt0FBt/FRt_purity‚0.08189694583415985 >&Purity/2FRt*SRqqt0FBt/event_proportion‚0.07579384744167328 ) Purity/*FRt4SRqqt*FBt/FRt_purity‚0.25 ?&Purity/*FRt0SRqqt3FBt/event_proportion‚0.003717237850651145 A&Purity/0FRt0SRqqt4FBt/event_proportion‚0.00025201612152159214 7 Purity/1FRt1SRqqt1FBt/FRt_purity‚0.2558540999889374 ("Purity/0FRt3SRqqt*FBt/event_purity‚0 *jet/accuracy_2_of_6‚0.2982456386089325 jet/accuracy_5_of_6‚0 7 Purity/1FRt3SRqqt*FBt/FRt_purity‚0.1823604851961136 ,&Purity/1FRt3SRqqt3FBt/event_proportion‚0 ("Purity/2FRt2SRqqt0FBt/event_purity‚0 8 Purity/2FRt2SRqqt0FBt/FRt_purity‚0.10309895128011703 7 Purity/3FRt0SRqqt1FBt/FRt_purity‚0.3333333134651184 ,&Purity/4FRt*SRqqt2FBt/event_proportion‚0 ,&Purity/4FRt4SRqqt*FBt/event_proportion‚0 jet/accuracy_4_of_7‚0 :"Purity/*FRt0SRqqt3FBt/event_purity‚0.13809522986412048 ?&Purity/0FRt3SRqqt*FBt/event_proportion‚0.006741431541740894 @&Purity/0FRt3SRqqt1FBt/event_proportion‚0.0006300403038039804 :"Purity/1FRt1SRqqt*FBt/event_purity‚0.01434949692338705 @&Purity/4FRt1SRqqt0FBt/event_proportion‚6.300403038039804e-05 =&Purity/0FRt1SRqqt*FBt/event_proportion‚0.2305317521095276 9"Purity/1FRt*SRqqt1FBt/SRqqt_purity‚0.4222720265388489 >&Purity/2FRt*SRqqt*FBt/event_proportion‚0.10244455933570862 ?&Purity/2FRt1SRqqt2FBt/event_proportion‚0.001890120911411941 ,&Purity/2FRt3SRqqt3FBt/event_proportion‚0 @&Purity/3FRt0SRqqt*FBt/event_proportion‚0.0027091733645647764 ,&Purity/3FRt4SRqqt3FBt/event_proportion‚0 jet/accuracy_2_of_7‚0 7 Purity/*FRt*SRqqt*FBt/FBt_purity‚0.9926795959472656 7 Purity/*FRt*SRqqt1FBt/FBt_purity‚0.9992988109588623 :"Purity/*FRt*SRqqt3FBt/event_purity‚0.11034481972455978 9"Purity/1FRt1SRqqt1FBt/SRqqt_purity‚0.4900919497013092 @&Purity/1FRt3SRqqt2FBt/event_proportion‚6.300403038039804e-05 ("Purity/2FRt*SRqqt1FBt/event_purity‚0 9"Purity/2FRt1SRqqt1FBt/SRqqt_purity‚0.4854070544242859 particle/accuracy_1_of_4‚1 @&Purity/0FRt4SRqqt1FBt/event_proportion‚6.300403038039804e-05 >&Purity/1FRt0SRqqt*FBt/event_proportion‚0.19745463132858276 8 Purity/3FRt1SRqqt0FBt/FRt_purity‚0.07361111044883728 A&Purity/3FRt1SRqqt2FBt/event_proportion‚0.00025201612152159214 9"Purity/2FRt3SRqqt1FBt/SRqqt_purity‚0.2666666805744171 particle/accuracy_5_of_7‚1 ("Purity/3FRt1SRqqt2FBt/event_purity‚0 & Purity/1FRt3SRqqt2FBt/FBt_purity‚1 9"Purity/0FRt1SRqqt*FBt/event_purity‚0.3393997251987457 :"Purity/1FRt0SRqqt*FBt/event_purity‚0.13203281164169312 +jet/accuracy_2_of_4‚0.27048465609550476 & Purity/0FRt3SRqqt*FBt/FBt_purity‚1 & Purity/0FRt3SRqqt1FBt/FBt_purity‚1 7 Purity/*FRt1SRqqt1FBt/FRt_purity‚0.2701636552810669 :"Purity/*FRt3SRqqt1FBt/SRqqt_purity‚0.26719576120376587 7 Purity/0FRt0SRqqt*FBt/FBt_purity‚0.9908480644226074 & Purity/0FRt0SRqqt1FBt/FBt_purity‚1 9"Purity/0FRt1SRqqt1FBt/SRqqt_purity‚0.4340507388114929 :"Purity/1FRt3SRqqt*FBt/SRqqt_purity‚0.21030040085315704 @&Purity/1FRt3SRqqt1FBt/event_proportion‚0.0011340725468471646 ("Purity/4FRt1SRqqt*FBt/SRqqt_purity‚1 ,&Purity/2FRt3SRqqt4FBt/event_proportion‚0 ("Purity/*FRt3SRqqt0FBt/event_purity‚0 8 Purity/2FRt0SRqqt2FBt/FRt_purity‚0.25535714626312256 ("Purity/4FRt1SRqqt0FBt/SRqqt_purity‚1 :"Purity/*FRt1SRqqt1FBt/SRqqt_purity‚0.46214184165000916 ;"Purity/0FRt2SRqqt1FBt/event_purity‚0.053763438016176224 ,&Purity/0FRt2SRqqt4FBt/event_proportion‚0 7 Purity/1FRt0SRqqt*FBt/FBt_purity‚0.9936372637748718 & Purity/1FRt0SRqqt1FBt/FBt_purity‚1 =&Purity/1FRt1SRqqt0FBt/event_proportion‚0.1265120953321457 >&Purity/2FRt1SRqqt0FBt/event_proportion‚0.03181703761219978 @&Purity/2FRt4SRqqt0FBt/event_proportion‚0.0003150201519019902 9"Purity/1FRt*SRqqt0FBt/SRqqt_purity‚0.3689264953136444 7 Purity/1FRt*SRqqt3FBt/FBt_purity‚0.9479166865348816 ?&Purity/1FRt0SRqqt2FBt/event_proportion‚0.015120967291295528 ("Purity/3FRt1SRqqt0FBt/event_purity‚0 8 Purity/3FRt2SRqqt*FBt/FRt_purity‚0.09953704476356506 _runtime‚1070.992495061 8 Purity/*FRt0SRqqt1FBt/FRt_purity‚0.31116926670074463 @&Purity/*FRt3SRqqt2FBt/event_proportion‚6.300403038039804e-05 ,&Purity/*FRt3SRqqt4FBt/event_proportion‚0 7 Purity/2FRt*SRqqt*FBt/FBt_purity‚0.9961161017417908 & Purity/2FRt*SRqqt1FBt/FBt_purity‚1 particle/accuracy_2_of_7‚1 ("Purity/2FRt*SRqqt3FBt/event_purity‚0 ( Purity/2FRt0SRqqt3FBt/FRt_purity‚0.5 9"Purity/*FRt2SRqqt0FBt/SRqqt_purity‚0.3059871792793274 9"Purity/0FRt*SRqqt1FBt/event_purity‚0.6276405453681946 particle/accuracy_2_of_6‚1 particle/accuracy_5_of_6‚1 & Purity/0FRt2SRqqt*FBt/FBt_purity‚1 & Purity/0FRt2SRqqt1FBt/FBt_purity‚1 8 Purity/2FRt1SRqqt2FBt/FRt_purity‚0.25219300389289856 *jet/accuracy_1_of_5‚0.8447754979133606 :"Purity/0FRt*SRqqt*FBt/event_purity‚0.43237245082855225 >&Purity/0FRt0SRqqt0FBt/event_proportion‚0.04082661122083664 :"Purity/1FRt1SRqqt2FBt/event_purity‚0.01785714365541935 & Purity/*FRt2SRqqt3FBt/FBt_purity‚1 ,&Purity/2FRt4SRqqt4FBt/event_proportion‚0 particle/accuracy_1_of_5‚1 ("Purity/*FRt1SRqqt4FBt/SRqqt_purity‚1 @&Purity/*FRt0SRqqt4FBt/event_proportion‚0.0003150201519019902 7 Purity/0FRt0SRqqt3FBt/FBt_purity‚0.9244444966316223 ?&Purity/0FRt3SRqqt0FBt/event_proportion‚0.006111391354352236 8 Purity/1FRt3SRqqt0FBt/FRt_purity‚0.16221198439598083 8 Purity/2FRt2SRqqt*FBt/FRt_purity‚0.12955538928508759 & Purity/3FRt*SRqqt*FBt/FBt_purity‚1 & Purity/3FRt*SRqqt1FBt/FBt_purity‚1 & Purity/3FRt1SRqqt2FBt/FBt_purity‚1 ,&Purity/0FRt4SRqqt4FBt/event_proportion‚0 ("Purity/2FRt1SRqqt0FBt/event_purity‚0 ,&Purity/3FRt3SRqqt2FBt/event_proportion‚0 ("Purity/4FRt*SRqqt0FBt/event_purity‚0 ,&Purity/*FRt3SRqqt3FBt/event_proportion‚0 ("Purity/2FRt0SRqqt2FBt/event_purity‚0 ("Purity/3FRt0SRqqt1FBt/event_purity‚0 *"Purity/0FRt1SRqqt3FBt/SRqqt_purity‚0.5 7 Purity/*FRt*SRqqt3FBt/FBt_purity‚0.9224138855934143¶Ò_ÝŸ ƒ¤ ("Purity/3FRt2SRqqt0FBt/event_purity‚0 9"Purity/0FRt4SRqqt0FBt/SRqqt_purity‚0.0357142873108387 :"Purity/1FRt3SRqqt1FBt/SRqqt_purity‚0.19629628956317902 ,&Purity/1FRt4SRqqt1FBt/event_proportion‚0 ("Purity/2FRt2SRqqt1FBt/event_purity‚0 ,&Purity/3FRt0SRqqt4FBt/event_proportion‚0 ,&Purity/4FRt3SRqqt0FBt/event_proportion‚0 particle/accuracy_5_of_5‚1 jet/accuracy_3_of_7‚0 & Purity/3FRt*SRqqt2FBt/FBt_purity‚1 :"Purity/1FRt*SRqqt2FBt/SRqqt_purity‚0.45879417657852173 ,&Purity/3FRt*SRqqt3FBt/event_proportion‚0 9"Purity/2FRt3SRqqt0FBt/SRqqt_purity‚0.2354617565870285 & Purity/1FRt2SRqqt2FBt/FBt_purity‚1 ,&Purity/0FRt1SRqqt4FBt/event_proportion‚0 & Purity/2FRt2SRqqt*FBt/FBt_purity‚1 & Purity/2FRt2SRqqt1FBt/FBt_purity‚1 8 Purity/3FRt*SRqqt*FBt/FRt_purity‚0.14176055788993835 jet/accuracy_6_of_6‚0 9"Purity/3FRt*SRqqt0FBt/SRqqt_purity‚0.3791525065898895 particle/accuracy_3_of_7‚1 ) Purity/4FRt1SRqqt0FBt/FRt_purity‚0.25 :"Purity/*FRt1SRqqt*FBt/event_purity‚0.18009153008460999 :"Purity/1FRt*SRqqt*FBt/SRqqt_purity‚0.38307252526283264 ,&Purity/2FRt1SRqqt3FBt/event_proportion‚0 ?&Purity/3FRt*SRqqt*FBt/event_proportion‚0.009954636916518211 *particle/specificity‚0.942259669303894 jet/accuracy_4_of_6‚0 ("Purity/4FRt2SRqqt0FBt/event_purity‚0 8 Purity/1FRt*SRqqt*FBt/FRt_purity‚0.17014014720916748 ?&Purity/2FRt0SRqqt0FBt/event_proportion‚0.024571573361754417 & Purity/*FRt2SRqqt2FBt/FBt_purity‚1 A&Purity/0FRt*SRqqt4FBt/event_proportion‚0.00025201612152159214 >&Purity/2FRt1SRqqt*FBt/event_proportion‚0.04158266261219978 jet/accuracy_7_of_7‚0 * Purity/1FRt*SRqqt4FBt/FBt_purity‚0.875 ("Purity/1FRt4SRqqt0FBt/event_purity‚0 & Purity/1FRt1SRqqt4FBt/FRt_purity‚1 ,&Purity/2FRt4SRqqt2FBt/event_proportion‚0 ("Purity/3FRt0SRqqt0FBt/event_purity‚0 ,&Purity/3FRt0SRqqt3FBt/event_proportion‚0 *jet/accuracy_1_of_3‚0.7843191623687744 ,jet/accuracy_3_of_3‚0.022122491151094437 ("Purity/3FRt2SRqqt1FBt/event_purity‚0 & Purity/*FRt1SRqqt4FBt/FRt_purity‚1 ("Purity/4FRt0SRqqt*FBt/event_purity‚0 >&Purity/*FRt*SRqqt1FBt/event_proportion‚0.27079132199287415 =&Purity/*FRt0SRqqt*FBt/event_proportion‚0.3558467626571655 @&Purity/*FRt2SRqqt3FBt/event_proportion‚6.300403038039804e-05 ("Purity/1FRt2SRqqt1FBt/event_purity‚0 8 Purity/2FRt*SRqqt0FBt/FRt_purity‚0.10540957003831863 @&Purity/3FRt3SRqqt*FBt/event_proportion‚0.0005670362734235823 ,&Purity/4FRt*SRqqt3FBt/event_proportion‚0 particle/accuracy_4_of_7‚1 7 Purity/*FRt2SRqqt*FBt/FRt_purity‚0.1517876237630844 >&Purity/0FRt0SRqqt*FBt/event_proportion‚0.11857359111309052 7 Purity/1FRt*SRqqt2FBt/FBt_purity‚0.9868507385253906 *jet/accuracy_1_of_2‚0.6850609183311462 8 Purity/2FRt4SRqqt*FBt/FRt_purity‚0.20000000298023224 ("Purity/*FRt*SRqqt4FBt/SRqqt_purity‚1 9"Purity/3FRt3SRqqt0FBt/SRqqt_purity‚0.3095238208770752 >&Purity/*FRt3SRqqt0FBt/event_proportion‚0.01827116869390011 9"Purity/0FRt*SRqqt2FBt/event_purity‚0.2762887179851532 8 Purity/1FRt0SRqqt0FBt/FRt_purity‚0.08189694583415985 >&Purity/2FRt*SRqqt0FBt/event_proportion‚0.07579384744167328 ) Purity/*FRt4SRqqt*FBt/FRt_purity‚0.25 ?&Purity/*FRt0SRqqt3FBt/event_proportion‚0.003717237850651145 A&Purity/0FRt0SRqqt4FBt/event_proportion‚0.00025201612152159214 7 Purity/1FRt1SRqqt1FBt/FRt_purity‚0.2558540999889374 ("Purity/0FRt3SRqqt*FBt/event_purity‚0 *jet/accuracy_2_of_6‚0.2982456386089325 jet/accuracy_5_of_6‚0 7 Purity/1FRt3SRqqt*FBt/FRt_purity‚0.1823604851961136 ,&Purity/1FRt3SRqqt3FBt/event_proportion‚0 ("Purity/2FRt2SRqqt0FBt/event_purity‚0 8 Purity/2FRt2SRqqt0FBt/FRt_purity‚0.10309895128011703 7 Purity/3FRt0SRqqt1FBt/FRt_purity‚0.3333333134651184 ,&Purity/4FRt*SRqqt2FBt/event_proportion‚0 ,&Purity/4FRt4SRqqt*FBt/event_proportion‚0 jet/accuracy_4_of_7‚0 :"Purity/*FRt0SRqqt3FBt/event_purity‚0.13809522986412048 ?&Purity/0FRt3SRqqt*FBt/event_proportion‚0.006741431541740894 @&Purity/0FRt3SRqqt1FBt/event_proportion‚0.0006300403038039804 :"Purity/1FRt1SRqqt*FBt/event_purity‚0.01434949692338705 @&Purity/4FRt1SRqqt0FBt/event_proportion‚6.300403038039804e-05 =&Purity/0FRt1SRqqt*FBt/event_proportion‚0.2305317521095276 9"Purity/1FRt*SRqqt1FBt/SRqqt_purity‚0.4222720265388489 >&Purity/2FRt*SRqqt*FBt/event_proportion‚0.10244455933570862 ?&Purity/2FRt1SRqqt2FBt/event_proportion‚0.001890120911411941 ,&Purity/2FRt3SRqqt3FBt/event_proportion‚0 @&Purity/3FRt0SRqqt*FBt/event_proportion‚0.0027091733645647764 ,&Purity/3FRt4SRqqt3FBt/event_proportion‚0 jet/accuracy_2_of_7‚0 7 Purity/*FRt*SRqqt*FBt/FBt_purity‚0.9926795959472656 7 Purity/*FRt*SRqqt1FBt/FBt_purity‚0.9992988109588623 :"Purity/*FRt*SRqqt3FBt/event_purity‚0.11034481972455978 9"Purity/1FRt1SRqqt1FBt/SRqqt_purity‚0.4900919497013092 @&Purity/1FRt3SRqqt2FBt/event_proportion‚6.300403038039804e-05 ("Purity/2FRt*SRqqt1FBt/event_purity‚0 9"Purity/2FRt1SRqqt1FBt/SRqqt_purity‚0.4854070544242859 particle/accuracy_1_of_4‚1 @&Purity/0FRt4SRqqt1FBt/event_proportion‚6.300403038039804e-05 >&Purity/1FRt0SRqqt*FBt/event_proportion‚0.19745463132858276 8 Purity/3FRt1SRqqt0FBt/FRt_purity‚0.07361111044883728 A&Purity/3FRt1SRÿìЙH}qqt2FBt/event_proportion‚0.00025201612152159214 9"Purity/2FRt3SRqqt1FBt/SRqqt_purity‚0.2666666805744171 particle/accuracy_5_of_7‚1 ("Purity/3FRt1SRqqt2FBt/event_purity‚0 & Purity/1FRt3SRqqt2FBt/FBt_purity‚1 9"Purity/0FRt1SRqqt*FBt/event_purity‚0.3393997251987457 :"Purity/1FRt0SRqqt*FBt/event_purity‚0.13203281164169312 +jet/accuracy_2_of_4‚0.27048465609550476 & Purity/0FRt3SRqqt*FBt/FBt_purity‚1 & Purity/0FRt3SRqqt1FBt/FBt_purity‚1 7 Purity/*FRt1SRqqt1FBt/FRt_purity‚0.2701636552810669 :"Purity/*FRt3SRqqt1FBt/SRqqt_purity‚0.26719576120376587 7 Purity/0FRt0SRqqt*FBt/FBt_purity‚0.9908480644226074 & Purity/0FRt0SRqqt1FBt/FBt_purity‚1 9"Purity/0FRt1SRqqt1FBt/SRqqt_purity‚0.4340507388114929 :"Purity/1FRt3SRqqt*FBt/SRqqt_purity‚0.21030040085315704 @&Purity/1FRt3SRqqt1FBt/event_proportion‚0.0011340725468471646 ("Purity/4FRt1SRqqt*FBt/SRqqt_purity‚1 ,&Purity/2FRt3SRqqt4FBt/event_proportion‚0 ("Purity/*FRt3SRqqt0FBt/event_purity‚0 8 Purity/2FRt0SRqqt2FBt/FRt_purity‚0.25535714626312256 ("Purity/4FRt1SRqqt0FBt/SRqqt_purity‚1 :"Purity/*FRt1SRqqt1FBt/SRqqt_purity‚0.46214184165000916 ;"Purity/0FRt2SRqqt1FBt/event_purity‚0.053763438016176224 ,&Purity/0FRt2SRqqt4FBt/event_proportion‚0 7 Purity/1FRt0SRqqt*FBt/FBt_purity‚0.9936372637748718 & Purity/1FRt0SRqqt1FBt/FBt_purity‚1 =&Purity/1FRt1SRqqt0FBt/event_proportion‚0.1265120953321457 >&Purity/2FRt1SRqqt0FBt/event_proportion‚0.03181703761219978 @&Purity/2FRt4SRqqt0FBt/event_proportion‚0.0003150201519019902 9"Purity/1FRt*SRqqt0FBt/SRqqt_purity‚0.3689264953136444 7 Purity/1FRt*SRqqt3FBt/FBt_purity‚0.9479166865348816 ?&Purity/1FRt0SRqqt2FBt/event_proportion‚0.015120967291295528 ("Purity/3FRt1SRqqt0FBt/event_purity‚0 8 Purity/3FRt2SRqqt*FBt/FRt_purity‚0.09953704476356506 _runtime‚1070.992495061 8 Purity/*FRt0SRqqt1FBt/FRt_purity‚0.31116926670074463 @&Purity/*FRt3SRqqt2FBt/event_proportion‚6.300403038039804e-05 ,&Purity/*FRt3SRqqt4FBt/event_proportion‚0 7 Purity/2FRt*SRqqt*FBt/FBt_purity‚0.9961161017417908 & Purity/2FRt*SRqqt1FBt/FBt_purity‚1 particle/accuracy_2_of_7‚1 ("Purity/2FRt*SRqqt3FBt/event_purity‚0 ( Purity/2FRt0SRqqt3FBt/FRt_purity‚0.5 9"Purity/*FRt2SRqqt0FBt/SRqqt_purity‚0.3059871792793274 9"Purity/0FRt*SRqqt1FBt/event_purity‚0.6276405453681946 particle/accuracy_2_of_6‚1 particle/accuracy_5_of_6‚1 & Purity/0FRt2SRqqt*FBt/FBt_purity‚1 & Purity/0FRt2SRqqt1FBt/FBt_purity‚1 8 Purity/2FRt1SRqqt2FBt/FRt_purity‚0.25219300389289856 *jet/accuracy_1_of_5‚0.8447754979133606 :"Purity/0FRt*SRqqt*FBt/event_purity‚0.43237245082855225 >&Purity/0FRt0SRqqt0FBt/event_proportion‚0.04082661122083664 :"Purity/1FRt1SRqqt2FBt/event_purity‚0.01785714365541935 & Purity/*FRt2SRqqt3FBt/FBt_purity‚1 ,&Purity/2FRt4SRqqt4FBt/event_proportion‚0 particle/accuracy_1_of_5‚1 ("Purity/*FRt1SRqqt4FBt/SRqqt_purity‚1 @&Purity/*FRt0SRqqt4FBt/event_proportion‚0.0003150201519019902 7 Purity/0FRt0SRqqt3FBt/FBt_purity‚0.9244444966316223 ?&Purity/0FRt3SRqqt0FBt/event_proportion‚0.006111391354352236 8 Purity/1FRt3SRqqt0FBt/FRt_purity‚0.16221198439598083 8 Purity/2FRt2SRqqt*FBt/FRt_purity‚0.12955538928508759 & Purity/3FRt*SRqqt*FBt/FBt_purity‚1 & Purity/3FRt*SRqqt1FBt/FBt_purity‚1 & Purity/3FRt1SRqqt2FBt/FBt_purity‚1 ,&Purity/0FRt4SRqqt4FBt/event_proportion‚0 ("Purity/2FRt1SRqqt0FBt/event_purity‚0 ,&Purity/3FRt3SRqqt2FBt/event_proportion‚0 ("Purity/4FRt*SRqqt0FBt/event_purity‚0 ,&Purity/*FRt3SRqqt3FBt/event_proportion‚0 ("Purity/2FRt0SRqqt2FBt/event_purity‚0 ("Purity/3FRt0SRqqt1FBt/event_purity‚0 *"Purity/0FRt1SRqqt3FBt/SRqqt_purity‚0.5 7 Purity/*FRt*SRqqt3FBt/FBt_purity‚0.9224138855934143 >&Purity/*FRt*SRqqt2FBt/event_proportion‚0.06747731566429138 >&Purity/*FRt0SRqqt0FBt/event_proportion‚0.15757308900356293 @&Purity/*FRt1SRqqt3FBt/event_proportion‚0.0006930443341843784 >&Purity/0FRt0SRqqt1FBt/event_proportion‚0.08606351166963577 9"Purity/0FRt0SRqqt3FBt/event_purity‚0.1599999964237213 :"Purity/1FRt3SRqqt0FBt/SRqqt_purity‚0.22186385095119476 ("Purity/0FRt*SRqqt4FBt/event_purity‚0 =&Purity/*FRt1SRqqt1FBt/event_proportion‚0.1001134067773819 >&Purity/*FRt2SRqqt1FBt/event_proportion‚0.02255544438958168 8 Purity/3FRt1SRqqt*FBt/FRt_purity‚0.16152264177799225 ("Purity/2FRt4SRqqt*FBt/event_purity‚0 ("Purity/2FRt0SRqqt3FBt/event_purity‚0 & Purity/2FRt2SRqqt2FBt/FBt_purity‚1 & Purity/1FRt2SRqqt*FBt/FBt_purity‚1 & Purity/1FRt2SRqqt1FBt/FBt_purity‚1 @&Purity/3FRt0SRqqt0FBt/event_proportion‚0.0015120967291295528 ?&Purity/*FRt3SRqqt1FBt/event_proportion‚0.002142137149348855 ("Purity/0FRt2SRqqt2FBt/event_purity‚0 ("Purity/3FRt*SRqqt2FBt/event_purity‚0 @&Purity/2FRt*SRqqt3FBt/event_proportion‚6.300403038039804e-05 7validation_average_jet_accuracy‚0.38774991035461426 +jet/accuracy_3_of_6‚0.05263157933950424 8 Purity/2FRt3SRqqt1FBt/FRt_purity‚0.15000000596046448 :"Purity/0FRt*SRqqt0FBt/event_purity‚0.33333590626716614 ("Purity/0FRt0SRqqt1FBt/event_purity‚1 >&Purity/1FRt1SRqqt1FBt/event_proportion‚0.03944052383303642 ("Purity/2FRt1SRqqt*FBt/event_purity‚0 ?&Purity/3FRt1SRqqt*FBt/event_proportion‚0.003717237850651145 8 Purity/3FRt2SRqqt0FBt/FRt_purity‚0.07004830986261368 & Purity/*FRt4SRqqt*FBt/FBt_purity‚1 & Purity/*FRt4SRqqt1FBt/FBt_purity‚1 9"Purity/*FRt0SRqqt2FBt/event_purity‚0.2601272463798523 & Purity/*FRt2SRqqt*FBt/FBt_purity‚1 & Purity/*FRt2SRqqt1FBt/FBt_purity‚1 9"Purity/1FRt2SRqqt0FBt/SRqqt_purity‚0.3070523738861084 ,&Purity/1FRt4SRqqt3FBt/event_proportion‚0 ,&Purity/4FRt4SRqqt0FBt/event_proportion‚0 ("Purity/3FRt4SRqqt0FBt/SRqqt_purity‚0 9"Purity/0FRt3SRqqt1FBt/SRqqt_purity‚0.3333333134651184 :"Purity/0FRt2SRqqt0FBt/event_purity‚0.03554140031337738 ("Purity/3FRt1SRqqt1FBt/event_purity‚0 & Purity/0FRt2SRqqt3FBt/FBt_purity‚1 A&Purity/0FRt1SRqqt3FBt/event_proportion‚0.00044102821266278625 @&Purity/2FRt4SRqqt*FBt/event_proportion‚0.0003150201519019902 ("Purity/*FRt4SRqqt1FBt/event_purity‚0 ."Purity/0FRt4SRqqt*FBt/SRqqt_purity‚0.03125 ("Purity/1FRt2SRqqt2FBt/event_purity‚0 & Purity/*FRt3SRqqt2FBt/FRt_purity‚1 9"Purity/*FRt2SRqqt*FBt/SRqqt_purity‚0.3108208477497101 ,&Purity/4FRt1SRqqt2FBt/event_proportion‚0 :"Purity/2FRt4SRqqt0FBt/SRqqt_purity‚0.05000000074505806 ("Purity/1FRt4SRqqt*FBt/event_purity‚0 & Purity/2FRt*SRqqt3FBt/FBt_purity‚1 6 Purity/1FRt*SRqqt*FBt/FBt_purity‚0.994266927242279 & Purity/1FRt*SRqqt1FBt/FBt_purity‚1 ("Purity/1FRt2SRqqt*FBt/event_purity‚0 @&Purity/4FRt2SRqqt0FBt/event_proportion‚6.300403038039804e-05 particle/accuracy_4_of_5‚1 9"Purity/1FRt3SRqqt2FBt/SRqqt_purity‚0.3333333432674408 7 Purity/1FRt0SRqqt3FBt/FBt_purity‚0.9523809552192688 @&Purity/3FRt1SRqqt1FBt/event_proportion‚0.0009450604557059705 @&Purity/4FRt0SRqqt0FBt/event_proportion‚6.300403038039804e-05 ,&Purity/4FRt3SRqqt3FBt/event_proportion‚0 ("Purity/0FRt0SRqqt4FBt/event_purity‚0 *"Purity/2FRt2SRqqt2FBt/SRqqt_purity‚0.5 % _timestamp‚1.7885019949111621e+09 :"Purity/*FRt*SRqqt0FBt/event_purity‚0.13784803450107574 @&Purity/2FRt3SRqqt1FBt/event_proportion‚0.0003780241822823882 @&Purity/3FRt2SRqqt1FBt/event_proportion‚0.0003780241822823882 :"Purity/0FRt3SRqqt0FBt/SRqqt_purity‚0.22013887763023376 ("Purity/3FRt4SRqqt*FBt/event_purity‚0 ("Purity/1FRt1SRqqt4FBt/SRqqt_purity‚1 8 Purity/*FRt0SRqqt3FBt/FRt_purity‚0.30000001192092896 =&Purity/*FRt2SRqqt*FBt/event_proportion‚0.1332535296678543 >&Purity/0FRt*SRqqt2FBt/event_proportion‚0.03944052383303642 <"Purity/2FRt*SRqqt*FBt/event_purity‚0.0011243795743212104 8 Purity/2FRt0SRqqt1FBt/FRt_purity‚0.25723543763160706 0"Purity/*FRt4SRqqt0FBt/SRqqt_purity‚ 0.1015625 >&Purity/*FRt0SRqqt1FBt/event_proportion‚0.14591734111309052 particle/accuracy_1_of_2‚1 * Purity/0FRt0SRqqt4FBt/FBt_purity‚0.875 ("Purity/3FRt0SRqqt2FBt/event_purity‚0 @&Purity/3FRt*SRqqt2FBt/event_proportion‚0.0003150201519019902 * Purity/*FRt*SRqqt4FBt/FBt_purity‚0.875 9"Purity/3FRt3SRqqt*FBt/SRqqt_purity‚0.3095238208770752 7 Purity/*FRt*SRqqt*FBt/FRt_purity‚0.1630123406648636 7 Purity/*FRt0SRqqt2FBt/FRt_purity‚0.3921659588813782 ,&Purity/0FRt3SRqqt4FBt/event_proportion‚0 ,&Purity/0FRt4SRqqt3FBt/event_proportion‚0 6 Purity/1FRt2SRqqt0FBt/FRt_purity‚0.152561217546463 7 Purity/0FRt0SRqqt2FBt/FBt_purity‚0.9885537028312683 ,&Purity/2FRt2SRqqt4FBt/event_proportion‚0 ("Purity/4FRt*SRqqt*FBt/SRqqt_purity‚1 ("Purity/4FRt2SRqqt*FBt/SRqqt_purity‚1 _step‚48 9"Purity/*FRt1SRqqt0FBt/SRqqt_purity‚0.4326682388782501 @&Purity/1FRt4SRqqt*FBt/event_proportion‚0.0003780241822823882 7 Purity/2FRt1SRqqt1FBt/FRt_purity‚0.3018241226673126 ,&Purity/2FRt2SRqqt3FBt/event_proportion‚0 ,&Purity/4FRt3SRqqt2FBt/event_proportion‚0 ("Purity/1FRt*SRqqt4FBt/SRqqt_purity‚1 ("Purity/4FRt0SRqqt0FBt/event_purity‚0 9"Purity/0FRt1SRqqt0FBt/SRqqt_purity‚0.4214657247066498 & Purity/1FRt3SRqqt*FBt/FBt_purity‚1 & Purity/1FRt3SRqqt1FBt/FBt_purity‚1 8 Purity/3FRt*SRqqt0FBt/FRt_purity‚0.09428572654724121 ,&Purity/3FRt3SRqqt3FBt/event_proportion‚0 9"Purity/*FRt*SRqqt3FBt/SRqqt_purity‚0.5952380895614624 =&Purity/*FRt*SRqqt*FBt/event_proportion‚0.9591733813285828 7 Purity/*FRt*SRqqt2FBt/FBt_purity‚0.9873462319374084 ?&Purity/0FRt0SRqqt3FBt/event_proportion‚0.002583165420219302 ?&Purity/0FRt1SRqqt2FBt/event_proportion‚0.008757560513913631 ,&Purity/0FRt3SRqqt3FBt/event_proportion‚0 +jet/accuracy_2_of_3‚0.21224664151668549 particle/accuracy_2_of_3‚1 8 Purity/1FRt1SRqqt2FBt/FRt_purity‚0.33877548575401306 A&Purity/1FRt1SRqqt3FBt/event_proportion‚0.00025201612152159214 7 Purity/2FRt*SRqqt2FBt/FBt_purity‚0.9911110997200012 *"Purity/3FRt*SRqqt1FBt/SRqqt_purity‚0.5 @&Purity/*FRt4SRqqt*FBt/event_proportion‚0.0013230846961960196 ;"Purity/0FRt2SRqqt*FBt/event_purity‚0.033947091549634933 ("Purity/0FRt4SRqqt1FBt/SRqqt_purity‚0 & Purity/3FRt0SRqqt2FBt/FRt_purity‚0 ) Purity/4FRt1SRqqt*FBt/FRt_purity‚0.25 7 Purity/1FRt0SRqqt2FBt/FBt_purity‚0.9862310886383057 >&Purity/*FRt1SRqqt0FBt/event_proportion‚0.33039313554763794 particle/accuracy_6_of_7‚1 8 Purity/1FRt*SRqqt0FBt/FRt_purity‚0.11462289839982986 ("Purity/*FRt2SRqqt2FBt/event_purity‚0 & Purity/1FRt1SRqqt3FBt/FRt_purity‚0 ,&Purity/*FRt4SRqqt2FBt/event_proportion‚0 7 Purity/2FRt*SRqqt*FBt/FRt_purity‚0.1508835405111313 ?&Purity/2FRt2SRqqt0FBt/event_proportion‚0.015435988083481789 ("Purity/2FRt3SRqqt*FBt/event_purity‚0 :"Purity/3FRt1SRqqt*FBt/SRqqt_purity‚0.48148152232170105 ("Purity/3FRt2SRqqt*FBt/event_purity‚0 & Purity/*FRt1SRqqt3FBt/FRt_purity‚0 @&Purity/*FRt4SRqqt1FBt/event_proportion‚6.300403038039804e-05 ;"Purity/0FRt1SRqqt2FBt/event_purity‚0.025317460298538208 =&Purity/1FRt*SRqqt*FBt/event_proportion‚0.4366179406642914 ?&Purity/2FRt2SRqqt1FBt/event_proportion‚0.003969253972172737 ("Purity/3FRt*SRqqt*FBt/event_purity‚0 @&Purity/3FRt1SRqqt0FBt/event_proportion‚0.0025201612152159214 particle/accuracy_2_of_5‚1 particle/accuracy_7_of_7‚1 8 Purity/1FRt0SRqqt*FBt/FRt_purity‚0.17214414477348328 9"Purity/0FRt0SRqqt*FBt/event_purity‚0.8201134204864502 ,&Purity/0FRt4SRqqt2FBt/event_proportion‚0 ?&Purity/1FRt1SRqqt2FBt/event_proportion‚0.006111391354352236 ,&Purity/2FRt4SRqqt1FBt/event_proportion‚0 ,&Purity/3FRt3SRqqt4FBt/event_proportion‚0 ) Purity/*FRt4SRqqt0FBt/FRt_purity‚0.25 & Purity/3FRt2SRqqt*FBt/FBt_purity‚1 9"Purity/*FRt*SRqqt*FBt/SRqqt_purity‚0.3787279725074768 ;"Purity/*FRt1SRqqt2FBt/event_purity‚0.020707787945866585 ?&Purity/*FRt1SRqqt2FBt/event_proportion‚0.017011089250445366 7 Purity/*FRt1SRqqt2FBt/FRt_purity‚0.2981066405773163 8 Purity/*FRt2SRqqt0FBt/FRt_purity‚0.12830613553524017 ("Purity/2FRt*SRqqt2FBt/event_purity‚0 ,&Purity/4FRt2SRqqt4FBt/event_proportion‚0 *jet/accuracy_1_of_4‚0.8258363604545593 @&Purity/*FRt*SRqqt4FBt/event_proportion‚0.0003780241822823882 7 Purity/*FRt1SRqqt2FBt/FBt_purity‚0.9832815527915955 ("Purity/2FRt0SRqqt1FBt/event_purity‚0 ,&Purity/4FRt*SRqqt4FBt/event_proportion‚0 ."Purity/*FRt4SRqqt*FBt/SRqqt_purity‚0.09375 8 Purity/2FRt4SRqqt0FBt/FRt_purity‚0.20000000298023224 & Purity/1FRt0SRqqt4FBt/FBt_purity‚1 trainer/global_step‚603 7 Purity/2FRt1SRqqt*FBt/FBt_purity‚0.9964157938957214 & Purity/2FRt1SRqqt1FBt/FBt_purity‚1 ,&Purity/4FRt0SRqqt4FBt/event_proportion‚0 & Purity/3FRt2SRqqt1FBt/FBt_purity‚1 & Purity/0FRt2SRqqt2FBt/FBt_purity‚1 jet/accuracy_5_of_7‚0 +"Purity/1FRt1SRqqt3FBt/SRqqt_purity‚0.75 9"Purity/0FRt*SRqqt1FBt/SRqqt_purity‚0.4040747582912445 >&Purity/0FRt1SRqqt1FBt/event_proportion‚0.05185231938958168 8 Purity/1FRt3SRqqt1FBt/FRt_purity‚0.30000001192092896 8 Purity/3FRt0SRqqt*FBt/FRt_purity‚0.15656568109989166 ,&Purity/3FRt4SRqqt4FBt/event_proportion‚0 * Purity/0FRt*SRqqt4FBt/FBt_purity‚0.875 & Purity/0FRt4SRqqt*FBt/FBt_purity‚1 & Purity/0FRt4SRqqt1FBt/FBt_purity‚1 :"Purity/0FRt0SRqqt2FBt/event_purity‚0.35592129826545715 8 Purity/1FRt1SRqqt*FBt/FRt_purity‚0.16569927334785461 ?&Purity/1FRt3SRqqt*FBt/event_proportion‚0.009135584346950054 particle/accuracy_2_of_4‚1 & Purity/1FRt*SRqqt4FBt/FRt_purity‚1 ) Purity/1FRt1SRqqt4FBt/FBt_purity‚0.75 7 Purity/3FRt3SRqqt0FBt/FRt_purity‚0.1190476268529892 ("Purity/4FRt*SRqqt*FBt/event_purity‚0 8 Purity/*FRt1SRqqt*FBt/FRt_purity‚0.16205354034900665 ,&Purity/1FRt4SRqqt2FBt/event_proportion‚0 7 Purity/2FRt0SRqqt*FBt/FBt_purity‚0.9967741966247559 & Purity/2FRt0SRqqt1FBt/FBt_purity‚1 -jet/accuracy_4_of_5‚0.0021505376789718866 +"Purity/3FRt1SRqqt2FBt/SRqqt_purity‚0.25 ( Purity/2FRt*SRqqt3FBt/FRt_purity‚0.5 ("Purity/4FRt2SRqqt*FBt/event_purity‚0 ,&Purity/0FRt3SRqqt2FBt/event_proportion‚0 8 Purity/1FRt0SRqqt2FBt/FRt_purity‚0.45895007252693176 @&Purity/1FRt2SRqqt2FBt/event_proportion‚0.0010080644860863686 8 Purity/2FRt3SRqqt0FBt/FRt_purity‚0.10005411505699158 ,&Purity/3FRt*SRqqt4FBt/event_proportion‚0 ,&Purity/4FRt0SRqqt1FBt/event_proportion‚0 ) Purity/*FRt1SRqqt4FBt/FBt_purity‚0.75 ("Purity/3FRt3SRqqt*FBt/event_purity‚0 7 Purity/1FRt0SRqqt3FBt/FRt_purity‚0.2857142984867096 & Purity/3FRt0SRqqt2FBt/FBt_purity‚1 ("Purity/*FRt2SRqqt3FBt/event_purity‚0 :"Purity/*FRt1SRqqt1FBt/event_purity‚0.07901166379451752 7 Purity/0FRt*SRqqt2FBt/FBt_purity‚0.9866310358047485 9"Purity/0FRt2SRqqt1FBt/SRqqt_purity‚0.3831029236316681 7 Purity/1FRt1SRqqt2FBt/FBt_purity‚0.9836309552192688 :"Purity/2FRt*SRqqt*FBt/SRqqt_purity‚0.37650057673454285 7 Purity/2FRt*SRqqt2FBt/FRt_purity‚0.2499999850988388 ,&Purity/4FRt2SRqqt1FBt/event_proportion‚0 :"Purity/*FRt*SRqqt*FBt/event_purity‚0.21480116248130798 7 Purity/*FRt*SRqqt2FBt/FRt_purity‚0.3559475839138031 8 Purity/*FRt0SRqqt*FBt/FRt_purity‚0.16779378056526184 :"Purity/*FRt1SRqqt0FBt/event_purity‚0.21873195469379425 7 Purity/0FRt1SRqqt2FBt/FBt_purity‚0.9829629063606262 ?&Purity/0FRt2SRqqt0FBt/event_proportion‚0.043283771723508835 A&Purity/0FRt4SRqqt0FBt/event_proportion‚0.00044102821266278625 jet/accuracy_5_of_5‚0 ("Purity/*FRt3SRqqt1FBt/event_purity‚0 ("Purity/0FRt0SRqqt0FBt/event_purity‚1 9"Purity/0FRt1SRqqt1FBt/event_purity‚0.1262608766555786 ("Purity/3FRt0SRqqt*FBt/event_purity‚0 ,&Purity/3FRt4SRqqt1FBt/event_proportion‚0 ,&Purity/4FRt0SRqqt2FBt/event_proportion‚0 *jet/accuracy_2_of_5‚0.3725011944770813 7 Purity/*FRt0SRqqt4FBt/FBt_purity‚0.8999999761581421 :"Purity/*FRt0SRqqt0FBt/event_purity‚0.06902338564395905 ,&Purity/*FRt2SRqqt4FBt/event_proportion‚0 -"Purity/1FRt*SRqqt3FBt/event_purity‚0.0625 :"Purity/1FRt1SRqqt1FBt/event_purity‚0.03787361457943916 9"Purity/2FRt*SRqqt1FBt/SRqqt_purity‚0.4233380854129791 ("Purity/2FRt1SRqqt1FBt/event_purity‚0 :"Purity/3FRt*SRqqt*FBt/SRqqt_purity‚0.39738646149635315 7 Purity/3FRt2SRqqt1FBt/FRt_purity‚0.2222222238779068 7 Purity/*FRt*SRqqt3FBt/FRt_purity‚0.2647058963775635 9"Purity/0FRt1SRqqt0FBt/event_purity‚0.4214657247066498 :"Purity/2FRt3SRqqt*FBt/SRqqt_purity‚0.25685715675354004 ,&Purity/2FRt3SRqqt2FBt/event_proportion‚0 ("Purity/3FRt1SRqqt*FBt/event_purity‚0 9"Purity/3FRt2SRqqt*FBt/SRqqt_purity‚0.2986111044883728 ,&Purity/3FRt2SRqqt2FBt/event_proportion‚0 particle/accuracy_4_of_4‚1 7 Purity/*FRt0SRqqt2FBt/FBt_purity‚0.9885528087615967 (particle/f_score‚0.39225271344184875 ("Purity/2FRt3SRqqt1FBt/event_purity‚0 ?&Purity/*FRt3SRqqt*FBt/event_proportion‚0.020476309582591057 9"Purity/1FRt*SRqqt1FBt/event_purity‚0.1456620991230011 @&Purity/3FRt3SRqqt0FBt/event_proportion‚0.0005670362734235823 ("Purity/*FRt4SRqqt0FBt/event_purity‚0 :"Purity/0FRt2SRqqt*FBt/SRqqt_purity‚0.30127444863319397 ,&Purity/1FRt2SRqqt4FBt/event_proportion‚0 ?&Purity/2FRt3SRqqt0FBt/event_proportion‚0.003654233878478408 @&Purity/4FRt0SRqqt*FBt/event_proportion‚6.300403038039804e-05 ("Purity/*FRt*SRqqt4FBt/event_purity‚0 ("Purity/*FRt4SRqqt*FBt/event_purity‚0 ("Purity/*FRt1SRqqt3FBt/event_purity‚0 :"Purity/1FRt0SRqqt1FBt/event_purity‚0.26097288727760315 9"Purity/1FRt1SRqqt2FBt/SRqqt_purity‚0.5383502840995789 7 Purity/1FRt4SRqqt0FBt/FRt_purity‚0.3333333432674408 9"Purity/*FRt*SRqqt2FBt/event_purity‚0.1929907351732254 8 Purity/*FRt3SRqqt*FBt/FRt_purity‚0.14518451690673828 ("Purity/*FRt3SRqqt2FBt/event_purity‚0 ,&Purity/3FRt2SRqqt4FBt/event_proportion‚0 :"Purity/1FRt0SRqqt0FBt/event_purity‚0.08189694583415985 >&Purity/1FRt0SRqqt0FBt/event_proportion‚0.13142640888690948 8 Purity/1FRt2SRqqt1FBt/FRt_purity‚0.26075270771980286 ,&Purity/4FRt1SRqqt4FBt/event_proportion‚0 ("Purity/2FRt4SRqqt0FBt/event_purity‚0 7 Purity/3FRt4SRqqt*FBt/FRt_purity‚0.3333333432674408 & Purity/4FRt2SRqqt0FBt/FRt_purity‚0 9"Purity/*FRt*SRqqt0FBt/SRqqt_purity‚0.3667586147785187 ?&Purity/1FRt0SRqqt1FBt/event_proportion‚0.049773186445236206 <"Purity/2FRt0SRqqt*FBt/event_purity‚0.0032583903521299362 8 Purity/2FRt0SRqqt0FBt/FRt_purity‚0.09273655712604523 ("Purity/2FRt1SRqqt2FBt/event_purity‚0 particle/accuracy_3_of_4‚1 ("Purity/0FRt3SRqqt0FBt/event_purity‚0 9"Purity/3FRt2SRqqt1FBt/SRqqt_purity‚0.0833333358168602 7 Purity/0FRt*SRqqt3FBt/FBt_purity‚0.9176955819129944 ?&Purity/1FRt2SRqqt1FBt/event_proportion‚0.008946572430431843 9"Purity/1FRt2SRqqt1FBt/SRqqt_purity‚0.3076421022415161 *jet/accuracy_1_of_6‚0.8684210777282715 & Purity/2FRt3SRqqt*FBt/FBt_purity‚1 & Purity/2FRt3SRqqt1FBt/FBt_purity‚1 7 Purity/1FRt1SRqqt3FBt/FBt_purity‚0.9166666865348816 :"Purity/1FRt*SRqqt2FBt/event_purity‚0.08447099477052689 ("Purity/1FRt2SRqqt0FBt/event_purity‚0 8 Purity/3FRt*SRqqt1FBt/FRt_purity‚0.27828285098075867 :"Purity/2FRt4SRqqt*FBt/SRqqt_purity‚0.05000000074505806 ("Purity/3FRt4SRqqt0FBt/event_purity‚0 6 Purity/*FRt1SRqqt3FBt/FBt_purity‚0.920634925365448 & Purity/*FRt3SRqqt2FBt/FBt_purity‚1 @&Purity/*FRt4SRqqt0FBt/event_proportion‚0.0012600806076079607 :"Purity/0FRt*SRqqt0FBt/SRqqt_purity‚0.36749303340911865 particle/accuracy_1_of_3‚1 :"Purity/0FRt*SRqqt3FBt/SRqqt_purity‚0.36666667461395264 9"Purity/*FRt0SRqqt1FBt/event_purity‚0.6806540489196777 =&Purity/*FRt0SRqqt2FBt/event_proportion‚0.0483240932226181 ?&Purity/*FRt2SRqqt2FBt/event_proportion‚0.002079133177176118 ,&Purity/*FRt4SRqqt4FBt/event_proportion‚0 :"Purity/0FRt2SRqqt0FBt/SRqqt_purity‚0.29334983229637146 8 Purity/2FRt1SRqqt0FBt/FRt_purity‚0.11505943536758423 particle/accuracy_4_of_6‚1 7 Purity/*FRt0SRqqt3FBt/FBt_purity‚0.9236112236976624 =&Purity/1FRt1SRqqt*FBt/event_proportion‚0.1723790317773819 @&Purity/4FRt*SRqqt0FBt/event_proportion‚0.0001890120911411941 ("Purity/3FRt4SRqqt*FBt/SRqqt_purity‚0 & Purity/4FRt0SRqqt*FBt/FRt_purity‚0 =&Purity/*FRt1SRqqt*FBt/event_proportion‚0.4482736885547638 6 Purity/1FRt*SRqqt1FBt/FRt_purity‚0.288621723651886 ,&Purity/4FRt3SRqqt*FBt/event_proportion‚0 particle/accuracy_1_of_6‚1 * Purity/4FRt*SRqqt*FBt/FRt_purity‚0.125 :"Purity/1FRt2SRqqt*FBt/SRqqt_purity‚0.30995163321495056 ("Purity/1FRt3SRqqt1FBt/event_purity‚0 ?&Purity/2FRt*SRqqt1FBt/event_proportion‚0.021169355139136314 9"Purity/2FRt2SRqqt1FBt/SRqqt_purity‚0.3619047999382019 ?&Purity/2FRt3SRqqt*FBt/event_proportion‚0.004032257944345474 ("Purity/1FRt0SRqqt4FBt/event_purity‚0 ("Purity/0FRt2SRqqt3FBt/event_purity‚0 epoch‚0 ;"Purity/*FRt2SRqqt*FBt/event_purity‚0.014175290241837502 +"Purity/1FRt4SRqqt*FBt/SRqqt_purity‚0.25 ( Purity/2FRt2SRqqt2FBt/FRt_purity‚0.5 ,&Purity/2FRt0SRqqt4FBt/event_proportion‚0 9"Purity/2FRt1SRqqt0FBt/SRqqt_purity‚0.4260632395744324 ,&Purity/4FRt0SRqqt3FBt/event_proportion‚0 ,&Purity/4FRt3SRqqt4FBt/event_proportion‚0 ("Purity/0FRt4SRqqt*FBt/event_purity‚0 jet/accuracy_6_of_7‚0 7 Purity/0FRt1SRqqt3FBt/FBt_purity‚0.8999999761581421 ("Purity/4FRt*SRqqt0FBt/SRqqt_purity‚1 9"Purity/*FRt*SRqqt1FBt/SRqqt_purity‚0.4152052402496338 8 Purity/*FRt2SRqqt1FBt/FRt_purity‚0.24810664355754852 ,&Purity/*FRt4SRqqt3FBt/event_proportion‚0 :"Purity/0FRt*SRqqt2FBt/SRqqt_purity‚0.42287036776542664 ("Purity/1FRt3SRqqt0FBt/event_purity‚0 @&Purity/1FRt4SRqqt0FBt/event_proportion‚0.0003780241822823882 ?&Purity/2FRt0SRqqt*FBt/event_proportion‚0.037046369165182114 @&Purity/1FRt0SRqqt3FBt/event_proportion‚0.0010710685746744275 9"Purity/*FRt2SRqqt1FBt/SRqqt_purity‚0.3412233889102936 @&Purity/0FRt2SRqqt2FBt/event_proportion‚0.0010080644860863686 ;"Purity/2FRt0SRqqt0FBt/event_purity‚0.004201680887490511 9"Purity/2FRt1SRqqt*FBt/SRqqt_purity‚0.4491351246833801 ) Purity/3FRt1SRqqt2FBt/FRt_purity‚0.25 ("Purity/*FRt1SRqqt4FBt/event_purity‚0 7 Purity/*FRt*SRqqt1FBt/FRt_purity‚0.2896887958049774 @&Purity/1FRt0SRqqt4FBt/event_proportion‚6.300403038039804e-05 ("Purity/1FRt3SRqqt*FBt/event_purity‚0 ?&Purity/2FRt2SRqqt*FBt/event_proportion‚0.019468246027827263 ,&Purity/4FRt4SRqqt1FBt/event_proportion‚0 7 Purity/3FRt4SRqqt0FBt/FRt_purity‚0.3333333432674408 & Purity/4FRt2SRqqt*FBt/FRt_purity‚0 ;"Purity/*FRt2SRqqt0FBt/event_purity‚0.014578606933355331 7 Purity/*FRt3SRqqt0FBt/FRt_purity‚0.1253616362810135 9"Purity/0FRt1SRqqt*FBt/SRqqt_purity‚0.4262346029281616 >&Purity/1FRt2SRqqt0FBt/event_proportion‚0.04731602966785431 ,&Purity/4FRt4SRqqt3FBt/event_proportion‚0 (particle/accuracy‚0.8451045155525208 >&Purity/0FRt*SRqqt*FBt/event_proportion‚0.40996724367141724 >&Purity/1FRt2SRqqt*FBt/event_proportion‚0.05727066472172737 7 Purity/2FRt1SRqqt2FBt/FBt_purity‚0.9868420958518982 ,&Purity/4FRt2SRqqt2FBt/event_proportion‚0 ,&Purity/4FRt3SRqqt1FBt/event_proportion‚0 7 Purity/*FRt1SRqqt*FBt/FBt_purity‚0.9923127293586731 7 Purity/*FRt1SRqqt1FBt/FBt_purity‚0.9982787370681763 8 Purity/2FRt0SRqqt*FBt/FRt_purity‚0.15170440077781677 ,&Purity/4FRt4SRqqt4FBt/event_proportion‚0 ("Purity/1FRt1SRqqt3FBt/event_purity‚0 ("Purity/4FRt1SRqqt*FBt/event_purity‚0 7 Purity/0FRt*SRqqt*FBt/FBt_purity‚0.9912183880805969 7 Purity/0FRt*SRqqt1FBt/FBt_purity‚0.9987989664077759 7 Purity/1FRt1SRqqt*FBt/FBt_purity‚0.9941508173942566 & Purity/1FRt1SRqqt1FBt/FBt_purity‚1 9"Purity/3FRt1SRqqt0FBt/SRqqt_purity‚0.5416666269302368 particle/accuracy_3_of_3‚1 9"Purity/*FRt3SRqqt0FBt/SRqqt_purity‚0.2284957617521286 >&Purity/0FRt*SRqqt0FBt/event_proportion‚0.21931703388690948 :"Purity/1FRt1SRqqt*FBt/SRqqt_purity‚0.45970261096954346 ,&Purity/2FRt*SRqqt4FBt/event_proportion‚0 & Purity/3FRt0SRqqt*FBt/FBt_purity‚1 & Purity/3FRt0SRqqt1FBt/FBt_purity‚1 =&Purity/0FRt1SRqqt0FBt/event_proportion‚0.1694808453321457 7 Purity/2FRt1SRqqt*FBt/FRt_purity‚0.1587657630443573 ("Purity/3FRt*SRqqt1FBt/event_purity‚0 ,&Purity/3FRt2SRqqt3FBt/event_proportion‚0 +jet/accuracy_2_of_2‚0.09339653700590134 particle/accuracy_3_of_6‚1 ?&Purity/0FRt2SRqqt1FBt/event_proportion‚0.009261592291295528 :"Purity/1FRt*SRqqt0FBt/event_purity‚0.03717515990138054 ,&Purity/1FRt4SRqqt4FBt/event_proportion‚0 7 Purity/2FRt0SRqqt2FBt/FBt_purity‚0.9940475821495056 :"Purity/0FRt3SRqqt*FBt/SRqqt_purity‚0.22051207721233368 ("Purity/0FRt4SRqqt1FBt/event_purity‚0 ("Purity/*FRt2SRqqt3FBt/SRqqt_purity‚0 ?&Purity/2FRt0SRqqt1FBt/event_proportion‚0.008946572430431843 +"Purity/1FRt*SRqqt3FBt/SRqqt_purity‚0.75 * Purity/4FRt*SRqqt0FBt/FRt_purity‚0.125 9"Purity/0FRt*SRqqt*FBt/SRqqt_purity‚0.3767129182815552 @&Purity/0FRt4SRqqt*FBt/event_proportion‚0.0005040322430431843 A&Purity/1FRt*SRqqt4FBt/event_proportion‚0.00012600806076079607 @&Purity/3FRt0SRqqt2FBt/event_proportion‚6.300403038039804e-05 +jet/accuracy_3_of_5‚0.08260243386030197 particle/accuracy_2_of_2‚1 & Purity/4FRt0SRqqt0FBt/FRt_purity‚0 :"Purity/*FRt2SRqqt2FBt/SRqqt_purity‚0.29385966062545776 7"Purity/*FRt1SRqqt3FBt/SRqqt_purity‚0.69047611951828 & Purity/1FRt3SRqqt2FBt/FRt_purity‚1 ("Purity/4FRt1SRqqt0FBt/event_purity‚0 7 Purity/0FRt1SRqqt*FBt/FBt_purity‚0.9909148812294006 7 Purity/0FRt1SRqqt1FBt/FBt_purity‚0.9969950318336487 >&Purity/1FRt*SRqqt1FBt/event_proportion‚0.09929435700178146 ?&Purity/2FRt0SRqqt2FBt/event_proportion‚0.003465221729129553 particle/accuracy_1_of_1‚1 particle/accuracy_3_of_5‚1 :"Purity/0FRt1SRqqt2FBt/SRqqt_purity‚0.44603174924850464 9"Purity/2FRt2SRqqt0FBt/SRqqt_purity‚0.3426436483860016 8 Purity/1FRt0SRqqt1FBt/FRt_purity‚0.31786155700683594 @&Purity/2FRt2SRqqt2FBt/event_proportion‚6.300403038039804e-05 @&Purity/4FRt1SRqqt*FBt/event_proportion‚6.300403038039804e-05 (jet/accuracy_3_of_4‚0.04249257594347 9"Purity/*FRt1SRqqt2FBt/SRqqt_purity‚0.5134053826332092 ?&Purity/1FRt3SRqqt0FBt/event_proportion‚0.007938507944345474 9"Purity/2FRt*SRqqt2FBt/SRqqt_purity‚0.5789473652839661 ,&Purity/2FRt1SRqqt4FBt/event_proportion‚0 ,&Purity/3FRt1SRqqt4FBt/event_proportion‚0 ,particle/sensitivity‚0.32016095519065857 7 Purity/*FRt0SRqqt*FBt/FBt_purity‚0.9920475482940674 & Purity/*FRt0SRqqt1FBt/FBt_purity‚1 >&Purity/0FRt2SRqqt*FBt/event_proportion‚0.05361643061041832 8 Purity/2FRt*SRqqt1FBt/FRt_purity‚0.28566113114356995 ,&Purity/3FRt3SRqqt1FBt/event_proportion‚0 +jet/accuracy_1_of_1‚0.42880913615226746 ("Purity/*FRt0SRqqt4FBt/event_purity‚0 ("Purity/1FRt1SRqqt4FBt/event_purity‚0 jet/accuracy_1_of_7‚1 ?&Purity/*FRt*SRqqt3FBt/event_proportion‚0.004473286215215921 7 Purity/1FRt*SRqqt2FBt/FRt_purity‚0.4272156357765198 ,&Purity/3FRt1SRqqt3FBt/event_proportion‚0 @&Purity/4FRt*SRqqt*FBt/event_proportion‚0.0001890120911411941 ) Purity/1FRt*SRqqt3FBt/FRt_purity‚0.25 7 Purity/1FRt1SRqqt0FBt/FRt_purity‚0.1297631561756134 ,&Purity/4FRt*SRqqt1FBt/event_proportion‚0 7 Purity/3FRt3SRqqt*FBt/FRt_purity‚0.1190476268529892 ,&Purity/1FRt3SRqqt4FBt/event_proportion‚0 8 Purity/2FRt2SRqqt1FBt/FRt_purity‚0.23273809254169464 8 Purity/3FRt0SRqqt0FBt/FRt_purity‚0.04575163498520851 ,jet/accuracy_4_of_4‚0.002061113715171814 & Purity/2FRt0SRqqt3FBt/FBt_purity‚1 ("Purity/3FRt3SRqqt0FBt/event_purity‚0 *loss/val_total_loss‚57.634858344452304 :"Purity/0FRt*SRqqt3FBt/event_purity‚0.12345678359270096 ?&Purity/1FRt*SRqqt2FBt/event_proportion‚0.022303426638245583 ?&Purity/2FRt*SRqqt2FBt/event_proportion‚0.005418346729129553 ?&Purity/2FRt1SRqqt1FBt/event_proportion‚0.007875503972172737 8 Purity/2FRt3SRqqt*FBt/FRt_purity‚0.11204762756824493 @&Purity/3FRt0SRqqt1FBt/event_proportion‚0.0011340725468471646 9"Purity/3FRt1SRqqt1FBt/SRqqt_purity‚0.5769230723381042 9"Purity/*FRt0SRqqt*FBt/event_purity‚0.3466196358203888 8 Purity/*FRt1SRqqt0FBt/FRt_purity‚0.12200312316417694 @&Purity/1FRt1SRqqt4FBt/event_proportion‚6.300403038039804e-05 ,&Purity/1FRt2SRqqt3FBt/event_proportion‚0 9"Purity/2FRt2SRqqt*FBt/SRqqt_purity‚0.3458143472671509 ("Purity/3FRt*SRqqt0FBt/event_purity‚0 @&Purity/3FRt*SRqqt1FBt/event_proportion‚0.0024571572430431843 ("Purity/*FRt4SRqqt1FBt/SRqqt_purity‚0 ("Purity/*FRt3SRqqt*FBt/event_purity‚0 ?&Purity/0FRt0SRqqt2FBt/event_proportion‚0.029674898833036423 @&Purity/3FRt2SRqqt0FBt/event_proportion‚0.0024571572430431843 7 Purity/*FRt2SRqqt2FBt/FRt_purity‚0.3205128312110901 ("Purity/0FRt3SRqqt1FBt/event_purity‚0 ("Purity/1FRt*SRqqt4FBt/event_purity‚0 & Purity/1FRt0SRqqt4FBt/FRt_purity‚1 +"Purity/1FRt4SRqqt0FBt/SRqqt_purity‚0.25 & Purity/*FRt3SRqqt*FBt/FBt_purity‚1 & Purity/*FRt3SRqqt1FBt/FBt_purity‚1 @&Purity/0FRt2SRqqt3FBt/event_proportion‚6.300403038039804e-05 8 Purity/3FRt1SRqqt1FBt/FRt_purity‚0.26923075318336487 @&Purity/3FRt2SRqqt*FBt/event_proportion‚0.0028351815417408943 A&Purity/3FRt4SRqqt*FBt/event_proportion‚0.00012600806076079607 @&Purity/4FRt2SRqqt*FBt/event_proportion‚6.300403038039804e-05 9"Purity/0FRt2SRqqt2FBt/SRqqt_purity‚0.3076923191547394 particle/accuracy_1_of_7‚1 +"Purity/3FRt*SRqqt2FBt/SRqqt_purity‚0.25 ("Purity/0FRt2SRqqt3FBt/SRqqt_purity‚0 8 Purity/*FRt0SRqqt0FBt/FRt_purity‚0.08425780385732651 @&Purity/*FRt1SRqqt4FBt/event_proportion‚6.300403038039804e-05 @&Purity/0FRt*SRqqt3FBt/event_proportion‚0.0030871976632624865 7 Purity/1FRt2SRqqt2FBt/FRt_purity‚0.3055555522441864 9"Purity/1FRt2SRqqt2FBt/SRqqt_purity‚0.1944444626569748 ("Purity/4FRt2SRqqt0FBt/SRqqt_purity‚1 >&Purity/0FRt*SRqqt1FBt/event_proportion‚0.14787046611309052 ("Purity/2FRt3SRqqt0FBt/event_purity‚0 ,&Purity/3FRt4SRqqt2FBt/event_proportion‚0 ,&Purity/4FRt2SRqqt3FBt/event_proportion‚0 particle/accuracy_6_of_6‚1 & Purity/*FRt*SRqqt4FBt/FRt_purity‚1 =&Purity/1FRt*SRqqt0FBt/event_proportion‚0.3135710656642914 A&Purity/3FRt4SRqqt0FBt/event_proportion‚0.00012600806076079607 8 Purity/3FRt*SRqqt2FBt/FRt_purity‚0.20000000298023224 ("Purity/1FRt3SRqqt2FBt/event_purity‚0 :"Purity/1FRt0SRqqt2FBt/event_purity‚0.12427285313606262 9"Purity/1FRt1SRqqt0FBt/SRqqt_purity‚0.4451372027397156 ,&Purity/2FRt4SRqqt3FBt/event_proportion‚0 9"Purity/1FRt0SRqqt3FBt/event_purity‚0.0714285746216774 9"Purity/*FRt1SRqqt*FBt/SRqqt_purity‚0.4429338872432709 >&Purity/*FRt2SRqqt0FBt/event_proportion‚0.10855595022439957 :"Purity/1FRt*SRqqt*FBt/event_purity‚0.06522779166698456 :"Purity/2FRt*SRqqt0FBt/SRqqt_purity‚0.35901257395744324 9"Purity/3FRt2SRqqt0FBt/SRqqt_purity‚0.3278985321521759 ("Purity/0FRt4SRqqt0FBt/event_purity‚0 7 Purity/1FRt4SRqqt*FBt/FRt_purity‚0.3333333432674408 @&Purity/1FRt*SRqqt3FBt/event_proportion‚0.0013230846961960196 @&Purity/3FRt*SRqqt0FBt/event_proportion‚0.0071824598126113415 ("Purity/2FRt2SRqqt2FBt/event_purity‚0 =&Purity/*FRt*SRqqt0FBt/event_proportion‚0.6160534024238586 :"Purity/1FRt1SRqqt0FBt/event_purity‚0.00696146534755826 ,&Purity/4FRt1SRqqt1FBt/event_proportion‚0 & Purity/*FRt0SRqqt4FBt/FRt_purity‚1 ("Purity/0FRt1SRqqt3FBt/event_purity‚0 9"Purity/*FRt*SRqqt2FBt/SRqqt_purity‚0.4694756269454956 ;"Purity/*FRt2SRqqt1FBt/event_purity‚0.014706171117722988 :"Purity/*FRt3SRqqt*FBt/SRqqt_purity‚0.22978773713111877 ,&Purity/4FRt1SRqqt3FBt/event_proportion‚0 9"Purity/*FRt3SRqqt2FBt/SRqqt_purity‚0.3333333432674408 8 Purity/*FRt*SRqqt0FBt/FRt_purity‚0.10999345779418945 6 Purity/1FRt2SRqqt*FBt/FRt_purity‚0.173463374376297 @&Purity/2FRt0SRqqt3FBt/event_proportion‚6.300403038039804e-05 ("Purity/2FRt2SRqqt*FBt/event_purity‚0 ,&Purity/4FRt4SRqqt2FBt/event_proportion‚0 9"Purity/*FRt*SRqqt1FBt/event_purity‚0.3965000808238983 7 Purity/*FRt3SRqqt1FBt/FRt_purity‚0.2888888716697693 :"Purity/2FRt*SRqqt0FBt/event_purity‚0.00136542075779289 9"Purity/2FRt1SRqqt2FBt/SRqqt_purity‚0.5964912176132202 & Purity/3FRt1SRqqt*FBt/FBt_purity‚1 & Purity/3FRt1SRqqt1FBt/FBt_purity‚10¼P¯“  ‚(:ˆ ó¿éÔ÷“¼º gpu.0.gpu‚41.0 gpu.0.memory‚3, gpu.0.memoryAllocated‚16.716384887695312* gpu.0.memoryAllocatedBytes‚ 14359265280 gpu.0.temp‚34.0 gpu.0.powerWatts‚141.574' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚28.3148 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0Xx6¡ ‚(:ü ó¿éÔ †Ž¼ disk.dm-0.out‚24220.16015625 disk.sda2.in‚0 disk.sda2.out‚0 disk.dm-1.in‚ 0.6953125 disk.sda1.out‚0 network.recv‚422654) disk./.usagePercent‚78.77566414088952 disk.dm-1.out‚ 166.43359375 disk.drbd1260.in‚|uxõw0( proc.memory.availableMB‚ 682.87890625 proc.memory.rssMB‚ 3039.6875* proc.memory.percent‚18.552780151367188 proc.cpu.threads‚17 network.sent‚ 1.019447e+06 disk.sda3.in‚ 0.6953125 disk.sda3.out‚ 166.43359375! disk.nvme0c0n1.in‚ 2581.609375% disk.nvme0c0n1.out‚24220.16015625 disk.dm-0.in‚ 2581.609375 disk.sda1.in‚ 0.1015625 disk.sda.out‚ 166.43359375$ disk./.usageGB‚1173.814323425293 disk.nvme0n1.in‚ 2581.609375# disk.nvme0n1.out‚24220.16015625 disk.drbd1260.out‚ 0.1484375 disk.sda.in‚0.796875$ memory_percent‚95.83203792572021 cpu‚25.84713594845254 è§Ú“¢ ‚(:ˆ ‚ÀéÔëüë¸ gpu.0.gpu‚56.0 gpu.0.memory‚6, gpu.0.memoryAllocated‚16.716384887695312* gpu.0.memoryAllocatedBytes‚ 14359265280 gpu.0.temp‚35.0 gpu.0.powerWatts‚164.057' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚32.8114 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0m'ô†£ ‚(:û ‚ÀéÔô˜Ýº disk.drbd1260.in‚0 disk.nvme0n1.in‚ 2732.71875$ memory_percent‚95.83449363708496" proc.memory.rssMB‚ 3039.6953125) disk./.usagePercent‚78.79405341808152 disk.drbd1260.out‚ 0.15234375 disk.sda.out‚ 166.79296875 disk.sda3.in‚ 0.6953125 disk.dm-0.out‚24894.10546875* proc.memory.percent‚18.552827835083008 cpu‚25.84307976627545# disk./.usageGB‚1174.08833694458 disk.sda2.in‚0 disk.sda2.out‚0 disk.sda3.out‚ 166.79296875 disk.nvme0c0n1.in‚ 2732.71875# disk.nvme0n1.out‚24894.10546875 proc.cpu.threads‚17 network.sent‚ 1.122237e+06 disk.dm-1.in‚ 0.6953125 disk.sda1.in‚ 0.1015625 disk.sda1.out‚0 disk.sda.in‚0.796875% disk.nvme0c0n1.out‚24894.10546875 disk.dm-0.in‚ 2732.71875' proc.memory.availableMB‚ 682.4765625 network.recv‚430083 disk.dm-1.out‚ 166.79296875^‚¥Ð‹¤ ‚(:€ ‘ÀéԎ늻 network.recv‚433624( disk./.usagePercent‚78.7939814799644 disk.sda2.in‚0 disk.nvme0n1.in‚ 2732.71875 disk.sda.out‚ 167.17578125$ memory_percent‚95.83451747894287( proc.memory.availableMB‚ 682.47265625* proc.memory.percent‚18.552851676940918% disk./.usageGB‚1174.0872650146484 disk.sda3.in‚ 0.6953125 disk.nvme0c0n1.in‚ 2732.71875 disk.dm-0.in‚ 2732.71875! disk.dm-0.out‚25255.111328125 disk.drbd1260.in‚0 disk.drbd1260.out‚ 0.15234375 disk.sda1.in‚ 0.1015625 network.sent‚ 1.12721e+06$ disk.nvme0n1.out‚25255.111328125 disk.dm-1.in‚ 0.6953125 disk.dm-1.out‚ 167.17578125 disk.sda.in‚0.796875# proc.memory.rssMB‚ 3039.69921875 proc.cpu.threads‚17 disk.sda2.out‚0 disk.sda3.out‚ 167.17578125& disk.nvme0c0n1.out‚25255.111328125 disk.sda1.out‚0 cpu‚25.83948213006868•¬ Úž¥ ‚(:“ ‘ÀéÔ÷¡Ðº gpu.0.gpu‚30.0 gpu.0.memory‚3, gpu.0.memoryAllocated‚16.718826293945312* gpu.0.memoryAllocatedBytes‚ 14361362432 gpu.0.temp‚34.0 gpu.0.powerWatts‚138.585' gpu.0.enforcedPowerLimitWatts‚500.0) gpu.0.powerPercent‚27.717000000000002 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0Ñ%~1“¦ ‚(:ˆ  ÀéÔ´¨óº gpu.0.gpu‚36.0 gpu.0.memory‚3, gpu.0.memoryAllocated‚16.718826293945312* gpu.0.memoryAllocatedBytes‚ 14361362432 gpu.0.temp‚31.0 gpu.0.powerWatts‚111.122' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚22.2244 gpu.0.smClock‚1275 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0ÞÝë§ ‚(:„  ÀéÔ…∼$ memory_percent‚95.83609104156494( proc.memory.availableMB‚ 682.21484375 network.recv‚437477% disk./.usageGB‚1174.0872650146484& disk.nvme0n1.out‚25735.53759765625 disk.drbd1260.in‚0 disk.sda.out‚ 167.59765625# proc.memory.rssMB‚ 3039.69921875 disk.drbd1260.out‚0.15625 disk.sda1.in‚ 0.1015625( disk.nvme0c0n1.out‚25735.53759765625* proc.memory.percent‚18.552851676940918 cpu‚25.83615836701079 proc.cpu.threads‚17 network.sent‚ 1.132384e+06( disk./.usagePercent‚78.7939814799644 disk.dm-0.in‚ 2732.71875# disk.dm-0.out‚25735.53759765625 disk.dm-1.out‚ 167.59765625 disk.sda2.in‚0 disk.sda3.in‚ 0.6953125 disk.sda3.out‚ 167.59765625 disk.nvme0n1.in‚ 2732.71875 disk.dm-1.in‚ 0.6953125 disk.sda1.out‚0 disk.sda.in‚0.796875 disk.sda2.out‚0 disk.nvme0c0n1.in‚ 2732.71875~h£^2¨ - _wandbruntime‚1137  _runtime‚1137i¢ ÊÅ© ¿ & lr-AdamW/pg1‚0.0013042582541210144 & lr-AdamW/pg2‚0.0013042582541210144 trainer/global_step‚649 % _timestamp‚1.7885020610845017e+09 _runtime‚1137.163660457 _step‚49êÉëɪ à _runtime‚1137.163660457 _step‚49 & lr-AdamW/pg1‚0.0013042582541210144 & lr-AdamW/pg2‚0.0013042582541210144 trainer/global_step‚649 % _timestamp‚1.7885020610845017e+091 :2« - _wandbruntime‚1137  _runtime‚1137‹&»¸Å'¬ ¿O TADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚52187062741761 G?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚609 UBDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated‚53335080260096 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 TDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 13807648768 YIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 13207863296 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚609 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 L?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚13458313 PCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚16292039 H?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚1321 MFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚33 O?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 13807648768 IDeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚3689 N@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 248284160 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚31684 @8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚895 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 TGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚13458315 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚4068586 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚31929 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚602 dQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚53466049655808 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 C;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚895 E8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚29750352 PCDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated‚16318125 SGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚7769314 RBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 13807648768 @8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚895 M@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚11837298 WGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 12589930496 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 TFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 599785472 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚286 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 231244800 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚7769316 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚2 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 586906112 SEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 384795648 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚24903680 $ _timestamp‚1.788502061087217e+09 YFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚54138379421184 XEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚52187306373461 ]LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 639769761893 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 599785472 \IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚52691723196928 ^MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 672354669056 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚26088 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 586906112 ]LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 643357063168 _LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚51547536611568 [KDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 13207863296 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 599785472 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 231244800 TGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚16318125 P@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 13568114688 E=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚895 WGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 13035530240 P@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 13118524025 XJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 226592340 G?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚286 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚17039360 SGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚4067984 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 152061440 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 UGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 582240769 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚26086 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚31684 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 320831488 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚31929 IDeviceStatsMonitor.on_train_batch_start/allocation.all.current‚26088 [HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚51547519572208 YHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 639543169553 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚26086 Q>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚53334831975936 _LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚52691723196928 OFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚1299 `MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚53466024752128 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 JBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚609 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚604 BDeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚31684 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 320831488 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚31929 IDeviceStatsMonitor.on_train_batch_start/allocation.all.current‚26088 [HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚51547519572208 YHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 639543169553 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚26086 Q>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚53334831975936 _LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚52691723196928 OFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚1299 `MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚53466024752128 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 JBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚609 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚604 BDeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚3689 N@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 248284160 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚31684 @8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚895 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 TGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚13458315 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚4068586 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚31929 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚602 dQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚53466049655808 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 C;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚895 E8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚29750352 PCDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated‚16318125 SGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚7769314 RBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 13807648768 @8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚8952&ë±'2® - _wandbruntime‚1138  _runtime‚1138×z!O ¯ ÊM I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚26112 FDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 13118524025 VCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚53414548953600 WFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 644083289600 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚3689 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚33 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 309544960 [MDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚ 284641280 R?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚52264861523699 K>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚11854839 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚2 ^KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚53545619612160 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚17039360 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚615 VCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚52770217366528 RBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 13035530240 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 YFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚51624366306752 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 N>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 13568114688 E6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚895 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 599785472 ZGDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚52770234405888 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 231258112 F8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 VCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚52265105155607 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 582240769 WGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 13207863296 UEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 13035530240 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1321 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 TCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 644083289600 NADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚13478270 OBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚11855454 QCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 384795648 L>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 248297472 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 243631908 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚609 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚26112 NADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚13478272 YFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚52770217366528 SEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 586906112 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 BDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚31929 XGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 644314547712 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 WDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚54218970323456 % _timestamp‚1.7885020620031514e+09 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚2 K>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚29821008 G:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚29794894 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚613 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚895 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚286 PBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 586906112 REDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚13478272 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 WFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 640495216947 RBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 13807648768 WIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 599785472 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 599785472 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚13478270 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17039360 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚4074645 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 UEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 12589930496 M=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 13807648768 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚31684 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 152061440 trainer/global_step‚649 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚7780809 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚895 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 Ž¨žÔ&° ÎM OADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 248297472 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 REDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚16342736 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚895 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 599785472 ZGDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚52770234405888 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 231258112 F8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 VCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚52265105155607 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 582240769 WGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 13207863296 UEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 13035530240 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1321 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 TCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 644083289600 NADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚13478270 OBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚11855454 QCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 384795648 L>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 248297472 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 243631908 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚609 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚26112 NADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚13478272 YFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚52770217366528 SEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 586906112 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 BDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚31929 XGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 644314547712 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 WDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚54218970323456 % _timestamp‚1.7885020620031514e+09 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚2 K>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚29821008 G:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚29794894 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚613 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚895 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚286 PBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 586906112 REDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚13478272 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 WFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 640495216947 RBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 13807648768 WIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 599785472 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 599785472 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚13478270 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17039360 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚4074645 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 UEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 12589930496 M=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 13807648768 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚31684 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 152061440 trainer/global_step‚649 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚7780809 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚895 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚26112 FDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 13118524025 VCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚53414548953600 WFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 644083289600 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚3689 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚33 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 309544960 [MDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚ 284641280 R?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚52264861523699 K>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚11854839 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚2 ^KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚53545619612160 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚17039360 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚615 VCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚52770217366528 RBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 13035530240 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 YFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚51624366306752 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 N>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 13568114688 E“¶ ‚(:ˆ ¾Àéԛݫº gpu.0.gpu‚47.0 gpu.0.memory‚6, gpu.0.memoryAllocated‚16.718826293945312* gpu.0.memoryAllocatedBytes‚ 14361362432 gpu.0.temp‚34.0 gpu.0.powerWatts‚116.868' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚23.3736 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚06ÿ“•· ‚(:Š ¾ÀéÔŠø‡»% disk./.usageGB‚1174.5414199829102 disk.nvme0n1.in‚ 2927.859375 disk.dm-0.in‚ 2927.859375 disk.dm-1.out‚ 168.37890625 disk.drbd1260.in‚0 disk.drbd1260.out‚ 0.16015625 disk.sda1.out‚0) disk./.usagePercent‚78.82446020094585! disk.nvme0c0n1.in‚ 2927.859375 disk.dm-1.in‚ 0.6953125 disk.sda1.in‚ 0.1015625 disk.sda2.in‚0 disk.sda2.out‚0 disk.sda3.in‚ 0.6953125 disk.sda3.out‚ 168.37890625( disk.nvme0c0n1.out‚26163.41259765625 disk.sda.in‚0.796875 disk.sda.out‚ 168.37890625# memory_percent‚95.8350419998169# proc.memory.rssMB‚ 3039.69921875* proc.memory.percent‚18.552851676940918 cpu‚25.82927094038694 network.sent‚ 1.225479e+06& disk.nvme0n1.out‚26163.41259765625# disk.dm-0.out‚26163.41259765625( proc.memory.availableMB‚ 682.38671875 proc.cpu.threads‚17 network.recv‚450020Gæ¸ ‚(:’ ÍÀéÔ´³Äº gpu.0.gpu‚41.0 gpu.0.memory‚3, gpu.0.memoryAllocated‚16.718826293945312* gpu.0.memoryAllocatedBytes‚ 14361362432 gpu.0.temp‚33.0 gpu.0.powerWatts‚126.26' gpu.0.enforcedPowerLimitWatts‚500.0) gpu.0.powerPercent‚25.252000000000002 gpu.0.smClock‚1335 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0’íï‹—¹ ‚(:Œ ÍÀéÔøÅû¼ network.recv‚453319) disk./.usagePercent‚78.83192128452374 disk.sda3.out‚ 168.71484375! disk.nvme0c0n1.in‚ 2927.859375 disk.sda1.out‚0 proc.cpu.threads‚17 disk.sda2.in‚0 disk.dm-1.out‚ 168.71484375 disk.sda1.in‚ 0.1015625 disk.sda.in‚0.796875$ memory_percent‚95.83508968353271* proc.memory.percent‚18.552851676940918% disk./.usageGB‚1174.6525955200195 disk.sda2.out‚0 disk.sda3.in‚ 0.6953125 disk.nvme0n1.in‚ 2927.859375& disk.nvme0n1.out‚26734.20556640625# disk.dm-0.out‚26734.20556640625 disk.dm-1.in‚ 0.6953125 disk.drbd1260.out‚ 0.16015625( disk.nvme0c0n1.out‚26734.20556640625 disk.dm-0.in‚ 2927.859375 disk.drbd1260.in‚0 disk.sda.out‚ 168.71484375( proc.memory.availableMB‚ 682.37890625# proc.memory.rssMB‚ 3039.69921875 cpu‚25.825745506433346 network.sent‚ 1.230365e+06ô@­“º ‚(:ˆ ÜÀéÔºÒ©¹ gpu.0.gpu‚40.0 gpu.0.memory‚5, gpu.0.memoryAllocated‚16.718826293945312* gpu.0.memoryAllocatedBytes‚ 14361362432 gpu.0.temp‚34.0 gpu.0.powerWatts‚114.748' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚22.9496 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0.5Å.Š» ‚(:ÿ ÜÀéÔ½¿ê» disk.sda3.in‚ 0.6953125 disk.sda3.out‚ 169.40234375& disk.nvme0c0n1.out‚27119.259765625 disk.dm-1.in‚ 0.6953125 disk.drbd1260.in‚0 disk.drbd1260.out‚ 0.1640625 disk.sda1.out‚0$ memory_percent‚95.83358764648438 network.sent‚ 1.234683e+06 disk.sda2.out‚0 disk.dm-0.in‚ 2927.859375# proc.memory.availableMB‚682.625* proc.memory.percent‚18.552851676940918 network.recv‚456095$ disk./.usageGB‚1174.652500152588 disk.sda.in‚0.796875 disk.sda.out‚ 169.40234375 disk.sda2.in‚0! disk.dm-0.out‚27119.259765625 disk.dm-1.out‚ 169.40234375 cpu‚25.822033087144078) disk./.usagePercent‚78.83191488433539! disk.nvme0c0n1.in‚ 2927.859375 disk.nvme0n1.in‚ 2927.859375$ disk.nvme0n1.out‚27119.259765625 disk.sda1.in‚ 0.1015625# proc.memory.rssMB‚ 3039.69921875 proc.cpu.threads‚17ÐYX˜’¼ ‚(:‡ ëÀéÔ®²„¹ gpu.0.gpu‚7.0 gpu.0.memory‚1, gpu.0.memoryAllocated‚16.718826293945312* gpu.0.memoryAllocatedBytes‚ 14361362432 gpu.0.temp‚31.0 gpu.0.powerWatts‚111.102' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚22.2204 gpu.0.smClock‚1275 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0<`½ ‚(:… ëÀéÔÇЛ» disk.sda.in‚0.796875 disk.sda.out‚ 169.91796875% disk./.usageGB‚1174.6576957702637 disk.dm-0.in‚ 2927.859375! disk.dm-0.out‚27485.369140625 disk.dm-1.in‚ 0.6953125 disk.sda2.in‚0 disk.sda2.out‚0 disk.sda3.in‚ 0.6953125# proc.memory.rssMB‚ 3039.69921875 network.recv‚460318! disk.nvme0c0n1.in‚ 2927.859375 disk.nvme0n1.in‚ 2927.859375 disk.dm-1.out‚ 169.91796875 disk.sda1.in‚ 0.1015625 network.sent‚ 1.240696e+06 disk.sda3.out‚ 169.91796875$ memory_percent‚95.83513736724854( proc.memory.availableMB‚ 682.37109375* proc.memory.percent‚18.552851676940918 cpu‚25.818788251000065 proc.cpu.threads‚17) disk./.usagePercent‚78.83226356659702& disk.nvme0c0n1.out‚27485.369140625$ disk.nvme0n1.out‚27485.369140625 disk.drbd1260.in‚0 disk.drbd1260.out‚ 0.1640625 disk.sda1.out‚0Mb÷2¾ - _wandbruntime‚1213  _runtime‚1213ߣ” Â¿ ¼ % lr-AdamW/pg1‚0.001303496030591741 % lr-AdamW/pg2‚0.001303496030591741 trainer/global_step‚699 $ _timestamp‚1.788502137462066e+09 _runtime‚1213.544282317 _step‚538A‘ÒÆÀ À % lr-AdamW/pg1‚0.001303496030591741 % lr-AdamW/pg2‚0.001303496030591741 trainer/global_step‚699 $ _timestamp‚1.788502137462066e+09 _runtime‚1213.544282317 _step‚535Y°212Á - _wandbruntime‚1213  _runtime‚1213~!ßçI' ÀO E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚31929 RBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 13807648768 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚2 IDeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚26088 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 TFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 599785472 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚26086 M=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 13568114688 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚26086 ]LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 691258452992 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 303266816 dQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚57445792657408 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 C;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚895 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚31929 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚614 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 320831488 YHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 687172711261 trainer/global_step‚699 [HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚55389856299408 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚286 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚2 TGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚17550727 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 JBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚609 SGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚8343964 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚31684 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚31684 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚4370128 TADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚56077029010669 L?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚17524641 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 231244800 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 _runtime‚1213.546377856 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 599785472 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚8343966 Q>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚57308293813760 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 248284160 H?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚1321 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 MFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚33 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 152061440 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 \IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚56617283644928 TDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 13035530240 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚17039360 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 231244800 IDeviceStatsMonitor.on_train_batch_start/allocation.all.current‚26088 YFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚58168278428160 P@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 13118524025 XJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 226592340 G?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚286 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 586906112 PCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚14456163 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚612 M@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚12713480 @8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚895 E8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚31980804 P@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 13568114688 YHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 691027208192 ]JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚58168581694976 SEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 384795648 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚3689 PCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚17524641 ^MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 722510674432 ZIDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 691258452992 UGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 582240769 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 _LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚55389873338768 PCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚14456165 VEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 691027208192 TADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚57308293813760 _LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚56617283644928 [HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚56617266605568 TGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚14456165 `MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚57445767753728 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 @8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚895 _step‚54 PCDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated‚17550727 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚3689 BDeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚26088 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 TFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 599785472 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚26086 M=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 13568114688 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚26086 ]LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 691258452992 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 303266816 dQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚57445792657408 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 C;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚895 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚31929 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚614 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 320831488 YHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 687172711261 trainer/global_step‚699 [HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚55389856299408 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚286 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚2 TGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚17550727 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 JBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚609 SGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚8343964 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚31684 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚31684 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚4370128 TADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚56077029010669 L?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚17524641 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 231244800 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 _runtime‚1213.546377856 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 599785472 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚8343966 Q>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚57308293813760 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 248284160 H?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚1321 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 MFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚33 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 152061440 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 \IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚56617283644928 TDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 13035530240 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚17039360 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 231244800 IDeviceStatsMonitor.on_train_batch_start/allocation.all.current‚26088 YFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚58168278428160 P@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 13118524025 XJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 226592340 G?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚286 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 586906112 PCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚14456163 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚612 M@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚12713480 @8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚895 E8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚31980804 P@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 13568114688 YHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 691027208192 ]JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚58168581694976 SEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 384795648 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚3689 PCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚17524641 ^MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 722510674432 ZIDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 691258452992 UGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 582240769 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 _LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚55389873338768 PCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚14456165 VEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 691027208192 TADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚57308293813760 _LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚56617283644928 [HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚56617266605568 TGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚14456165 `MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚57445767753728 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 @8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚895 _step‚54 PCDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated‚17550727 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚3689 B8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 SEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 586906112 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚8355459 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 [JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 688351351203 VCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚57388010791424 NADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚17549226 \KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 723515222016 WFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 688124758655 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚609 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚31929 FDeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚26112 K>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚12730931 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚2 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 231258112 G:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚32025346 REDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚14476122 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17039360 VCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚56155071424515 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚895 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚31929 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚8355457 MDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚1299 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 301156352 `ODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 723791474688 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 599785472 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚609 K>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚32051460 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 599785472 trainer/global_step‚699 VCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚56695777814528 [JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 692215937536 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚17039360 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚609 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚286 WFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 691984679424 ^KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚57525362613760 bODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚57525387517440 RBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 13807648768 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 $ _timestamp‚1.788502139566557e+09 NADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚17575338 RBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 13035530240 P@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 13807648768 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚286 N>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 13568114688 PBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 586906112 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚895 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 VHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 226592548 K;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 13568114688 OBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚12731562 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚631 [MDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚ 276252672 YIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 13207863296 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚26114 R?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚56154827792607 UEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 12589930496 WGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 13207863296 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚895 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 231258112 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 582240769 EDeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 248297472 ZGDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚56695794853888 WIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 599785472 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 R?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚57387762493952 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1321 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚4375474 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 N>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 13118524025 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚14476120ØdÀµÓ&È ÍM G:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚32051460 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 152061440 NADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚14476122 TCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 691984679424 QCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 384795648 ]JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚55466720073312 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚17039360 REDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚17575338 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24903680 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚26112 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 243631908 _runtime‚1215.647510435 FDeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚26112 K>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚12730931 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚2 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 231258112 G:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚32025346 REDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚14476122 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17039360 VCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚56155071424515 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚895 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚31929 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚8355457 MDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚1299 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 301156352 `ODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 723791474688 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 599785472 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚609 K>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚32051460 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 599785472 trainer/global_step‚699 VCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚56695777814528 [JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 692215937536 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚17039360 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚609 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚286 WFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 691984679424 ^KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚57525362613760 bODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚57525387517440 RBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 13807648768 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 $ _timestamp‚1.788502139566557e+09 NADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚17575338 RBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 13035530240 P@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 13807648768 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚286 N>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 13568114688 PBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 586906112 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚895 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 VHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 226592548 K;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 13568114688 OBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚12731562 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚631 [MDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚ 276252672 YIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 13207863296 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚26114 R?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚56154827792607 UEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 12589930496 WGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 13207863296 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚895 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 231258112 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 582240769 EDeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 248297472 ZGDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚56695794853888 WIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 599785472 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 R?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚57387762493952 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1321 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚4375474 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 N>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 13118524025 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚14476120 [HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚58249178992128 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 _step‚55 ]JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚56695794853888 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚4376103 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚629 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 SEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 586906112 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚8355459 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 [JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 688351351203 VCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚57388010791424 NADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚17549226 \KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 723515222016 WFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 688124758655 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚609 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚31929 FDeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 YHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 734802252969 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚17039360 WGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 12589930496 TDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 13807648768 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚286 _step‚58 [HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚60542827053568 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚31929 H?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚1321 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 TGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚15454015 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 M@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚34237344 ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 267877376 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 [KDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 13207863296 IDeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚61281755651584 ]LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 739159842816 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 320831488 N@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 248284160 `MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚61425510755328 ^MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 772704283136 C;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚895 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 292781056 JBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚609 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 TADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚61281755651584 _LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚60542844092928 TGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚18783329 OFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚1299 YFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚62198215038464 dQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚61425535659008 trainer/global_step‚749 M=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 13568114688 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚24903680 P@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 13118524025 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚26088 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚586 G?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚286 O?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 13807648768 YIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 13207863296 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚8918616 SGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚4668569 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚31684 TDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 13035530240 ZIDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 739159842816 P@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 13568114688 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚2 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 % _timestamp‚1.7885022268553445e+09 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 586906112 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚26088 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚588 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 XJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 226592340 E8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚34211256 XEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚60542827053568 XEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚61282003935744 UGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 582240769 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 TFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 599785472 @8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚895 PCDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated‚18783329 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 248284160 ]LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 735028845309 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚609 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚26086 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚31684 XEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚59967238911277 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 [HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚59232193026608 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 231244800 M@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚13587183 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 VFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 13207863296 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 599785472 WGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 13035530240 YHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 738928598016 SGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚8918614 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚26086 UBDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated‚61282003935744 G?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚609 L?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚15454013 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 152061440 _LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚59232210065968 G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚3689 TADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚59966995279577 PCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚15454015 \IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚60542844092928 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚17039360 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 586906112 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚3689 SEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 384795648 bQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 772972160512 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚31929 VEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 738928598016 ]JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚62198507819520 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 599785472 E=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚895 RBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 13807648768 PCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚15454013b{¡2É'Û ÃO WKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚4669155 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 _runtime‚ 1302.93447527 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚2 L?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚18757243 QCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 243631700 PCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚18757243 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 231244800 @8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚895 JBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚286 MFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚33 IDeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚61281755651584 ]LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 739159842816 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 320831488 N@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 248284160 `MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚61425510755328 ^MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 772704283136 C;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚895 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 292781056 JBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚609 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 TADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚61281755651584 _LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚60542844092928 TGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚18783329 OFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚1299 YFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚62198215038464 dQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚61425535659008 trainer/global_step‚749 M=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 13568114688 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚24903680 P@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 13118524025 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚26088 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚586 G?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚286 O?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 13807648768 YIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 13207863296 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚8918616 SGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚4668569 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚31684 TDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 13035530240 ZIDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 739159842816 P@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 13568114688 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚2 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 % _timestamp‚1.7885022268553445e+09 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 586906112 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚26088 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚588 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 XJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 226592340 E8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚34211256 XEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚60542827053568 XEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚61282003935744 UGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 582240769 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 TFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 599785472 @8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚895 PCDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated‚18783329 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 248284160 ]LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 735028845309 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚609 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚26086 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚31684 XEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚59967238911277 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 [HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚59232193026608 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 231244800 M@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚13587183 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 VFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 13207863296 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 599785472 WGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 13035530240 YHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 738928598016 SGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚8918614 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚26086 UBDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated‚61282003935744 G?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚609 L?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚15454013 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 152061440 _LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚59232210065968 G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚3689 TADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚59966995279577 PCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚15454015 \IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚60542844092928 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚17039360 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 586906112 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚3689 SEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 384795648 bQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 772972160512 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚31929 VEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 738928598016 ]JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚62198507819520 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 599785472 E=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚895 RBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 13807648768 PCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚15454013 BDeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 YHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 734802252969 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚17039360 WGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 12589930496 TDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 13807648768 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚286 _step‚58 [HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚60542827053568 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚31929 H?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚1321 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 TGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚15454015 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 M@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚34237344 ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 267877376 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 [KDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 13207863296 IDeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚34281912 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚31929 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 305350656 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 NADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚15473972 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚895 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 RBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 13807648768 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚609 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚31929 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚15473970 UEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 13035530240 [JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 735980892911 WFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 735754300363 S@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚61361472629248 ]JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚60621355301888 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 YFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚59309039761152 TDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 13207863296 _step‚59 XGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 740117327360 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚31684 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1321 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚26114 NADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚18807940 ODeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 248297472 [MDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚ 280446976 \KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 773703733248 N>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 13118524025 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚895 PBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 586906112 C6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚34255798 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚607 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚286 trainer/global_step‚749 K>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚13604710 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚4674603 FDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 VCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚60621338262528 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚26112 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 OBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚13605319 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24903680 WGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 13207863296 WIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 599785472 FDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 13568114688 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚609 R?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚60044794061515 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 599785472 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17039360 R?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚61361224331776 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚26112 J=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚18781828 B6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚895 YFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚60621338262528 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 % _timestamp‚1.7885022277642882e+09 RBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 13035530240 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 231258112 MDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚1299 UEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 12589930496 M=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 13807648768 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 REDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚18807940 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 599785472 [HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚62279114699264 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 VCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚60045037693423 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚17039360E  µÔ&Þ ÎM MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚2 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚609 L>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 248297472 [MDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚ 280446976 \KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 773703733248 N>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 13118524025 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚895 PBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 586906112 C6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚34255798 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚607 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚286 trainer/global_step‚749 K>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚13604710 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚4674603 FDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 VCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚60621338262528 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚26112 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 OBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚13605319 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24903680 WGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 13207863296 WIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 599785472 FDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 13568114688 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚609 R?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚60044794061515 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 599785472 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17039360 R?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚61361224331776 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚26112 J=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚18781828 B6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚895 YFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚60621338262528 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 % _timestamp‚1.7885022277642882e+09 RBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 13035530240 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 231258112 MDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚1299 UEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 12589930496 M=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 13807648768 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 REDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚18807940 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 599785472 [HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚62279114699264 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 VCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚60045037693423 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚17039360 _runtime‚1303.844132586 EDeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚34281912 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚31929 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 305350656 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 NADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚15473972 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚895 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 RBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 13807648768 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚609 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚31929 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚15473970 UEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 13035530240 [JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 735980892911 WFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 735754300363 S@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚61361472629248 ]JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚60621355301888 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 YFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚59309039761152 TDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 13207863296 _step‚59 XGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 740117327360 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚31684 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1321 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚26114 NADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚18807940 ODeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚3689 Q>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚65255217489408 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚3689 YFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚66228142512640 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 TFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 599785472 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 248284160 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 586906112 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚26088 TADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚63856961548485 ]LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 782658387017 @8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚895 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 \IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚64468404540928 ZIDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 787061232640 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 trainer/global_step‚799 IDeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚31684 QDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚14463075 C;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚895 JBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚609 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚17039360 TADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚65255217489408 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚2 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚26088 P@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 13568114688 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚24903680 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 [KDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 13207863296 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 586906112 OFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚1299 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 PCDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated‚20015931 TGDeviceStatsMonitor.on_train_batch_start/allocation.large_po¿_׸¬ol.allocated‚16451865 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 299072512 bQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 823162924544 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 WGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 12589930496 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚9493266 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 599785472 IDeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 QCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 243631700 MFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚33 L?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚19989845 UBDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated‚65255465773568 YHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 786829987840 ]LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 787061232640 `MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚65405253756928 SGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚4969203 @8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚895 RBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 13807648768 JBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚286 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚2 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚26086 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 231244800 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚26086 PCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚16451863 TDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 13807648768 N@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 248284160 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚31929 TDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 13035530240 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚31929 H?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚1321 [HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚63074529753808 _runtime‚1392.538103296 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 231244800 PCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚19989845 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 599785472 % _timestamp‚1.7885023164574008e+09 M@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚14462467 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚606 XJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 226592340 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚286 XEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚65255465773568 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 dQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚65405278660608 P@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 13118524025 XEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚64468387501568 ]JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚66228441585152 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 M=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 13568114688 TGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚20015931 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 YIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 13207863296 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 152061440 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 320831488­¡!Ê'ó ÄO ZIDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 787061232640 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 trainer/global_step‚799 IDeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚31684 QDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚14463075 C;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚895 JBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚609 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚17039360 TADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚65255217489408 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚2 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚26088 P@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 13568114688 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚24903680 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 [KDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 13207863296 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 586906112 OFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚1299 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 PCDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated‚20015931 TGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚16451865 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 299072512 bQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 823162924544 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 WGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 12589930496 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚9493266 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 599785472 IDeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 QCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 243631700 MFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚33 L?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚19989845 UBDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated‚65255465773568 YHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 786829987840 ]LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 787061232640 `MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚65405253756928 SGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚4969203 @8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚895 RBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 13807648768 JBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚286 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚2 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚26086 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 231244800 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚26086 PCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚16451863 TDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 13807648768 N@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 248284160 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚31929 TDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 13035530240 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚31929 H?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚1321 [HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚63074529753808 _runtime‚1392.538103296 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 231244800 PCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚19989845 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 599785472 % _timestamp‚1.7885023164574008e+09 M@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚14462467 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚606 XJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 226592340 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚286 XEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚65255465773568 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 dQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚65405278660608 P@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 13118524025 XEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚64468387501568 ]JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚66228441585152 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 M=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 13568114688 TGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚20015931 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 YIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 13207863296 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 152061440 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 320831488 M@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚36467796 SGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚9493264 _LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚63074546793168 ^MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 822888755712 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 WGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 13035530240 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚4969809 SEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 384795648 ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 274168832 VFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 13207863296 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚609 G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚3689 Q>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚65255217489408 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚3689 YFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚66228142512640 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 TFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 599785472 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 248284160 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 586906112 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚26088 TADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚63856961548485 ]LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 782658387017 @8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚895 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 \IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚64468404540928>ºõÉ2ô - _wandbruntime‚1394  _runtime‚1394ô'œÐ&õ ÊM C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚31929 NADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚16471820 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 582240769 NADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚20040542 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚33 QCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 384795648 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚17039360 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚286 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚4975862 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 VCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚64546898710528 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚17039360 P@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 13807648768 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 RBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 13807648768 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 PBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 586906112 WFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 787787459072 F6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚895 % _timestamp‚1.7885023188193386e+09 J=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚20014430 [MDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚ 278349824 [JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 783610434619 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚609 RBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 13035530240 ]JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚64546915749888 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 231258112 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 WFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 783383842071 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 599785472 YIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 13207863296 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚895 S@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚65334934467072 N>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 13568114688 REDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚20040542 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 320831488 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚609 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚3689 NADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚20014430 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1321 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 243631908 UEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 12589930496 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚2 MDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚1299 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 152061440 [HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚66309040455168 WDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚66308737201664 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 R?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚63934760330423 VHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 226592548 G:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚36486250 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 599785472 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚627 bODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚65484873520640 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 ]JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚63151393527712 WGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 13207863296 TDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 13207863296 _runtime‚1394.901839637 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚16471820 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17039360 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚2 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚4975237 [JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 788018717184 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚895 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚609 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚9504757 `ODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 824166934528 \KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 823888584704 N>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 13118524025 L>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 248297472 YFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚64546898710528 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚31684 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 303253504 WIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 599785472 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚895 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚286 SEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 586906112 OBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚14480621 BDeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 NADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚16471822 O8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚26112 UEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 13035530240 K>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚36512364 REDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚16471822 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚31929 FDeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚14479994 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚286nWsŽeö ÎM UIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚4975862 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 VCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚64546898710528 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚17039360 P@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 13807648768 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 RBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 13807648768 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 PBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 586906112 WFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 787787459072 F6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚895 % _timestamp‚1.7885023188193386e+09 J=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚20014430 [MDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚ 278349824 [JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 783610434619 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚609 RBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 13035530240 ]JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚64546915749888 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 231258112 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 WFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 783383842071 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 599785472 YIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 13207863296 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚895 S@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚65334934467072 N>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 13568114688 REDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚20040542 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 320831488 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚609 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚3689 NADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚20014430 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1321 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 243631908 UEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 12589930496 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚2 MDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚1299 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 152061440 [HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚66309040455168 WDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚66308737201664 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 R?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚63934760330423 VHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 226592548 G:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚36486250 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 599785472 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚627 bODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚65484873520640 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 ]JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚63151393527712 WGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 13207863296 TDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 13207863296 _runtime‚1394.901839637 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚16471820 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17039360 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚2 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚4975237 [JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 788018717184 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚895 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚609 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚9504757 `ODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 824166934528 \KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 823888584704 N>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 13118524025 L>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 248297472 YFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚64546898710528 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚31684 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 303253504 WIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 599785472 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚895 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚286 SEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 586906112 OBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚14480621 BDeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 NADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚16471822 O8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚26112 UEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 13035530240 K>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚36512364 REDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚16471822 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚31929 FDeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚14479994 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚286 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚31929 NADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚16471820 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 582240769 NADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚20040542 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚33 QCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 384795648 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚17039360 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚286?΂PR2÷ - _wandbruntime‚1394  _runtime‚1394”û}îXø Ò 1loss/SRqqt2/detection_loss‚2.2984844915072125 &loss/total_loss‚49.888686993753694 epoch‚1 2loss/SRqqt3/assignment_loss‚2.7484045005312154 2loss/SRqqt4/assignment_loss‚1.2305405721862168 0loss/SRqqt3/detection_loss‚2.490623405166701 /loss/FBt2/detection_loss‚2.4682022057699435 _step‚64 /loss/FRt3/detection_loss‚1.9977250698122007 /loss/FBt3/detection_loss‚0.9833464465748806 % _timestamp‚1.7885023188249328e+09 _runtime‚1394.904895088 /loss/FRt1/detection_loss‚2.7240106382153253 .loss/FRt2/assignment_loss‚4.24222504157646 0loss/FRt3/assignment_loss‚2.7200124778793855 1loss/FBt2/assignment_loss‚0.44043526495079277 .loss/FRt2/detection_loss‚3.118240005829755 /loss/FRt4/detection_loss‚1.7101834262377464 0loss/FRt4/assignment_loss‚2.1781890180286947 1loss/SRqqt4/detection_loss‚1.0127261188902936 trainer/global_step‚799 /loss/FRt1/assignment_loss‚4.793981929620107 1loss/FBt3/assignment_loss‚0.18798614384288345 0loss/FBt4/assignment_loss‚0.2271894055357734 1loss/SRqqt1/detection_loss‚2.5447724500999733 ,loss/FBt1/detection_loss‚2.3317536099537 .loss/FBt4/detection_loss‚3.046011209487915 2loss/SRqqt1/assignment_loss‚2.0373604559503935 2loss/SRqqt2/assignment_loss‚2.0137303682168324 1loss/FBt1/assignment_loss‚0.34255273788929463?þ‚+\ù Ö /loss/FRt1/detection_loss‚2.7240106382153253 .loss/FRt2/assignment_loss‚4.24222504157646 0loss/FRt3/assignment_loss‚2.7200124778793855 1loss/FBt2/assignment_loss‚0.44043526495079277 .loss/FRt2/detection_loss‚3.118240005829755 /loss/FRt4/detection_loss‚1.7101834262377464 0loss/FRt4/assignment_loss‚2.1781890180286947 1loss/SRqqt4/detection_loss‚1.0127261188902936 trainer/global_step‚799 /loss/FRt1/assignment_loss‚4.793981929620107 1loss/FBt3/assignment_loss‚0.18798614384288345 0loss/FBt4/assignment_loss‚0.2271894055357734 1loss/SRqqt1/detection_loss‚2.5447724500999733 ,loss/FBt1/detection_loss‚2.3317536099537 .loss/FBt4/detection_loss‚3.046011209487915 2loss/SRqqt1/assignment_loss‚2.0373604559503935 2loss/SRqqt2/assignment_loss‚2.0137303682168324 1loss/FBt1/assignment_loss‚0.34255273788929463 1loss/SRqqt2/detection_loss‚2.2984844915072125 &loss/total_loss‚49.888686993753694 epoch‚1 2loss/SRqqt3/assignment_loss‚2.7484045005312154 2loss/SRqqt4/assignment_loss‚1.2305405721862168 0loss/SRqqt3/detection_loss‚2.490623405166701 /loss/FBt2/detection_loss‚2.4682022057699435 _step‚64 /loss/FRt3/detection_loss‚1.9977250698122007 /loss/FBt3/detection_loss‚0.9833464465748806 % _timestamp‚1.7885023188249328e+09 _runtime‚1394.904895088@&Vé‘ú ‚(:† ®ÂéÔ¹£×½ gpu.0.gpu‚0.0 gpu.0.memory‚0, gpu.0.memoryAllocated‚16.718826293945312* gpu.0.memoryAllocatedBytes‚ 14361362432 gpu.0.temp‚31.0 gpu.0.powerWatts‚93.386' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚18.6772 gpu.0.smClock‚1275 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0ë•-€û ‚(:õ ®ÂéÔí«Ô¿% disk./.usageGB‚1174.7010231018066 disk.sda2.in‚0 disk.sda3.in‚ 0.6953125 disk.nvme0n1.in‚3409.5 disk.dm-1.in‚ 0.6953125 disk.drbd1260.in‚0 disk.sda.out‚ 175.31640625) disk./.usagePercent‚78.83517130017093$ memory_percent‚97.90270328521729( proc.memory.availableMB‚ 343.62109375) proc.memory.percent‚18.55292320251465 proc.cpu.threads‚17$ disk.nvme0n1.out‚31979.255859375 disk.dm-0.in‚3409.5! disk.dm-0.out‚31979.255859375 disk.sda2.out‚0 disk.sda3.out‚ 175.31640625& disk.nvme0c0n1.out‚31979.255859375 disk.sda1.in‚ 0.1015625 disk.sda1.out‚0 network.sent‚ 1.534859e+06 disk.nvme0c0n1.in‚3409.5 disk.dm-1.out‚ 175.31640625 disk.drbd1260.out‚ 0.19140625 disk.sda.in‚0.796875" proc.memory.rssMB‚ 3039.7109375 cpu‚25.775348573480844 network.recv‚520604Ì4C’“ü ‚(:ˆ ½ÂéÔû¶¡¹ gpu.0.gpu‚27.0 gpu.0.memory‚2, gpu.0.memoryAllocated‚16.718826293945312* gpu.0.memoryAllocatedBytes‚ 14361362432 gpu.0.temp‚31.0 gpu.0.powerWatts‚109.486' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚21.8972 gpu.0.smClock‚1275 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0SER÷˜ý ‚(: ½ÂéÔÚÛû network.sent‚ 1.620713e+06 disk.sda2.in‚0 disk.sda2.out‚0 disk.dm-1.in‚ 0.6953125 disk.dm-1.out‚ 175.69921875 disk.sda1.out‚0" proc.memory.rssMB‚ 3039.7109375) disk./.usagePercent‚78.83359378174494 disk.sda3.out‚ 175.69921875# disk.nvme0c0n1.in‚ 3541.16796875 disk.drbd1260.in‚0& proc.memory.availableMB‚ 344.046875) proc.memory.percent‚18.55292320251465 cpu‚25.77198040838526% disk./.usageGB‚1174.6775169372559 disk.sda3.in‚ 0.6953125( disk.nvme0c0n1.out‚32394.62255859375! disk.nvme0n1.in‚ 3541.16796875& disk.nvme0n1.out‚32394.62255859375 proc.cpu.threads‚17 network.recv‚526374 disk.dm-0.in‚ 3541.16796875 disk.drbd1260.out‚ 0.19140625 disk.sda1.in‚ 0.1015625 disk.sda.in‚0.796875 disk.sda.out‚ 175.69921875$ memory_percent‚97.90010452270508# disk.dm-0.out‚32394.62255859375^´V"“þ ‚(:ˆ ÌÂéÔìüù gpu.0.gpu‚10.0 gpu.0.memory‚1, gpu.0.memoryAllocated‚16.718826293945312* gpu.0.memoryAllocatedBytes‚ 14361362432 gpu.0.temp‚31.0 gpu.0.powerWatts‚113.773' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚22.7546 gpu.0.smClock‚1275 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0@‘V‰ÿ ‚(:þ ÌÂéÔ›·èº) disk./.usagePercent‚78.83540913117024 disk.drbd1260.in‚0 disk.sda.out‚176.125 disk.sda3.in‚ 0.6953125 disk.sda3.out‚176.125& disk.nvme0n1.out‚32750.39404296875 cpu‚25.76804624858485 disk.sda1.in‚ 0.1015625 disk.sda2.in‚0# disk.nvme0c0n1.in‚ 3541.16796875 disk.dm-0.in‚ 3541.16796875$ memory_percent‚97.90012836456299 proc.cpu.threads‚17 network.sent‚ 1.62606e+06 disk.sda.in‚0.796875( disk.nvme0c0n1.out‚32750.39404296875 disk.dm-1.in‚ 0.6953125 disk.dm-1.out‚176.125) proc.memory.percent‚18.55292320251465% disk./.usageGB‚1174.7045669555664 disk.drbd1260.out‚ 0.1953125 disk.sda1.out‚0 disk.sda2.out‚0! disk.nvme0n1.in‚ 3541.16796875# disk.dm-0.out‚32750.39404296875( proc.memory.availableMB‚ 344.04296875" proc.memory.rssMB‚ 3039.7109375 network.recv‚530195Nn°|ž€ ‚(:“ ÛÂéÔ„ÛÖ½ gpu.0.gpu‚33.0 gpu.0.memory‚3, gpu.0.memoryAllocated‚16.718826293945312* gpu.0.memoryAllocatedBytes‚ 14361362432 gpu.0.temp‚34.0 gpu.0.powerWatts‚165.938' gpu.0.enforcedPowerLimitWatts‚500.0) gpu.0.powerPercent‚33.187599999999996 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0˜âš ‚(: ÛÂéÔûúùÀ disk.sda3.in‚ 0.6953125" proc.memory.rssMB‚ 3039.7109375( disk.nvme0c0n1.out‚33005.64013671875! disk.nvme0n1.in‚ 3541.16796875 disk.drbd1260.in‚0 disk.sda.in‚0.796875 disk.sda2.in‚0 disk.sda3.out‚ 176.57421875$ memory_percent‚97.90017604827881 cpu‚25.764450118565588 network.sent‚ 1.632095e+06 disk.dm-0.in‚ 3541.16796875# disk.dm-0.out‚33005.64013671875 disk.dm-1.in‚ 0.6953125 disk.dm-1.out‚ 176.57421875 disk.drbd1260.out‚ 0.1953125( proc.memory.availableMB‚ 344.03515625) proc.memory.percent‚18.55292320251465) disk./.usagePercent‚78.83594393090925# disk.nvme0c0n1.in‚ 3541.16796875 disk.sda1.out‚0 disk.sda.out‚ 176.57421875 proc.cpu.threads‚17 network.recv‚534635% disk./.usageGB‚1174.7125358581543& disk.nvme0n1.out‚33005.64013671875 disk.sda1.in‚ 0.1015625 disk.sda2.out‚0]ˆk“‚ ‚(:ˆ êÂéÔáÿ§¹ gpu.0.gpu‚16.0 gpu.0.memory‚2, gpu.0.memoryAllocated‚16.718826293945312* gpu.0.memoryAllocatedBytes‚ 14361362432 gpu.0.temp‚34.0 gpu.0.powerWatts‚163.531' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚32.7062 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0/"ˆr™ƒ ‚(:Ž êÂéÔßÆõ») proc.memory.percent‚18.55292320251465 network.recv‚539339 disk.sda2.out‚0 disk.sda3.in‚ 0.6953125# disk.nvme0c0n1.in‚ 3541.16796875& disk.nvme0n1.out‚33197.45849609375 disk.dm-1.out‚ 176.92578125 disk.drbd1260.in‚0 disk.sda1.in‚ 0.1015625 disk.sda3.out‚ 176.92578125 disk.dm-1.in‚ 0.6953125 disk.drbd1260.out‚ 0.19921875 disk.sda1.out‚0 disk.sda.in‚0.796875$ memory_percent‚97.90024757385254' proc.memory.availableMB‚ 344.0234375# disk.dm-0.out‚33197.45849609375 disk.sda2.in‚0( disk.nvme0c0n1.out‚33197.45849609375 disk.dm-0.in‚ 3541.16796875" proc.memory.rssMB‚ 3039.7109375 cpu‚25.76099070321581 proc.cpu.threads‚17 network.sent‚ 1.638194e+06) disk./.usagePercent‚78.83436718050586% disk./.usageGB‚1174.6890411376953! disk.nvme0n1.in‚ 3541.16796875 disk.sda.out‚ 176.92578125×NEž„ ‚(:“ ùÂéÔƒÓ®¹ gpu.0.gpu‚22.0 gpu.0.memory‚2, gpu.0.memoryAllocated‚16.718826293945312* gpu.0.memoryAllocatedBytes‚ 14361362432 gpu.0.temp‚33.0 gpu.0.powerWatts‚134.057' gpu.0.enforcedPowerLimitWatts‚500.0) gpu.0.powerPercent‚26.811399999999995 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0T¾îsˆ… ‚(:ý ùÂéÔÓÕ¬¼ disk.drbd1260.in‚0 disk.drbd1260.out‚ 0.19921875 disk.sda1.in‚ 0.1015625 disk.sda.out‚ 177.296875 network.sent‚ 1.643272e+06& disk.nvme0n1.out‚33476.67333984375$ memory_percent‚97.89869785308838( proc.memory.availableMB‚ 344.27734375" proc.memory.rssMB‚ 3039.7109375 cpu‚25.757406679955352 disk.sda2.in‚0 disk.sda3.out‚ 177.296875 disk.nvme0c0n1.in‚ 3673.0625( disk.nvme0c0n1.out‚33476.67333984375 disk.dm-0.in‚ 3673.0625) proc.memory.percent‚18.55292320251465 proc.cpu.threads‚17 network.recv‚542646 disk.sda2.out‚0 disk.sda3.in‚ 0.6953125 disk.nvme0n1.in‚ 3673.0625# disk.dm-0.out‚33476.67333984375 disk.sda1.out‚0 disk.sda.in‚0.796875) disk./.usagePercent‚78.83593881075858$ disk./.usageGB‚1174.712459564209 disk.dm-1.in‚ 0.6953125 disk.dm-1.out‚ 177.296875êÂ3{2† - _wandbruntime‚1484  _runtime‚1484LÙ¾ìć ¾ & lr-AdamW/pg1‚0.0012979516482304891 & lr-AdamW/pg2‚0.0012979516482304891 trainer/global_step‚849 $ _timestamp‚1.788502408374096e+09 _runtime‚1484.453678881 _step‚65b:۩Ȉ  & lr-AdamW/pg2‚0.0012979516482304891 trainer/global_step‚849 $ _timestamp‚1.788502408374096e+09 _runtime‚1484.453678881 _step‚65 & lr-AdamW/pg1‚0.0012979516482304891An¾À2‰ - _wandbruntime‚1484  _runtime‚1484,¿aßÈ'Š ÂO ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 TDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 13807648768 E8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚38672160 IDeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚3689 QDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚15339926 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 586906112 M=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 13568114688 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚566 ]JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚70258331290112 TADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚67746927817393 _LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚66916883520368 XJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 226592340 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 O?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 13807648768 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚286 YHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 834731377664 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 586906112 bQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 873309627904 C;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚895 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚2 TGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚21248533 PCDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated‚21248533 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 231244800 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 XEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚68393947949568 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚31929 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚564 QCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 243631700 G?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚609 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 231244800 SGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚5271446 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 % _timestamp‚1.7885024083777454e+09 RBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 13807648768 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 599785472 BDeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 YHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 830061336385 ]LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 834962622464 PCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚21222447 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚5272010 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 IDeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚31684 E=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚895 G?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚286 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚26088 [HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚66916866481008 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 599785472 _step‚66 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚17039360 OFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚1299 dQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚69385021662208 XEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚67747171449093 VFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 13207863296 M@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚15339360 N@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 248284160 TDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 13035530240 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚31684 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 [KDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 13207863296 TFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 599785472 ZIDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 834962622464 M@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚38698248 PCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚17449713 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚3689 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 152061440 ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 267877376 \IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚68393964988928 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 P@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 13568114688 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚26086 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 WGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 12589930496 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚26086 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚10067916 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 JBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚609 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 248284160 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚2 P@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 13118524025 ]LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 830287928725 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚609 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚26088 L?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚17449713 Q>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚69228679327232 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚24903680¾¦[w"‹ ÆO OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚26086 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚10067916 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 JBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚609 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 248284160 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚2 P@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 13118524025 ]LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 830287928725 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚609 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚26088 L?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚17449713 Q>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚69228679327232 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚24903680 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 TDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 13807648768 E8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚38672160 IDeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚3689 QDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚15339926 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 586906112 M=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 13568114688 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚566 ]JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚70258331290112 TADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚67746927817393 _LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚66916883520368 XJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 226592340 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 O?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 13807648768 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚286 YHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 834731377664 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 586906112 bQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 873309627904 C;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚895 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚2 TGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚21248533 PCDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated‚21248533 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 231244800 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 XEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚68393947949568 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚31929 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚564 QCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 243631700 G?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚609 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 231244800 SGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚5271446 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 % _timestamp‚1.7885024083777454e+09 RBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 13807648768 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 599785472 BDeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 YHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 830061336385 ]LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 834962622464 PCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚21222447 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚5272010 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 IDeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚31684 E=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚895 G?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚286 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚26088 [HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚66916866481008 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 599785472 _step‚66 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚17039360 OFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚1299 dQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚69385021662208 XEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚67747171449093 VFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 13207863296 M@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚15339360 N@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 248284160 TDDeviceStatsMonitor.on_train_batch_start/aêêSVUctive_bytes.large_pool.peak‚ 13035530240 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚31684 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 [KDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 13207863296 TFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 599785472 ZIDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 834962622464 M@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚38698248 PCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚17449713 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚3689 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 152061440 ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 267877376 \IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚68393964988928 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 P@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 13568114688 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚26086 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 WGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 12589930496B¶6^‹“Œ ‚(:ˆ ˆÃéÔ°£Ëº gpu.0.gpu‚29.0 gpu.0.memory‚2, gpu.0.memoryAllocated‚16.718826293945312* gpu.0.memoryAllocatedBytes‚ 14361362432 gpu.0.temp‚32.0 gpu.0.powerWatts‚111.922' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚22.3844 gpu.0.smClock‚1275 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0”_àôŒ ‚(: ˆÃéÔ¿‚˼ disk.sda1.out‚0 disk.sda3.in‚ 0.6953125( disk.nvme0c0n1.out‚33925.63818359375 disk.dm-0.in‚ 3673.0625 disk.dm-1.out‚ 177.98828125 network.sent‚ 1.648404e+06 disk.sda1.in‚ 0.1015625 disk.sda2.in‚0 disk.nvme0n1.in‚ 3673.0625# disk.dm-0.out‚33925.63818359375 disk.dm-1.in‚ 0.6953125$ memory_percent‚97.89750576019287" proc.memory.rssMB‚ 3039.7109375) disk./.usagePercent‚78.81741026546794 disk.sda.in‚0.796875 disk.sda2.out‚0 disk.sda3.out‚ 177.98828125) proc.memory.percent‚18.55292320251465 cpu‚25.75373625203242 disk.drbd1260.out‚0.203125 disk.sda.out‚ 177.98828125 disk.nvme0c0n1.in‚ 3673.0625& disk.nvme0n1.out‚33925.63818359375( proc.memory.availableMB‚ 344.47265625 proc.cpu.threads‚17 network.recv‚546196% disk./.usageGB‚1174.4363708496094 disk.drbd1260.in‚0T¸þ2Ž - _wandbruntime‚1486  _runtime‚1486ã›Â?Ò& ÌM SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 231258112 ]JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚68472476197888 UEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 13035530240 NADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚17469670 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚33 QCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 384795648 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚5277340 MDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚1299 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 309544960 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 BDeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 WFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 835688848896 REDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚10079407 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚286 NADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚17469672 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1321 WDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚70338629739008 M=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 13807648768 ZGDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚68472476197888 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚17039360 RBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 13035530240 YFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚66993713215552 TCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 835688848896 PBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 586906112 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚17039360 [HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚70338939283968 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 WFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 831013383779 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 _step‚67 L>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 248297472 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 231258112 ^KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚69464591618560 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 G:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚38742816 N>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 13568114688 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 VHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 226592548 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚17469670 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚3689 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚895 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 TDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 13207863296 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚286 VCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚69308396304896 OBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚15357334 YIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 13207863296 K;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 13568114688 G:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚38716702 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 152061440 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚2 `ODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 874322761728 VCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚68472459158528 S@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚69308396304896 \KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 874038120448 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚609 % _timestamp‚1.7885024107152534e+09 _runtime‚1486.796729569 OADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 248297472 FDeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚15356747 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 582240769 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚609 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚286 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚31929 REDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚17469672 [MDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚ 284641280 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 243631908 N>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 13118524025 RBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 13807648768 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚895 C6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚38716702 F6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚895 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 R?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚67824726599331 K>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚38742816 REDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚21273144 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚5277925 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24903680 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 NADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚21273144›>Ö& ÐM [JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 831239976327 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚26112 YFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚68472459158528 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 WFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 835688848896 REDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚10079407 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚286 NADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚17469672 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1321 WDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚70338629739008 M=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 13807648768 ZGDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚68472476197888 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚17039360 RBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 13035530240 YFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚66993713215552 TCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 835688848896 PBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 586906112 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚17039360 [HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚70338939283968 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 WFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 831013383779 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 _step‚67 L>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 248297472 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 231258112 ^KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚69464591618560 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 G:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚38742816 N>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 13568114688 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 VHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 226592548 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚17469670 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚3689 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚895 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 TDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 13207863296 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚286 VCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚69308396304896 OBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚15357334 YIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 13207863296 K;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 13568114688 G:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚38716702 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 152061440 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚2 `ODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 874322761728 VCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚68472459158528 S@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚69308396304896 \KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 874038120448 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚609 % _timestamp‚1.7885024107152534e+09 _runtime‚1486.796729569 OADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 248297472 FDeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚15356747 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 582240769 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚609 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚286 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚31929 REDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚17469672 [MDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚ 284641280 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 243631908 N>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 13118524025 RBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 13807648768 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚895 C6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚38716702 F6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚895 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 R?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚67824726599331 K>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚38742816 REDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚21273144 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚5277925 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24903680 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 NADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚21273144 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 231258112 ]JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚68472476197888 UEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 13035530240 NADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚17469670 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚33 QCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 384795648 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚5277340 MDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚1299 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 309544960 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 BùA'art/active_bytes.small_pool.peak‚ 586906112 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚2 YHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 882632767488 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚24903680 TADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚73202141165056 MFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚33 SGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚5574893 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 311655424 C;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚895 E=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚895 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 248284160 TGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚22481135 ]JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚74288256021504 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 320831488 YHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 877690878093 PCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚22455049 QDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚16218035 P@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 13118524025 IDeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚31684 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚31684 H?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚1321 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚5575469 ^MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 923204605952 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 XEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚71637137718001 RBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 13807648768 ]LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 882864012288 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 599785472 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 [HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚70759203208208 G?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚286 bQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 923491357696 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 ZIDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 882864012288 [HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚72319508397568 OFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚1299 ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 286751744 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 ]LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 877917470433 YIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 13207863296 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚26086 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚31929 M@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚16217457 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚10642566 YFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚74287944366080 @8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚895 VEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 882632767488 _LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚70759220247568 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 G?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚609 P@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 13568114688 JBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚286 _LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚72319525436928 WGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 13035530240 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚26088 `MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚73364739760128 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 XJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 226592340 [KDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 13207863296 WGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 12589930496 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚26088 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 231244800 PCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚18447563 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚10642564 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 O?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 13807648768 TFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 599785472 PCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚18447565 G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚3689 TDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 13035530240 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 TDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 13807648768 _step‚70 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚31929 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚26086 BDeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 E8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚40902612 M@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚40928700 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 152061440 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚609 _runtime‚ 1573.4087379 UBDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated‚73202389449216 N@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 248284160 IDeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚73202141165056 TGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚18447565 dQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚73364764663808 QCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 243631700 UGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 582240769 % _timestamp‚1.7885024973297894e+09 M=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 13568114688 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 586906112 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚3689 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 599785472¸LÜÊ'£ ÄO PCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚22455049 QDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚16218035 P@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 13118524025 IDeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚31684 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚31684 H?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚1321 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚5575469 ^MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 923204605952 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 XEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚71637137718001 RBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 13807648768 ]LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 882864012288 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 599785472 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 [HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚70759203208208 G?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚286 bQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 923491357696 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 ZIDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 882864012288 [HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚72319508397568 OFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚1299 ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 286751744 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 ]LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 877917470433 YIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 13207863296 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚26086 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚31929 M@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚16217457 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚10642566 YFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚74287944366080 @8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚895 VEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 882632767488 _LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚70759220247568 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 G?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚609 P@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 13568114688 JBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚286 _LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚72319525436928 WGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 13035530240 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚26088 `MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚73364739760128 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 XJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 226592340 [KDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 13207863296 WGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 12589930496 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚26088 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 231244800 PCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚18447563 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚10642564 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 O?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 13807648768 TFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 599785472 PCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚18447565 G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚3689 TDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 13035530240 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 TDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 13807648768 _step‚70 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚31929 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚26086 BDeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 E8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚40902612 M@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚40928700 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 152061440 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚609 _runtime‚ 1573.4087379 UBDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated‚73202389449216 N@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 248284160 IDeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚73202141165056 TGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚18447565 dQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚73364764663808 QCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 243631700 UGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 582240769 % _timestamp‚1.7885024973297894e+09 M=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 13568114688 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 586906112 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚3689 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 599785472 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 231244800 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 586906112 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚2 YHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 882632767488 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚24903680 TADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚73202141165056 MFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚33 SGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚5574893 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 311655424 C;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚895 E=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚895 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 248284160 TGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚22481135 ]JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚74288256021504 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 320831488 YHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 877690878093Fj‹„Õž¤ ‚(:“ âÃéԹ똺 gpu.0.gpu‚53.0 gpu.0.memory‚6, gpu.0.memoryAllocated‚16.718826293945312* gpu.0.memoryAllocatedBytes‚ 14361362432 gpu.0.temp‚34.0 gpu.0.powerWatts‚150.208' gpu.0.enforcedPowerLimitWatts‚500.0) gpu.0.powerPercent‚30.041600000000003 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0ºW— ‘¥ ‚(:† âÃéÔ·¥ ¼" disk.dm-0.out‚36318.8544921875 disk.drbd1260.out‚ 0.21484375$ memory_percent‚97.90089130401611" proc.memory.rssMB‚ 3039.7109375) proc.memory.percent‚18.55292320251465) disk./.usagePercent‚78.87627228576306 disk.sda2.out‚0 disk.sda3.out‚ 180.61328125 disk.dm-1.in‚ 0.6953125 disk.dm-1.out‚ 180.61328125 disk.drbd1260.in‚0( proc.memory.availableMB‚ 343.91796875 proc.cpu.threads‚17 disk.sda.out‚ 180.61328125 disk.sda2.in‚0 disk.sda3.in‚ 0.6953125' disk.nvme0c0n1.out‚36318.8544921875% disk.nvme0n1.out‚36318.8544921875 cpu‚25.73443124222371 network.sent‚ 1.822197e+06 network.recv‚575202% disk./.usageGB‚1175.3134574890137 disk.sda1.in‚ 0.1015625 disk.sda1.out‚0 disk.sda.in‚0.796875! disk.nvme0c0n1.in‚ 3825.671875 disk.dm-0.in‚ 3825.671875 disk.nvme0n1.in‚ 3825.671875—ê:^2¦ - _wandbruntime‚1575  _runtime‚1575ï0FAÒ&§ ÌM FDeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 `ODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 924484095488 M=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 13807648768 S@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚73281858142720 K>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚16235055 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 XGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 883821496832 [HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚74368843619328 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 152061440 WFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 878642925487 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚895 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚286 K>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚40973268 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚605 C6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚40947154 K;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 13568114688 RBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 13035530240 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 WGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 13207863296 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 599785472 _runtime‚1575.125650074 L>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 248297472 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 231258112 N>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 13118524025 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚895 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚33 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17039360 ^KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚73444334620160 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚31929 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 E6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚895 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚2 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚18467520 OADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 248297472 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 231258112 WDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚74368540365824 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚286 VCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚72398019606528 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 BDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 % _timestamp‚1.7885024990459268e+09 G:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚40947154 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 582240769 N>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 13568114688 REDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚18467522 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚5581601 UEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 12589930496 P@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 13807648768 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚895 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚286 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚17039360 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚2 R?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚71714692868239 J=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚22479634 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1321 REDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚10654057 QCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 384795648 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 NADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚22479634 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚31684 R?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚73281609845248 ZGDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚72398036645888 TCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 883590238720 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚603 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚609 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚3689 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚26112 YFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚72398019606528 B\‘¨ ÐM A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚895 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚286 K>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚40973268 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚605 C6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚40947154 K;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 13568114688 RBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 13035530240 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 WGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 13207863296 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 599785472 _runtime‚1575.125650074 L>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 248297472 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 231258112 N>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 13118524025 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚895 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.cÅíVŠE"urrent‚0 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚33 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17039360 ^KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚73444334620160 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚31929 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 E6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚895 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚2 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚18467520 OADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 248297472 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 231258112 WDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚74368540365824 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚286 VCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚72398019606528 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 BDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 % _timestamp‚1.7885024990459268e+09 G:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚40947154 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 582240769 N>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 13568114688 REDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚18467522 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚5581601 UEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 12589930496 P@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 13807648768 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚895 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚286 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚17039360 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚2 R?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚71714692868239 J=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚22479634 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1321 REDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚10654057 QCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 384795648 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 NADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚22479634 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚31684 R?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚73281609845248 ZGDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚72398036645888 TCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 883590238720 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚603 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚609 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚3689 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚26112 YFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚72398019606528 FDeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 `ODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 924484095488 M=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 13807648768 S@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚73281858142720 K>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚16235055 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 XGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 883821496832 [HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚74368843619328 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 152061440 WFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 878642925487GrF·å2© - _wandbruntime‚1575  _runtime‚1575ônŽïVª Ð _runtime‚1575.126461894 /loss/FRt1/assignment_loss‚5.290414333343506 1loss/FBt1/assignment_loss‚0.35573988824103253 0loss/FBt3/assignment_loss‚0.2373405557144737 1loss/FBt4/assignment_loss‚0.18464132169118294 /loss/FRt4/detection_loss‚1.6890603257150485 1loss/SRqqt1/assignment_loss‚2.064064197833576 2loss/SRqqt2/assignment_loss‚1.6603484118686003 /loss/FRt2/detection_loss‚2.9182647236755916 .loss/FBt4/detection_loss‚2.849647962130033 epoch‚1 2loss/SRqqt3/assignment_loss‚2.2200979594443297 0loss/FBt2/assignment_loss‚0.4323302261409306 trainer/global_step‚899 0loss/FRt3/assignment_loss‚2.9561660874772957 1loss/SRqqt2/detection_loss‚2.2681285962462425 1loss/SRqqt4/detection_loss‚0.9042995547356664 _step‚72 1loss/SRqqt3/detection_loss‚2.7188943213707693 %loss/total_loss‚48.58683500859895 2loss/SRqqt4/assignment_loss‚1.1557760231471697 /loss/FBt3/detection_loss‚0.9315771165176525 .loss/FRt2/assignment_loss‚4.17147673538753 /loss/FRt3/detection_loss‚1.9026598660620275 /loss/FRt4/assignment_loss‚2.181457612843349 .loss/FRt1/detection_loss‚2.281379120349906 1loss/SRqqt1/detection_loss‚2.4096210835471985 .loss/FBt1/detection_loss‚2.303032424300909 .loss/FBt2/detection_loss‚2.500416560814931 % _timestamp‚1.7885024990479074e+09úIZ« Ô .loss/FRt2/assignment_loss‚4.17147673538753 /loss/FRt3/detection_loss‚1.9026598660620275 /loss/FRt4/assignment_loss‚2.181457612843349 .loss/FRt1/detection_loss‚2.281379120349906 1loss/SRqqt1/detection_loss‚2.4096210835471985 .loss/FBt1/detection_loss‚2.303032424300909 .loss/FBt2/detection_loss‚2.500416560814931 % _timestamp‚1.7885024990479074e+09 _runtime‚1575.126461894 /loss/FRt1/assignment_loss‚5.290414333343506 1loss/FBt1/assignment_loss‚0.35573988824103253 0loss/FBt3/assignment_loss‚0.2373405557144737 1loss/FBt4/assignment_loss‚0.18464132169118294 /loss/FRt4/detection_loss‚1.6890603257150485 1loss/SRqqt1/assignment_loss‚2.064064197833576 2loss/SRqqt2/assignment_loss‚1.6603484118686003 /loss/FRt2/detection_loss‚2.9182647236755916 .loss/FBt4/detection_loss‚2.849647962130033 epoch‚1 2loss/SRqqt3/assignment_loss‚2.2200979594443297 0loss/FBt2/assignment_loss‚0.4323302261409306 trainer/global_step‚899 0loss/FRt3/assignment_loss‚2.9561660874772957 1loss/SRqqt2/detection_loss‚2.2681285962462425 1loss/SRqqt4/detection_loss‚0.9042995547356664 _step‚72 1loss/SRqqt3/detection_loss‚2.7188943213707693 %loss/total_loss‚48.58683500859895 2loss/SRqqt4/assignment_loss‚1.1557760231471697 /loss/FBt3/detection_loss‚0.9315771165176525HíîÚÒ“¬ ‚(:ˆ ñÃéÔ¶”÷½ gpu.0.gpu‚62.0 gpu.0.memory‚5, gpu.0.memoryAllocated‚16.718826293945312* gpu.0.memoryAllocatedBytes‚ 14361362432 gpu.0.temp‚35.0 gpu.0.powerWatts‚162.718' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚32.5436 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0{ddc–­ ‚(:‹ ñÃéÔŠè„ disk.sda3.in‚ 0.6953125% disk.nvme0n1.out‚36874.6630859375 disk.dm-0.in‚ 3899.28515625 disk.drbd1260.out‚ 0.21484375 disk.sda1.in‚ 0.1015625$ memory_percent‚97.90124893188477" proc.memory.rssMB‚ 3039.7109375 network.sent‚ 1.896865e+06) disk./.usagePercent‚78.83127435348472" disk.dm-0.out‚36874.6630859375 disk.dm-1.out‚ 181.23828125 disk.sda.in‚0.796875 proc.cpu.threads‚17 network.recv‚580854! disk.nvme0n1.in‚ 3899.53515625 disk.sda1.out‚0& proc.memory.availableMB‚ 343.859375) proc.memory.percent‚18.55292320251465 cpu‚25.733162552344286 disk.sda2.in‚0 disk.sda2.out‚0 disk.sda3.out‚ 181.23828125# disk.nvme0c0n1.in‚ 3899.53515625' disk.nvme0c0n1.out‚36874.6630859375 disk.dm-1.in‚ 0.6953125 disk.drbd1260.in‚0 disk.sda.out‚ 181.23828125% disk./.usageGB‚1174.6429557800293f‰Ú“® ‚(:ˆ €ÄéÔç‡Ó» gpu.0.gpu‚56.0 gpu.0.memory‚4, gpu.0.memoryAllocated‚16.718826293945312* gpu.0.memoryAllocatedBytes‚ 14361362432 gpu.0.temp‚35.0 gpu.0.powerWatts‚151.841' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚30.3682 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0·B Ñ‹¯ ‚(:€ €ÄéÔºäðÀ disk.dm-1.out‚ 181.62890625 disk.sda.out‚ 181.62890625) proc.memory.percent‚18.55292320251465 proc.cpu.threads‚17 disk.sda2.in‚0! disk.nvme0c0n1.in‚ 3964.328125$ disk.nvme0n1.out‚37091.568359375 disk.drbd1260.out‚0.21875 disk.sda.in‚0.796875 network.sent‚ 1.974379e+06 disk.sda3.in‚ 0.6953125 disk.nvme0n1.in‚ 3964.328125! disk.dm-0.out‚37091.568359375 disk.sda1.out‚0( proc.memory.availableMB‚ 343.83984375" proc.memory.rssMB‚ 3039.7109375 cpu‚25.73214310532703) disk./.usagePercent‚78.82969939513407% disk./.usageGB‚1174.6194877624512 disk.sda2.out‚0 disk.sda3.out‚ 181.62890625& disk.nvme0c0n1.out‚37091.568359375 disk.drbd1260.in‚0 disk.sda1.in‚ 0.1015625$ memory_percent‚97.90136814117432 network.recv‚586761 disk.dm-0.in‚ 3964.328125 disk.dm-1.in‚ 0.6953125Ú5€“° ‚(:ˆ ÄéÔט©» gpu.0.gpu‚46.0 gpu.0.memory‚6, gpu.0.memoryAllocated‚16.718826293945312* gpu.0.memoryAllocatedBytes‚ 14361362432 gpu.0.temp‚34.0 gpu.0.powerWatts‚115.521' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚23.1042 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0:¸ï‹± ‚(:€ ÄéÔïù£¿ disk.sda1.out‚0$ memory_percent‚97.90151119232178) proc.memory.percent‚18.55292320251465 cpu‚25.73070534191254) disk./.usagePercent‚78.82968787479503 disk.sda.in‚0.796875 disk.sda.out‚ 182.16015625$ disk.nvme0n1.out‚37548.490234375 disk.dm-1.out‚ 182.16015625 disk.drbd1260.in‚0( proc.memory.availableMB‚ 343.81640625" proc.memory.rssMB‚ 3039.7109375 network.recv‚591423& disk.nvme0c0n1.out‚37548.490234375 disk.dm-0.in‚ 3964.328125 proc.cpu.threads‚17 network.sent‚ 1.980249e+06% disk./.usageGB‚1174.6193161010742 disk.sda2.out‚0 disk.sda3.in‚ 0.6953125 disk.sda3.out‚ 182.16015625 disk.nvme0n1.in‚ 3964.328125 disk.dm-1.in‚ 0.6953125 disk.drbd1260.out‚0.21875 disk.sda2.in‚0! disk.nvme0c0n1.in‚ 3964.328125! disk.dm-0.out‚37548.490234375 disk.sda1.in‚ 0.1015625¼÷Üy2² - _wandbruntime‚1627  _runtime‚1627fÞų ¿ & lr-AdamW/pg2‚0.0012915571930065578 trainer/global_step‚949 % _timestamp‚1.7885025519025378e+09 _runtime‚1627.981484538 _step‚73 & lr-AdamW/pg1‚0.0012915571930065578—ìYÉ´ à & lr-AdamW/pg1‚0.0012915571930065578 & lr-AdamW/pg2‚0.0012915571930065578 trainer/global_step‚949 % _timestamp‚1.7885025519025378e+09 _runtime‚1627.981484538 _step‚73IG6¶c2µ - _wandbruntime‚1627  _runtime‚1627…£·íÈ'¶ ÂO N@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 248284160 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 586906112 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚31929 WGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 12589930496 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚286 PCDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated‚23713737 BDeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚31684 UBDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated‚77175851287040 XEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚77175851287040 TADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚77175603002880 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 152061440 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 H?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚1321 SEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 384795648 RBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 13807648768 JBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚609 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 599785472 XEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚76245068845568 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 231244800 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚621 XEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚75527103986909 UGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 582240769 YHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 930534157312 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚26088 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 JBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚286 SGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚5878613 [HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚74601539935408 @8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚895 IDeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚3689 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚31929 OFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚1299 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚619 dQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚77344507665408 bQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 973646895104 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚609 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚17039360 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 586906112 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚11217216 TDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 13807648768 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚17039360 PCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚19445413 YHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 925320419801 YIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 13207863296 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚24903680 @8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚895 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 D>DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 ]LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 925547012141 G?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚286 _runtime‚1627.982993872 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚3689 TGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚23713737 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 TFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 599785472 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚31684 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 O?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 13807648768 L?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚19445413 TDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 13035530240 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 320044032 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 ]JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚78318154560512 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚26086 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚5879232 YFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚78317834516480 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 % _timestamp‚1.7885025519040296e+09 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚26088 M=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 13568114688 _LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚76245085884928 TGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚19445415 QCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 243631700 PCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚19445415 PCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚23687651 E=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚895 ZIDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 930765402112 P@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 13568114688 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 320831488 \IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚76245085884928 ]LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 930765402112 IDeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚77175603002880 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚26086 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚2 P@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 13118524025 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 trainer/global_step‚949 QDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚17096448 M@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚17095827 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 599785472º1èJ· ÆO P@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 13118524025 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 trainer/global_step‚949 QDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚17096448 M@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚17095827 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 599785472 N@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 248284160 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 586906112 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚31929 WGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 12589930496 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚286 PCDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated‚23713737 BDeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚31684 UBDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated‚77175851287040 XEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚77175851287040 TADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚77175603002880 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 152061440 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 H?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚1321 SEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 384795648 RBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 13807648768 JBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚609 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 599785472 XEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚76245068845568 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 231244800 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚621 XEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚75527103986909 UGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 582240769 YHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 930534157312 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚26088 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 JBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚286 SGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚5878613 [HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚74601539935408 @8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚895 IDeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚3689 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚31929 OFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚1299 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚619 dQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚77344507665408 bQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 973646895104 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚609 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚17039360 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 586906112 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚11217216 TDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 13807648768 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚17039360 PCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚19445413 YHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 925320419801 YIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 13207863296 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚24903680 @8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚895 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 D½ûت‚>DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 ]LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 925547012141 G?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚286 _runtime‚1627.982993872 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚3689 TGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚23713737 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 TFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 599785472 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚31684 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 O?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 13807648768 L?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚19445413 TDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 13035530240 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 320044032 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 ]JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚78318154560512 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚26086 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚5879232 YFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚78317834516480 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 % _timestamp‚1.7885025519040296e+09 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚26088 M=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 13568114688 _LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚76245085884928 TGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚19445415 QCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 243631700 PCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚19445415 PCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚23687651 E=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚895 ZIDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 930765402112 P@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 13568114688 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 320831488 \IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚76245085884928 ]LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 930765402112 IDeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚77175603002880 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚26086 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚2Ju¯Úñ2¸ - _wandbruntime‚1628  _runtime‚1628Ê8rFÒ&¹ ÌM ODeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 WGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 13207863296 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚286 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 OADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 248297472 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 231258112 K>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚43203720 [HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚78398741065728 _step‚75 YIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 13207863296 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚26112 WDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚78398437812224 N>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 13118524025 % _timestamp‚1.7885025528498778e+09 G:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚43203720 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17039360 XGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 931722886656 QCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 384795648 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 243631908 NADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚23738348 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚26112 TCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 931491628544 FDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 13568114688 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚286 C6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚43177606 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚31684 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚2 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚33 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 582240769 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚895 [JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 931722886656 ^KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚77424077621760 YFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚76323580054528 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 152061440 [JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 926499059743 M=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 13807648768 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 L>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 248297472 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚17039360 WFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 931491628544 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 NADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚23712236 bODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚77424102525440 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚609 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚3689 REDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚23738348 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚286 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚5884582 UEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 12589930496 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚31929 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚5885172 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚26114 VCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚77255319980544 [MDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚ 278349824 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 VCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚75604902769055 NADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚19465372 NADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚19465370 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚895 WIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 599785472 trainer/global_step‚949 K;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 13568114688 VCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚76323580054528 G:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚43177606 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚11228709 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 P@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 13807648768 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚609 RBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 13035530240 PBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 586906112 ]JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚76323597093888 REDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚11228707 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 B6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚895 ZGDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚76323597093888 R?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚75604659137147 EDeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚17113289 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 303253504 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 599785472 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚609 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1321 `ODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 974638540288 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 WFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 926272467195 UEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 13035530240 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚590 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24903680 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚2Ý™ãVÖ&º ÐM H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚592 MDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚1299 _runtime‚1628.929741204 J=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚23712236 FDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 WGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 13207863296 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚286 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 OADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 248297472 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 231258112 K>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚43203720 [HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚78398741065728 _step‚75 YIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 13207863296 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚26112 WDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚78398437812224 N>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 13118524025 % _timestamp‚1.7885025528498778e+09 G:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚43203720 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17039360 XGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 931722886656 QCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 384795648 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 243631908 NADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚23738348 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚26112 TCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 931491628544 FDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 13568114688 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚286 C6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚43177606 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚31684 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚2 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚33 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 582240769 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚895 [JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 931722886656 ^KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚77424077621760 YFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚76323580054528 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 152061440 [JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 926499059743 M=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 13807648768 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 L>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 248297472 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚17039360 WFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 931491628544 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 NADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚23712236 bODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚77424102525440 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚609 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚3689 REDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚23738348 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚286 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚5884582 UEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 12589930496 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚31929 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚5885172 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚26114 VCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚77255319980544 [MDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚ 278349824 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 VCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚75604902769055 NADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚19465372 NADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚19465370 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚895 WIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 599785472 trainer/global_step‚949 K;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 13568114688 VCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚76323580054528 G:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚43177606 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚11228709 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 P@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 13807648768 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚609 RBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 13035530240 PBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 586906112 ]JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚76323597093888 REDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚11228707 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 B6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚895 ZGDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚76323597093888 R?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚75604659137147 EDeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚17113289 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 303253504 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 599785472 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚609 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1321 `ODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 974638540288 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 WFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 926272467195 UEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 13035530240 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚590 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24903680 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚2 ODeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 ]LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 973176553849 O?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 13807648768 TFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 599785472 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 M=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 13568114688 dQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚81324250667008 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 VFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 13207863296 E=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚895 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚286 QDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚17971901 G?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚286 _step‚78 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚26086 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 E8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚45363516 XEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚80170629293568 `MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚81324225763328 BDeviceStatsMonitor.on_train_batch_start/allocation.all.current‚26088 SGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚6179395 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 P@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 13568114688 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚642 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚640 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 JBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚286 XEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚81149313124864 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚31929 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚31684 SEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 384795648 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 152061440 _MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 1023465996288 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 248284160 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 586906112 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 599785472 @8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚895 IDeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚3689 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚24903680 UBDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated‚81149313124864 PCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚24920253 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚31929 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 YFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚82347691759616 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 599785472 VEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 978435547136 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 586906112 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚17039360 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚3689 TADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚79416826624117 _LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚78443893701968 H?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚1321 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚26088 PCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚20443265 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚26086 YHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 978435547136 XJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 226592340 WGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 13035530240 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚11791864 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 P@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 13118524025 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚609 G?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚609 TDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 13035530240 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 RBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 13807648768 M@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚45389604 ]JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚82348003415040 QCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 243631700 YHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 972949961509 C;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚895 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 Q>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚81149064840704 _LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚80170646332928 MFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚33 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚6180035 ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 286751744 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 231244800 TADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚81149064840704 M@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚17971259 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 UGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 582240769 [HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚80170629293568 TGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚24946339 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚11791866 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 311655424 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 % _timestamp‚1.7885026201017525e+09 L?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚20443263 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚2 cQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 1023752748032 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚31684 PCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚20443263 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚17039360 [HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚78443876662608 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 JBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚609 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 320831488 ]LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 978666791936 XEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚79417070255817 [KDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 13207863296º¡+¼Î'Í ÈO D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚26088 PCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚20443265 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚26086 YHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 978435547136 XJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 226592340 WGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 13035530240 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚11791864 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 P@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 13118524025 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚609 G?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚609 TDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 13035530240 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 RBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 13807648768 M@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚45389604 ]JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚82348003415040 QCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 243631700 YHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 972949961509 C;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚895 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 Q>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚81149064840704 _LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚80170646332928 MFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚33 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚6180035 ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 286751744 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 231244800 TADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚81149064840704 M@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚17971259 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 UGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 582240769 [HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚80170629293568 TGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚24946339 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚11791866 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 311655424 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 % _timestamp‚1.7885026201017525e+09 L?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚20443263 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚2 cQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 1023752748032 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚31684 PCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚20443263 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚17039360 [HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚78443876662608 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 JBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚609 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 320831488 ]LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 978666791936 XEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚79417070255817 [KDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 13207863296 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚2 \IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚80170646332928 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 WGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 12589930496 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 @8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚895 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 TDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 13807648768 _runtime‚1696.181804211 PCDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated‚24946339 L?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚24920253 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 231244800 IDeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 ]LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 973176553849 O?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 13807648768 TFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 599785472 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 M=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 13568114688 dQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚81324250667008 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 VFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 13207863296 E=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚895 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚286 QDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚17971901 G?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚286 _step‚78 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚26086 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 E8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚45363516 XEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚80170629293568 `MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚81324225763328 BDeviceStatsMonitor.on_train_batch_start/allocation.all.current‚26088 SGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚6179395 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 P@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 13568114688 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚642 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚640 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 JBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚286 XEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚81149313124864 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚31929 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚31684 SEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 384795648 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 152061440 _MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 1023465996288 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 248284160 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 586906112 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 599785472 @8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚895 IDeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚3689 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚24903680 UBDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated‚81149313124864 PCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚24920253 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚31929 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 YFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚82347691759616 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 599785472 VEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 978435547136 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 586906112 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚17039360 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚3689 TADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚79416826624117 _LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚78443893701968 H?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚1321N‚g H2Î - _wandbruntime‚1697  _runtime‚1697#†éOÔ&Ï ÎM FDeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 YFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚80249140502528 UEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 13035530240 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚6185432 MDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚1299 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚895 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 M=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 13807648768 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚286 [JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 979624276480 K>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚17988789 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 NADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚24970950 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚895 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚17039360 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚286 _step‚79 RBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 13035530240 aODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 1024747189760 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 ODeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚609 S@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚81228781818368 K;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 13568114688 R?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚81228533520896 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚2 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚6186039 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 599785472 L>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 248297472 XGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 979624276480 TCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 979393018368 VCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚81228781818368 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚609 trainer/global_step‚999 _runtime‚1697.852283959 PBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 586906112 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚31929 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 UEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 12589930496 P@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 13807648768 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚286 % _timestamp‚1.7885026217698689e+09 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚26114 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚26112 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚3689 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚895 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 231258112 WFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 979393018368 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 320831488 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 599785472 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17039360 VCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚80249140502528 N>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 13568114688 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚26112 YFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚78520723397152 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 582240769 C6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚45408058 ZGDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚80249157541888 OBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚17989398 REDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚24970950 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚31684 QCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 384795648 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚31929 E6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚895 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚2 REDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚20463222 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚20463220 WDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚82428299949056 WGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 13207863296 K>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚45434172 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚609 REDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚11803357 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24903680 N>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 13118524025 G:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚45434172 SEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 586906112 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚609 [MDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚ 267864064 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 VCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚79494869037963 ]JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚78520740436512 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚11803359 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚607 R?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚79494625406055 WFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 973902008903 FDeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 YFDeviceStatsMonitor.on_train_batch_end/allocated_byƒî±d !tes.large_pool.freed‚80249140502528 UEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 13035530240 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚6185432 MDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚1299 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚895 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 M=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 13807648768 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚286 [JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 979624276480 K>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚17988789 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 NADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚24970950 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚895 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚17039360 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚286 _step‚79 RBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 13035530240 aODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 1024747189760 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 ODeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚609 S@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚81228781818368 K;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 13568114688 R?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚81228533520896 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚2 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚6186039 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 599785472 L>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 248297472 XGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 979624276480 TCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 979393018368 VCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚81228781818368 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚609 trainer/global_step‚999 _runtime‚1697.852283959 PBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 586906112 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚31929 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 UEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 12589930496 P@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 13807648768 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚286 % _timestamp‚1.7885026217698689e+09 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚26114 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚26112 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚3689 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚895 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 231258112 WFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 979393018368 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 320831488 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 599785472 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17039360 VCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚80249140502528 N>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 13568114688 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚26112 YFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚78520723397152 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 582240769 C6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚45408058 ZGDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚80249157541888 OBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚17989398 REDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚24970950 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚31684 QCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 384795648 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚31929 E6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚895 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚2 REDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚20463222 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚20463220 WDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚82428299949056 WGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 13207863296 K>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚45434172 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚609 REDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚11803357 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24903680 N>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 13118524025O?k2Ñ - _wandbruntime‚1697  _runtime‚1697<é^TÒ Î .loss/FRt3/detection_loss‚2.393813397936724 .loss/FBt2/detection_loss‚2.375064570903778 epoch‚1 _runtime‚1697.854432998 _step‚80 /loss/FBt1/detection_loss‚2.4381506058477584 &loss/total_loss‚48.563802686585404 trainer/global_step‚999 .loss/FRt1/detection_loss‚2.454575484139579 /loss/SRqqt4/assignment_loss‚1.1066280917825 .loss/FRt2/detection_loss‚2.884059538966731 1loss/SRqqt3/detection_loss‚2.3328054753633647 1loss/FBt1/assignment_loss‚0.19771677151490416 2loss/FBt4/assignment_loss‚0.006786332752098032 /loss/FRt4/detection_loss‚1.7301412188581058 /loss/FRt2/assignment_loss‚4.478048406149211 1loss/SRqqt3/assignment_loss‚2.146576226903842 /loss/FBt3/detection_loss‚0.7857789865849044 /loss/FRt1/assignment_loss‚4.764439737796783 2loss/SRqqt2/assignment_loss‚1.6637035231680086 0loss/FBt2/assignment_loss‚0.3709960161270647 1loss/FBt3/assignment_loss‚0.13102311063013888 1loss/SRqqt1/detection_loss‚2.7225615744497262 0loss/SRqqt2/detection_loss‚2.152776627507928 0loss/SRqqt4/detection_loss‚1.077713468259546 /loss/FRt4/assignment_loss‚2.241175017612321 1loss/SRqqt1/assignment_loss‚2.071614673909019 % _timestamp‚1.7885026217754047e+09 0loss/FRt3/assignment_loss‚2.5967886174047314 /loss/FBt4/detection_loss‚3.4408652120166354ýmÄžXÓ Ò /loss/SRqqt4/assignment_loss‚1.1066280917825 .loss/FRt2/detection_loss‚2.884059538966731 1loss/SRqqt3/detection_loss‚2.3328054753633647 1loss/FBt1/assignment_loss‚0.19771677151490416 2loss/FBt4/assignment_loss‚0.006786332752098032 /loss/FRt4/detection_loss‚1.7301412188581058 /loss/FRt2/assignment_loss‚4.478048406149211 1loss/SRqqt3/assignment_loss‚2.146576226903842 /loss/FBt3/detection_loss‚0.7857789865849044 /loss/FRt1/assignment_loss‚4.764439737796783 2loss/SRqqt2/assignment_loss‚1.6637035231680086 0loss/FBt2/assignment_loss‚0.3709960161270647 1loss/FBt3/assignment_loss‚0.13102311063013888 1loss/SRqqt1/detection_loss‚2.7225615744497262 0loss/SRqqt2/detection_loss‚2.152776627507928 0loss/SRqqt4/detection_loss‚1.077713468259546 /loss/FRt4/assignment_loss‚2.241175017612321 1loss/SRqqt1/assignment_loss‚2.071614673909019 % _timestamp‚1.7885026217754047e+09 0loss/FRt3/assignment_loss‚2.5967886174047314 /loss/FBt4/detection_loss‚3.4408652120166354 .loss/FRt3/detection_loss‚2.393813397936724 .loss/FBt2/detection_loss‚2.375064570903778 epoch‚1 _runtime‚1697.854432998 _step‚80 /loss/FBt1/detection_loss‚2.4381506058477584 &loss/total_loss‚48.563802686585404 trainer/global_step‚999 .loss/FRt1/detection_loss‚2.454575484139579Pc…â“Ô ‚(:ˆ éÄéԙΪº gpu.0.gpu‚25.0 gpu.0.memory‚2, gpu.0.memoryAllocated‚16.718826293945312* gpu.0.memoryAllocatedBytes‚ 14361362432 gpu.0.temp‚31.0 gpu.0.powerWatts‚110.466' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚22.0932 gpu.0.smClock‚1275 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0ÇÄljÕ ‚(:ô éÄéÔ¥½“¼% disk./.usageGB‚1174.6243858337402! disk.nvme0n1.out‚ 39461.890625 disk.dm-1.in‚ 0.6953125 disk.sda.out‚ 184.71875# disk.nvme0c0n1.out‚ 39461.890625 network.sent‚ 2.09281e+06 disk.sda1.out‚0 disk.sda.in‚0.796875 disk.sda2.in‚0" disk.nvme0c0n1.in‚ 4089.4453125) proc.memory.percent‚18.55294704437256 proc.cpu.threads‚17 disk.nvme0n1.in‚ 4089.4453125 disk.dm-0.in‚ 4089.4453125 disk.dm-0.out‚ 39461.890625 disk.drbd1260.in‚0 disk.drbd1260.out‚ 0.23046875 disk.sda1.in‚ 0.1015625' proc.memory.availableMB‚ 343.1328125 cpu‚25.721830553484942 network.recv‚618936) disk./.usagePercent‚78.83002810880804 disk.dm-1.out‚ 184.71875 disk.sda2.out‚0 disk.sda3.in‚ 0.6953125 disk.sda3.out‚ 184.71875$ memory_percent‚97.90568351745605# proc.memory.rssMB‚ 3039.71484375­êÊþžÖ ‚(:“ øÄéÔÏij¹ gpu.0.gpu‚30.0 gpu.0.memory‚3, gpu.0.memoryAllocated‚16.718826293945312* gpu.0.memoryAllocatedBytes‚ 14361362432 gpu.0.temp‚32.0 gpu.0.powerWatts‚114.594' gpu.0.enforcedPowerLimitWatts‚500.0) gpu.0.powerPercent‚22.918799999999997 gpu.0.smClock‚1275 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0)6Š¾•× ‚(:Š øÄéÔÔ³» disk.sda.in‚0.796875 proc.cpu.threads‚17 network.sent‚ 2.180876e+06 disk.nvme0n1.in‚ 4231.359375 disk.dm-0.in‚ 4231.359375 disk.drbd1260.out‚ 0.23828125 disk.sda.out‚ 185.27734375$ memory_percent‚97.90432453155518( proc.memory.availableMB‚ 343.35546875$ disk./.usageGB‚1174.617862701416 disk.sda2.out‚0 disk.sda3.out‚ 185.27734375! disk.nvme0c0n1.in‚ 4231.359375& disk.nvme0n1.out‚39852.41845703125# disk.dm-0.out‚39852.41845703125 disk.dm-1.out‚ 185.27734375 disk.sda1.in‚ 0.1015625) disk./.usagePercent‚78.82959033592448 disk.sda2.in‚0 disk.sda3.in‚ 0.6953125( disk.nvme0c0n1.out‚39852.41845703125 disk.drbd1260.in‚0# proc.memory.rssMB‚ 3039.71484375) proc.memory.percent‚18.55294704437256 cpu‚25.720399014594683 network.recv‚625026 disk.dm-1.in‚ 0.6953125 disk.sda1.out‚0eq2Ú’Ø ‚(:‡ ‡ÅéÔÒ …º gpu.0.gpu‚1.0 gpu.0.memory‚0, gpu.0.memoryAllocated‚16.718826293945312* gpu.0.memoryAllocatedBytes‚ 14361362432 gpu.0.temp‚31.0 gpu.0.powerWatts‚112.742' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚22.5484 gpu.0.smClock‚1275 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0i¦¬È–Ù ‚(:‹ ‡ÅéԶ disk.sda.out‚ 185.82421875 disk.sda3.in‚ 0.6953125 disk.sda3.out‚ 185.82421875( disk.nvme0c0n1.out‚40213.19189453125( proc.memory.availableMB‚ 343.33203125 network.sent‚ 2.184819e+06 disk.dm-0.in‚ 4231.359375 disk.sda2.in‚0 disk.nvme0n1.in‚ 4231.359375# proc.memory.rssMB‚ 3039.71484375 cpu‚25.718997408611482 disk.dm-1.in‚ 0.6953125 disk.drbd1260.in‚0 disk.sda1.out‚0 disk.sda2.out‚0! disk.nvme0c0n1.in‚ 4231.359375) proc.memory.percent‚18.55294704437256 network.recv‚629701) disk./.usagePercent‚78.83116478226006# disk.dm-0.out‚40213.19189453125 disk.dm-1.out‚ 185.82421875 disk.sda.in‚0.796875& disk.nvme0n1.out‚40213.19189453125$ memory_percent‚97.90446758270264 proc.cpu.threads‚17% disk./.usageGB‚1174.6413230895996 disk.drbd1260.out‚ 0.23828125 disk.sda1.in‚ 0.1015625š ‹‘Ú ‚(:† –ÅéÔ™£ó¹ gpu.0.gpu‚34.0 gpu.0.memory‚4, gpu.0.memoryAllocated‚16.718826293945312* gpu.0.memoryAllocatedBytes‚ 14361362432 gpu.0.temp‚31.0 gpu.0.powerWatts‚93.345' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚18.669 gpu.0.smClock‚1275 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0.M3+‚Û ‚(:÷ –ÅéÔüÌ» disk.sda2.in‚0 disk.sda3.in‚ 0.6953125 network.recv‚634286 disk.drbd1260.out‚ 0.2421875 disk.sda.out‚ 186.30078125! disk.nvme0n1.out‚ 40661.984375 disk.dm-1.out‚ 186.30078125 disk.nvme0n1.in‚ 4231.359375 disk.dm-0.in‚ 4231.359375$ memory_percent‚97.90616035461426# proc.memory.rssMB‚ 3039.71484375 cpu‚25.71756740225884 proc.cpu.threads‚17 disk.sda2.out‚0 network.sent‚ 2.19109e+06( disk./.usagePercent‚78.8353551135805 disk.sda1.in‚ 0.1015625 disk.sda1.out‚0 disk.sda.in‚0.796875! disk.nvme0c0n1.in‚ 4231.359375) proc.memory.percent‚18.55294704437256% disk./.usageGB‚1174.7037620544434 disk.drbd1260.in‚0 disk.sda3.out‚ 186.30078125# disk.nvme0c0n1.out‚ 40661.984375 disk.dm-0.out‚ 40661.984375 disk.dm-1.in‚ 0.6953125' proc.memory.availableMB‚ 343.0546875mÅ&N“Ü ‚(:ˆ ¥ÅéÔ©€îº gpu.0.gpu‚34.0 gpu.0.memory‚3, gpu.0.memoryAllocated‚16.718826293945312* gpu.0.memoryAllocatedBytes‚ 14361362432 gpu.0.temp‚33.0 gpu.0.powerWatts‚163.004' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚32.6008 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0WO©í‰Ý ‚(:þ ¥ÅéÔ¾Á¸¾# disk.nvme0n1.out‚40869.21484375 disk.dm-1.out‚ 186.68359375 disk.sda1.in‚ 0.1015625 disk.sda.in‚0.796875$ memory_percent‚97.90620803833008& proc.memory.availableMB‚ 343.046875 disk.sda2.in‚0 disk.drbd1260.in‚0 disk.drbd1260.out‚ 0.2421875 cpu‚25.71636483433729 proc.cpu.threads‚17) disk./.usagePercent‚78.83797663073113% disk./.usageGB‚1174.7428245544434 disk.sda3.in‚ 0.6953125 disk.sda3.out‚ 186.68359375% disk.nvme0c0n1.out‚40869.21484375 disk.nvme0n1.in‚ 4231.359375 disk.dm-0.in‚ 4231.359375 disk.dm-1.in‚ 0.6953125 network.sent‚ 2.197998e+06 disk.sda2.out‚0! disk.nvme0c0n1.in‚ 4231.359375 disk.dm-0.out‚40869.21484375 disk.sda1.out‚0 disk.sda.out‚ 186.68359375# proc.memory.rssMB‚ 3039.71484375) proc.memory.percent‚18.55294704437256 network.recv‚637601§9v2Þ - _wandbruntime‚1784  _runtime‚1784¦?“1Äß ¾ % lr-AdamW/pg1‚0.001283028038240036 % lr-AdamW/pg2‚0.001283028038240036 trainer/global_step‚1049 % _timestamp‚1.7885027083692183e+09 _runtime‚1784.448256716 _step‚81hýœÅÈà  trainer/global_step‚1049 % _timestamp‚1.7885027083692183e+09 _runtime‚1784.448256716 _step‚81 % lr-AdamW/pg1‚0.001283028038240036 % lr-AdamW/pg2‚0.001283028038240036Q„Õ¿2á - _wandbruntime‚1784  _runtime‚1784­¥øÄÑ'â ËO XEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚84096189741568 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚31684 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 TDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 13807648768 [KDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 13207863296 ^LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 1020806095557 O?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 13807648768 XEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚85122774962688 P@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 13568114688 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 231244800 PCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚26152855 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 WGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 13035530240 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 M@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚18846345 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚17039360 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 586906112 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 TDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 13035530240 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚26088 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 309558272 BDeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚85122526678528 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 WGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 12589930496 VFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 13207863296 OFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚1299 UGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 582240769 G?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚609 cQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 1074006669312 E=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚895 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚286 IDeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 XJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 226592340 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚31929 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚2 TADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚83306792893025 PCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚21441115 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚601 YFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚86377690779648 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 % _timestamp‚1.7885027083707232e+09 _step‚82 PCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚21441113 M=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 13568114688 [HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚84096189741568 IDeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚31684 [IDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 1026568181760 ZHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 1026336936960 M@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚47620056 G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚3689 QCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 243631700 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚12366516 SEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 384795648 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 ZHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 1020579503217 RBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 13807648768 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 248284160 L?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚21441113 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 231244800 TGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚26178941 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 C;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚895 JBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚609n`À»-ã ÏO XKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚12366516 SEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 384795648 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 ZHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 1020579503217 RBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 13807648768 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 248284160 L?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚21441113 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 231244800 TGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚26178941 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 C;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚895 JBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚609 XEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚84096189741568 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚31684 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 TDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 13807648768 [KDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 13207863296 ^LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 1020806095557 O?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 13807648768 XEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚85122774962688 P@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 13568114688 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 231244800 PCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚26152855 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 WGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 13035530240 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 M@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚18846345 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚17039360 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 586906112 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 TDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 13035530240 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚26088 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 309558272 BDeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚85122526678528 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 WGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 12589930496 VFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 13207863296 OFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚1299 UGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 582240769 G?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚609 cQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 1074006669312 E=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚895 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚286 IDeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 XJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 226592340 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚31929 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚2 TADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚83306792893025 PCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚21441115 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚601 YFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚86377690779648 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 % _timestamp‚1.7885027083707232e+09 _step‚82 PCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚21441113 M=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 13568114688 [HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚84096189741568 IDeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚31684 [IDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 1026568181760 ZHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 1026336936960 M@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚47620056 G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚3689 QCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 243631700RÉáA÷“ä ‚(:ˆ ´ÅéÔÁðæº gpu.0.gpu‚62.0 gpu.0.memory‚5, gpu.0.memoryAllocated‚16.718826293945312* gpu.0.memoryAllocatedBytes‚ 14361362432 gpu.0.temp‚34.0 gpu.0.powerWatts‚162.623' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚32.5246 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0ù%°’å ‚(:‡ ´ÅéÔ“øƒ¾ disk.nvme0n1.in‚ 4384.515625 disk.sda.out‚ 187.10546875& proc.memory.availableMB‚ 342.984375) proc.memory.percent‚18.55294704437256 proc.cpu.threads‚17( disk./.usagePercent‚78.8467564091179 disk.sda2.in‚0 disk.sda3.out‚ 187.10546875& disk.nvme0n1.out‚41268.71630859375 disk.dm-0.in‚ 4384.515625 disk.drbd1260.in‚0 disk.drbd1260.out‚ 0.24609375 cpu‚25.715014882446727( disk.nvme0c0n1.out‚41268.71630859375 disk.sda1.out‚0 disk.sda.in‚0.796875$ memory_percent‚97.90658950805664# proc.memory.rssMB‚ 3039.71484375 network.sent‚ 2.214392e+06 network.recv‚641573$ disk./.usageGB‚1174.873649597168 disk.sda3.in‚ 0.6953125# disk.dm-0.out‚41268.84130859375 disk.dm-1.in‚ 0.6953125 disk.dm-1.out‚ 187.10546875 disk.sda1.in‚ 0.1015625 disk.sda2.out‚0! disk.nvme0c0n1.in‚ 4384.515625¯·2æ - _wandbruntime‚1785  _runtime‚1785ö?Û&ç ÕM N>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 13118524025 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 C6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚47638510 K;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 13568114688 YFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚84174700950528 K>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚47664624 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚895 REDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚26203552 ZGDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚84174717989888 SEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 586906112 QCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 384795648 ]KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 1074734262784 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 trainer/global_step‚1049 YFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚82363060124352 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚609 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚286 NADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚26177440 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚286 _step‚83 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚31684 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚571 WDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚86458297887744 \JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 1021758143159 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 FDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 OBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚18864432 NADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚21461072 TDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 13207863296 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚609 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚286 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚2 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 599785472 E6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚895 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚21461070 J=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚26177440 VCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚85202243656192 ^KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚85383563624960 P@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 13807648768 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚895 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚2 bODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚85383588528640 [MDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚ 267864064 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 582240769 YIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 13207863296 ODeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚18863861 REDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚12378007 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 _runtime‚1785.390114586 YGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 1027525666304 ]JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚84174717989888 REDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚21461072 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 292767744 N>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 13568114688 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 231258112 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 243631908 S@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚85202243656192 WGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 13207863296 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚31929 MDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚1299 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚31929 NADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚21461070 R?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚85201995358720 G:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚47664624 PBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 586906112 UEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 13035530240 \JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 1027525666304 L>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 248297472 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚17039360 aODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 1075002126848 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 320831488 M=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 13807648768 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚569 R?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚83384591674963 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚895 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 152061440 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0IûÉß&è ÙM OBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚18864432 NADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚21461072 TDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 13207863296 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚609 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚286 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚2 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 599785472 E6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚895 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚21461070 J=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚26177440 VCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚85202243656192 ^KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚85383563624960 P@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 13807648768 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚895 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚2 bODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚85383588528640 [MDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚ 267864064 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 582240769 YIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 13207863296 ODeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚18863861 REDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚12378007 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 _runtime‚1785.390114586 YGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 1027525666304 ]JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚84174717989888 REDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚21461072 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 292767744 N>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 13568114688 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 231258112 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 243631908 S@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚85202243656192 WGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 13207863296 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚31929 MDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚1299 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚31929 NADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚21461070 R?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚85201995358720 G:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚47664624 PBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 586906112 UEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 13035530240 \JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 1027525666304 L>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 248297472 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚17039360 aODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 1075002126848 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 320831488 M=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 13807648768 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚569 R?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚83384591674963 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚895 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 152061440 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 N>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 13118524025 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 C6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚47638510 K;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 13568114688 YFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚84174700950528 K>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚47664624 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚895 REDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚26203552 ZGDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚84174717989888 SEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 586906112 QCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 384795648 ]KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 1074734262784 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 trainer/global_step‚1049 YFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚82363060124352 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚609 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚286 NADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚26177440 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚286 _step‚83 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚31684 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚571 WDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚86458297887744 \JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 1021758143159 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 FDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0SJ³‚ 2é - _wandbruntime‚1785  _runtime‚1785ê£Zê Ô 1loss/SRqqt1/detection_loss‚2.9211418134100895 &loss/total_loss‚46.825081988797535 epoch‚1 2loss/SRqqt2/assignment_loss‚1.4681419207114879 0loss/FBt2/assignment_loss‚0.3210102707974264 /loss/FRt2/detection_loss‚3.0455915740891997 /loss/FRt3/detection_loss‚1.9416837826067088 _step‚84 0loss/FRt4/assignment_loss‚1.9816815757145316 0loss/SRqqt1/assignment_loss‚1.96571561789259 2loss/SRqqt4/assignment_loss‚1.1090166509071955 1loss/FBt4/assignment_loss‚0.27334474356885524 0loss/SRqqt2/detection_loss‚2.264943626237242 .loss/FBt2/detection_loss‚2.321900365182332 2loss/SRqqt3/assignment_loss‚2.3281076218594206 1loss/FBt1/assignment_loss‚0.20911441481681928 1loss/FBt3/assignment_loss‚0.19274586558849477 /loss/FRt1/detection_loss‚2.5595378618973954 /loss/FRt4/detection_loss‚1.7193831204736636 .loss/FBt4/detection_loss‚2.395392028724446 trainer/global_step‚1049 1loss/SRqqt3/detection_loss‚2.1157428859309717 _runtime‚1785.392325842 /loss/FBt1/detection_loss‚3.0789685721198716 /loss/FBt3/detection_loss‚0.8535060179650995 % _timestamp‚1.7885027093137004e+09 /loss/FRt1/assignment_loss‚4.400333222976098 /loss/FRt2/assignment_loss‚4.093754967053731 /loss/FRt3/assignment_loss‚2.292220124784781 1loss/SRqqt4/detection_loss‚0.9721033434890877ÙÑ"š^ë Ø /loss/FBt1/detection_loss‚3.0789685721198716 /loss/FBt3/detection_loss‚0.8535060179650995 % _timestamp‚1.7885027093137004e+09 /loss/FRt1/assignment_loss‚4.400333222976098 /loss/FRt2/assignment_loss‚4.093754967053731 /loss/FRt3/assignment_loss‚2.292220124784781 1loss/SRqqt4/detection_loss‚0.9721033434890877 1loss/SRqqt1/detection_loss‚2.9211418134100895 &loss/total_loss‚46.825081988797535 epoch‚1 2loss/SRqqt2/assignment_loss‚1.4681419207114879 0loss/FBt2/assignment_loss‚0.3210102707974264 /loss/FRt2/detection_loss‚3.0455915740891997 /loss/FRt3/detection_loss‚1.9416837826067088 _step‚84 0loss/FRt4/assignment_loss‚1.9816815757145316 0loss/SRqqt1/assignment_loss‚1.96571561789259 2loss/SRqqt4/assignment_loss‚1.1090166509071955 1loss/FBt4/assignment_loss‚0.27334474356885524 0loss/SRqqt2/detection_loss‚2.264943626237242 .loss/FBt2/detection_loss‚2.321900365182332 2loss/SRqqt3/assignment_loss‚2.3281076218594206 1loss/FBt1/assignment_loss‚0.20911441481681928 1loss/FBt3/assignment_loss‚0.19274586558849477 /loss/FRt1/detection_loss‚2.5595378618973954 /loss/FRt4/detection_loss‚1.7193831204736636 .loss/FBt4/detection_loss‚2.395392028724446 trainer/global_step‚1049 1loss/SRqqt3/detection_loss‚2.1157428859309717 _runtime‚1785.392325842T|Åk<“ì ‚(:ˆ ÃÅéÔý·ò¹ gpu.0.gpu‚34.0 gpu.0.memory‚4, gpu.0.memoryAllocated‚16.718826293945312* gpu.0.memoryAllocatedBytes‚ 14361362432 gpu.0.temp‚33.0 gpu.0.powerWatts‚116.868' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚23.3736 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0º»Å”í ‚(:‰ ÃÅéÔŸï®» disk.sda3.out‚ 187.54296875 disk.sda1.out‚0 disk.sda2.out‚0 cpu‚25.713887880315784 proc.cpu.threads‚17 disk.dm-1.in‚ 0.6953125 disk.dm-1.out‚ 187.54296875 network.sent‚ 2.276855e+06! disk.nvme0c0n1.in‚ 4384.515625 disk.nvme0n1.in‚ 4384.515625& disk.nvme0n1.out‚41797.52490234375 disk.dm-0.in‚ 4384.515625 disk.sda1.in‚ 0.1015625 network.recv‚647954$ disk./.usageGB‚1174.873649597168# disk.dm-0.out‚41797.52490234375 disk.drbd1260.out‚ 0.24609375 disk.sda.out‚ 187.54296875 disk.sda2.in‚0$ memory_percent‚97.90675640106201# proc.memory.rssMB‚ 3039.71484375( disk./.usagePercent‚78.8467564091179 disk.sda3.in‚ 0.6953125( disk.nvme0c0n1.out‚41797.52490234375 disk.drbd1260.in‚0 disk.sda.in‚0.796875( proc.memory.availableMB‚ 342.95703125) proc.memory.percent‚18.55294704437256Ð-—Žî ‚(:ƒ ÒÅéÔ… gpu.0.gpu‚4.0 gpu.0.memory‚0, gpu.0.memoryAllocated‚16.718826293945312* gpu.0.memoryAllocatedBytes‚ 14361362432 gpu.0.temp‚33.0 gpu.0.powerWatts‚163.1' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚32.62 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0¸Æa•Mï ‚(:ú ÒÅéÔ“ܬº disk.dm-1.out‚ 187.90625 disk.sda1.in‚ 0.1015625 disk.sda.out‚ 187.90625$ memory_percent‚97.90832996368408( proc.memory.availableMB‚ 342.69921875 network.sent‚ 2.35338e+06 disk.sda3.in‚ 0.6953125 disk.nvme0n1.in‚ 4384.515625 disk.dm-0.in‚ 4384.515625# disk.dm-0.out‚42222.21337890625# proc.memory.rssMB‚ 3039.71484375) proc.memory.percent‚18.55294704437256 cpu‚25.71299992162178 disk.sda2.in‚0 disk.sda2.out‚0 disk.sda3.out‚ 187.90625( disk.nvme0c0n1.out‚42222.21337890625 proc.cpu.threads‚17 network.recv‚654699& disk.nvme0n1.out‚42222.21337890625 disk.dm-1.in‚ 0.6953125 disk.drbd1260.in‚0 disk.drbd1260.out‚0.25 disk.sda1.out‚0 disk.sda.in‚0.796875) disk./.usagePercent‚78.86195916053123% disk./.usageGBR~‚™8‚1175.1001815795898! disk.nvme0c0n1.in‚ 4384.515625¸ClÝ“ð ‚(:ˆ áÅéÔŠŠý¸ gpu.0.gpu‚49.0 gpu.0.memory‚5, gpu.0.memoryAllocated‚16.718826293945312* gpu.0.memoryAllocatedBytes‚ 14361362432 gpu.0.temp‚34.0 gpu.0.powerWatts‚163.243' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚32.6486 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0ºð€•ñ ‚(:„ áÅéÔ®Ëܺ) disk./.usagePercent‚78.86510907723252 disk.dm-1.in‚ 0.6953125 disk.drbd1260.in‚0 disk.sda1.out‚0( proc.memory.availableMB‚ 342.91015625# proc.memory.rssMB‚ 3039.71484375) proc.memory.percent‚18.55294704437256 disk.sda.out‚ 188.35546875 disk.sda3.in‚ 0.6953125 cpu‚25.711686430879926 proc.cpu.threads‚17 network.sent‚ 2.357519e+06( disk.nvme0c0n1.out‚42525.49462890625 disk.nvme0n1.in‚ 4384.515625& disk.nvme0n1.out‚42525.49462890625 disk.dm-0.in‚ 4384.515625 disk.dm-1.out‚ 188.35546875 disk.drbd1260.out‚0.25 disk.sda2.out‚0$ disk./.usageGB‚1175.147117614746! disk.nvme0c0n1.in‚ 4384.515625# disk.dm-0.out‚42525.49462890625 disk.sda1.in‚ 0.1015625 disk.sda.in‚0.796875 disk.sda2.in‚0 disk.sda3.out‚ 188.35546875$ memory_percent‚97.90704250335693 network.recv‚657385j•J“ò ‚(:ˆ ðÅéÔ¸ÖÖ¹ gpu.0.gpu‚30.0 gpu.0.memory‚2, gpu.0.memoryAllocated‚16.718826293945312* gpu.0.memoryAllocatedBytes‚ 14361362432 gpu.0.temp‚33.0 gpu.0.powerWatts‚135.334' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚27.0668 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0ièh–ó ‚(:‹ ðÅéÔÆÊ»& disk.nvme0n1.out‚42782.13525390625# disk.dm-0.out‚42782.13525390625 disk.dm-1.in‚ 0.6953125 disk.sda1.out‚0 disk.sda.in‚0.796875 disk.sda.out‚ 188.8359375 disk.sda3.in‚ 0.6953125 disk.sda3.out‚ 188.8359375 network.sent‚ 2.362622e+06) disk./.usagePercent‚78.86510574913457 disk.dm-1.out‚ 188.8359375 disk.drbd1260.out‚ 0.25390625 disk.sda2.out‚0# disk.nvme0c0n1.in‚ 4523.96484375( disk.nvme0c0n1.out‚42782.13525390625# proc.memory.rssMB‚ 3039.71484375! disk.nvme0n1.in‚ 4523.96484375 disk.dm-0.in‚ 4523.96484375 disk.sda1.in‚ 0.1015625 disk.sda2.in‚0$ memory_percent‚97.90706634521484) proc.memory.percent‚18.55294704437256 cpu‚25.710512903577442 proc.cpu.threads‚17 network.recv‚661125% disk./.usageGB‚1175.1470680236816 disk.drbd1260.in‚0% proc.memory.availableMB‚ 342.906259H€“ô ‚(:ˆ ÿÅéԚ建 gpu.0.gpu‚14.0 gpu.0.memory‚1, gpu.0.memoryAllocated‚16.718826293945312* gpu.0.memoryAllocatedBytes‚ 14361362432 gpu.0.temp‚32.0 gpu.0.powerWatts‚115.521' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚23.1042 gpu.0.smClock‚1275 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0Nædõ ‚(:… ÿÅéÔüɽ% disk./.usageGB‚1175.6490859985352 disk.sda3.in‚ 0.6953125 disk.sda3.out‚ 189.21875# disk.nvme0c0n1.in‚ 4523.96484375 disk.dm-0.in‚ 4523.96484375 disk.dm-1.in‚ 0.6953125 disk.drbd1260.in‚0 disk.sda1.out‚0) disk./.usagePercent‚78.89879659664813 disk.sda2.out‚0# disk.dm-0.out‚43206.68603515625 disk.drbd1260.out‚ 0.25390625 disk.sda1.in‚ 0.1015625% proc.memory.availableMB‚ 342.90625) proc.memory.percent‚18.55294704437256 proc.cpu.threads‚17 network.recv‚665352( disk.nvme0c0n1.out‚43206.68603515625& disk.nvme0n1.out‚43206.68603515625 disk.dm-1.out‚ 189.21875$ memory_percent‚97.90706634521484 network.sent‚ 2.368537e+06 disk.sda2.in‚0! disk.nvme0n1.in‚ 4523.96484375 disk.sda.in‚0.796875 disk.sda.out‚ 189.21875# proc.memory.rssMB‚ 3039.71484375 cpu‚25.709340616571946ž›!2ö - _wandbruntime‚1874  _runtime‚1874xP¥úÄ÷ ¾ _runtime‚1874.311887078 _step‚85 % lr-AdamW/pg1‚0.001277971572249279 % lr-AdamW/pg2‚0.001277971572249279 trainer/global_step‚1099 % _timestamp‚1.7885027982316265e+09@í^—Èø  % lr-AdamW/pg2‚0.001277971572249279 trainer/global_step‚1099 % _timestamp‚1.7885027982316265e+09 _runtime‚1874.311887078 _step‚85 % lr-AdamW/pg1‚0.001277971572249279U{¨š2ù - _wandbruntime‚1874  _runtime‚1874…wˆXÑ'ú ËO LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 599785472 PCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚27385457 cQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 1124134138368 ^LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 1068435637265 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 P@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 13118524025 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 UBDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated‚89096236800512 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚17039360 @8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚895 XEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚88021750189568 dQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚89283736670208 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 152061440 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 BDeviceStatsMonitor.on_train_batch_start/allocation.all.current‚26088 G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚3689 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚26086 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚31684 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚595 H?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚1321 OFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚1299 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚609 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚17039360 [HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚88021750189568 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚12941166 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 G?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚609 % _timestamp‚1.7885027982353497e+09 _step‚86 O?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 13807648768 YIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 13207863296 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚3689 QDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚19724205 P@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 13568114688 PCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚22438963 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚12941164 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 320831488 [IDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 1074469571584 TGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚22438965 SEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 384795648 `MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚89283711766528 _MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 1123853678080 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 E8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚49824420 SGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚6782446 VFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 13207863296 TFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 599785472 JBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚609 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 586906112 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚26086 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚6783039 ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 280460288 TGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚27411543 ZHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 1068209044925 WEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 1074238326784 TADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚89095988516352 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚24903680 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 WGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 12589930496 M=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 13568114688 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 L?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚22438963 TDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 13035530240 _LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚88021767228928 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 231244800 YFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚90407565444608 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 [HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚86128550117008 N@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 248284160 M@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚49850508 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 QCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 243631700 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚31929 PCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚22438965 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚31684 WGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 13035530240 M@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚19723610 XEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚87197002793633 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 trainer/global_step‚1099 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 248284160 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 TADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚87196759161933 _LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚86128567156368 TDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 13807648768 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 231244800 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚593 IDeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 RBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 13807648768 Q>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚89095988516352 UGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 582240769 _runtime‚1874.316729571 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 G?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚286 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚2 ]JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚90407870808576 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 599785472 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚286 JBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚286"² ÙÕ'û ÏO @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 BDeviceStatsMonitor.on_train_batch_start/allocation.all.current‚26088 G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚3689 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚26086 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚31684 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚595 H?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚1321 OFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚1299 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚609 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚17039360 [HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚88021750189568 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚12941166 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 G?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚609 % _timestamp‚1.7885027982353497e+09 _step‚86 O?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 13807648768 YIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 13207863296 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚3689 QDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚19724205 P@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 13568114688 PCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚22438963 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚12941164 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 320831488 [IDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 1074469571584 TGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚22438965 SEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 384795648 `MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚89283711766528 _MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 1123853678080 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 E8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚49824420 SGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚6782446 VFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 13207863296 TFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 599785472 JBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚609 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 586906112 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚26086 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚6783039 ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 280460288 TGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚27411543 ZHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 1068209044925 WEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 1074238326784 TADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚89095988516352 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚24903680 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 WGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 12589930496 M=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 13568114688 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 L?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚22438963 TDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 13035530240 _LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚88021767228928 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 231244800 YFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚90407565444608 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 [HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚86128550117008 N@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 248284160 M@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚49850508 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 QCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 243631700 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚31929 PCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚22438965 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚31684 WGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 13035530240 M@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚19723610 XEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚87197002793633 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 trainer/global_step‚1099 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 248284160 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 TADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚87196759161933 _LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚86128567156368 TDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 13807648768 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 231244800 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚593 IDeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 RBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 13807648768 Q>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚89095988516352 UGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 582240769 _runtime‚1874.316729571 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 G?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚286 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚2 ]JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚90407870808576 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 599785472 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚286 JBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚286 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 599785472 PCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚27385457 cQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 1124134138368 ^LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 1068435637265 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 P@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 13118524025 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 UBDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated‚89096236800512 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚17039360 @8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚895 XEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚88021750189568 dQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚89283736670208 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 152061440V&‹"¸‘ü ‚(:† ŽÆéÔƒªá» gpu.0.gpu‚22.0 gpu.0.memory‚2, gpu.0.memoryAllocated‚16.718826293945312* gpu.0.memoryAllocatedBytes‚ 14361362432 gpu.0.temp‚31.0 gpu.0.powerWatts‚105.89' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚21.178 gpu.0.smClock‚1275 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0«Q›ý ‚(: ŽÆéÔõÖÿ¼ disk.drbd1260.out‚ 0.26171875 disk.sda1.out‚0 proc.cpu.threads‚17) disk./.usagePercent‚78.89866116866251% disk./.usageGB‚1175.6470680236816# disk.nvme0c0n1.in‚ 4523.96484375( disk.nvme0c0n1.out‚43655.37158203125& disk.nvme0n1.out‚43655.37158203125 disk.dm-1.in‚ 0.6953125 disk.sda.out‚ 189.58984375$ memory_percent‚97.90711402893066 disk.sda2.in‚0 disk.sda3.out‚ 189.58984375! disk.nvme0n1.in‚ 4523.96484375 disk.drbd1260.in‚0 disk.sda.in‚0.796875' proc.memory.availableMB‚ 342.8984375) proc.memory.percent‚18.55294704437256 network.recv‚670008 disk.sda2.out‚0# disk.dm-0.out‚43655.37158203125 disk.dm-1.out‚ 189.58984375 disk.sda1.in‚ 0.1015625# proc.memory.rssMB‚ 3039.71484375 cpu‚25.708358231331733 network.sent‚ 2.374754e+06 disk.sda3.in‚ 0.6953125 disk.dm-0.in‚ 4523.96484375 ~¼¤2þ - _wandbruntime‚1876  _runtime‚1876V]EÝïÿ ÔM H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚599 \JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 1069387684867 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚17039360 \JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 1075427056128 SEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 586906112 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚31684 [HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚90488472496640 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 P@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 13807648768 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚286 UEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 13035530240 K>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚19741137 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 317933568 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚286 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚286 _runtime‚1876.771883904 S@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚89175705494016 VCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚88100261398528 MDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚1299 ^KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚89363306626560 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚3689 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚6788480 OADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 248297472 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 WIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 599785472 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 F6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚895 ZGDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚88100278437888 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚597 [MDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚ 293029888 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚2 R?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚89175457196544 OBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚19741736 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 K;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 13568114688 G:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚49868962 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚6789077 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 YFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚88100261398528 XFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 1075195798016 K>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚49895076 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24903680 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚895 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 C6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚49868962 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 231258112 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚33 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 BDeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 248297472 ]JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚88100278437888 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 NADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚27410042 ]KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 1124847936512 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 243631908 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚609 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚12952659 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚2 ]JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚86205413890912 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚609 trainer/global_step‚1099 G:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚49895076 NADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚22458920 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 R?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚87274557 áÔýë 943871 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚31929 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚895 REDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚27436154 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1321 bODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚89363331530240 VCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚87274801575779 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 VCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚89175705494016 REDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚22458922 WDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚90488154563072 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 VHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 226592548 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 599785472 NADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚22458922 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚22458920 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 231258112 WGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 13207863296 UCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 1075195798016 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚31929 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 N>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 13568114688 FDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 ODeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 13118524025 M=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 13807648768 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17039360 YIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 13207863296 E6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚895 ZGDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚88100278437888 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚597 [MDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚ 293029888 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚2 R?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚89175457196544 OBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚19741736 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 K;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 13568114688 G:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚49868962 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚6789077 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 YFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚88100261398528 XFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 1075195798016 K>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚49895076 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24903680 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚895 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 C6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚49868962 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 231258112 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚33 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 BDeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 248297472 ]JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚88100278437888 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 NADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚27410042 ]KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 1124847936512 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 243631908 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚609 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚12952659 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚2 ]JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚86205413890912 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚609 trainer/global_step‚1099 G:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚49895076 NADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚22458920 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 R?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚87274557943871 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚31929 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚895 REDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚27436154 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1321 bODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚89363331530240 VCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚87274801575779 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 VCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚89175705494016 REDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚22458922 WDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚90488154563072 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 VHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 226592548 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 599785472 NADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚22458922 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚22458920 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 231258112 WGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 13207863296 UCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 1075195798016 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚31929 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 N>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 13568114688 FDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 ODeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 13118524025 M=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 13807648768 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17039360 YIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 13207863296 EDeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚19741137 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 317933568 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚286 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚286 _runtime‚1876.771883904 S@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚89175705494016 VCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚88100261398528 MDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚1299W MùÒ2 - _wandbruntime‚1876  _runtime‚1876~¥V‚ Ð 0loss/FRt3/assignment_loss‚2.4531304285496094 .loss/FBt2/assignment_loss‚0.29717673425214 1loss/SRqqt3/detection_loss‚2.8438681871504397 epoch‚1 trainer/global_step‚1099 _step‚88 /loss/FRt1/assignment_loss‚3.784873788036517 0loss/FRt4/assignment_loss‚1.9704485535621643 0loss/FBt1/assignment_loss‚0.1791648408625406 /loss/FBt1/detection_loss‚2.5615729416410127 2loss/SRqqt2/assignment_loss‚1.8300921535246992 2loss/SRqqt4/assignment_loss‚1.1578060563655577 1loss/FBt4/assignment_loss‚0.42435208830813115 0loss/SRqqt1/detection_loss‚2.530274744828542 1loss/SRqqt3/assignment_loss‚2.527695694686593 .loss/FRt2/detection_loss‚3.074361098055937 1loss/FBt3/assignment_loss‚0.20432453067999035 1loss/SRqqt2/detection_loss‚2.0760787394765305 .loss/FBt2/detection_loss‚2.120305598491714 /loss/FBt3/detection_loss‚0.9108716070500483 %loss/total_loss‚47.00857624910054 /loss/FRt1/detection_loss‚2.6302591931464065 /loss/FBt4/detection_loss‚3.2350575761361555 % _timestamp‚1.7885028006959283e+09 _runtime‚1876.775371535 /loss/FRt2/assignment_loss‚3.837317353608657 0loss/SRqqt4/detection_loss‚0.984565860356193 2loss/SRqqt1/assignment_loss‚1.5340286537694434 .loss/FRt3/detection_loss‚2.119982443844216 /loss/FRt4/detection_loss‚1.7209673827173004„¾ÓhZƒ Ô /loss/FRt2/assignment_loss‚3.837317353608657 0loss/SRqqt4/detection_loss‚0.984565860356193 2loss/SRqqt1/assignment_loss‚1.5340286537694434 .loss/FRt3/detection_loss‚2.119982443844216 /loss/FRt4/detection_loss‚1.7209673827173004 0loss/FRt3/assignment_loss‚2.4531304285496094 .loss/FBt2/assignment_loss‚0.29717673425214 1loss/SRqqt3/detection_loss‚2.8438681871504397 epoch‚1 trainer/global_step‚1099 _step‚88 /loss/FRt1/assignment_loss‚3.784873788036517 0loss/FRt4/assignment_loss‚1.9704485535621643 0loss/FBt1/assignment_loss‚0.1791648408625406 /loss/FBt1/detection_loss‚2.5615729416410127 2loss/SRqqt2/assignment_loss‚1.8300921535246992 2loss/SRqqt4/assignment_loss‚1.1578060563655577 1loss/FBt4/assignment_loss‚0.42435208830813115 0loss/SRqqt1/detection_loss‚2.530274744828542 1loss/SRqqt3/assignment_loss‚2.527695694686593 .loss/FRt2/detection_loss‚3.074361098055937 1loss/FBt3/assignment_loss‚0.20432453067999035 1loss/SRqqt2/detection_loss‚2.0760787394765305 .loss/FBt2/detection_loss‚2.120305598491714 /loss/FBt3/detection_loss‚0.9108716070500483 %loss/total_loss‚47.00857624910054 /loss/FRt1/detection_loss‚2.6302591931464065 /loss/FBt4/detection_loss‚3.2350575761361555 % _timestamp‚1.7885028006959283e+09 _runtime‚1876.775371535Xh ò‘„ ‚(:† ÆéÔ£–º gpu.0.gpu‚0.0 gpu.0.memory‚0, gpu.0.memoryAllocated‚16.718826293945312* gpu.0.memoryAllocatedBytes‚ 14361362432 gpu.0.temp‚31.0 gpu.0.powerWatts‚93.386' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚18.6772 gpu.0.smClock‚1275 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0&³ˆD›… ‚(: ÆéÔ‘З»# disk.dm-0.out‚44143.62548828125 disk.drbd1260.in‚0 disk.sda2.out‚0 disk.sda3.out‚ 190.07421875$ memory_percent‚97.90740013122559) disk./.usagePercent‚78.89866116866251 disk.drbd1260.out‚ 0.26171875 disk.sda3.in‚ 0.6953125# disk.nvme0c0n1.in‚ 4523.96484375( disk.nvme0c0n1.out‚44143.62548828125) proc.memory.percent‚18.55294704437256 proc.cpu.threads‚17% disk./.usageGB‚1175.6470680236816! disk.nvme0n1.in‚ 4523.96484375 disk.sda1.in‚ 0.1015625 disk.sda.out‚ 190.07421875 disk.sda2.in‚0' proc.memory.availableMB‚ 342.8515625# proc.memory.rssMB‚ 3039.71484375 cpu‚25.705762948335042 network.recv‚675603 disk.dm-1.in‚ 0.6953125 disk.dm-1.out‚ 190.07421875 disk.sda1.out‚0 disk.sda.in‚0.796875 network.sent‚ 2.449776e+06& disk.nvme0n1.out‚44143.62548828125 disk.dm-0.in‚ 4523.96484375Ü;D¢“† ‚(:ˆ ¬ÆéԦʵ¹ gpu.0.gpu‚53.0 gpu.0.memory‚6, gpu.0.memoryAllocated‚16.718826293945312* gpu.0.memoryAllocatedBytes‚ 14361362432 gpu.0.temp‚34.0 gpu.0.powerWatts‚168.771' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚33.7542 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0Z?åk‰‡ ‚(:þ ¬ÆéÔ“º» network.sent‚ 2.525305e+06 network.recv‚681133$ disk./.usageGB‚1175.647174835205 disk.sda2.in‚0" disk.nvme0c0n1.in‚ 4664.9296875 disk.nvme0n1.in‚ 4664.9296875 disk.dm-0.in‚ 4664.9296875 disk.drbd1260.in‚0 disk.sda.out‚ 190.4453125 disk.sda3.in‚ 0.6953125 disk.sda3.out‚ 190.4453125 disk.drbd1260.out‚0.265625 disk.sda1.out‚0# memory_percent‚97.9074239730835# proc.memory.rssMB‚ 3039.71484375) proc.memory.percent‚18.55294704437256 disk.sda.in‚0.796875# disk.nvme0n1.out‚44471.08203125 disk.dm-1.in‚ 0.6953125 disk.sda1.in‚ 0.1015625 cpu‚25.703543182654514) disk./.usagePercent‚78.89866833687348 disk.sda2.out‚0% disk.nvme0c0n1.out‚44471.08203125 disk.dm-0.out‚44471.08203125 disk.dm-1.out‚ 190.4453125( proc.memory.availableMB‚ 342.84765625 proc.cpu.threads‚17&¹^zžˆ ‚(:“ »ÆéǪ̂˜¹ gpu.0.gpu‚56.0 gpu.0.memory‚6, gpu.0.memoryAllocated‚16.718826293945312* gpu.0.memoryAllocatedBytes‚ 14361362432 gpu.0.temp‚34.0 gpu.0.powerWatts‚147.744' gpu.0.enforcedPowerLimitWatts‚500.0) gpu.0.powerPercent‚29.548799999999996 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0L¬¶Ž‰ ‚(:ƒ »ÆéÔýãº( proc.memory.availableMB‚ 342.10546875# proc.memory.rssMB‚ 3039.71484375 cpu‚25.701492373526772" disk.nvme0n1.out‚ 44775.5234375 network.sent‚ 2.531232e+06 disk.sda2.out‚0 disk.sda3.in‚ 0.6953125 disk.sda3.out‚ 190.87890625# disk.nvme0c0n1.in‚ 4701.40234375$ disk.nvme0c0n1.out‚ 44775.5234375! disk.nvme0n1.in‚ 4701.40234375 network.recv‚685418) disk./.usagePercent‚78.82573972660197 disk.sda2.in‚0 disk.dm-1.out‚ 190.87890625 disk.drbd1260.in‚0 disk.drbd1260.out‚0.265625 disk.sda1.out‚0 disk.sda.in‚0.796875 disk.dm-0.out‚ 44775.5234375% disk./.usageGB‚1174.5604858398438 disk.dm-1.in‚ 0.6953125 disk.sda.out‚ 190.87890625) proc.memory.percent‚18.55294704437256 proc.cpu.threads‚17 disk.dm-0.in‚ 4701.40234375 disk.sda1.in‚ 0.1015625$ memory_percent‚97.91195392608643u—D“Š ‚(:ˆ ÊÆéÔî„ ¹ gpu.0.gpu‚50.0 gpu.0.memory‚6, gpu.0.memoryAllocated‚16.718826293945312* gpu.0.memoryAllocatedBytes‚ 14361362432 gpu.0.temp‚34.0 gpu.0.powerWatts‚165.218' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚33.0436 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0a`ë4›‹ ‚(: ÊÆéÔùüÒº! disk.nvme0n1.in‚ 4701.40234375 disk.dm-0.in‚ 4701.40234375# disk.dm-0.out‚45098.46337890625 disk.drbd1260.in‚0 disk.drbd1260.out‚ 0.26953125 disk.sda.out‚ 191.37109375 proc.cpu.threads‚17 network.recv‚690198) disk./.usagePercent‚78.82567316464308 disk.sda3.in‚ 0.6953125 disk.sda3.out‚ 191.37109375 disk.dm-1.in‚ 0.6953125 disk.sda1.in‚ 0.1015625 disk.sda1.out‚0 disk.sda.in‚0.796875 network.sent‚ 2.537437e+06( disk.nvme0c0n1.out‚45098.46337890625$ memory_percent‚97.90892601013184' proc.memory.availableMB‚ 342.6015625# proc.memory.rssMB‚ 3039.71484375) proc.memory.percent‚18.55294704437256 cpu‚25.699340530199503% disk./.usageGB‚1174.5594940185547 disk.sda2.in‚0 disk.sda2.out‚0# disk.nvme0c0n1.in‚ 4701.40234375& disk.nvme0n1.out‚45098.46337890625 disk.dm-1.out‚ 191.37109375¥ «œŒ ‚(:‘ ÙÆéÔÈÝÞ¹ gpu.0.gpu‚1.0 gpu.0.memory‚0, gpu.0.memoryAllocated‚16.718826293945312* gpu.0.memoryAllocatedBytes‚ 14361362432 gpu.0.temp‚31.0 gpu.0.powerWatts‚93.564' gpu.0.enforcedPowerLimitWatts‚500.0) gpu.0.powerPercent‚18.712799999999998 gpu.0.smClock‚1275 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0ÓЕ• ‚(:Š ÙÆéÔ–¿» disk.sda1.in‚ 0.1015625 disk.sda1.out‚0$ memory_percent‚97.90842533111572& disk.nvme0n1.out‚45570.00244140625) disk./.usagePercent‚78.82409820629242 disk.sda.out‚ 191.796875 disk.sda3.out‚ 191.796875 cpu‚25.69681957449817 proc.cpu.threads‚17% disk./.usageGB‚1174.5360260009766 disk.sda3.in‚ 0.6953125! disk.nvme0n1.in‚ 4701.40234375( proc.memory.availableMB‚ 342.68359375# proc.memory.rssMB‚ 3039.71484375) proc.memory.percent‚18.55294704437256 network.sent‚ 2.542542e+06 network.recv‚693504 disk.sda2.in‚0 disk.sda2.out‚0# disk.nvme0c0n1.in‚ 4701.40234375 disk.dm-1.in‚ 0.6953125 disk.dm-1.out‚ 191.796875 disk.drbd1260.out‚ 0.26953125 disk.sda.in‚0.796875( disk.nvme0c0n1.out‚45570.00244140625 disk.dm-0.in‚ 4701.40234375# disk.dm-0.out‚45570.00244140625 disk.drbd1260.in‚0}áLE2Ž - _wandbruntime‚1959  _runtime‚1959ιфÅ ¿ & lr-AdamW/pg2‚0.0012723926630114671 trainer/global_step‚1149 % _timestamp‚1.7885028836715987e+09 _runtime‚ 1959.75041009 _step‚89 & lr-AdamW/pg1‚0.0012723926630114671ÙCÉ Ã & lr-AdamW/pg2‚0.0012723926630114671 trainer/global_step‚1149 % _timestamp‚1.7885028836715987e+09 _runtime‚ 1959.75041009 _step‚89 & lr-AdamW/pg1‚0.0012723926630114671Yå‘~f2‘ - _wandbruntime‚1959  _runtime‚1959áOλ$’ ËO P@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 13568114688 L?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚23436813 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 [HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚89970886844208 WGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 13035530240 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚3689 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 JBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚286 MFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚33 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚31929 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 586906112 TGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚28644145 O?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 13807648768 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 M@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚52080960 IDeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚3689 L?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚28618059 ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 286751744 XJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 226592340 \IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚91947327676928 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 599785472 _runtime‚1959.755366805 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚13515816 YIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 13207863296 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚286 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 N@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 248284160 [IDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 1122370961408 dQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚93263479671808 @8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚895 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 ZHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 1122139716608 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 586906112 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚31684 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚13515814 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 E=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚895 G?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚286 IDeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚31684 P@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 13118524025 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚17039360 QDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚20601858 H?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚1321 OFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚1299 TADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚91086725430841 QCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 243631700 C;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚895 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚31929 `MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚93263454768128 BDeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 XEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚91947310637568 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚26088 PCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚28618059 TDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 13035530240 TADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚93069450354176 TDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 13807648768 [KDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 13207863296 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 599785472 VFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 13207863296 PCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚23436815 M=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 13568114688 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚26086 SGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚7085418 cQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 1174304092672 UGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 582240769 @8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚895 WEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.€bŸyÍfreed‚ 1122139716608 _LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚91947327676928 ^LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 1122370961408 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 320831488 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 ZHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 1115838586633 G?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚609 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚2 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 _LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚89970903883568 UBDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated‚93069698638336 Q>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚93069450354176:.3 Õ'“ ÏO XEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚93069698638336 TGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚23436815 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚626 YFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚94437472109056 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 _step‚90 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚31684 P@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 13118524025 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚17039360 QDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚20601858 H?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚1321 OFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚1299 TADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚91086725430841 QCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 243631700 C;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚895 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚31929 `MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚93263454768128 BDeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 XEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚91947310637568 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚26088 PCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚28618059 TDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 13035530240 TADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚93069450354176 TDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 13807648768 [KDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 13207863296 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 599785472 VFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 13207863296 PCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚23436815 M=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 13568114688 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚26086 SGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚7085418 cQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 1174304092672 UGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 582240769 @8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚895 WEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 1122139716608 _LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚91947327676928 ^LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 1122370961408 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 320831488 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 ZHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 1115838586633 G?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚609 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚2 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 _LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚89970903883568 UBDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated‚93069698638336 Q>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚93069450354176 P@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 13568114688 L?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚23436813 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 [HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚89970886844208 WGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 13035530240 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚3689 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 JBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚286 MFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚33 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚31929 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 586906112 TGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚28644145 O?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 13807648768 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 M@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚52080960 IDeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚3689 L?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚28618059 ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 286751744 XJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 226592340 \IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚91947327676928 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 599785472 _runtime‚1959.755366805 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚13515816 YIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 13207863296 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚286 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 N@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 248284160 [IDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 1122370961408 dQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚93263479671808 @8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚895 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 ZHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 1122139716608 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 586906112 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚31684 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚13515814 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 E=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚895 G?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚286 IDeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚286 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚17039360 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 BDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 13118524025 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 599785472 EDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 13568114688 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚2 aODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 1175297911808 P@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 13807648768 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 NADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚28642644 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 M=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 13807648768 % _timestamp‚1.7885028849252782e+09 OBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚20619366 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚33 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 VCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚91164767844687 ]JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚92025838885888 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 152061440 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚26112 PBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 586906112 FDeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚52125528 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚31684 RBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 13807648768 WGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 13207863296 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 599785472 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚609 YGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 1123328445952 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚13527309 NADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚23456772 VCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚92025821846528 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 231258112 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚26112 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 243631908 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚895 VCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚93149167331840 XFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 1123097187840 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚3689 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚895 G:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚52125528 \JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 1123328445952 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚7092057 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚23456770 L>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 248297472 K>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚20618746 XFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 1116790634027 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚895 C6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚52099414 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚17039360 _runtime‚1961.006211976 K;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 13568114688 R?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚91164524212779 OADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 248297472 [HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚94518372443648 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚7091439 trainer/global_step‚1149 _step‚91Š@¶!ß&– ÙM GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚17039360 _runtime‚1961.006211976 K;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 13568114688 R?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚91164524212779 OADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 248297472 [HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚94518372443648 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚7091439 trainer/global_step‚1149 _step‚91 NADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚23456770 MDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚1299 TDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 13207863296 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚26114 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 ]JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚90047750618112 YIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 13207863296 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚286 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚17039360 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 BDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 13118524025 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 599785472 EDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 13568114688 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚2 aODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 1175297911808 P@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 13807648768 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 NADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚28642644 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 M=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 13807648768 % _timestamp‚1.7885028849252782e+09 OBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚20619366 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚33 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 VCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚91164767844687 ]JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚92025838885888 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 152061440 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚26112 PBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 586906112 FDeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚52125528 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚31684 RBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 13807648768 WGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 13207863296 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 599785472 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚609 YGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 1123328445952 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚13527309 NADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚23456772 VCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚92025821846528 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 231258112 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚26112 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 243631908 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚895 VCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚93149167331840 XFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 1123097187840 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚3689 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚895 G:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚52125528 \JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 1123328445952 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚7092057 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚23456770 L>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 248297472 K>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚20618746 XFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 1116790634027 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚895 C6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚52099414[­ð02— - _wandbruntime‚1961  _runtime‚1961ž{Z˜ Ô /loss/FRt3/assignment_loss‚2.199563095764238 0loss/FRt4/assignment_loss‚2.4761244654655457 1loss/FBt2/assignment_loss‚0.38270286039330337 /loss/FRt3/detection_loss‚1.7358182177555805 1loss/SRqqt4/detection_loss‚0.9979952294830112 /loss/FBt1/detection_loss‚2.2354120184213686 _runtime‚1961.008663012 /loss/FRt1/assignment_loss‚4.655251453320186 2loss/SRqqt3/assignment_loss‚2.5210578109536854 0loss/FBt1/assignment_loss‚0.2672853750251913 0loss/SRqqt2/detection_loss‚2.120329190771301 _step‚92 2loss/SRqqt2/assignment_loss‚1.6218401893199264 2loss/SRqqt4/assignment_loss‚1.1420109889303633 /loss/FRt4/detection_loss‚1.8665674404541324 1loss/SRqqt3/detection_loss‚2.5945032477378844 epoch‚1 $ _timestamp‚1.788502884930041e+09 0loss/FRt2/assignment_loss‚3.6644944802478507 /loss/FBt3/detection_loss‚0.8240903863805706 2loss/SRqqt1/assignment_loss‚2.0529728360333532 /loss/FRt1/detection_loss‚2.5618643214305243 0loss/SRqqt1/detection_loss‚2.669618785943625 trainer/global_step‚1149 1loss/FBt3/assignment_loss‚0.13222203527541782 0loss/FBt4/assignment_loss‚0.2148037449934653 /loss/FRt2/detection_loss‚3.0029053445215577 /loss/FBt2/detection_loss‚2.3011057265102863 %loss/total_loss‚46.57503740649369 .loss/FBt4/detection_loss‚2.334498161361331Jh¸]®™ Ø /loss/FRt3/assignment_loss‚2.199563095764238 0loss/FRt4/assignment_loss‚2.4761244654655457 1loss/FBt2/assignment_loss‚0.38270286039330337 /loss/FRt3/dete›2I°ction_loss‚1.7358182177555805 1loss/SRqqt4/detection_loss‚0.9979952294830112 /loss/FBt1/detection_loss‚2.2354120184213686 _runtime‚1961.008663012 /loss/FRt1/assignment_loss‚4.655251453320186 2loss/SRqqt3/assignment_loss‚2.5210578109536854 0loss/FBt1/assignment_loss‚0.2672853750251913 0loss/SRqqt2/detection_loss‚2.120329190771301 _step‚92 2loss/SRqqt2/assignment_loss‚1.6218401893199264 2loss/SRqqt4/assignment_loss‚1.1420109889303633 /loss/FRt4/detection_loss‚1.8665674404541324 1loss/SRqqt3/detection_loss‚2.5945032477378844 epoch‚1 $ _timestamp‚1.788502884930041e+09 0loss/FRt2/assignment_loss‚3.6644944802478507 /loss/FBt3/detection_loss‚0.8240903863805706 2loss/SRqqt1/assignment_loss‚2.0529728360333532 /loss/FRt1/detection_loss‚2.5618643214305243 0loss/SRqqt1/detection_loss‚2.669618785943625 trainer/global_step‚1149 1loss/FBt3/assignment_loss‚0.13222203527541782 0loss/FBt4/assignment_loss‚0.2148037449934653 /loss/FRt2/detection_loss‚3.0029053445215577 /loss/FBt2/detection_loss‚2.3011057265102863 %loss/total_loss‚46.57503740649369 .loss/FBt4/detection_loss‚2.334498161361331\ÔaóÚ“š ‚(:ˆ èÆéÔº¿¦º gpu.0.gpu‚35.0 gpu.0.memory‚4, gpu.0.memoryAllocated‚16.718826293945312* gpu.0.memoryAllocatedBytes‚ 14361362432 gpu.0.temp‚32.0 gpu.0.powerWatts‚114.461' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚22.8922 gpu.0.smClock‚1275 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0<‡%î•› ‚(:Š èÆéÔݦ²» disk.sda3.out‚ 192.17578125# disk.nvme0c0n1.in‚ 4701.40234375% disk.nvme0n1.out‚45961.2587890625 disk.sda1.in‚ 0.1015625 disk.sda1.out‚0$ memory_percent‚97.91021347045898$ disk./.usageGB‚1174.559970855713' disk.nvme0c0n1.out‚45961.2587890625 disk.dm-1.in‚ 0.6953125 disk.dm-1.out‚ 192.17578125 disk.sda.out‚ 192.17578125& proc.memory.availableMB‚ 342.390625 cpu‚25.694734003415345 proc.cpu.threads‚17 network.recv‚697350 disk.sda2.out‚0 disk.dm-0.in‚ 4701.40234375" disk.dm-0.out‚45961.2587890625 disk.drbd1260.in‚0 disk.drbd1260.out‚ 0.2734375# proc.memory.rssMB‚ 3039.71484375 network.sent‚ 2.547673e+06 disk.sda3.in‚ 0.6953125! disk.nvme0n1.in‚ 4701.40234375 disk.sda.in‚0.796875) proc.memory.percent‚18.55294704437256) disk./.usagePercent‚78.82570516558486 disk.sda2.in‚0Ø®¶V“œ ‚(:ˆ ÷ÆéÔ¨åù¿ gpu.0.gpu‚31.0 gpu.0.memory‚3, gpu.0.memoryAllocated‚16.718826293945312* gpu.0.memoryAllocatedBytes‚ 14361362432 gpu.0.temp‚32.0 gpu.0.powerWatts‚109.628' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚21.9256 gpu.0.smClock‚1275 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0Åñû” ‚(:‰ ÷ÆéԼƕÁ disk.sda.in‚0.796875 disk.dm-0.in‚ 4829.65234375 network.recv‚704288% disk./.usageGB‚1174.5599517822266 disk.sda2.in‚0 disk.sda2.out‚0 disk.sda3.in‚ 0.6953125 disk.sda3.out‚ 192.609375( disk.nvme0c0n1.out‚46312.87548828125 disk.dm-1.out‚ 192.609375 disk.sda1.in‚ 0.1015625 disk.sda.out‚ 192.609375 cpu‚25.692647178362733& disk.nvme0n1.out‚46312.87548828125# disk.dm-0.out‚46312.87548828125! disk.nvme0n1.in‚ 4829.65234375# memory_percent‚97.9088544845581( proc.memory.availableMB‚ 342.61328125# proc.memory.rssMB‚ 3039.71484375) proc.memory.percent‚18.55294704437256 proc.cpu.threads‚17 network.sent‚ 2.634564e+06) disk./.usagePercent‚78.82570388554718# disk.nvme0c0n1.in‚ 4829.65234375 disk.dm-1.in‚ 0.6953125 disk.drbd1260.in‚0 disk.drbd1260.out‚ 0.2734375 disk.sda1.out‚0q–‚žž ‚(:“ †ÇéÔ¡¯¹ gpu.0.gpu‚55.0 gpu.0.memory‚7, gpu.0.memoryAllocated‚16.718826293945312* gpu.0.memoryAllocatedBytes‚ 14361362432 gpu.0.temp‚34.0 gpu.0.powerWatts‚167.755' gpu.0.enforcedPowerLimitWatts‚500.0) gpu.0.powerPercent‚33.550999999999995 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0Ÿw‰«•Ÿ ‚(:Š †ÇéÔö‚ ½) proc.memory.percent‚18.55294704437256 cpu‚25.690969420749653 proc.cpu.threads‚17 network.sent‚ 2.640818e+06 disk.sda1.out‚0 disk.sda.in‚0.796875 disk.sda2.in‚0 disk.dm-0.in‚ 4829.65234375# disk.dm-0.out‚46472.47998046875 disk.dm-1.in‚ 0.6953125 disk.dm-1.out‚ 192.921875 network.recv‚709210% disk./.usageGB‚1174.5599784851074 disk.sda1.in‚ 0.1015625( disk.nvme0c0n1.out‚46472.47998046875& disk.nvme0n1.out‚46472.47998046875# memory_percent‚97.9088544845581( proc.memory.availableMB‚ 342.61328125# proc.memory.rssMB‚ 3039.71484375 disk.sda.out‚ 192.921875 disk.sda2.out‚0 disk.sda3.out‚ 192.921875# disk.nvme0c0n1.in‚ 4829.65234375! disk.nvme0n1.in‚ 4829.65234375 disk.drbd1260.out‚ 0.27734375) disk./.usagePercent‚78.82570567759993 disk.sda3.in‚ 0.6953125 disk.drbd1260.in‚0’  ‚(:‡ •ÇéÔïËî¸ gpu.0.gpu‚8.0 gpu.0.memory‚1, gpu.0.memoryAllocated‚16.718826293945312* gpu.0.memoryAllocatedBytes‚ 14361362432 gpu.0.temp‚33.0 gpu.0.powerWatts‚117.293' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚23.4586 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0²4ËÆ•¡ ‚(:Š •ÇéԶݢ»# disk.dm-0.out‚46824.00146484375 disk.sda1.out‚0 disk.sda.out‚ 193.421875 network.recv‚712528 disk.sda3.out‚ 193.421875( disk.nvme0c0n1.out‚46824.00146484375 disk.dm-1.out‚ 193.421875 disk.drbd1260.out‚ 0.27734375 disk.sda1.in‚ 0.1015625 disk.sda.in‚0.796875( proc.memory.availableMB‚ 342.60546875 disk.sda2.in‚0 disk.sda3.in‚ 0.6953125# disk.nvme0c0n1.in‚ 4829.65234375 disk.drbd1260.in‚0 cpu‚25.68902003433456 proc.cpu.threads‚17& disk.nvme0n1.out‚46824.00146484375 disk.dm-0.in‚ 4829.65234375 disk.dm-1.in‚ 0.6953125$ memory_percent‚97.90890216827393# proc.memory.rssMB‚ 3039.71484375) proc.memory.percent‚18.55294704437256 network.sent‚ 2.645809e+06) disk./.usagePercent‚78.82567393266568% disk./.usageGB‚1174.5595054626465 disk.sda2.out‚0! disk.nvme0n1.in‚ 4829.65234375ŒÉs“¢ ‚(:ˆ ¤ÇéÔÌù‡¼ gpu.0.gpu‚47.0 gpu.0.memory‚5, gpu.0.memoryAllocated‚16.718826293945312* gpu.0.memoryAllocatedBytes‚ 14361362432 gpu.0.temp‚34.0 gpu.0.powerWatts‚160.063' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚32.0126 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0‰e(–£ ‚(:‹ ¤ÇéÔ¸÷ä disk.sda3.in‚ 0.6953125' disk.nvme0c0n1.out‚47127.6201171875 disk.dm-1.in‚ 0.6953125$ memory_percent‚97.90890216827393( proc.memory.availableMB‚ 342.60546875# proc.memory.rssMB‚ 3039.71484375 network.sent‚ 2.651116e+06) disk./.usagePercent‚78.82409641423969% disk./.usageGB‚1174.5359992980957 disk.sda.in‚0.796875 disk.sda.out‚ 194.16796875 disk.sda2.in‚0! disk.nvme0n1.in‚ 4829.65234375 disk.dm-1.out‚ 194.16796875 disk.sda1.out‚0 disk.sda2.out‚0 disk.sda3.out‚ 194.16796875# disk.nvme0c0n1.in‚ 4829.65234375 disk.dm-0.in‚ 4829.65234375" disk.dm-0.out‚47127.6201171875 disk.drbd1260.in‚0 disk.drbd1260.out‚0.28125% disk.nvme0n1.out‚47127.6201171875) proc.memory.percent‚18.55294704437256 cpu‚25.687216565282036 proc.cpu.threads‚17 network.recv‚716225 disk.sda1.in‚ 0.1015625òGëfž¤ ‚(:“ ³ÇéԸ뚹 gpu.0.gpu‚59.0 gpu.0.memory‚5, gpu.0.memoryAllocated‚16.718826293945312* gpu.0.memoryAllocatedBytes‚ 14361362432 gpu.0.temp‚34.0 gpu.0.powerWatts‚147.803' gpu.0.enforcedPowerLimitWatts‚500.0) gpu.0.powerPercent‚29.560599999999997 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0OÉ[’’¥ ‚(:‡ ³ÇéÔ³‰³¼ network.sent‚ 2.656931e+06 network.recv‚720709) disk./.usagePercent‚78.81453606888101% disk.nvme0n1.out‚47480.6005859375 disk.dm-0.in‚ 4993.34765625" disk.dm-0.out‚47480.6005859375' proc.memory.availableMB‚ 342.6015625# disk.nvme0c0n1.in‚ 4993.34765625' disk.nvme0c0n1.out‚47480.6005859375 disk.dm-1.out‚ 194.5078125 disk.drbd1260.in‚0 disk.drbd1260.out‚0.28125 disk.sda1.out‚0) proc.memory.percent‚18.55294704437256% disk./.usageGB‚1174.3935432434082! disk.nvme0n1.in‚ 4993.34765625 disk.sda1.in‚ 0.1015625 disk.sda.out‚ 194.5078125 disk.sda2.out‚0 cpu‚25.685555380775703 disk.sda3.in‚ 0.6953125 disk.sda3.out‚ 194.5078125 disk.dm-1.in‚ 0.6953125 disk.sda.in‚0.796875 disk.sda2.in‚0$ memory_percent‚97.90892601013184# proc.memory.rssMB‚ 3039.71484375 proc.cpu.threads‚17[—2¦ - _wandbruntime‚2045  _runtime‚2045—A5Ƨ À & lr-AdamW/pg1‚0.0012662959685314302 & lr-AdamW/pg2‚0.0012662959685314302 trainer/global_step‚1199 % _timestamp‚1.7885029698792522e+09 _runtime‚2045.958668018 _step‚93asþʨ Ä _runtime‚2045.958668018 _step‚93 & lr-AdamW/pg1‚0.0012662959685314302 & lr-AdamW/pg2‚0.0012662959685314302 trainer/global_step‚1199 % _timestamp‚1.7885029698792522e+09]¾²ƒ,2© - _wandbruntime‚2045  _runtime‚2045F;ܽÐ'ª ÊO XEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚97043160476160 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚3689 M@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚21478266 ^LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 1163694720681 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚7388380 PCDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated‚29876747 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚31684 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 586906112 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 320831488 [KDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 13207863296 WGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 13035530240 cQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 1224454631424 JBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚609 G?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚609 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 IDeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚97042912192000 ^LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 1170272351232 ZHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 1170041106432 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 599785472 @8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚895 $ _timestamp‚1.788502969882788e+09 _runtime‚2045.962874519 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚26088 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚31684 H?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚1321 P@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 13118524025 _LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚93813240610768 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚26086 dQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚97243222673408 _MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 1224176268288 G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚3689 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚578 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 152061440 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 WGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 12589930496 TDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 13807648768 VFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 13207863296 E8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚54285324 L?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚29850661 [IDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 1170272351232 @8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚895 [HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚93813223571408 trainer/global_step‚1199 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚2 XEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚94976935331449 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 L?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚24434663 TGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚29876747 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚14090466 OFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚1299 YIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 13207863296 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 599785472 M=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 13568114688 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚26086 IDeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 ZHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 1163468128341 UGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 582240769 C;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚895 _LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚95872888124928 QDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚21478846 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚286zf¤íÔ'« ÎO 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 E=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚895 JBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚286 G?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚286 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 231244800 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚31929 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚14090464 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 248284160 [HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚95872871085568 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 XEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚95872871085568 TDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 13035530240 TADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚94976691699749 O?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 13807648768 P@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 13568114688 PCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚24434665 QCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 243631700 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 D>DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 ZHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 1163468128341 UGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 582240769 C;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚895 _LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚95872888124928 QDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚21478846 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚286 XEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚97043160476160 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚3689 M@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚21478266 ^LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 1163694720681 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚7388380 PCDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated‚29876747 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚31684 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 586906112 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 320831488 [KDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 13207863296 WGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 13035530240 cQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 1224454631424 JBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚609 G?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚609 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 IDeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚97042912192000 ^LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 1170272351232 ZHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 1170041106432 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 599785472 @8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚895 $ _timestamp‚1.788502969882788e+09 _runtime‚2045.962874519 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚26088 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚31684 H?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚1321 P@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 13118524025 _LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚93813240610768 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚26086 dQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚97243222673408 _MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 1224176268288 G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚3689 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚578 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 152061440 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 WGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 12589930496 TDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 13807648768 VFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 13207863296 E8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚54285324 L?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚29850661 [IDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 1170272351232 @8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚895 [HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚93813223571408 trainer/global_step‚1199 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚2 XEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚94976935331449 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 L?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚24434663 TGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚29876747 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚14090466 OFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚1299 YIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 13207863296 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 599785472 M=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 13568114688 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚26086 IDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 13118524025 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚26112 BDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 N>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 13568114688 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚609 NADeviceStatsMonitor.on_train_ ?älêbatch_end/active.large_pool.allocated‚24454622 REDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚24454622 NADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚29875246 R?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚95054490481687 L>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 248297472 PBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 586906112 ^KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚97322792629760 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 243631908 XFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 1164420175735 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17039360 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 317933568 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚31684 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚7393726 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 152061440 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚609 UEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 13035530240 OBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚21496296 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 P@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 13807648768 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 599785472 G:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚54355980 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚613 K>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚21495683 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1321 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚611 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚31929 FDeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚54355980 MDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚1299 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚17039360 \JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 1164646768283 WGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 13207863296 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚609 K;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 13568114688 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚17039360 XFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 1170998577664 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚895 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚286 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚895 ]JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚95951399333888 NADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚24454620 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚14101959 S@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚97122629169664 ZGDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚95951399333888 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 231258112 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚24454620 J=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚29875246 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚7394337 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 ]JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚93890087345312 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 582240769 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 _step‚95 VCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚97122629169664 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 [HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚98548281057280 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚31929 WDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚98547963123712 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚286 YGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 1171229835776 M=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 13807648768 NADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚29901358 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚895 WIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 599785472 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 YIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 13207863296 TDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 13207863296 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 599785472 ODeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 N>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 13568114688 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚609 NADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚24454622 REDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚24454622 NADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚29875246 R?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚95054490481687 L>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 248297472 PBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 586906112 ^KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚97322792629760 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 243631908 XFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 1164420175735 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17039360 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 317933568 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚31684 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚7393726 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 152061440 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚609 UEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 13035530240 OBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚21496296 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 P@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 13807648768 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 599785472 G:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚54355980 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚613 K>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚21495683 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1321 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚611 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚31929 FDeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚54355980 MDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚1299 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚17039360 \JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 1164646768283 WGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 13207863296 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚609 K;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 13568114688 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚17039360 XFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 1170998577664 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚895 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚286 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚895 ]JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚95951399333888 NADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚24454620 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚14101959 S@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚97122629169664 ZGDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚95951399333888 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 231258112 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚24454620 J=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚29875246 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚7394337 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 ]JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚93890087345312 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 582240769 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 _step‚95 VCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚97122629169664 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 [HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚98548281057280 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚31929 WDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚98547963123712 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚286 YGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 1171229835776 M=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 13807648768 NADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚29901358 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚895 WIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 599785472 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 YIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 13207863296 TDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 13207863296 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 599785472 ODeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 13118524025 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚26112 B&Purity/2FRt1SRqqt0FBt/event_proportion‚0.03181703761219978 particle/accuracy_4_of_6‚1 & Purity/3FRt0SRqqt1FBt/FBt_purity‚1 particle/accuracy_1_of_2‚1 <"Purity/0FRt3SRqqt0FBt/event_purity‚0.0071428571827709675 @&Purity/*FRt0SRqqt4FBt/event_proportion‚0.0003150201519019902 9"Purity/0FRt*SRqqt2FBt/event_purity‚0.2553229033946991 ?&Purity/1FRt*SRqqt2FBt/event_proportion‚0.022303426638245583 ?&Purity/2FRt3SRqqt*FBt/event_proportion‚0.004032257944345474 5validation_average_jet_accuracy‚0.421050488948822 jet/accuracy_3_of_7‚0 & Purity/1FRt1SRqqt4FBt/FRt_purity‚0 ("Purity/2FRt2SRqqt2FBt/event_purity‚0 :"Purity/*FRt2SRqqt1FBt/SRqqt_purity‚0.47805899381637573 7 Purity/1FRt*SRqqt3FBt/FBt_purity‚0.9583333134651184 8 Purity/1FRt2SRqqt1FBt/FRt_purity‚0.26574504375457764 :"Purity/2FRt0SRqqt2FBt/event_purity‚0.01785714365541935 particle/accuracy_1_of_6‚1 & Purity/*FRt3SRqqt2FBt/FRt_purity‚1 7 Purity/*FRt*SRqqt2FBt/FBt_purity‚0.9858043789863586 :"Purity/*FRt*SRqqt0FBt/event_purity‚0.17130959033966064 =&Purity/0FRt1SRqqt0FBt/event_proportion‚0.1694808453321457 8 Purity/1FRt*SRqqt0FBt/FRt_purity‚0.14385321736335754 jet/accuracy_4_of_7‚0 ("Purity/3FRt3SRqqt0FBt/event_purity‚0 :"Purity/*FRt0SRqqt*FBt/event_purity‚0.38084423542022705 ?&Purity/1FRt2SRqqt1FBt/event_proportion‚0.008946572430431843 7 Purity/1FRt3SRqqt0FBt/FRt_purity‚0.1711597591638565 ?&Purity/2FRt0SRqqt0FBt/event_proportion‚0.024571573361754417 ?&Purity/2FRt2SRqqt0FBt/event_proportion‚0.015435988083481789 & Purity/3FRt0SRqqt*FBt/FBt_purity‚1 +jet/accuracy_3_of_4‚0.06692469865083694 ("Purity/4FRt1SRqqt0FBt/event_purity‚0 ,&Purity/*FRt4SRqqt3FBt/event_proportion‚0 9"Purity/1FRt0SRqqt0FBt/event_purity‚0.1551836133003235 7 Purity/2FRt*SRqqt2FBt/FBt_purity‚0.9911110997200012 :"Purity/3FRt1SRqqt0FBt/SRqqt_purity‚0.44999998807907104 7 Purity/0FRt1SRqqt3FBt/FBt_purity‚0.8999999761581421 9"Purity/0FRt2SRqqt1FBt/SRqqt_purity‚0.4907834529876709 8 Purity/3FRt*SRqqt0FBt/FRt_purity‚0.14253968000411987 ) Purity/1FRt1SRqqt4FBt/FBt_purity‚0.75 9"Purity/1FRt1SRqqt2FBt/SRqqt_purity‚0.5365645885467529 @&Purity/*FRt1SRqqt4FBt/event_proportion‚6.300403038039804e-05 & Purity/*FRt2SRqqt1FBt/FBt_purity‚1 7 Purity/1FRt0SRqqt2FBt/FRt_purity‚0.5763627290725708 & Purity/*FRt4SRqqt*FBt/FBt_purity‚1 jet/accuracy_5_of_7‚0 ("Purity/1FRt2SRqqt*FBt/event_purity‚0 :"Purity/1FRt2SRqqt*FBt/SRqqt_purity‚0.41354405879974365 8 Purity/2FRt0SRqqt2FBt/FRt_purity‚0.49047619104385376 ("Purity/2FRt4SRqqt*FBt/event_purity‚0 9"Purity/2FRt4SRqqt*FBt/SRqqt_purity‚0.4000000059604645 ;"Purity/*FRt2SRqqt1FBt/event_purity‚0.022938141599297523 >&Purity/1FRt0SRqqt*FBt/event_proportion‚0.19745463132858276 ?&Purity/1FRt1SRqqt2FBt/event_proportion‚0.006111391354352236 7 Purity/3FRt2SRqqt1FBt/FRt_purity‚0.2777777910232544 ("Purity/0FRt2SRqqt3FBt/event_purity‚0 ,&Purity/4FRt3SRqqt1FBt/event_proportion‚0 & Purity/2FRt2SRqqt2FBt/FBt_purity‚1 7 Purity/*FRt2SRqqt*FBt/FBt_purity‚0.9961516261100769 @&Purity/4FRt*SRqqt0FBt/event_proportion‚0.0001890120911411941 & Purity/*FRt4SRqqt1FBt/FBt_purity‚1 ("Purity/3FRt4SRqqt0FBt/event_purity‚0 ("Purity/1FRt0SRqqt4FBt/event_purity‚0 ("Purity/1FRt4SRqqt0FBt/event_purity‚0 ("Purity/4FRt2SRqqt0FBt/SRqqt_purity‚1 @&Purity/*FRt*SRqqt4FBt/event_proportion‚0.0003780241822823882 A&Purity/3FRt1SRqqt2FBt/event_proportion‚0.00025201612152159214 :"Purity/0FRt2SRqqt1FBt/event_purity‚0.05478750914335251 8 Purity/1FRt1SRqqt*FBt/FRt_purity‚0.18231597542762756 :"Purity/1FRt1SRqqt0FBt/SRqqt_purity‚0.47499245405197144 ,&Purity/4FRt1SRqqt4FBt/event_proportion‚0 @&Purity/3FRt0SRqqt2FBt/event_proportion‚6.300403038039804e-05 ,&Purity/3FRt0SRqqt4FBt/event_proportion‚0 8 Purity/2FRt3SRqqt1FBt/FRt_purity‚0.30000001192092896 ("Purity/3FRt3SRqqt*FBt/event_purity‚0 9"Purity/3FRt3SRqqt*FBt/SRqqt_purity‚0.4523809552192688 ("Purity/*FRt2SRqqt3FBt/event_purity‚0 ("Purity/4FRt2SRqqt*FBt/event_purity‚0 ("Purity/4FRt2SRqqt*FBt/SRqqt_purity‚1 :"Purity/*FRt*SRqqt*FBt/event_purity‚0.24101600050926208 9"Purity/*FRt*SRqqt*FBt/SRqqt_purity‚0.4574120342731476 :"Purity/0FRt1SRqqt2FBt/event_purity‚0.04753968492150307 ("Purity/3FRt1SRqqt2FBt/event_purity‚0 & Purity/4FRt2SRqqt*FBt/FRt_purity‚0 _step‚97 7 Purity/0FRt*SRqqt1FBt/FBt_purity‚0.9983571171760559 ,&Purity/3FRt0SRqqt3FBt/event_proportion‚0 ,&Purity/4FRt3SRqqt2FBt/event_proportion‚0 *jet/accuracy_1_of_3‚0.8399849534034729 ("Purity/0FRt0SRqqt1FBt/event_purity‚1 @&Purity/3FRt0SRqqt1FBt/event_proportion‚0.0011340725468471646 & Purity/2FRt3SRqqt1FBt/FBt_purity‚1 7 Purity/3FRt4SRqqt0FBt/FRt_purity‚0.1666666716337204 & Purity/*FRt3SRqqt2FBt/FBt_purity‚1 8 Purity/*FRt*SRqqt2FBt/FRt_purity‚0.46986985206604004 7 Purity/*FRt0SRqqt*FBt/FRt_purity‚0.2473047524690628 7 Purity/0FRt0SRqqt*FBt/FBt_purity‚0.9898556470870972 9"Purity/2FRt2SRqqt1FBt/SRqqt_purity‚0.5651785731315613 ,&Purity/2FRt3SRqqt4FBt/event_proportion‚0 ,&Purity/3FRt4SRqqt1FBt/event_proportion‚0 & Purity/3FRt2SRqqt*FBt/FBt_purity‚1 =&Purity/*FRt1SRqqt*FBt/event_proportion‚0.4482736885547638 @&Purity/*FRt2SRqqt3FBt/event_proportion‚6.300403038039804e-05 ."Purity/1FRt*SRqqt3FBt/event_purity‚0.15625 9"Purity/3FRt*SRqqt1FBt/SRqqt_purity‚0.6333333253860474 ,&Purity/4FRt2SRqqt4FBt/event_proportion‚0 9"Purity/1FRt3SRqqt2FBt/SRqqt_purity‚0.3333333432674408 8 Purity/*FRt2SRqqt1FBt/FRt_purity‚0.24782153964042664 6 Purity/0FRt*SRqqt*FBt/FBt_purity‚0.990129292011261 :"Purity/0FRt*SRqqt0FBt/event_purity‚0.37732023000717163 :"Purity/0FRt0SRqqt3FBt/event_purity‚0.09333334118127823 8 Purity/2FRt*SRqqt2FBt/FRt_purity‚0.46166667342185974 9"Purity/3FRt1SRqqt1FBt/SRqqt_purity‚0.5384615659713745 >&Purity/2FRt*SRqqt0FBt/event_proportion‚0.07579384744167328 ,&Purity/2FRt0SRqqt4FBt/event_proportion‚0 ("Purity/3FRt0SRqqt0FBt/event_purity‚0 8 Purity/3FRt1SRqqt*FBt/FRt_purity‚0.19547326862812042 & Purity/2FRt3SRqqt*FBt/FBt_purity‚1 9"Purity/1FRt0SRqqt2FBt/event_purity‚0.1716790497303009 ;"Purity/*FRt2SRqqt*FBt/event_purity‚0.023970510810613632 :"Purity/*FRt2SRqqt*FBt/SRqqt_purity‚0.42671146988868713 & Purity/0FRt0SRqqt1FBt/FBt_purity‚1 7 Purity/2FRt1SRqqt*FBt/FRt_purity‚0.2017263025045395 particle/accuracy_4_of_5‚1 & Purity/3FRt2SRqqt1FBt/FBt_purity‚1 epoch‚1 @&Purity/*FRt3SRqqt2FBt/event_proportion‚6.300403038039804e-05 7 Purity/1FRt0SRqqt2FBt/FBt_purity‚0.9874216318130493 >&Purity/1FRt2SRqqt*FBt/event_proportion‚0.05727066472172737 :"Purity/2FRt0SRqqt1FBt/event_purity‚0.07518518716096878 7 Purity/2FRt0SRqqt2FBt/FBt_purity‚0.9940475821495056 +jet/accuracy_3_of_3‚0.02672634832561016 ) Purity/4FRt1SRqqt0FBt/FRt_purity‚0.25 9"Purity/2FRt1SRqqt*FBt/SRqqt_purity‚0.5213163495063782 @&Purity/2FRt3SRqqt1FBt/event_proportion‚0.0003780241822823882 ?&Purity/1FRt0SRqqt1FBt/event_proportion‚0.049773186445236206 ?&Purity/1FRt0S¼ytù‘wRqqt2FBt/event_proportion‚0.015120967291295528 :"Purity/*FRt1SRqqt0FBt/event_purity‚0.24877195060253143 ?&Purity/2FRt0SRqqt2FBt/event_proportion‚0.003465221729129553 ;"Purity/2FRt1SRqqt*FBt/event_purity‚0.002016128972172737 ,&Purity/3FRt4SRqqt3FBt/event_proportion‚0 :"Purity/*FRt*SRqqt3FBt/event_purity‚0.07183908671140671 ,&Purity/0FRt2SRqqt4FBt/event_proportion‚0 ?&Purity/0FRt3SRqqt*FBt/event_proportion‚0.006741431541740894 7 Purity/1FRt2SRqqt*FBt/FBt_purity‚0.9959677457809448 ("Purity/2FRt2SRqqt1FBt/event_purity‚0 ,&Purity/3FRt*SRqqt4FBt/event_proportion‚0 @&Purity/3FRt2SRqqt*FBt/event_proportion‚0.0028351815417408943 @&Purity/4FRt0SRqqt0FBt/event_proportion‚6.300403038039804e-05 @&Purity/1FRt4SRqqt*FBt/event_proportion‚0.0003780241822823882 ("Purity/3FRt*SRqqt1FBt/event_purity‚0 @&Purity/4FRt2SRqqt*FBt/event_proportion‚6.300403038039804e-05 ,&Purity/4FRt2SRqqt3FBt/event_proportion‚0 ,&Purity/4FRt3SRqqt*FBt/event_proportion‚0 +particle/specificity‚0.9575158953666687 7 Purity/*FRt0SRqqt4FBt/FBt_purity‚0.8999999761581421 8 Purity/2FRt4SRqqt0FBt/FRt_purity‚0.20000000298023224 ,&Purity/*FRt3SRqqt3FBt/event_proportion‚0 7 Purity/3FRt0SRqqt1FBt/FRt_purity‚0.3910256028175354 ,&Purity/3FRt4SRqqt4FBt/event_proportion‚0 particle/accuracy_2_of_4‚1 ("Purity/*FRt4SRqqt0FBt/event_purity‚0 & Purity/*FRt0SRqqt4FBt/FRt_purity‚0 9"Purity/*FRt3SRqqt2FBt/SRqqt_purity‚0.3333333432674408 ("Purity/3FRt1SRqqt1FBt/event_purity‚0 ("Purity/0FRt4SRqqt0FBt/event_purity‚0 *"Purity/3FRt*SRqqt2FBt/SRqqt_purity‚0.5 7 Purity/*FRt1SRqqt3FBt/FBt_purity‚0.9126983880996704 ( Purity/2FRt2SRqqt2FBt/FRt_purity‚0.5 trainer/global_step‚1207 7 Purity/*FRt1SRqqt0FBt/FRt_purity‚0.1374981850385666 9"Purity/1FRt0SRqqt3FBt/event_purity‚0.1785714328289032 & Purity/1FRt2SRqqt1FBt/FBt_purity‚1 @&Purity/3FRt1SRqqt0FBt/event_proportion‚0.0025201612152159214 jet/accuracy_7_of_7‚0 6 Purity/0FRt*SRqqt2FBt/FBt_purity‚0.985823929309845 ,&Purity/1FRt4SRqqt1FBt/event_proportion‚0 8 Purity/3FRt1SRqqt0FBt/FRt_purity‚0.10833333432674408 8 Purity/2FRt1SRqqt0FBt/FRt_purity‚0.14667023718357086 particle/accuracy_2_of_6‚1 ("Purity/2FRt0SRqqt3FBt/event_purity‚0 7 Purity/3FRt0SRqqt2FBt/FRt_purity‚0.3333333432674408 ) Purity/*FRt1SRqqt4FBt/FBt_purity‚0.75 >&Purity/*FRt2SRqqt0FBt/event_proportion‚0.10855595022439957 @&Purity/3FRt1SRqqt1FBt/event_proportion‚0.0009450604557059705 9"Purity/0FRt4SRqqt0FBt/SRqqt_purity‚0.2142857164144516 9"Purity/2FRt3SRqqt1FBt/SRqqt_purity‚0.2666666805744171 & Purity/*FRt3SRqqt1FBt/FBt_purity‚1 @&Purity/0FRt2SRqqt2FBt/event_proportion‚0.0010080644860863686 A&Purity/0FRt4SRqqt0FBt/event_proportion‚0.00044102821266278625 ,&Purity/0FRt4SRqqt3FBt/event_proportion‚0 +"Purity/0FRt4SRqqt1FBt/SRqqt_purity‚0.25 particle/accuracy_5_of_7‚1 particle/accuracy_7_of_7‚0.5 7 Purity/*FRt0SRqqt3FBt/FBt_purity‚0.9212302565574646 ,&Purity/2FRt4SRqqt4FBt/event_proportion‚0 7 Purity/3FRt4SRqqt*FBt/FRt_purity‚0.1666666716337204 8 Purity/*FRt0SRqqt0FBt/FRt_purity‚0.15270301699638367 9"Purity/*FRt1SRqqt0FBt/SRqqt_purity‚0.4754506051540375 >&Purity/0FRt*SRqqt*FBt/event_proportion‚0.40996724367141724 7 Purity/0FRt0SRqqt2FBt/FBt_purity‚0.9871290922164917 ,&Purity/0FRt1SRqqt4FBt/event_proportion‚0 @&Purity/3FRt0SRqqt0FBt/event_proportion‚0.0015120967291295528 A&Purity/3FRt4SRqqt*FBt/event_proportion‚0.00012600806076079607 particle/accuracy_1_of_1‚1 +"Purity/*FRt4SRqqt0FBt/SRqqt_purity‚0.25 & Purity/*FRt3SRqqt*FBt/FBt_purity‚1 >&Purity/1FRt1SRqqt1FBt/event_proportion‚0.03944052383303642 8 Purity/2FRt2SRqqt1FBt/FRt_purity‚0.23005953431129456 9"Purity/1FRt2SRqqt2FBt/SRqqt_purity‚0.4791666567325592 & Purity/*FRt1SRqqt4FBt/FRt_purity‚0 +"Purity/1FRt1SRqqt3FBt/SRqqt_purity‚0.75 & Purity/4FRt2SRqqt0FBt/FRt_purity‚0 =&Purity/*FRt2SRqqt*FBt/event_proportion‚0.1332535296678543 ,&Purity/*FRt3SRqqt4FBt/event_proportion‚0 & Purity/2FRt0SRqqt1FBt/FBt_purity‚1 ,&Purity/2FRt1SRqqt4FBt/event_proportion‚0 ,&Purity/2FRt4SRqqt2FBt/event_proportion‚0 8 Purity/*FRt1SRqqt*FBt/FRt_purity‚0.18926188349723816 >&Purity/*FRt3SRqqt0FBt/event_proportion‚0.01827116869390011 9"Purity/0FRt*SRqqt0FBt/SRqqt_purity‚0.4433155357837677 =&Purity/1FRt*SRqqt0FBt/event_proportion‚0.3135710656642914 & Purity/1FRt0SRqqt1FBt/FBt_purity‚1 ,&Purity/1FRt3SRqqt4FBt/event_proportion‚0 8 Purity/2FRt*SRqqt1FBt/FRt_purity‚0.35202425718307495 ,&Purity/4FRt2SRqqt1FBt/event_proportion‚0 >&Purity/0FRt0SRqqt0FBt/event_proportion‚0.04082661122083664 ("Purity/2FRt2SRqqt*FBt/event_purity‚0 9"Purity/2FRt2SRqqt*FBt/SRqqt_purity‚0.4868183732032776 ,&Purity/4FRt3SRqqt3FBt/event_proportion‚0 +jet/accuracy_2_of_3‚0.24337103962898254 7 Purity/*FRt2SRqqt2FBt/FRt_purity‚0.3717948794364929 ("Purity/2FRt3SRqqt1FBt/event_purity‚0 @&Purity/2FRt0SRqqt3FBt/event_proportion‚6.300403038039804e-05 ("Purity/0FRt4SRqqt1FBt/event_purity‚0 jet/accuracy_6_of_7‚0 >&Purity/*FRt2SRqqt1FBt/event_proportion‚0.02255544438958168 7 Purity/2FRt0SRqqt*FBt/FBt_purity‚0.9967741966247559 7 Purity/1FRt0SRqqt*FBt/FBt_purity‚0.9939948916435242 ,&Purity/4FRt0SRqqt1FBt/event_proportion‚0 8 Purity/2FRt4SRqqt*FBt/FRt_purity‚0.20000000298023224 particle/accuracy_4_of_7‚1 @&Purity/2FRt4SRqqt*FBt/event_proportion‚0.0003150201519019902 7 Purity/1FRt2SRqqt2FBt/FBt_purity‚0.9583333134651184 ( Purity/2FRt0SRqqt3FBt/FRt_purity‚0.5 ("Purity/4FRt0SRqqt0FBt/event_purity‚0 7 Purity/*FRt*SRqqt1FBt/FRt_purity‚0.3547112047672272 >&Purity/0FRt0SRqqt*FBt/event_proportion‚0.11857359111309052 7 Purity/1FRt0SRqqt3FBt/FRt_purity‚0.5357142686843872 ("Purity/0FRt*SRqqt4FBt/event_purity‚0 ) Purity/4FRt1SRqqt*FBt/FRt_purity‚0.25 9"Purity/0FRt0SRqqt2FBt/event_purity‚0.3267888128757477 @&Purity/1FRt*SRqqt3FBt/event_proportion‚0.0013230846961960196 @&Purity/1FRt3SRqqt2FBt/event_proportion‚6.300403038039804e-05 7 Purity/2FRt*SRqqt*FBt/FBt_purity‚0.9961161017417908 ?&Purity/2FRt2SRqqt*FBt/event_proportion‚0.019468246027827263 ("Purity/3FRt*SRqqt*FBt/event_purity‚0 7 Purity/3FRt*SRqqt*FBt/FRt_purity‚0.1971753090620041 9"Purity/3FRt*SRqqt*FBt/SRqqt_purity‚0.4000958204269409 A&Purity/1FRt1SRqqt3FBt/event_proportion‚0.00025201612152159214 >&Purity/1FRt2SRqqt0FBt/event_proportion‚0.04731602966785431 ?&Purity/3FRt*SRqqt*FBt/event_proportion‚0.009954636916518211 *jet/accuracy_1_of_2‚0.7193155884742737 +"Purity/3FRt4SRqqt0FBt/SRqqt_purity‚0.25 9"Purity/1FRt4SRqqt0FBt/SRqqt_purity‚0.1666666716337204 & Purity/3FRt0SRqqt2FBt/FBt_purity‚1 ("Purity/4FRt2SRqqt0FBt/event_purity‚0 7 Purity/*FRt*SRqqt1FBt/FBt_purity‚0.9988539218902588 ;"Purity/1FRt1SRqqt0FBt/event_purity‚0.015516032464802265 7 Purity/1FRt3SRqqt*FBt/FRt_purity‚0.1678699553012848 ("Purity/2FRt1SRqqt2FBt/event_purity‚0 @&Purity/3FRt2SRqqt0FBt/event_proportion‚0.0024571572430431843 ("Purity/*FRt4SRqqt*FBt/event_purity‚0 0"Purity/*FRt4SRqqt*FBt/SRqqt_purity‚ 0.2421875 +"Purity/*FRt4SRqqt1FBt/SRqqt_purity‚0.25 :"Purity/0FRt1SRqqt1FBt/event_purity‚0.16842396557331085 ?&Purity/2FRt2SRqqt1FBt/event_proportion‚0.003969253972172737 ,&Purity/2FRt4SRqqt3FBt/event_proportion‚0 ("Purity/3FRt0SRqqt1FBt/event_purity‚0 particle/accuracy_1_of_3‚1 (particle/accuracy‚0.8539512753486633 & Purity/0FRt3SRqqt1FBt/FBt_purity‚1 ("Purity/2FRt*SRqqt3FBt/event_purity‚0 8 Purity/*FRt*SRqqt3FBt/FRt_purity‚0.47058823704719543 :"Purity/0FRt*SRqqt3FBt/event_purity‚0.04320988059043884 8 Purity/1FRt0SRqqt1FBt/FRt_purity‚0.41656604409217834 & Purity/2FRt*SRqqt1FBt/FBt_purity‚1 8 Purity/1FRt*SRqqt*FBt/FRt_purity‚0.21107767522335052 9"Purity/1FRt*SRqqt2FBt/SRqqt_purity‚0.5407834053039551 8 Purity/2FRt0SRqqt1FBt/FRt_purity‚0.34785714745521545 ?&Purity/3FRt1SRqqt*FBt/event_proportion‚0.003717237850651145 ( Purity/2FRt*SRqqt3FBt/FRt_purity‚0.5 ("Purity/*FRt1SRqqt3FBt/event_purity‚0 7 Purity/0FRt2SRqqt3FBt/FBt_purity‚0.6666666865348816 7 Purity/*FRt*SRqqt*FBt/FBt_purity‚0.9916454553604126 ,&Purity/3FRt2SRqqt4FBt/event_proportion‚0 @&Purity/4FRt*SRqqt*FBt/event_proportion‚0.0001890120911411941 ("Purity/4FRt0SRqqt*FBt/event_purity‚0 9"Purity/*FRt*SRqqt1FBt/SRqqt_purity‚0.5145480632781982 ,&Purity/0FRt3SRqqt2FBt/event_proportion‚0 9"Purity/1FRt0SRqqt*FBt/event_purity‚0.1943524330854416 9"Purity/2FRt*SRqqt2FBt/SRqqt_purity‚0.7061403393745422 particle/accuracy_4_of_4‚1 & Purity/0FRt3SRqqt*FBt/FBt_purity‚1 ;"Purity/*FRt1SRqqt2FBt/event_purity‚0.017626726999878883 :"Purity/0FRt*SRqqt*FBt/event_purity‚0.45918840169906616 :"Purity/0FRt*SRqqt*FBt/SRqqt_purity‚0.45577847957611084 8 Purity/1FRt1SRqqt0FBt/FRt_purity‚0.13386237621307373 @&Purity/2FRt4SRqqt0FBt/event_proportion‚0.0003150201519019902 ("Purity/4FRt1SRqqt0FBt/SRqqt_purity‚0 >&Purity/1FRt*SRqqt1FBt/event_proportion‚0.09929435700178146 ("Purity/3FRt1SRqqt0FBt/event_purity‚0 jet/accuracy_5_of_5‚0 7 Purity/*FRt2SRqqt2FBt/FBt_purity‚0.9736841917037964 7 Purity/1FRt1SRqqt3FBt/FBt_purity‚0.9166666865348816 :"Purity/*FRt*SRqqt2FBt/event_purity‚0.18838714063167572 >&Purity/0FRt0SRqqt1FBt/event_proportion‚0.08606351166963577 A&Purity/0FRt1SRqqt3FBt/event_proportion‚0.00044102821266278625 & Purity/2FRt2SRqqt1FBt/FBt_purity‚1 ,&Purity/2FRt3SRqqt2FBt/event_proportion‚0 ("Purity/*FRt4SRqqt1FBt/event_purity‚0 & Purity/1FRt*SRqqt4FBt/FRt_purity‚0 >&Purity/*FRt*SRqqt2FBt/event_proportion‚0.06747731566429138 9"Purity/0FRt1SRqqt1FBt/SRqqt_purity‚0.5024889707565308 ,&Purity/*FRt4SRqqt4FBt/event_proportion‚0 =&Purity/1FRt1SRqqt*FBt/event_proportion‚0.1723790317773819 =&Purity/1FRt1SRqqt0FBt/event_proportion‚0.1265120953321457 @&Purity/3FRt0SRqqt*FBt/event_proportion‚0.0027091733645647764 ("Purity/3FRt2SRqqt*FBt/event_purity‚0 9"Purity/3FRt2SRqqt*FBt/SRqqt_purity‚0.4114583432674408 @&Purity/4FRt2SRqqt0FBt/event_proportion‚6.300403038039804e-05 9"Purity/0FRt3SRqqt0FBt/SRqqt_purity‚0.3983134925365448 >&Purity/*FRt0SRqqt0FBt/event_proportion‚0.15757308900356293 ?&Purity/0FRt2SRqqt1FBt/event_proportion‚0.009261592291295528 +jet/accuracy_3_of_5‚0.08656997978687286 ("Purity/*FRt2SRqqt2FBt/event_purity‚0 *"Purity/0FRt2SRqqt2FBt/SRqqt_purity‚0.5 * Purity/1FRt*SRqqt4FBt/FBt_purity‚0.875 9"Purity/0FRt1SRqqt3FBt/SRqqt_purity‚0.6000000238418579 )loss/val_total_loss‚46.00053281292241 >&Purity/*FRt1SRqqt0FBt/event_proportion‚0.33039313554763794 ?&Purity/*FRt1SRqqt2FBt/event_proportion‚0.017011089250445366 7 Purity/*FRt3SRqqt1FBt/FRt_purity‚0.2777777910232544 & Purity/2FRt2SRqqt*FBt/FBt_purity‚1 7 Purity/1FRt2SRqqt2FBt/FRt_purity‚0.3611111342906952 :"Purity/*FRt*SRqqt0FBt/SRqqt_purity‚0.43976476788520813 9"Purity/*FRt*SRqqt1FBt/event_purity‚0.4162077009677887 ,&Purity/0FRt4SRqqt2FBt/event_proportion‚0 ,&Purity/1FRt4SRqqt2FBt/event_proportion‚0 @&Purity/4FRt1SRqqt*FBt/event_proportion‚6.300403038039804e-05 ,&Purity/4FRt4SRqqt1FBt/event_proportion‚0 *jet/accuracy_2_of_4‚0.3586917817592621 ("Purity/0FRt4SRqqt*FBt/event_purity‚0 8 Purity/*FRt*SRqqt0FBt/FRt_purity‚0.14294034242630005 7 Purity/1FRt*SRqqt2FBt/FBt_purity‚0.9672889709472656 ."Purity/0FRt4SRqqt*FBt/SRqqt_purity‚0.21875 7 Purity/3FRt3SRqqt*FBt/FRt_purity‚0.2142857313156128 9"Purity/3FRt3SRqqt0FBt/SRqqt_purity‚0.4523809552192688 7 Purity/*FRt*SRqqt3FBt/FBt_purity‚0.9162836670875549 9"Purity/*FRt1SRqqt2FBt/SRqqt_purity‚0.5574929714202881 & Purity/0FRt2SRqqt1FBt/FBt_purity‚1 ,&Purity/0FRt3SRqqt4FBt/event_proportion‚0 ("Purity/1FRt3SRqqt1FBt/event_purity‚0 ,&Purity/4FRt0SRqqt2FBt/event_proportion‚0 9"Purity/3FRt2SRqqt1FBt/SRqqt_purity‚0.6666666865348816 ("Purity/0FRt3SRqqt1FBt/event_purity‚0 ;"Purity/2FRt1SRqqt1FBt/event_purity‚0.008064515888690948 ,&Purity/3FRt1SRqqt4FBt/event_proportion‚0 particle/accuracy_3_of_3‚1 particle/accuracy_1_of_4‚1 jet/accuracy_1_of_7‚1 * Purity/4FRt*SRqqt*FBt/FRt_purity‚0.125 9"Purity/*FRt*SRqqt2FBt/SRqqt_purity‚0.5448167324066162 ?&Purity/0FRt3SRqqt0FBt/event_proportion‚0.006111391354352236 @&Purity/0FRt4SRqqt1FBt/event_proportion‚6.300403038039804e-05 ,&Purity/3FRt2SRqqt2FBt/event_proportion‚0 ,&Purity/4FRt1SRqqt3FBt/event_proportion‚0 8 Purity/3FRt0SRqqt*FBt/FRt_purity‚0.21464644372463226 particle/accuracy_2_of_5‚1 ("Purity/0FRt2SRqqt2FBt/event_purity‚0 >&Purity/0FRt*SRqqt1FBt/event_proportion‚0.14787046611309052 ?&Purity/0FRt0SRqqt3FBt/event_proportion‚0.002583165420219302 :"Purity/0FRt1SRqqt0FBt/event_purity‚0.47332024574279785 7 Purity/0FRt2SRqqt*FBt/FBt_purity‚0.9946236610412598 3"Purity/1FRt*SRqqt1FBt/SRqqt_purity‚ 0.5185546875 ,&Purity/2FRt3SRqqt3FBt/event_proportion‚0 -jet/accuracy_4_of_4‚0.0029922511894255877 particle/accuracy_3_of_4‚1 ?&Purity/0FRt2SRqqt0FBt/event_proportion‚0.043283771723508835 7 Purity/2FRt*SRqqt0FBt/FRt_purity‚0.1416560262441635 ("Purity/3FRt2SRqqt0FBt/event_purity‚0 ,&Purity/4FRt4SRqqt4FBt/event_proportion‚0 :"Purity/*FRt2SRqqt2FBt/SRqqt_purity‚0.48245614767074585 & Purity/3FRt*SRqqt2FBt/FBt_purity‚1 ("Purity/3FRt0SRqqt2FBt/event_purity‚0 ("Purity/1FRt1SRqqt3FBt/event_purity‚0 7 Purity/0FRt1SRqqt2FBt/FBt_purity‚0.9839351773262024 @&Purity/1FRt2SRqqt2FBt/event_proportion‚0.0010080644860863686 ?&Purity/2FRt1SRqqt2FBt/event_proportion‚0.001890120911411941 & Purity/2FRt*SRqqt3FBt/FBt_purity‚1 ("Purity/*FRt*SRqqt4FBt/SRqqt_purity‚1 & Purity/1FRt3SRqqt2FBt/FBt_purity‚1 :"Purity/*FRt0SRqqt0FBt/event_purity‚0.13171575963497162 ?&Purity/2FRt*SRqqt2FBt/event_proportion‚0.005418346729129553 8 Purity/2FRt2SRqqt0FBt/FRt_purity‚0.12786389887332916 7 Purity/*FRt4SRqqt*FBt/FRt_purity‚0.2638888955116272 *jet/accuracy_2_of_6‚0.6403508186340332 ("Purity/1FRt*SRqqt4FBt/event_purity‚0 :"Purity/1FRt3SRqqt1FBt/SRqqt_purity‚0.42222216725349426 @&Purity/4FRt0SRqqt*FBt/event_proportion‚6.300403038039804e-05 +jet/accuracy_2_of_5‚0.41565755009651184 ("Purity/3FRt2SRqqt1FBt/event_purity‚0 -"Purity/0FRt3SRqqt1FBt/SRqqt_purity‚0.4375 ("Purity/*FRt1SRqqt4FBt/SRqqt_purity‚1 8 Purity/*FRt0SRqqt1FBt/FRt_purity‚0.40624064207077026 9"Purity/0FRt2SRqqt0FBt/SRqqt_purity‚0.4075649082660675 @&Purity/0FRt2SRqqt3FBt/event_proportion‚6.300403038039804e-05 :"Purity/1FRt*SRqqt0FBt/event_purity‚0.07126323878765106 7 Purity/1FRt1SRqqt1FBt/FBt_purity‚0.9988052845001221 :"Purity/1FRt2SRqqt0FBt/SRqqt_purity‚0.40822798013687134 9"Purity/2FRt1SRqqt1FBt/SRqqt_purity‚0.6000000834465027 @&Purity/3FRt3SRqqt*FBt/event_proportion‚0.0005670362734235823 :"Purity/1FRt*SRqqt*FBt/event_purity‚0.09583548456430435 :"Purity/1FRt*SRqqt*FBt/SRqqt_purity‚0.45532092452049255 9"Purity/2FRt1SRqqt2FBt/SRqqt_purity‚0.7412280440330505 ,&Purity/4FRt4SRqqt3FBt/event_proportion‚0 *jet/accuracy_1_of_6‚0.9824560880661011 ("Purity/1FRt1SRqqt4FBt/event_purity‚0 ("Purity/4FRt*SRqqt0FBt/event_purity‚0 :"Purity/*FRt1SRqqt*FBt/event_purity‚0.20619556307792664 8"Purity/*FRt1SRqqt*FBt/SRqqt_purity‚0.493158221244812 7 Purity/*FRt1SRqqt2FBt/FRt_purity‚0.3620350956916809 6 Purity/*FRt2SRqqt*FBt/FRt_purity‚0.153945192694664 >&Purity/0FRt2SRqqt*FBt/event_proportion‚0.05361643061041832 :"Purity/1FRt*SRqqt2FBt/event_purity‚0.11141553521156311 & Purity/1FRt1SRqqt3FBt/FRt_purity‚0 =&Purity/*FRt*SRqqt*FBt/event_proportion‚0.9591733813285828 :"Purity/1FRt*SRqqt1FBt/event_purity‚0.16953250765800476 ?&Purity/1FRt3SRqqt0FBt/event_proportion‚0.007938507944345474 <"Purity/2FRt*SRqqt0FBt/event_purity‚0.0040725404396653175 ,&Purity/3FRt*SRqqt3FBt/event_proportion‚0 7 Purity/1FRt1SRqqt*FBt/FBt_purity‚0.9913498759269714 ;"Purity/2FRt*SRqqt2FBt/event_purity‚0.011111111380159855 7 Purity/2FRt1SRqqt1FBt/FRt_purity‚0.3931259512901306 >&Purity/*FRt0SRqqt1FBt/event_proportion‚0.14591734111309052 :"Purity/2FRt1SRqqt0FBt/SRqqt_purity‚0.48947006464004517 8 Purity/3FRt1SRqqt1FBt/FRt_purity‚0.38461539149284363 ,&Purity/4FRt*SRqqt2FBt/event_proportion‚0 @&Purity/0FRt4SRqqt*FBt/event_proportion‚0.0005040322430431843 7 Purity/1FRt*SRqqt2FBt/FRt_purity‚0.5245682597160339 ,&Purity/4FRt4SRqqt2FBt/event_proportion‚0 particle/accuracy_2_of_3‚1 % _timestamp‚1.7885030323192105e+09 7 Purity/*FRt0SRqqt3FBt/FRt_purity‚0.5333333611488342 7 Purity/*FRt3SRqqt0FBt/FRt_purity‚0.1702105700969696 :"Purity/1FRt3SRqqt0FBt/SRqqt_purity‚0.32165902853012085 ,&Purity/4FRt4SRqqt*FBt/event_proportion‚0 ?&Purity/*FRt0SRqqt3FBt/event_proportion‚0.003717237850651145 9"Purity/0FRt*SRqqt1FBt/SRqqt_purity‚0.4918311536312103 @&Purity/1FRt1SRqqt4FBt/event_proportion‚6.300403038039804e-05 9"Purity/2FRt2SRqqt0FBt/SRqqt_purity‚0.4574425518512726 @&Purity/3FRt*SRqqt2FBt/event_proportion‚0.0003150201519019902 & Purity/3FRt1SRqqt1FBt/FBt_purity‚1 particle/accuracy_3_of_6‚1 8 Purity/3FRt2SRqqt*FBt/FRt_purity‚0.14814814925193787 7 Purity/*FRt0SRqqt*FBt/FBt_purity‚0.9917316436767578 :"Purity/*FRt0SRqqt2FBt/event_purity‚0.25607988238334656 :"Purity/*FRt3SRqqt1FBt/SRqqt_purity‚0.39550262689590454 =&Purity/0FRt1SRqqt*FBt/event_proportion‚0.2305317521095276 ;"Purity/2FRt*SRqqt*FBt/event_purity‚0.009259711019694805 9"Purity/2FRt*SRqqt*FBt/SRqqt_purity‚0.4774433374404907 & Purity/2FRt1SRqqt1FBt/FBt_purity‚1 A&Purity/0FRt0SRqqt4FBt/event_proportion‚0.00025201612152159214 @&Purity/1FRt0SRqqt3FBt/event_proportion‚0.0010710685746744275 :"Purity/1FRt2SRqqt1FBt/SRqqt_purity‚0.39420127868652344 ("Purity/3FRt1SRqqt*FBt/event_purity‚0 :"Purity/3FRt1SRqqt*FBt/SRqqt_purity‚0.42592597007751465 ,&Purity/3FRt1SRqqt3FBt/event_proportion‚0 particle/accuracy_3_of_7‚1 7 Purity/1FRt4SRqqt0FBt/FRt_purity‚0.3333333432674408 ?&Purity/*FRt3SRqqt*FBt/event_proportion‚0.020476309582591057 ,&Purity/1FRt2SRqqt3FBt/event_proportion‚0 ;"Purity/2FRt*SRqqt1FBt/event_purity‚0.027065463364124298 ("Purity/2FRt3SRqqt*FBt/event_purity‚0 :"Purity/2FRt3SRqqt*FBt/SRqqt_purity‚0.24228572845458984 ,&Purity/3FRt3SRqqt1FBt/event_proportion‚0 8 Purity/3FRt*SRqqt2FBt/FRt_purity‚0.20000000298023224 9"Purity/*FRt1SRqqt3FBt/SRqqt_purity‚0.7619047164916992 & Purity/3FRt1SRqqt*FBt/FBt_purity‚1 ;"Purity/2FRt0SRqqt0FBt/event_purity‚0.013001693412661552 8 Purity/2FRt3SRqqt*FBt/FRt_purity‚0.13147619366645813 ?&Purity/2FRt3SRqqt0FBt/event_proportion‚0.003654233878478408 9"Purity/0FRt*SRqqt3FBt/SRqqt_purity‚0.6000000238418579 _runtime‚2108.400911616 7 Purity/2FRt1SRqqt*FBt/FBt_purity‚0.9964157938957214 =&Purity/*FRt1SRqqt1FBt/event_proportion‚0.1001134067773819 & Purity/*FRt0SRqqt1FBt/FBt_purity‚1 7 Purity/1FRt1SRqqt1FBt/FRt_purity‚0.3093251585960388 ?&Purity/2FRt*SRqqt1FBt/event_proportion‚0.021169355139136314 ?&Purity/2FRt0SRqqt1FBt/event_proportion‚0.008946572430431843 & Purity/1FRt3SRqqt2FBt/FRt_purity‚1 :"Purity/0FRt1SRqqt*FBt/event_purity‚0.38540440797805786 8"Purity/0FRt1SRqqt*FBt/SRqqt_purity‚0.482045978307724 ,&Purity/0FRt3SRqqt3FBt/event_proportion‚0 ,&Purity/1FRt2SRqqt4FBt/event_proportion‚0 :"Purity/2FRt0SRqqt*FBt/event_purity‚0.02383982203900814 ,&Purity/4FRt*SRqqt3FBt/event_proportion‚0 9"Purity/2FRt4SRqqt0FBt/SRqqt_purity‚0.4000000059604645 @&Purity/*FRt4SRqqt0FBt/event_proportion‚0.0012600806076079607 8"Purity/0FRt*SRqqt1FBt/event_purity‚0.643706202507019 >&Purity/0FRt1SRqqt1FBt/event_proportion‚0.05185231938958168 7 Purity/1FRt0SRqqt3FBt/FBt_purity‚0.9523809552192688 8 Purity/2FRt0SRqqt0FBt/FRt_purity‚0.14386755228042603 ("Purity/*FRt3SRqqt1FBt/event_purity‚0 ?&Purity/0FRt1SRqqt2FBt/event_proportion‚0.008757560513913631 >&Purity/1FRt0SRqqt0FBt/event_proportion‚0.13142640888690948 7 Purity/1FRt0SRqqt0FBt/FRt_purity‚0.1551836133003235 7 Purity/1FRt2SRqqt*FBt/FRt_purity‚0.1584923416376114 * Purity/0FRt0SRqqt4FBt/FBt_purity‚0.875 <"Purity/0FRt3SRqqt*FBt/event_purity‚0.0071428571827709675 :"Purity/0FRt3SRqqt*FBt/SRqqt_purity‚0.40017005801200867 9"Purity/*FRt0SRqqt1FBt/event_purity‚0.7005100250244141 7 Purity/*FRt1SRqqt2FBt/FBt_purity‚0.9816062450408936 @&Purity/*FRt1SRqqt3FBt/event_proportion‚0.0006930443341843784 ;"Purity/*FRt2SRqqt0FBt/event_purity‚0.024302344769239426 :"Purity/0FRt2SRqqt*FBt/event_purity‚0.06023978814482689 :"Purity/0FRt2SRqqt*FBt/SRqqt_purity‚0.42005398869514465 ("Purity/1FRt2SRqqt1FBt/event_purity‚0 ,&Purity/1FRt4SRqqt4FBt/event_proportion‚0 :"Purity/*FRt3SRqqt0FBt/SRqqt_purity‚0.34414586424827576 9"Purity/2FRt*SRqqt1FBt/SRqqt_purity‚0.5635807514190674 :"Purity/2FRt3SRqqt0FBt/SRqqt_purity‚0.25115442276000977 * Purity/0FRt*SRqqt4FBt/FBt_purity‚0.875 ("Purity/0FRt1SRqqt3FBt/event_purity‚0 A&Purity/3FRt4SRqqt0FBt/event_proportion‚0.00012600806076079607 & Purity/2FRt0SRqqt3FBt/FBt_purity‚1 & Purity/4FRt0SRqqt*FBt/FRt_purity‚0 ("Purity/3FRt*SRqqt0FBt/event_purity‚0 ("Purity/1FRt2SRqqt2FBt/event_purity‚0 :"Purity/*FRt2SRqqt0FBt/SRqqt_purity‚0.41437655687332153 >&Purity/0FRt*SRqqt0FBt/event_proportion‚0.21931703388690948 8 Purity/2FRt3SRqqt0FBt/FRt_purity‚0.13382035493850708 ("Purity/3FRt0SRqqt*FBt/event_purity‚0 ,&Purity/3FRt3SRqqt4FBt/event_proportion‚0 ,&Purity/4FRt0SRqqt3FBt/event_proportion‚0 jet/accuracy_6_of_6‚0 =&Purity/*FRt0SRqqt2FBt/event_proportion‚0.0483240932226181 ;"Purity/*FRt3SRqqt0FBt/event_purity‚0.003225806402042508 @&Purity/*FRt4SRqqt*FBt/event_proportion‚0.0013230846961960196 7"Purity/1FRt1SRqqt1FBt/event_purity‚0.04088119789958 ,&Purity/2FRt2SRqqt4FBt/event_proportion‚0 ("Purity/2FRt3SRqqt0FBt/event_purity‚0 ,&Purity/4FRt0SRqqt4FBt/event_proportion‚0 7 Purity/0FRt*SRqqt3FBt/FBt_purity‚0.9094650745391846 9"Purity/0FRt0SRqqt*FBt/event_purity‚0.8111569881439209 7 Purity/2FRt1SRqqt2FBt/FBt_purity‚0.9868420958518982 @&Purity/3FRt3SRqqt0FBt/event_proportion‚0.0005670362734235823 ("Purity/0FRt0SRqqt4FBt/event_purity‚0 +jet/accuracy_3_of_6‚0.07894736528396606 ("Purity/3FRt*SRqqt2FBt/event_purity‚0 & Purity/1FRt0SRqqt4FBt/FRt_purity‚0 :"Purity/3FRt*SRqqt0FBt/SRqqt_purity‚0.38703233003616333 & Purity/3FRt1SRqqt2FBt/FBt_purity‚1 ("Purity/*FRt3SRqqt2FBt/event_purity‚0 7 Purity/0FRt0SRqqt3FBt/FBt_purity‚0.9200000762939453 *particle/sensitivity‚0.294413685798645 @&Purity/1FRt0SRqqt4FBt/event_proportion‚6.300403038039804e-05 8 Purity/3FRt2SRqqt0FBt/FRt_purity‚0.11714975535869598 +jet/accuracy_2_of_2‚0.10854209959506989 ("Purity/2FRt4SRqqt0FBt/event_purity‚0 & Purity/1FRt0SRqqt4FBt/FBt_purity‚1 7 Purity/1FRt4SRqqt*FBt/FRt_purity‚0.3333333432674408 7 Purity/*FRt0SRqqt2FBt/FBt_purity‚0.9885271191596985 ,&Purity/3FRt3SRqqt3FBt/event_proportion‚0 8 Purity/*FRt3SRqqt*FBt/FRt_purity‚0.16085723042488098 >&Purity/0FRt*SRqqt2FBt/event_proportion‚0.03944052383303642 =&Purity/*FRt0SRqqt*FBt/event_proportion‚0.3558467626571655 *jet/accuracy_1_of_4‚0.8980698585510254 particle/accuracy_3_of_5‚1 6 Purity/*FRt1SRqqt*FBt/FBt_purity‚0.990800142288208 @&Purity/0FRt*SRqqt3FBt/event_proportion‚0.0030871976632624865 9"Purity/0FRt1SRqqt2FBt/SRqqt_purity‚0.5089947581291199 8"Purity/1FRt1SRqqt1FBt/SRqqt_purity‚0.575208306312561 7 Purity/3FRt*SRqqt1FBt/FRt_purity‚0.3583333492279053 *"Purity/3FRt1SRqqt2FBt/SRqqt_purity‚0.5 @&Purity/*FRt4SRqqt1FBt/event_proportion‚6.300403038039804e-05 ,&Purity/4FRt*SRqqt4FBt/event_proportion‚0 particle/accuracy_1_of_7‚1 =&Purity/*FRt*SRqqt0FBt/event_proportion‚0.6160534024238586 6 Purity/1FRt1SRqqt2FBt/FRt_purity‚0.392687052488327 7 Purity/1FRt3SRqqt1FBt/FRt_purity‚0.2888889014720917 & Purity/*FRt1SRqqt3FBt/FRt_purity‚0 ("Purity/0FRt2SRqqt3FBt/SRqqt_purity‚1 ("Purity/0FRt0SRqqt0FBt/event_purity‚1 8 Purity/1FRt*SRqqt1FBt/FRt_purity‚0.35555925965309143 8 Purity/2FRt0SRqqt*FBt/FRt_purity‚0.22357922792434692 *jet/accuracy_1_of_1‚0.4695148169994354 particle/accuracy_6_of_7‚1 7 Purity/*FRt1SRqqt1FBt/FBt_purity‚0.9970335364341736 A&Purity/0FRt*SRqqt4FBt/event_proportion‚0.00025201612152159214 7 Purity/1FRt0SRqqt*FBt/FRt_purity‚0.2534314692020416 8 Purity/1FRt2SRqqt0FBt/FRt_purity‚0.13723468780517578 ("Purity/1FRt3SRqqt0FBt/event_purity‚0 ("Purity/1FRt3SRqqt2FBt/event_purity‚0 & Purity/4FRt0SRqqt0FBt/FRt_purity‚0 @&Purity/0FRt3SRqqt1FBt/event_proportion‚0.0006300403038039804 >&Purity/2FRt*SRqqt*FBt/event_proportion‚0.10244455933570862 ("Purity/2FRt2SRqqt0FBt/event_purity‚0 @&Purity/4FRt1SRqqt0FBt/event_proportion‚6.300403038039804e-05 'particle/f_score‚0.3862621486186981 particle/accuracy_6_of_6‚1 ("Purity/*FRt2SRqqt3FBt/SRqqt_purity‚1 ?&Purity/*FRt*SRqqt3FBt/event_proportion‚0.004473286215215921 ?&Purity/*FRt3SRqqt1FBt/event_proportion‚0.002142137149348855 ,&Purity/3FRt4SRqqt2FBt/event_proportion‚0 particle/accuracy_2_of_2‚1 7 Purity/*FRt1SRqqt1FBt/FRt_purity‚0.3303362727165222 8 Purity/2FRt*SRqqt*FBt/FRt_purity‚0.19861142337322235 & Purity/3FRt*SRqqt*FBt/FBt_purity‚1 & Purity/0FRt2SRqqt2FBt/FBt_purity‚1 =&Purity/1FRt*SRqqt*FBt/event_proportion‚0.4366179406642914 7 Purity/1FRt*SRqqt1FBt/FBt_purity‚0.9994797110557556 9"Purity/2FRt*SRqqt0FBt/SRqqt_purity‚0.4514434039592743 >&Purity/2FRt1SRqqt*FBt/event_proportion‚0.04158266261219978 ,&Purity/4FRt1SRqqt2FBt/event_proportion‚0 ,&Purity/4FRt3SRqqt4FBt/event_proportion‚0 :"Purity/*FRt1SRqqt1FBt/event_purity‚0.10275526344776154 7 Purity/0FRt1SRqqt*FBt/FBt_purity‚0.9902821779251099 & Purity/1FRt3SRqqt*FBt/FBt_purity‚1 7 Purity/2FRt1SRqqt2FBt/FRt_purity‚0.4166666865348816 @&Purity/3FRt*SRqqt0FBt/event_proportion‚0.0071824598126113415 8 Purity/3FRt0SRqqt0FBt/FRt_purity‚0.09803920984268188 )jet/accuracy_1_of_5‚0.914513349533081 particle/accuracy_5_of_6‚1 ;"Purity/1FRt1SRqqt*FBt/event_purity‚0.020242972299456596 9"Purity/1FRt1SRqqt*FBt/SRqqt_purity‚0.5014470815658569 ("Purity/2FRt1SRqqt0FBt/event_purity‚0 ,&Purity/2FRt1SRqqt3FBt/event_proportion‚0 ,&Purity/4FRt4SRqqt0FBt/event_proportion‚0 +jet/accuracy_4_of_5‚0.01075268816202879 particle/accuracy_5_of_5‚1 ("Purity/*FRt0SRqqt4FBt/event_purity‚0 & Purity/3FRt*SRqqt1FBt/FBt_purity‚1 ,&Purity/4FRt*SRqqt1FBt/event_proportion‚0 +jet/accuracy_4_of_6‚0.02631578966975212 7 Purity/3FRt1SRqqt2FBt/FRt_purity‚0.1666666716337204 ("Purity/*FRt1SRqqt4FBt/event_purity‚0 * Purity/4FRt*SRqqt0FBt/FRt_purity‚0.125 7 Purity/1FRt*SRqqt*FBt/FBt_purity‚0.9928846955299377 ?&Purity/1FRt3SRqqt*FBt/event_proportion‚0.009135584346950054 @&Purity/1FRt3SRqqt1FBt/event_proportion‚0.0011340725468471646 ,&Purity/2FRt2SRqqt3FBt/event_proportion‚0 ,&Purity/4FRt3SRqqt0FBt/event_proportion‚0 ("Purity/1FRt*SRqqt4FBt/SRqqt_purity‚1 +"Purity/1FRt*SRqqt3FBt/SRqqt_purity‚0.75 ("Purity/4FRt1SRqqt*FBt/event_purity‚0 8 Purity/*FRt*SRqqt*FBt/FRt_purity‚0.20710377395153046 7 Purity/*FRt0SRqqt2FBt/FRt_purity‚0.5285031199455261 ,&Purity/*FRt2SRqqt4FBt/event_proportion‚0 ,&Purity/*FRt4SRqqt2FBt/event_proportion‚0 7 Purity/0FRt1SRqqt1FBt/FBt_purity‚0.9955925345420837 :"Purity/0FRt2SRqqt0FBt/event_purity‚0.06241914629936218 :"Purity/1FRt*SRqqt0FBt/SRqqt_purity‚0.43540725111961365 ("Purity/1FRt2SRqqt0FBt/event_purity‚0 :"Purity/1FRt0SRqqt1FBt/event_purity‚0.31030845642089844 ?&Purity/*FRt2SRqqt2FBt/event_proportion‚0.002079133177176118 , Purity/1FRt*SRqqt3FBt/FRt_purity‚0.46875 & Purity/1FRt3SRqqt1FBt/FBt_purity‚1 ,&Purity/1FRt4SRqqt3FBt/event_proportion‚0 @&Purity/3FRt*SRqqt1FBt/event_proportion‚0.0024571572430431843 @&Purity/3FRt2SRqqt1FBt/event_proportion‚0.0003780241822823882 particle/accuracy_1_of_5‚1 9"Purity/0FRt*SRqqt2FBt/SRqqt_purity‚0.5000000596046448 ("Purity/1FRt3SRqqt*FBt/event_purity‚0 :"Purity/1FRt3SRqqt*FBt/SRqqt_purity‚0.35547447204589844 ,&Purity/2FRt4SRqqt1FBt/event_proportion‚0 ,&Purity/4FRt2SRqqt2FBt/event_proportion‚0 particle/accuracy_2_of_7‚1 ("Purity/1FRt4SRqqt*FBt/event_purity‚0 9"Purity/1FRt4SRqqt*FBt/SRqqt_purity‚0.1666666716337204 :"Purity/0FRt1SRqqt0FBt/SRqqt_purity‚0.47332024574279785 ?&Purity/2FRt0SRqqt*FBt/event_proportion‚0.037046369165182114 ?&Purity/2FRt1SRqqt1FBt/event_proportion‚0.007875503972172737 ,&Purity/3FRt3SRqqt2FBt/event_proportion‚0 & Purity/0FRt4SRqqt*FBt/FBt_purity‚1 & Purity/*FRt*SRqqt4FBt/FRt_purity‚0 7 Purity/3FRt3SRqqt0FBt/FRt_purity‚0.2142857313156128 ("Purity/4FRt*SRqqt*FBt/event_purity‚0 9"Purity/*FRt1SRqqt1FBt/SRqqt_purity‚0.5382260084152222 8 Purity/*FRt2SRqqt0FBt/FRt_purity‚0.13332627713680267 @&Purity/2FRt2SRqqt2FBt/event_proportion‚6.300403038039804e-05 :"Purity/3FRt2SRqqt0FBt/SRqqt_purity‚0.37137681245803833 ,&Purity/3FRt2SRqqt3FBt/event_proportion‚0 ("Purity/*FRt*SRqqt4FBt/event_purity‚0 jet/accuracy_5_of_6‚0 ("Purity/3FRt4SRqqt*FBt/event_purity‚0 >&Purity/*FRt*SRqqt1FBt/event_proportion‚0.27079132199287415 <"Purity/*FRt3SRqqt*FBt/event_purity‚0.0029325513169169426 9"Purity/*FRt3SRqqt*FBt/SRqqt_purity‚0.3520544171333313 ,&Purity/0FRt4SRqqt4FBt/event_proportion‚0 7 Purity/1FRt1SRqqt2FBt/FBt_purity‚0.9732142686843872 +"Purity/3FRt4SRqqt*FBt/SRqqt_purity‚0.25 jet/accuracy_2_of_7‚0 9"Purity/*FRt*SRqqt3FBt/SRqqt_purity‚0.7619047164916992 9"Purity/*FRt0SRqqt3FBt/event_purity‚0.0892857164144516 A&Purity/1FRt*SRqqt4FBt/event_proportion‚0.00012600806076079607 ,&Purity/1FRt3SRqqt3FBt/event_proportion‚0 *"Purity/4FRt*SRqqt*FBt/SRqqt_purity‚0.5 ("Purity/4FRt1SRqqt*FBt/SRqqt_purity‚0 7 Purity/*FRt2SRqqt3FBt/FBt_purity‚0.6666666865348816 @&Purity/2FRt*SRqqt3FBt/event_proportion‚6.300403038039804e-05 ,&Purity/2FRt*SRqqt4FBt/event_proportion‚0 ,&Purity/4FRt1SRqqt1FBt/event_proportion‚0 & Purity/0FRt4SRqqt1FBt/FBt_purity‚1 ("Purity/1FRt1SRqqt4FBt/SRqqt_purity‚1 *"Purity/4FRt*SRqqt0FBt/SRqqt_purity‚0.5 ("Purity/1FRt1SRqqt2FBt/event_purity‚0 8 Purity/2FRt2SRqqt*FBt/FRt_purity‚0.14428581297397614 * Purity/*FRt*SRqqt4FBt/FBt_purity‚0.875 7 Purity/*FRt4SRqqt0FBt/FRt_purity‚0.2638888955116272 ("Purity/2FRt2SRqqt2FBt/SRqqt_purity‚0 ?&Purity/0FRt0SRqqt2FBt/event_proportion‚0.0296748988330364234QõTa¼ ±§ @&Purity/4FRt0SRqqt*FBt/event_proportion‚6.300403038039804e-05 +jet/accuracy_2_of_5‚0.41565755009651184 ("Purity/3FRt2SRqqt1FBt/event_purity‚0 -"Purity/0FRt3SRqqt1FBt/SRqqt_purity‚0.4375 ("Purity/*FRt1SRqqt4FBt/SRqqt_purity‚1 8 Purity/*FRt0SRqqt1FBt/FRt_purity‚0.40624064207077026 9"Purity/0FRt2SRqqt0FBt/SRqqt_purity‚0.4075649082660675 @&Purity/0FRt2SRqqt3FBt/event_proportion‚6.300403038039804e-05 :"Purity/1FRt*SRqqt0FBt/event_purity‚0.07126323878765106 7 Purity/1FRt1SRqqt1FBt/FBt_purity‚0.9988052845001221 :"Purity/1FRt2SRqqt0FBt/SRqqt_purity‚0.40822798013687134 9"Purity/2FRt1SRqqt1FBt/SRqqt_purity‚0.6000000834465027 @&Purity/3FRt3SRqqt*FBt/event_proportion‚0.0005670362734235823 :"Purity/1FRt*SRqqt*FBt/event_purity‚0.09583548456430435 :"Purity/1FRt*SRqqt*FBt/SRqqt_purity‚0.45532092452049255 9"Purity/2FRt1SRqqt2FBt/SRqqt_purity‚0.7412280440330505 ,&Purity/4FRt4SRqqt3FBt/event_proportion‚0 *jet/accuracy_1_of_6‚0.9824560880661011 ("Purity/1FRt1SRqqt4FBt/event_purity‚0 ("Purity/4FRt*SRqqt0FBt/event_purity‚0 :"Purity/*FRt1SRqqt*FBt/event_purity‚0.20619556307792664 8"Purity/*FRt1SRqqt*FBt/SRqqt_purity‚0.493158221244812 7 Purity/*FRt1SRqqt2FBt/FRt_purity‚0.3620350956916809 6 Purity/*FRt2SRqqt*FBt/FRt_purity‚0.153945192694664 >&Purity/0FRt2SRqqt*FBt/event_proportion‚0.05361643061041832 :"Purity/1FRt*SRqqt2FBt/event_purity‚0.11141553521156311 & Purity/1FRt1SRqqt3FBt/FRt_purity‚0 =&Purity/*FRt*SRqqt*FBt/event_proportion‚0.9591733813285828 :"Purity/1FRt*SRqqt1FBt/event_purity‚0.16953250765800476 ?&Purity/1FRt3SRqqt0FBt/event_proportion‚0.007938507944345474 <"Purity/2FRt*SRqqt0FBt/event_purity‚0.0040725404396653175 ,&Purity/3FRt*SRqqt3FBt/event_proportion‚0 7 Purity/1FRt1SRqqt*FBt/FBt_purity‚0.9913498759269714 ;"Purity/2FRt*SRqqt2FBt/event_purity‚0.011111111380159855 7 Purity/2FRt1SRqqt1FBt/FRt_purity‚0.3931259512901306 >&Purity/*FRt0SRqqt1FBt/event_proportion‚0.14591734111309052 :"Purity/2FRt1SRqqt0FBt/SRqqt_purity‚0.48947006464004517 8 Purity/3FRt1SRqqt1FBt/FRt_purity‚0.38461539149284363 ,&Purity§½ù/4FRt*SRqqt2FBt/event_proportion‚0 @&Purity/0FRt4SRqqt*FBt/event_proportion‚0.0005040322430431843 7 Purity/1FRt*SRqqt2FBt/FRt_purity‚0.5245682597160339 ,&Purity/4FRt4SRqqt2FBt/event_proportion‚0 particle/accuracy_2_of_3‚1 % _timestamp‚1.7885030323192105e+09 7 Purity/*FRt0SRqqt3FBt/FRt_purity‚0.5333333611488342 7 Purity/*FRt3SRqqt0FBt/FRt_purity‚0.1702105700969696 :"Purity/1FRt3SRqqt0FBt/SRqqt_purity‚0.32165902853012085 ,&Purity/4FRt4SRqqt*FBt/event_proportion‚0 ?&Purity/*FRt0SRqqt3FBt/event_proportion‚0.003717237850651145 9"Purity/0FRt*SRqqt1FBt/SRqqt_purity‚0.4918311536312103 @&Purity/1FRt1SRqqt4FBt/event_proportion‚6.300403038039804e-05 9"Purity/2FRt2SRqqt0FBt/SRqqt_purity‚0.4574425518512726 @&Purity/3FRt*SRqqt2FBt/event_proportion‚0.0003150201519019902 & Purity/3FRt1SRqqt1FBt/FBt_purity‚1 particle/accuracy_3_of_6‚1 8 Purity/3FRt2SRqqt*FBt/FRt_purity‚0.14814814925193787 7 Purity/*FRt0SRqqt*FBt/FBt_purity‚0.9917316436767578 :"Purity/*FRt0SRqqt2FBt/event_purity‚0.25607988238334656 :"Purity/*FRt3SRqqt1FBt/SRqqt_purity‚0.39550262689590454 =&Purity/0FRt1SRqqt*FBt/event_proportion‚0.2305317521095276 ;"Purity/2FRt*SRqqt*FBt/event_purity‚0.009259711019694805 9"Purity/2FRt*SRqqt*FBt/SRqqt_purity‚0.4774433374404907 & Purity/2FRt1SRqqt1FBt/FBt_purity‚1 A&Purity/0FRt0SRqqt4FBt/event_proportion‚0.00025201612152159214 @&Purity/1FRt0SRqqt3FBt/event_proportion‚0.0010710685746744275 :"Purity/1FRt2SRqqt1FBt/SRqqt_purity‚0.39420127868652344 ("Purity/3FRt1SRqqt*FBt/event_purity‚0 :"Purity/3FRt1SRqqt*FBt/SRqqt_purity‚0.42592597007751465 ,&Purity/3FRt1SRqqt3FBt/event_proportion‚0 particle/accuracy_3_of_7‚1 7 Purity/1FRt4SRqqt0FBt/FRt_purity‚0.3333333432674408 ?&Purity/*FRt3SRqqt*FBt/event_proportion‚0.020476309582591057 ,&Purity/1FRt2SRqqt3FBt/event_proportion‚0 ;"Purity/2FRt*SRqqt1FBt/event_purity‚0.027065463364124298 ("Purity/2FRt3SRqqt*FBt/event_purity‚0 :"Purity/2FRt3SRqqt*FBt/SRqqt_purity‚0.24228572845458984 ,&Purity/3FRt3SRqqt1FBt/event_proportion‚0 8 Purity/3FRt*SRqqt2FBt/FRt_purity‚0.20000000298023224 9"Purity/*FRt1SRqqt3FBt/SRqqt_purity‚0.7619047164916992 & Purity/3FRt1SRqqt*FBt/FBt_purity‚1 ;"Purity/2FRt0SRqqt0FBt/event_purity‚0.013001693412661552 8 Purity/2FRt3SRqqt*FBt/FRt_purity‚0.13147619366645813 ?&Purity/2FRt3SRqqt0FBt/event_proportion‚0.003654233878478408 9"Purity/0FRt*SRqqt3FBt/SRqqt_purity‚0.6000000238418579 _runtime‚2108.400911616 7 Purity/2FRt1SRqqt*FBt/FBt_purity‚0.9964157938957214 =&Purity/*FRt1SRqqt1FBt/event_proportion‚0.1001134067773819 & Purity/*FRt0SRqqt1FBt/FBt_purity‚1 7 Purity/1FRt1SRqqt1FBt/FRt_purity‚0.3093251585960388 ?&Purity/2FRt*SRqqt1FBt/event_proportion‚0.021169355139136314 ?&Purity/2FRt0SRqqt1FBt/event_proportion‚0.008946572430431843 & Purity/1FRt3SRqqt2FBt/FRt_purity‚1 :"Purity/0FRt1SRqqt*FBt/event_purity‚0.38540440797805786 8"Purity/0FRt1SRqqt*FBt/SRqqt_purity‚0.482045978307724 ,&Purity/0FRt3SRqqt3FBt/event_proportion‚0 ,&Purity/1FRt2SRqqt4FBt/event_proportion‚0 :"Purity/2FRt0SRqqt*FBt/event_purity‚0.02383982203900814 ,&Purity/4FRt*SRqqt3FBt/event_proportion‚0 9"Purity/2FRt4SRqqt0FBt/SRqqt_purity‚0.4000000059604645 @&Purity/*FRt4SRqqt0FBt/event_proportion‚0.0012600806076079607 8"Purity/0FRt*SRqqt1FBt/event_purity‚0.643706202507019 >&Purity/0FRt1SRqqt1FBt/event_proportion‚0.05185231938958168 7 Purity/1FRt0SRqqt3FBt/FBt_purity‚0.9523809552192688 8 Purity/2FRt0SRqqt0FBt/FRt_purity‚0.14386755228042603 ("Purity/*FRt3SRqqt1FBt/event_purity‚0 ?&Purity/0FRt1SRqqt2FBt/event_proportion‚0.008757560513913631 >&Purity/1FRt0SRqqt0FBt/event_proportion‚0.13142640888690948 7 Purity/1FRt0SRqqt0FBt/FRt_purity‚0.1551836133003235 7 Purity/1FRt2SRqqt*FBt/FRt_purity‚0.1584923416376114 * Purity/0FRt0SRqqt4FBt/FBt_purity‚0.875 <"Purity/0FRt3SRqqt*FBt/event_purity‚0.0071428571827709675 :"Purity/0FRt3SRqqt*FBt/SRqqt_purity‚0.40017005801200867 9"Purity/*FRt0SRqqt1FBt/event_purity‚0.7005100250244141 7 Purity/*FRt1SRqqt2FBt/FBt_purity‚0.9816062450408936 @&Purity/*FRt1SRqqt3FBt/event_proportion‚0.0006930443341843784 ;"Purity/*FRt2SRqqt0FBt/event_purity‚0.024302344769239426 :"Purity/0FRt2SRqqt*FBt/event_purity‚0.06023978814482689 :"Purity/0FRt2SRqqt*FBt/SRqqt_purity‚0.42005398869514465 ("Purity/1FRt2SRqqt1FBt/event_purity‚0 ,&Purity/1FRt4SRqqt4FBt/event_proportion‚0 :"Purity/*FRt3SRqqt0FBt/SRqqt_purity‚0.34414586424827576 9"Purity/2FRt*SRqqt1FBt/SRqqt_purity‚0.5635807514190674 :"Purity/2FRt3SRqqt0FBt/SRqqt_purity‚0.25115442276000977 * Purity/0FRt*SRqqt4FBt/FBt_purity‚0.875 ("Purity/0FRt1SRqqt3FBt/event_purity‚0 A&Purity/3FRt4SRqqt0FBt/event_proportion‚0.00012600806076079607 & Purity/2FRt0SRqqt3FBt/FBt_purity‚1 & Purity/4FRt0SRqqt*FBt/FRt_purity‚0 ("Purity/3FRt*SRqqt0FBt/event_purity‚0 ("Purity/1FRt2SRqqt2FBt/event_purity‚0 :"Purity/*FRt2SRqqt0FBt/SRqqt_purity‚0.41437655687332153 >&Purity/0FRt*SRqqt0FBt/event_proportion‚0.21931703388690948 8 Purity/2FRt3SRqqt0FBt/FRt_purity‚0.13382035493850708 ("Purity/3FRt0SRqqt*FBt/event_purity‚0 ,&Purity/3FRt3SRqqt4FBt/event_proportion‚0 ,&Purity/4FRt0SRqqt3FBt/event_proportion‚0 jet/accuracy_6_of_6‚0 =&Purity/*FRt0SRqqt2FBt/event_proportion‚0.0483240932226181 ;"Purity/*FRt3SRqqt0FBt/event_purity‚0.003225806402042508 @&Purity/*FRt4SRqqt*FBt/event_proportion‚0.0013230846961960196 7"Purity/1FRt1SRqqt1FBt/event_purity‚0.04088119789958 ,&Purity/2FRt2SRqqt4FBt/event_proportion‚0 ("Purity/2FRt3SRqqt0FBt/event_purity‚0 ,&Purity/4FRt0SRqqt4FBt/event_proportion‚0 7 Purity/0FRt*SRqqt3FBt/FBt_purity‚0.9094650745391846 9"Purity/0FRt0SRqqt*FBt/event_purity‚0.8111569881439209 7 Purity/2FRt1SRqqt2FBt/FBt_purity‚0.9868420958518982 @&Purity/3FRt3SRqqt0FBt/event_proportion‚0.0005670362734235823 ("Purity/0FRt0SRqqt4FBt/event_purity‚0 +jet/accuracy_3_of_6‚0.07894736528396606 ("Purity/3FRt*SRqqt2FBt/event_purity‚0 & Purity/1FRt0SRqqt4FBt/FRt_purity‚0 :"Purity/3FRt*SRqqt0FBt/SRqqt_purity‚0.38703233003616333 & Purity/3FRt1SRqqt2FBt/FBt_purity‚1 ("Purity/*FRt3SRqqt2FBt/event_purity‚0 7 Purity/0FRt0SRqqt3FBt/FBt_purity‚0.9200000762939453 *particle/sensitivity‚0.294413685798645 @&Purity/1FRt0SRqqt4FBt/event_proportion‚6.300403038039804e-05 8 Purity/3FRt2SRqqt0FBt/FRt_purity‚0.11714975535869598 +jet/accuracy_2_of_2‚0.10854209959506989 ("Purity/2FRt4SRqqt0FBt/event_purity‚0 & Purity/1FRt0SRqqt4FBt/FBt_purity‚1 7 Purity/1FRt4SRqqt*FBt/FRt_purity‚0.3333333432674408 7 Purity/*FRt0SRqqt2FBt/FBt_purity‚0.9885271191596985 ,&Purity/3FRt3SRqqt3FBt/event_proportion‚0 8 Purity/*FRt3SRqqt*FBt/FRt_purity‚0.16085723042488098 >&Purity/0FRt*SRqqt2FBt/event_proportion‚0.03944052383303642 =&Purity/*FRt0SRqqt*FBt/event_proportion‚0.3558467626571655 *jet/accuracy_1_of_4‚0.8980698585510254 particle/accuracy_3_of_5‚1 6 Purity/*FRt1SRqqt*FBt/FBt_purity‚0.990800142288208 @&Purity/0FRt*SRqqt3FBt/event_proportion‚0.0030871976632624865 9"Purity/0FRt1SRqqt2FBt/SRqqt_purity‚0.5089947581291199 8"Purity/1FRt1SRqqt1FBt/SRqqt_purity‚0.575208306312561 7 Purity/3FRt*SRqqt1FBt/FRt_purity‚0.3583333492279053 *"Purity/3FRt1SRqqt2FBt/SRqqt_purity‚0.5 @&Purity/*FRt4SRqqt1FBt/event_proportion‚6.300403038039804e-05 ,&Purity/4FRt*SRqqt4FBt/event_proportion‚0 particle/accuracy_1_of_7‚1 =&Purity/*FRt*SRqqt0FBt/event_proportion‚0.6160534024238586 6 Purity/1FRt1SRqqt2FBt/FRt_purity‚0.392687052488327 7 Purity/1FRt3SRqqt1FBt/FRt_purity‚0.2888889014720917 & Purity/*FRt1SRqqt3FBt/FRt_purity‚0 ("Purity/0FRt2SRqqt3FBt/SRqqt_purity‚1 ("Purity/0FRt0SRqqt0FBt/event_purity‚1 8 Purity/1FRt*SRqqt1FBt/FRt_purity‚0.35555925965309143 8 Purity/2FRt0SRqqt*FBt/FRt_purity‚0.22357922792434692 *jet/accuracy_1_of_1‚0.4695148169994354 particle/accuracy_6_of_7‚1 7 Purity/*FRt1SRqqt1FBt/FBt_purity‚0.9970335364341736 A&Purity/0FRt*SRqqt4FBt/event_proportion‚0.00025201612152159214 7 Purity/1FRt0SRqqt*FBt/FRt_purity‚0.2534314692020416 8 Purity/1FRt2SRqqt0FBt/FRt_purity‚0.13723468780517578 ("Purity/1FRt3SRqqt0FBt/event_purity‚0 ("Purity/1FRt3SRqqt2FBt/event_purity‚0 & Purity/4FRt0SRqqt0FBt/FRt_purity‚0 @&Purity/0FRt3SRqqt1FBt/event_proportion‚0.0006300403038039804 >&Purity/2FRt*SRqqt*FBt/event_proportion‚0.10244455933570862 ("Purity/2FRt2SRqqt0FBt/event_purity‚0 @&Purity/4FRt1SRqqt0FBt/event_proportion‚6.300403038039804e-05 'particle/f_score‚0.3862621486186981 particle/accuracy_6_of_6‚1 ("Purity/*FRt2SRqqt3FBt/SRqqt_purity‚1 ?&Purity/*FRt*SRqqt3FBt/event_proportion‚0.004473286215215921 ?&Purity/*FRt3SRqqt1FBt/event_proportion‚0.002142137149348855 ,&Purity/3FRt4SRqqt2FBt/event_proportion‚0 particle/accuracy_2_of_2‚1 7 Purity/*FRt1SRqqt1FBt/FRt_purity‚0.3303362727165222 8 Purity/2FRt*SRqqt*FBt/FRt_purity‚0.19861142337322235 & Purity/3FRt*SRqqt*FBt/FBt_purity‚1 & Purity/0FRt2SRqqt2FBt/FBt_purity‚1 =&Purity/1FRt*SRqqt*FBt/event_proportion‚0.4366179406642914 7 Purity/1FRt*SRqqt1FBt/FBt_purity‚0.9994797110557556 9"Purity/2FRt*SRqqt0FBt/SRqqt_purity‚0.4514434039592743 >&Purity/2FRt1SRqqt*FBt/event_proportion‚0.04158266261219978 ,&Purity/4FRt1SRqqt2FBt/event_proportion‚0 ,&Purity/4FRt3SRqqt4FBt/event_proportion‚0 :"Purity/*FRt1SRqqt1FBt/event_purity‚0.10275526344776154 7 Purity/0FRt1SRqqt*FBt/FBt_purity‚0.9902821779251099 & Purity/1FRt3SRqqt*FBt/FBt_purity‚1 7 Purity/2FRt1SRqqt2FBt/FRt_purity‚0.4166666865348816 @&Purity/3FRt*SRqqt0FBt/event_proportion‚0.0071824598126113415 8 Purity/3FRt0SRqqt0FBt/FRt_purity‚0.09803920984268188 )jet/accuracy_1_of_5‚0.914513349533081 particle/accuracy_5_of_6‚1 ;"Purity/1FRt1SRqqt*FBt/event_purity‚0.020242972299456596 9"Purity/1FRt1SRqqt*FBt/SRqqt_purity‚0.5014470815658569 ("Purity/2FRt1SRqqt0FBt/event_purity‚0 ,&Purity/2FRt1SRqqt3FBt/event_proportion‚0 ,&Purity/4FRt4SRqqt0FBt/event_proportion‚0 +jet/accuracy_4_of_5‚0.01075268816202879 particle/accuracy_5_of_5‚1 ("Purity/*FRt0SRqqt4FBt/event_purity‚0 & Purity/3FRt*SRqqt1FBt/FBt_purity‚1 ,&Purity/4FRt*SRqqt1FBt/event_proportion‚0 +jet/accuracy_4_of_6‚0.02631578966975212 7 Purity/3FRt1SRqqt2FBt/FRt_purity‚0.1666666716337204 ("Purity/*FRt1SRqqt4FBt/event_purity‚0 * Purity/4FRt*SRqqt0FBt/FRt_purity‚0.125 7 Purity/1FRt*SRqqt*FBt/FBt_purity‚0.9928846955299377 ?&Purity/1FRt3SRqqt*FBt/event_proportion‚0.009135584346950054 @&Purity/1FRt3SRqqt1FBt/event_proportion‚0.0011340725468471646 ,&Purity/2FRt2SRqqt3FBt/event_proportion‚0 ,&Purity/4FRt3SRqqt0FBt/event_proportion‚0 ("Purity/1FRt*SRqqt4FBt/SRqqt_purity‚1 +"Purity/1FRt*SRqqt3FBt/SRqqt_purity‚0.75 ("Purity/4FRt1SRqqt*FBt/event_purity‚0 8 Purity/*FRt*SRqqt*FBt/FRt_purity‚0.20710377395153046 7 Purity/*FRt0SRqqt2FBt/FRt_purity‚0.5285031199455261 ,&Purity/*FRt2SRqqt4FBt/event_proportion‚0 ,&Purity/*FRt4SRqqt2FBt/event_proportion‚0 7 Purity/0FRt1SRqqt1FBt/FBt_purity‚0.9955925345420837 :"Purity/0FRt2SRqqt0FBt/event_purity‚0.06241914629936218 :"Purity/1FRt*SRqqt0FBt/SRqqt_purity‚0.43540725111961365 ("Purity/1FRt2SRqqt0FBt/event_purity‚0 :"Purity/1FRt0SRqqt1FBt/event_purity‚0.31030845642089844 ?&Purity/*FRt2SRqqt2FBt/event_proportion‚0.002079133177176118 , Purity/1FRt*SRqqt3FBt/FRt_purity‚0.46875 & Purity/1FRt3SRqqt1FBt/FBt_purity‚1 ,&Purity/1FRt4SRqqt3FBt/event_proportion‚0 @&Purity/3FRt*SRqqt1FBt/event_proportion‚0.0024571572430431843 @&Purity/3FRt2SRqqt1FBt/event_proportion‚0.0003780241822823882 particle/accuracy_1_of_5‚1 9"Purity/0FRt*SRqqt2FBt/SRqqt_purity‚0.5000000596046448 ("Purity/1FRt3SRqqt*FBt/event_purity‚0 :"Purity/1FRt3SRqqt*FBt/SRqqt_purity‚0.35547447204589844 ,&Purity/2FRt4SRqqt1FBt/event_proportion‚0 ,&Purity/4FRt2SRqqt2FBt/event_proportion‚0 particle/accuracy_2_of_7‚1 ("Purity/1FRt4SRqqt*FBt/event_purity‚0 9"Purity/1FRt4SRqqt*FBt/SRqqt_purity‚0.1666666716337204 :"Purity/0FRt1SRqqt0FBt/SRqqt_purity‚0.47332024574279785 ?&Purity/2FRt0SRqqt*FBt/event_proportion‚0.037046369165182114 ?&Purity/2FRt1SRqqt1FBt/event_proportion‚0.007875503972172737 ,&Purity/3FRt3SRqqt2FBt/event_proportion‚0 & Purity/0FRt4SRqqt*FBt/FBt_purity‚1 & Purity/*FRt*SRqqt4FBt/FRt_purity‚0 7 Purity/3FRt3SRqqt0FBt/FRt_purity‚0.2142857313156128 ("Purity/4FRt*SRqqt*FBt/event_purity‚0 9"Purity/*FRt1SRqqt1FBt/SRqqt_purity‚0.5382260084152222 8 Purity/*FRt2SRqqt0FBt/FRt_purity‚0.13332627713680267 @&Purity/2FRt2SRqqt2FBt/event_proportion‚6.300403038039804e-05 :"Purity/3FRt2SRqqt0FBt/SRqqt_purity‚0.37137681245803833 ,&Purity/3FRt2SRqqt3FBt/event_proportion‚0 ("Purity/*FRt*SRqqt4FBt/event_purity‚0 jet/accuracy_5_of_6‚0 ("Purity/3FRt4SRqqt*FBt/event_purity‚0 >&Purity/*FRt*SRqqt1FBt/event_proportion‚0.27079132199287415 <"Purity/*FRt3SRqqt*FBt/event_purity‚0.0029325513169169426 9"Purity/*FRt3SRqqt*FBt/SRqqt_purity‚0.3520544171333313 ,&Purity/0FRt4SRqqt4FBt/event_proportion‚0 7 Purity/1FRt1SRqqt2FBt/FBt_purity‚0.9732142686843872 +"Purity/3FRt4SRqqt*FBt/SRqqt_purity‚0.25 jet/accuracy_2_of_7‚0 9"Purity/*FRt*SRqqt3FBt/SRqqt_purity‚0.7619047164916992 9"Purity/*FRt0SRqqt3FBt/event_purity‚0.0892857164144516 A&Purity/1FRt*SRqqt4FBt/event_proportion‚0.00012600806076079607 ,&Purity/1FRt3SRqqt3FBt/event_proportion‚0 *"Purity/4FRt*SRqqt*FBt/SRqqt_purity‚0.5 ("Purity/4FRt1SRqqt*FBt/SRqqt_purity‚0 7 Purity/*FRt2SRqqt3FBt/FBt_purity‚0.6666666865348816 @&Purity/2FRt*SRqqt3FBt/event_proportion‚6.300403038039804e-05 ,&Purity/2FRt*SRqqt4FBt/event_proportion‚0 ,&Purity/4FRt1SRqqt1FBt/event_proportion‚0 & Purity/0FRt4SRqqt1FBt/FBt_purity‚1 ("Purity/1FRt1SRqqt4FBt/SRqqt_purity‚1 *"Purity/4FRt*SRqqt0FBt/SRqqt_purity‚0.5 ("Purity/1FRt1SRqqt2FBt/event_purity‚0 8 Purity/2FRt2SRqqt*FBt/FRt_purity‚0.14428581297397614 * Purity/*FRt*SRqqt4FBt/FBt_purity‚0.875 7 Purity/*FRt4SRqqt0FBt/FRt_purity‚0.2638888955116272 ("Purity/2FRt2SRqqt2FBt/SRqqt_purity‚0 ?&Purity/0FRt0SRqqt2FBt/event_proportion‚0.029674898833036423 @&Purity/1FRt4SRqqt0FBt/event_proportion‚0.0003780241822823882 >&Purity/2FRt1SRqqt0FBt/event_proportion‚0.03181703761219978 particle/accuracy_4_of_6‚1 & Purity/3FRt0SRqqt1FBt/FBt_purity‚1 particle/accuracy_1_of_2‚1 <"Purity/0FRt3SRqqt0FBt/event_purity‚0.0071428571827709675 @&Purity/*FRt0SRqqt4FBt/event_proportion‚0.0003150201519019902 9"Purity/0FRt*SRqqt2FBt/event_purity‚0.2553229033946991 ?&Purity/1FRt*SRqqt2FBt/event_proportion‚0.022303426638245583 ?&Purity/2FRt3SRqqt*FBt/event_proportion‚0.004032257944345474 5validation_average_jet_accuracy‚0.421050488948822 jet/accuracy_3_of_7‚0 & Purity/1FRt1SRqqt4FBt/FRt_purity‚0 ("Purity/2FRt2SRqqt2FBt/event_purity‚0 :"Purity/*FRt2SRqqt1FBt/SRqqt_purity‚0.47805899381637573 7 Purity/1FRt*SRqqt3FBt/FBt_purity‚0.9583333134651184 8 Purity/1FRt2SRqqt1FBt/FRt_purity‚0.26574504375457764 :"Purity/2FRt0SRqqt2FBt/event_purity‚0.01785714365541935 particle/accuracy_1_of_6‚1 & Purity/*FRt3SRqqt2FBt/FRt_purity‚1 7 Purity/*FRt*SRqqt2FBt/FBt_purity‚0.9858043789863586 :"Purity/*FRt*SRqqt0FBt/event_purity‚0.17130959033966064 =&Purity/0FRt1SRqqt0FBt/event_proportion‚0.1694808453321457 8 Purity/1FRt*SRqqt0FBt/FRt_purity‚0.14385321736335754 jet/accuracy_4_of_7‚0 ("Purity/3FRt3SRqqt0FBt/event_purity‚0 :"Purity/*FRt0SRqqt*FBt/event_purity‚0.38084423542022705 ?&Purity/1FRt2SRqqt1FBt/event_proportion‚0.008946572430431843 7 Purity/1FRt3SRqqt0FBt/FRt_purity‚0.1711597591638565 ?&Purity/2FRt0SRqqt0FBt/event_proportion‚0.024571573361754417 ?&Purity/2FRt2SRqqt0FBt/event_proportion‚0.015435988083481789 & Purity/3FRt0SRqqt*FBt/FBt_purity‚1 +jet/accuracy_3_of_4‚0.06692469865083694 ("Purity/4FRt1SRqqt0FBt/event_purity‚0 ,&Purity/*FRt4SRqqt3FBt/event_proportion‚0 9"Purity/1FRt0SRqqt0FBt/event_purity‚0.1551836133003235 7 Purity/2FRt*SRqqt2FBt/FBt_purity‚0.9911110997200012 :"Purity/3FRt1SRqqt0FBt/SRqqt_purity‚0.44999998807907104 7 Purity/0FRt1SRqqt3FBt/FBt_purity‚0.8999999761581421 9"Purity/0FRt2SRqqt1FBt/SRqqt_purity‚0.4907834529876709 8 Purity/3FRt*SRqqt0FBt/FRt_purity‚0.14253968000411987 ) Purity/1FRt1SRqqt4FBt/FBt_purity‚0.75 9"Purity/1FRt1SRqqt2FBt/SRqqt_purity‚0.5365645885467529 @&Purity/*FRt1SRqqt4FBt/event_proportion‚6.300403038039804e-05 & Purity/*FRt2SRqqt1FBt/FBt_purity‚1 7 Purity/1FRt0SRqqt2FBt/FRt_purity‚0.5763627290725708 & Purity/*FRt4SRqqt*FBt/FBt_purity‚1 jet/accuracy_5_of_7‚0 ("Purity/1FRt2SRqqt*FBt/event_purity‚0 :"Purity/1FRt2SRqqt*FBt/SRqqt_purity‚0.41354405879974365 8 Purity/2FRt0SRqqt2FBt/FRt_purity‚0.49047619104385376 ("Purity/2FRt4SRqqt*FBt/event_purity‚0 9"Purity/2FRt4SRqqt*FBt/SRqqt_purity‚0.4000000059604645 ;"Purity/*FRt2SRqqt1FBt/event_purity‚0.022938141599297523 >&Purity/1FRt0SRqqt*FBt/event_proportion‚0.19745463132858276 ?&Purity/1FRt1SRqqt2FBt/event_proportion‚0.006111391354352236 7 Purity/3FRt2SRqqt1FBt/FRt_purity‚0.2777777910232544 ("Purity/0FRt2SRqqt3FBt/event_purity‚0 ,&Purity/4FRt3SRqqt1FBt/event_proportion‚0 & Purity/2FRt2SRqqt2FBt/FBt_purity‚1 7 Purity/*FRt2SRqqt*FBt/FBt_purity‚0.9961516261100769 @&Purity/4FRt*SRqqt0FBt/event_proportion‚0.0001890120911411941 & Purity/*FRt4SRqqt1FBt/FBt_purity‚1 ("Purity/3FRt4SRqqt0FBt/event_purity‚0 ("Purity/1FRt0SRqqt4FBt/event_purity‚0 ("Purity/1FRt4SRqqt0FBt/event_purity‚0 ("Purity/4FRt2SRqqt0FBt/SRqqt_purity‚1 @&Purity/*FRt*SRqqt4FBt/event_proportion‚0.0003780241822823882 A&Purity/3FRt1SRqqt2FBt/event_proportion‚0.00025201612152159214 :"Purity/0FRt2SRqqt1FBt/event_purity‚0.05478750914335251 8 Purity/1FRt1SRqqt*FBt/FRt_purity‚0.18231597542762756 :"Purity/1FRt1SRqqt0FBt/SRqqt_purity‚0.47499245405197144 ,&Purity/4FRt1SRqqt4FBt/event_proportion‚0 @&Purity/3FRt0SRqqt2FBt/event_proportion‚6.300403038039804e-05 ,&Purity/3FRt0SRqqt4FBt/event_proportion‚0 8 Purity/2FRt3SRqqt1FBt/FRt_purity‚0.30000001192092896 ("Purity/3FRt3SRqqt*FBt/event_purity‚0 9"Purity/3FRt3SRqqt*FBt/SRqqt_purity‚0.4523809552192688 ("Purity/*FRt2SRqqt3FBt/event_purity‚0 ("Purity/4FRt2SRqqt*FBt/event_purity‚0 ("Purity/4FRt2SRqqt*FBt/SRqqt_purity‚1 :"Purity/*FRt*SRqqt*FBt/event_purity‚0.24101600050926208 9"Purity/*FRt*SRqqt*FBt/SRqqt_purity‚0.4574120342731476 :"Purity/0FRt1SRqqt2FBt/event_purity‚0.04753968492150307 ("Purity/3FRt1SRqqt2FBt/event_purity‚0 & Purity/4FRt2SRqqt*FBt/FRt_purity‚0 _step‚97 7 Purity/0FRt*SRqqt1FBt/FBt_purity‚0.9983571171760559 ,&Purity/3FRt0SRqqt3FBt/event_proportion‚0 ,&Purity/4FRt3SRqqt2FBt/event_proportion‚0 *jet/accuracy_1_of_3‚0.8399849534034729 ("Purity/0FRt0SRqqt1FBt/event_purity‚1 @&Purity/3FRt0SRqqt1FBt/event_proportion‚0.0011340725468471646 & Purity/2FRt3SRqqt1FBt/FBt_purity‚1 7 Purity/3FRt4SRqqt0FBt/FRt_purity‚0.1666666716337204 & Purity/*FRt3SRqqt2FBt/FBt_purity‚1 8 Purity/*FRt*SRqqt2FBt/FRt_purity‚0.46986985206604004 7 Purity/*FRt0SRqqt*FBt/FRt_purity‚0.2473047524690628 7 Purity/0FRt0SRqqt*FBt/FBt_purity‚0.9898556470870972 9"Purity/2FRt2SRqqt1FBt/SRqqt_purity‚0.5651785731315613 ,&Purity/2FRt3SRqqt4FBt/event_proportion‚0 ,&Purity/3FRt4SRqqt1FBt/event_proportion‚0 & Purity/3FRt2SRqqt*FBt/FBt_purity‚1 =&Purity/*FRt1SRqqt*FBt/event_proportion‚0.4482736885547638 @&Purity/*FRt2SRqqt3FBt/event_proportion‚6.300403038039804e-05 ."Purity/1FRt*SRqqt3FBt/event_purity‚0.15625 9"Purity/3FRt*SRqqt1FBt/SRqqt_purity‚0.6333333253860474 ,&Purity/4FRt2SRqqt4FBt/event_proportion‚0 9"Purity/1FRt3SRqqt2FBt/SRqqt_purity‚0.3333333432674408 8 Purity/*FRt2SRqqt1FBt/FRt_purity‚0.24782153964042664 6 Purity/0FRt*SRqqt*FBt/FBt_purity‚0.990129292011261 :"Purity/0FRt*SRqqt0FBt/event_purity‚0.37732023000717163 :"Purity/0FRt0SRqqt3FBt/event_purity‚0.09333334118127823 8 Purity/2FRt*SRqqt2FBt/FRt_purity‚0.46166667342185974 9"Purity/3FRt1SRqqt1FBt/SRqqt_purity‚0.5384615659713745 >&Purity/2FRt*SRqqt0FBt/event_proportion‚0.07579384744167328 ,&Purity/2FRt0SRqqt4FBt/event_proportion‚0 ("Purity/3FRt0SRqqt0FBt/event_purity‚0 8 Purity/3FRt1SRqqt*FBt/FRt_purity‚0.19547326862812042 & Purity/2FRt3SRqqt*FBt/FBt_purity‚1 9"Purity/1FRt0SRqqt2FBt/event_purity‚0.1716790497303009 ;"Purity/*FRt2SRqqt*FBt/event_purity‚0.023970510810613632 :"Purity/*FRt2SRqqt*FBt/SRqqt_purity‚0.42671146988868713 & Purity/0FRt0SRqqt1FBt/FBt_purity‚1 7 Purity/2FRt1SRqqt*FBt/FRt_purity‚0.2017263025045395 particle/accuracy_4_of_5‚1 & Purity/3FRt2SRqqt1FBt/FBt_purity‚1 epoch‚1 @&Purity/*FRt3SRqqt2FBt/event_proportion‚6.300403038039804e-05 7 Purity/1FRt0SRqqt2FBt/FBt_purity‚0.9874216318130493 >&Purity/1FRt2SRqqt*FBt/event_proportion‚0.05727066472172737 :"Purity/2FRt0SRqqt1FBt/event_purity‚0.07518518716096878 7 Purity/2FRt0SRqqt2FBt/FBt_purity‚0.9940475821495056 +jet/accuracy_3_of_3‚0.02672634832561016 ) Purity/4FRt1SRqqt0FBt/FRt_purity‚0.25 9"Purity/2FRt1SRqqt*FBt/SRqqt_purity‚0.5213163495063782 @&Purity/2FRt3SRqqt1FBt/event_proportion‚0.0003780241822823882 ?&Purity/1FRt0SRqqt1FBt/event_proportion‚0.049773186445236206 ?&Purity/1FRt0SRqqt2FBt/event_proportion‚0.015120967291295528 :"Purity/*FRt1SRqqt0FBt/event_purity‚0.24877195060253143 ?&Purity/2FRt0SRqqt2FBt/event_proportion‚0.003465221729129553 ;"Purity/2FRt1SRqqt*FBt/event_purity‚0.002016128972172737 ,&Purity/3FRt4SRqqt3FBt/event_proportion‚0 :"Purity/*FRt*SRqqt3FBt/event_purity‚0.07183908671140671 ,&Purity/0FRt2SRqqt4FBt/event_proportion‚0 ?&Purity/0FRt3SRqqt*FBt/event_proportion‚0.006741431541740894 7 Purity/1FRt2SRqqt*FBt/FBt_purity‚0.9959677457809448 ("Purity/2FRt2SRqqt1FBt/event_purity‚0 ,&Purity/3FRt*SRqqt4FBt/event_proportion‚0 @&Purity/3FRt2SRqqt*FBt/event_proportion‚0.0028351815417408943 @&Purity/4FRt0SRqqt0FBt/event_proportion‚6.300403038039804e-05 @&Purity/1FRt4SRqqt*FBt/event_proportion‚0.0003780241822823882 ("Purity/3FRt*SRqqt1FBt/event_purity‚0 @&Purity/4FRt2SRqqt*FBt/event_proportion‚6.300403038039804e-05 ,&Purity/4FRt2SRqqt3FBt/event_proportion‚0 ,&Purity/4FRt3SRqqt*FBt/event_proportion‚0 +particle/specificity‚0.9575158953666687 7 Purity/*FRt0SRqqt4FBt/FBt_purity‚0.8999999761581421 8 Purity/2FRt4SRqqt0FBt/FRt_purity‚0.20000000298023224 ,&Purity/*FRt3SRqqt3FBt/event_proportion‚0 7 Purity/3FRt0SRqqt1FBt/FRt_purity‚0.3910256028175354 ,&Purity/3FRt4SRqqt4FBt/event_proportion‚0 particle/accuracy_2_of_4‚1 ("Purity/*FRt4SRqqt0FBt/event_purity‚0 & Purity/*FRt0SRqqt4FBt/FRt_purity‚0 9"Purity/*FRt3SRqqt2FBt/SRqqt_purity‚0.3333333432674408 ("Purity/3FRt1SRqqt1FBt/event_purity‚0 ("Purity/0FRt4SRqqt0FBt/event_purity‚0 *"Purity/3FRt*SRqqt2FBt/SRqqt_purity‚0.5 7 Purity/*FRt1SRqqt3FBt/FBt_purity‚0.9126983880996704 ( Purity/2FRt2SRqqt2FBt/FRt_purity‚0.5 trainer/global_step‚1207 7 Purity/*FRt1SRqqt0FBt/FRt_purity‚0.1374981850385666 9"Purity/1FRt0SRqqt3FBt/event_purity‚0.1785714328289032 & Purity/1FRt2SRqqt1FBt/FBt_purity‚1 @&Purity/3FRt1SRqqt0FBt/event_proportion‚0.0025201612152159214 jet/accuracy_7_of_7‚0 6 Purity/0FRt*SRqqt2FBt/FBt_purity‚0.985823929309845 ,&Purity/1FRt4SRqqt1FBt/event_proportion‚0 8 Purity/3FRt1SRqqt0FBt/FRt_purity‚0.10833333432674408 8 Purity/2FRt1SRqqt0FBt/FRt_purity‚0.14667023718357086 particle/accuracy_2_of_6‚1 ("Purity/2FRt0SRqqt3FBt/event_purity‚0 7 Purity/3FRt0SRqqt2FBt/FRt_purity‚0.3333333432674408 ) Purity/*FRt1SRqqt4FBt/FBt_purity‚0.75 >&Purity/*FRt2SRqqt0FBt/event_proportion‚0.10855595022439957 @&Purity/3FRt1SRqqt1FBt/event_proportion‚0.0009450604557059705 9"Purity/0FRt4SRqqt0FBt/SRqqt_purity‚0.2142857164144516 9"Purity/2FRt3SRqqt1FBt/SRqqt_purity‚0.2666666805744171 & Purity/*FRt3SRqqt1FBt/FBt_purity‚1 @&Purity/0FRt2SRqqt2FBt/event_proportion‚0.0010080644860863686 A&Purity/0FRt4SRqqt0FBt/event_proportion‚0.00044102821266278625 ,&Purity/0FRt4SRqqt3FBt/event_proportion‚0 +"Purity/0FRt4SRqqt1FBt/SRqqt_purity‚0.25 particle/accuracy_5_of_7‚1 particle/accuracy_7_of_7‚0.5 7 Purity/*FRt0SRqqt3FBt/FBt_purity‚0.9212302565574646 ,&Purity/2FRt4SRqqt4FBt/event_proportion‚0 7 Purity/3FRt4SRqqt*FBt/FRt_purity‚0.1666666716337204 8 Purity/*FRt0SRqqt0FBt/FRt_purity‚0.15270301699638367 9"Purity/*FRt1SRqqt0FBt/SRqqt_purity‚0.4754506051540375 >&Purity/0FRt*SRqqt*FBt/event_proportion‚0.40996724367141724 7 Purity/0FRt0SRqqt2FBt/FBt_purity‚0.9871290922164917 ,&Purity/0FRt1SRqqt4FBt/event_proportion‚0 @&Purity/3FRt0SRqqt0FBt/event_proportion‚0.0015120967291295528 A&Purity/3FRt4SRqqt*FBt/event_proportion‚0.00012600806076079607 particle/accuracy_1_of_1‚1 +"Purity/*FRt4SRqqt0FBt/SRqqt_purity‚0.25 & Purity/*FRt3SRqqt*FBt/FBt_purity‚1 >&Purity/1FRt1SRqqt1FBt/event_proportion‚0.03944052383303642 8 Purity/2FRt2SRqqt1FBt/FRt_purity‚0.23005953431129456 9"Purity/1FRt2SRqqt2FBt/SRqqt_purity‚0.4791666567325592 & Purity/*FRt1SRqqt4FBt/FRt_purity‚0 +"Purity/1FRt1SRqqt3FBt/SRqqt_purity‚0.75 & Purity/4FRt2SRqqt0FBt/FRt_purity‚0 =&Purity/*FRt2SRqqt*FBt/event_proportion‚0.1332535296678543 ,&Purity/*FRt3SRqqt4FBt/event_proportion‚0 & Purity/2FRt0SRqqt1FBt/FBt_purity‚1 ,&Purity/2FRt1SRqqt4FBt/event_proportion‚0 ,&Purity/2FRt4SRqqt2FBt/event_proportion‚0 8 Purity/*FRt1SRqqt*FBt/FRt_purity‚0.18926188349723816 >&Purity/*FRt3SRqqt0FBt/event_proportion‚0.01827116869390011 9"Purity/0FRt*SRqqt0FBt/SRqqt_purity‚0.4433155357837677 =&Purity/1FRt*SRqqt0FBt/event_proportion‚0.3135710656642914 & Purity/1FRt0SRqqt1FBt/FBt_purity‚1 ,&Purity/1FRt3SRqqt4FBt/event_proportion‚0 8 Purity/2FRt*SRqqt1FBt/FRt_purity‚0.35202425718307495 ,&Purity/4FRt2SRqqt1FBt/event_proportion‚0 >&Purity/0FRt0SRqqt0FBt/event_proportion‚0.04082661122083664 ("Purity/2FRt2SRqqt*FBt/event_purity‚0 9"Purity/2FRt2SRqqt*FBt/SRqqt_purity‚0.4868183732032776 ,&Purity/4FRt3SRqqt3FBt/event_proportion‚0 +jet/accuracy_2_of_3‚0.24337103962898254 7 Purity/*FRt2SRqqt2FBt/FRt_purity‚0.3717948794364929 ("Purity/2FRt3SRqqt1FBt/event_purity‚0 @&Purity/2FRt0SRqqt3FBt/event_proportion‚6.300403038039804e-05 ("Purity/0FRt4SRqqt1FBt/event_purity‚0 jet/accuracy_6_of_7‚0 >&Purity/*FRt2SRqqt1FBt/event_proportion‚0.02255544438958168 7 Purity/2FRt0SRqqt*FBt/FBt_purity‚0.9967741966247559 7 Purity/1FRt0SRqqt*FBt/FBt_purity‚0.9939948916435242 ,&Purity/4FRt0SRqqt1FBt/event_proportion‚0 8 Purity/2FRt4SRqqt*FBt/FRt_purity‚0.20000000298023224 particle/accuracy_4_of_7‚1 @&Purity/2FRt4SRqqt*FBt/event_proportion‚0.0003150201519019902 7 Purity/1FRt2SRqqt2FBt/FBt_purity‚0.9583333134651184 ( Purity/2FRt0SRqqt3FBt/FRt_purity‚0.5 ("Purity/4FRt0SRqqt0FBt/event_purity‚0 7 Purity/*FRt*SRqqt1FBt/FRt_purity‚0.3547112047672272 >&Purity/0FRt0SRqqt*FBt/event_proportion‚0.11857359111309052 7 Purity/1FRt0SRqqt3FBt/FRt_purity‚0.5357142686843872 ("Purity/0FRt*SRqqt4FBt/event_purity‚0 ) Purity/4FRt1SRqqt*FBt/FRt_purity‚0.25 9"Purity/0FRt0SRqqt2FBt/event_purity‚0.3267888128757477 @&Purity/1FRt*SRqqt3FBt/event_proportion‚0.0013230846961960196 @&Purity/1FRt3SRqqt2FBt/event_proportion‚6.300403038039804e-05 7 Purity/2FRt*SRqqt*FBt/FBt_purity‚0.9961161017417908 ?&Purity/2FRt2SRqqt*FBt/event_proportion‚0.019468246027827263 ("Purity/3FRt*SRqqt*FBt/event_purity‚0 7 Purity/3FRt*SRqqt*FBt/FRt_purity‚0.1971753090620041 9"Purity/3FRt*SRqqt*FBt/SRqqt_purity‚0.4000958204269409 A&Purity/1FRt1SRqqt3FBt/event_proportion‚0.00025201612152159214 >&Purity/1FRt2SRqqt0FBt/event_proportion‚0.04731602966785431 ?&Purity/3FRt*SRqqt*FBt/event_proportion‚0.009954636916518211 *jet/accuracy_1_of_2‚0.7193155884742737 +"Purity/3FRt4SRqqt0FBt/SRqqt_purity‚0.25 9"Purity/1FRt4SRqqt0FBt/SRqqt_purity‚0.1666666716337204 & Purity/3FRt0SRqqt2FBt/FBt_purity‚1 ("Purity/4FRt2SRqqt0FBt/event_purity‚0 7 Purity/*FRt*SRqqt1FBt/FBt_purity‚0.9988539218902588 ;"Purity/1FRt1SRqqt0FBt/event_purity‚0.015516032464802265 7 Purity/1FRt3SRqqt*FBt/FRt_purity‚0.1678699553012848 ("Purity/2FRt1SRqqt2FBt/event_purity‚0 @&Purity/3FRt2SRqqt0FBt/event_proportion‚0.0024571572430431843 ("Purity/*FRt4SRqqt*FBt/event_purity‚0 0"Purity/*FRt4SRqqt*FBt/SRqqt_purity‚ 0.2421875 +"Purity/*FRt4SRqqt1FBt/SRqqt_purity‚0.25 :"Purity/0FRt1SRqqt1FBt/event_purity‚0.16842396557331085 ?&Purity/2FRt2SRqqt1FBt/event_proportion‚0.003969253972172737 ,&Purity/2FRt4SRqqt3FBt/event_proportion‚0 ("Purity/3FRt0SRqqt1FBt/event_purity‚0 particle/accuracy_1_of_3‚1 (particle/accuracy‚0.8539512753486633 & Purity/0FRt3SRqqt1FBt/FBt_purity‚1 ("Purity/2FRt*SRqqt3FBt/event_purity‚0 8 Purity/*FRt*SRqqt3FBt/FRt_purity‚0.47058823704719543 :"Purity/0FRt*SRqqt3FBt/event_purity‚0.04320988059043884 8 Purity/1FRt0SRqqt1FBt/FRt_purity‚0.41656604409217834 & Purity/2FRt*SRqqt1FBt/FBt_purity‚1 8 Purity/1FRt*SRqqt*FBt/FRt_purity‚0.21107767522335052 9"Purity/1FRt*SRqqt2FBt/SRqqt_purity‚0.5407834053039551 8 Purity/2FRt0SRqqt1FBt/FRt_purity‚0.34785714745521545 ?&Purity/3FRt1SRqqt*FBt/event_proportion‚0.003717237850651145 ( Purity/2FRt*SRqqt3FBt/FRt_purity‚0.5 ("Purity/*FRt1SRqqt3FBt/event_purity‚0 7 Purity/0FRt2SRqqt3FBt/FBt_purity‚0.6666666865348816 7 Purity/*FRt*SRqqt*FBt/FBt_purity‚0.9916454553604126 ,&Purity/3FRt2SRqqt4FBt/event_proportion‚0 @&Purity/4FRt*SRqqt*FBt/event_proportion‚0.0001890120911411941 ("Purity/4FRt0SRqqt*FBt/event_purity‚0 9"Purity/*FRt*SRqqt1FBt/SRqqt_purity‚0.5145480632781982 ,&Purity/0FRt3SRqqt2FBt/event_proportion‚0 9"Purity/1FRt0SRqqt*FBt/event_purity‚0.1943524330854416 9"Purity/2FRt*SRqqt2FBt/SRqqt_purity‚0.7061403393745422 particle/accuracy_4_of_4‚1 & Purity/0FRt3SRqqt*FBt/FBt_purity‚1 ;"Purity/*FRt1SRqqt2FBt/event_purity‚0.017626726999878883 :"Purity/0FRt*SRqqt*FBt/event_purity‚0.45918840169906616 :"Purity/0FRt*SRqqt*FBt/SRqqt_purity‚0.45577847957611084 8 Purity/1FRt1SRqqt0FBt/FRt_purity‚0.13386237621307373 @&Purity/2FRt4SRqqt0FBt/event_proportion‚0.0003150201519019902 ("Purity/4FRt1SRqqt0FBt/SRqqt_purity‚0 >&Purity/1FRt*SRqqt1FBt/event_proportion‚0.09929435700178146 ("Purity/3FRt1SRqqt0FBt/event_purity‚0 jet/accuracy_5_of_5‚0 7 Purity/*FRt2SRqqt2FBt/FBt_purity‚0.9736841917037964 7 Purity/1FRt1SRqqt3FBt/FBt_purity‚0.9166666865348816 :"Purity/*FRt*SRqqt2FBt/event_purity‚0.18838714063167572 >&Purity/0FRt0SRqqt1FBt/event_proportion‚0.08606351166963577 A&Purity/0FRt1SRqqt3FBt/event_proportion‚0.00044102821266278625 & Purity/2FRt2SRqqt1FBt/FBt_purity‚1 ,&Purity/2FRt3SRqqt2FBt/event_proportion‚0 ("Purity/*FRt4SRqqt1FBt/event_purity‚0 & Purity/1FRt*SRqqt4FBt/FRt_purity‚0 >&Purity/*FRt*SRqqt2FBt/event_proportion‚0.06747731566429138 9"Purity/0FRt1SRqqt1FBt/SRqqt_purity‚0.5024889707565308 ,&Purity/*FRt4SRqqt4FBt/event_proportion‚0 =&Purity/1FRt1SRqqt*FBt/event_proportion‚0.1723790317773819 =&Purity/1FRt1SRqqt0FBt/event_proportion‚0.1265120953321457 @&Purity/3FRt0SRqqt*FBt/event_proportion‚0.0027091733645647764 ("Purity/3FRt2SRqqt*FBt/event_purity‚0 9"Purity/3FRt2SRqqt*FBt/SRqqt_purity‚0.4114583432674408 @&Purity/4FRt2SRqqt0FBt/event_proportion‚6.300403038039804e-05 9"Purity/0FRt3SRqqt0FBt/SRqqt_purity‚0.3983134925365448 >&Purity/*FRt0SRqqt0FBt/event_proportion‚0.15757308900356293 ?&Purity/0FRt2SRqqt1FBt/event_proportion‚0.009261592291295528 +jet/accuracy_3_of_5‚0.08656997978687286 ("Purity/*FRt2SRqqt2FBt/event_purity‚0 *"Purity/0FRt2SRqqt2FBt/SRqqt_purity‚0.5 * Purity/1FRt*SRqqt4FBt/FBt_purity‚0.875 9"Purity/0FRt1SRqqt3FBt/SRqqt_purity‚0.6000000238418579 )loss/val_total_loss‚46.00053281292241 >&Purity/*FRt1SRqqt0FBt/event_proportion‚0.33039313554763794 ?&Purity/*FRt1SRqqt2FBt/event_proportion‚0.017011089250445366 7 Purity/*FRt3SRqqt1FBt/FRt_purity‚0.2777777910232544 & Purity/2FRt2SRqqt*FBt/FBt_purity‚1 7 Purity/1FRt2SRqqt2FBt/FRt_purity‚0.3611111342906952 :"Purity/*FRt*SRqqt0FBt/SRqqt_purity‚0.43976476788520813 9"Purity/*FRt*SRqqt1FBt/event_purity‚0.4162077009677887 ,&Purity/0FRt4SRqqt2FBt/event_proportion‚0 ,&Purity/1FRt4SRqqt2FBt/event_proportion‚0 @&Purity/4FRt1SRqqt*FBt/event_proportion‚6.300403038039804e-05 ,&Purity/4FRt4SRqqt1FBt/event_proportion‚0 *jet/accuracy_2_oò`õ^ f_4‚0.3586917817592621 ("Purity/0FRt4SRqqt*FBt/event_purity‚0 8 Purity/*FRt*SRqqt0FBt/FRt_purity‚0.14294034242630005 7 Purity/1FRt*SRqqt2FBt/FBt_purity‚0.9672889709472656 ."Purity/0FRt4SRqqt*FBt/SRqqt_purity‚0.21875 7 Purity/3FRt3SRqqt*FBt/FRt_purity‚0.2142857313156128 9"Purity/3FRt3SRqqt0FBt/SRqqt_purity‚0.4523809552192688 7 Purity/*FRt*SRqqt3FBt/FBt_purity‚0.9162836670875549 9"Purity/*FRt1SRqqt2FBt/SRqqt_purity‚0.5574929714202881 & Purity/0FRt2SRqqt1FBt/FBt_purity‚1 ,&Purity/0FRt3SRqqt4FBt/event_proportion‚0 ("Purity/1FRt3SRqqt1FBt/event_purity‚0 ,&Purity/4FRt0SRqqt2FBt/event_proportion‚0 9"Purity/3FRt2SRqqt1FBt/SRqqt_purity‚0.6666666865348816 ("Purity/0FRt3SRqqt1FBt/event_purity‚0 ;"Purity/2FRt1SRqqt1FBt/event_purity‚0.008064515888690948 ,&Purity/3FRt1SRqqt4FBt/event_proportion‚0 particle/accuracy_3_of_3‚1 particle/accuracy_1_of_4‚1 jet/accuracy_1_of_7‚1 * Purity/4FRt*SRqqt*FBt/FRt_purity‚0.125 9"Purity/*FRt*SRqqt2FBt/SRqqt_purity‚0.5448167324066162 ?&Purity/0FRt3SRqqt0FBt/event_proportion‚0.006111391354352236 @&Purity/0FRt4SRqqt1FBt/event_proportion‚6.300403038039804e-05 ,&Purity/3FRt2SRqqt2FBt/event_proportion‚0 ,&Purity/4FRt1SRqqt3FBt/event_proportion‚0 8 Purity/3FRt0SRqqt*FBt/FRt_purity‚0.21464644372463226 particle/accuracy_2_of_5‚1 ("Purity/0FRt2SRqqt2FBt/event_purity‚0 >&Purity/0FRt*SRqqt1FBt/event_proportion‚0.14787046611309052 ?&Purity/0FRt0SRqqt3FBt/event_proportion‚0.002583165420219302 :"Purity/0FRt1SRqqt0FBt/event_purity‚0.47332024574279785 7 Purity/0FRt2SRqqt*FBt/FBt_purity‚0.9946236610412598 3"Purity/1FRt*SRqqt1FBt/SRqqt_purity‚ 0.5185546875 ,&Purity/2FRt3SRqqt3FBt/event_proportion‚0 -jet/accuracy_4_of_4‚0.0029922511894255877 particle/accuracy_3_of_4‚1 ?&Purity/0FRt2SRqqt0FBt/event_proportion‚0.043283771723508835 7 Purity/2FRt*SRqqt0FBt/FRt_purity‚0.1416560262441635 ("Purity/3FRt2SRqqt0FBt/event_purity‚0 ,&Purity/4FRt4SRqqt4FBt/event_proportion‚0 :"Purity/*FRt2SRqqt2FBt/SRqqt_purity‚0.48245614767074585 & Purity/3FRt*SRqqt2FBt/FBt_purity‚1 ("Purity/3FRt0SRqqt2FBt/event_purity‚0 ("Purity/1FRt1SRqqt3FBt/event_purity‚0 7 Purity/0FRt1SRqqt2FBt/FBt_purity‚0.9839351773262024 @&Purity/1FRt2SRqqt2FBt/event_proportion‚0.0010080644860863686 ?&Purity/2FRt1SRqqt2FBt/event_proportion‚0.001890120911411941 & Purity/2FRt*SRqqt3FBt/FBt_purity‚1 ("Purity/*FRt*SRqqt4FBt/SRqqt_purity‚1 & Purity/1FRt3SRqqt2FBt/FBt_purity‚1 :"Purity/*FRt0SRqqt0FBt/event_purity‚0.13171575963497162 ?&Purity/2FRt*SRqqt2FBt/event_proportion‚0.005418346729129553 8 Purity/2FRt2SRqqt0FBt/FRt_purity‚0.12786389887332916 7 Purity/*FRt4SRqqt*FBt/FRt_purity‚0.2638888955116272 *jet/accuracy_2_of_6‚0.6403508186340332 ("Purity/1FRt*SRqqt4FBt/event_purity‚0 :"Purity/1FRt3SRqqt1FBt/SRqqt_purity‚0.42222216725349426aŸ?Ôl’½ ‚(:‡ þÇéÔÎÚ ½ gpu.0.gpu‚45.0 gpu.0.memory‚5, gpu.0.memoryAllocated‚16.718826293945312* gpu.0.memoryAllocatedBytes‚ 14361362432 gpu.0.temp‚34.0 gpu.0.powerWatts‚164.685' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚32.937 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0S}A“¾ ‚(:ˆ þÇéÔñ¨ÝÁ( disk./.usagePercent‚78.7687734420979 disk.sda2.out‚0 disk.sda3.in‚ 0.6953125 disk.dm-0.in‚ 5116.828125 disk.dm-1.out‚ 196.48046875 disk.sda.in‚0.796875$ memory_percent‚97.91178703308105* proc.memory.percent‚18.510007858276367 network.sent‚ 2.766605e+06& disk.nvme0n1.out‚49559.77490234375# disk.dm-0.out‚49559.77490234375 disk.sda1.in‚ 0.1015625" proc.memory.rssMB‚ 3032.6796875 cpu‚25.81402814914246% disk./.usageGB‚1173.7116470336914 disk.sda3.out‚ 196.48046875 disk.dm-1.in‚ 0.6953125 disk.drbd1260.out‚ 0.29296875' proc.memory.availableMB‚ 342.1328125 proc.cpu.threads‚17 network.recv‚744437 disk.sda2.in‚0! disk.nvme0c0n1.in‚ 5116.828125( disk.nvme0c0n1.out‚49559.77490234375 disk.nvme0n1.in‚ 5116.828125 disk.drbd1260.in‚0 disk.sda1.out‚0 disk.sda.out‚ 196.48046875—Äs·“¿ ‚(:ˆ ÈéÔ°ñ™º gpu.0.gpu‚47.0 gpu.0.memory‚4, gpu.0.memoryAllocated‚16.718826293945312* gpu.0.memoryAllocatedBytes‚ 14361362432 gpu.0.temp‚35.0 gpu.0.powerWatts‚163.243' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚32.6486 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚01”‘À ‚(:† ÈéÔ§ø–» network.recv‚750809 disk.dm-1.in‚ 0.6953125 disk.dm-1.out‚ 196.8828125 disk.sda.in‚0.796875 disk.sda.out‚ 196.8828125( disk.nvme0c0n1.out‚49839.39599609375# disk.dm-0.out‚49839.39599609375 disk.drbd1260.out‚ 0.29296875 disk.sda3.in‚ 0.6953125 disk.sda3.out‚ 196.8828125" proc.memory.rssMB‚ 3032.6796875 proc.cpu.threads‚17 disk.drbd1260.in‚0 network.sent‚ 2.869101e+06) disk./.usagePercent‚78.77030615920539% disk./.usageGB‚1173.7344856262207 disk.nvme0n1.in‚ 5116.828125 disk.dm-0.in‚ 5116.828125 disk.sda2.out‚0! disk.nvme0c0n1.in‚ 5116.828125& disk.nvme0n1.out‚49839.39599609375 disk.sda1.in‚ 0.1015625 disk.sda2.in‚0$ memory_percent‚97.91216850280762' proc.memory.availableMB‚ 342.0703125* proc.memory.percent‚18.510007858276367 cpu‚25.81237159292951 disk.sda1.out‚0•]jˆÁ ‚(:ý œÈéÔÀÖº$ memory_percent‚97.90759086608887" proc.memory.rssMB‚ 3031.6796875 disk.sda.in‚0.796875 disk.sda2.out‚0 disk.drbd1260.in‚0* proc.memory.percent‚18.503904342651367 cpu‚25.810706096208225 network.sent‚ 2.874226e+06% disk./.usageGB‚1173.7423515319824 disk.sda2.in‚0& disk.nvme0c0n1.out‚50095.845703125$ disk.nvme0n1.out‚50095.845703125 disk.dm-1.out‚ 197.3828125 proc.cpu.threads‚17 network.recv‚754382 disk.sda3.out‚ 197.3828125 disk.nvme0n1.in‚ 5116.828125 disk.dm-0.in‚ 5116.828125! disk.dm-0.out‚50095.845703125 disk.drbd1260.out‚0.296875 disk.sda1.in‚ 0.1015625' proc.memory.availableMB‚ 342.8203125' disk./.usagePercent‚78.770834046741 disk.sda.out‚ 197.3828125 disk.sda3.in‚ 0.6953125! disk.nvme0c0n1.in‚ 5116.828125 disk.dm-1.in‚ 0.6953125 disk.sda1.out‚0‘(1èž ‚(:“ œÈéÔêü°º gpu.0.gpu‚13.0 gpu.0.memory‚1, gpu.0.memoryAllocated‚16.718826293945312* gpu.0.memoryAllocatedBytes‚ 14361362432 gpu.0.temp‚32.0 gpu.0.powerWatts‚112.077' gpu.0.enforcedPowerLimitWatts‚500.0) gpu.0.powerPercent‚22.415399999999998 gpu.0.smClock‚1275 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0]1Áá“à ‚(:ˆ «ÈéÔ¾Ù¿¾ gpu.0.gpu‚14.0 gpu.0.memory‚1, gpu.0.memoryAllocated‚16.718826293945312* gpu.0.memoryAllocatedBytes‚ 14361362432 gpu.0.temp‚31.0 gpu.0.powerWatts‚102.782' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚20.5564 gpu.0.smClock‚1275 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0™Nr:Ä ‚(:„ «ÈéÔÉ…íÀ% disk./.usageGB‚1173.7188835144043 disk.sda3.in‚ 0.6953125& disk.nvme0c0n1.out‚50335.392578125! disk.dm-0.out‚50335.392578125 disk.drbd1260.in‚0 disk.drbd1260.out‚0.296875) disk./.usagePercent‚78.76925908839036 disk.dm-1.in‚ 0.6953125 disk.sda1.in‚ 0.1015625 disk.sda.in‚0.796875( proc.memory.availableMB‚ 343.31640625# proc.memory.rssMB‚ 3031.68359375* proc.memory.percent‚18.503928184509277 network.sent‚ 2.879228e+06 network.recv‚758111 disk.sda2.in‚0$ disk.nvme0n1.out‚50335.392578125 disk.dm-0.in‚ 5271.515625 disk.dm-1.out‚ 197.77734375 disk.sda1.out‚0 proc.cpu.threads‚17 disk.sda2.out‚0 disk.sda3.out‚ 197.77734375! disk.nvme0c0n1.in‚ 5271.515625 disk.nvme0n1.in‚ 5271.515625 disk.sda.out‚ 197.77734375$ memory_percent‚97.90456295013428 cpu‚25.809233156200552k°m"2Å - _wandbruntime‚2169  _runtime‚2169ÂRzÅÆ ¿ & lr-AdamW/pg2‚0.0012596865791289998 trainer/global_step‚1249 $ _timestamp‚1.788503093250324e+09 _runtime‚2169.329304779 _step‚98 & lr-AdamW/pg1‚0.0012596865791289998ŽzÉÇ Ã $ _timestamp‚1.788503093250324e+09 _runtime‚2169.329304779 _step‚98 & lr-AdamW/pg1‚0.0012596865791289998 & lr-AdamW/pg2‚0.0012596865791289998 trainer/global_step‚1249béD!\2È - _wandbruntime‚2169  _runtime‚2169ýšÓšß'É ÙO QDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚22775743 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 586906624 PCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚25908053 SGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚7818557 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚610 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 290683392 G?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚286 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 231245312 BDeviceStatsMonitor.on_train_batch_start/allocation.all.current‚26089 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚2 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 D>DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚3689 QCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 243631704 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 599785472 PCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚31361223 IDeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚102667964645888 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 320831488 TDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 13807648768 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚2 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 248284672 ZHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 1238065532928 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚26087 UGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 582240773 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 599785472 PCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚25908055 L?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚25908053 WEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 1238065532928 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 [HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚99228095281872 ]IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚101429916152320 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚31930 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚7819165 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 _LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚99228112321232 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 P@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 13118524029 RBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 13807648768 YIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 13207863296 MFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚33 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚3689 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚26087 L?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚31361223 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚31685 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚17039360 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 231245312 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚17039360 cQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 1294839318528 YEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚100459509382871 G?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚609 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚286 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚24903680 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 M@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚22775133 ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 265779712 TGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚31387310 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 O?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 13807648768 JBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚286 VBDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated‚102668212930560 WGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 13035530240 TGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚25908055 eQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚102918967028736 _runtime‚2169.330743658 N@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 248284672 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 XJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 226592344 ^LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 1238296778240 UADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚100459265751167 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚14956576 M@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚57295365 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 [IDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 1238296778240 IDeviceStatsMonitor.on_train_batch_start/allocation.all.current‚26089 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚2 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 D>DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚3689 QCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 243631704 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 599785472 PCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚31361223 IDeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚102667964645888 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 320831488 TDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 13807648768 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚2 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 248284672 ZHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 1238065532928 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚26087 UGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 582240773 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 599785472 PCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚25908055 L?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚25908053 WEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 1238065532928 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 [HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚99228095281872 ]IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚101429916152320 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚31930 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚7819165 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 _LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚99228112321232 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 P@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 13118524029 RBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 13807648768 YIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 13207863296 MFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚33 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚3689 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚26087 L?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚31361223 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚31685 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚17039360 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 231245312 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚17039360 cQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 1294839318528 YEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚100459509382871 G?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚609 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚286 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚24903680 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 M@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚22775133 ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 265779712 TGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚31387310 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 O?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 13807648768 JBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚286 VBDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated‚102668212930560 WGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 13035530240 TGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚25908055 eQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚102918967028736 _runtime‚2169.330743658 N@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 248284672 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 XJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 226592344 ^LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 1238296778240 UADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚100459265751167 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚14956576 M@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚57295365 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 [IDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 1238296778240 I6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚895 BDeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚22792347 [MDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚ 274155008 VHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 226592552 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚609 _step‚100 N>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 13568115200 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 231258624 REDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚31411921 RBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 13807648768 G:DeviceStatsMonitor.on_train_batch_end/active/ÙyÇ6.all.allocated‚57339933 UEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 13035530240 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 599785472 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚26115 \JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 1239254262784 REDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚25928012 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 UEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 12589930496 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚609 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚286 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚25928010 UCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 1239023004160 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 N>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 13118524029 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚17039360 PDeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚57339933 WGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 13207863296 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚2 YGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 1239254262784 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚17039360 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚620 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚895 J=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚31385808 XFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 1239023004160 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚26113 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 T@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚102747681624064 QCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 384795648 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 XFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 1232122516689 EDeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 248297984 K;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 13568115200 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 S?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚100537064533105 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 C6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚57313818 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 231258624 PBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 586906624 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚609 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚31685 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚7824278 \JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 1232349109241 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 NADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚31411921 \HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚104294412684288 M=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 13807648768 TDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 13207863296 F6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚895 OADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 248297984 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1321 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚14968071 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 299058688 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚2 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 152061440 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 599785472 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0¥Ú›Õî&Í èM YFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚99304942016416 YIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 13207863296 RBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 13035530240 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚31930 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 243631912 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚895 % _timestamp‚1.7885030941680136e+09 ZFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚101508410321920 K>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚57339933 WGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 13207863296 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚2 YGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 1239254262784 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚17039360 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚620 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚895 J=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚31385808 XFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 1239023004160 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚26113 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 T@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚102747681624064 QCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 384795648 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 XFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 1232122516689 EDeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 248297984 K;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 13568115200 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 S?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚100537064533105 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 C6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚57313818 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 231258624 PBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 586906624 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚609 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚31685 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚7824278 \JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 1232349109241 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 NADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚31411921 \HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚104294412684288 M=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 13807648768 TDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 13207863296 F6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚895 OADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 248297984 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1321 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚14968071 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 299058688 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚2 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 152061440 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 599785472 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 NADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚25928010 XDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚104294113625600 S?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚102747433326080 G:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚57313818 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 F6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚895 BDeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚22792347 [MDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚ 274155008 VHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 226592552 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚609 _step‚100 N>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 13568115200 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 231258624 REDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚31411921 RBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 13807648768 G:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚57339933 UEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 13035530240 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 599785472 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚26115 \JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 1239254262784 REDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚25928012 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 UEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 12589930496 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚609 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚286 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚25928010 UCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 1239023004160 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 N>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 13118524029 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚17039360 Pì˜Ú ‚(: öÈéÔÑÔÜ» disk.sda1.out‚0( proc.memory.availableMB‚ 342.76953125 network.recv‚781667) disk./.usagePercent‚78.80750098184009& disk.nvme0n1.out‚51939.87646484375 disk.sda1.in‚ 0.1015625# memory_percent‚97.9079008102417# proc.memory.rssMB‚ 3031.68359375* proc.memory.percent‚18.503928184509277% disk./.usageGB‚1174.2887153625488 disk.sda3.out‚ 200.2109375( disk.nvme0c0n1.out‚51939.87646484375# disk.dm-0.out‚51939.87646484375 proc.cpu.threads‚17 network.sent‚ 2.989304e+06 disk.sda.in‚0.796875 disk.sda.out‚ 200.2109375 disk.sda2.in‚0! disk.nvme0n1.in‚ 5417.22265625 disk.dm-0.in‚ 5417.22265625 disk.drbd1260.in‚0 cpu‚25.80116401300289 disk.sda2.out‚0 disk.sda3.in‚ 0.6953125# disk.nvme0c0n1.in‚ 5417.22265625 disk.dm-1.in‚ 0.6953125 disk.dm-1.out‚ 200.2109375 disk.drbd1260.out‚ 0.30859375ÆîK¹2Û - _wandbruntime‚2245  _runtime‚2245d¥„ÇÜ Á & lr-AdamW/pg2‚0.0012525700131889432 trainer/global_step‚1299 % _timestamp‚1.7885031696128101e+09 _runtime‚2245.692456112 _step‚102 & lr-AdamW/pg1‚0.0012525700131889432¢ˆ–ŠËÝ Å _step‚102 & lr-AdamW/pg1‚0.0012525700131889432 & lr-AdamW/pg2‚0.0012525700131889432 trainer/global_step‚1299 % _timestamp‚1.7885031696128101e+09 _runtime‚2245.692456112fZwàf2Þ - _wandbruntime‚2245  _runtime‚2245 ¼†ß ÚO `LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚105355476600320 M@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚59525817 RBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 13807648768 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 trainer/global_step‚1299 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 248284672 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚31685 ZFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚108243477551616 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚616 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚614 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 599785472 PCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚26905903 H?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚1321 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 IDeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 ^LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 1279026603347 TDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 13035530240 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 @8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚895 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚24903680 JBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚609 WGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 13035530240 ^LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 1286198168064 TGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚32619912 ^JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚108243778720768 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 O?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 13807648768 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 C;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚895 _runtime‚2245.696790506 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 320831488 BDeviceStatsMonitor.on_train_batch_start/allocation.all.current‚26089 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚31930 YEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚106641674768384 P@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 13568115200 UADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚104349232020075 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 599785472 G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚¾4ζß3689 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚8116730 # _timestamp‚1.78850316961669e+09 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 586906624 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚15531226 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 152061440 PCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚32593825 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚31685 R>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚106641426483712 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚3689 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 PCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚26905905 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚17039360 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚17039360 QDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚23647958 MFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚33 E8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚59499728 \HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚105355459560960 eQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚106898710030336 VFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 13207863296 ZHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 1285966922752 P@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 13118524029 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 231245312 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 JBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚286 UADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚106641426483712 TGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚26905905 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚15531228 WGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 12589930496 _step‚103 IDeviceStatsMonitor.on_train_batch_start/allocation.all.current‚26089 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚31930 YEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚106641674768384 P@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 13568115200 UADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚104349232020075 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 599785472 G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚3689 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚8116730 # _timestamp‚1.78850316961669e+09 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 586906624 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚15531226 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 152061440 PCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚32593825 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚31685 R>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚106641426483712 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚3689 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 PCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚26905905 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚17039360 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚17039360 QDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚23647958 MFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚33 E8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚59499728 \HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚105355459560960 eQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚106898710030336 VFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 13207863296 ZHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 1285966922752 P@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 13118524029 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 231245312 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 JBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚286 UADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚106641426483712 TGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚26905905 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚15531228 WGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 12589930496 _step‚103 IDeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 ^LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 1279026603347 TDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 13035530240 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 @8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚895gÙ®‹2á - _wandbruntime‚2247  _runtime‚2247þÔ›ì&â æM I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚26113 SEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 586906624 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚617 _runtime‚2247.675290727 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1321 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚33 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 ZFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚103147278743616 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17039360 ^JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚105433987809280 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚31930 XDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚108324092347904 WGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 13207863296 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚286 C6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚59544270 RBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 13035530240 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚2 YIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 13207863296 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 MDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚1299 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 BDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 L>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 248297984 YGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 1287155652608 \JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 1287155652608 ]KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 1345812361216 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 WCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚104427274433925 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚895 K>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚23664784 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚8122065 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚17039360 aODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 1346069739008 S?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚104427030802013 N>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 13118524029 QCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 384795648 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚2 PDeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚26115 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚3689 RBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 13807648768 WIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 599785472 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚609 % _timestamp‚1.7885031715951002e+09 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 231258624 N>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 13568115200 NADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚32618410 \HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚108324374629376 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 [GDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚105433987809280 UCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 1286924393984 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 VHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 226592552 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚609 trainer/global_step‚1299 _step‚104 T@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚106721143461888 _KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚106978279986688 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 582240773 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚286 OADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 248297984 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚31685 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 320831488 REDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚26925862 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚895 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 NADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚26925862 J=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚32618410 WCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚105433970769920 REDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚32644523 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚26925860 PBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 586906624 G:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚59570385 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚15542721 UEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 12589930496 NADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚26925860 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚615 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚17039360 XFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 1279752058397 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 231258624 cODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚106978304890368 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24903680 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 NADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚32644523 WCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚106721143461888 UEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 13035530240 K>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚59570385 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚286 OBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚23665401 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚609 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚895 EDeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚26115 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚3689 RBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 13807648768 WIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 599785472 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚609 % _timestamp‚1.7885031715951002e+09 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 231258624 N>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 13568115200 NADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚32618410 \HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚108324374629376 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 [GDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚105433987809280 UCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 1286924393984 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 VHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 226592552 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚609 trainer/global_step‚1299 _step‚104 T@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚106721143461888 _KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚106978279986688 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 582240773 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚286 OADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 248297984 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚31685 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 320831488 REDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚26925862 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚895 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 NADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚26925862 J=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚32618410 WCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚105433970769920 REDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚32644523 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚26925860 PBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 586906624 G:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚59570385 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚15542721 UEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 12589930496 NADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚26925860 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚615 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚17039360 XFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 1279752058397 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 231258624 cODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚106978304890368 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24903680 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 NADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚32644523 WCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚106721143461888 UEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 13035530240 K>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚59570385 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚286 OBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚23665401 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚609 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚895 EDeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 MDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚1299 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 BDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 L>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 248297984 YGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 1287155652608 \JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 1287155652608 ]KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 1345812361216 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 WCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚104427274433925 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚895 K>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚23664784 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚8122065 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚17039360 aODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 1346069739008 S?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚104427030802013 N>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 13118524029 QCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 384795648hE7·T2ä - _wandbruntime‚2247  _runtime‚2247÷£†\å Ö 0loss/SRqqt2/detection_loss‚2.328476024207784 /loss/FRt4/assignment_loss‚1.985989247471833 1loss/FBt1/assignment_loss‚0.25291634782283484 .loss/FBt1/detection_loss‚2.495069140425095 _step‚105 0loss/FRt1/assignment_loss‚3.7635305469686333 1loss/FBt3/assignment_loss‚0.12504231505298774 /loss/FBt4/detection_loss‚2.0894279024180245 %loss/total_loss‚45.12063983068246 _runtime‚2247.676840175 2loss/SRqqt2/assignment_loss‚1.6522938263282847 1loss/FBt2/assignment_loss‚0.37244751783230573 0loss/FRt2/assignment_loss‚3.7925740196154667 1loss/FBt4/assignment_loss‚0.20077736719780231 /loss/FBt2/detection_loss‚1.9139875322580338 epoch‚2 2loss/SRqqt1/assignment_loss‚1.9841155730306157 .loss/FRt3/detection_loss‚2.074799741010951 trainer/global_step‚1299 1loss/SRqqt3/assignment_loss‚1.852521651639388 1loss/SRqqt3/detection_loss‚2.7083097134645167 % _timestamp‚1.7885031715976088e+09 2loss/SRqqt4/assignment_loss‚1.1829890907620606 /loss/FRt4/detection_loss‚1.8834250576424698 0loss/SRqqt1/detection_loss‚2.438547909259796 1loss/SRqqt4/detection_loss‚1.0225173049646876 /loss/FBt3/detection_loss‚0.8121236717753758 0loss/FRt3/assignment_loss‚2.9186182955036992 .loss/FRt1/detection_loss‚2.398972536578324 /loss/FRt2/detection_loss‚2.8711674974514887=ÑeØ`æ Ú 0loss/FRt2/assignment_loss‚3.7925740196154667 1loss/FBt4/assignment_loss‚0.20077736719780231 /loss/FBt2/detection_loss‚1.9139875322580338 epoch‚2 2loss/SRqqt1/assignment_loss‚1.9841155730306157 .loss/FRt3/detection_loss‚2.074799741010951 trainer/global_step‚1299 1loss/SRqqt3/assignment_loss‚1.852521651639388 1loss/SRqqt3/detection_loss‚2.7083097134645167 % _timestamp‚1.7885031715976088e+09 2loss/SRqqt4/assignment_loss‚1.1829890907620606 /loss/FRt4/detection_loss‚1.8834250576424698 0loss/SRqqt1/detection_loss‚2.438547909259796 1loss/SRqqt4/detection_loss‚1.0225173049646876 /loss/FBt3/detection_loss‚0.8121236717753758 0loss/FRt3/assignment_loss‚2.9186182955036992 .loss/FRt1/detection_loss‚2.398972536578324 /loss/FRt2/detection_loss‚2.8711674974514887 0loss/SRqqt2/detection_loss‚2.328476024207784 /loss/FRt4/assignment_loss‚1.985989247471833 1loss/FBt1/assignment_loss‚0.25291634782283484 .loss/FBt1/detection_loss‚2.495069140425095 _step‚105 0loss/FRt1/assignment_loss‚3.7635305469686333 1loss/FBt3/assignment_loss‚0.12504231505298774 /loss/FBt4/detection_loss‚2.0894279024180245 %loss/total_loss‚45.12063983068246 _runtime‚2247.676840175 2loss/SRqqt2/assignment_loss‚1.6522938263282847 1loss/FBt2/assignment_loss‚0.37244751783230573iFóH’ç ‚(:‡ …ÉéÔíË“½ gpu.0.gpu‚51.0 gpu.0.memory‚5, gpu.0.memoryAllocated‚16.718826293945312* gpu.0.memoryAllocatedBytes‚ 14361362432 gpu.0.temp‚34.0 gpu.0.powerWatts‚136.465' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚27.293 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0ºp3D‘è ‚(:† …ÉéÔñ¼Ï¿ disk.drbd1260.out‚ 0.30859375 disk.sda.in‚0.796875 network.sent‚ 2.994342e+06 disk.sda3.in‚ 0.6953125 disk.sda1.in‚ 0.1015625 disk.sda.out‚ 200.65625$ memory_percent‚98.93865585327148 network.recv‚785389 disk.sda3.out‚ 200.65625! disk.nvme0n1.in‚ 5417.22265625& disk.nvme0n1.out‚52276.60302734375* proc.memory.percent‚18.503928184509277 cpu‚25.79975914224498 proc.cpu.threads‚17) disk./.usagePercent‚78.80907619619828 disk.sda2.in‚0 disk.dm-0.in‚ 5417.22265625# disk.dm-0.out‚52276.60302734375 disk.dm-1.in‚ 0.6953125 disk.sda1.out‚0& proc.memory.availableMB‚ 173.890625# proc.memory.rssMB‚ 3031.68359375% disk./.usageGB‚1174.3121871948242 disk.sda2.out‚0# disk.nvme0c0n1.in‚ 5417.22265625( disk.nvme0c0n1.out‚52276.60302734375 disk.dm-1.out‚ 200.65625 disk.drbd1260.in‚0œ¸¨žé ‚(:“ ”ÉéÔ†‚‡¹ gpu.0.gpu‚53.0 gpu.0.memory‚4, gpu.0.memoryAllocated‚16.718826293945312* gpu.0.memoryAllocatedBytes‚ 14361362432 gpu.0.temp‚34.0 gpu.0.powerWatts‚143.688' gpu.0.enforcedPowerLimitWatts‚500.0) gpu.0.powerPercent‚28.737599999999997 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0 F‹ç”ê ‚(:‰ ”ÉéÔÉ¡Ÿ½' disk.nvme0c0n1.out‚52869.7041015625% disk.nvme0n1.out‚52869.7041015625" disk.dm-0.out‚52869.7041015625# proc.memory.rssMB‚ 3031.68359375) disk./.usagePercent‚78.84694380663296 disk.dm-1.out‚ 201.06640625 disk.sda1.in‚ 0.1015625 disk.sda2.out‚0' proc.memory.availableMB‚ 173.8828125* proc.memory.percent‚18.503928184509277 cpu‚25.798410727692374 proc.cpu.threads‚17 network.sent‚ 3.082807e+06 network.recv‚791353% disk./.usageGB‚1174.8764419555664 disk.dm-1.in‚ 0.6953125 disk.drbd1260.out‚0.3125 disk.sda1.out‚0 disk.sda.out‚ 201.06640625 disk.sda2.in‚0 disk.drbd1260.in‚0 disk.sda.in‚0.796875 disk.sda3.in‚ 0.6953125 disk.sda3.out‚ 201.06640625# disk.nvme0c0n1.in‚ 5417.22265625! disk.nvme0n1.in‚ 5417.22265625 disk.dm-0.in‚ 5417.22265625# memory_percent‚98.9387035369873ÒC",“ë ‚(:ˆ £ÉéÔÛÚ¶¹ gpu.0.gpu‚56.0 gpu.0.memory‚5, gpu.0.memoryAllocated‚16.718826293945312* gpu.0.memoryAllocatedBytes‚ 14361362432 gpu.0.temp‚33.0 gpu.0.powerWatts‚122.114' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚24.4228 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0º¶…]ì ‚(:… £ÉéÔðö‰» disk.sda1.in‚ 0.1015625 disk.sda1.out‚0 disk.sda.in‚0.796875$ memory_percent‚98.93872737884521 cpu‚25.7964297374066 proc.cpu.threads‚17$ disk./.usageGB‚1174.936923980713 disk.sda2.in‚0! disk.dm-0.out‚53306.373046875# proc.memory.rssMB‚ 3031.68359375 network.sent‚ 3.086893e+06 network.recv‚796017 disk.sda2.out‚0 disk.sda3.out‚ 201.45703125# disk.nvme0c0n1.in‚ 5417.22265625$ disk.nvme0n1.out‚53306.373046875 disk.dm-0.in‚ 5417.22265625 disk.drbd1260.in‚0) disk./.usagePercent‚78.85100280608835 disk.sda3.in‚ 0.6953125! disk.nvme0n1.in‚ 5417.22265625 disk.dm-1.out‚ 201.45703125 disk.drbd1260.out‚0.3125 disk.sda.out‚ 201.45703125( proc.memory.availableMB‚ 173.87890625* proc.memory.percent‚18.503928184509277& disk.nvme0c0n1.out‚53306.373046875 disk.dm-1.in‚ 0.6953125¿/¨P“í ‚(:ˆ ²ÉéÔåÚß¹ gpu.0.gpu‚58.0 gpu.0.memory‚4, gpu.0.memoryAllocated‚16.718826293945312* gpu.0.memoryAllocatedBytes‚ 14361362432 gpu.0.temp‚34.0 gpu.0.powerWatts‚165.412' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚33.0824 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0±¯C–î ‚(:‹ ²ÉéÔùÀÏ»* proc.memory.percent‚18.503928184509277 cpu‚25.793985172164067 network.sent‚ 3.093123e+06 disk.sda2.out‚0! disk.nvme0c0n1.in‚ 5557.890625 disk.nvme0n1.in‚ 5557.890625 disk.sda1.in‚ 0.1015625 disk.sda.in‚0.796875 disk.sda.out‚ 201.78515625$ memory_percent‚98.93872737884521 network.recv‚800690 disk.sda3.in‚ 0.6953125( proc.memory.availableMB‚ 173.87890625# proc.memory.rssMB‚ 3031.68359375 proc.cpu.threads‚17) disk./.usagePercent‚78.85939166097032 disk.sda2.in‚0 disk.sda3.out‚ 201.78515625& disk.nvme0n1.out‚53695.33544921875 disk.dm-0.in‚ 5557.890625 disk.dm-1.out‚ 201.78515625 disk.drbd1260.out‚ 0.31640625$ disk./.usageGB‚1175.061923980713( disk.nvme0c0n1.out‚53695.33544921875# disk.dm-0.out‚53695.33544921875 disk.dm-1.in‚ 0.6953125 disk.drbd1260.in‚0 disk.sda1.out‚0 ÷û’ï ‚(:‡ ÁÉéÔöæç¸ gpu.0.gpu‚44.0 gpu.0.memory‚4, gpu.0.memoryAllocated‚16.718826293945312* gpu.0.memoryAllocatedBytes‚ 14361362432 gpu.0.temp‚34.0 gpu.0.powerWatts‚158.715' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚31.743 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0Ä<ûð ‚(:„ ÁÉéÔƒ³Áº disk.dm-1.out‚ 202.3671875 disk.drbd1260.in‚0 disk.drbd1260.out‚ 0.31640625 disk.sda2.in‚0 disk.sda2.out‚0 disk.sda3.in‚ 0.6953125( proc.memory.availableMB‚ 173.87890625 network.recv‚804516 disk.dm-0.in‚ 5557.890625 disk.dm-1.in‚ 0.6953125 disk.sda1.in‚ 0.1015625 disk.sda1.out‚0 disk.sda.in‚0.796875# proc.memory.rssMB‚ 3031.68359375 cpu‚25.79139744441779) disk./.usagePercent‚78.85939166097032$ disk./.usageGB‚1175.061923980713% disk.nvme0n1.out‚53951.6748046875" disk.dm-0.out‚53951.6748046875 disk.sda3.out‚ 202.3671875$ memory_percent‚98.93872737884521 network.sent‚ 3.098324e+06' disk.nvme0c0n1.out‚53951.6748046875 disk.nvme0n1.in‚ 5557.890625 disk.sda.out‚ 202.3671875* proc.memory.percent‚18.503928184509277 proc.cpu.threads‚17! disk.nvme0c0n1.in‚ 5557.890625!R³»žñ ‚(:“ ÐÉéÔÍé’¹ gpu.0.gpu‚31.0 gpu.0.memory‚2, gpu.0.memoryAllocated‚16.718826293945312* gpu.0.memoryAllocatedBytes‚ 14361362432 gpu.0.temp‚33.0 gpu.0.powerWatts‚121.299' gpu.0.enforcedPowerLimitWatts‚500.0) gpu.0.powerPercent‚24.259800000000002 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0•]®•ò ‚(:Š ÐÉéÔ†ž³» disk.sda3.in‚ 0.6953125$ memory_percent‚98.93877506256104 cpu‚25.789008102783452 proc.cpu.threads‚17 disk.sda2.out‚0 disk.sda3.out‚ 202.85546875 disk.nvme0n1.in‚ 5557.890625& disk.nvme0n1.out‚54207.31494140625# disk.dm-0.out‚54207.31494140625 disk.dm-1.out‚ 202.85546875 disk.drbd1260.out‚ 0.3203125 disk.sda.in‚0.796875 network.sent‚ 3.103447e+06 disk.sda2.in‚0! disk.nvme0c0n1.in‚ 5557.890625( disk.nvme0c0n1.out‚54207.31494140625 disk.dm-0.in‚ 5557.890625 disk.drbd1260.in‚0 disk.sda1.out‚0 disk.sda.out‚ 202.85546875 network.recv‚808192 disk.dm-1.in‚ 0.6953125 disk.sda1.in‚ 0.1015625( proc.memory.availableMB‚ 173.87109375# proc.memory.rssMB‚ 3031.68359375* proc.memory.percent‚18.503928184509277) disk./.usagePercent‚78.87616937073429$ disk./.usageGB‚1175.311923980713뮯2ó - _wandbruntime‚2333  _runtime‚2333©ªƒÇô Á trainer/global_step‚1349 % _timestamp‚1.7885032571077647e+09 _runtime‚2333.187276578 _step‚106 & lr-AdamW/pg1‚0.0012449522125534925 & lr-AdamW/pg2‚0.0012449522125534925j­Î9Ëõ Å % _timestamp‚1.7885032571077647e+09 _runtime‚2333.187276578 _step‚106 & lr-AdamW/pg1‚0.0012449522125534925 & lr-AdamW/pg2‚0.0012449522125534925 trainer/global_step‚1349jw‚p2ö - _wandbruntime‚2333  _runtime‚2333Ó_ŸÆâ'÷ ÜO IDeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚110614888321536 UADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚110614888321536 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚3689 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚3689 UGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 582240773 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚286 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 231245312 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚658 @8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚895 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 599785472 PCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚33826427 UADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚108239198288983 ^LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 1326656145055 JBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚609 trainer/global_step‚1349 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚16105876 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚2 L?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚33826427 TGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚27903755 YEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚110615136606208 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚31930 ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 295139840 _runtime‚2333.190507617 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 TFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 599785472 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 586906624 M@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚24521576 XJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 226592344 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚609 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚17039360 [IDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 1334099557888 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚17039360 SGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚8415700 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 QCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 243631704 ^LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 1334099557888 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 QDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚24522236 OFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚1299 @8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚895 G?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚286 ZHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 1333868312576 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚24903680 O?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 13807648768 YIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 13207863296 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 C;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚895 WEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 1333868312576 `LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚109281037048320 ZFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚112273454468096 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 VFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 13207863296 P@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 13568115200 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚31930 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚26089 ZHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 1326429552711 JBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚286 N@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 248284672 TDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 13035530240 aMDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚110878428128256 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 320831488 `LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚106912785775632 E=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚895 % _timestamp‚1.7885032571105812e+09 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 231245312 SEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 384795648 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 _step‚107 M=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 13568115200 TGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚33852514 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚26087 H?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚1321 eQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚110878453031936 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 E8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚61730180 \HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚109281020008960 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 599785472 IDeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 G?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚609 ]IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚109281037048320 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 586906624 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚2 RBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 13807648768 PCDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated‚33852514 YEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚109281020008960 BDeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚31685 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 248284672 PCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚27903753 MFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚33 _MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 1395026339840 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚660 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 PCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚27903755 L?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚27903753 M@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚61756269 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚8416358 cQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 1395321479680 [KDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 13207863296—ºú¢·&ø àO WKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚8416358 cQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 1395321479680 [KDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 13207863296 IDeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚110614888321536 UADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚110614888321536 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚3689 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚3689 UGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 582240773 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚286 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 231245312 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚658 @8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚895 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 599785472 PCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚33826427 UADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚108239198288983 ^LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 1326656145055 JBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚609 trainer/global_step‚1349 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚16105876 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚2 L?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚33826427 TGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚27903755 YEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚110615136606208 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚31930 ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 295139840 _runtime‚2333.190507617 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 TFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 599785472 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 586906624 M@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚24521576 XJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 226592344 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚609 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚17039360 [IDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 1334099557888 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚17039360 SGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚8415700 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 QCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 243631704 ^LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 1334099557888 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 QDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚24522236 OFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚1299 @8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚895 G?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚286 ZHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 1333868312576 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚24903680 O?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 13807648768 YIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 13207863296 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 C;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚895 WEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 1333868312576 `LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚109281037048320 ZFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚112273454468096 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 VFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 13207863296 P@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 13568115200 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚31930 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚26089 ZHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 1326429552711 JBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚286 N@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 248284672 TDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 13035530240 aMDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚110878428128256 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 320831488 `LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚106912785775632 E=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚895 % _timestamp‚1.7885032571105812e+09 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 231245312 SEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 384795648 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 _step‚107 M=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 13568115200 TGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚33852514 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚26087 H?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚1321 eQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚110878453031936 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 E8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚61730180 \HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚109281020008960 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 599785472 IDeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 G?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚609 ]IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚109281037048320 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 586906624 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚2 RBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 13807648768 PCDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated‚33852514 YEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚109281020008960 BDeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚31685 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 248284672 PCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚27903753 MFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚33 _MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 1395026339840 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚660 =M°BÅ/6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 PCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚27903755 L?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚27903753 M@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚61756269krȦ2ù - _wandbruntime‚2335  _runtime‚2335¦&Scì&ú æM NADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚33877125 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚26113 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 _runtime‚2335.234322871 C6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚61774722 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚27923710 \JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 1335057042432 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 294864384 RBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 13807648768 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚895 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚895 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚286 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚17039360 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 243631912 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 582240773 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 _step‚108 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚8422077 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 WCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚108317240702833 \JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 1327608192657 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 231258624 cODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚110958047891968 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚286 YGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 1335057042432 NADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚27923710 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚26113 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚609 T@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚110694605299712 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚3689 NADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚33851012 aODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 1396307422208 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚286 ^JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚106989632510176 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 599785472 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚895 WCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚109359531217920 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 231258624 ^JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚109359548257280 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚600 TDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 13207863296 MDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚1299 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 OADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 248297984 QCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 384795648 N>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 13118524029 % _timestamp‚1.7885032591548736e+09 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚31930 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24903680 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 NADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚27923712 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 WGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 13207863296 K>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚24538848 FDeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 trainer/global_step‚1349 P6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚895 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17039360 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚609 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚31930 FDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 13568115200 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1321 REDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚16117369 \HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚112354355314176 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 L>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 248297984 G:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚61800837 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚26115 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚2 ]KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 1396037461504 PBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 586906624 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 S?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚108316997070921 WIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 599785472 UCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 1334825783808 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚33 [GDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚109359548257280 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 BDeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚61800837 REDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚27923712 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚598 [MDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚ 269960704 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0t[•’ð&û êM UCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 1334825783808 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚33 [GDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚109359548257280 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 BDeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚61800837 REDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚27923712 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚598 [MDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚ 269960704 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 NADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚33877125 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚26113 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 _runtime‚2335.234322871 C6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚61774722 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚27923710 \JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 1335057042432 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 294864384 RBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 13807648768 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚895 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚895 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚286 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚17039360 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 243631912 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 582240773 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 _step‚108 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚8422077 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 WCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚108317240702833 \JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 1327608192657 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 231258624 cODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚110958047891968 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚286 YGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 1335057042432 NADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚27923710 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚26113 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚609 T@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚110694605299712 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚3689 NADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚33851012 aODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 1396307422208 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚286 ^JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚106989632510176 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 599785472 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚895 WCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚109359531217920 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 231258624 ^JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚109359548257280 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚600 TDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 13207863296 MDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚1299 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 OADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 248297984 QCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 384795648 N>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 13118524029 % _timestamp‚1.7885032591548736e+09 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚31930 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24903680 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 NADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚27923712 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 WGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 13207863296 K>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚24538848 FDeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 trainer/global_step‚1349 P6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚895 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17039360 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚609 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚31930 FDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 13568115200 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1321 REDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚16117369 \HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚112354355314176 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 L>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 248297984 G:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚61800837 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚26115 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚2 ]KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 1396037461504 PBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 586906624 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 S?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚108316997070921 WIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 599785472lîcy2ü - _wandbruntime‚2335  _runtime‚2335îÁ§?Vý Ð /loss/FRt3/assignment_loss‚2.601473150284667 1loss/FBt3/assignment_loss‚0.12171006496516368 /loss/FBt3/detection_loss‚0.7338306793966626 %loss/total_loss‚45.86000398470694 # _timestamp‚1.78850325915653e+09 2loss/SRqqt4/assignment_loss‚1.1496464335821364 0loss/SRqqt3/detection_loss‚2.327792296806971 /loss/FBt2/detection_loss‚2.8622390263610415 _runtime‚2335.235244608 0loss/FRt2/assignment_loss‚3.5696412324905396 0loss/FRt4/assignment_loss‚1.9801186910042397 0loss/FBt4/assignment_loss‚0.2162568960138742 -loss/FRt2/detection_loss‚2.62588774283727 _step‚109 /loss/FRt4/detection_loss‚1.7214976551441046 epoch‚2 /loss/FBt2/assignment_loss‚0.336992152815267 1loss/SRqqt1/detection_loss‚2.4735282161196723 1loss/SRqqt4/detection_loss‚0.9680941637589711 trainer/global_step‚1349 .loss/FBt4/detection_loss‚2.185078563598486 0loss/FRt1/assignment_loss‚4.0559218135618025 2loss/SRqqt2/assignment_loss‚1.9129901536955283 1loss/FBt1/assignment_loss‚0.20441566961181662 /loss/FRt3/detection_loss‚2.0623754957984937 .loss/FBt1/detection_loss‚2.737849601677486 2loss/SRqqt1/assignment_loss‚2.2545562068947027 1loss/SRqqt3/assignment_loss‚2.084479107033639 .loss/FRt1/detection_loss‚2.415707043239899 1loss/SRqqt2/detection_loss‚2.2579219280145106PboŽZþ Ô /loss/FRt4/detection_loss‚1.7214976551441046 epoch‚2 /loss/FBt2/assignment_loss‚0.336992152815267 1loss/SRqqt1/detection_loss‚2.4735282161196723 1loss/SRqqt4/detection_loss‚0.9680941637589711 trainer/global_step‚1349 .loss/FBt4/detection_loss‚2.185078563598486 0loss/FRt1/assignment_loss‚4.0559218135618025 2loss/SRqqt2/assignment_loss‚1.9129901536955283 1loss/FBt1/assignment_loss‚0.20441566961181662 /loss/FRt3/detection_loss‚2.0623754957984937 .loss/FBt1/detection_loss‚2.737849601677486 2loss/SRqqt1/assignment_loss‚2.2545562068947027 1loss/SRqqt3/assignment_loss‚2.084479107033639 .loss/FRt1/detection_loss‚2.415707043239899 1loss/SRqqt2/detection_loss‚2.2579219280145106 /loss/FRt3/assignment_loss‚2.601473150284667 1loss/FBt3/assignment_loss‚0.12171006496516368 /loss/FBt3/detection_loss‚0.7338306793966626 %loss/total_loss‚45.86000398470694 # _timestamp‚1.78850325915653e+09 2loss/SRqqt4/assignment_loss‚1.1496464335821364 0loss/SRqqt3/detection_loss‚2.327792296806971 /loss/FBt2/detection_loss‚2.8622390263610415 _runtime‚2335.235244608 0loss/FRt2/assignment_loss‚3.5696412324905396 0loss/FRt4/assignment_loss‚1.9801186910042397 0loss/FBt4/assignment_loss‚0.2162568960138742 -loss/FRt2/detection_loss‚2.62588774283727 _step‚109m±Ìkžÿ ‚(:“ ßÉéÔ”“Œº gpu.0.gpu‚53.0 gpu.0.memory‚6, gpu.0.memoryAllocated‚16.718826293945312* gpu.0.memoryAllocatedBytes‚ 14361362432 gpu.0.temp‚34.0 gpu.0.powerWatts‚132.956' gpu.0.enforcedPowerLimitWatts‚500.0) gpu.0.powerPercent‚26.591199999999997 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0$Km~•€ ‚(:Š ßÉéÔóθ» disk.dm-1.in‚ 0.6953125 disk.drbd1260.out‚ 0.3203125 disk.sda1.in‚ 0.1015625 disk.sda.out‚ 203.28515625 disk.sda2.out‚0# proc.memory.rssMB‚ 3031.68359375* proc.memory.percent‚18.503928184509277) disk./.usagePercent‚78.87617756297539 disk.dm-0.in‚ 5557.890625# disk.dm-0.out‚54752.04150390625( disk.nvme0c0n1.out‚54752.04150390625 disk.nvme0n1.in‚ 5557.890625& disk.nvme0n1.out‚54752.04150390625% disk./.usageGB‚1175.3120460510254 disk.dm-1.out‚ 203.28515625 disk.sda1.out‚0 disk.sda.in‚0.796875 disk.sda3.in‚ 0.6953125 disk.sda3.out‚ 203.28515625! disk.nvme0c0n1.in‚ 5557.890625$ memory_percent‚98.93906116485596 network.recv‚812734 disk.drbd1260.in‚0 disk.sda2.in‚0( proc.memory.availableMB‚ 173.82421875 cpu‚25.78656299370763 proc.cpu.threads‚17 network.sent‚ 3.109257e+06¹†KPž ‚(:“ îÉéÔ˜üû¸ gpu.0.gpu‚60.0 gpu.0.memory‚5, gpu.0.memoryAllocated‚16.718826293945312* gpu.0.memoryAllocatedBytes‚ 14361362432 gpu.0.temp‚34.0 gpu.0.powerWatts‚159.536' gpu.0.enforcedPowerLimitWatts‚500.0) gpu.0.powerPercent‚31.907200000000003 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0r:¾„‚ ‚(:ù îÉéÔ £Ÿ» network.recv‚819854 disk.sda1.in‚ 0.1015625 disk.sda2.in‚0 disk.sda2.out‚0' disk.nvme0c0n1.out‚55176.7958984375 disk.dm-1.in‚ 0.6953125$ memory_percent‚98.93903732299805# proc.memory.rssMB‚ 3031.68359375 network.sent‚ 3.196871e+06) disk./.usagePercent‚78.87460260462476% disk./.usageGB‚1175.2885780334473 disk.sda1.out‚0 disk.sda.in‚0.796875 disk.sda3.out‚203.625" disk.dm-0.out‚55176.7958984375* proc.memory.percent‚18.503928184509277 disk.sda3.in‚ 0.6953125" disk.nvme0c0n1.in‚ 5673.8359375 disk.nvme0n1.in‚ 5673.8359375% disk.nvme0n1.out‚55176.7958984375 disk.dm-1.out‚203.625& proc.memory.availableMB‚ 173.828125 cpu‚25.7842576754536 disk.sda.out‚203.625 disk.dm-0.in‚ 5673.8359375 disk.drbd1260.in‚0 disk.drbd1260.out‚ 0.32421875 proc.cpu.threads‚17pt㞃 ‚(:“ ýÉéÔ±¡¹ gpu.0.gpu‚57.0 gpu.0.memory‚5, gpu.0.memoryAllocated‚16.718826293945312* gpu.0.memoryAllocatedBytes‚ 14361362432 gpu.0.temp‚34.0 gpu.0.powerWatts‚146.573' gpu.0.enforcedPowerLimitWatts‚500.0) gpu.0.powerPercent‚29.314600000000002 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0±§Pú•„ ‚(:Š ýÉéÔ䥞»( proc.memory.availableMB‚ 173.82421875 network.recv‚823677 disk.sda2.in‚0 disk.sda3.in‚ 0.6953125 disk.nvme0n1.in‚ 5673.8359375% disk.nvme0n1.out‚55601.2529296875 disk.dm-1.out‚ 203.94140625 disk.drbd1260.out‚ 0.32421875$ memory_percent‚98.93906116485596 network.sent‚ 3.202126e+06 disk.sda2.out‚0 disk.sda3.out‚ 203.94140625" disk.dm-0.out‚55601.2529296875 disk.drbd1260.in‚0 disk.sda.out‚ 203.94140625* proc.memory.percent‚18.503928184509277 cpu‚25.78187374482459) disk./.usagePercent‚78.90986841048895' disk.nvme0c0n1.out‚55601.2529296875# proc.memory.rssMB‚ 3031.68359375 proc.cpu.threads‚17$ disk./.usageGB‚1175.814064025879" disk.nvme0c0n1.in‚ 5673.8359375 disk.dm-0.in‚ 5673.8359375 disk.dm-1.in‚ 0.6953125 disk.sda1.in‚ 0.1015625 disk.sda1.out‚0 disk.sda.in‚0.796875éÉ·cž… ‚(:“ ŒÊéÔÿ°—¹ gpu.0.gpu‚31.0 gpu.0.memory‚3, gpu.0.memoryAllocated‚16.718826293945312* gpu.0.memoryAllocatedBytes‚ 14361362432 gpu.0.temp‚33.0 gpu.0.powerWatts‚143.645' gpu.0.enforcedPowerLimitWatts‚500.0) gpu.0.powerPercent‚28.729000000000006 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0,€ï•† ‚(:Š ŒÊéÔûÝϺ" disk.dm-0.out‚55953.8642578125 disk.drbd1260.in‚0 disk.sda.out‚ 204.41015625# proc.memory.rssMB‚ 3031.68359375 cpu‚25.779428972557636 disk.sda3.out‚ 204.41015625 disk.dm-0.in‚ 5673.8359375 disk.dm-1.in‚ 0.6953125 disk.dm-1.out‚ 204.41015625 disk.sda1.out‚0$ memory_percent‚98.93901348114014 network.sent‚ 3.207304e+06) disk./.usagePercent‚78.90973298250334 disk.sda2.out‚0' disk.nvme0c0n1.out‚55953.8642578125 disk.drbd1260.out‚0.328125( proc.memory.availableMB‚ 173.83203125* proc.memory.percent‚18.503928184509277 network.recv‚827373% disk./.usageGB‚1175.8120460510254 disk.nvme0n1.in‚ 5673.8359375% disk.nvme0n1.out‚55953.8642578125 disk.sda1.in‚ 0.1015625 disk.sda.in‚0.796875 proc.cpu.threads‚17 disk.sda2.in‚0 disk.sda3.in‚ 0.6953125" disk.nvme0c0n1.in‚ 5673.8359375±¶ø²ž‡ ‚(:“ ›ÊéÔ¹øøº gpu.0.gpu‚15.0 gpu.0.memory‚2, gpu.0.memoryAllocated‚16.718826293945312* gpu.0.memoryAllocatedBytes‚ 14361362432 gpu.0.temp‚32.0 gpu.0.powerWatts‚116.595' gpu.0.enforcedPowerLimitWatts‚500.0) gpu.0.powerPercent‚23.319000000000003 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0 N/ˆ ‚(:„ ›ÊéÔ™ÙÒ½ disk.drbd1260.in‚0 disk.sda2.in‚0# proc.memory.rssMB‚ 3031.68359375 cpu‚25.776934804748496 proc.cpu.threads‚17 disk.dm-1.in‚ 0.6953125 disk.sda2.out‚0( proc.memory.availableMB‚ 174.06640625* proc.memory.percent‚18.503928184509277 disk.sda1.in‚ 0.1015625 disk.sda1.out‚0 network.sent‚ 3.213196e+06) disk./.usagePercent‚78.90973298250334 disk.nvme0n1.in‚ 5673.8359375% disk.nvme0n1.out‚56210.0947265625 disk.sda.in‚0.796875 disk.sda3.in‚ 0.6953125 disk.dm-0.in‚ 5673.8359375 disk.dm-1.out‚ 204.859375 disk.drbd1260.out‚0.328125 disk.sda.out‚ 204.859375 disk.sda3.out‚ 204.859375" disk.nvme0c0n1.in‚ 5673.8359375' disk.nvme0c0n1.out‚56210.0947265625$ memory_percent‚98.93758296966553 network.recv‚831704% disk./.usageGB‚1175.8120460510254" disk.dm-0.out‚56210.0947265625J îa‘‰ ‚(:† ªÊéÔ²´×¹ gpu.0.gpu‚0.0 gpu.0.memory‚0, gpu.0.memoryAllocated‚16.718826293945312* gpu.0.memoryAllocatedBytes‚ 14361362432 gpu.0.temp‚31.0 gpu.0.powerWatts‚94.201' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚18.8402 gpu.0.smClock‚1275 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0ƒf׊ ‚(:… ªÊéÔò¸»½" disk.dm-0.out‚56513.5986328125 disk.dm-1.in‚ 0.6953125 network.recv‚836693$ disk./.usageGB‚1175.812068939209 disk.dm-0.in‚ 5847.50390625 disk.drbd1260.in‚0 disk.sda.in‚0.796875$ memory_percent‚98.93615245819092 cpu‚25.774642971192588 proc.cpu.threads‚17 network.sent‚ 3.219467e+06 disk.sda2.in‚0 disk.sda3.in‚ 0.6953125' disk.nvme0c0n1.out‚56513.5986328125% disk.nvme0n1.out‚56513.5986328125 disk.dm-1.out‚ 205.21875 disk.sda1.in‚ 0.1015625# proc.memory.rssMB‚ 3031.68359375 disk.sda2.out‚0 disk.sda3.out‚ 205.21875! disk.nvme0n1.in‚ 5847.50390625 disk.drbd1260.out‚ 0.33203125 disk.sda1.out‚0 disk.sda.out‚ 205.21875( proc.memory.availableMB‚ 174.30078125* proc.memory.percent‚18.503928184509277) disk./.usagePercent‚78.90973451854855# disk.nvme0c0n1.in‚ 5847.50390625º2‹ - _wandbruntime‚2422  _runtime‚2422wÔQ)ÇŒ Á & lr-AdamW/pg1‚0.0012368395375613153 & lr-AdamW/pg2‚0.0012368395375613153 trainer/global_step‚1399 % _timestamp‚1.7885033467142801e+09 _runtime‚2422.794398498 _step‚110Ψ¦ñË Å & lr-AdamW/pg1‚0.0012368395375613153 & lr-AdamW/pg2‚0.0012368395375613153 trainer/global_step‚1399 % _timestamp‚1.7885033467142801e+09 _runtime‚2422.794398498 _step‚110nŽ‹¹e2Ž - _wandbruntime‚2422  _runtime‚2422e]w: ÜO RBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 13807648768 PCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚28901603 BDeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚114588350159360 TDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 13035530240 MFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚33 ^JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚116303749297152 cQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 1445553263616 P@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 13118524029 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚286 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚3689 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 YEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚112129408189595 XJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 226592344 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚31685 ]IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚113206597496320 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚17039360 `LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚113206597496320 \HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚113206580456960 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 % _timestamp‚1.7885033467177308e+09 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚2 aMDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚114858171129856 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 trainer/global_step‚1399 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 248284672 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚16680528 UADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚112129164557891 L?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚28901603 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 231245312 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚26087 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚16680526 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 599785472 TGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚35085116 G?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚286 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚26087 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚8715423 E=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚895 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 231245312 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚17039360 H?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚1321 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 320831488 ^LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 1374285686763 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚605 OFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚1299 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 152061440 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 599785472 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚26089 @8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚895 YEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚114588598444032 TGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚28901605 \HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚110755105463472 [KDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 13207863296 JBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚609 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚31930 N@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 248284672 [IDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 1382000947712 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 WGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 12589930496 _step‚111 ^LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 1382000947712 ZHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 1381769702400 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚31685 SGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚8714820 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚26089 YEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚113206580456960 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚603 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚24903680 G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚3689 M=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 13568115200 `LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚110755122502832 _runtime‚2422.797419009 M@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚63986721 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 586906624 D>DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0#‹ €æ' àO ZHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 1374059094419 E8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚63960632 PCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚28901605 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚31930 PCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚35059029 M@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚25395346 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 SEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 384795648 eQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚114858196033536 UADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚114588350159360 _MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 1445270706688 O?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 13807648768 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 307460608 VFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 13207863296 VBDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated‚114588598444032 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 586906624 YIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 13207863296 R>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚114588350159360 TDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 13035530240 MFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚33 ^JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚116303749297152 cQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 1445553263616 P@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 13118524029 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚286 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚3689 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 YEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚112129408189595 XJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 226592344 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚31685 ]IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚113206597496320 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚17039360 `LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚113206597496320 \HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚113206580456960 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 % _timestamp‚1.7885033467177308e+09 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚2 aMDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚114858171129856 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 trainer/global_step‚1399 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 248284672 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚16680528 UADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚112129164557891 L?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚28901603 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 231245312 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚26087 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚16680526 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 599785472 TGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚35085116 G?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚286 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚26087 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚8715423 E=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚895 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 231245312 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚17039360 H?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚1321 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 320831488 ^LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 1374285686763 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚605 OFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚1299 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 152061440 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 599785472 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚26089 @8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚895 YEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚114588598444032 TGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚28901605 \HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚110755105463472 [KDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 13207863296 JBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚609 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚31930 N@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 248284672 [IDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 1382000947712 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 WGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 12589930496 _step‚111 ^LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 1382000947712 ZHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 1381769702400 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚31685 SGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚8714820 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚26089 YEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚113206580456960 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚603 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚24903680 G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚3689 M=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 13568115200 `LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚110755122502832 _runtime‚2422.797419009 M@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚63986721 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 586906624 D>DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 RBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 13807648768 PCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚28901603 BDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 13118524029 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚895 REDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚28921562 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚286 _runtime‚2423.731922271 NADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚28921562 S?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚114667818839552 K>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚25412611 _KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚114937765989888 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 231258624 K>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚64031289 NADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚35083614 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚2 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24903680 ]KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 1446281054208 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 REDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚16692019 \HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚116384341908480 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚895 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚604 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚609 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚286 G:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚64031289 UEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 13035530240 aODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 1446551014912 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 ^JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚110831969237376 RBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 13807648768 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 243631912 M=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 13807648768 WIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 599785472 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 599785472 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 PDeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 248297984 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 599785472 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 231258624 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 XDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚116384047044096 N>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 13568115200 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚17039360 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚8721194 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚602 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 320831488 TDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 13207863296 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚31930 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 S?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚112206963339829 XFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 1375011141813 J=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚35083614 ZFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚110831952198016 UEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 12589930496 VHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 226592552 _step‚112 [GDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚113285108705280 YGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 1382958432256 PBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 586906624 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚2 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 294864384 cODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚114937790893568 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚895 C6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚64005174 FDeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚31685 WCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚112207206971741 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 NADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚35109727 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17039360 RBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 13035530240 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 YIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 13207863296 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚31930 WCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚113285091665920 QCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 384795648 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚17039360 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚609 UCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 1382727173632 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1321 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚609 K;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 13568115200 \JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 1382958432256 NADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚28921560 % _timestamp‚1.7885033476528053e+09 FDeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚31685 WCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚112207206971741 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 NADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚35109727 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17039360 RBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 13035530240 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 YIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 13207863296 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚31930 WCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚113285091665920 QCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 384795648 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚17039360 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚609 UCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 1382727173632 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1321 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚609 K;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 13568115200 \JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 1382958432256 NADeviceStatsMonitor.on_train_batch_end/alf‚'Ñ¿location.large_pool.freed‚28921560 % _timestamp‚1.7885033476528053e+09 FDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 13118524029 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚895 REDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚28921562 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚286 _runtime‚2423.731922271 NADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚28921562 S?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚114667818839552 K>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚25412611 _KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚114937765989888 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 231258624 K>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚64031289 NADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚35083614 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚2 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24903680 ]KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 1446281054208 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 REDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚16692019 \HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚116384341908480 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚895 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚604 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚609 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚286 G:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚64031289 UEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 13035530240 aODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 1446551014912 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 ^JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚110831969237376 RBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 13807648768 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 243631912 M=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 13807648768 WIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 599785472 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 599785472 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 PDeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 248297984 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 599785472 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 231258624 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 XDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚116384047044096 N>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 13568115200 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚17039360 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚8721194 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚602 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 320831488 TDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 13207863296 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚31930 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 S?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚112206963339829 XFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 1375011141813 J=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚35083614 ZFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚110831952198016 UEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 12589930496 VHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 226592552 _step‚112pZi@‹2” - _wandbruntime‚2423  _runtime‚2423#x*W• Ñ /loss/FBt1/detection_loss‚2.3971420122731115 /loss/FBt3/detection_loss‚0.7084054053231752 2loss/SRqqt3/assignment_loss‚1.8843687856022049 0loss/FBt4/assignment_loss‚0.1653004157744969 /loss/FRt3/detection_loss‚2.3008382375593537 .loss/FBt2/detection_loss‚2.332460882572027 trainer/global_step‚1399 0loss/SRqqt1/detection_loss‚2.514527247577417 1loss/SRqqt2/detection_loss‚2.1278866480474603 0loss/FRt2/assignment_loss‚3.9104162074517514 1loss/FBt2/assignment_loss‚0.23799431589803205 1loss/SRqqt3/detection_loss‚2.6465859316727696 2loss/SRqqt1/assignment_loss‚1.7173973327411003 2loss/SRqqt4/assignment_loss‚1.1735850135117298 0loss/FBt3/assignment_loss‚0.1360814810766989 -loss/FRt1/detection_loss‚2.50296966234843 /loss/FRt4/detection_loss‚1.8474161949853938 epoch‚2 _runtime‚2423.733873678 .loss/FRt3/assignment_loss‚2.95451819970284 /loss/FRt2/detection_loss‚2.8223222520040427 .loss/FBt4/detection_loss‚2.227440383036931 %loss/total_loss‚46.42720775161452 0loss/FRt4/assignment_loss‚2.1370779175718293 0loss/FBt1/assignment_loss‚0.2214283647166403 % _timestamp‚1.7885033476553671e+09 _step‚113 /loss/FRt1/assignment_loss‚4.595167546700209 2loss/SRqqt2/assignment_loss‚1.7871452694733256 0loss/SRqqt4/detection_loss‚1.078732043993554>°ø[– Õ 2loss/SRqqt3/assignment_loss‚1.8843687856022049 0loss/FBt4/assignment_loss‚0.1653004157744969 /loss/FRt3/detection_loss‚2.3008382375593537 .loss/FBt2/detection_loss‚2.332460882572027 trainer/global_step‚1399 0loss/SRqqt1/detection_loss‚2.514527247577417 1loss/SRqqt2/detection_loss‚2.1278866480474603 0loss/FRt2/assignment_loss‚3.9104162074517514 1loss/FBt2/assignment_loss‚0.23799431589803205 1loss/SRqqt3/detection_loss‚2.6465859316727696 2loss/SRqqt1/assignment_loss‚1.7173973327411003 2loss/SRqqt4/assignment_loss‚1.1735850135117298 0loss/FBt3/assignment_loss‚0.1360814810766989 -loss/FRt1/detection_loss‚2.50296966234843 /loss/FRt4/detection_loss‚1.8474161949853938 epoch‚2 _runtime‚2423.733873678 .loss/FRt3/assignment_loss‚2.95451819970284 /loss/FRt2/detection_loss‚2.8223222520040427 .loss/FBt4/detection_loss‚2.227440383036931 %loss/total_loss‚46.42720775161452 0loss/FRt4/assignment_loss‚2.1370779175718293 0loss/FBt1/assignment_loss‚0.2214283647166403 % _timestamp‚1.7885033476553671e+09 _step‚113 /loss/FRt1/assignment_loss‚4.595167546700209 2loss/SRqqt2/assignment_loss‚1.7871452694733256 0loss/SRqqt4/detection_loss‚1.078732043993554 /loss/FBt1/detection_loss‚2.3971420122731115 /loss/FBt3/detection_loss‚0.7084054053231752qîí-i“— ‚(:ˆ ¹ÊéÔÏêÓ¾ gpu.0.gpu‚28.0 gpu.0.memory‚2, gpu.0.memoryAllocated‚16.718826293945312* gpu.0.memoryAllocatedBytes‚ 14361362432 gpu.0.temp‚31.0 gpu.0.powerWatts‚109.246' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚21.8492 gpu.0.smClock‚1275 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0~댘 ‚(: ¹ÊéÔÖÞ¤Ä disk.dm-1.in‚ 0.6953125 disk.sda1.in‚ 0.1015625 disk.sda1.out‚0 disk.sda2.in‚0# disk.nvme0c0n1.in‚ 5847.50390625 disk.dm-1.out‚205.5625 disk.sda.in‚0.796875 disk.sda.out‚205.5625# proc.memory.rssMB‚ 3031.68359375! disk.nvme0n1.in‚ 5847.50390625 disk.sda2.out‚0 disk.sda3.in‚ 0.6953125 disk.sda3.out‚205.5625* proc.memory.percent‚18.503928184509277 cpu‚25.772428591576872 proc.cpu.threads‚17 network.sent‚ 3.22453e+06) disk./.usagePercent‚78.90973323851087% disk.nvme0n1.out‚56914.0947265625 disk.dm-0.in‚ 5847.50390625 disk.drbd1260.in‚0 disk.drbd1260.out‚ 0.33203125$ memory_percent‚98.93507957458496' proc.memory.availableMB‚ 174.4765625 network.recv‚839942% disk./.usageGB‚1175.8120498657227' disk.nvme0c0n1.out‚56914.0947265625" disk.dm-0.out‚56914.0947265625ïI‹“™ ‚(:ˆ ÈÊéÔ–õ¹ gpu.0.gpu‚32.0 gpu.0.memory‚3, gpu.0.memoryAllocated‚16.718826293945312* gpu.0.memoryAllocatedBytes‚ 14361362432 gpu.0.temp‚32.0 gpu.0.powerWatts‚112.209' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚22.4418 gpu.0.smClock‚1275 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚00˜T–š ‚(:‹ ÈÊéÔõÏؼ disk.drbd1260.in‚0 disk.sda3.out‚ 206.1328125# disk.nvme0c0n1.in‚ 5847.50390625' disk.nvme0c0n1.out‚57362.7548828125 disk.dm-0.in‚ 5847.50390625 disk.sda.out‚ 206.1328125$ memory_percent‚98.93670082092285# proc.memory.rssMB‚ 3031.68359375* proc.memory.percent‚18.503928184509277 disk.sda2.in‚0' proc.memory.availableMB‚ 174.2109375 cpu‚25.770150200361694 proc.cpu.threads‚17 network.recv‚845913 disk.sda2.out‚0 disk.sda3.in‚ 0.6953125% disk.nvme0n1.out‚57362.7548828125 disk.drbd1260.out‚ 0.33984375 disk.sda1.in‚ 0.1015625 disk.sda1.out‚0 disk.sda.in‚0.796875 network.sent‚ 3.311274e+06) disk./.usagePercent‚78.90815981620544% disk./.usageGB‚1175.7886047363281! disk.nvme0n1.in‚ 5847.50390625" disk.dm-0.out‚57362.7548828125 disk.dm-1.in‚ 0.6953125 disk.dm-1.out‚ 206.1328125C<“› ‚(:ˆ ×ÊéÔ¦÷ƹ gpu.0.gpu‚33.0 gpu.0.memory‚2, gpu.0.memoryAllocated‚16.718826293945312* gpu.0.memoryAllocatedBytes‚ 14361362432 gpu.0.temp‚32.0 gpu.0.powerWatts‚117.018' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚23.4036 gpu.0.smClock‚1275 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0Œ—BŽœ ‚(:ƒ ×ÊéÔæݨ»# proc.memory.rssMB‚ 3031.68359375 cpu‚25.767964585997195 proc.cpu.threads‚17 network.recv‚850230( disk./.usagePercent‚78.9097334945184# disk./.usageGB‚1175.81205368042 disk.sda2.out‚0 disk.sda3.in‚ 0.6953125( proc.memory.availableMB‚ 174.45703125 network.sent‚ 3.317146e+06 disk.sda2.in‚0" disk.dm-0.out‚57811.2978515625 disk.dm-1.in‚ 0.6953125 disk.dm-1.out‚ 206.46875 disk.sda.in‚0.796875* proc.memory.percent‚18.503928184509277# disk.nvme0c0n1.in‚ 5847.50390625% disk.nvme0n1.out‚57811.2978515625 disk.dm-0.in‚ 5847.50390625 disk.drbd1260.in‚0 disk.drbd1260.out‚ 0.33984375 disk.sda1.in‚ 0.1015625 disk.sda1.out‚0 disk.sda.out‚ 206.46875 disk.sda3.out‚ 206.46875' disk.nvme0c0n1.out‚57811.2978515625! disk.nvme0n1.in‚ 5847.50390625$ memory_percent‚98.93519878387451Â"\õ“ ‚(:ˆ æÊéÔ«£°» gpu.0.gpu‚34.0 gpu.0.memory‚3, gpu.0.memoryAllocated‚16.718826293945312* gpu.0.memoryAllocatedBytes‚ 14361362432 gpu.0.temp‚32.0 gpu.0.powerWatts‚108.331' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚21.6662 gpu.0.smClock‚1275 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0çb¾šž ‚(: æÊéÔôþ“À! disk.nvme0n1.in‚ 5983.08203125 disk.sda2.out‚0# proc.memory.rssMB‚ 3031.68359375* proc.memory.percent‚18.503928184509277 cpu‚25.765574642381935 proc.cpu.threads‚17 disk.dm-1.out‚ 206.90234375 network.recv‚855131( disk.nvme0c0n1.out‚58067.83544921875# disk.dm-0.out‚58067.83544921875 disk.sda2.in‚0$ memory_percent‚98.93519878387451 network.sent‚ 3.323512e+06% disk./.usageGB‚1175.8120765686035 disk.drbd1260.in‚0 disk.drbd1260.out‚0.34375 disk.sda1.in‚ 0.1015625 disk.sda.in‚0.796875 disk.sda.out‚ 206.90234375 disk.sda3.in‚ 0.6953125 disk.sda3.out‚ 206.90234375& disk.nvme0n1.out‚58067.83544921875 disk.dm-0.in‚ 5983.08203125 disk.dm-1.in‚ 0.6953125 disk.sda1.out‚0( proc.memory.availableMB‚ 174.45703125) disk./.usagePercent‚78.90973503056361# disk.nvme0c0n1.in‚ 5983.08203125èUÞ%’Ÿ ‚(:‡ õÊéÔ—Õñ¸ gpu.0.gpu‚32.0 gpu.0.memory‚2, gpu.0.memoryAllocated‚16.718826293945312* gpu.0.memoryAllocatedBytes‚ 14361362432 gpu.0.temp‚31.0 gpu.0.powerWatts‚110.705' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚22.141 gpu.0.smClock‚1275 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0y¡Gô˜  ‚(: õÊéԛᔿ( disk./.usagePercent‚78.9081587921753# disk.nvme0c0n1.in‚ 5983.08203125! disk.nvme0n1.in‚ 5983.08203125 cpu‚25.763533563235743 proc.cpu.threads‚17 network.sent‚ 3.330334e+06 network.recv‚858118 disk.sda2.in‚0 disk.sda3.out‚ 207.23046875# disk.dm-0.out‚58411.87060546875 disk.sda1.in‚ 0.1015625( proc.memory.availableMB‚ 174.19921875# proc.memory.rssMB‚ 3031.68359375$ disk./.usageGB‚1175.788589477539 disk.sda.out‚ 207.23046875 disk.sda2.out‚0 disk.sda3.in‚ 0.6953125 disk.dm-1.out‚ 207.23046875 disk.sda1.out‚0$ memory_percent‚98.93677234649658 disk.sda.in‚0.796875( disk.nvme0c0n1.out‚58411.87060546875& disk.nvme0n1.out‚58411.87060546875 disk.dm-0.in‚ 5983.08203125 disk.dm-1.in‚ 0.6953125 disk.drbd1260.in‚0 disk.drbd1260.out‚0.34375* proc.memory.percent‚18.503928184509277e=R&‘¡ ‚(:† „ËéÔº¨ß» gpu.0.gpu‚1.0 gpu.0.memory‚0, gpu.0.memoryAllocated‚16.718826293945312* gpu.0.memoryAllocatedBytes‚ 14361362432 gpu.0.temp‚31.0 gpu.0.powerWatts‚93.168' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚18.6336 gpu.0.smClock‚1275 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0¨·—¢ ‚(:Œ „ËéÔÃðé¾ network.sent‚ 3.334755e+06 disk.dm-1.in‚ 0.6953125 disk.drbd1260.in‚0 disk.sda2.out‚0( disk.nvme0c0n1.out‚58865.08056640625& proc.memory.availableMB‚ 174.453125* proc.memory.percent‚18.503928184509277 disk.dm-1.out‚ 207.7578125 disk.sda1.in‚ 0.1015625 disk.sda1.out‚0 disk.sda.in‚0.796875 disk.sda2.in‚0 disk.sda3.in‚ 0.6953125& disk.nvme0n1.out‚58865.08056640625# disk.dm-0.out‚58865.08056640625 network.recv‚861591) disk./.usagePercent‚78.89735655426803$ disk./.usageGB‚1175.627628326416 disk.sda.out‚ 207.7578125 disk.sda3.out‚ 207.7578125$ memory_percent‚98.93522262573242# proc.memory.rssMB‚ 3031.68359375 proc.cpu.threads‚17 disk.drbd1260.out‚ 0.34765625# disk.nvme0c0n1.in‚ 5983.08203125! disk.nvme0n1.in‚ 5983.08203125 disk.dm-0.in‚ 5983.08203125 cpu‚25.761110607760127°B¥2£ - _wandbruntime‚2513  _runtime‚251333¤dǤ Á & lr-AdamW/pg1‚0.0012282387617370708 & lr-AdamW/pg2‚0.0012282387617370708 trainer/global_step‚1449 % _timestamp‚1.7885034377651734e+09 _runtime‚2513.845388902 _step‚114ÕÚ<Ë¥ Å & lr-AdamW/pg1‚0.0012282387617370708 & lr-AdamW/pg2‚0.0012282387617370708 trainer/global_step‚1449 % _timestamp‚1.7885034377651734e+09 _runtime‚2513.845388902 _step‚114r,Œéz2¦ - _wandbruntime‚2513  _runtime‚2513fæ"ßá'§ ÛO G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚3689 M@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚26267981 `LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚114597459230032 VBDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated‚118562060281856 YEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚117132140904960 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚26087 [KDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 13207863296 P@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 13568115200 H?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚1321 G?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚609 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 G?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚286 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚26089 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚286 QCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 243631704 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚609 M=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 13568115200 ]IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚117132157944320 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚17039360 M@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚66217173 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚3689 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 ZHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 1421688636127 ^JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚120333654416384 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚24903680 BDeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 R>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚118561811997184 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚2 eQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚118837939035136 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 152061440 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 C;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚895 ^LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 1421915228471 E=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚895 E8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚66191084 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 248284672 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚17039360 ^LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 1429902337536 $ _timestamp‚1.788503437768319e+09 \HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚117132140904960 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚31930 ZFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚120333353247232 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 599785472 L?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚36291631 P@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 13118524029 WGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 12589930496 O?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 13807648768 `LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚117132157944320 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚31685 aMDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚118837914131456 \HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚114597442190672 YIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 13207863296 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 586906624 TGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚29899455 SEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 384795648 UADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚116019130826799 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚604 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 TDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 13035530240 PCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚29899453 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 @8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚895 _step‚115 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚31685 YEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚118562060281856 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 231245312 RBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 13807648768 VFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 13207863296 TFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 599785472 JBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚286 _MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 1495439115776 N@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 248284672 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚17255178 cQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 1495715381248 [IDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 1429902337536 UADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚118561811997184 PCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚36291631 XJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 226592344 _runtime‚2513.848240435 IDeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚3689 M@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚26267981 `LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚114597459230032 VBDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated‚118562060281856 YEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚117132140904960 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚26087 [KDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 13207863296 P@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 13568115200 H?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚1321 G?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚609 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 G?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚286 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚26089 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚286 QCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 243631704 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚609 M=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 13568115200 ]IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚117132157944320 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚17039360 M@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚66217173 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚3689 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 ZHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 1421688636127 ^JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚120333654416384 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚24903680 BDeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 R>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚118561811997184 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚2 eQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚118837939035136 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 152061440 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 C;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚895 ^LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 1421915228471 E=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚895 E8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚66191084 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 248284672 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚17039360 ^LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 1429902337536 $ _timestamp‚1.788503437768319e+09 \HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚117132140904960 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚31930 ZFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚120333353247232 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 599785472 L?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚36291631 P@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 13118524029 WGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 12589930496 O?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 13807648768 `LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚117132157944320 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚31685 aMDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚118837914131456 \HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚114597442190672 YIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 13207863296 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 586906624 TGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚29899455 SEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 384795648 UADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚116019130826799 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚604 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 TDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 13035530240 PCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚29899453 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 @8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚895 _step‚115 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚31685 YEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚118562060281856 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 231245312 RBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 13807648768 VFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 13207863296 TFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 599785472 JBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚286 _MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 1495439115776 N@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 248284672 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚17255178 cQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 1495715381248 [IDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 1429902337536 UADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚118561811997184 PCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚36291631 XJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 226592344 _runtime‚2513.848240435 IDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 13568115200 BDeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 # _timestamp‚1.78850343979422e+09 J=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚36316216 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17039360 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 231258624 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚895 M=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 13807648768 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚895 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚26115 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24903680 ]KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 1496449090560 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚17039360 UCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 1430628563456 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 294864384 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 WCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚118641528975360 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 152061440 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 YIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 13207863296 C6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚66235626 ZFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚114674288925216 \JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 1422867276073 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 599785472 N>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 13118524029 XFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 1422640683521 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 582240773 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 243631912 S?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚118641280677376 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 RBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 13807648768 WGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 13207863296 K;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 13568115200 G:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚66235626 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 UEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 12589930496 P@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 13807648768 [MDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚ 269960704 _step‚116 REDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚29919412 REDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚17266669 trainer/global_step‚1449 NADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚36342329 P6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚895 FDeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚26285521 \HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚120414252946432 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 RBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 13035530240 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚586 _KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚118917508991488 WCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚117210652113920 UEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 13035530240 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚9018852 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 231258624 NADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚36316216 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚33 cODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚118917533895168 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 _runtime‚2515.878563244 EDeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 248297984 XFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 1430628563456 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚26113 ^JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚117210669153280 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚17039360 QCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 384795648 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚609 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚2 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚17266671 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚609 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚286 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚2 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚31930 OBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚26286109 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1321 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚9019438 XDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚120413958082048 aODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 1496719051264 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 VHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 226592552 G:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚66261741 T@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚118641528975360 K>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚66261741 NADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚29919410 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 599785472 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚609 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚588 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚3689 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚31685 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 320831488 F6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚895 FDeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚26285521 \HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚120414252946432 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 RBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 13035530240 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚586 _KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚118917508991488 WCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚117210652113920 UEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 13035530240 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚9018852 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 231258624 NADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚36316216 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚33 cODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚118917533895168 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 _runtime‚2515.878563244 EDeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 248297984 XFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 1430628563456 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚26113 ^JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚117210669153280 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚17039360 QCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 384795648 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚609 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚2 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚17266671 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚609 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚286 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚2 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚31930 OBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚26286109 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1321 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚9019438 XDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚120413958082048 aODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 1496719051264 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 VHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 226592552 G:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚66261741 T@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚118641528975360 K>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚66261741 NADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚29919410 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 599785472 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚609 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚588 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚3689 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚31685 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 320831488 FDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 13568115200 BDeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 # _timestamp‚1.78850343979422e+09 J=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚36316216 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17039360 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 231258624 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚895 M=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 13807648768 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚895 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚26115 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24903680 ]KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 1496449090560 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚17039360 UCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 1430628563456 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 294864384 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 WCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚118641528975360 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 152061440 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 YIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 13207863296 C6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚66235626 ZFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚114674288925216 \JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 1422867276073 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 599785472 N>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 13118524029 XFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 1422640683521 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 582240773 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 243631912 S?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚118641280677376 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 RBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 13807648768 WGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 13207863296 K;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 13568115200 G:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚66235626 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 UEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 12589930496 P@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 13807648768 [MDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚ 269960704 _step‚116 REDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚29919412 REDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚17266669 trainer/global_step‚1449 NADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚36342329 PŽ² ‚(:ƒ ¢ËéÔß“á¼% disk./.usageGB‚1175.8388404846191 disk.sda.in‚0.796875 disk.sda2.out‚0! disk.nvme0n1.in‚ 6073.23046875# proc.memory.rssMB‚ 3031.68359375* proc.memory.percent‚18.503928184509277 cpu‚25.75686060769539 disk.dm-1.in‚ 0.6953125 disk.drbd1260.out‚ 0.3515625 disk.sda1.in‚ 0.1015625 disk.sda1.out‚0 disk.sda.out‚208.9375# disk.nvme0c0n1.in‚ 6073.23046875( disk.nvme0c0n1.out‚59689.47314453125' proc.memory.availableMB‚ 174.4140625 network.recv‚873061 disk.sda3.in‚ 0.6953125 disk.dm-0.in‚ 6073.23046875 proc.cpu.threads‚17 disk.sda2.in‚0) disk./.usagePercent‚78.91153117942385 disk.dm-1.out‚208.9375 disk.drbd1260.in‚0 disk.sda3.out‚208.9375& disk.nvme0n1.out‚59689.47314453125# disk.dm-0.out‚59689.47314453125$ memory_percent‚98.93546104431152 network.sent‚ 3.426299e+06`³«“³ ‚(:ˆ ±ËéÔ®¤®¹ gpu.0.gpu‚34.0 gpu.0.memory‚3, gpu.0.memoryAllocated‚16.718826293945312* gpu.0.memoryAllocatedBytes‚ 14361362432 gpu.0.temp‚31.0 gpu.0.powerWatts‚118.897' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚23.7794 gpu.0.smClock‚1275 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0Šjy“´ ‚(:ˆ ±ËéÔ«¸» proc.cpu.threads‚17% disk./.usageGB‚1175.8388404846191& disk.nvme0n1.out‚59969.74267578125 disk.dm-0.in‚ 6120.4921875 disk.sda.out‚ 209.2578125$ memory_percent‚98.93817901611328 network.sent‚ 3.431506e+06 network.recv‚876739) disk./.usagePercent‚78.91153117942385( disk.nvme0c0n1.out‚59969.74267578125 disk.dm-1.in‚ 0.6953125 disk.dm-1.out‚ 209.2578125 disk.drbd1260.out‚ 0.3515625 disk.sda1.out‚0 disk.sda2.out‚0" disk.nvme0c0n1.in‚ 6120.4921875 disk.nvme0n1.in‚ 6120.4921875 disk.drbd1260.in‚0 disk.sda.in‚0.796875% proc.memory.availableMB‚ 173.96875# proc.memory.rssMB‚ 3031.68359375 cpu‚25.754929399912562 disk.sda2.in‚0 disk.sda3.in‚ 0.6953125 disk.sda3.out‚ 209.2578125# disk.dm-0.out‚59969.74267578125 disk.sda1.in‚ 0.1015625* proc.memory.percent‚18.503928184509277$­jÚZµ ‚(:“ ÀËéÔïåǹ gpu.0.gpu‚19.0 gpu.0.memory‚1, gpu.0.memoryAllocated‚16.718826293945312* gpu.0.memoryAllocatedBytes‚ 14361362432 gpu.0.temp‚31.0 gpu.0.powerWatts‚130.529' gpu.0.enforcedPowerLimitWatts‚500.0) gpu.0.powerPercent‚26.105800000000002 gpu.0.smClock‚1275 gpu.0.memoryClock‚1593! %›ý±Dgpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0`ã*—¶ ‚(:Œ ÀËéÔö¬´¼ network.sent‚ 3.436727e+06' disk./.usagePercent‚78.911517355017 disk.sda3.in‚ 0.6953125 disk.nvme0n1.in‚ 6120.4921875 disk.drbd1260.in‚0 disk.drbd1260.out‚ 0.35546875 disk.sda.in‚0.796875$ memory_percent‚98.93667697906494 network.recv‚880140 disk.sda2.in‚0 disk.sda3.out‚ 209.72265625& disk.nvme0n1.out‚60443.03466796875 disk.dm-0.in‚ 6120.4921875 disk.dm-1.in‚ 0.6953125 disk.sda1.in‚ 0.1015625# proc.memory.rssMB‚ 3031.68359375% disk./.usageGB‚1175.8386344909668 disk.sda2.out‚0" disk.nvme0c0n1.in‚ 6120.4921875 disk.dm-1.out‚ 209.72265625 disk.sda.out‚ 209.72265625* proc.memory.percent‚18.503928184509277 cpu‚25.75276953628143( disk.nvme0c0n1.out‚60443.03466796875# disk.dm-0.out‚60443.03466796875 disk.sda1.out‚0( proc.memory.availableMB‚ 174.21484375 proc.cpu.threads‚17<Þ¬£· ‚(:’ ÏËéÔê±â¸ gpu.0.gpu‚24.0 gpu.0.memory‚2, gpu.0.memoryAllocated‚16.718826293945312* gpu.0.memoryAllocatedBytes‚ 14361362432 gpu.0.temp‚32.0 gpu.0.powerWatts‚139.08' gpu.0.enforcedPowerLimitWatts‚500.0) gpu.0.powerPercent‚27.816000000000003 gpu.0.smClock‚1275 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0¾½T—¸ ‚(:Œ ÏËéÔ¾úÕº network.sent‚ 3.442642e+06 network.recv‚884604) disk./.usagePercent‚78.91153373949919 disk.sda2.in‚0 disk.sda3.in‚ 0.6953125( disk.nvme0c0n1.out‚60699.46728515625 disk.dm-0.in‚ 6120.4921875 disk.dm-1.in‚ 0.6953125 disk.sda2.out‚0 disk.nvme0n1.in‚ 6120.4921875# disk.dm-0.out‚60699.46728515625 disk.dm-1.out‚ 210.09765625 disk.drbd1260.out‚ 0.35546875 disk.sda1.in‚ 0.1015625 disk.sda.in‚0.796875 disk.sda.out‚ 210.09765625% disk./.usageGB‚1175.8388786315918 disk.sda3.out‚ 210.09765625& disk.nvme0n1.out‚60699.46728515625 disk.drbd1260.in‚0 disk.sda1.out‚0$ memory_percent‚98.93841743469238' proc.memory.availableMB‚ 173.9296875# proc.memory.rssMB‚ 3031.68359375" disk.nvme0c0n1.in‚ 6120.4921875* proc.memory.percent‚18.503928184509277 cpu‚25.7506954381862 proc.cpu.threads‚17±C‰‘¹ ‚(:† ÞËéÔÒ ‹¹ gpu.0.gpu‚43.0 gpu.0.memory‚4, gpu.0.memoryAllocated‚16.718826293945312* gpu.0.memoryAllocatedBytes‚ 14361362432 gpu.0.temp‚33.0 gpu.0.powerWatts‚158.87' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚31.774 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0¥Ò´`’º ‚(:‡ ÞËéÔ £¼ disk.drbd1260.in‚0 disk.drbd1260.out‚ 0.36328125& disk.nvme0n1.out‚60955.01318359375$ disk./.usageGB‚1175.838665008545 disk.sda1.out‚0 disk.sda.in‚0.796875 disk.sda2.out‚0 disk.sda3.in‚ 0.6953125( disk.nvme0c0n1.out‚60955.01318359375$ memory_percent‚98.93689155578613* proc.memory.percent‚18.503928184509277 network.sent‚ 3.448877e+06 disk.dm-0.in‚ 6120.4921875 disk.sda2.in‚0" disk.nvme0c0n1.in‚ 6120.4921875# proc.memory.rssMB‚ 3031.68359375 cpu‚25.748866991837176 proc.cpu.threads‚17 disk.dm-1.in‚ 0.6953125 disk.sda.out‚ 210.484375 disk.sda3.out‚ 210.484375 disk.nvme0n1.in‚ 6120.4921875' proc.memory.availableMB‚ 174.1796875 disk.sda1.in‚ 0.1015625 network.recv‚889295) disk./.usagePercent‚78.91151940307726# disk.dm-0.out‚60955.01318359375 disk.dm-1.out‚ 210.484375åGE„2» - _wandbruntime‚2608  _runtime‚2608‡Îâ6Ƽ À _step‚118 & lr-AdamW/pg1‚0.0012191570661359832 & lr-AdamW/pg2‚0.0012191570661359832 trainer/global_step‚1499 $ _timestamp‚1.788503532202161e+09 _runtime‚2608.281154677êR¯!ʽ Ä _runtime‚2608.281154677 _step‚118 & lr-AdamW/pg1‚0.0012191570661359832 & lr-AdamW/pg2‚0.0012191570661359832 trainer/global_step‚1499 $ _timestamp‚1.788503532202161e+09vyÞî[2¾ - _wandbruntime‚2608  _runtime‚2608SÚ´á'¿ ÛO F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 586906624 `LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚121057718392320 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 JBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚609 PCDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated‚37550320 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚26087 VFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 13207863296 JBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚286 [IDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 1477803727360 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 WGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 12589930496 ]IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚121057718392320 WEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 1477572482048 TGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚37550320 H?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚1321 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚17829826 SEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 384795648 [KDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 13207863296 P@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 13568115200 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚9312288 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 152061440 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚17039360 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 231245312 G?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚609 ^LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 1477803727360 IDeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚3689 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚17039360 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 XJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 226592344 UGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 582240773 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚609 ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 288848384 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 UADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚119909097095707 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 @8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚895 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 YEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚122535522119680 SGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚9311684 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚31930 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚24903680 R>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚122535273835008 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 231245312 PCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚37524233 QDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚27142116 cQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 1545860971520 @8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚895 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 586906624 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 313752064 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚26089 TGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚30897305 _MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 1545572123136 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 G?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚286 YEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚121057701352960 PCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚30897303 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 E=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚895 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚31930 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚26087 L?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚37524233 VBDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated‚122535522119680 M@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚68447625 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚31685 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚2 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚26089 ^JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚124363543008256 TDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 13807648768 trainer/global_step‚1499 TDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 13035530240 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 248284672 aMDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚122817657133056 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 ^LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 1469544770179 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 599785472 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚31685 WGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 13035530240 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 RBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 13807648768 YIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 13207863296 D>DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 O?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 13807648768 _step‚119 IDeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚3689 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚17039360 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 XJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 226592344 UGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 582240773 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚609 ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 288848384 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 UADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚119909097095707 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 @8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚895 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 YEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚122535522119680 SGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚9311684 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚31930 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚24903680 R>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚122535273835008 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 231245312 PCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚37524233 QDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚27142116 cQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 1545860971520 @8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚895 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 586906624 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 313752064 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚26089 TGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚30897305 _MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 1545572123136 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 G?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚286 YEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚121057701352960 PCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚30897303 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 E=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚895 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚31930 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚26087 L?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚37524233 VBDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated‚122535522119680 M@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚68447625 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚31685 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚2 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚26089 ^JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚124363543008256 TDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 13807648768 trainer/global_step‚1499 TDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 13035530240 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 248284672 aMDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚122817657133056 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 ^LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 1469544770179 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 599785472 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚31685 WGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 13035530240 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 RBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 13807648768 YIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 13207863296 D>DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 O?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 13807648768 _step‚119 IDeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 248297984 PDeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚27159086 E8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 599785472 N>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 13568115200 REDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚17841319 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚577 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 YIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 13207863296 K;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 13568115200 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚9318344 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 243631912 TDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 13207863296 G:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚68492193 NADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚30917262 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚26113 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 ^JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚121136229601280 NADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚37548818 _runtime‚2609.181964268 WCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚121136212561920 K>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚68492193 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚895 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 WIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 599785472 NADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚37574931 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚2 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚2 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 311641600 P@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 13807648768 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚17841321 aODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 1546856607232 N>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 13118524029 [MDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚ 286737920 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 ^JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚118516642691776 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 ZFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚121136212561920 NADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚30917260 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚33 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚17039360 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚895 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚609 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 152061440 F6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚895 _step‚120 REDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚37574931 WCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚119987139509557 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 582240773 XFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 1478529953280 FDeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚27159086 E8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 599785472 N>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 13568115200 REDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚17841319 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚577 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 YIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 13207863296 K;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 13568115200 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚9318344 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 243631912 TDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 13207863296 G:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚68492193 NADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚30917262 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚26113 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 ^JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚121136229601280 NADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚37548818 _runtime‚2609.181964268 WCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚121136212561920 K>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚68492193 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚895 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 WIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 599785472 NADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚37574931 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚2 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚2 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 311641600 P@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 13807648768 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚17841321 aODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 1546856607232 N>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 13118524029 [MDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚ 286737920 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 ^JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚118516642691776 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 ZFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚121136212561920 NADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚30917260 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚33 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚17039360 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚895 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚609 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 152061440 F6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚895 _step‚120 REDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚37574931 WCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚119987139509557 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 582240773 XFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 1478529953280 FDeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 248297984 PDeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚3689 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚3689 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚31685 SEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 384795648 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚24903680 P@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 13118524029 R>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚126508735672832 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 152061440 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 M=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 13568115200 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 248284672 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 RBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 13807648768 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚286 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 E8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚70651988 ZHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 1525473871872 M@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚28017185 cQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 1596017136128 ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 284654080 TDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 13807648768 UGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 582240773 eQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚126797425038336 BDeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 O?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 13807648768 L?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚38756835 @8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚895 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚26089 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 309557760 ^LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 1525705117184 TGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚38782922 PCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚38756835 ^LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 1517174311887 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 H?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚1321 SGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚9612709 ZFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚128393132616704 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 C;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚895 G?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚609 $ _timestamp‚1.788503586214445e+09 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚31685 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 586906624 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚609 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 599785472 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚595 PCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚31895155 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚17039360 \HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚124983261800960 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 QDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚28017780 _MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 1595732482048 L?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚31895153 N@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 248284672 UADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚126508735672832 G?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚286 WEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 1525473871872 IDeviceStatsMonitor.on_train_batch_start/allocation.all.current‚26089 JBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚609 M@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚70678077 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚2 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 trainer/global_step‚1549%)˜Ô ßO F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 152061440 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 M=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 13568115200 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 248284672 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 RBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 13807648768 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚286 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 E8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚70651988 ZHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 1525473871872 M@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚28017185 cQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 1596017136128 ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 284654080 TDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 13807648768 UGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 582240773 eQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚126797425038336 BDeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 O?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 13807648768 L?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚38756835 @8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚895 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚26089 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 309557760 ^LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 1525705117184 TGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚38782922 PCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚38756835 ^LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 1517174311887 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 H?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚1321 SGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚9612709 ZFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚128393132616704 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 C;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚895 G?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚609 $ _timestamp‚1.788503586214445e+09 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚31685 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 586906624 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚609 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 599785472 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚595 PCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚31895155 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚17039360 \HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚124983261800960 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 QDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚28017780 _MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 1595732482048 L?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚31895153 N@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 248284672 UADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚126508735672832 G?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚286 WEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 1525473871872 IDeviceStatsMonitor.on_train_batch_start/allocation.all.current‚26089 JBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚609 M@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚70678077 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚2 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 trainer/global_step‚1549 QCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 243631704 TDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 13035530240 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 231245312 aMDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚126797400134656 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 VBDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated‚126508983957504 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚18404476 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 \HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚122282115645072 G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚3689 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚3689 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚31685 SEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 384795648 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚24903680 P@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 13118524029 R>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚126508735672832{/á5P2Õ - _wandbruntime‚2663  _runtime‚2663)PNàë&Ö åM ^JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚122358979418976 trainer/global_step‚1549 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚17039360 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚609 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 231258624 N>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 13568115200 ^JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚125061790049280 K;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 13568115200 XFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 1526431343104 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 152061440 YIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 13207863296 TDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 13207863296 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚609 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚3689 J=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚38781420 OADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 248297984 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚31930 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚31685 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚286 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚18415971 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚895 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚33 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1321 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 WCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚123877105778465 XFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 1517899766937 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 243631912 \JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 1518126359489 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚895 L>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 248297984 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚17039360 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚2 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 320831488 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 WCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚125061773009920 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚286 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 K>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚70722645 NADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚31915110 OBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚28035277 % _timestamp‚1.7885035871970558e+09 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚580 M=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 13807648768 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚26115 S?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚126588204353024 _KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚126876994994688 NADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚38781420 WGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 13207863296 E6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚895 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 599785472 _runtime‚ 2663.27661599 UEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 13035530240 cODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚126877019898368 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 P@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 13807648768 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 599785472 ZFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚125061773009920 FDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 13118524029 ZFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚122358962379616 VHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 226592552 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚895 PDeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚28034695 XDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚128473720307712 B6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚895 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 599785472 _runtime‚ 2663.27661599 UEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 13035530240 cODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚126877019898368 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 P@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 13807648768 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 599785472 ZFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚125061773009920 FDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 13118524029 ZFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚122358962379616 VHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 226592552 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚895 PDeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚28034695 XDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚128473720307712 BDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 13568115200 ^JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚125061790049280 K;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 13568115200 XFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 1526431343104 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 152061440 YIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 13207863296 TDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 13207863296 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚609 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚3689 J=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚38781420 OADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 248297984 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚31930 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚31685 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚286 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚18415971 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚895 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚33 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1321 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 WCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚123877105778465 XFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 1517899766937 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 243631912 \JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 1518126359489 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚895 L>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 248297984 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚17039360 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚2 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 320831488 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 WCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚125061773009920 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚286 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 K>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚70722645 NADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚31915110 OBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚28035277 % _timestamp‚1.7885035871970558e+09 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚580 M=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 13807648768 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚26115|­y.2Ø - _wandbruntime‚2663  _runtime‚2663°ó/[Ù Õ /loss/FRt4/detection_loss‚1.7024251917838795 2loss/SRqqt3/assignment_loss‚1.6127458906942798 1loss/FBt4/assignment_loss‚0.08257771852495921 1loss/FBt1/assignment_loss‚0.24336406004642544 .loss/FRt2/detection_loss‚2.602144420998437 epoch‚2 trainer/global_step‚1549 /loss/FRt3/assignment_loss‚2.493173270783526 /loss/FRt4/assignment_loss‚1.682765651971866 1loss/SRqqt2/detection_loss‚2.0815844507102508 2loss/SRqqt2/assignment_loss‚1.7305594608973127 0loss/SRqqt1/detection_loss‚2.597693256303376 1loss/SRqqt3/detection_loss‚2.9301494167697046 1loss/SRqqt4/detection_loss‚0.9446755186733553 .loss/FBt1/detection_loss‚2.333054700068065 .loss/FRt1/detection_loss‚2.515604824988873 /loss/FRt3/detection_loss‚1.8794534967697047 .loss/FBt2/detection_loss‚2.139887161327131 /loss/FBt3/detection_loss‚0.8000791049463317 _step‚125 % _timestamp‚1.7885035871989794e+09 0loss/FRt2/assignment_loss‚3.6979235104152135 2loss/SRqqt1/assignment_loss‚1.7923548326772802 1loss/FBt2/assignment_loss‚0.42832902711136217 1loss/FBt3/assignment_loss‚0.19920617960802917 /loss/FBt4/detection_loss‚2.2899684011936188 _runtime‚2663.277713428 %loss/total_loss‚43.46987971810284 /loss/FRt1/assignment_loss‚3.563127998407785 2loss/SRqqt4/assignment_loss‚1.1270321724320747»ŽF_Ú Ù 1loss/SRqqt2/detection_loss‚2.0815844507102508 2loss/SRqqt2/assignment_loss‚1.7305594608973127 0loss/SRqqt1/detection_loss‚2.597693256303376 1loss/SRqqt3/detection_loss‚2.9301494167697046 1loss/SRqqt4/detection_loss‚0.9446755186733553 .loss/FBt1/detection_loss‚2.333054700068065 .loss/FRt1/detection_loss‚2.515604824988873 /loss/FRt3/detection_loss‚1.8794534967697047 .loss/FBt2/detection_loss‚2.139887161327131 /loss/FBt3/detection_loss‚0.8000791049463317 _step‚125 % _timestamp‚1.7885035871989794e+09 0loss/FRt2/assignment_loss‚3.6979235104152135 2loss/SRqqt1/assignment_loss‚1.7923548326772802 1loss/FBt2/assignment_loss‚0.42832902711136217 1loss/FBt3/assignment_loss‚0.19920617960802917 /loss/FBt4/detection_loss‚2.2899684011936188 _runtime‚2663.277713428 %loss/total_loss‚43.46987971810284 /loss/FRt1/assignment_loss‚3.563127998407785 2loss/SRqqt4/assignment_loss‚1.1270321724320747 /loss/FRt4/detection_loss‚1.7024251917838795 2loss/SRqqt3/assignment_loss‚1.6127458906942798 1loss/FBt4/assignment_loss‚0.08257771852495921 1loss/FBt1/assignment_loss‚0.24336406004642544 .loss/FRt2/detection_loss‚2.602144420998437 epoch‚2 trainer/global_step‚1549 /loss/FRt3/assignment_loss‚2.493173270783526 /loss/FRt4/assignment_loss‚1.682765651971866}Ó¿fÔ“Û ‚(:ˆ ©ÌéÔú…ź gpu.0.gpu‚62.0 gpu.0.memory‚5, gpu.0.memoryAllocated‚16.718826293945312* gpu.0.memoryAllocatedBytes‚ 14361362432 gpu.0.temp‚35.0 gpu.0.powerWatts‚116.356' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚23.2712 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0T0Ä~Ü ‚(:ó ©ÌéÔÆÍöº disk.dm-1.out‚ 212.3828125 disk.drbd1260.in‚0# proc.memory.rssMB‚ 3031.68359375* proc.memory.percent‚18.503928184509277 proc.cpu.threads‚17) disk./.usagePercent‚78.76672538182397 disk.sda3.in‚ 0.6953125 disk.sda3.out‚ 212.3828125 disk.nvme0c0n1.in‚6447 disk.dm-0.in‚6447 disk.dm-1.in‚ 0.6953125 disk.sda.out‚ 212.3828125 network.sent‚ 3.557716e+06( disk.nvme0c0n1.out‚62685.17626953125& disk.nvme0n1.out‚62685.17626953125 disk.sda1.in‚ 0.1015625 disk.sda1.out‚0( proc.memory.availableMB‚ 174.10546875 cpu‚25.74229526148861 network.recv‚910986% disk./.usageGB‚1173.6811294555664# disk.dm-0.out‚62685.17626953125 disk.drbd1260.out‚ 0.37109375 disk.sda.in‚0.796875$ memory_percent‚98.93734455108643 disk.sda2.in‚0 disk.sda2.out‚0 disk.nvme0n1.in‚6447j›ÀdÝ ‚(:ˆ ¸ÌéÔ÷¨ª¹ gpu.0.gpu‚13.0 gpu.0.memory‚1, gpu.0.memoryAllocated‚16.718826293945312* gpu.0.memoryAllocatedBytes‚ 14361362432 gpu.0.temp‚31.0 gpu.0.powerWatts‚115.831' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚23.1662 gpu.0.smClock‚1275 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryL ‰/Errors‚0# gpu.0.uncorrectedMemoryErrors‚0ÛÒ7zÞ ‚(:ï ¸ÌéÔž‡åº) disk./.usagePercent‚78.76672845391438 disk.sda3.out‚ 212.82421875 disk.nvme0c0n1.in‚6447% disk.nvme0n1.out‚62840.9951171875 disk.drbd1260.in‚0 disk.sda.in‚0.796875( proc.memory.availableMB‚ 174.09765625 network.sent‚ 3.646212e+06% disk./.usageGB‚1173.6811752319336' disk.nvme0c0n1.out‚62840.9951171875 disk.sda2.in‚0* proc.memory.percent‚18.503928184509277 cpu‚25.740144035305264 proc.cpu.threads‚17 disk.sda3.in‚ 0.6953125" disk.dm-0.out‚62840.9951171875 disk.dm-1.in‚ 0.6953125 disk.drbd1260.out‚0.375 disk.sda.out‚ 212.82421875 disk.sda2.out‚0$ memory_percent‚98.93739223480225# proc.memory.rssMB‚ 3031.68359375 network.recv‚916674 disk.nvme0n1.in‚6447 disk.dm-0.in‚6447 disk.dm-1.out‚ 212.82421875 disk.sda1.in‚ 0.1015625 disk.sda1.out‚0Dâ ø’ß ‚(:‡ ÇÌéÔ®ßû¸ gpu.0.gpu‚0.0 gpu.0.memory‚0, gpu.0.memoryAllocated‚16.718826293945312* gpu.0.memoryAllocatedBytes‚ 14361362432 gpu.0.temp‚33.0 gpu.0.powerWatts‚115.012' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚23.0024 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0ì}†ºnà ‚(:ã ÇÌéÔĘ”» network.recv‚920989) disk./.usagePercent‚78.75934647666456% disk./.usageGB‚1173.5711784362793 disk.drbd1260.in‚0# proc.memory.rssMB‚ 3031.68359375* proc.memory.percent‚18.503928184509277" disk.dm-0.out‚63033.7255859375 disk.dm-1.in‚ 0.6953125 disk.sda1.in‚ 0.1015625 disk.sda.in‚0.796875 disk.sda2.out‚0 disk.sda3.in‚ 0.6953125 disk.sda3.out‚213.3125( proc.memory.availableMB‚ 174.06640625 network.sent‚ 3.652166e+06 disk.nvme0n1.in‚6447 disk.dm-1.out‚213.3125 disk.drbd1260.out‚0.375 disk.sda1.out‚0 disk.sda2.in‚0' disk.nvme0c0n1.out‚63033.7255859375$ memory_percent‚98.93758296966553% disk.nvme0n1.out‚63033.7255859375 disk.dm-0.in‚6447 disk.sda.out‚213.3125 disk.nvme0c0n1.in‚6447 cpu‚25.738552764868235 proc.cpu.threads‚17˜©à¶á ‚(:’ ÖÌéÔé¦õ¸ gpu.0.gpu‚7.0 gpu.0.memory‚1, gpu.0.memoryAllocated‚16.718826293945312* gpu.0.memoryAllocatedBytes‚ 14361362432 gpu.0.temp‚35.0 gpu.0.powerWatts‚162.432' gpu.0.enforcedPowerLimitWatts‚500.0) gpu.0.powerPercent‚32.486399999999996 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0¸ ›ž~â ‚(:ó ÖÌéÔß¡ëº disk.sda.out‚ 213.53515625$ memory_percent‚98.93927574157715 network.recv‚925699# proc.memory.rssMB‚ 3031.68359375 cpu‚25.737716070735114 proc.cpu.threads‚17 network.sent‚ 3.656547e+06% disk./.usageGB‚1173.7469329833984 disk.sda2.in‚0 disk.sda3.in‚ 0.6953125 disk.drbd1260.in‚0 disk.sda1.in‚ 0.1015625 disk.sda1.out‚0* proc.memory.percent‚18.503928184509277) disk./.usagePercent‚78.77114151178962 disk.sda2.out‚0% disk.nvme0n1.out‚63363.3544921875 disk.dm-0.in‚6447" disk.dm-0.out‚63363.3544921875 disk.dm-1.in‚ 0.6953125 disk.dm-1.out‚ 213.53515625' proc.memory.availableMB‚ 173.7890625 disk.sda3.out‚ 213.53515625 disk.nvme0c0n1.in‚6447' disk.nvme0c0n1.out‚63363.3544921875 disk.nvme0n1.in‚6447 disk.drbd1260.out‚ 0.37890625 disk.sda.in‚0.796875ÊÕ°“ã ‚(:’ åÌéÔ÷‡œ¹ gpu.0.gpu‚31.0 gpu.0.memory‚2, gpu.0.memoryAllocated‚16.718826293945312* gpu.0.memoryAllocatedBytes‚ 14361362432 gpu.0.temp‚31.0 gpu.0.powerWatts‚93.564' gpu.0.enforcedPowerLimitWatts‚500.0) gpu.0.powerPercent‚18.712799999999998 gpu.0.smClock‚1275 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚09×5ÿ‘ä ‚(:† åÌéÔ¬çÜ» disk.sda.in‚0.796875) disk./.usagePercent‚78.77111437499099 disk.sda3.in‚ 0.6953125! disk.nvme0c0n1.in‚ 6574.390625 disk.nvme0n1.in‚ 6574.390625% disk.nvme0n1.out‚63716.8037109375$ memory_percent‚98.93772602081299 disk.sda2.in‚0* proc.memory.percent‚18.503928184509277 cpu‚25.736235124689536 proc.cpu.threads‚17 disk.sda2.out‚0 network.sent‚ 3.663468e+06 network.recv‚929049% disk./.usageGB‚1173.7465286254883 disk.dm-1.in‚ 0.6953125 disk.dm-1.out‚ 213.9296875 disk.drbd1260.out‚ 0.37890625 disk.sda1.in‚ 0.1015625 disk.drbd1260.in‚0 disk.sda.out‚ 213.9296875 disk.sda3.out‚ 213.9296875' disk.nvme0c0n1.out‚63716.8037109375 disk.dm-0.in‚ 6574.390625" disk.dm-0.out‚63716.8037109375( proc.memory.availableMB‚ 174.04296875# proc.memory.rssMB‚ 3031.68359375 disk.sda1.out‚0~þ:Ä2å - _wandbruntime‚2733  _runtime‚2733v± Çæ Á _runtime‚2733.386366873 _step‚126 & lr-AdamW/pg1‚0.0011995816411676263 & lr-AdamW/pg2‚0.0011995816411676263 trainer/global_step‚1599 % _timestamp‚1.7885036573067617e+09.fkžËç Å trainer/global_step‚1599 % _timestamp‚1.7885036573067617e+09 _runtime‚2733.386366873 _step‚126 & lr-AdamW/pg1‚0.0011995816411676263 & lr-AdamW/pg2‚0.0011995816411676263~ü vº2è - _wandbruntime‚2733  _runtime‚2733/òWâ'é ÜO @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 YEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚130482445795328 PCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚32893003 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚26087 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚603 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 YEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚127689273265227 UGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 582240773 R>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚130482197510656 _MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 1645848644096 PCDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated‚40015524 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 TGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚40015524 ^JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚132423280366592 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 _runtime‚2733.389958411 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 E=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚895 MFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚33 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 IDeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚31685 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚9913183 WGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 12589930496 TDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 13807648768 E8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚72882440 ^LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 1573606507008 SEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 384795648 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚24903680 ZHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 1564577261251 JBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚609 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 ^LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 1564803853595 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 586906624 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚26089 eQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚130777168039936 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 152061440 UADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚127689029633523 YIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 13207863296 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚17039360 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 231245312 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚605 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 288586240 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 C;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚895 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚17039360 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚31930 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚2 ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 263682560 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 O?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 13807648768 TDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 13035530240 G?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚609 @8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚895 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 QCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 243631704 L?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚32893003 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 248284672 TGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚32893005 OFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚1299 [KDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 13207863296 L?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚39989437 IDeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚3689 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚26087 N@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 248284672 ZHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 1573375261696 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚26089 WGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 13035530240 D>DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 BDeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚3689 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚26087 N@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 248284672 ZHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 1573375261696 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚26089 WGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 13035530240 D>DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 BDeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚130482197510656 _MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 1645848644096 PCDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated‚40015524 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 TGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚40015524 ^JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚132423280366592 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 _runtime‚2733.389958411 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 E=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚895 MFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚33 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 IDeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚31685 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚9913183 WGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 12589930496 TDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 13807648768 E8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚72882440 ^LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 1573606507008 SEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 384795648 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚24903680 ZHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 1564577261251 JBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚609 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 ^LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 1564803853595 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 586906624 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚26089 eQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚130777168039936 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 152061440 UADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚127689029633523 YIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 13207863296 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚17039360 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 231245312 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚605 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 288586240 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 C;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚895 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚17039360 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚31930 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚2 ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 263682560 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 O?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 13807648768 TDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 13035530240 G?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚609 @8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚895 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 QCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 243631704hE¯2ë - _wandbruntime‚2735  _runtime‚2735LØîl®ì åM I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚26113 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 \JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 1565755901197 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚286 RBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 13035530240 ZFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚128987333457920 NADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚32912960 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 599785472 NADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚40014022 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 ^JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚128987350497280 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 M=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 13807648768 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚2 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 231258624 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚31930 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 TDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 13207863296 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 REDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚18990619 cODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚130856762899968 FDeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 248297984 REDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚40040135 WCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚127767072047373 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚609 G:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚72953097 NADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚40040135 QCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 384795648 ]KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 1646848653312 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚609 FDeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚72953097 MDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚1299 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 243631912 ^JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚126201316146176 UEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 12589930496 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 582240773 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚609 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17039360 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 309544448 WCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚130561914488832 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚609 \HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚132503896194048 aODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 1647133294080 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 C6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚72926982 PDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 NADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚32912962 N>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 13568115200 UEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 13035530240 SEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 586906624 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚3689 P@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 13807648768 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 231258624 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚611 XDDeviceStatsMonitor.on_trainÒ,õ= _batch_end/inactive_split_bytes.all.freed‚132503586649600 B6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚895 N>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 13118524029 PBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 586906624 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 ZFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚126201299106816 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚895 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚32912960 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚26113 K>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚28909108 YIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 13207863296 WIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 599785472 REDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚32912962 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1321 E8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚609 FDeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚72953097 MDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚1299 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 243631912 ^JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚126201316146176 UEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 12589930496 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 582240773 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚609 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17039360 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 309544448 WCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚130561914488832 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚609 \HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚132503896194048 aODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 1647133294080 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 C6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚72926982 PDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 NADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚32912962 N>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 13568115200 UEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 13035530240 SEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 586906624 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚3689 P@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 13807648768 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 231258624 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚611 XDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚132503586649600 B6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚895 N>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 13118524029 PBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 586906624 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 ZFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚126201299106816 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚895 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚32912960 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚26113 K>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚28909108 YIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 13207863296 WIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 599785472 REDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚32912962 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1321 EDeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚286 RBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 13035530240 ZFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚128987333457920 NADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚32912960 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 599785472 NADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚40014022 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 ^JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚128987350497280 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 M=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 13807648768 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚2 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 231258624 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚31930 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 TDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 13207863296 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 REDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚18990619 cODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚130856762899968 FDeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 248297984 REDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚40040135 WCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚127767072047373 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚609 G:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚72953097€ôÜB2î - _wandbruntime‚2735  _runtime‚2735!oìS[ï Õ 2loss/SRqqt4/assignment_loss‚1.0643463590083124 /loss/FRt1/detection_loss‚2.2703209955518053 1loss/SRqqt4/detection_loss‚1.0065988570024245 /loss/FRt1/assignment_loss‚4.099507871197491 1loss/FBt2/assignment_loss‚0.19117835753063445 /loss/FRt4/detection_loss‚1.4533076525231203 /loss/FBt3/detection_loss‚0.9245535288974642 $loss/total_loss‚44.8885610195313 2loss/SRqqt1/assignment_loss‚1.7424797945552402 .loss/FBt2/detection_loss‚2.840503749847412 .loss/FBt4/detection_loss‚2.171859409411748 1loss/FBt1/assignment_loss‚0.31244536453769717 _runtime‚2735.389979739 epoch‚2 trainer/global_step‚1599 % _timestamp‚1.7885036593106003e+09 2loss/SRqqt2/assignment_loss‚1.4155281832103024 0loss/FBt4/assignment_loss‚0.5080412445022375 .loss/FRt3/detection_loss‚2.331723597710547 1loss/SRqqt3/detection_loss‚2.9012323723119846 _step‚129 2loss/SRqqt3/assignment_loss‚1.8871281374903286 1loss/FBt3/assignment_loss‚0.20526052585816523 /loss/FRt2/detection_loss‚2.6969642820881634 1loss/SRqqt2/detection_loss‚2.1308019560846416 /loss/FBt1/detection_loss‚2.3283357692487314 /loss/FRt2/assignment_loss‚3.502900508118839 0loss/FRt3/assignment_loss‚2.5718363031097082 /loss/FRt4/assignment_loss‚1.781143213113149 1loss/SRqqt1/detection_loss‚2.5505629866211503¡òu.`ð Ú /loss/FRt4/assignment_loss‚1.781143213113149 1loss/SRqqt1/detection_loss‚2.5505629866211503 2loss/SRqqt4/assignment_loss‚1.0643463590083124 /loss/FRt1/detection_loss‚2.2703209955518053 1loss/SRqqt4/detection_loss‚1.0065988570024245 /loss/FRt1/assignment_loss‚4.099507871197491 1loss/FBt2/assignment_loss‚0.19117835753063445 /loss/FRt4/detection_loss‚1.4533076525231203 /loss/FBt3/detection_loss‚0.9245535288974642 $loss/total_loss‚44.8885610195313 2loss/SRqqt1/assignment_loss‚1.7424797945552402 .loss/FBt2/detection_loss‚2.840503749847412 .loss/FBt4/detection_loss‚2.171859409411748 1loss/FBt1/assignment_loss‚0.31244536453769717 _runtime‚2735.389979739 epoch‚2 trainer/global_step‚1599 % _timestamp‚1.7885036593106003e+09 2loss/SRqqt2/assignment_loss‚1.4155281832103024 0loss/FBt4/assignment_loss‚0.5080412445022375 .loss/FRt3/detection_loss‚2.331723597710547 1loss/SRqqt3/detection_loss‚2.9012323723119846 _step‚129 2loss/SRqqt3/assignment_loss‚1.8871281374903286 1loss/FBt3/assignment_loss‚0.20526052585816523 /loss/FRt2/detection_loss‚2.6969642820881634 1loss/SRqqt2/detection_loss‚2.1308019560846416 /loss/FBt1/detection_loss‚2.3283357692487314 /loss/FRt2/assignment_loss‚3.502900508118839 0loss/FRt3/assignment_loss‚2.5718363031097082Ú%žñ ‚(:“ ôÌéÔ©«†º gpu.0.gpu‚51.0 gpu.0.memory‚6, gpu.0.memoryAllocated‚16.718826293945312* gpu.0.memoryAllocatedBytes‚ 14361362432 gpu.0.temp‚34.0 gpu.0.powerWatts‚139.899' gpu.0.enforcedPowerLimitWatts‚500.0) gpu.0.powerPercent‚27.979799999999997 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0e1_â”ò ‚(:‰ ôÌéÔÛ¿º disk.dm-1.out‚ 214.39453125 disk.sda1.out‚0# proc.memory.rssMB‚ 3031.68359375 cpu‚25.734362934906542 network.recv‚932711 disk.sda2.out‚0 disk.sda3.out‚ 214.39453125 disk.nvme0n1.in‚ 6574.390625 disk.dm-1.in‚ 0.6953125 disk.drbd1260.in‚0 disk.sda1.in‚ 0.1015625$ memory_percent‚98.93944263458252 disk.sda2.in‚0 disk.sda3.in‚ 0.6953125# disk.dm-0.out‚63886.67626953125 disk.sda.out‚ 214.39453125( proc.memory.availableMB‚ 173.76171875* proc.memory.percent‚18.503928184509277 network.sent‚ 3.66672e+06( disk.nvme0c0n1.out‚63886.67626953125& disk.nvme0n1.out‚63886.67626953125 disk.drbd1260.out‚ 0.3828125 disk.sda.in‚0.796875 proc.cpu.threads‚17) disk./.usagePercent‚78.68726601935401$ disk./.usageGB‚1172.497127532959! disk.nvme0c0n1.in‚ 6574.390625 disk.dm-0.in‚ 6574.390625IKA7ó ‚(:’ ƒÍéÔ…鎹 gpu.0.gpu‚16.0 gpu.0.memory‚2, gpu.0.memoryAllocated‚16.718826293945312* gpu.0.memoryAllocatedBytes‚ 14361362432 gpu.0.temp‚35.0 gpu.0.powerWatts‚168.675' gpu.0.enforcedPowerLimitWatts‚500.0( gpu.0.powerPercent‚33.73500000000001 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0_5 áô ‚(:… ƒÍéÔò¢áº# disk.dm-0.out‚64178.67626953125 disk.sda1.out‚0( proc.memory.availableMB‚ 173.99609375 network.recv‚939416 disk.sda3.in‚ 0.6953125! disk.nvme0c0n1.in‚ 6574.390625( disk.nvme0c0n1.out‚64178.67626953125 disk.drbd1260.out‚ 0.3828125 disk.sda.in‚0.796875$ memory_percent‚98.93801212310791* proc.memory.percent‚18.503928184509277% disk./.usageGB‚1172.7319107055664& disk.nvme0n1.out‚64178.67626953125 disk.dm-0.in‚ 6574.390625 disk.dm-1.out‚ 214.890625 disk.drbd1260.in‚0 disk.sda1.in‚ 0.1015625 disk.sda.out‚ 214.890625 cpu‚25.732927462551228 disk.nvme0n1.in‚ 6574.390625 network.sent‚ 3.754244e+06) disk./.usagePercent‚78.70302251506389 disk.sda2.in‚0 disk.sda2.out‚0 disk.dm-1.in‚ 0.6953125# proc.memory.rssMB‚ 3031.68359375 proc.cpu.threads‚17 disk.sda3.out‚ 214.890625xÈUà’õ ‚(:‡ ’ÍéÔý˜Ý¸ gpu.0.gpu‚40.0 gpu.0.memory‚4, gpu.0.memoryAllocated‚16.718826293945312* gpu.0.memoryAllocatedBytes‚ 14361362432 gpu.0.temp‚35.0 gpu.0.powerWatts‚145.925' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚29.185 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0”0a–ö ‚(:‹ ’ÍéÔ¼ó§º network.sent‚ 3.760475e+06% disk./.usageGB‚1172.8120651245117 disk.nvme0n1.in‚ 6574.390625 disk.dm-0.in‚ 6574.390625# disk.dm-0.out‚64361.51708984375 disk.dm-1.in‚ 0.6953125 disk.dm-1.out‚ 215.29296875 disk.drbd1260.in‚0) disk./.usagePercent‚78.70840174537335 disk.sda2.in‚0 disk.sda2.out‚0 disk.sda3.out‚ 215.29296875! disk.nvme0c0n1.in‚ 6574.390625 disk.sda1.in‚ 0.1015625 disk.sda1.out‚0 disk.sda.in‚0.796875 disk.sda3.in‚ 0.6953125( disk.nvme0c0n1.out‚64361.51708984375 disk.drbd1260.out‚ 0.38671875 disk.sda.out‚ 215.29296875$ memory_percent‚98.93803596496582' proc.memory.availableMB‚ 173.9921875# proc.memory.rssMB‚ 3031.68359375 cpu‚25.731166499337466 network.recv‚944147& disk.nvme0n1.out‚64361.51708984375* proc.memory.percent‚18.503928184509277 proc.cpu.threads‚17*Ð “÷ ‚(:ˆ ¡ÍéÔÈîí¸ gpu.0.gpu‚48.0 gpu.0.memory‚5, gpu.0.memoryAllocated‚16.718826293945312* gpu.0.memoryAllocatedBytes‚ 14361362432 gpu.0.temp‚35.0 gpu.0.powerWatts‚162.957' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚32.5914 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚08Øø^…ø ‚(:ú ¡ÍéÔèÍé¾( disk.nvme0c0n1.out‚64610.47412109375& disk.nvme0n1.out‚64610.47412109375 disk.drbd1260.out‚ 0.38671875 disk.sda.in‚0.796875 disk.sda2.out‚0# disk.dm-0.out‚64610.47412109375 disk.dm-1.in‚ 0.6953125 disk.dm-1.out‚ 215.859375 disk.drbd1260.in‚0 disk.sda1.out‚0 disk.sda.out‚ 215.859375" proc.memory.availableMB‚173.75 network.recv‚947894 disk.sda2.in‚0 disk.nvme0c0n1.in‚ 6693.4375 disk.dm-0.in‚ 6693.4375$ memory_percent‚98.93951416015625 cpu‚25.729419063199515 proc.cpu.threads‚17) disk./.usagePercent‚78.71258900460339% disk./.usageGB‚1172.8744583129883 disk.sda3.out‚ 215.859375 disk.nvme0n1.in‚ 6693.4375 disk.sda1.in‚ 0.1015625# proc.memory.rssMB‚ 3031.68359375* proc.memory.percent‚18.503928184509277 network.sent‚ 3.767449e+06 disk.sda3.in‚ 0.6953125ˆãVžù ‚(:“ °ÍéÔЋª¹ gpu.0.gpu‚18.0 gpu.0.memory‚2, gpu.0.memoryAllocated‚16.718826293945312* gpu.0.memoryAllocatedBytes‚ 14361362432 gpu.0.temp‚33.0 gpu.0.powerWatts‚132.956' gpu.0.enforcedPowerLimitWatts‚500.0) gpu.0.powerPercent‚26.591199999999997 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0_£¯ª‰ú ‚(:þ °ÍéÔÐœ» disk.dm-0.out‚64674.96484375 disk.dm-1.in‚ 0.6953125 disk.drbd1260.out‚0.390625( proc.memory.availableMB‚ 173.99609375# proc.memory.rssMB‚ 3031.68359375 network.sent‚ 3.770802e+06 disk.sda2.in‚0 disk.dm-1.out‚ 216.296875 disk.drbd1260.in‚0 network.recv‚951546 disk.sda3.in‚ 0.6953125# disk.nvme0n1.out‚64674.96484375 disk.dm-0.in‚ 6767.30859375$ memory_percent‚98.93801212310791* proc.memory.percent‚18.503928184509277 cpu‚25.7277559232553 proc.cpu.threads‚17# disk.nvme0c0n1.in‚ 6767.30859375% disk.nvme0c0n1.out‚64674.96484375) disk./.usagePercent‚78.71099945382328$ disk./.usageGB‚1172.850772857666 disk.sda1.in‚ 0.1015625 disk.sda1.out‚0 disk.sda.in‚0.796875 disk.sda3.out‚ 216.296875 disk.sda.out‚ 216.296875 disk.sda2.out‚0! disk.nvme0n1.in‚ 6767.30859375繄22û - _wandbruntime‚2816  _runtime‚2816ˆ[Æü À _runtime‚2816.842310575 _step‚130 & lr-AdamW/pg1‚0.0011891042559314684 & lr-AdamW/pg2‚0.0011891042559314684 trainer/global_step‚1649 $ _timestamp‚1.788503740760509e+09G‡dËý Å & lr-AdamW/pg1‚0.0011891042559314684 & lr-AdamW/pg2‚0.0011891042559314684 trainer/global_step‚1649 $ _timestamp‚1.788503740760509e+09 _runtime‚2816.842310575 _step‚130‚{ /í2þ - _wandbruntime‚2816  _runtime‚2816ýÆô‘Ÿ%ÿ ÝO KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚26087 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 `LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚132834399736320 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚2 ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 282556928 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 $ _timestamp‚1.788503740761687e+09 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚2 _MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 1696084076544 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 599785472 M=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 13568115200 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚624 UGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 582240773 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚26089 E8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚75112892 WGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 13035530240 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚3689 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚26087 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 cQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 1696366633472 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 YIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 13207863296 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚609 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 586906624 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 [KDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 13207863296 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 \HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚132834382696960 WGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 12589930496 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚26089 TGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚33890855 IDeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚134455659348480 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚19553776 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚622 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚31685 IDeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 G?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚286 trainer/global_step‚1649 L?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚41222039 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 586906624 QDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚29766684 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 248284672 SEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 384795648 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 JBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚286 YEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚134455907633152 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚10212284 YEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚131579239534135 QCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 243631704 ^LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 1612433395303 XJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 226592344 WEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 1621276651520 ZHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 1621276651520 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚31930 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚17039360 UADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚134455659348480 ^JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocatÆk–Ded‚136453277675008 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚24903680 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 320831488 @8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚895 C;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚895 M@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚29766060 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 231245312 G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚3689ZÆ.ÿè'€âO OFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚1299 eQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚134756911041536 VBDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated‚134455907633152 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚17039360 `LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚129966806138832 O?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 13807648768 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 599785472 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 E=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚895 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 307460608 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 L?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚33890853 BDeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 G?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚286 trainer/global_step‚1649 L?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚41222039 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 586906624 QDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚29766684 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 248284672 SEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 384795648 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 JBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚286 YEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚134455907633152 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚10212284 YEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚131579239534135 QCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 243631704 ^LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 1612433395303 XJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 226592344 WEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 1621276651520 ZHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 1621276651520 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚31930 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚17039360 UADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚134455659348480 ^JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚136453277675008 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚24903680 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 320831488 @8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚895 C;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚895 M@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚29766060 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 231245312 G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚3689 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚26087 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 `LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚132834399736320 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚2 ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 282556928 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 $ _timestamp‚1.788503740761687e+09 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚2 _MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 1696084076544 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 599785472 M=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 13568115200 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚624 UGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 582240773 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚26089 E8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚75112892 WGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 13035530240 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚3689 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚26087 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 cQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 1696366633472 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 YIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 13207863296 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚609 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 586906624 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 [KDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 13207863296 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 \HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚132834382696960 WGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 12589930496 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚26089 TGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚33890855 IDeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚134455659348480 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚19553776 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚622 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚31685 IDeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚286 [GDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚132912910945280 RBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 13035530240 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 ZFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚130043635834016 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚286 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚31930 UEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 13035530240 FDeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 248297984 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚17039360 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚17039360 WCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚132912893905920 REDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚33910812 ]KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 1697096388096 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚26113 XFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 1613158850353 ZFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚132912893905920 aODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 1697366348800 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 P@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 13807648768 ^JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚132912910945280 SEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 586906624 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1321 REDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚10218325 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚895 S?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚131656794684369 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚33 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚26115 WCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚134535376326656 \JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 1622465381376 NADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚33910810 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚31685 T@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚134535376326656 PDeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚75183549 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚31930 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24903680 _KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚134836480997888 M=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 13807648768 C6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚75157434 FDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 13118524029 \JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 1613385442905 WIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 599785472 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 NADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚41246624 REDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚19565269 \HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚136533872250368 TDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 13207863296 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚609 % _timestamp‚1.7885037416982443e+09 NADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚33910812 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚2 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚26113 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 152061440 OADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 248297984 UEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 12589930496 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚895 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17039360 MDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚1299 XDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚136533577385984 WCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚131657038316281 VHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 226592552 OBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚29784190 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚609 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚609 J=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚41246624 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 582240773 G:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚75157434 K>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚29783594 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚594 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 _step‚132 G:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚75183549 YGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 1622465381376 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 294864384 YIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 13207863296 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 NADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚41272737 cODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚134836505901568 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 K;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 13568115200 PBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 586906624 N>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 13568115200 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚596 RBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 13807648768 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 599785472 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 320831488 trainer/global_step‚1649ÿ‹¬áó&ƒíM aODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 1697366348800 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 P@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 13807648768 ^JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚132912910945280 SEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 586906624 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1321 REDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚10218325 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚895 S?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚131656794684369 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚33 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚26115 WCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚134535376326656 \JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 1622465381376 NADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚33910810 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚31685 T@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚134535376326656 PDeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚75183549 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚31930 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24903680 _KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚134836480997888 M=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 13807648768 C6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚75157434 FDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 13118524029 \JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 1613385442905 WIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 599785472 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 NADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚41246624 REDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚19565269 \HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚136533872250368 TDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 13207863296 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚609 % _timestamp‚1.7885037416982443e+09 NADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚33910812 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚2 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚26113 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 152061440 OADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 248297984 UEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 12589930496 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚895 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17039360 MDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚1299 XDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚136533577385984 WCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚131657038316281 VHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 226592552 OBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚29784190 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚609 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚609 J=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚41246624 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 582240773 G:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚75157434 K>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚29783594 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚594 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 _step‚132 G:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚75183549 YGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 1622465381376 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 294864384 YIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 13207863296 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 NADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚41272737 cODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚134836505901568 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 K;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 13568115200 PBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 586906624 N>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 13568115200 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚596 RBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 13807648768 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 599785472 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 320831488 trainer/global_step‚1649 _runtime‚2817.777572893 QCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 384795648 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚286 [GDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚132912910945280 RBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 13035530240 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 ZFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚130043635834016 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚286 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚31930 UEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 13035530240 FDeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 248297984 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚17039360 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚17039360 WCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚132912893905920 REDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚33910812 ]KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 1697096388096 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚26113 XFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 1613158850353 ZFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚132912893905920„G2f 2„- _wandbruntime‚2817  _runtime‚2817än&W…Ñ 0loss/FRt2/assignment_loss‚3.6296340153767512 0loss/FRt3/assignment_loss‚2.3623568836380455 0loss/FBt2/assignment_loss‚0.3482081794669615 % _timestamp‚1.7885037417005668e+09 2loss/SRqqt1/assignment_loss‚2.0971957042261407 .loss/FBt1/detection_loss‚2.742685310840607 /loss/FRt3/detection_loss‚1.8947146710227518 0loss/SRqqt2/detection_loss‚2.044652602672577 .loss/FRt1/assignment_loss‚4.18935087594119 2loss/SRqqt2/assignment_loss‚1.4917051361004512 0loss/FBt1/assignment_loss‚0.2338345046496579 /loss/FBt2/detection_loss‚2.3224754017410856 1loss/FBt3/assignment_loss‚0.15143334650064094 /loss/FBt4/assignment_loss‚0.314572887999847 /loss/FRt4/detection_loss‚1.7045607035055415 /loss/FBt4/detection_loss‚1.7407221380959859 _step‚133 2loss/SRqqt3/assignment_loss‚2.1379326195879416 %loss/total_loss‚44.41434645737648 epoch‚2 trainer/global_step‚1649 /loss/FRt4/assignment_loss‚1.758347699205384 /loss/FRt2/detection_loss‚2.6850099435219397 0loss/SRqqt3/detection_loss‚2.974787846659169 1loss/SRqqt4/detection_loss‚0.9842053127491421 /loss/FBt3/detection_loss‚0.7437941563218388 _runtime‚ 2817.77906793 2loss/SRqqt4/assignment_loss‚1.0145521662565635 /loss/FRt1/detection_loss‚2.4620674523440282 0loss/SRqqt1/detection_loss‚2.385546898952237½a°†Ö 0loss/FRt2/assignment_loss‚3.6296340153767512 0loss/FRt3/assignment_loss‚2.3623568836380455 0loss/FBt2/assignment_loss‚0.3482081794669615 % _timestamp‚1.7885037417005668e+09 2loss/SRqqt1/assignment_loss‚2.0971957042261407 .loss/FBt1/detection_loss‚2.742685310840607 /loss/FRt3/detection_loss‚1.8947146710227518 0loss/SRqqt2/detection_loss‚2.044652602672577 .loss/FRt1/assignment_loss‚4.18935087594119 2loss/SRqqt2/assignment_loss‚1.4917051361004512 0loss/FBt1/assi¼§d“\gnment_loss‚0.2338345046496579 /loss/FBt2/detection_loss‚2.3224754017410856 1loss/FBt3/assignment_loss‚0.15143334650064094 /loss/FBt4/assignment_loss‚0.314572887999847 /loss/FRt4/detection_loss‚1.7045607035055415 /loss/FBt4/detection_loss‚1.7407221380959859 _step‚133 2loss/SRqqt3/assignment_loss‚2.1379326195879416 %loss/total_loss‚44.41434645737648 epoch‚2 trainer/global_step‚1649 /loss/FRt4/assignment_loss‚1.758347699205384 /loss/FRt2/detection_loss‚2.6850099435219397 0loss/SRqqt3/detection_loss‚2.974787846659169 1loss/SRqqt4/detection_loss‚0.9842053127491421 /loss/FBt3/detection_loss‚0.7437941563218388 _runtime‚ 2817.77906793 2loss/SRqqt4/assignment_loss‚1.0145521662565635 /loss/FRt1/detection_loss‚2.4620674523440282 0loss/SRqqt1/detection_loss‚2.385546898952237…õço“‡‚(:ˆ ¿ÍéÔíë–º gpu.0.gpu‚24.0 gpu.0.memory‚2, gpu.0.memoryAllocated‚16.718826293945312* gpu.0.memoryAllocatedBytes‚ 14361362432 gpu.0.temp‚34.0 gpu.0.powerWatts‚173.982' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚34.7964 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0³Íó2Žˆ‚(:ƒ ¿ÍéÔðÃ¥» disk.dm-0.out‚64931.32421875 disk.dm-1.in‚ 0.6953125 disk.sda1.in‚ 0.1015625 disk.sda1.out‚0# disk.nvme0c0n1.in‚ 6767.30859375 disk.sda.in‚0.796875$ memory_percent‚98.93841743469238* proc.memory.percent‚18.503928184509277 cpu‚25.726125130529166 disk.sda2.in‚0 disk.sda2.out‚0 disk.sda3.out‚ 216.8515625 disk.drbd1260.in‚0 disk.sda.out‚ 216.8515625' proc.memory.availableMB‚ 173.9296875# proc.memory.rssMB‚ 3031.68359375 proc.cpu.threads‚17 network.recv‚955006) disk./.usagePercent‚78.71781207031695% disk./.usageGB‚1172.9522857666016! disk.nvme0n1.in‚ 6767.30859375# disk.nvme0n1.out‚64931.32421875 disk.dm-1.out‚ 216.8515625 disk.drbd1260.out‚0.390625 network.sent‚ 3.776497e+06 disk.sda3.in‚ 0.6953125% disk.nvme0c0n1.out‚64931.32421875 disk.dm-0.in‚ 6767.30859375aY픉‚(:’ ÎÍéÔ¿Ðê¸ gpu.0.gpu‚2.0 gpu.0.memory‚0, gpu.0.memoryAllocated‚16.718826293945312* gpu.0.memoryAllocatedBytes‚ 14361362432 gpu.0.temp‚33.0 gpu.0.powerWatts‚122.743' gpu.0.enforcedPowerLimitWatts‚500.0) gpu.0.powerPercent‚24.548599999999997 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0YÒ„—Š‚(:Œ ÎÍéÔˆ¼Íº disk.sda2.out‚0& disk.nvme0n1.out‚65003.78173828125# memory_percent‚98.9384412765503* proc.memory.percent‚18.503928184509277) disk./.usagePercent‚78.71834354195803 disk.sda.out‚ 217.3359375 disk.sda3.out‚ 217.3359375 disk.dm-1.out‚ 217.3359375 disk.drbd1260.in‚0 disk.drbd1260.out‚ 0.39453125# proc.memory.rssMB‚ 3031.68359375 cpu‚25.72452739414972# disk.nvme0c0n1.in‚ 6767.30859375( disk.nvme0c0n1.out‚65003.78173828125 disk.dm-1.in‚ 0.6953125 proc.cpu.threads‚17 network.sent‚ 3.863403e+06 network.recv‚962087 disk.sda2.in‚0 disk.sda3.in‚ 0.6953125! disk.nvme0n1.in‚ 6767.30859375 disk.dm-0.in‚ 6767.30859375# disk.dm-0.out‚65003.78173828125( proc.memory.availableMB‚ 173.92578125$ disk./.usageGB‚1172.960205078125 disk.sda1.in‚ 0.1015625 disk.sda1.out‚0 disk.sda.in‚0.796875«ƒ“‹‚(:ˆ ÝÍéÔ´»ý¸ gpu.0.gpu‚49.0 gpu.0.memory‚5, gpu.0.memoryAllocated‚16.718826293945312* gpu.0.memoryAllocatedBytes‚ 14361362432 gpu.0.temp‚34.0 gpu.0.powerWatts‚169.256' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚33.8512 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0ó1+Œ‚(:’ ÝÍéÔ±“äº disk.sda.in‚0.796875( proc.memory.availableMB‚ 173.69921875# proc.memory.rssMB‚ 3031.68359375 cpu‚25.722585204598104 proc.cpu.threads‚17 network.recv‚965340 disk.drbd1260.in‚0 disk.sda3.out‚ 217.67578125$ memory_percent‚98.93982410430908* proc.memory.percent‚18.503928184509277! disk.nvme0n1.in‚ 6767.30859375& disk.nvme0n1.out‚65069.35986328125# disk.dm-0.out‚65069.35986328125 disk.sda1.in‚ 0.1015625 disk.sda.out‚ 217.67578125 disk.sda2.in‚0 disk.sda2.out‚0 disk.sda3.in‚ 0.6953125) disk./.usagePercent‚78.72574983992361% disk./.usageGB‚1173.0705642700195 disk.dm-0.in‚ 6767.30859375 disk.dm-1.in‚ 0.6953125 disk.dm-1.out‚ 217.67578125 disk.drbd1260.out‚ 0.39453125# disk.nvme0c0n1.in‚ 6767.30859375( disk.nvme0c0n1.out‚65069.35986328125 network.sent‚ 3.868455e+06 disk.sda1.out‚0Šä“‚(:ˆ ìÍéÔƒ¯Ì¹ gpu.0.gpu‚46.0 gpu.0.memory‚6, gpu.0.memoryAllocated‚16.718826293945312* gpu.0.memoryAllocatedBytes‚ 14361362432 gpu.0.temp‚35.0 gpu.0.powerWatts‚163.339' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚32.6678 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0;«ŠŽ‚(:ÿ ìÍéÔñ—§»( proc.memory.availableMB‚ 173.69140625 proc.cpu.threads‚17 network.sent‚ 3.875577e+06 network.recv‚969208 disk.sda2.out‚0 disk.sda3.in‚ 0.6953125 disk.dm-0.in‚ 6916.453125 disk.dm-1.in‚ 0.6953125* proc.memory.percent‚18.503928184509277 cpu‚25.72099966327391 disk.sda2.in‚0 disk.sda3.out‚ 218.03125! disk.nvme0c0n1.in‚ 6916.453125 disk.nvme0n1.in‚ 6916.453125& disk.nvme0n1.out‚65326.13330078125 disk.sda1.out‚0 disk.sda.out‚ 218.03125# memory_percent‚98.9398717880249( disk./.usagePercent‚78.7340744369145% disk./.usageGB‚1173.1946067810059# disk.dm-0.out‚65326.13330078125 disk.drbd1260.in‚0 disk.drbd1260.out‚ 0.3984375 disk.sda1.in‚ 0.1015625 disk.sda.in‚0.796875# proc.memory.rssMB‚ 3031.68359375( disk.nvme0c0n1.out‚65326.13330078125 disk.dm-1.out‚ 218.03125kèš“‚(:ˆ ûÍéÔ¡É™¹ gpu.0.gpu‚28.0 gpu.0.memory‚2, gpu.0.memoryAllocated‚16.718826293945312* gpu.0.memoryAllocatedBytes‚ 14361362432 gpu.0.temp‚34.0 gpu.0.powerWatts‚138.956' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚27.7912 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0«^N®‰‚(:þ ûÍéÔåçâº" disk.dm-0.out‚65848.0966796875 disk.dm-1.out‚ 218.453125 disk.sda1.out‚0* proc.memory.percent‚18.503928184509277 cpu‚25.719535598349214 proc.cpu.threads‚17 network.sent‚ 3.879728e+06 network.recv‚973523 disk.sda2.in‚0 disk.sda2.out‚0 disk.nvme0n1.in‚ 6916.453125% disk.nvme0n1.out‚65848.0966796875 disk.dm-1.in‚ 0.6953125$ proc.memory.availableMB‚173.6875) disk./.usagePercent‚78.73246517355426 disk.sda3.in‚ 0.6953125 disk.sda3.out‚ 218.453125! disk.nvme0c0n1.in‚ 6916.453125' disk.nvme0c0n1.out‚65848.0966796875 disk.sda1.in‚ 0.1015625 disk.sda.in‚0.796875 disk.sda.out‚ 218.453125% disk./.usageGB‚1173.1706275939941 disk.dm-0.in‚ 6916.453125 disk.drbd1260.in‚0 disk.drbd1260.out‚ 0.3984375$ memory_percent‚98.93989562988281# proc.memory.rssMB‚ 3031.68359375˜›EÌ’‘‚(:‡ ŠÎéÔ¤­É¹ gpu.0.gpu‚14.0 gpu.0.memory‚1, gpu.0.memoryAllocated‚16.718826293945312* gpu.0.memoryAllocatedBytes‚ 14361362432 gpu.0.temp‚32.0 gpu.0.powerWatts‚93.694' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚18.7388 gpu.0.smClock‚1275 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0ø-L„‘’‚(:† ŠÎéÔȿͼ disk.sda.in‚0.796875& disk.nvme0c0n1.out‚66144.478515625$ disk.nvme0n1.out‚66144.603515625 disk.dm-0.in‚ 6916.453125 disk.dm-1.out‚ 218.88671875# proc.memory.rssMB‚ 3031.68359375 proc.cpu.threads‚17% disk./.usageGB‚1173.8211135864258 disk.drbd1260.in‚0 disk.sda2.out‚0 disk.sda3.in‚ 0.6953125 disk.sda3.out‚ 218.88671875! disk.dm-0.out‚66135.853515625 disk.dm-1.in‚ 0.6953125$ memory_percent‚98.93991947174072 network.recv‚978250 disk.sda1.in‚ 0.1015625 disk.sda1.out‚0 disk.sda.out‚ 218.88671875 disk.sda2.in‚0! disk.nvme0c0n1.in‚ 6916.453125 disk.nvme0n1.in‚ 6916.453125( proc.memory.availableMB‚ 173.68359375 network.sent‚ 3.885829e+06) disk./.usagePercent‚78.77611983430045 disk.drbd1260.out‚ 0.40234375* proc.memory.percent‚18.503928184509277 cpu‚25.717788465707997ßÊØ12“- _wandbruntime‚2899  _runtime‚2899­íxVÆ”À trainer/global_step‚1699 % _timestamp‚1.7885038239045758e+09 _runtime‚ 2899.98430167 _step‚134 & lr-AdamW/pg1‚0.0011781786255344667 & lr-AdamW/pg2‚0.0011781786255344667Ü+gòË•Å & lr-AdamW/pg1‚0.0011781786255344667 & lr-AdamW/pg2‚0.0011781786255344667 trainer/global_step‚1699 % _timestamp‚1.7885038239045758e+09 _runtime‚ 2899.98430167 _step‚134†CSsî2–- _wandbruntime‚2899  _runtime‚28994.FÜä'—ÞO E=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚895 JBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚286 TDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 13035530240 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 586906624 ^JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚140483269330944 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 YEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚138429369470976 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 @8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚895 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚17039360 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 586906624 TGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚42480728 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 303266304 D>DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 G?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚609 IDeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚3689 WGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 13035530240 ZHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 1669178041344 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚26089 ZFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚140482966064640 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 [KDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 13207863296 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 \HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚136759943144960 _runtime‚2899.988491534 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚26089 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 231245312 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 G?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚286 trainer/global_step‚1699 ]IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚136759960184320 `LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚136759960184320 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚26087 VBDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated‚138429369470976 QDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚30639357 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚607 SEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 384795648 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚17039360 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚31685 _MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 1746336925184 P@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 13568115200 PCDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated‚42480728 YEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚136759943144960 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚10510324 _step‚135 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 E8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚77343344 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚31930 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚2 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚10510929 L?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚34888703 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚3689 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚24903680 TFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 599785472 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚286 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚26087 N@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 248284672 M=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 13568115200 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚20128426 `LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚133809142866032 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 WGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 12589930496 PCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚34888705 ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 278362624 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 320831488 QCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 243631704 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 248284672 M@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚77369433 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 YEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚135469205803043 PCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚34888703 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 152061440 C;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚895 [IDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 1669409286656 PCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚42454641 H?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚1321 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 599785472 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚31685 eQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚138736654043136 O?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 13807648768 YIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 13207863296 @8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚895 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚2 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚605 OFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚1299 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 UGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 582240773 R>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚138429121186304 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚20128428 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 UADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚135468962171339 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 599785472 cQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 1746615287808 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0ž#Ö}é'˜ãO UADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚138429121186304 aMDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚138736629139456 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 ^LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 1660062937011 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚609 ^LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 1669409286656 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚31930 XJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 226592344 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 TGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚34888705 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 WEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 1669178041344 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 231245312 MFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚33 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 ZHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 1659836344667 RBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 13807648768 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 % _timestamp‚1.7885038239080076e+09 M@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚30638750 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 \HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚133809125826672 TDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 13807648768 G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚3689 WGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 13035530240 ZHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 1669178041344 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚26089 ZFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚140482966064640 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 [KDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 13207863296 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 \HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚136759943144960 _runtime‚2899.988491534 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚26089 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 231245312 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 G?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚286 trainer/global_step‚1699 ]IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚136759960184320 `LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚136759960184320 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚26087 VBDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated‚138429369470976 QDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚30639357 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚607 SEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 384795648 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚17039360 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚31685 _MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 1746336925184 P@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 13568115200 PCDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated‚42480728 YEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚136759943144960 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚10510324 _step‚135 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 E8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚77343344 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚31930 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚2 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚10510929 L?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚34888703 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚3689 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚24903680 TFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 599785472 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚286 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚26087 N@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 248284672 M=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 13568115200 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚20128426 `LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚133809142866032 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 WGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 12589930496 PCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚34888705 ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 278362624 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 320831488 QCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 243631704 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 248284672 M@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚77369433 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 YEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚135469205803043 PCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚34888703 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 152061440 C;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚895 [IDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 1669409286656 PCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚42454641 H?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚1321 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 599785472 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚31685 eQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚138736654043136 O?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 13807648768 YIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 13207863296 @8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚895 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚2 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚605 OFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚1299 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 UGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 582240773 R>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚138429121186304 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚20128428 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 UADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚135468962171339 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 599785472 cQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 1746615287808 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 E=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚895 JBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚286 TDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 13035530240 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 586906624 ^JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚140483269330944 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 YEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚138429369470976 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 @8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚895 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚17039360 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 586906624 TGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚42480728 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 303266304 D>DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 G?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚609 IDeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚77414001 NADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚34908660 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 303252992 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 243631912 P@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 13807648768 cODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚138816248903168 [GDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚136838471393280 OADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 248297984 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 582240773 ZFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚136838454353920 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24903680 UEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 13035530240 MDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚1299 YIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 13207863296 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 trainer/global_step‚1699 NADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚42505339 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 231258624 ]KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 1747344520192 XFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 1660788392061 M=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 13807648768 WGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 13207863296 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚2 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 231258624 QCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 384795648 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚895 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 UCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 1670135512576 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚31930 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 ^JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚136838471393280 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 320831488 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚17039360 S?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚138508589866496 I?DeviceStatsMoniMß6²tor.on_train_batch_end/active.small_pool.current‚26113 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚895 C6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚77387886 N>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 13568115200 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚31685 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 RBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 13807648768 WIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 599785472 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 599785472 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚34908660 [MDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚ 278349312 OBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚30656754 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚10516833 \HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚140563871772672 ZFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚133885972561216 TDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 13207863296 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚286 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1321 REDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚20139919 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚33 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚609 EDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 _runtime‚2901.176394467 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚26115 FDeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 % _timestamp‚1.7885038250968983e+09 L>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 248297984 PDeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚30656138 REDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚10516219 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚895 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚609 G:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚77414001 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚614 aODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 1747622869504 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚895 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚286 FDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 13118524029 WCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚136838454353920 S?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚135546760953277 ^JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚133885989600576 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0Ö!ó&›íM ^JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚136838471393280 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 320831488 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚17039360 S?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚138508589866496 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚26113 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚895 C6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚77387886 N>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 13568115200 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚31685 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 RBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 13807648768 WIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 599785472 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 599785472 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚34908660 [MDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚ 278349312 OBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚30656754 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚10516833 \HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚140563871772672 ZFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚133885972561216 TDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 13207863296 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚286 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1321 REDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚20139919 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚33 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚609 EDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 _runtime‚2901.176394467 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚26115 FDeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 % _timestamp‚1.7885038250968983e+09 L>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 248297984 PDeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚30656138 REDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚10516219 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚895 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚609 G:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚77414001 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚614 aODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 1747622869504 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚895 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚286 FDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 13118524029 WCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚136838454353920 S?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚135546760953277 ^JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚133885989600576 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 K>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚77414001 NADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚34908660 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 303252992 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 243631912 P@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 13807648768 cODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚138816248903168 [GDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚136838471393280 OADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 248297984 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 582240773 ZFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚136838454353920 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24903680 UEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 13035530240 MDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚1299 YIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 13207863296 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 trainer/global_step‚1699 NADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚42505339 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 231258624 ]KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 1747344520192 XFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 1660788392061 M=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 13807648768 WGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 13207863296 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚2 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 231258624 QCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 384795648 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚895 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 UCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 1670135512576 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚31930 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0ˆìUd2œ- _wandbruntime‚2901  _runtime‚2901sVŸgVÐ 1loss/SRqqt1/detection_loss‚2.6491991732009623 $loss/total_loss‚44.5859987245432 0loss/FRt4/assignment_loss‚2.0021140287842667 1loss/SRqqt4/assignment_loss‚1.090498169656429 .loss/FRt1/detection_loss‚2.415276685486669 1loss/SRqqt2/detection_loss‚1.9005950874295727 2loss/SRqqt3/assignment_loss‚1.6849065171554685 1loss/SRqqt4/detection_loss‚0.9837031855020258 .loss/FBt1/detection_loss‚2.276211272447537 /loss/FBt2/detection_loss‚2.3578577959859692 _runtime‚2901.177482427 /loss/FBt2/assignment_loss‚0.294007568852681 1loss/FBt4/assignment_loss‚0.25825102961789526 % _timestamp‚1.7885038250989404e+09 .loss/FRt1/assignment_loss‚3.79325460869333 .loss/FRt3/detection_loss‚2.406548264045869 .loss/FRt4/detection_loss‚1.787071405142023 trainer/global_step‚1699 0loss/FRt2/assignment_loss‚3.1522341503037348 1loss/FBt3/assignment_loss‚0.17583069679641786 /loss/FRt2/detection_loss‚2.5837516548732915 0loss/SRqqt3/detection_loss‚2.784137084148824 _step‚137 /loss/FRt3/assignment_loss‚2.756242560763513 /loss/FBt3/detection_loss‚0.9203465490488054 /loss/FBt4/detection_loss‚2.6143880039453506 epoch‚2 2loss/SRqqt1/assignment_loss‚1.8891741882923037 1loss/SRqqt2/assignment_loss‚1.578027746629441 1loss/FBt1/assignment_loss‚0.23237129774082485mèÔ[žÕ epoch‚2 2loss/SRqqt1/assignment_loss‚1.8891741882923037 1loss/SRqqt2/assignment_loss‚1.578027746629441 1loss/FBt1/assignment_loss‚0.23237129774082485 1loss/SRqqt1/detection_loss‚2.6491991732009623 $loss/total_loss‚44.5859987245432 0loss/FRt4/assignment_loss‚2.0021140287842667 1loss/SRqqt4/assignment_loss‚1.090498169656429 .loss/FRt1/detection_loss‚2.415276685486669 1loss/SRqqt2/detection_loss‚1.9005950874295727 2loss/SRqqt3/assignment_loss‚1.6849065171554685 1loss/SRqqt4/detection_loss‚0.9837031855020258 .loss/FBt1/detection_loss‚2.276211272447537 /loss/FBt2/detection_loss‚2.3578577959859692 _runtime‚2901.177482427 /loss/FBt2/assignment_loss‚0.294007568852681 1loss/FBt4/assignment_loss‚0.25825102961789526 % _timestamp‚1.7885038250989404e+09 .loss/FRt1/assignment_loss‚3.79325460869333 .loss/FRt3/detection_loss‚2.406548264045869 .loss/FRt4/detection_loss‚1.787071405142023 trainer/global_step‚1699 0loss/FRt2/assignment_loss‚3.1522341503037348 1loss/FBt3/assignment_loss‚0.17583069679641786 /loss/FRt2/detection_loss‚2.5837516548732915 0loss/SRqqt3/detection_loss‚2.784137084148824 _step‚137 /loss/FRt3/assignment_loss‚2.756242560763513 /loss/FBt3/detection_loss‚0.9203465490488054 /loss/FBt4/detection_loss‚2.6143880039453506‰ ™Ì‘Ÿ‚(:† ™ÎéÔ®ùí¹ gpu.0.gpu‚1.0 gpu.0.memory‚0, gpu.0.memoryAllocated‚16.718826293945312* gpu.0.memoryAllocatedBytes‚ 14361362432 gpu.0.temp‚31.0 gpu.0.powerWatts‚93.981' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚18.7962 gpu.0.smClock‚1275 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0n+k’ ‚(:‡ ™ÎéÔõ–ߺ* proc.memory.percent‚18.503928184509277 proc.cpu.threads‚17 disk.sda2.out‚0 disk.nvme0n1.in‚ 6916.453125# disk.dm-0.out‚66266.43310546875 disk.drbd1260.out‚ 0.40234375 disk.sda1.out‚0 disk.sda.in‚0.796875 network.sent‚ 3.890918e+06 network.recv‚981774! disk.nvme0c0n1.in‚ 6916.453125( disk.nvme0c0n1.out‚66266.43310546875 disk.dm-0.in‚ 6916.453125 disk.dm-1.in‚ 0.6953125 disk.dm-1.out‚ 219.3984375 disk.sda.out‚ 219.3984375( disk./.usagePercent‚78.7928637510699 disk.sda2.in‚0 disk.sda3.in‚ 0.6953125 disk.drbd1260.in‚0 disk.sda1.in‚ 0.1015625$ memory_percent‚98.94013404846191# proc.memory.rssMB‚ 3031.68359375 cpu‚25.716210206042145% disk./.usageGB‚1174.0706100463867 disk.sda3.out‚ 219.3984375& disk.nvme0n1.out‚66266.43310546875' proc.memory.availableMB‚ 173.6484375éû!ž¡‚(:“ ¨ÎéÔ‘¡Ú» gpu.0.gpu‚17.0 gpu.0.memory‚1, gpu.0.memoryAllocated‚16.718826293945312* gpu.0.memoryAllocatedBytes‚ 14361362432 gpu.0.temp‚31.0 gpu.0.powerWatts‚111.896' gpu.0.enforcedPowerLimitWatts‚500.0) gpu.0.powerPercent‚22.379199999999997 gpu.0.smClock‚1275 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0Ö|k¯…¢‚(:ú ¨ÎéÔí‘­¿& proc.memory.availableMB‚ 173.640625* proc.memory.percent‚18.503952026367188 cpu‚25.714615341993575 disk.sda3.in‚ 0.6953125" disk.nvme0c0n1.in‚ 7116.1484375$ disk.nvme0n1.out‚66696.693359375 disk.sda1.out‚0 proc.cpu.threads‚17 network.sent‚ 3.979503e+06) disk./.usagePercent‚78.80821703491337% disk./.usageGB‚1174.2993850708008 disk.sda3.out‚ 220.078125 disk.nvme0n1.in‚ 7116.1484375! disk.dm-0.out‚66696.693359375 disk.dm-1.out‚ 220.078125 disk.drbd1260.out‚0.40625 network.recv‚987804 disk.sda2.out‚0& disk.nvme0c0n1.out‚66696.693359375 disk.dm-1.in‚ 0.6953125 disk.sda1.in‚ 0.1015625 disk.sda.in‚0.796875 disk.sda.out‚ 220.078125 proc.memory.rssMB‚ 3031.6875 disk.sda2.in‚0 disk.dm-0.in‚ 7116.1484375 disk.drbd1260.in‚0$ memory_percent‚98.94018173217773Ÿ³V‘£‚(:† ·ÎéÔ…Ùé¹ gpu.0.gpu‚1.0 gpu.0.memory‚0, gpu.0.memoryAllocated‚16.718826293945312* gpu.0.memoryAllocatedBytes‚ 14361362432 gpu.0.temp‚31.0 gpu.0.powerWatts‚94.489' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚18.8978 gpu.0.smClock‚1275 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0Ï8wËŠ¤‚(:ÿ ·ÎéÔúŽú¼ disk.drbd1260.out‚0.40625! disk.dm-0.out‚67026.166015625 disk.dm-1.out‚ 220.4609375( proc.memory.availableMB‚ 173.63671875% disk./.usageGB‚1174.2574157714844 disk.sda1.in‚ 0.1015625 disk.sda1.out‚0 disk.sda.in‚0.796875 disk.sda2.in‚0& disk.nvme0c0n1.out‚67026.166015625$ memory_percent‚98.94020557403564 proc.memory.rssMB‚ 3031.6875) disk./.usagePercent‚78.80540044002166 disk.sda.out‚ 220.4609375 disk.sda3.out‚ 220.4609375 disk.nvme0n1.in‚ 7116.1484375$ disk.nvme0n1.out‚67026.166015625 disk.dm-0.in‚ 7116.1484375* proc.memory.percent‚18.503952026367188 cpu‚25.713078938957327 network.sent‚ 3.983446e+06 network.recv‚992281 disk.sda2.out‚0 disk.sda3.in‚ 0.6953125" disk.nvme0c0n1.in‚ 7116.1484375 disk.dm-1.in‚ 0.6953125 proc.cpu.threads‚17 disk.drbd1260.in‚0~\Ç‘¥‚(:† ÆÎéÔ´§Þ¹ gpu.0.gpu‚8.0 gpu.0.memory‚1, gpu.0.memoryAllocated‚16.718826293945312* gpu.0.memoryAllocatedBytes‚ 14361362432 gpu.0.temp‚31.0 gpu.0.powerWatts‚110.705' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚22.141 gpu.0.smClock‚1275 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0¹{­~‘¦‚(:† ÆÎéÔÐÍÓ¼* proc.memory.percent‚18.503952026367188 network.sent‚ 3.991626e+06# disk.dm-0.out‚67656.90869140625 disk.drbd1260.in‚0 disk.sda2.out‚0$ memory_percent‚98.94022941589355 cpu‚25.71193616325881 disk.dm-1.in‚ 0.6953125 disk.dm-1.out‚ 220.9609375) disk./.usagePercent‚78.85172653938764 disk.sda3.in‚ 0.6953125( disk.nvme0c0n1.out‚67656.90869140625 disk.nvme0n1.in‚ 7116.1484375& disk.nvme0n1.out‚67656.90869140625 disk.sda1.in‚ 0.1015625% disk./.usageGB‚1174.9477081298828 disk.sda1.out‚0 disk.sda.in‚0.796875 disk.sda.out‚ 220.9609375' proc.memory.availableMB‚ 173.6328125 proc.memory.rssMB‚ 3031.6875 proc.cpu.threads‚17 disk.dm-0.in‚ 7116.1484375 network.recv‚996998 disk.sda3.out‚ 220.9609375" disk.nvme0c0n1.in‚ 7116.1484375 disk.drbd1260.out‚ 0.41015625 disk.sda2.in‚0<8Òd“§‚(:ˆ ÕÎéԮ骹 gpu.0.gpu‚32.0 gpu.0.memory‚2, gpu.0.memoryAllocated‚16.718826293945312* gpu.0.memoryAllocatedBytes‚ 14361362432 gpu.0.temp‚32.0 gpu.0.powerWatts‚111.526' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚22.3052 gpu.0.smClock‚1275 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0]Ìf‘•¨‚(:Š ÕÎéÔۻܺ$ memory_percent‚98.94027709960938# proc.memory.availableMB‚173.625% disk./.usageGB‚1174.9691619873047 disk.sda2.in‚0 disk.sda2.out‚0 disk.sda3.in‚ 0.6953125 disk.nvme0n1.in‚ 7116.1484375 disk.dm-0.in‚ 7116.1484375# disk.dm-0.out‚68082.43603515625 disk.dm-1.in‚ 0.6953125 network.sent‚ 3.994722e+06 network.recv‚ 1.000472e+06( disk./.usagePercent‚78.8531663257602" disk.nvme0c0n1.in‚ 7116.1484375& disk.nvme0n1.out‚68082.43603515625 disk.dm-1.out‚ 221.33203125 disk.drbd1260.in‚0 disk.drbd1260.out‚ 0.41015625 disk.sda3.out‚ 221.33203125 disk.sda.out‚ 221.33203125 proc.memory.rssMB‚ 3031.6875* proc.memory.percent‚18.503952026367188 cpu‚25.71099951845049 proc.cpu.threads‚17( disk.nvme0c0n1.out‚68082.43603515625 disk.sda1.in‚ 0.1015625 disk.sda1.out‚0 disk.sda.in‚0.796875+Ǻ¯ž©‚(:“ äÎéÔýÖ›¹ gpu.0.gpu‚34.0 gpu.0.memory‚3, gpu.0.memoryAllocated‚16.718826293945312* gpu.0.memoryAllocatedBytes‚ 14361362432 gpu.0.temp‚32.0 gpu.0.powerWatts‚111.504' gpu.0.enforcedPowerLimitWatts‚500.0) gpu.0.powerPercent‚22.300800000000002 gpu.0.smClock‚1275 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0æ蚪‚(: äÎéÔ³×Ǻ disk.sda1.out‚0( disk.nvme0c0n1.out‚68364.13134765625& disk.nvme0n1.out‚68364.13134765625 disk.dm-0.in‚ 7238.47265625 disk.drbd1260.out‚ 0.4140625 disk.sda.in‚0.796875* proc.memory.percent‚18.503952026367188 cpu‚25.709463930209168( disk./.usagePercent‚78.8531663257602 disk.sda2.in‚0# disk.nvme0c0n1.in‚ 7238.47265625 disk.sda.out‚ 221.7421875$ memory_percent‚98.94025325775146 disk.sda2.out‚0 disk.sda3.in‚ 0.6953125 disk.dm-1.in‚ 0.6953125 disk.dm-1.out‚ 221.7421875 disk.sda1.in‚ 0.1015625( proc.memory.availableMB‚ 173.62890625 proc.memory.rssMB‚ 3031.6875 proc.cpu.threads‚17# disk.dm-0.out‚68364.13134765625 network.sent‚ 3.999906e+06 network.recv‚ 1.004457e+06% disk./.usageGB‚1174.9691619873047 disk.sda3.out‚ 221.7421875! disk.nvme0n1.in‚ 7238.47265625 disk.drbd1260.in‚0Ÿ…ž2«- _wandbruntime‚2986  _runtime‚2986Þc´ªÇ¬Á _runtime‚2986.191248754 _step‚138 & lr-AdamW/pg1‚0.0011668138721252317 & lr-AdamW/pg2‚0.0011668138721252317 trainer/global_step‚1749 % _timestamp‚1.7885039101115158e+09(Uâ²Ì­Æ & lr-AdamW/pg1‚0.0011668138721252317 & lr-AdamW/pg2‚0.0011668138721252317 trainer/global_step‚1749 % _timestamp‚1.7885039101115158e+09 _runtime‚2986.191248754 _step‚138Š.A2®- _wandbruntime‚2986  _runtime‚2986e[ôQ¯ÞO TGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚43713330 aMDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚142716372141056 BDeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚3689 N@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 248284672 P@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 13118524029 SEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 384795648 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 ]IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚140685520632320 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚596 XJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 226592344 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚26089 YEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚140685503592960 \HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚140685503592960 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 YIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 13207863296 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 PCDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated‚43713330 `LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚140685520632320 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚17039360 OFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚1299 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 586906624 YEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚142402831308800 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚31930 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 PCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚35886553 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 599785472 @8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚895 VBDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated‚142402831308800 WGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 13035530240 cQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 1796742626816 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 QCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 243631704 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚286 JBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚286 WEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 1717079431168 ^LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 1717310676480 TFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 599785472 PCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚35886555 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 586906624 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚10810506 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚10809910 TGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚35886555 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 \HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚137651462553872 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚31685 ?9DeviceS̃³ÐàtatsMonitor.on_train_batch_start/num_alloc_retries‚0 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚609 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚31930 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 C;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚895 E8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚79573796 VFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 13207863296 `LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚137651479593232 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚20703076 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 G?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚286 TDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 13035530240 PCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚43687243 ZFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚144512836405248 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 152061440 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚26087 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚2 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚24903680 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚2 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚31685 D>DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 P@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 13568115200 IDeviceStatsMonitor.on_train_batch_start/allocation.all.current‚26089 ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 278362624 UGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 582240773 E=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚895 R>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚142402583024128 M=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 13568115200 H?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚1321 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 303266304 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 248284672 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚3689 QDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚31513584 M@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚31512986 MFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚33 eQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚142716397044736 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 RBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 13807648768 UADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚142402583024128 G?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚609 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚17039360 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚20703078 @8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚895 ZHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 1707465886375 L?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚35886553 [IDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 1717310676480 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚598 YEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚139359172071951 O?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 13807648768 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚26087 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 231245312 M@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚79599885Ëò¨é'°ãO ]IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚140685520632320 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚596 XJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 226592344 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚26089 YEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚140685503592960 \HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚140685503592960 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 YIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 13207863296 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 PCDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated‚43713330 `LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚140685520632320 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚17039360 OFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚1299 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 586906624 YEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚142402831308800 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚31930 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 PCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚35886553 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 599785472 @8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚895 VBDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated‚142402831308800 WGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 13035530240 cQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 1796742626816 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 QCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 243631704 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚286 JBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚286 WEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 1717079431168 ^LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 1717310676480 TFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 599785472 PCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚35886555 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 586906624 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚10810506 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚10809910 TGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚35886555 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 \HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚137651462553872 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚31685 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚609 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚31930 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 C;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚895 E8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚79573796 VFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 13207863296 `LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚137651479593232 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚20703076 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 G?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚286 TDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 13035530240 PCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚43687243 ZFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚144512836405248 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 152061440 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚26087 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚2 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚24903680 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚2 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚31685 D>DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 P@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 13568115200 IDeviceStatsMonitor.on_train_batch_start/allocation.all.current‚26089 ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 278362624 UGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 582240773 E=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚895 R>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚142402583024128 M=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 13568115200 H?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚1321 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 303266304 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 248284672 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚3689 QDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚31513584 M@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚31512986 MFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚33 eQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚142716397044736 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 RBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 13807648768 UADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚142402583024128 G?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚609 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚17039360 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚20703078 @8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚895 ZHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 1707465886375 L?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚35886553 [IDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 1717310676480 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚598 YEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚139359172071951 O?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 13807648768 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚26087 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 231245312 M@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚79599885 TGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚43713330 aMDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚142716372141056 BDeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚3689 N@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 248284672 P@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 13118524029 SEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 384795648 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0‹Kö|p2±- _wandbruntime‚2987  _runtime‚29878À‚í&²çM ^JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚140764031841280 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 599785472 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 320831488 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 G:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚79618338 NADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚43711828 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 $ _timestamp‚1.788503911848512e+09 OADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 248297984 T@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚142482300002304 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17039360 WCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚140764014801920 UCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 1718036902400 ZFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚140764014801920 NADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚35906510 REDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚20714569 ^JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚137728326327776 G:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚79644453 P6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚895 \JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 1718268161024 XFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 1718036902400 WCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚139436970854097 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 RBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 13035530240 cODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚142795991904768 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚895 _step‚140 [GDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚140764031841280 UEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 13035530240 \HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚144593743903232 VHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 226592552 WGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 13207863296 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 K;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 13568115200 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚17039360 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 TDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 13207863296 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚2 F6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚895 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚286 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 313738752 S?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚142482051704320 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚2 SEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 586906624 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚35906510 N>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 13118524029 UEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 12589930496 ZFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚137728309288416 P@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 13807648768 NADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚35906512 J=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚43711828 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚31685 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚31930 _KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚142795967001088 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 582240773 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 YIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 13207863296 [MDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚ 288835072 ]KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 1797463163392 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚17039360 trainer/global_step‚1749 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 599785472 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚609 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚286 REDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚43737941 L>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 248297984 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24903680 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 152061440 \JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 1708644526321 WIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 599785472 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚895 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1321 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 BDeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚79644453 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚33 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 QCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 384795648 RBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 13807648768 M=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 13807648768 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 PBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 586906624 K>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚31530531 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 243631912 C6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚79618338 YGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 1718268161024 OBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚31531160 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚627 XFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 1708417933769 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚629 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚26113 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚26113 EDeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚609 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚286 MDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚1299 WCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚142482300002304 N>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 13568115200 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚26115 REDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚35906512 XDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚144593430164480 S?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚139436727222185^0‡Òõ³ìM G:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚79618338 NADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚43711828 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 $ _timestamp‚1.788503911848512e+09 OADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 248297984 T@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚142482300002304 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17039360 WCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚140764014801920 UCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 1718036902400 ZFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚140764014801920 NADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚35906510 REDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚20714569 ^JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚137728326327776 G:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚79644453 P6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚895 \JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 1718268161024 XFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 1718036902400 WCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚139436970854097 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 RBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 13035530240 cODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚142795991904768 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚895 _step‚140 [GDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚140764031841280 UEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 13035530240 \HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚144593743903232 VHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 226592552 WGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 13207863296 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 K;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 13568115200 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚17039360 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 TDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 13207863296 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚2 F6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚895 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚286 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 313738752 S?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚142482051704320 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚2 SEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 586906624 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚35906510 N>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 13118524029 UEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 12589930496 ZFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚137728309288416 P@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 13807648768 NADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚35906512 J=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚43711828 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚31685 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚31930 _KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚142795967001088 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 582240773 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 YIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 13207863296 [MDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚ 288835072 ]KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 1797463163392 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚17039360 trainer/global_step‚1749 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 599785472 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚609 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚286 REDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚43737941 L>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 248297984 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24903680 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 152061440 \JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 1708644526321 WIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 599785472 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚895 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1321 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 Byý onitor.on_train_batch_end/oversize_segments.peak‚0 K>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚79644453 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚33 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 QCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 384795648 RBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 13807648768 M=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 13807648768 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 PBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 586906624 K>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚31530531 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 243631912 C6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚79618338 YGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 1718268161024 OBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚31531160 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚627 XFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 1708417933769 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚629 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚26113 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚26113 EDeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚609 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚286 MDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚1299 WCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚142482300002304 N>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 13568115200 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚26115 REDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚35906512 XDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚144593430164480 S?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚139436727222185 ^JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚140764031841280 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 599785472 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 320831488 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0Œ×oׯ2´- _wandbruntime‚2987  _runtime‚2987®HÔl[µÕ /loss/FRt1/detection_loss‚2.5609594791654557 0loss/SRqqt3/detection_loss‚2.447884405321545 /loss/FBt4/detection_loss‚2.5321971515814465 $loss/total_loss‚45.3667235557163 _runtime‚2987.929515415 /loss/FRt3/detection_loss‚2.1457815492111774 epoch‚2 _step‚141 2loss/SRqqt4/assignment_loss‚1.0650803776980753 0loss/SRqqt2/detection_loss‚2.130394978613793 0loss/FRt3/assignment_loss‚2.2752469273077116 2loss/SRqqt3/assignment_loss‚1.9671695050679976 /loss/FRt4/detection_loss‚1.9408616773089615 1loss/FBt3/assignment_loss‚0.12191396483546205 /loss/FBt2/assignment_loss‚0.126964667954735 1loss/SRqqt4/detection_loss‚1.0954966352468258 /loss/FBt1/detection_loss‚2.2113321479409933 .loss/FBt2/detection_loss‚2.748267501592636 0loss/FRt2/assignment_loss‚3.5187466909855973 2loss/SRqqt1/assignment_loss‚2.3280389697424004 1loss/FBt1/assignment_loss‚0.18867122234538414 /loss/FRt2/detection_loss‚2.4338890395046753 trainer/global_step‚1749 % _timestamp‚1.7885039118510046e+09 1loss/SRqqt2/assignment_loss‚1.753137232193464 1loss/FBt4/assignment_loss‚0.27161692033987495 1loss/SRqqt1/detection_loss‚2.6691138595342636 /loss/FBt3/detection_loss‚0.7392597755521432 /loss/FRt1/assignment_loss‚4.109137691202617 0loss/FRt4/assignment_loss‚1.9855611854690616M«£÷`¶Ú /loss/FRt3/detection_loss‚2.1457815492111774 epoch‚2 _step‚141 2loss/SRqqt4/assignment_loss‚1.0650803776980753 0loss/SRqqt2/detection_loss‚2.130394978613793 0loss/FRt3/assignment_loss‚2.2752469273077116 2loss/SRqqt3/assignment_loss‚1.9671695050679976 /loss/FRt4/detection_loss‚1.9408616773089615 1loss/FBt3/assignment_loss‚0.12191396483546205 /loss/FBt2/assignment_loss‚0.126964667954735 1loss/SRqqt4/detection_loss‚1.0954966352468258 /loss/FBt1/detection_loss‚2.2113321479409933 .loss/FBt2/detection_loss‚2.748267501592636 0loss/FRt2/assignment_loss‚3.5187466909855973 2loss/SRqqt1/assignment_loss‚2.3280389697424004 1loss/FBt1/assignment_loss‚0.18867122234538414 /loss/FRt2/detection_loss‚2.4338890395046753 trainer/global_step‚1749 % _timestamp‚1.7885039118510046e+09 1loss/SRqqt2/assignment_loss‚1.753137232193464 1loss/FBt4/assignment_loss‚0.27161692033987495 1loss/SRqqt1/detection_loss‚2.6691138595342636 /loss/FBt3/detection_loss‚0.7392597755521432 /loss/FRt1/assignment_loss‚4.109137691202617 0loss/FRt4/assignment_loss‚1.9855611854690616 /loss/FRt1/detection_loss‚2.5609594791654557 0loss/SRqqt3/detection_loss‚2.447884405321545 /loss/FBt4/detection_loss‚2.5321971515814465 $loss/total_loss‚45.3667235557163 _runtime‚2987.929515415?Íå“·‚(:ˆ óÎéÔÙ®­» gpu.0.gpu‚19.0 gpu.0.memory‚1, gpu.0.memoryAllocated‚16.718826293945312* gpu.0.memoryAllocatedBytes‚ 14361362432 gpu.0.temp‚31.0 gpu.0.powerWatts‚109.102' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚21.8204 gpu.0.smClock‚1275 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0:¥‹¸‚(:€ óÎéÔ„«ó½# disk.nvme0c0n1.in‚ 7238.47265625( disk.nvme0c0n1.out‚68428.17431640625 disk.dm-1.out‚222.25 disk.sda1.in‚ 0.1015625 proc.memory.rssMB‚ 3031.6875 cpu‚25.708456868207527! disk.nvme0n1.in‚ 7238.47265625# disk.dm-0.out‚68428.17431640625 disk.dm-1.in‚ 0.6953125 disk.sda.in‚0.796875$ memory_percent‚98.94070625305176' proc.memory.availableMB‚ 173.5546875 proc.cpu.threads‚17 network.recv‚ 1.008781e+06 disk.drbd1260.in‚0 disk.sda2.in‚0 disk.sda3.out‚222.25* proc.memory.percent‚18.503952026367188) disk./.usagePercent‚78.85319909472459& disk.nvme0n1.out‚68428.17431640625 disk.dm-0.in‚ 7238.47265625 disk.drbd1260.out‚ 0.4140625 disk.sda1.out‚0 disk.sda.out‚222.25 disk.sda2.out‚0 disk.sda3.in‚ 0.6953125 network.sent‚ 4.005828e+06% disk./.usageGB‚1174.9696502685547ä«£‘¹‚(:† ‚ÏéÔ§Þµ¹ gpu.0.gpu‚0.0 gpu.0.memory‚0, gpu.0.memoryAllocated‚16.718826293945312* gpu.0.memoryAllocatedBytes‚ 14361362432 gpu.0.temp‚31.0 gpu.0.powerWatts‚93.277' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚18.6554 gpu.0.smClock‚1275 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0X›ÑY–º‚(:‹ ‚ÏéԸ׬»% disk./.usageGB‚1174.9461822509766 disk.sda2.in‚0# disk.nvme0c0n1.in‚ 7238.47265625 disk.drbd1260.in‚0 disk.sda1.in‚ 0.1015625 disk.sda.in‚0.796875( proc.memory.availableMB‚ 173.55859375* proc.memory.percent‚18.503952026367188 disk.sda2.out‚0 disk.sda3.in‚ 0.6953125 disk.sda3.out‚ 222.7734375& disk.nvme0c0n1.out‚68661.857421875 disk.dm-1.out‚ 222.7734375 proc.memory.rssMB‚ 3031.6875 cpu‚25.706908379355834! disk.dm-0.out‚68661.857421875 disk.drbd1260.out‚ 0.41796875 disk.sda.out‚ 222.7734375$ memory_percent‚98.94068241119385 proc.cpu.threads‚17$ disk.nvme0n1.out‚68661.857421875 disk.dm-0.in‚ 7238.47265625 network.sent‚ 4.093541e+06 network.recv‚ 1.016033e+06) disk./.usagePercent‚78.85162413637394! disk.nvme0n1.in‚ 7238.47265625 disk.dm-1.in‚ 0.6953125 disk.sda1.out‚0ur¤íœ»‚(:‘ ‘ÏéÔé¶öº gpu.0.gpu‚0.0 gpu.0.memory‚0, gpu.0.memoryAllocated‚16.718826293945312* gpu.0.memoryAllocatedBytes‚ 14361362432 gpu.0.temp‚31.0 gpu.0.powerWatts‚93.564' gpu.0.enforcedPowerLimitWatts‚500.0) gpu.0.powerPercent‚18.712799999999998 gpu.0.smClock‚1275 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0¨Îÿ–¼‚(:‹ ‘ÏéÔ’°©¼ disk.dm-0.in‚ 7238.47265625 cpu‚25.705720500814206) disk./.usagePercent‚78.82629167884579 disk.drbd1260.in‚0 disk.sda1.in‚ 0.1015625 disk.sda.out‚ 223.1015625! disk.nvme0n1.in‚ 7238.47265625$ disk.nvme0n1.out‚69111.474609375$ memory_percent‚98.94063472747803 network.sent‚ 4.098593e+06% disk./.usageGB‚1174.5687103271484 disk.dm-1.out‚ 223.1015625 disk.drbd1260.out‚ 0.41796875 disk.sda2.in‚0! disk.dm-0.out‚69111.474609375( proc.memory.availableMB‚ 173.56640625* proc.memory.percent‚18.503952026367188 disk.dm-1.in‚ 0.6953125 disk.sda1.out‚0 disk.sda.in‚0.796875 disk.sda2.out‚0& disk.nvme0c0n1.out‚69111.474609375 proc.memory.rssMB‚ 3031.6875 proc.cpu.threads‚17 network.recv‚ 1.019481e+06 disk.sda3.in‚ 0.6953125 disk.sda3.out‚ 223.1015625# disk.nvme0c0n1.in‚ 7238.47265625b÷ª™œ½‚(:‘  ÏéÔÆẹ gpu.0.gpu‚0.0 gpu.0.memory‚0, gpu.0.memoryAllocated‚16.718826293945312* gpu.0.memoryAllocatedBytes‚ 14361362432 gpu.0.temp‚31.0 gpu.0.powerWatts‚93.564' gpu.0.enforcedPowerLimitWatts‚500.0) gpu.0.powerPercent‚18.712799999999998 gpu.0.smClock‚1275 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0ˆÁR´›¾‚(:  ÏéÔ•²¹¼* proc.memory.percent‚18.503952026367188 network.sent‚ 4.103859e+06 network.recv‚ 1.02307e+06 disk.sda3.in‚ 0.6953125( disk.nvme0c0n1.out‚69465.20654296875! disk.nvme0n1.in‚ 7363.70703125 disk.dm-0.in‚ 7363.70703125# disk.dm-0.out‚69465.20654296875 disk.sda1.out‚0) disk./.usagePercent‚78.82786663719644 disk.dm-1.out‚ 223.48046875 disk.drbd1260.out‚0.421875 disk.sda1.in‚ 0.1015625' proc.memory.availableMB‚ 173.5546875 proc.memory.rssMB‚ 3031.6875 cpu‚25.704559126276667 proc.cpu.threads‚17 disk.sda3.out‚ 223.48046875 disk.dm-1.in‚ 0.6953125 disk.drbd1260.in‚0$ memory_percent‚98.94070625305176% disk./.usageGB‚1174.5921783447266 disk.sda2.in‚0 disk.sda2.out‚0# disk.nvme0c0n1.in‚ 7363.70703125& disk.nvme0n1.out‚69465.20654296875 disk.sda.in‚0.796875 disk.sda.out‚ 223.48046875Ôò«“¿‚(:ˆ ¯ÏéÔ˜Êع gpu.0.gpu‚12.0 gpu.0.memory‚1, gpu.0.memoryAllocated‚16.718826293945312* gpu.0.memoryAllocatedBytes‚ 14361362432 gpu.0.temp‚32.0 gpu.0.powerWatts‚113.954' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚22.7908 gpu.0.smClock‚1275 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0¾“4Ò›À‚(: ¯ÏéÔø­ ¼ disk.sda2.in‚0 disk.sda3.in‚ 0.6953125( disk.nvme0c0n1.out‚69673.33935546875! disk.nvme0n1.in‚ 7375.83203125& disk.nvme0n1.out‚69673.33935546875 disk.dm-1.out‚ 223.95703125 disk.sda.in‚0.796875 disk.sda.out‚ 223.95703125 disk.sda2.out‚0 disk.sda3.out‚ 223.95703125 disk.dm-1.in‚ 0.6953125 disk.drbd1260.in‚0 disk.sda1.out‚0$ memory_percent‚98.94232749938965' proc.memory.availableMB‚ 173.2890625* proc.memory.percent‚18.503952026367188 disk.dm-0.in‚ 7375.83203125) disk./.usagePercent‚78.82786817324165% disk./.usageGB‚1174.5922012329102# disk.nvme0c0n1.in‚ 7375.83203125 disk.sda1.in‚ 0.1015625 proc.memory.rssMB‚ 3031.6875 proc.cpu.threads‚17 network.sent‚ 4.109792e+06 network.recv‚ 1.027299e+06# disk.dm-0.out‚69673.33935546875 disk.drbd1260.out‚0.421875 cpu‚25.70317237039934E°)2Á- _wandbruntime‚3071  _runtime‚3071æn–ÂÇÂÁ _runtime‚3071.358265799 _step‚142 & lr-AdamW/pg1‚0.0011550194844898345 & lr-AdamW/pg2‚0.0011550194844898345 trainer/global_step‚1799 % _timestamp‚1.7885039952786374e+09Æ@!ÌÃÆ & lr-AdamW/pg1‚0.0011550194844898345 & lr-AdamW/pg2‚0.0011550194844898345 trainer/global_step‚1799 % _timestamp‚1.7885039952786374e+09 _runtime‚3071.358265799 _step‚142ŽÙ)²ö2Ä- _wandbruntime‚3071  _runtime‚3071§Þ ëä'ÅÞO P@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 13118524029 ^LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 1755322020427 YIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 13207863296 SEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 384795648 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 G?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚286 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 152061440 WGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 12589930496 E=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚895 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚2 N@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 248284672 [KDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 13207863296 C;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚895 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 JBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚609 trainer/global_step‚1799 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚613 [IDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 1765212066304 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 O?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 13807648768 TFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 599785472 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚609 YEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚144611064040960 QDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚32388521 H?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚1321 G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚3689 aMDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚146696115142656 G?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚609 E8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚81804248 P@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 13568115200 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚2 eQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚146696140046336 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 586906624 `LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚144611081080320 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚26089 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚21277726 PCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚36884405 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 320831488 \HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚141493799281072 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 IDeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚146376044861952 YEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚146376293146624 IDeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 % _timestamp‚1.7885039952816637e+09 _runtime‚3071.364323942 TGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚36884405 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚286 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚26087 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 QCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 243631704 MFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚33 cQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 1846796897280 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚31685 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚11110793 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚31685 ]IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚144611081080320 WGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 13035530240 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚26087 _MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 1846514340352 @8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚895 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚24903680 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚17039360 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 231245312 M@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚81830337 PCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚44919845 OFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚1299 XJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 226592344 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚31930 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 599785472 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 L?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚44919845 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 231245312 TGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚44945932 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚611 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 307460608‰ÿä—Ó#ÆãO YEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚144611064040960 QDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚32388521 H?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚1321 G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚3689 aMDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚146696115142656 G?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚609 E8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚81804248 P@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 13568115200 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚2 eQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚146696140046336 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 586906624 `LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚144611081080320 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚26089 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚21277726 PCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚36884405 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 320831488 \HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚141493799281072 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 IDeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚146376044861952 YEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚146376293146624 IDeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 % _timestamp‚1.7885039952816637e+09 _runtime‚3071.364323942 TGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚36884405 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚286 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚26087 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 QCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 243631704 MFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚33 cQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 1846796897280 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚31685 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚11110793 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚31685 ]IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚144611081080320 WGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 13035530240 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚26087 _MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 1846514340352 @8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚895 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚24903680 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚17039360 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 231245312 M@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚81830337 PCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚44919845 OFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚1299 XJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 226592344 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚31930 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 599785472 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 L?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚44919845 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 231245312 TGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚44945932 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚611 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 307460608 P@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 13118524029 ^LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 1755322020427 YIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 13207863296 SEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 384795648 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 G?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚286 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 152061440 WGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 125899Îlð30496 E=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚895 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚2 N@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 248284672 [KDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 13207863296 C;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚895 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 JBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚609 trainer/global_step‚1799 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚613 [IDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 1765212066304 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 O?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 13807648768 TFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 599785472 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚609 Ië˜2Ç- _wandbruntime‚3073  _runtime‚3073"ÀÂÇí&ÈçM SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 582240773 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚286 NADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚44970543 XFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 1765938292224 _runtime‚3073.790803967 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17039360 F6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚895 REDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚36904362 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚26113 YIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 13207863296 WCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚144689575249920 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 XFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 1756047475477 C6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚81848790 N>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 13118524029 ^JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚141570663054976 VHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 226592552 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚895 WCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚146455761840128 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1321 QCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 384795648 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 RBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 13807648768 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚21289221 REDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚21289219 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚33 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 \JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 1756274068029 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚609 trainer/global_step‚1799 YGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 1766169550848 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚31930 WIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 599785472 [GDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚144689592289280 \JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 1766169550848 [MDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚ 282543616 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 599785472 L>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 248297984 PBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 586906624 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 152061440 S?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚143326693491093 TDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 13207863296 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚286 G:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚81874905 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚31685 _KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚146775710002688 aODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 1847799944704 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 599785472 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 S?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚146455513542144 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 WCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚143326937123005 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚609 OADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 248297984 XDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚148623227403776 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 B6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚895 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚17039360 _step‚144 N>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 13568115200 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚17039360 ZFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚141570646015616 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚609 EDeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 231258624 SEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 586906624 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 UEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 12589930496 P@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 13807648768 M=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 13807648768 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚31930 ^JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚144689592289280 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚26115 NADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚36904360 NADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚44944430 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 243631912 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚895 K;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 13568115200 RBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 13035530240 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 231258624 K>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚32405494 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 307447296 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚286 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚594 ZFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚144689575249920 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚11116867 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 OBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚32406088 K>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚81874905 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 NADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚36904362 P8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚36904360 UEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 13035530240 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 320831488·–}ò&ÉìM [GDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚144689592289280 \JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 1766169550848 [MDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚ 282543616 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 599785472 L>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 248297984 PBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 586906624 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 152061440 S?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚143326693491093 TDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 13207863296 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚286 G:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚81874905 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚31685 _KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚146775710002688 aODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 1847799944704 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 599785472 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 S?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚146455513542144 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 WCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚143326937123005 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚609 OADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 248297984 XDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚148623227403776 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 B6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚895 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚17039360 _step‚144 N>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 13568115200 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚17039360 ZFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚141570646015616 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚609 EDeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 231258624 SEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 586906624 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 UEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 12589930496 P@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 13807648768 M=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 13807648768 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚31930 ^JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚144689592289280 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚26115 NADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚36904360 NADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚44944430 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 243631912 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚895 K;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 13568115200 RBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 13035530240 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 231258624 K>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚32405494 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 307447296 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚286 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚594 ZFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚144689575249920 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚11116867 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 OBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚32406088 K>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚81874905 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 NADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚36904362 P8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚36904360 UEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 13035530240 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 320831488 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 582240773 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚286 NADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚44970543 XFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 1765938292224 _runtime‚3073.790803967 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17039360 F6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚895 REDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚36904362 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚26113 YIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 13207863296 WCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚144689575249920 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 XFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 1756047475477 C6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚81848790 N>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 13118524029 ^JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚141570663054976 VHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 226592552 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚895 WCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚146455761840128 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1321 QCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 384795648 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 RBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 13807648768 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚21289221 REDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚21289219 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚33 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 \JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 1756274068029 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚609 trainer/global_step‚1799 YGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 1766169550848 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚31930 WIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 599785472½§æ2Ê- _wandbruntime‚3073  _runtime‚3073þvYËÓ .loss/FRt1/detection_loss‚2.315104944481943 1loss/FBt1/assignment_loss‚0.45141577337282496 2loss/SRqqt2/assignment_loss‚1.6525855344202784 1loss/SRqqt3/assignment_loss‚1.715637313883479 0loss/SRqqt2/detection_loss‚2.085553084479438 1loss/SRqqt3/detection_loss‚2.6685538924321897 .loss/FBt1/detection_loss‚2.632386960917049 epoch‚2 0loss/FRt4/assignment_loss‚1.9794595971588016 0loss/SRqqt1/assignment_loss‚1.70065961626824 0loss/FBt4/assignment_loss‚0.1583927339559427 /loss/FRt3/detection_loss‚2.0221922265697425 1loss/SRqqt1/detection_loss‚2.3385405857115984 1loss/SRqqt4/detection_loss‚0.9507380881225963 1loss/FBt2/assignment_loss‚0.13870568753840315 /loss/FBt2/detection_loss‚3.0979348288642035 _runtime‚3073.795342456 /loss/FRt2/assignment_loss‚3.105518788099289 1loss/FBt3/assignment_loss‚0.13365616189700188 trainer/global_step‚1799 .loss/FRt2/detection_loss‚2.464072456726661 0loss/FRt1/assignment_loss‚3.7800931882213904 0loss/FRt3/assignment_loss‚2.3805309566732955 1loss/SRqqt4/assignment_loss‚1.186461734439336 /loss/FRt4/detection_loss‚1.8384223958665087 /loss/FBt3/detection_loss‚0.6023702595776282 .loss/FBt4/detection_loss‚2.331597229367808 &loss/total_loss‚43.730584039045645 % _timestamp‚1.7885039977154639e+09 _step‚145†¸Y^ÌØ 1loss/FBt3/assignment_loss‚0.13365616189700188 trainer/global_step‚1799 .loss/FRt2/detection_loss‚2.464072456726661 0loss/FRt1/assignment_loss‚3.7800931882213904 0loss/FRt3/assignment_loss‚2.3805309566732955 1loss/SRqqt4/assignment_loss‚1.186461734439336 /loss/FRt4/detection_loss‚1.8384223958665087 /loss/FBt3/detection_loss‚0.6023702595776282 .loss/FBt4/detection_loss‚2.331597229367808 &loss/total_loss‚43.730584039045645 % _timestamp‚1.7885039977154639e+09 _step‚145 .loss/FRt1/detection_loss‚2.315104944481943 1loss/FBt1/assignment_loss‚0.45141577337282496 2loss/SRqqt2/assignment_loss‚1.6525855344202784 1loss/SRqqt3/assignment_loss‚1.715637313883479 0loss/SRqqt2/detection_loss‚2.085553084479438 1loss/SRqqt3/detection_loss‚2.6685538924321897 .loss/FBt1/detection_loss‚2.632386960917049 epoch‚2 0loss/FRt4/assignment_loss‚1.9794595971588016 0loss/SRqqt1/assignment_loss‚1.70065961626824 0loss/FBt4/assignment_loss‚0.1583927339559427 /loss/FRt3/detection_loss‚2.0221922265697425 1loss/SRqqt1/detection_loss‚2.3385405857115984 1loss/SRqqt4/detection_loss‚0.9507380881225963 1loss/FBt2/assignment_loss‚0.13870568753840315 /loss/FBt2/detection_loss‚3.0979348288642035 _runtime‚3073.795342456 /loss/FRt2/assignment_loss‚3.105518788099289‘ÔC³oÍ‚(:… ¾Ïéԃμ gpu.0.gpu‚0.0 gpu.0.memory‚0, gpu.0.memoryAllocated‚16.718826293945312* gpu.0.memoryAllocatedBytes‚ 14361362432 gpu.0.temp‚31.0 gpu.0.powerWatts‚93.345' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚18.669 gpu.0.smClock‚1275 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0®jÿœÎ‚(:‘ ¾ÏéÔçÎß¿ disk.drbd1260.out‚ 0.42578125 disk.sda.out‚ 224.44140625 proc.memory.rssMB‚ 3031.6875* proc.memory.percent‚18.503952026367188 cpu‚25.70173675047557# disk.dm-0.out‚69895.12060546875 network.sent‚ 4.116028e+06) disk./.usagePercent‚78.82867766906492 disk.sda2.out‚0 disk.sda3.in‚ 0.6953125& disk.nvme0n1.out‚69895.12060546875 disk.dm-1.out‚ 224.44140625 disk.drbd1260.in‚0 disk.sda3.out‚ 224.44140625! disk.nvme0n1.in‚ 7375.83203125 disk.dm-1.in‚ 0.6953125 proc.cpu.threads‚17 disk.sda2.in‚0# disk.nvme0c0n1.in‚ 7375.83203125 disk.dm-0.in‚ 7375.83203125 disk.sda1.in‚ 0.1015625 disk.sda1.out‚0 disk.sda.in‚0.796875# memory_percent‚98.9424467086792( proc.memory.availableMB‚ 173.26953125 network.recv‚ 1.032115e+06$ disk./.usageGB‚1174.604263305664( disk.nvme0c0n1.out‚69895.12060546875Þ#`·‘Ï‚(:† ÍÏéÔ¤¹ gpu.0.gpu‚27.0 gpu.0.memory‚2, gpu.0.memoryAllocated‚16.718826293945312* gpu.0.memoryAllocatedBytes‚ 14361362432 gpu.0.temp‚33.0 gpu.0.powerWatts‚114.86' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚22.972 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0p]G@™Ð‚(:Ž ÍÏéԌ disk.sda1.in‚ 0.1015625 disk.sda1.out‚0$ memory_percent‚98.94101619720459 network.sent‚ 4.202592e+06 disk.dm-0.in‚ 7375.83203125 disk.sda.in‚0.796875 disk.sda.out‚ 224.7578125 disk.sda2.out‚0 disk.sda3.in‚ 0.6953125& disk.nvme0n1.out‚70016.63232421875 proc.memory.rssMB‚ 3031.6875# disk.dm-0.out‚70016.63232421875 disk.dm-1.out‚ 224.7578125 disk.sda3.out‚ 224.7578125( disk.nvme0c0n1.out‚70016.63232421875* proc.memory.percent‚18.503952026367188 cpu‚25.7007682354928 proc.cpu.threads‚17 network.recv‚ 1.038077e+06) disk./.usagePercent‚78.82710271071427 disk.drbd1260.in‚0 disk.drbd1260.out‚ 0.42578125 disk.sda2.in‚0# disk.nvme0c0n1.in‚ 7375.83203125! disk.nvme0n1.in‚ 7375.83203125( proc.memory.availableMB‚ 173.50390625$ disk./.usageGB‚1174.580795288086 disk.dm-1.in‚ 0.6953125vú1’Ñ‚(:‡ ÜÏéÔâúû¸ gpu.0.gpu‚3.0 gpu.0.memory‚0, gpu.0.memoryAllocated‚16.718826293945312* gpu.0.memoryAllocatedBytes‚ 14361362432 gpu.0.temp‚34.0 gpu.0.powerWatts‚175.617' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚35.1234 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0‹ý¡„“Ò‚(:ˆ ÜÏéÔ޼ɺ# disk.nvme0c0n1.in‚ 7375.83203125! disk.nvme0n1.in‚ 7375.83203125 disk.dm-1.out‚225.1875 disk.sda.in‚0.796875 cpu‚25.715210093535955 network.sent‚ 4.207929e+06 disk.sda3.in‚ 0.6953125( disk.nvme0c0n1.out‚70554.42431640625& disk.nvme0n1.out‚70554.42431640625 disk.drbd1260.in‚0 disk.drbd1260.out‚ 0.4296875 disk.sda1.in‚ 0.1015625 disk.sda1.out‚0( disk./.usagePercent‚78.8169113067837% disk./.usageGB‚1174.4289360046387 disk.sda2.out‚0 disk.dm-0.in‚ 7375.83203125# disk.dm-0.out‚70554.42431640625 disk.dm-1.in‚ 0.6953125$ memory_percent‚98.94278049468994( proc.memory.availableMB‚ 173.21484375 network.recv‚ 1.042237e+06 disk.sda2.in‚0 disk.sda.out‚225.1875" proc.memory.rssMB‚ 3030.9921875) proc.memory.percent‚18.49970817565918 proc.cpu.threads‚17 disk.sda3.out‚225.1875’Ó‚(:‡ ëÏéÔÿªŽ» gpu.0.gpu‚0.0 gpu.0.memory‚0, gpu.0.memoryAllocated‚16.718826293945312* gpu.0.memoryAllocatedBytes‚ 14361362432 gpu.0.temp‚33.0 gpu.0.powerWatts‚114.748' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚22.9496 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0À#G”Ô‚(:‰ ëÏéÔ²˜ß¾$ memory_percent‚98.85106086730957 network.recv‚ 1.046659e+06) disk./.usagePercent‚78.78216801229688& disk.nvme0n1.out‚70830.12744140625 disk.dm-0.in‚ 7516.15625 disk.sda.in‚0.796875 disk.sda2.out‚0 disk.sda3.out‚ 225.5859375# proc.memory.rssMB‚ 3015.48828125 network.sent‚ 4.213914e+06 disk.nvme0c0n1.in‚ 7516.15625 disk.nvme0n1.in‚ 7516.15625 disk.dm-1.out‚ 225.5859375 disk.drbd1260.in‚0 disk.sda1.in‚ 0.1015625' proc.memory.availableMB‚ 188.2421875) proc.memory.percent‚18.40507984161377% disk./.usageGB‚1173.9112358093262 disk.sda.out‚ 225.5859375 disk.sda2.in‚0 cpu‚25.740641733661953 proc.cpu.threads‚17( disk.nvme0c0n1.out‚70830.12744140625# disk.dm-0.out‚70830.12744140625 disk.dm-1.in‚ 0.6953125 disk.drbd1260.out‚ 0.4296875 disk.sda1.out‚0 disk.sda3.in‚ 0.6953125¿¯¤“Õ‚(:ˆ úÏéÔ²’ÿ¸ gpu.0.gpu‚64.0 gpu.0.memory‚3, gpu.0.memoryAllocated‚16.718826293945312* gpu.0.memoryAllocatedBytes‚ 14361362432 gpu.0.temp‚33.0 gpu.0.powerWatts‚115.012' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚23.0024 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0†Ù–Ö‚(:‹ úÏéÔñš»# proc.memory.rssMB‚ 3091.16015625 disk.sda.out‚ 226.0859375( disk.nvme0c0n1.out‚71120.04833984375# disk.dm-0.out‚71120.04833984375' proc.memory.availableMB‚ 112.9140625 proc.cpu.threads‚17 disk.drbd1260.in‚0 disk.sda1.in‚ 0.1015625 disk.sda2.in‚0 disk.nvme0n1.in‚ 7516.15625& disk.nvme0n1.out‚71120.04833984375 disk.dm-0.in‚ 7516.15625$ memory_percent‚99.31082725524902* proc.memory.percent‚18.866944313049316 network.recv‚ 1.050695e+06% disk./.usageGB‚1173.9112167358398 disk.sda1.out‚0 disk.sda.in‚0.796875 disk.sda3.out‚ 226.0859375 disk.dm-1.in‚ 0.6953125 disk.dm-1.out‚ 226.0859375 cpu‚25.766983791417516 network.sent‚ 4.219892e+06) disk./.usagePercent‚78.78216673225921 disk.drbd1260.out‚ 0.43359375 disk.sda2.out‚0 disk.sda3.in‚ 0.6953125 disk.nvme0c0n1.in‚ 7516.15625È‹Å2×- _wandbruntime‚3138  _runtime‚3138n“.¼oØ›§ 9"Purity/1FRt3SRqqt2FBt/SRqqt_purity‚0.3333333432674408 ("Purity/*FRt2SRqqt3FBt/event_purity‚0 9"Purity/0FRt2SRqqt*FBt/SRqqt_purity‚0.4508831202983856 ("Purity/1FRt3SRqqt0FBt/event_purity‚0 ,&Purity/1FRt3SRqqt4FBt/event_proportion‚0 particle/accuracy_5_of_7‚1 ("Purity/*FRt1SRqqt4FBt/SRqqt_purity‚1 & Purity/*FRt3SRqqt2FBt/FBt_purity‚1 9"Purity/0FRt1SRqqt0FBt/event_purity‚0.4938444197177887 ?&Purity/0FRt2SRqqt1FBt/event_proportion‚0.009261592291295528 :"Purity/1FRt3SRqqt1FBt/SRqqt_purity‚0.43703705072402954 ("Purity/3FRt1SRqqt0FBt/event_purity‚0 & Purity/0FRt4SRqqt*FBt/FBt_purity‚1 9"Purity/*FRt2SRqqt2FBt/SRqqt_purity‚0.4649122655391693 % _timestamp‚1.7885040622031276e+09 >&Purity/*FRt2SRqqt1FBt/event_proportion‚0.02255544438958168 >&Purity/1FRt0SRqqt0FBt/event_proportion‚0.13142640888690948 :"Purity/1FRt0SRqqt2FBt/event_purity‚0.15249039232730865 @&Purity/3FRt*SRqqt0FBt/event_proportion‚0.0071824598126113415 & Purity/3FRt0SRqqt*FBt/FBt_purity‚1 ("Purity/2FRt2SRqqt2FBt/event_purity‚0 @&Purity/2FRt4SRqqt*FBt/event_proportion‚0.0003150201519019902 8"Purity/*FRt2SRqqt1FBt/SRqqt_purity‚0.549780011177063 & Purity/2FRt3SRqqt1FBt/FBt_purity‚1 & Purity/*FRt3SRqqt1FBt/FBt_purity‚1 @&Purity/0FRt4SRqqt*FBt/event_proportion‚0.0005040322430431843 particle/accuracy_4_of_7‚1 ("Purity/3FRt0SRqqt*FBt/event_purity‚0 9"Purity/*FRt*SRqqt1FBt/SRqqt_purity‚0.5592327117919922 >&Purity/0FRt1SRqqt1FBt/event_proportion‚0.05185231938958168 ?&Purity/0FRt2SRqqt0FBt/event_proportion‚0.043283771723508835 =&Purity/1FRt*SRqqt*FBt/event_proportion‚0.4366179406642914 ,&Purity/2FRt1SRqqt4FBt/event_proportion‚0 ,&Purity/4FRt*SRqqt1FBt/event_proportion‚0 & Purity/3FRt1SRqqt2FBt/FBt_purity‚1 7 Purity/*FRt0SRqqt2FBt/FRt_purity‚0.5110097527503967 =&Purity/*FRt1SRqqt1FBt/event_proportion‚0.1001134067773819 ?&Purity/1FRt*SRqqt2FBt/event_proportion‚0.022303426638245583 >&Purity/2FRt*SRqqt0FBt/event_proportion‚0.07579384744167328 8 Purity/2FRt*SRqqt2FBt/FRt_purity‚0.47333332896232605 ?&Purity/2FRt3SRqqt0FBt/event_proportion‚0.003654233¤)Cù878478408 ,&Purity/4FRt4SRqqt4FBt/event_proportion‚0 ("Purity/1FRt1SRqqt3FBt/event_purity‚0 <"Purity/1FRt1SRqqt2FBt/event_purity‚0.0071428571827709675 @&Purity/3FRt3SRqqt0FBt/event_proportion‚0.0005670362734235823 ("Purity/3FRt4SRqqt*FBt/event_purity‚0 9"Purity/0FRt*SRqqt3FBt/SRqqt_purity‚0.8999999761581421 & Purity/1FRt3SRqqt2FBt/FBt_purity‚1 @&Purity/3FRt1SRqqt0FBt/event_proportion‚0.0025201612152159214 :"Purity/*FRt0SRqqt0FBt/event_purity‚0.15092168748378754 7 Purity/*FRt0SRqqt0FBt/FRt_purity‚0.1726149618625641 7 Purity/2FRt*SRqqt0FBt/FRt_purity‚0.1508571207523346 8"Purity/2FRt0SRqqt0FBt/event_purity‚0.024053905159235 & Purity/2FRt1SRqqt1FBt/FBt_purity‚1 <"Purity/0FRt3SRqqt*FBt/event_purity‚0.0071428571827709675 ("Purity/4FRt0SRqqt*FBt/event_purity‚0 ?&Purity/2FRt2SRqqt*FBt/event_proportion‚0.019468246027827263 ,&Purity/0FRt4SRqqt4FBt/event_proportion‚0 =&Purity/*FRt*SRqqt*FBt/event_proportion‚0.9591733813285828 ;"Purity/0FRt1SRqqt2FBt/event_purity‚0.054206352680921555 ,&Purity/4FRt1SRqqt2FBt/event_proportion‚0 ,&Purity/4FRt3SRqqt4FBt/event_proportion‚0 6 Purity/0FRt*SRqqt1FBt/FBt_purity‚0.998268187046051 ,&Purity/3FRt4SRqqt3FBt/event_proportion‚0 ("Purity/0FRt2SRqqt3FBt/SRqqt_purity‚1 :"Purity/*FRt*SRqqt2FBt/event_purity‚0.19206121563911438 ?&Purity/2FRt2SRqqt1FBt/event_proportion‚0.003969253972172737 @&Purity/2FRt4SRqqt0FBt/event_proportion‚0.0003150201519019902 ("Purity/3FRt2SRqqt*FBt/event_purity‚0 :"Purity/0FRt2SRqqt2FBt/SRqqt_purity‚0.42307692766189575 =&Purity/*FRt0SRqqt2FBt/event_proportion‚0.0483240932226181 :"Purity/*FRt1SRqqt*FBt/event_purity‚0.21589429676532745 @&Purity/0FRt2SRqqt2FBt/event_proportion‚0.0010080644860863686 ,&Purity/2FRt3SRqqt4FBt/event_proportion‚0 *"Purity/4FRt*SRqqt*FBt/SRqqt_purity‚0.5 9"Purity/*FRt1SRqqt2FBt/SRqqt_purity‚0.5673344731330872 @&Purity/1FRt0SRqqt4FBt/event_proportion‚6.300403038039804e-05 :"Purity/2FRt0SRqqt1FBt/event_purity‚0.12346561253070831 ,&Purity/2FRt4SRqqt2FBt/event_proportion‚0 8 Purity/3FRt1SRqqt1FBt/FRt_purity‚0.25641024112701416 +jet/accuracy_2_of_2‚0.11253254115581512 particle/accuracy_6_of_6‚1 & Purity/4FRt0SRqqt*FBt/FRt_purity‚0 A&Purity/1FRt1SRqqt3FBt/event_proportion‚0.00025201612152159214 ?&Purity/2FRt3SRqqt*FBt/event_proportion‚0.004032257944345474 jet/accuracy_4_of_7‚0 9"Purity/*FRt1SRqqt3FBt/SRqqt_purity‚0.8333333134651184 7 Purity/0FRt2SRqqt3FBt/FBt_purity‚0.6666666865348816 @&Purity/0FRt4SRqqt1FBt/event_proportion‚6.300403038039804e-05 :"Purity/1FRt*SRqqt1FBt/event_purity‚0.17277376353740692 7 Purity/1FRt3SRqqt1FBt/FRt_purity‚0.3888888955116272 @&Purity/3FRt0SRqqt0FBt/event_proportion‚0.0015120967291295528 A&Purity/3FRt4SRqqt*FBt/event_proportion‚0.00012600806076079607 A&Purity/3FRt4SRqqt0FBt/event_proportion‚0.00012600806076079607 ("Purity/2FRt3SRqqt1FBt/event_purity‚0 ("Purity/1FRt0SRqqt4FBt/event_purity‚0 ("Purity/*FRt0SRqqt4FBt/event_purity‚0 ("Purity/3FRt*SRqqt2FBt/event_purity‚0 & Purity/4FRt2SRqqt*FBt/FRt_purity‚0 9"Purity/*FRt0SRqqt3FBt/event_purity‚0.1369047611951828 ("Purity/2FRt*SRqqt2FBt/event_purity‚0 ("Purity/2FRt1SRqqt2FBt/event_purity‚0 *"Purity/*FRt4SRqqt1FBt/SRqqt_purity‚0.5 & Purity/0FRt2SRqqt2FBt/FBt_purity‚1 >&Purity/0FRt*SRqqt0FBt/event_proportion‚0.21931703388690948 7 Purity/0FRt2SRqqt1FBt/FBt_purity‚0.9953917264938354 7 Purity/1FRt2SRqqt*FBt/FBt_purity‚0.9831901788711548 *jet/accuracy_1_of_4‚0.9053216576576233 ( Purity/4FRt1SRqqt*FBt/FRt_purity‚0.5 ;"Purity/1FRt2SRqqt1FBt/event_purity‚0.006451612804085016 8 Purity/2FRt1SRqqt1FBt/FRt_purity‚0.39268431067466736 ,&Purity/2FRt4SRqqt3FBt/event_proportion‚0 ;"Purity/1FRt1SRqqt1FBt/event_purity‚0.047421231865882874 ,&Purity/1FRt2SRqqt4FBt/event_proportion‚0 ,"Purity/3FRt4SRqqt0FBt/SRqqt_purity‚0.375 9"Purity/0FRt3SRqqt1FBt/SRqqt_purity‚0.3333333432674408 9"Purity/1FRt4SRqqt*FBt/SRqqt_purity‚0.2083333283662796 ("Purity/3FRt0SRqqt2FBt/event_purity‚0 >&Purity/*FRt*SRqqt1FBt/event_proportion‚0.27079132199287415 ,&Purity/*FRt3SRqqt3FBt/event_proportion‚0 @&Purity/1FRt3SRqqt1FBt/event_proportion‚0.0011340725468471646 ("Purity/3FRt1SRqqt*FBt/event_purity‚0 ,&Purity/4FRt0SRqqt2FBt/event_proportion‚0 particle/accuracy_2_of_6‚1 7 Purity/0FRt*SRqqt3FBt/FBt_purity‚0.8868313431739807 8 Purity/1FRt0SRqqt0FBt/FRt_purity‚0.17710725963115692 9"Purity/1FRt3SRqqt0FBt/SRqqt_purity‚0.3567715883255005 9"Purity/3FRt*SRqqt*FBt/SRqqt_purity‚0.5041598081588745 @&Purity/3FRt2SRqqt1FBt/event_proportion‚0.0003780241822823882 7 Purity/*FRt3SRqqt1FBt/FRt_purity‚0.3166666626930237 @&Purity/1FRt*SRqqt3FBt/event_proportion‚0.0013230846961960196 9"Purity/3FRt1SRqqt1FBt/SRqqt_purity‚0.7307692170143127 7 Purity/1FRt0SRqqt2FBt/FRt_purity‚0.5464561581611633 ?&Purity/0FRt3SRqqt*FBt/event_proportion‚0.006741431541740894 :"Purity/1FRt*SRqqt2FBt/event_purity‚0.10331588983535767 7 Purity/1FRt2SRqqt*FBt/FRt_purity‚0.1338045448064804 ?&Purity/2FRt*SRqqt1FBt/event_proportion‚0.021169355139136314 *"Purity/0FRt4SRqqt1FBt/SRqqt_purity‚0.5 8 Purity/2FRt3SRqqt1FBt/FRt_purity‚0.30000001192092896 ("Purity/*FRt1SRqqt4FBt/event_purity‚0 7 Purity/0FRt*SRqqt2FBt/FBt_purity‚0.9856252074241638 ("Purity/0FRt0SRqqt0FBt/event_purity‚1 ?&Purity/0FRt0SRqqt2FBt/event_proportion‚0.029674898833036423 9"Purity/0FRt1SRqqt0FBt/SRqqt_purity‚0.4938444197177887 :"Purity/3FRt1SRqqt0FBt/SRqqt_purity‚0.47083336114883423 7 Purity/3FRt3SRqqt0FBt/FRt_purity‚0.0476190485060215 ,&Purity/*FRt3SRqqt4FBt/event_proportion‚0 :"Purity/0FRt0SRqqt2FBt/event_purity‚0.34515297412872314 9"Purity/0FRt1SRqqt2FBt/SRqqt_purity‚0.4933994710445404 ?&Purity/1FRt2SRqqt1FBt/event_proportion‚0.008946572430431843 +jet/accuracy_3_of_3‚0.03174355998635292 *particle/specificity‚0.949479877948761 ("Purity/1FRt4SRqqt*FBt/event_purity‚0 9"Purity/1FRt4SRqqt0FBt/SRqqt_purity‚0.2083333283662796 >&Purity/0FRt0SRqqt1FBt/event_proportion‚0.08606351166963577 9"Purity/2FRt0SRqqt*FBt/event_purity‚0.0406825989484787 9"Purity/3FRt1SRqqt*FBt/SRqqt_purity‚0.5586420297622681 @&Purity/3FRt3SRqqt*FBt/event_proportion‚0.0005670362734235823 ("Purity/1FRt*SRqqt4FBt/SRqqt_purity‚1 9"Purity/1FRt1SRqqt2FBt/SRqqt_purity‚0.5886054635047913 & Purity/2FRt2SRqqt1FBt/FBt_purity‚1 ,&Purity/3FRt3SRqqt1FBt/event_proportion‚0 particle/accuracy_1_of_4‚1 ("Purity/4FRt*SRqqt0FBt/event_purity‚0 8 Purity/*FRt*SRqqt1FBt/FRt_purity‚0.37070155143737793 ,&Purity/3FRt4SRqqt2FBt/event_proportion‚0 ( Purity/1FRt1SRqqt3FBt/FRt_purity‚0.5 8 Purity/*FRt1SRqqt1FBt/FRt_purity‚0.35375159978866577 9"Purity/1FRt0SRqqt1FBt/event_purity‚0.3090205490589142 ;"Purity/2FRt*SRqqt1FBt/event_purity‚0.045593470335006714 & Purity/2FRt0SRqqt1FBt/FBt_purity‚1 @&Purity/3FRt*SRqqt1FBt/event_proportion‚0.0024571572430431843 8 Purity/3FRt0SRqqt0FBt/FRt_purity‚0.14705882966518402 ,particle/sensitivity‚0.31455308198928833 _step‚146 7 Purity/1FRt*SRqqt1FBt/FRt_purity‚0.3738571107387543 +jet/accuracy_2_of_4‚0.34317338466644287 7 Purity/*FRt2SRqqt2FBt/FBt_purity‚0.9473684430122375 7 Purity/3FRt*SRqqt2FBt/FRt_purity‚0.2666666805744171 =&Purity/0FRt1SRqqt0FBt/event_proportion‚0.1694808453321457 @&Purity/0FRt3SRqqt1FBt/event_proportion‚0.0006300403038039804 ,&Purity/0FRt4SRqqt2FBt/event_proportion‚0 ,&Purity/1FRt2SRqqt3FBt/event_proportion‚0 ("Purity/3FRt*SRqqt*FBt/event_purity‚0 & Purity/1FRt0SRqqt4FBt/FBt_purity‚1 7 Purity/3FRt0SRqqt2FBt/FRt_purity‚0.3333333432674408 7 Purity/*FRt2SRqqt3FBt/FBt_purity‚0.6666666865348816 ("Purity/3FRt*SRqqt0FBt/event_purity‚0 8 Purity/3FRt*SRqqt0FBt/FRt_purity‚0.13373017311096191 ,&Purity/4FRt4SRqqt2FBt/event_proportion‚0 8 Purity/2FRt4SRqqt*FBt/FRt_purity‚0.30000001192092896 & Purity/3FRt4SRqqt0FBt/FRt_purity‚0 8"Purity/*FRt1SRqqt*FBt/SRqqt_purity‚0.521054208278656 +"Purity/0FRt4SRqqt0FBt/SRqqt_purity‚0.25 & Purity/2FRt0SRqqt3FBt/FBt_purity‚1 ("Purity/4FRt*SRqqt*FBt/event_purity‚0 ("Purity/2FRt2SRqqt2FBt/SRqqt_purity‚1 particle/accuracy_2_of_3‚1 7validation_average_jet_accuracy‚0.43256518244743347 7 Purity/3FRt2SRqqt1FBt/FRt_purity‚0.2222222238779068 particle/accuracy_1_of_6‚1 ("Purity/*FRt2SRqqt2FBt/event_purity‚0 7 Purity/*FRt*SRqqt3FBt/FRt_purity‚0.5588235259056091 7 Purity/*FRt2SRqqt1FBt/FBt_purity‚0.9956210851669312 :"Purity/1FRt*SRqqt0FBt/event_purity‚0.07915782183408737 7 Purity/1FRt1SRqqt1FBt/FRt_purity‚0.3446482717990875 :"Purity/2FRt3SRqqt0FBt/SRqqt_purity‚0.34044012427330017 8 Purity/*FRt0SRqqt*FBt/FRt_purity‚0.26280391216278076 8 Purity/2FRt*SRqqt*FBt/FRt_purity‚0.21181133389472961 7 Purity/2FRt0SRqqt2FBt/FBt_purity‚0.9940475821495056 9"Purity/3FRt2SRqqt*FBt/SRqqt_purity‚0.5017361044883728 ,&Purity/4FRt*SRqqt2FBt/event_proportion‚0 ("Purity/*FRt1SRqqt3FBt/event_purity‚0 ("Purity/1FRt3SRqqt2FBt/event_purity‚0 + Purity/1FRt*SRqqt3FBt/FRt_purity‚0.5625 9"Purity/2FRt2SRqqt0FBt/SRqqt_purity‚0.5050190687179565 +jet/accuracy_3_of_4‚0.05882061645388603 7 Purity/*FRt0SRqqt*FBt/FBt_purity‚0.9911262392997742 7 Purity/2FRt*SRqqt*FBt/FBt_purity‚0.9961161017417908 ?&Purity/2FRt2SRqqt0FBt/event_proportion‚0.015435988083481789 ,&Purity/4FRt*SRqqt4FBt/event_proportion‚0 ( Purity/*FRt1SRqqt3FBt/FRt_purity‚0.5 & Purity/2FRt2SRqqt2FBt/FBt_purity‚1 >&Purity/*FRt1SRqqt0FBt/event_proportion‚0.33039313554763794 ?&Purity/1FRt3SRqqt0FBt/event_proportion‚0.007938507944345474 @&Purity/1FRt4SRqqt*FBt/event_proportion‚0.0003780241822823882 jet/accuracy_7_of_7‚0 >&Purity/*FRt3SRqqt0FBt/event_proportion‚0.01827116869390011 7 Purity/0FRt0SRqqt*FBt/FBt_purity‚0.9892858266830444 particle/accuracy_2_of_4‚1 ("Purity/2FRt4SRqqt0FBt/event_purity‚0 jet/accuracy_5_of_6‚0 9"Purity/*FRt3SRqqt2FBt/SRqqt_purity‚0.3333333432674408 ("Purity/0FRt2SRqqt3FBt/event_purity‚0 :"Purity/*FRt*SRqqt1FBt/event_purity‚0.42205241322517395 7 Purity/*FRt0SRqqt2FBt/FBt_purity‚0.9891031384468079 7 Purity/2FRt*SRqqt2FBt/FBt_purity‚0.9911110997200012 ?&Purity/2FRt0SRqqt*FBt/event_proportion‚0.037046369165182114 8 Purity/2FRt0SRqqt*FBt/FRt_purity‚0.24078749120235443 & Purity/2FRt2SRqqt*FBt/FBt_purity‚1 ,&Purity/4FRt2SRqqt1FBt/event_proportion‚0 * Purity/*FRt4SRqqt0FBt/FRt_purity‚0.125 ?&Purity/*FRt*SRqqt3FBt/event_proportion‚0.004473286215215921 7 Purity/0FRt0SRqqt3FBt/FBt_purity‚0.9133334159851074 ,&Purity/4FRt3SRqqt*FBt/event_proportion‚0 jet/accuracy_5_of_5‚0 particle/accuracy_1_of_3‚1 jet/accuracy_3_of_7‚0 & Purity/1FRt3SRqqt2FBt/FRt_purity‚1 ("Purity/4FRt2SRqqt*FBt/SRqqt_purity‚1 :"Purity/*FRt2SRqqt1FBt/event_purity‚0.02797956019639969 7 Purity/0FRt0SRqqt2FBt/FBt_purity‚0.9880027174949646 9"Purity/0FRt1SRqqt*FBt/SRqqt_purity‚0.5083165168762207 9"Purity/2FRt*SRqqt*FBt/SRqqt_purity‚0.5139579772949219 ,&Purity/3FRt2SRqqt4FBt/event_proportion‚0 particle/accuracy_6_of_7‚1 ("Purity/4FRt1SRqqt*FBt/event_purity‚0 :"Purity/*FRt*SRqqt*FBt/event_purity‚0.24976526200771332 ?&Purity/*FRt0SRqqt3FBt/event_proportion‚0.003717237850651145 7 Purity/*FRt0SRqqt3FBt/FBt_purity‚0.9047619700431824 7 Purity/1FRt0SRqqt3FBt/FRt_purity‚0.5357142686843872 =&Purity/1FRt1SRqqt0FBt/event_proportion‚0.1265120953321457 7 Purity/1FRt3SRqqt0FBt/FRt_purity‚0.1218893975019455 ,&Purity/3FRt*SRqqt4FBt/event_proportion‚0 ,&Purity/3FRt0SRqqt3FBt/event_proportion‚0 ,&Purity/0FRt3SRqqt2FBt/event_proportion‚0 :"Purity/1FRt1SRqqt0FBt/event_purity‚0.01242572721093893 >&Purity/2FRt1SRqqt0FBt/event_proportion‚0.03181703761219978 ?&Purity/3FRt1SRqqt*FBt/event_proportion‚0.003717237850651145 ,&Purity/3FRt2SRqqt3FBt/event_proportion‚0 @&Purity/4FRt0SRqqt0FBt/event_proportion‚6.300403038039804e-05 ("Purity/*FRt*SRqqt4FBt/event_purity‚0 9"Purity/0FRt3SRqqt0FBt/SRqqt_purity‚0.3853174149990082 :"Purity/0FRt2SRqqt0FBt/SRqqt_purity‚0.43564850091934204 9"Purity/1FRt3SRqqt*FBt/SRqqt_purity‚0.3651007413864136 ,&Purity/2FRt2SRqqt4FBt/event_proportion‚0 7 Purity/3FRt1SRqqt0FBt/FRt_purity‚0.1249999850988388 ("Purity/*FRt4SRqqt0FBt/event_purity‚0 & Purity/*FRt0SRqqt4FBt/FRt_purity‚1 @&Purity/*FRt1SRqqt3FBt/event_proportion‚0.0006930443341843784 :"Purity/1FRt*SRqqt*FBt/event_purity‚0.10202563554048538 >&Purity/1FRt0SRqqt*FBt/event_proportion‚0.19745463132858276 7 Purity/2FRt0SRqqt*FBt/FBt_purity‚0.9967741966247559 8 Purity/2FRt2SRqqt*FBt/FRt_purity‚0.16146066784858704 jet/accuracy_5_of_7‚0 & Purity/2FRt*SRqqt3FBt/FBt_purity‚1 ) Purity/1FRt1SRqqt4FBt/FBt_purity‚0.75 @&Purity/2FRt2SRqqt2FBt/event_proportion‚6.300403038039804e-05 ) Purity/3FRt1SRqqt2FBt/FRt_purity‚0.25 7 Purity/*FRt2SRqqt*FBt/FBt_purity‚0.9861812591552734 @&Purity/1FRt1SRqqt4FBt/event_proportion‚6.300403038039804e-05 8 Purity/2FRt3SRqqt0FBt/FRt_purity‚0.08852814137935638 0"Purity/*FRt4SRqqt*FBt/SRqqt_purity‚ 0.2421875 jet/accuracy_6_of_7‚0 ) Purity/*FRt1SRqqt4FBt/FBt_purity‚0.75 :"Purity/0FRt2SRqqt1FBt/event_purity‚0.05138249322772026 ?&Purity/1FRt1SRqqt2FBt/event_proportion‚0.006111391354352236 ,&Purity/4FRt3SRqqt3FBt/event_proportion‚0 * Purity/1FRt*SRqqt4FBt/FBt_purity‚0.875 & Purity/1FRt4SRqqt0FBt/FRt_purity‚0 & Purity/*FRt3SRqqt2FBt/FRt_purity‚1 ."Purity/1FRt*SRqqt3FBt/event_purity‚0.09375 8 Purity/1FRt0SRqqt1FBt/FRt_purity‚0.42688214778900146 9"Purity/2FRt1SRqqt1FBt/SRqqt_purity‚0.6427803039550781 particle/accuracy_2_of_2‚1 8 Purity/*FRt3SRqqt0FBt/FRt_purity‚0.09330908954143524 ,&Purity/1FRt4SRqqt2FBt/event_proportion‚0 ,&Purity/2FRt*SRqqt4FBt/event_proportion‚0 ("Purity/2FRt2SRqqt1FBt/event_purity‚0 @&Purity/4FRt0SRqqt*FBt/event_proportion‚6.300403038039804e-05 ,&Purity/4FRt2SRqqt3FBt/event_proportion‚0 @&Purity/2FRt0SRqqt3FBt/event_proportion‚6.300403038039804e-05 +jet/accuracy_2_of_5‚0.47330114245414734 7 Purity/0FRt0SRqqt1FBt/FBt_purity‚0.9992319345474243 ?&Purity/0FRt0SRqqt3FBt/event_proportion‚0.002583165420219302 ?&Purity/1FRt0SRqqt1FBt/event_proportion‚0.049773186445236206 =&Purity/1FRt1SRqqt*FBt/event_proportion‚0.1723790317773819 ("Purity/1FRt3SRqqt1FBt/event_purity‚0 7 Purity/2FRt1SRqqt2FBt/FRt_purity‚0.4714912176132202 particle/accuracy_1_of_5‚1 jet/accuracy_6_of_6‚0 8 Purity/*FRt2SRqqt*FBt/FRt_purity‚0.14368775486946106 @&Purity/*FRt3SRqqt2FBt/event_proportion‚6.300403038039804e-05 ,&Purity/3FRt4SRqqt4FBt/event_proportion‚0 :"Purity/2FRt4SRqqt*FBt/SRqqt_purity‚0.30000001192092896 9"Purity/1FRt2SRqqt2FBt/SRqqt_purity‚0.4236110746860504 ("Purity/1FRt1SRqqt4FBt/SRqqt_purity‚1 ("Purity/4FRt2SRqqt0FBt/event_purity‚0 7 Purity/*FRt0SRqqt1FBt/FBt_purity‚0.9995864033699036 >&Purity/2FRt*SRqqt*FBt/event_proportion‚0.10244455933570862 & Purity/2FRt*SRqqt1FBt/FBt_purity‚1 ,&Purity/2FRt0SRqqt4FBt/event_proportion‚0 8 Purity/2FRt1SRqqt0FBt/FRt_purity‚0.15364539623260498 7 Purity/*FRt*SRqqt4FBt/FBt_purity‚0.8333333134651184 +jet/accuracy_3_of_6‚0.18421052396297455 8"Purity/*FRt1SRqqt0FBt/SRqqt_purity‚0.500934898853302 7 Purity/*FRt1SRqqt2FBt/FRt_purity‚0.4474572241306305 A&Purity/0FRt0SRqqt4FBt/event_proportion‚0.00025201612152159214 :"Purity/2FRt*SRqqt0FBt/event_purity‚0.00918419286608696 ("Purity/3FRt2SRqqt0FBt/event_purity‚0 particle/accuracy_2_of_7‚1 8"Purity/3FRt3SRqqt0FBt/SRqqt_purity‚0.523809552192688 ) Purity/4FRt*SRqqt0FBt/FRt_purity‚0.25 9"Purity/*FRt*SRqqt0FBt/SRqqt_purity‚0.4675922989845276 >&Purity/*FRt0SRqqt1FBt/event_proportion‚0.14591734111309052 :"Purity/*FRt1SRqqt1FBt/event_purity‚0.11066509038209915 9"Purity/0FRt*SRqqt1FBt/SRqqt_purity‚0.5411539673805237 7 Purity/1FRt*SRqqt2FBt/FRt_purity‚0.5215697884559631 6 Purity/3FRt*SRqqt1FBt/FRt_purity‚0.302272766828537 8 Purity/*FRt1SRqqt0FBt/FRt_purity‚0.15396557748317719 6 Purity/0FRt*SRqqt*FBt/FBt_purity‚0.989280641078949 >&Purity/0FRt*SRqqt1FBt/event_proportion‚0.14787046611309052 ,&Purity/0FRt4SRqqt3FBt/event_proportion‚0 :"Purity/1FRt0SRqqt*FBt/event_purity‚0.20826943218708038 @&Purity/2FRt3SRqqt1FBt/event_proportion‚0.0003780241822823882 7 Purity/3FRt0SRqqt1FBt/FRt_purity‚0.4038461446762085 particle/accuracy_3_of_3‚1 7 Purity/1FRt*SRqqt0FBt/FRt_purity‚0.1576627641916275 ,&Purity/4FRt0SRqqt3FBt/event_proportion‚0 ("Purity/*FRt4SRqqt*FBt/event_purity‚0 particle/accuracy_3_of_6‚1 ,&Purity/*FRt4SRqqt4FBt/event_proportion‚0 8"Purity/0FRt1SRqqt1FBt/SRqqt_purity‚0.553488552570343 ,&Purity/0FRt2SRqqt4FBt/event_proportion‚0 ("Purity/2FRt4SRqqt*FBt/event_purity‚0 ("Purity/2FRt0SRqqt3FBt/event_purity‚0 8 Purity/*FRt*SRqqt0FBt/FRt_purity‚0.15404744446277618 ,&Purity/4FRt0SRqqt4FBt/event_proportion‚0 7 Purity/1FRt2SRqqt2FBt/FBt_purity‚0.9097222685813904 *"Purity/1FRt*SRqqt3FBt/SRqqt_purity‚0.5 ("Purity/4FRt1SRqqt0FBt/event_purity‚0 ,&Purity/3FRt4SRqqt1FBt/event_proportion‚0 +"Purity/3FRt1SRqqt2FBt/SRqqt_purity‚0.75 ("Purity/4FRt0SRqqt0FBt/event_purity‚0 8 Purity/*FRt*SRqqt2FBt/FRt_purity‚0.47851860523223877 9"Purity/*FRt3SRqqt0FBt/SRqqt_purity‚0.3676312565803528 9"Purity/*FRt3SRqqt1FBt/SRqqt_purity‚0.3743385672569275 :"Purity/0FRt*SRqqt0FBt/event_purity‚0.39407363533973694 :"Purity/0FRt1SRqqt*FBt/event_purity‚0.40472614765167236 ;"Purity/2FRt*SRqqt*FBt/event_purity‚0.016239898279309273 ?&Purity/2FRt1SRqqt2FBt/event_proportion‚0.001890120911411941 ("Purity/4FRt1SRqqt*FBt/SRqqt_purity‚0 _runtime‚3138.284434494 :"Purity/*FRt*SRqqt*FBt/SRqqt_purity‚0.48824450373649597 :"Purity/0FRt*SRqqt2FBt/SRqqt_purity‚0.48462963104248047 A&Purity/1FRt*SRqqt4FBt/event_proportion‚0.00012600806076079607 :"Purity/1FRt0SRqqt0FBt/event_purity‚0.17710725963115692 :"Purity/*FRt2SRqqt0FBt/SRqqt_purity‚0.44606953859329224 ?&Purity/*FRt3SRqqt*FBt/event_proportion‚0.020476309582591057 7 Purity/0FRt2SRqqt*FBt/FBt_purity‚0.9900153875350952 ("Purity/1FRt2SRqqt0FBt/event_purity‚0 >&Purity/1FRt2SRqqt0FBt/event_proportion‚0.04731602966785431 7 Purity/1FRt2SRqqt1FBt/FBt_purity‚0.9953917264938354 @&Purity/3FRt*SRqqt2FBt/event_proportion‚0.0003150201519019902 ("Purity/4FRt2SRqqt*FBt/event_purity‚0 @&Purity/0FRt2SRqqt3FBt/event_proportion‚6.300403038039804e-05 9"Purity/3FRt*SRqqt1FBt/SRqqt_purity‚0.7777777910232544 *jet/accuracy_2_of_3‚0.2570105195045471 *jet/accuracy_2_of_6‚0.6491227746009827 =&Purity/*FRt2SRqqt*FBt/event_proportion‚0.1332535296678543 :"Purity/0FRt0SRqqt3FBt/event_purity‚0.12666666507720947 =&Purity/0FRt1SRqqt*FBt/event_proportion‚0.2305317521095276 9"Purity/1FRt*SRqqt*FBt/SRqqt_purity‚0.4878331422805786 8 Purity/1FRt1SRqqt0FBt/FRt_purity‚0.15521308779716492 @&Purity/3FRt2SRqqt0FBt/event_proportion‚0.0024571572430431843 ,&Purity/4FRt3SRqqt2FBt/event_proportion‚0 @&Purity/4FRt*SRqqt*FBt/event_proportion‚0.0001890120911411941 9"Purity/*FRt0SRqqt1FBt/event_purity‚0.7033302783966064 ?&Purity/1FRt0SRqqt2FBt/event_proportion‚0.015120967291295528 7 Purity/1FRt1SRqqt2FBt/FRt_purity‚0.4557822644710541 7"Purity/2FRt1SRqqt0FBt/SRqqt_purity‚0.51542729139328 *jet/accuracy_1_of_5‚0.9132077693939209 +"Purity/3FRt2SRqqt1FBt/SRqqt_purity‚0.75 jet/accuracy_2_of_7‚0 @&Purity/3FRt0SRqqt1FBt/event_proportion‚0.0011340725468471646 8 Purity/3FRt2SRqqt0FBt/FRt_purity‚0.14371980726718903 ,&Purity/*FRt4SRqqt2FBt/event_proportion‚0 >&Purity/0FRt0SRqqt0FBt/event_proportion‚0.04082661122083664 9"Purity/0FRt0SRqqt1FBt/event_purity‚0.9992319345474243 ("Purity/1FRt3SRqqt*FBt/event_purity‚0 ,&Purity/4FRt3SRqqt0FBt/event_proportion‚0 jet/accuracy_1_of_7‚1 :"Purity/*FRt2SRqqt*FBt/SRqqt_purity‚0.46504104137420654 & Purity/1FRt3SRqqt*FBt/FBt_purity‚1 7 Purity/3FRt1SRqqt*FBt/FRt_purity‚0.1769547462463379 ,&Purity/*FRt2SRqqt4FBt/event_proportion‚0 9"Purity/0FRt*SRqqt0FBt/SRqqt_purity‚0.4653698205947876 @&Purity/3FRt0SRqqt*FBt/event_proportion‚0.0027091733645647764 particle/accuracy_2_of_5‚1 ( Purity/2FRt2SRqqt2FBt/FRt_purity‚0.5 ,&Purity/2FRt4SRqqt1FBt/event_proportion‚0 ("Purity/1FRt2SRqqt2FBt/event_purity‚0 :"Purity/*FRt0SRqqt*FBt/event_purity‚0.39130041003227234 =&Purity/*FRt1SRqqt*FBt/event_proportion‚0.4482736885547638 ;"Purity/*FRt2SRqqt0FBt/event_purity‚0.024824894964694977 9"Purity/1FRt2SRqqt0FBt/SRqqt_purity‚0.4377133548259735 <"Purity/2FRt1SRqqt*FBt/event_purity‚0.0010405826615169644 8 Purity/2FRt2SRqqt0FBt/FRt_purity‚0.14358215034008026 ,&Purity/3FRt1SRqqt4FBt/event_proportion‚0 ,&Purity/4FRt2SRqqt4FBt/event_proportion‚0 ,&Purity/3FRt0SRqqt4FBt/event_proportion‚0 ,&Purity/4FRt3SRqqt1FBt/event_proportion‚0 +jet/accuracy_3_of_5‚0.09134620428085327 & Purity/*FRt4SRqqt*FBt/FBt_purity‚1 & Purity/0FRt4SRqqt1FBt/FBt_purity‚1 >&Purity/0FRt*SRqqt2FBt/event_proportion‚0.03944052383303642 8 Purity/2FRt0SRqqt0FBt/FRt_purity‚0.16031105816364288 & Purity/3FRt0SRqqt1FBt/FBt_purity‚1 particle/accuracy_3_of_5‚1 * Purity/*FRt4SRqqt*FBt/FRt_purity‚0.125 =&Purity/*FRt*SRqqt0FBt/event_proportion‚0.6160534024238586 ,&Purity/0FRt3SRqqt3FBt/event_proportion‚0 7 Purity/1FRt3SRqqt*FBt/FRt_purity‚0.1557091623544693 ,&Purity/1FRt4SRqqt3FBt/event_proportion‚0 & Purity/3FRt*SRqqt1FBt/FBt_purity‚1 @&Purity/3FRt0SRqqt2FBt/event_proportion‚6.300403038039804e-05 & Purity/3FRt1SRqqt*FBt/FBt_purity‚1 ,&Purity/4FRt0SRqqt1FBt/event_proportion‚0 A&Purity/0FRt1SRqqt3FBt/event_proportion‚0.00044102821266278625 =&Purity/1FRt*SRqqt0FBt/event_proportion‚0.3135710656642914 7 Purity/2FRt0SRqqt2FBt/FRt_purity‚0.4821428656578064 ;"Purity/2FRt1SRqqt0FBt/event_purity‚0.001240694778971374 ("Purity/2FRt2SRqqt*FBt/event_purity‚0 ,&Purity/4FRt1SRqqt1FBt/event_proportion‚0 ,&Purity/4FRt2SRqqt2FBt/event_proportion‚0 'particle/f_score‚0.3961953818798065 :"Purity/*FRt*SRqqt0FBt/event_purity‚0.18183115124702454 & Purity/*FRt3SRqqt*FBt/FBt_purity‚1 >&Purity/1FRt2SRqqt*FBt/event_proportion‚0.05727066472172737 8"Purity/*FRt1SRqqt0FBt/event_purity‚0.258332759141922 9"Purity/2FRt*SRqqt0FBt/SRqqt_purity‚0.4921001195907593 :"Purity/3FRt2SRqqt0FBt/SRqqt_purity‚0.44927534461021423 7 Purity/*FRt2SRqqt2FBt/FRt_purity‚0.3717948794364929 & Purity/3FRt*SRqqt2FBt/FBt_purity‚1 ("Purity/3FRt3SRqqt0FBt/event_purity‚0 epoch‚2 ,&Purity/1FRt3SRqqt3FBt/event_proportion‚0 ?&Purity/2FRt*SRqqt2FBt/event_proportion‚0.005418346729129553 7 Purity/2FRt1SRqqt*FBt/FRt_purity‚0.2144928127527237 & Purity/3FRt0SRqqt2FBt/FBt_purity‚1 ("Purity/0FRt1SRqqt3FBt/event_purity‚0 8 Purity/*FRt2SRqqt0FBt/FRt_purity‚0.12367933243513107 @&Purity/3FRt1SRqqt1FBt/event_proportion‚0.0009450604557059705 particle/accuracy_3_of_4‚1 + Purity/0FRt*SRqqt4FBt/FBt_purity‚0.8125 & Purity/2FRt3SRqqt*FBt/FBt_purity‚1 ?&Purity/2FRt0SRqqt0FBt/event_proportion‚0.024571573361754417 7 Purity/2FRt1SRqqt*FBt/FBt_purity‚0.9964157938957214 & Purity/1FRt4SRqqt*FBt/FRt_purity‚0 ("Purity/4FRt1SRqqt0FBt/SRqqt_purity‚0 :"Purity/0FRt*SRqqt*FBt/SRqqt_purity‚0.48193591833114624 8 Purity/1FRt2SRqqt1FBt/FRt_purity‚0.19577571749687195 8 Purity/2FRt3SRqqt*FBt/FRt_purity‚0.08895239233970642 ,&Purity/3FRt*SRqqt3FBt/event_proportion‚0 particle/accuracy_4_of_5‚1 particle/accuracy_5_of_6‚1 particle/accuracy_3_of_7‚1 @&Purity/*FRt0SRqqt4FBt/event_proportion‚0.0003150201519019902 8 Purity/*FRt3SRqqt*FBt/FRt_purity‚0.12199730426073074 :"Purity/*FRt3SRqqt0FBt/event_purity‚0.00358422938734293 ?&Purity/*FRt3SRqqt1FBt/event_proportion‚0.002142137149348855 ,&Purity/0FRt3SRqqt4FBt/event_proportion‚0 8"Purity/1FRt1SRqqt*FBt/SRqqt_purity‚0.531638503074646 :"Purity/1FRt2SRqqt*FBt/SRqqt_purity‚0.45721402764320374 @&Purity/0FRt*SRqqt3FBt/event_proportion‚0.0030871976632624865 >&Purity/1FRt1SRqqt1FBt/event_proportion‚0.03944052383303642 9"Purity/2FRt*SRqqt1FBt/SRqqt_purity‚0.5850185751914978 ("Purity/3FRt0SRqqt0FBt/event_purity‚0 ("Purity/0FRt4SRqqt*FBt/event_purity‚0 9"Purity/*FRt*SRqqt3FBt/SRqqt_purity‚0.8333333134651184 8 Purity/*FRt*SRqqt*FBt/FRt_purity‚0.21905387938022614 :"Purity/*FRt*SRqqt3FBt/event_purity‚0.09195401519536972 ,&Purity/*FRt4SRqqt3FBt/event_proportion‚0 7 Purity/1FRt*SRqqt*FBt/FBt_purity‚0.9917953610420227 jet/accuracy_4_of_6‚0 7 Purity/*FRt0SRqqt3FBt/FRt_purity‚0.5333333611488342 7 Purity/*FRt1SRqqt*FBt/FBt_purity‚0.9916169047355652 7 Purity/1FRt0SRqqt3FBt/FBt_purity‚0.9285714030265808 9"Purity/1FRt1SRqqt1FBt/SRqqt_purity‚0.5985221266746521 ("Purity/0FRt3SRqqt1FBt/event_purity‚0 ( Purity/2FRt*SRqqt3FBt/FRt_purity‚0.5 ?&Purity/*FRt2SRqqt2FBt/event_proportion‚0.002079133177176118 A&Purity/0FRt4SRqqt0FBt/event_proportion‚0.00044102821266278625 7 Purity/1FRt0SRqqt2FBt/FBt_purity‚0.9874216318130493 9"Purity/1FRt2SRqqt1FBt/SRqqt_purity‚0.5266897082328796 ("Purity/2FRt0SRqqt2FBt/event_purity‚0 *jet/accuracy_1_of_1‚0.4814811646938324 *jet/accuracy_1_of_3‚0.8552227020263672 7 Purity/0FRt1SRqqt*FBt/FBt_purity‚0.9901571869850159 ,&Purity/2FRt3SRqqt3FBt/event_proportion‚0 particle/accuracy_1_of_7‚1 9"Purity/0FRt0SRqqt*FBt/event_purity‚0.8144093751907349 :"Purity/1FRt1SRqqt*FBt/event_purity‚0.01986980438232422 <"Purity/1FRt2SRqqt*FBt/event_purity‚0.0011520737316459417 ,&Purity/2FRt1SRqqt3FBt/event_proportion‚0 @&Purity/4FRt2SRqqt*FBt/event_proportion‚6.300403038039804e-05 @&Purity/3FRt2SRqqt*FBt/event_proportion‚0.0028351815417408943 ."Purity/0FRt4SRqqt*FBt/SRqqt_purity‚0.28125 ("Purity/0FRt4SRqqt1FBt/event_purity‚0 9"Purity/0FRt1SRqqt3FBt/SRqqt_purity‚0.8999999761581421 ) Purity/4FRt*SRqqt*FBt/FRt_purity‚0.25 6 Purity/*FRt*SRqqt*FBt/FBt_purity‚0.990805447101593 @&Purity/*FRt4SRqqt*FBt/event_proportion‚0.0013230846961960196 :"Purity/0FRt*SRqqt2FBt/event_purity‚0.27149394154548645 7 Purity/1FRt*SRqqt*FBt/FRt_purity‚0.2252509444952011 ("Purity/3FRt1SRqqt1FBt/event_purity‚0 ,jet/accuracy_4_of_5‚0.009984639473259449 ("Purity/0FRt0SRqqt4FBt/event_purity‚0 & Purity/1FRt0SRqqt4FBt/FRt_purity‚1 7 Purity/*FRt1SRqqt*FBt/FRt_purity‚0.2090562880039215 9"Purity/0FRt*SRqqt*FBt/event_purity‚0.4712952673435211 ,&Purity/3FRt3SRqqt2FBt/event_proportion‚0 particle/accuracy_1_of_1‚1 ("Purity/1FRt1SRqqt4FBt/event_purity‚0 ("Purity/4FRt2SRqqt0FBt/SRqqt_purity‚1 7 Purity/*FRt0SRqqt1FBt/FRt_purity‚0.4175725281238556 >&Purity/1FRt*SRqqt1FBt/event_proportion‚0.09929435700178146 8 Purity/2FRt*SRqqt1FBt/FRt_purity‚0.37670913338661194 ("Purity/*FRt*SRqqt4FBt/SRqqt_purity‚1 >&Purity/*FRt2SRqqt0FBt/event_proportion‚0.10855595022439957 8 Purity/1FRt1SRqqt*FBt/FRt_purity‚0.20882214605808258 9"Purity/2FRt1SRqqt*FBt/SRqqt_purity‚0.5457957983016968 ,&Purity/4FRt1SRqqt4FBt/event_proportion‚0 ("Purity/3FRt1SRqqt2FBt/event_purity‚0 & Purity/0FRt3SRqqt1FBt/FBt_purity‚1 :"Purity/*FRt2SRqqt*FBt/event_purity‚0.02513105981051922 @&Purity/1FRt2SRqqt2FBt/event_proportion‚0.0010080644860863686 ?&Purity/2FRt0SRqqt2FBt/event_proportion‚0.003465221729129553 8 Purity/3FRt2SRqqt*FBt/FRt_purity‚0.16087962687015533 ,&Purity/4FRt4SRqqt3FBt/event_proportion‚0 <"Purity/0FRt3SRqqt0FBt/event_purity‚0.0071428571827709675 @&Purity/*FRt1SRqqt4FBt/event_proportion‚6.300403038039804e-05 :"Purity/0FRt2SRqqt0FBt/event_purity‚0.06375109404325485 0"Purity/*FRt4SRqqt0FBt/SRqqt_purity‚ 0.2421875 ("Purity/*FRt4SRqqt1FBt/event_purity‚0 & Purity/3FRt2SRqqt*FBt/FBt_purity‚1 >&Purity/*FRt*SRqqt2FBt/event_proportion‚0.06747731566429138 & Purity/1FRt0SRqqt1FBt/FBt_purity‚1 9"Purity/1FRt1SRqqt0FBt/SRqqt_purity‚0.5078604221343994 ,&Purity/1FRt4SRqqt4FBt/event_proportion‚0 9"Purity/2FRt2SRqqt*FBt/SRqqt_purity‚0.5299275517463684 @&Purity/4FRt1SRqqt*FBt/event_proportion‚6.300403038039804e-05 trainer/global_step‚1811 9"Purity/1FRt*SRqqt1FBt/SRqqt_purity‚0.5670551657676697 ,&Purity/4FRt1SRqqt3FBt/event_proportion‚0 :"Purity/2FRt3SRqqt1FBt/SRqqt_purity‚0.20000000298023224 A&Purity/0FRt*SRqqt4FBt/event_proportion‚0.00025201612152159214 6 Purity/1FRt1SRqqt*FBt/FBt_purity‚0.993652880191803 (particle/accuracy‚0.8503233790397644 ,&Purity/2FRt2SRqqt3FBt/event_proportion‚0 ("Purity/3FRt*SRqqt1FBt/event_purity‚0 +jet/accuracy_4_of_4‚0.00247434014454484 & Purity/4FRt2SRqqt0FBt/FRt_purity‚0 7 Purity/0FRt1SRqqt1FBt/FBt_purity‚0.9979166984558105 ?&Purity/0FRt3SRqqt0FBt/event_proportion‚0.006111391354352236 9"Purity/1FRt*SRqqt2FBt/SRqqt_purity‚0.5328981280326843 ,&Purity/2FRt3SRqqt2FBt/event_proportion‚0 ,&Purity/3FRt3SRqqt4FBt/event_proportion‚0 ("Purity/0FRt*SRqqt4FBt/event_purity‚0 particle/accuracy_4_of_6‚1 ("Purity/0FRt4SRqqt0FBt/event_purity‚0 :"Purity/0FRt*SRqqt3FBt/event_purity‚0.09259258955717087 7 Purity/1FRt*SRqqt1FBt/FBt_purity‚0.9988213777542114 7 Purity/1FRt1SRqqt2FBt/FBt_purity‚0.9806547164916992 @&Purity/4FRt*SRqqt0FBt/event_proportion‚0.0001890120911411941 & Purity/1FRt1SRqqt3FBt/FBt_purity‚1 *loss/val_total_loss‚43.738315762282525 7 Purity/*FRt1SRqqt1FBt/FBt_purity‚0.9983752965927124 :"Purity/1FRt*SRqqt0FBt/SRqqt_purity‚0.46393752098083496 @&Purity/1FRt3SRqqt2FBt/event_proportion‚6.300403038039804e-05 7 Purity/2FRt0SRqqt1FBt/FRt_purity‚0.3699338734149933 ("Purity/2FRt3SRqqt0FBt/event_purity‚0 7 Purity/*FRt*SRqqt1FBt/FBt_purity‚0.9986271262168884 >&Purity/0FRt2SRqqt*FBt/event_proportion‚0.05361643061041832 7 Purity/*FRt0SRqqt4FBt/FBt_purity‚0.8500000238418579 9"Purity/0FRt3SRqqt*FBt/SRqqt_purity‚0.3859599232673645 *"Purity/1FRt1SRqqt3FBt/SRqqt_purity‚0.5 ("Purity/2FRt2SRqqt0FBt/event_purity‚0 7 Purity/2FRt2SRqqt1FBt/FRt_purity‚0.2526785731315613 ,&Purity/4FRt*SRqqt3FBt/event_proportion‚0 @&Purity/4FRt2SRqqt0FBt/event_proportion‚6.300403038039804e-05 ("Purity/3FRt2SRqqt1FBt/event_purity‚0 ?&Purity/3FRt*SRqqt*FBt/event_proportion‚0.009954636916518211 + Purity/0FRt0SRqqt4FBt/FBt_purity‚0.8125 :"Purity/2FRt4SRqqt0FBt/SRqqt_purity‚0.30000001192092896 & Purity/1FRt1SRqqt4FBt/FRt_purity‚0 & Purity/4FRt0SRqqt0FBt/FRt_purity‚0 @&Purity/*FRt*SRqqt4FBt/event_proportion‚0.0003780241822823882 @&Purity/2FRt*SRqqt3FBt/event_proportion‚6.300403038039804e-05 ,&Purity/4FRt4SRqqt1FBt/event_proportion‚0 ,"Purity/3FRt4SRqqt*FBt/SRqqt_purity‚0.375 7 Purity/*FRt*SRqqt3FBt/FBt_purity‚0.8975096940994263 @&Purity/*FRt4SRqqt1FBt/event_proportion‚6.300403038039804e-05 9"Purity/2FRt*SRqqt2FBt/SRqqt_purity‚0.7236841917037964 9"Purity/2FRt1SRqqt2FBt/SRqqt_purity‚0.7236841917037964 ("Purity/3FRt0SRqqt1FBt/event_purity‚0 A&Purity/3FRt1SRqqt2FBt/event_proportion‚0.00025201612152159214 @&Purity/4FRt1SRqqt0FBt/event_proportion‚6.300403038039804e-05 & Purity/*FRt1SRqqt4FBt/FRt_purity‚0 6 Purity/1FRt0SRqqt*FBt/FRt_purity‚0.269497275352478 @&Purity/1FRt4SRqqt0FBt/event_proportion‚0.0003780241822823882 ("Purity/2FRt3SRqqt*FBt/event_purity‚0 ,&Purity/2FRt4SRqqt4FBt/event_proportion‚0 particle/accuracy_1_of_2‚1 particle/accuracy_5_of_5‚1 7 Purity/1FRt2SRqqt2FBt/FRt_purity‚0.3611111342906952 +"Purity/3FRt*SRqqt2FBt/SRqqt_purity‚0.75 9"Purity/*FRt1SRqqt1FBt/SRqqt_purity‚0.5767184495925903 @&Purity/*FRt4SRqqt0FBt/event_proportion‚0.0012600806076079607 9"Purity/0FRt*SRqqt1FBt/event_purity‚0.6490389704704285 9"Purity/1FRt0SRqqt3FBt/event_purity‚0.1071428582072258 ,&Purity/3FRt1SRqqt3FBt/event_proportion‚0 *jet/accuracy_1_of_2‚0.7427793145179749 ("Purity/1FRt*SRqqt4FBt/event_purity‚0 ( Purity/1FRt*SRqqt4FBt/FRt_purity‚0.5 7 Purity/*FRt*SRqqt2FBt/FBt_purity‚0.9853708148002625 8 Purity/1FRt2SRqqt0FBt/FRt_purity‚0.11791838705539703 ?&Purity/2FRt1SRqqt1FBt/event_proportion‚0.007875503972172737 & Purity/0FRt3SRqqt*FBt/FBt_purity‚1 ( Purity/2FRt0SRqqt3FBt/FRt_purity‚0.5 ("Purity/3FRt3SRqqt*FBt/event_purity‚0 7 Purity/3FRt3SRqqt*FBt/FRt_purity‚0.0476190485060215 *"Purity/4FRt*SRqqt0FBt/SRqqt_purity‚0.5 >&Purity/*FRt0SRqqt0FBt/event_proportion‚0.15757308900356293 ;"Purity/*FRt1SRqqt2FBt/event_purity‚0.024884795770049095 7 Purity/*FRt1SRqqt2FBt/FBt_purity‚0.9829049110412598 ("Purity/*FRt3SRqqt1FBt/event_purity‚0 *jet/accuracy_1_of_6‚0.9473684430122375 ("Purity/1FRt4SRqqt0FBt/event_purity‚0 ("Purity/*FRt2SRqqt3FBt/5UF¡: SRqqt_purity‚1 8 Purity/*FRt2SRqqt1FBt/FRt_purity‚0.23405133187770844 7 Purity/1FRt*SRqqt2FBt/FBt_purity‚0.9656379818916321 7 Purity/1FRt1SRqqt1FBt/FBt_purity‚0.9988052845001221 ,&Purity/4FRt4SRqqt*FBt/event_proportion‚0 particle/accuracy_7_of_7‚1 6 Purity/0FRt1SRqqt3FBt/FBt_purity‚0.800000011920929 ( Purity/4FRt1SRqqt0FBt/FRt_purity‚0.5 @&Purity/1FRt0SRqqt3FBt/event_proportion‚0.0010710685746744275 9"Purity/*FRt*SRqqt2FBt/SRqqt_purity‚0.5525772571563721 :"Purity/*FRt3SRqqt*FBt/event_purity‚0.00358422938734293 7 Purity/0FRt1SRqqt2FBt/FBt_purity‚0.9810184836387634 7 Purity/1FRt*SRqqt3FBt/FBt_purity‚0.9479167461395264 8"Purity/3FRt*SRqqt0FBt/SRqqt_purity‚0.440532922744751 ,&Purity/3FRt2SRqqt2FBt/event_proportion‚0 & Purity/3FRt2SRqqt1FBt/FBt_purity‚1 >&Purity/0FRt*SRqqt*FBt/event_proportion‚0.40996724367141724 @&Purity/*FRt2SRqqt3FBt/event_proportion‚6.300403038039804e-05 :"Purity/0FRt1SRqqt1FBt/event_purity‚0.18086053431034088 ?&Purity/0FRt1SRqqt2FBt/event_proportion‚0.008757560513913631 :"Purity/0FRt2SRqqt*FBt/event_purity‚0.06152655556797981 7 Purity/1FRt0SRqqt*FBt/FBt_purity‚0.9930731654167175 ?&Purity/1FRt3SRqqt*FBt/event_proportion‚0.009135584346950054 ?&Purity/2FRt0SRqqt1FBt/event_proportion‚0.008946572430431843 =&Purity/*FRt0SRqqt*FBt/event_proportion‚0.3558467626571655 >&Purity/0FRt0SRqqt*FBt/event_proportion‚0.11857359111309052 ,&Purity/1FRt4SRqqt1FBt/event_proportion‚0 9"Purity/2FRt2SRqqt1FBt/SRqqt_purity‚0.5869048237800598 ,&Purity/3FRt3SRqqt3FBt/event_proportion‚0 ,&Purity/4FRt4SRqqt0FBt/event_proportion‚0 particle/accuracy_4_of_4‚1 ( Purity/*FRt*SRqqt4FBt/FRt_purity‚0.5 6 Purity/3FRt0SRqqt*FBt/FRt_purity‚0.252525269985199 >&Purity/2FRt1SRqqt*FBt/event_proportion‚0.04158266261219978 ("Purity/2FRt1SRqqt1FBt/event_purity‚0 7 Purity/2FRt1SRqqt2FBt/FBt_purity‚0.9868420958518982 & Purity/*FRt4SRqqt1FBt/FBt_purity‚1 ("Purity/0FRt2SRqqt2FBt/event_purity‚0 ("Purity/2FRt*SRqqt3FBt/event_purity‚0 ("Purity/*FRt3SRqqt2FBt/event_purity‚0 ?&Purity/*FRt1SRqqt2FBt/event_proportion‚0.017011089250445366 9"Purity/0FRt2SRqqt1FBt/SRqqt_purity‚0.5393753051757812 & Purity/1FRt3SRqqt1FBt/FBt_purity‚1 8 Purity/3FRt*SRqqt*FBt/FRt_purity‚0.18332910537719727 8 Purity/2FRt4SRqqt0FBt/FRt_purity‚0.30000001192092896 & Purity/3FRt4SRqqt*FBt/FRt_purity‚0 7 Purity/*FRt1SRqqt3FBt/FBt_purity‚0.8888888955116272 8"Purity/3FRt3SRqqt*FBt/SRqqt_purity‚0.523809552192688 9"Purity/2FRt3SRqqt*FBt/SRqqt_purity‚0.3173650801181793 ("Purity/3FRt4SRqqt0FBt/event_purity‚0 & Purity/3FRt1SRqqt1FBt/FBt_purity‚1 :"Purity/*FRt0SRqqt2FBt/event_purity‚0.25802281498908997 :"Purity/*FRt3SRqqt*FBt/SRqqt_purity‚0.36791568994522095 ,&Purity/0FRt1SRqqt4FBt/event_proportion‚0 & Purity/3FRt*SRqqt*FBt/FBt_purity‚1Fž)¸tÙ § & Purity/*FRt4SRqqt1FBt/FBt_purity‚1 ("Purity/0FRt2SRqqt2FBt/event_purity‚0 ("Purity/2FRt*SRqqt3FBt/event_purity‚0 ("Purity/*FRt3SRqqt2FBt/event_purity‚0 ?&Purity/*FRt1SRqqt2FBt/event_proportion‚0.017011089250445366 9"Purity/0FRt2SRqqt1FBt/SRqqt_purity‚0.5393753051757812 & Purity/1FRt3SRqqt1FBt/FBt_purity‚1 8 Purity/3FRt*SRqqt*FBt/FRt_purity‚0.18332910537719727 8 Purity/2FRt4SRqqt0FBt/FRt_purity‚0.30000001192092896 & Purity/3FRt4SRqqt*FBt/FRt_purity‚0 7 Purity/*FRt1SRqqt3FBt/FBt_purity‚0.8888888955116272 8"Purity/3FRt3SRqqt*FBt/SRqqt_purity‚0.523809552192688 9"Purity/2FRt3SRqqt*FBt/SRqqt_purity‚0.3173650801181793 ("Purity/3FRt4SRqqt0FBt/event_purity‚0 & Purity/3FRt1SRqqt1FBt/FBt_purity‚1 :"Purity/*FRt0SRqqt2FBt/event_purity‚0.25802281498908997 :"Purity/*FRt3SRqqt*FBt/SRqqt_purity‚0.36791568994522095 ,&Purity/0FRt1SRqqt4FBt/event_proportion‚0 & Purity/3FRt*SRqqt*FBt/FBt_purity‚1 9"Purity/1FRt3SRqqt2FBt/SRqqt_purity‚0.3333333432674408 ("Purity/*FRt2SRqqt3FBt/event_purity‚0 9"Purity/0FRt2SRqqt*FBt/SRqqt_purity‚0.4508831202983856 ("Purity/1FRt3SRqqt0FBt/event_purity‚0 ,&Purity/1FRt3SRqqt4FBt/event_proportion‚0 particle/accuracy_5_of_7‚1 ("Purity/*FRt1SRqqt4FBt/SRqqt_purity‚1 & Purity/*FRt3SRqqt2FBt/FBt_purity‚1 9"Purity/0FRt1SRqqt0FBt/event_purity‚0.4938444197177887 ?&Purity/0FRt2SRqqt1FBt/event_proportion‚0.009261592291295528 :"Purity/1FRt3SRqqt1FBt/SRqqt_purity‚0.43703705072402954 ("Purity/3FRt1SRqqt0FBt/event_purity‚0 & Purity/0FRt4SRqqt*FBt/FBt_purity‚1 9"Purity/*FRt2SRqqt2FBt/SRqqt_purity‚0.4649122655391693 % _timestamp‚1.7885040622031276e+09 >&Purity/*FRt2SRqqt1FBt/event_proportion‚0.02255544438958168 >&Purity/1FRt0SRqqt0FBt/event_proportion‚0.13142640888690948 :"Purity/1FRt0SRqqt2FBt/event_purity‚0.15249039232730865 @&Purity/3FRt*SRqqt0FBt/event_proportion‚0.0071824598126113415 & Purity/3FRt0SRqqt*FBt/FBt_purity‚1 ("Purity/2FRt2SRqqt2FBt/event_purity‚0 @&Purity/2FRt4SRqqt*FBt/event_proportion‚0.0003150201519019902 8"Purity/*FRt2SRqqt1FBt/SRqqt_purity‚0.549780011177063 & Purity/2FRt3SRqqt1FBt/FBt_purity‚1 & Purity/*FRt3SRqqt1FBt/FBt_purity‚1 @&Purity/0FRt4SRqqt*FBt/event_proportion‚0.0005040322430431843 particle/accuracy_4_of_7‚1 ("Purity/3FRt0SRqqt*FBt/event_purity‚0 9"Purity/*FRt*SRqqt1FBt/SRqqt_purity‚0.5592327117919922 >&Purity/0FRt1SRqqt1FBt/event_proportion‚0.05185231938958168 ?&Purity/0FRt2SRqqt0FBt/event_proportion‚0.043283771723508835 =&Purity/1FRt*SRqqt*FBt/event_proportion‚0.4366179406642914 ,&Purity/2FRt1SRqqt4FBt/event_proportion‚0 ,&Purity/4FRt*SRqqt1FBt/event_proportion‚0 & Purity/3FRt1SRqqt2FBt/FBt_purity‚1 7 Purity/*FRt0SRqqt2FBt/FRt_purity‚0.5110097527503967 =&Purity/*FRt1SRqqt1FBt/event_proportion‚0.1001134067773819 ?&Purity/1FRt*SRqqt2FBt/event_proportion‚0.022303426638245583 >&Purity/2FRt*SRqqt0FBt/event_proportion‚0.07579384744167328 8 Purity/2FRt*SRqqt2FBt/FRt_purity‚0.47333332896232605 ?&Purity/2FRt3SRqqt0FBt/event_proportion‚0.003654233878478408 ,&Purity/4FRt4SRqqt4FBt/event_proportion‚0 ("Purity/1FRt1SRqqt3FBt/event_purity‚0 <"Purity/1FRt1SRqqt2FBt/event_purity‚0.0071428571827709675 @&Purity/3FRt3SRqqt0FBt/event_proportion‚0.0005670362734235823 ("Purity/3FRt4SRqqt*FBt/event_purity‚0 9"Purity/0FRt*SRqqt3FBt/SRqqt_purity‚0.8999999761581421 & Purity/1FRt3SRqqt2FBt/FBt_purity‚1 @&Purity/3FRt1SRqqt0FBt/event_proportion‚0.0025201612152159214 :"Purity/*FRt0SRqqt0FBt/event_purity‚0.15092168748378754 7 Purity/*FRt0SRqqt0FBt/FRt_purity‚0.1726149618625641 7 Purity/2FRt*SRqqt0FBt/FRt_purity‚0.1508571207523346 8"Purity/2FRt0SRqqt0FBt/event_purity‚0.024053905159235 & Purity/2FRt1SRqqt1FBt/FBt_purity‚1 <"Purity/0FRt3SRqqt*FBt/event_purity‚0.0071428571827709675 ("Purity/4FRt0SRqqt*FBt/event_purity‚0 ?&Purity/2FRt2SRqqt*FBt/event_proportion‚0.019468246027827263 ,&Purity/0FRt4SRqqt4FBt/event_proportion‚0 =&Purity/*FRt*SRqqt*FBt/event_proportion‚0.9591733813285828 ;"Purity/0FRt1SRqqt2FBt/event_purity‚0.054206352680921555 ,&Purity/4FRt1SRqqt2FBt/event_proportion‚0 ,&Purity/4FRt3SRqqt4FBt/event_proportion‚0 6 Purity/0FRt*SRqqt1FBt/FBt_purity‚0.998268187046051 ,&Purity/3FRt4SRqqt3FBt/event_proportion‚0 ("Purity/0FRt2SRqqt3FBt/SRqqt_purity‚1 :"Purity/*FRt*SRqqt2FBt/event_purity‚0.19206121563911438 ?&Purity/2FRt2SRqqt1FBt/event_proportion‚0.003969253972172737 @&Purity/2FRt4SRqqt0FBt/event_proportion‚0.0003150201519019902 ("Purity/3FRt2SRqqt*FBt/event_purity‚0 :"Purity/0FRt2SRqqt2FBt/SRqqt_purity‚0.42307692766189575 =&Purity/*FRt0SRqqt2FBt/event_proportion‚0.0483240932226181 :"Purity/*FRt1SRqqt*FBt/event_purity‚0.21589429676532745 @&Purity/0FRt2SRqqt2FBt/event_proportion‚0.0010080644860863686 ,&Purity/2FRt3SRqqt4FBt/event_proportion‚0 *"Purity/4FRt*SRqqt*FBt/SRqqt_purity‚0.5 9"Purity/*FRt1SRqqt2FBt/SRqqt_purity‚0.5673344731330872 @&Purity/1FRt0SRqqt4FBt/event_proportion‚6.300403038039804e-05 :"Purity/2FRt0SRqqt1FBt/event_purity‚0.12346561253070831 ,&Purity/2FRt4SRqqt2FBt/event_proportion‚0 8 Purity/3FRt1SRqqt1FBt/FRt_purity‚0.25641024112701416 +jet/accuracy_2_of_2‚0.11253254115581512 particle/accuracy_6_of_6‚1 & Purity/4FRt0SRqqt*FBt/FRt_purity‚0 A&Purity/1FRt1SRqqt3FBt/event_proportion‚0.00025201612152159214 ?&Purity/2FRt3SRqqt*FBt/event_proportion‚0.004032257944345474 jet/accuracy_4_of_7‚0 9"Purity/*FRt1SRqqt3FBt/SRqqt_purity‚0.8333333134651184 7 Purity/0FRt2SRqqt3FBt/FBt_purity‚0.6666666865348816 @&Purity/0FRt4SRqqt1FBt/event_proportion‚6.300403038039804e-05 :"Purity/1FRt*SRqqt1FBt/event_purity‚0.17277376353740692 7 Purity/1FRt3SRqqt1FBt/FRt_purity‚0.3888888955116272 @&Purity/3FRt0SRqqt0FBt/event_proportion‚0.0015120967291295528 A&Purity/3FRt4SRqqt*FBt/event_proportion‚0.00012600806076079607 A&Purity/3FRt4SRqqt0FBt/event_proportion‚0.00012600806076079607 ("Purity/2FRt3SRqqt1FBt/event_purity‚0 ("Purity/1FRt0SRqqt4FBt/event_purity‚0 ("Purity/*FRt0SRqqt4FBt/event_purity‚0 ("Purity/3FRt*SRqqt2FBt/event_purity‚0 & Purity/4FRt2SRqqt*FBt/FRt_purity‚0 9"Purity/*FRt0SRqqt3FBt/event_purity‚0.1369047611951828 ("Purity/2FRt*SRqqt2FBt/event_purity‚0 ("Purity/2FRt1SRqqt2FBt/event_purity‚0 *"Purity/*FRt4SRqqt1FBt/SRqqt_purity‚0.5 & Purity/0FRt2SRqqt2FBt/FBt_purity‚1 >&Purity/0FRt*SRqqt0FBt/event_proportion‚0.21931703388690948 7 Purity/0FRt2SRqqt1FBt/FBt_purity‚0.9953917264938354 7 Purity/1FRt2SRqqt*FBt/FBt_purity‚0.9831901788711548 *jet/accuracy_1_of_4‚0.9053216576576233 ( Purity/4FRt1SRqqt*FBt/FRt_purity‚0.5 ;"Purity/1FRt2SRqqt1FBt/event_purity‚0.006451612804085016 8 Purity/2FRt1SRqqt1FBt/FRt_purity‚0.39268431067466736 ,&Purity/2FRt4SRqqt3FBt/event_proportion‚0 ;"Purity/1FRt1SRqqt1FBt/event_purity‚0.047421231865882874 ,&Purity/1FRt2SRqqt4FBt/event_proportion‚0 ,"Purity/3FRt4SRqqt0FBt/SRqqt_purity‚0.375 9"Purity/0FRt3SRqqt1FBt/SRqqt_purity‚0.3333333432674408 9"Purity/1FRt4SRqqt*FBt/SRqqt_purity‚0.2083333283662796 ("Purity/3FRt0SRqqt2FBt/event_purity‚0 >&Purity/*FRt*SRqqt1FBt/event_proportion‚0.27079132199287415 ,&Purity/*FRt3SRqqt3FBt/event_proportion‚0 @&Purity/1FRt3SRqqt1FBt/event_proportion‚0.0011340725468471646 ("Purity/3FRt1SRqqt*FBt/event_purity‚0 ,&Purity/4FRt0SRqqt2FBt/event_proportion‚0 particle/accuracy_2_of_6‚1 7 Purity/0FRt*SRqqt3FBt/FBt_purity‚0.8868313431739807 8 Purity/1FRt0SRqqt0FBt/FRt_purity‚0.17710725963115692 9"Purity/1FRt3SRqqt0FBt/SRqqt_purity‚0.3567715883255005 9"Purity/3FRt*SRqqt*FBt/SRqqt_purity‚0.5041598081588745 @&Purity/3FRt2SRqqt1FBt/event_proportion‚0.0003780241822823882 7 Purity/*FRt3SRqqt1FBt/FRt_purity‚0.3166666626930237 @&Purity/1FRt*SRqqt3FBt/event_proportion‚0.0013230846961960196 9"Purity/3FRt1SRqqt1FBt/SRqqt_purity‚0.7307692170143127 7 Purity/1FRt0SRqqt2FBt/FRt_purity‚0.5464561581611633 ?&Purity/0FRt3SRqqt*FBt/event_proportion‚0.006741431541740894 :"Purity/1FRt*SRqqt2FBt/event_purity‚0.10331588983535767 7 Purity/1FRt2SRqqt*FBt/FRt_purity‚0.1338045448064804 ?&Purity/2FRt*SRqqt1FBt/event_proportion‚0.021169355139136314 *"Purity/0FRt4SRqqt1FBt/SRqqt_purity‚0.5 8 Purity/2FRt3SRqqt1FBt/FRt_purity‚0.30000001192092896 ("Purity/*FRt1SRqqt4FBt/event_purity‚0 7 Purity/0FRt*SRqqt2FBt/FBt_purity‚0.9856252074241638 ("Purity/0FRt0SRqqt0FBt/event_purity‚1 ?&Purity/0FRt0SRqqt2FBt/event_proportion‚0.029674898833036423 9"Purity/0FRt1SRqqt0FBt/SRqqt_purity‚0.4938444197177887 :"Purity/3FRt1SRqqt0FBt/SRqqt_purity‚0.47083336114883423 7 Purity/3FRt3SRqqt0FBt/FRt_purity‚0.0476190485060215 ,&Purity/*FRt3SRqqt4FBt/event_proportion‚0 :"Purity/0FRt0SRqqt2FBt/event_purity‚0.34515297412872314 9"Purity/0FRt1SRqqt2FBt/SRqqt_purity‚0.4933994710445404 ?&Purity/1FRt2SRqqt1FBt/event_proportion‚0.008946572430431843 +jet/accuracy_3_of_3‚0.03174355998635292 *particle/specificity‚0.949479877948761 ("Purity/1FRt4SRqqt*FBt/event_purity‚0 9"Purity/1FRt4SRqqt0FBt/SRqqt_purity‚0.2083333283662796 >&Purity/0FRt0SRqqt1FBt/event_proportion‚0.08606351166963577 9"Purity/2FRt0SRqqt*FBt/event_purity‚0.0406825989484787 9"Purity/3FRt1SRqqt*FBt/SRqqt_purity‚0.5586420297622681 @&Purity/3FRt3SRqqt*FBt/event_proportion‚0.0005670362734235823 ("Purity/1FRt*SRqqt4FBt/SRqqt_purity‚1 9"Purity/1FRt1SRqqt2FBt/SRqqt_purity‚0.5886054635047913 & Purity/2FRt2SRqqt1FBt/FBt_purity‚1 ,&Purity/3FRt3SRqqt1FBt/event_proportion‚0 particle/accuracy_1_of_4‚1 ("Purity/4FRt*SRqqt0FBt/event_purity‚0 8 Purity/*FRt*SRqqt1FBt/FRt_purity‚0.37070155143737793 ,&Purity/3FRt4SRqqt2FBt/event_proportion‚0 ( Purity/1FRt1SRqqt3FBt/FRt_purity‚0.5 8 Purity/*FRt1SRqqt1FBt/FRt_purity‚0.35375159978866577 9"Purity/1FRt0SRqqt1FBt/event_purity‚0.3090205490589142 ;"Purity/2FRt*SRqqt1FBt/event_purity‚0.045593470335006714 & Purity/2FRt0SRqqt1FBt/FBt_purity‚1 @&Purity/3FRt*SRqqt1FBt/event_proportion‚0.0024571572430431843 8 Purity/3FRt0SRqqt0FBt/FRt_purity‚0.14705882966518402 ,particle/sensitivity‚0.31455308198928833 _step‚146 7 Purity/1FRt*SRqqt1FBt/FRt_purity‚0.3738571107387543 +jet/accuracy_2_of_4‚0.34317338466644287 7 Purity/*FRt2SRqqt2FBt/FBt_purity‚0.9473684430122375 7 Purity/3FRt*SRqqt2FBt/FRt_purity‚0.2666666805744171 =&Purity/0FRt1SRqqt0FBt/event_proportion‚0.1694808453321457 @&Purity/0FRt3SRqqt1FBt/event_proportion‚0.0006300403038039804 ,&Purity/0FRt4SRqqt2FBt/event_proportion‚0 ,&Purity/1FRt2SRqqt3FBt/event_proportion‚0 ("Purity/3FRt*SRqqt*FBt/event_purity‚0 & Purity/1FRt0SRqqt4FBt/FBt_purity‚1 7 Purity/3FRt0SRqqt2FBt/FRt_purity‚0.3333333432674408 7 Purity/*FRt2SRqqt3FBt/FBt_purity‚0.6666666865348816 ("Purity/3FRt*SRqqt0FBt/event_purity‚0 8 Purity/3FRt*SRqqt0FBt/FRt_purity‚0.13373017311096191 ,&Purity/4FRt4SRqqt2FBt/event_proportion‚0 8 Purity/2FRt4SRqqt*FBt/FRt_purity‚0.30000001192092896 & Purity/3FRt4SRqqt0FBt/FRt_purity‚0 8"Purity/*FRt1SRqqt*FBt/SRqqt_purity‚0.521054208278656 +"Purity/0FRt4SRqqt0FBt/SRqqt_purity‚0.25 & Purity/2FRt0SRqqt3FBt/FBt_purity‚1 ("Purity/4FRt*SRqqt*FBt/event_purity‚0 ("Purity/2FRt2SRqqt2FBt/SRqqt_purity‚1 particle/accuracy_2_of_3‚1 7validation_average_jet_accuracy‚0.43256518244743347 7 Purity/3FRt2SRqqt1FBt/FRt_purity‚0.2222222238779068 particle/accuracy_1_of_6‚1 ("Purity/*FRt2SRqqt2FBt/event_purity‚0 7 Purity/*FRt*SRqqt3FBt/FRt_purity‚0.5588235259056091 7 Purity/*FRt2SRqqt1FBt/FBt_purity‚0.9956210851669312 :"Purity/1FRt*SRqqt0FBt/event_purity‚0.07915782183408737 7 Purity/1FRt1SRqqt1FBt/FRt_purity‚0.3446482717990875 :"Purity/2FRt3SRqqt0FBt/SRqqt_purity‚0.34044012427330017 8 Purity/*FRt0SRqqt*FBt/FRt_purity‚0.26280391216278076 8 Purity/2FRt*SRqqt*FBt/FRt_purity‚0.21181133389472961 7 Purity/2FRt0SRqqt2FBt/FBt_purity‚0.9940475821495056 9"Purity/3FRt2SRqqt*FBt/SRqqt_purity‚0.5017361044883728 ,&Purity/4FRt*SRqqt2FBt/event_proportion‚0 ("Purity/*FRt1SRqqt3FBt/event_purity‚0 ("Purity/1FRt3SRqqt2FBt/event_purity‚0 + Purity/1FRt*SRqqt3FBt/FRt_purity‚0.5625 9"Purity/2FRt2SRqqt0FBt/SRqqt_purity‚0.5050190687179565 +jet/accuracy_3_of_4‚0.05882061645388603 7 Purity/*FRt0SRqqt*FBt/FBt_purity‚0.9911262392997742 7 Purity/2FRt*SRqqt*FBt/FBt_purity‚0.9961161017417908 ?&Purity/2FRt2SRqqt0FBt/event_proportion‚0.015435988083481789 ,&Purity/4FRt*SRqqt4FBt/event_proportion‚0 ( Purity/*FRt1SRqqt3FBt/FRt_purity‚0.5 & Purity/2FRt2SRqqt2FBt/FBt_purity‚1 >&Purity/*FRt1SRqqt0FBt/event_proportion‚0.33039313554763794 ?&Purity/1FRt3SRqqt0FBt/event_proportion‚0.007938507944345474 @&Purity/1FRt4SRqqt*FBt/event_proportion‚0.0003780241822823882 jet/accuracy_7_of_7‚0 >&Purity/*FRt3SRqqt0FBt/event_proportion‚0.01827116869390011 7 Purity/0FRt0SRqqt*FBt/FBt_purity‚0.9892858266830444 particle/accuracy_2_of_4‚1 ("Purity/2FRt4SRqqt0FBt/event_purity‚0 jet/accuracy_5_of_6‚0 9"Purity/*FRt3SRqqt2FBt/SRqqt_purity‚0.3333333432674408 ("Purity/0FRt2SRqqt3FBt/event_purity‚0 :"Purity/*FRt*SRqqt1FBt/event_purity‚0.42205241322517395 7 Purity/*FRt0SRqqt2FBt/FBt_purity‚0.9891031384468079 7 Purity/2FRt*SRqqt2FBt/FBt_purity‚0.9911110997200012 ?&Purity/2FRt0SRqqt*FBt/event_proportion‚0.037046369165182114 8 Purity/2FRt0SRqqt*FBt/FRt_purity‚0.24078749120235443 & Purity/2FRt2SRqqt*FBt/FBt_purity‚1 ,&Purity/4FRt2SRqqt1FBt/event_proportion‚0 * Purity/*FRt4SRqqt0FBt/FRt_purity‚0.125 ?&Purity/*FRt*SRqqt3FBt/event_proportion‚0.004473286215215921 7 Purity/0FRt0SRqqt3FBt/FBt_purity‚0.9133334159851074 ,&Purity/4FRt3SRqqt*FBt/event_proportion‚0 jet/accuracy_5_of_5‚0 particle/accuracy_1_of_3‚1 jet/accuracy_3_of_7‚0 & Purity/1FRt3SRqqt2FBt/FRt_purity‚1 ("Purity/4FRt2SRqqt*FBt/SRqqt_purity‚1 :"Purity/*FRt2SRqqt1FBt/event_purity‚0.02797956019639969 7 Purity/0FRt0SRqqt2FBt/FBt_purity‚0.9880027174949646 9"Purity/0FRt1SRqqt*FBt/SRqqt_purity‚0.5083165168762207 9"Purity/2FRt*SRqqt*FBt/SRqqt_purity‚0.5139579772949219 ,&Purity/3FRt2SRqqt4FBt/event_proportion‚0 particle/accuracy_6_of_7‚1 ("Purity/4FRt1SRqqt*FBt/event_purity‚0 :"Purity/*FRt*SRqqt*FBt/event_purity‚0.24976526200771332 ?&Purity/*FRt0SRqqt3FBt/event_proportion‚0.003717237850651145 7 Purity/*FRt0SRqqt3FBt/FBt_purity‚0.9047619700431824 7 Purity/1FRt0SRqqt3FBt/FRt_purity‚0.5357142686843872 =&Purity/1FRt1SRqqt0FBt/event_proportion‚0.1265120953321457 7 Purity/1FRt3SRqqt0FBt/FRt_purity‚0.1218893975019455 ,&Purity/3FRt*SRqqt4FBt/event_proportion‚0 ,&Purity/3FRt0SRqqt3FBt/event_proportion‚0 ,&Purity/0FRt3SRqqt2FBt/event_proportion‚0 :"Purity/1FRt1SRqqt0FBt/event_purity‚0.01242572721093893 >&Purity/2FRt1SRqqt0FBt/event_proportion‚0.03181703761219978 ?&Purity/3FRt1SRqqt*FBt/event_proportion‚0.003717237850651145 ,&Purity/3FRt2SRqqt3FBt/event_proportion‚0 @&Purity/4FRt0SRqqt0FBt/event_proportion‚6.300403038039804e-05 ("Purity/*FRt*SRqqt4FBt/event_purity‚0 9"Purity/0FRt3SRqqt0FBt/SRqqt_purity‚0.3853174149990082 :"Purity/0FRt2SRqqt0FBt/SRqqt_purity‚0.43564850091934204 9"Purity/1FRt3SRqqt*FBt/SRqqt_purity‚0.3651007413864136 ,&Purity/2FRt2SRqqt4FBt/event_proportion‚0 7 Purity/3FRt1SRqqt0FBt/FRt_purity‚0.1249999850988388 ("Purity/*FRt4SRqqt0FBt/event_purity‚0 & Purity/*FRt0SRqqt4FBt/FRt_purity‚1 @&Purity/*FRt1SRqqt3FBt/event_proportion‚0.0006930443341843784 :"Purity/1FRt*SRqqt*FBt/event_purity‚0.10202563554048538 >&Purity/1FRt0SRqqt*FBt/event_proportion‚0.19745463132858276 7 Purity/2FRt0SRqqt*FBt/FBt_purity‚0.9967741966247559 8 Purity/2FRt2SRqqt*FBt/FRt_purity‚0.16146066784858704 jet/accuracy_5_of_7‚0 & Purity/2FRt*SRqqt3FBt/FBt_purity‚1 ) Purity/1FRt1SRqqt4FBt/FBt_purity‚0.75 @&Purity/2FRt2SRqqt2FBt/event_proportion‚6.300403038039804e-05 ) Purity/3FRt1SRqqt2FBt/FRt_purity‚0.25 7 Purity/*FRt2SRqqt*FBt/FBt_purity‚0.9861812591552734 @&Purity/1FRt1SRqqt4FBt/event_proportion‚6.300403038039804e-05 8 Purity/2FRt3SRqqt0FBt/FRt_purity‚0.08852814137935638 0"Purity/*FRt4SRqqt*FBt/SRqqt_purity‚ 0.2421875 jet/accuracy_6_of_7‚0 ) Purity/*FRt1SRqqt4FBt/FBt_purity‚0.75 :"Purity/0FRt2SRqqt1FBt/event_purity‚0.05138249322772026 ?&Purity/1FRt1SRqqt2FBt/event_proportion‚0.006111391354352236 ,&Purity/4FRt3SRqqt3FBt/event_proportion‚0 * Purity/1FRt*SRqqt4FBt/FBt_purity‚0.875 & Purity/1FRt4SRqqt0FBt/FRt_purity‚0 & Purity/*FRt3SRqqt2FBt/FRt_purity‚1 ."Purity/1FRt*SRqqt3FBt/event_purity‚0.09375 8 Purity/1FRt0SRqqt1FBt/FRt_purity‚0.42688214778900146 9"Purity/2FRt1SRqqt1FBt/SRqqt_purity‚0.6427803039550781 particle/accuracy_2_of_2‚1 8 Purity/*FRt3SRqqt0FBt/FRt_purity‚0.09330908954143524 ,&Purity/1FRt4SRqqt2FBt/event_proportion‚0 ,&Purity/2FRt*SRqqt4FBt/event_proportion‚0 ("Purity/2FRt2SRqqt1FBt/event_purity‚0 @&Purity/4FRt0SRqqt*FBt/event_proportion‚6.300403038039804e-05 ,&Purity/4FRt2SRqqt3FBt/event_proportion‚0 @&Purity/2FRt0SRqqt3FBt/event_proportion‚6.300403038039804e-05 +jet/accuracy_2_of_5‚0.47330114245414734 7 Purity/0FRt0SRqqt1FBt/FBt_purity‚0.9992319345474243 ?&Purity/0FRt0SRqqt3FBt/event_proportion‚0.002583165420219302 ?&Purity/1FRt0SRqqt1FBt/event_proportion‚0.049773186445236206 =&Purity/1FRt1SRqqt*FBt/event_proportion‚0.1723790317773819 ("Purity/1FRt3SRqqt1FBt/event_purity‚0 7 Purity/2FRt1SRqqt2FBt/FRt_purity‚0.4714912176132202 particle/accuracy_1_of_5‚1 jet/accuracy_6_of_6‚0 8 Purity/*FRt2SRqqt*FBt/FRt_purity‚0.14368775486946106 @&Purity/*FRt3SRqqt2FBt/event_proportion‚6.300403038039804e-05 ,&Purity/3FRt4SRqqt4FBt/event_proportion‚0 :"Purity/2FRt4SRqqt*FBt/SRqqt_purity‚0.30000001192092896 9"Purity/1FRt2SRqqt2FBt/SRqqt_purity‚0.4236110746860504 ("Purity/1FRt1SRqqt4FBt/SRqqt_purity‚1 ("Purity/4FRt2SRqqt0FBt/event_purity‚0 7 Purity/*FRt0SRqqt1FBt/FBt_purity‚0.9995864033699036 >&Purity/2FRt*SRqqt*FBt/event_proportion‚0.10244455933570862 & Purity/2FRt*SRqqt1FBt/FBt_purity‚1 ,&Purity/2FRt0SRqqt4FBt/event_proportion‚0 8 Purity/2FRt1SRqqt0FBt/FRt_purity‚0.15364539623260498 7 Purity/*FRt*SRqqt4FBt/FBt_purity‚0.8333333134651184 +jet/accuracy_3_of_6‚0.18421052396297455 8"Purity/*FRt1SRqqt0FBt/SRqqt_purity‚0.500934898853302 7 Purity/*FRt1SRqqt2FBt/FRt_purity‚0.4474572241306305 A&Purity/0FRt0SRqqt4FBt/event_proportion‚0.00025201612152159214 :"Purity/2FRt*SRqqt0FBt/event_purity‚0.00918419286608696 ("Purity/3FRt2SRqqt0FBt/event_purity‚0 particle/accuracy_2_of_7‚1 8"Purity/3FRt3SRqqt0FBt/SRqqt_purity‚0.523809552192688 ) Purity/4FRt*SRqqt0FBt/FRt_purity‚0.25 9"Purity/*FRt*SRqqt0FBt/SRqqt_purity‚0.4675922989845276 >&Purity/*FRt0SRqqt1FBt/event_proportion‚0.14591734111309052 :"Purity/*FRt1SRqqt1FBt/event_purity‚0.11066509038209915 9"Purity/0FRt*SRqqt1FBt/SRqqt_purity‚0.5411539673805237 7 Purity/1FRt*SRqqt2FBt/FRt_purity‚0.5215697884559631 6 Purity/3FRt*SRqqt1FBt/FRt_purity‚0.302272766828537 8 Purity/*FRt1SRqqt0FBt/FRt_purity‚0.15396557748317719 6 Purity/0FRt*SRqqt*FBt/FBt_purity‚0.989280641078949 >&Purity/0FRt*SRqqt1FBt/event_proportion‚0.14787046611309052 ,&Purity/0FRt4SRqqt3FBt/event_proportion‚0 :"Purity/1FRt0SRqqt*FBt/event_purity‚0.20826943218708038 @&Purity/2FRt3SRqqt1FBt/event_proportion‚0.0003780241822823882 7 Purity/3FRt0SRqqt1FBt/FRt_purity‚0.4038461446762085 particle/accuracy_3_of_3‚1 7 Purity/1FRt*SRqqt0FBt/FRt_purity‚0.1576627641916275 ,&Purity/4FRt0SRqqt3FBt/event_proportion‚0 ("Purity/*FRt4SRqqt*FBt/event_purity‚0 particle/accuracy_3_of_6‚1 ,&Purity/*FRt4SRqqt4FBt/event_proportion‚0 8"Purity/0FRt1SRqqt1FBt/SRqqt_purity‚0.553488552570343 ,&Purity/0FRt2SRqqt4FBt/event_proportion‚0 ("Purity/2FRt4SRqqt*FBt/event_purity‚0 ("Purity/2FRt0SRqqt3FBt/event_purity‚0 8 Purity/*FRt*SRqqt0FBt/FRt_purity‚0.15404744446277618 ,&Purity/4FRt0SRqqt4FBt/event_proportion‚0 7 Purity/1FRt2SRqqt2FBt/FBt_purity‚0.9097222685813904 *"Purity/1FRt*SRqqt3FBt/SRqqt_purity‚0.5 ("Purity/4FRt1SRqqt0FBt/event_purity‚0 ,&Purity/3FRt4SRqqt1FBt/event_proportion‚0 +"Purity/3FRt1SRqqt2FBt/SRqqt_purity‚0.75 ("Purity/4FRt0SRqqt0FBt/event_purity‚0 8 Purity/*FRt*SRqqt2FBt/FRt_purity‚0.47851860523223877 9"Purity/*FRt3SRqqt0FBt/SRqqt_purity‚0.3676312565803528 9"Purity/*FRt3SRqqt1FBt/SRqqt_purity‚0.3743385672569275 :"Purity/0FRt*SRqqt0FBt/event_purity‚0.39407363533973694 :"Purity/0FRt1SRqqt*FBt/event_purity‚0.40472614765167236 ;"Purity/2FRt*SRqqt*FBt/event_purity‚0.016239898279309273 ?&Purity/2FRt1SRqqt2FBt/event_proportion‚0.001890120911411941 ("Purity/4FRt1SRqqt*FBt/SRqqt_purity‚0 _runtime‚3138.284434494 :"Purity/*FRt*SRqqt*FBt/SRqqt_purity‚0.48824450373649597 :"Purity/0FRt*SRqqt2FBt/SRqqt_purity‚0.48462963104248047 A&Purity/1FRt*SRqqt4FBt/event_proportion‚0.00012600806076079607 :"Purity/1FRt0SRqqt0FBt/event_purity‚0.17710725963115692 :"Purity/*FRt2SRqqt0FBt/SRqqt_purity‚0.44606953859329224 ?&Purity/*FRt3SRqqt*FBt/event_proportion‚0.020476309582591057 7 Purity/0FRt2SRqqt*FBt/FBt_purity‚0.9900153875350952 ("Purity/1FRt2SRqqt0FBt/event_purity‚0 >&Purity/1FRt2SRqqt0FBt/event_proportion‚0.04731602966785431 7 Purity/1FRt2SRqqt1FBt/FBt_purity‚0.9953917264938354 @&Purity/3FRt*SRqqt2FBt/event_proportion‚0.0003150201519019902 ("Purity/4FRt2SRqqt*FBt/event_purity‚0 @&Purity/0FRt2SRqqt3FBt/event_proportion‚6.300403038039804e-05 9"Purity/3FRt*SRqqt1FBt/SRqqt_purity‚0.7777777910232544 *jet/accuracy_2_of_3‚0.2570105195045471 *jet/accuracy_2_of_6‚0.6491227746009827 =&Purity/*FRt2SRqqt*FBt/event_proportion‚0.1332535296678543 :"Purity/0FRt0SRqqt3FBt/event_purity‚0.12666666507720947 =&Purity/0FRt1SRqqt*FBt/event_proportion‚0.2305317521095276 9"Purity/1FRt*SRqqt*FBt/SRqqt_purity‚0.4878331422805786 8 Purity/1FRt1SRqqt0FBt/FRt_purity‚0.15521308779716492 @&Purity/3FRt2SRqqt0FBt/event_proportion‚0.0024571572430431843 ,&Purity/4FRt3SRqqt2FBt/event_proportion‚0 @&Purity/4FRt*SRqqt*FBt/event_proportion‚0.0001890120911411941 9"Purity/*FRt0SRqqt1FBt/event_purity‚0.7033302783966064 ?&Purity/1FRt0SRqqt2FBt/event_proportion‚0.015120967291295528 7 Purity/1FRt1SRqqt2FBt/FRt_purity‚0.4557822644710541 7"Purity/2FRt1SRqqt0FBt/SRqqt_purity‚0.51542729139328 *jet/accuracy_1_of_5‚0.9132077693939209 +"Purity/3FRt2SRqqt1FBt/SRqqt_purity‚0.75 jet/accuracy_2_of_7‚0 @&Purity/3FRt0SRqqt1FBt/event_proportion‚0.0011340725468471646 8 Purity/3FRt2SRqqt0FBt/FRt_purity‚0.14371980726718903 ,&Purity/*FRt4SRqqt2FBt/event_proportion‚0 >&Purity/0FRt0SRqqt0FBt/event_proportion‚0.04082661122083664 9"Purity/0FRt0SRqqt1FBt/event_purity‚0.9992319345474243 ("Purity/1FRt3SRqqt*FBt/event_purity‚0 ,&Purity/4FRt3SRqqt0FBt/event_proportion‚0 jet/accuracy_1_of_7‚1 :"Purity/*FRt2SRqqt*FBt/SRqqt_purity‚0.46504104137420654 & Purity/1FRt3SRqqt*FBt/FBt_purity‚1 7 Purity/3FRt1SRqqt*FBt/FRt_purity‚0.1769547462463379 ,&Purity/*FRt2SRqqt4FBt/event_proportion‚0 9"Purity/0FRt*SRqqt0FBt/SRqqt_purity‚0.4653698205947876 @&Purity/3FRt0SRqqt*FBt/event_proportion‚0.0027091733645647764 particle/accuracy_2_of_5‚1 ( Purity/2FRt2SRqqt2FBt/FRt_purity‚0.5 ,&Purity/2FRt4SRqqt1FBt/event_proportion‚0 ("Purity/1FRt2SRqqt2FBt/event_purity‚0 :"Purity/*FRt0SRqqt*FBt/event_purity‚0.39130041003227234 =&Purity/*FRt1SRqqt*FBt/event_proportion‚0.4482736885547638 ;"Purity/*FRt2SRqqt0FBt/event_purity‚0.024824894964694977 9"Purity/1FRt2SRqqt0FBt/SRqqt_purity‚0.4377133548259735 <"Purity/2FRt1SRqqt*FBt/event_purity‚0.0010405826615169644 8 Purity/2FRt2SRqqt0FBt/FRt_purity‚0.14358215034008026 ,&Purity/3FRt1SRqqt4FBt/event_proportion‚0 ,&Purity/4FRt2SRqqt4FBt/event_proportion‚0 ,&Purity/3FRt0SRqqt4FBt/event_proportion‚0 ,&Purity/4FRt3SRqqt1FBt/event_proportion‚0 +jet/accuracy_3_of_5‚0.09134620428085327 & Purity/*FRt4SRqqt*FBt/FBt_purity‚1 & Purity/0FRt4SRqqt1FBt/FBt_purity‚1 >&Purity/0FRt*SRqqt2FBt/event_proportion‚0.03944052383303642 8 Purity/2FRt0SRqqt0FBt/FRt_purity‚0.16031105816364288 & Purity/3FRt0SRqqt1FBt/FBt_purity‚1 particle/accuracy_3_of_5‚1 * Purity/*FRt4SRqqt*FBt/FRt_purity‚0.125 =&Purity/*FRt*SRqqt0FBt/event_proportion‚0.6160534024238586 ,&Purity/0FRt3SRqqt3FBt/event_proportion‚0 7 Purity/1FRt3SRqqt*FBt/FRt_purity‚0.1557091623544693 ,&Purity/1FRt4SRqqt3FBt/event_proportion‚0 & Purity/3FRt*SRqqt1FBt/FBt_purity‚1 @&Purity/3FRt0SRqqt2FBt/event_proportion‚6.300403038039804e-05 & Purity/3FRt1SRqqt*FBt/FBt_purity‚1 ,&Purity/4FRt0SRqqt1FBt/event_proportion‚0 A&Purity/0FRt1SRqqt3FBt/event_proportion‚0.00044102821266278625 =&Purity/1FRt*SRqqt0FBt/event_proportion‚0.3135710656642914 7 Purity/2FRt0SRqqt2FBt/FRt_purity‚0.4821428656578064 ;"Purity/2FRt1SRqqt0FBt/event_purity‚0.001240694778971374 ("Purity/2FRt2SRqqt*FBt/event_purity‚0 ,&Purity/4FRt1SRqqt1FBt/event_proportion‚0 ,&Purity/4FRt2SRqqt2FBt/event_proportion‚0 'particle/f_score‚0.3961953818798065 :"Purity/*FRt*SRqqt0FBt/event_purity‚0.18183115124702454 & Purity/*FRt3SRqqt*FBt/FBt_purity‚1 >&Purity/1FRt2SRqqt*FBt/event_proportion‚0.05727066472172737 8"Purity/*FRt1SRqqt0FBt/event_purity‚0.258332759141922 9"Purity/2FRt*SRqqt0FBt/SRqqt_purity‚0.4921001195907593 :"Purity/3FRt2SRqqt0FBt/SRqqt_purity‚0.44927534461021423 7 Purity/*FRt2SRqqt2FBt/FRt_purity‚0.3717948794364929 & Purity/3FRt*SRqqt2FBt/FBt_purity‚1 ("Purity/3FRt3SRqqt0FBt/event_purity‚0 epoch‚2 ,&Purity/1FRt3SRqqt3FBt/event_proportion‚0 ?&Purity/2FRt*SRqqt2FBt/event_proportion‚0.005418346729129553 7 Purity/2FRt1SRqqt*FBt/FRt_purity‚0.2144928127527237 & Purity/3FRt0SRqqt2FBt/FBt_purity‚1 ("Purity/0FRt1SRqqt3FBt/event_purity‚0 8 Purity/*FRt2SRqqt0FBt/FRt_purity‚0.12367933243513107 @&Purity/3FRt1SRqqt1FBt/event_proportion‚0.0009450604557059705 particle/accuracy_3_of_4‚1 + Purity/0FRt*SRqqt4FBt/FBt_purity‚0.8125 & Purity/2FRt3SRqqt*FBt/FBt_purity‚1 ?&Purity/2FRt0SRqqt0FBt/event_proportion‚0.024571573361754417 7 Purity/2FRt1SRqqt*FBt/FBt_purity‚0.9964157938957214 & Purity/1FRt4SRqqt*FBt/FRt_purity‚0 ("Purity/4FRt1SRqqt0FBt/SRqqt_purity‚0 :"Purity/0FRt*SRqqt*FBt/SRqqt_purity‚0.48193591833114624 8 Purity/1FRt2SRqqt1FBt/FRt_purity‚0.19577571749687195 8 Purity/2FRt3SRqqt*FBt/FRt_purity‚0.08895239233970642 ,&Purity/3FRt*SRqqt3FBt/event_proportion‚0 particle/accuracy_4_of_5‚1 particle/accuracy_5_of_6‚1 particle/accuracy_3_of_7‚1 @&Purity/*FRt0SRqqt4FBt/event_proportion‚0.0003150201519019902 8 Purity/*FRt3SRqqt*FBt/FRt_purity‚0.12199730426073074 :"Purity/*FRt3SRqqt0FBt/event_purity‚0.00358422938734293 ?&Purity/*FRt3SRqqt1FBt/event_proportion‚0.002142137149348855 ,&Purity/0FRt3SRqqt4FBt/event_proportion‚0 8"Purity/1FRt1SRqqt*FBt/SRqqt_purity‚0.531638503074646 :"Purity/1FRt2SRqqt*FBt/SRqqt_purity‚0.45721402764320374 @&Purity/0FRt*SRqqt3FBt/event_proportion‚0.0030871976632624865 >&Purity/1FRt1SRqqt1FBt/event_proportion‚0.03944052383303642 9"Purity/2FRt*SRqqt1FBt/SRqqt_purity‚0.5850185751914978 ("Purity/3FRt0SRqqt0FBt/event_purity‚0 ("Purity/0FRt4SRqqt*FBt/event_purity‚0 9"Purity/*FRt*SRqqt3FBt/SRqqt_purity‚0.8333333134651184 8 Purity/*FRt*SRqqt*FBt/FRt_purity‚0.21905387938022614 :"Purity/*FRt*SRqqt3FBt/event_purity‚0.09195401519536972 ,&Purity/*FRt4SRqqt3FBt/event_proportion‚0 7 Purity/1FRt*SRqqt*FBt/FBt_purity‚0.9917953610420227 jet/accuracy_4_of_6‚0 7 Purity/*FRt0SRqqt3FBt/FRt_purity‚0.5333333611488342 7 Purity/*FRt1SRqqt*FBt/FBt_purity‚0.9916169047355652 7 Purity/1FRt0SRqqt3FBt/FBt_purity‚0.9285714030265808 9"Purity/1FRt1SRqqt1FBt/SRqqt_purity‚0.5985221266746521 ("Purity/0FRt3SRqqt1FBt/event_purity‚0 ( Purity/2FRt*SRqqt3FBt/FRt_purity‚0.5 ?&Purity/*FRt2SRqqt2FBt/event_proportion‚0.002079133177176118 A&Purity/0FRt4SRqqt0FBt/event_proportion‚0.00044102821266278625 7 Purity/1FRt0SRqqt2FBt/FBt_purity‚0.9874216318130493 9"Purity/1FRt2SRqqt1FBt/SRqqt_purity‚0.5266897082328796 ("Purity/2FRt0SRqqt2FBt/event_purity‚0 *jet/accuracy_1_of_1‚0.4814811646938324 *jet/accuracy_1_of_3‚0.8552227020263672 7 Purity/0FRt1SRqqt*FBt/FBt_purity‚0.9901571869850159 ,&Purity/2FRt3SRqqt3FBt/event_proportion‚0 particle/accuracy_1_of_7‚1 9"Purity/0FRt0SRqqt*FBt/event_purity‚0.8144093751907349 :"Purity/1FRt1SRqqt*FBt/event_purity‚0.01986980438232422 <"Purity/1FRt2SRqqt*FBt/event_purity‚0.0011520737316459417 ,&Purity/2FRt1SRqqt3FBt/event_proportion‚0 @&Purity/4FRt2SRqqt*FBt/event_proportion‚6.300403038039804e-05 @&Purity/3FRt2SRqqt*FBt/event_proportion‚0.0028351815417408943 ."Purity/0FRt4SRqqt*FBt/SRqqt_purity‚0.28125 ("Purity/0FRt4SRqqt1FBt/event_purity‚0 9"Purity/0FRt1SRqqt3FBt/SRqqt_purity‚0.8999999761581421 ) Purity/4FRt*SRqqt*FBt/FRt_purity‚0.25 6 Purity/*FRt*SRqqt*FBt/FBt_purity‚0.990805447101593 @&Purity/*FRt4SRqqt*FBt/event_proportion‚0.0013230846961960196 :"Purity/0FRt*SRqqt2FBt/event_purity‚0.27149394154548645 7 Purity/1FRt*SRqqt*FBt/FRt_purity‚0.2252509444952011 ("Purity/3FRt1SRqqt1FBt/event_purity‚0 ,jet/accuracy_4_of_5‚0.009984639473259449 ("Purity/0FRt0SRqqt4FBt/event_purity‚0 & Purity/1FRt0SRqqt4FBt/FRt_purity‚1 7 Purity/*FRt1SRqqt*FBt/FRt_purity‚0.2090562880039215 9"Purity/0FRt*SRqqt*FBt/event_purity‚0.4712952673435211 ,&Purity/3FRt3SRqqt2FBt/event_proportion‚0 particle/accuracy_1_of_1‚1 ("Purity/1FRt1SRqqt4FBt/event_purity‚0 ("Purity/4FRt2SRqqt0FBt/SRqqt_purity‚1 7 Purity/*FRt0S‰¨O0ïRqqt1FBt/FRt_purity‚0.4175725281238556 >&Purity/1FRt*SRqqt1FBt/event_proportion‚0.09929435700178146 8 Purity/2FRt*SRqqt1FBt/FRt_purity‚0.37670913338661194 ("Purity/*FRt*SRqqt4FBt/SRqqt_purity‚1 >&Purity/*FRt2SRqqt0FBt/event_proportion‚0.10855595022439957 8 Purity/1FRt1SRqqt*FBt/FRt_purity‚0.20882214605808258 9"Purity/2FRt1SRqqt*FBt/SRqqt_purity‚0.5457957983016968 ,&Purity/4FRt1SRqqt4FBt/event_proportion‚0 ("Purity/3FRt1SRqqt2FBt/event_purity‚0 & Purity/0FRt3SRqqt1FBt/FBt_purity‚1 :"Purity/*FRt2SRqqt*FBt/event_purity‚0.02513105981051922 @&Purity/1FRt2SRqqt2FBt/event_proportion‚0.0010080644860863686 ?&Purity/2FRt0SRqqt2FBt/event_proportion‚0.003465221729129553 8 Purity/3FRt2SRqqt*FBt/FRt_purity‚0.16087962687015533 ,&Purity/4FRt4SRqqt3FBt/event_proportion‚0 <"Purity/0FRt3SRqqt0FBt/event_purity‚0.0071428571827709675 @&Purity/*FRt1SRqqt4FBt/event_proportion‚6.300403038039804e-05 :"Purity/0FRt2SRqqt0FBt/event_purity‚0.06375109404325485 0"Purity/*FRt4SRqqt0FBt/SRqqt_purity‚ 0.2421875 ("Purity/*FRt4SRqqt1FBt/event_purity‚0 & Purity/3FRt2SRqqt*FBt/FBt_purity‚1 >&Purity/*FRt*SRqqt2FBt/event_proportion‚0.06747731566429138 & Purity/1FRt0SRqqt1FBt/FBt_purity‚1 9"Purity/1FRt1SRqqt0FBt/SRqqt_purity‚0.5078604221343994 ,&Purity/1FRt4SRqqt4FBt/event_proportion‚0 9"Purity/2FRt2SRqqt*FBt/SRqqt_purity‚0.5299275517463684 @&Purity/4FRt1SRqqt*FBt/event_proportion‚6.300403038039804e-05 trainer/global_step‚1811 9"Purity/1FRt*SRqqt1FBt/SRqqt_purity‚0.5670551657676697 ,&Purity/4FRt1SRqqt3FBt/event_proportion‚0 :"Purity/2FRt3SRqqt1FBt/SRqqt_purity‚0.20000000298023224 A&Purity/0FRt*SRqqt4FBt/event_proportion‚0.00025201612152159214 6 Purity/1FRt1SRqqt*FBt/FBt_purity‚0.993652880191803 (particle/accuracy‚0.8503233790397644 ,&Purity/2FRt2SRqqt3FBt/event_proportion‚0 ("Purity/3FRt*SRqqt1FBt/event_purity‚0 +jet/accuracy_4_of_4‚0.00247434014454484 & Purity/4FRt2SRqqt0FBt/FRt_purity‚0 7 Purity/0FRt1SRqqt1FBt/FBt_purity‚0.9979166984558105 ?&Purity/0FRt3SRqqt0FBt/event_proportion‚0.006111391354352236 9"Purity/1FRt*SRqqt2FBt/SRqqt_purity‚0.5328981280326843 ,&Purity/2FRt3SRqqt2FBt/event_proportion‚0 ,&Purity/3FRt3SRqqt4FBt/event_proportion‚0 ("Purity/0FRt*SRqqt4FBt/event_purity‚0 particle/accuracy_4_of_6‚1 ("Purity/0FRt4SRqqt0FBt/event_purity‚0 :"Purity/0FRt*SRqqt3FBt/event_purity‚0.09259258955717087 7 Purity/1FRt*SRqqt1FBt/FBt_purity‚0.9988213777542114 7 Purity/1FRt1SRqqt2FBt/FBt_purity‚0.9806547164916992 @&Purity/4FRt*SRqqt0FBt/event_proportion‚0.0001890120911411941 & Purity/1FRt1SRqqt3FBt/FBt_purity‚1 *loss/val_total_loss‚43.738315762282525 7 Purity/*FRt1SRqqt1FBt/FBt_purity‚0.9983752965927124 :"Purity/1FRt*SRqqt0FBt/SRqqt_purity‚0.46393752098083496 @&Purity/1FRt3SRqqt2FBt/event_proportion‚6.300403038039804e-05 7 Purity/2FRt0SRqqt1FBt/FRt_purity‚0.3699338734149933 ("Purity/2FRt3SRqqt0FBt/event_purity‚0 7 Purity/*FRt*SRqqt1FBt/FBt_purity‚0.9986271262168884 >&Purity/0FRt2SRqqt*FBt/event_proportion‚0.05361643061041832 7 Purity/*FRt0SRqqt4FBt/FBt_purity‚0.8500000238418579 9"Purity/0FRt3SRqqt*FBt/SRqqt_purity‚0.3859599232673645 *"Purity/1FRt1SRqqt3FBt/SRqqt_purity‚0.5 ("Purity/2FRt2SRqqt0FBt/event_purity‚0 7 Purity/2FRt2SRqqt1FBt/FRt_purity‚0.2526785731315613 ,&Purity/4FRt*SRqqt3FBt/event_proportion‚0 @&Purity/4FRt2SRqqt0FBt/event_proportion‚6.300403038039804e-05 ("Purity/3FRt2SRqqt1FBt/event_purity‚0 ?&Purity/3FRt*SRqqt*FBt/event_proportion‚0.009954636916518211 + Purity/0FRt0SRqqt4FBt/FBt_purity‚0.8125 :"Purity/2FRt4SRqqt0FBt/SRqqt_purity‚0.30000001192092896 & Purity/1FRt1SRqqt4FBt/FRt_purity‚0 & Purity/4FRt0SRqqt0FBt/FRt_purity‚0 @&Purity/*FRt*SRqqt4FBt/event_proportion‚0.0003780241822823882 @&Purity/2FRt*SRqqt3FBt/event_proportion‚6.300403038039804e-05 ,&Purity/4FRt4SRqqt1FBt/event_proportion‚0 ,"Purity/3FRt4SRqqt*FBt/SRqqt_purity‚0.375 7 Purity/*FRt*SRqqt3FBt/FBt_purity‚0.8975096940994263 @&Purity/*FRt4SRqqt1FBt/event_proportion‚6.300403038039804e-05 9"Purity/2FRt*SRqqt2FBt/SRqqt_purity‚0.7236841917037964 9"Purity/2FRt1SRqqt2FBt/SRqqt_purity‚0.7236841917037964 ("Purity/3FRt0SRqqt1FBt/event_purity‚0 A&Purity/3FRt1SRqqt2FBt/event_proportion‚0.00025201612152159214 @&Purity/4FRt1SRqqt0FBt/event_proportion‚6.300403038039804e-05 & Purity/*FRt1SRqqt4FBt/FRt_purity‚0 6 Purity/1FRt0SRqqt*FBt/FRt_purity‚0.269497275352478 @&Purity/1FRt4SRqqt0FBt/event_proportion‚0.0003780241822823882 ("Purity/2FRt3SRqqt*FBt/event_purity‚0 ,&Purity/2FRt4SRqqt4FBt/event_proportion‚0 particle/accuracy_1_of_2‚1 particle/accuracy_5_of_5‚1 7 Purity/1FRt2SRqqt2FBt/FRt_purity‚0.3611111342906952 +"Purity/3FRt*SRqqt2FBt/SRqqt_purity‚0.75 9"Purity/*FRt1SRqqt1FBt/SRqqt_purity‚0.5767184495925903 @&Purity/*FRt4SRqqt0FBt/event_proportion‚0.0012600806076079607 9"Purity/0FRt*SRqqt1FBt/event_purity‚0.6490389704704285 9"Purity/1FRt0SRqqt3FBt/event_purity‚0.1071428582072258 ,&Purity/3FRt1SRqqt3FBt/event_proportion‚0 *jet/accuracy_1_of_2‚0.7427793145179749 ("Purity/1FRt*SRqqt4FBt/event_purity‚0 ( Purity/1FRt*SRqqt4FBt/FRt_purity‚0.5 7 Purity/*FRt*SRqqt2FBt/FBt_purity‚0.9853708148002625 8 Purity/1FRt2SRqqt0FBt/FRt_purity‚0.11791838705539703 ?&Purity/2FRt1SRqqt1FBt/event_proportion‚0.007875503972172737 & Purity/0FRt3SRqqt*FBt/FBt_purity‚1 ( Purity/2FRt0SRqqt3FBt/FRt_purity‚0.5 ("Purity/3FRt3SRqqt*FBt/event_purity‚0 7 Purity/3FRt3SRqqt*FBt/FRt_purity‚0.0476190485060215 *"Purity/4FRt*SRqqt0FBt/SRqqt_purity‚0.5 >&Purity/*FRt0SRqqt0FBt/event_proportion‚0.15757308900356293 ;"Purity/*FRt1SRqqt2FBt/event_purity‚0.024884795770049095 7 Purity/*FRt1SRqqt2FBt/FBt_purity‚0.9829049110412598 ("Purity/*FRt3SRqqt1FBt/event_purity‚0 *jet/accuracy_1_of_6‚0.9473684430122375 ("Purity/1FRt4SRqqt0FBt/event_purity‚0 ("Purity/*FRt2SRqqt3FBt/SRqqt_purity‚1 8 Purity/*FRt2SRqqt1FBt/FRt_purity‚0.23405133187770844 7 Purity/1FRt*SRqqt2FBt/FBt_purity‚0.9656379818916321 7 Purity/1FRt1SRqqt1FBt/FBt_purity‚0.9988052845001221 ,&Purity/4FRt4SRqqt*FBt/event_proportion‚0 particle/accuracy_7_of_7‚1 6 Purity/0FRt1SRqqt3FBt/FBt_purity‚0.800000011920929 ( Purity/4FRt1SRqqt0FBt/FRt_purity‚0.5 @&Purity/1FRt0SRqqt3FBt/event_proportion‚0.0010710685746744275 9"Purity/*FRt*SRqqt2FBt/SRqqt_purity‚0.5525772571563721 :"Purity/*FRt3SRqqt*FBt/event_purity‚0.00358422938734293 7 Purity/0FRt1SRqqt2FBt/FBt_purity‚0.9810184836387634 7 Purity/1FRt*SRqqt3FBt/FBt_purity‚0.9479167461395264 8"Purity/3FRt*SRqqt0FBt/SRqqt_purity‚0.440532922744751 ,&Purity/3FRt2SRqqt2FBt/event_proportion‚0 & Purity/3FRt2SRqqt1FBt/FBt_purity‚1 >&Purity/0FRt*SRqqt*FBt/event_proportion‚0.40996724367141724 @&Purity/*FRt2SRqqt3FBt/event_proportion‚6.300403038039804e-05 :"Purity/0FRt1SRqqt1FBt/event_purity‚0.18086053431034088 ?&Purity/0FRt1SRqqt2FBt/event_proportion‚0.008757560513913631 :"Purity/0FRt2SRqqt*FBt/event_purity‚0.06152655556797981 7 Purity/1FRt0SRqqt*FBt/FBt_purity‚0.9930731654167175 ?&Purity/1FRt3SRqqt*FBt/event_proportion‚0.009135584346950054 ?&Purity/2FRt0SRqqt1FBt/event_proportion‚0.008946572430431843 =&Purity/*FRt0SRqqt*FBt/event_proportion‚0.3558467626571655 >&Purity/0FRt0SRqqt*FBt/event_proportion‚0.11857359111309052 ,&Purity/1FRt4SRqqt1FBt/event_proportion‚0 9"Purity/2FRt2SRqqt1FBt/SRqqt_purity‚0.5869048237800598 ,&Purity/3FRt3SRqqt3FBt/event_proportion‚0 ,&Purity/4FRt4SRqqt0FBt/event_proportion‚0 particle/accuracy_4_of_4‚1 ( Purity/*FRt*SRqqt4FBt/FRt_purity‚0.5 6 Purity/3FRt0SRqqt*FBt/FRt_purity‚0.252525269985199 >&Purity/2FRt1SRqqt*FBt/event_proportion‚0.04158266261219978 ("Purity/2FRt1SRqqt1FBt/event_purity‚0 7 Purity/2FRt1SRqqt2FBt/FBt_purity‚0.9868420958518982’ô2mJžÚ‚(:“ ‰ÐéÔÍÂý¹ gpu.0.gpu‚63.0 gpu.0.memory‚5, gpu.0.memoryAllocated‚16.718826293945312* gpu.0.memoryAllocatedBytes‚ 14361362432 gpu.0.temp‚34.0 gpu.0.powerWatts‚171.259' gpu.0.enforcedPowerLimitWatts‚500.0) gpu.0.powerPercent‚34.251799999999996 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0%ÀV‡Û‚(:„ ‰ÐéÔœ°ò¿ network.sent‚ 4.224244e+06 disk.sda3.out‚ 226.453125( disk.nvme0c0n1.out‚71554.79833984375 disk.nvme0n1.in‚ 7516.15625 disk.dm-0.in‚ 7516.15625# disk.dm-0.out‚71554.79833984375 disk.drbd1260.in‚0 disk.drbd1260.out‚ 0.43359375) disk./.usagePercent‚78.79320040097744 disk.sda2.out‚0 disk.sda3.in‚ 0.6953125 disk.nvme0c0n1.in‚ 7516.15625 disk.dm-1.in‚ 0.6953125 disk.sda1.out‚0 disk.sda.in‚0.796875 disk.sda.out‚ 226.453125$ disk./.usageGB‚1174.075626373291 disk.sda2.in‚0 disk.dm-1.out‚ 226.453125# proc.memory.rssMB‚ 3016.53515625) proc.memory.percent‚18.41146945953369 cpu‚25.77097004198733 proc.cpu.threads‚17 network.recv‚ 1.054093e+06& disk.nvme0n1.out‚71554.79833984375 disk.sda1.in‚ 0.1015625# memory_percent‚98.9438533782959' proc.memory.availableMB‚ 173.0390625VÊg`“Ü‚(:ˆ ˜ÐéÔŸ·¹¹ gpu.0.gpu‚49.0 gpu.0.memory‚6, gpu.0.memoryAllocated‚16.718826293945312* gpu.0.memoryAllocatedBytes‚ 14361362432 gpu.0.temp‚35.0 gpu.0.powerWatts‚165.699' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚33.1398 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0\lþ—Ý‚(:Œ ˜ÐéÔô«Ó» disk.sda3.out‚ 226.81640625 disk.dm-1.in‚ 0.6953125 disk.sda1.out‚0' proc.memory.availableMB‚ 173.2734375# proc.memory.rssMB‚ 3016.53515625 cpu‚25.77055376945139 proc.cpu.threads‚17% disk./.usageGB‚1174.3030967712402 disk.sda1.in‚ 0.1015625 disk.sda.out‚ 226.81640625$ memory_percent‚98.94242286682129) proc.memory.percent‚18.41146945953369 disk.sda2.out‚0 disk.nvme0c0n1.in‚ 7516.15625 disk.nvme0n1.in‚ 7516.15625 disk.dm-0.in‚ 7516.15625 disk.dm-1.out‚ 226.81640625 disk.drbd1260.in‚0 disk.drbd1260.out‚ 0.44140625 disk.sda.in‚0.796875# disk.dm-0.out‚71652.62060546875 network.sent‚ 4.326682e+06 network.recv‚ 1.060589e+06) disk./.usagePercent‚78.80846613024418 disk.sda2.in‚0 disk.sda3.in‚ 0.6953125( disk.nvme0c0n1.out‚71652.62060546875& disk.nvme0n1.out‚71652.62060546875‰½*{“Þ‚(:ˆ §ÐéÔàûü¸ gpu.0.gpu‚28.0 gpu.0.memory‚2, gpu.0.memoryAllocated‚16.718826293945312* gpu.0.memoryAllocatedBytes‚ 14361362432 gpu.0.temp‚32.0 gpu.0.powerWatts‚114.748' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚22.9496 gpu.0.smClock‚1275 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0¿3þ ˜ß‚(: §ÐéÔæ·ˆ¼$ memory_percent‚98.94239902496338 cpu‚25.769676898419384 network.sent‚ 4.33261e+06 network.recv‚ 1.064796e+06" disk.nvme0c0n1.in‚ 7653.4609375% disk.nvme0n1.out‚72211.1025390625 disk.dm-1.out‚ 227.1484375 disk.sda.in‚0.796875) proc.memory.percent‚18.41146945953369 proc.cpu.threads‚17% disk./.usageGB‚1174.3255348205566 disk.sda2.in‚0 disk.sda2.out‚0 disk.sda3.in‚ 0.6953125 disk.dm-1.in‚ 0.6953125 disk.sda1.in‚ 0.1015625( proc.memory.availableMB‚ 173.27734375# proc.memory.rssMB‚ 3016.53515625) disk./.usagePercent‚78.80997196656058" disk.dm-0.out‚72211.1025390625 disk.drbd1260.in‚0 disk.sda.out‚ 227.1484375 disk.sda3.out‚ 227.1484375' disk.nvme0c0n1.out‚72211.1025390625 disk.nvme0n1.in‚ 7653.4609375 disk.dm-0.in‚ 7653.4609375 disk.drbd1260.out‚ 0.44140625 disk.sda1.out‚0TI!2à- _wandbruntime‚3192  _runtime‚3192$®³ÇáÁ _step‚147 & lr-AdamW/pg1‚0.0011428053101293228 & lr-AdamW/pg2‚0.0011428053101293228 trainer/global_step‚1849 % _timestamp‚1.7885041167153523e+09 _runtime‚3192.794229368~Ï7³ÌâÆ trainer/global_step‚1849 % _timestamp‚1.7885041167153523e+09 _runtime‚3192.794229368 _step‚147 & lr-AdamW/pg1‚0.0011428053101293228 & lr-AdamW/pg2‚0.0011428053101293228“ eÐk2ã- _wandbruntime‚3192  _runtime‚3192Rñ>›ä'äÞO UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 231245824 M@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚33694249 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚2 WEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 1833005249024 \HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚150168092068352 TGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚46456498 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 JBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚286 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚31686 YEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚150168092068352 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚2 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 UGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 582240777 O?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 13807648768 % _timestamp‚1.7885041167166755e+09 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚26090 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚31686 YEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚152001345602560 `LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚150168109107712 OFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚1299 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 BDeviceStatsMonitor.on_train_batch_start/allocation.all.current‚26090 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚607 ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 265779200 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 320831488 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 [KDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 13207863296 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚609 IDeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚152001097317376 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 248285184 WGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 13035530240 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 TGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚38357795 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚605 RBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 13807648768 PCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚38357793 H?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚1321 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 YIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 13207863296 QDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚33694856 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚3689 QCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 243631708 M=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 13568115712 \HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚146908670991536 PCDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated‚46456498 N@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 248285184 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 599785472 L?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚46430410 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚11550411 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 TFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 599785472 D>DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 YEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚148731712392284 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚17039360 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚22143840 eQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚152371884401664 E8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚84788203 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 586907136 ]IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚150168109107712 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚31931 PCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚46430410 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 ^LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 1823024361388 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚24903680 UADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚148731468760576 P@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 13118524033 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 IDeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚3689 [IDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 1833236494848 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 152061440 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 WGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 12589930496 @8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚895 C;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚895 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚286 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 599785472 MFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚33 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚26088 P@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 13568115712 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 231245824 ZFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚154288805624320 ^LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 1833236494848 TDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 13807648768 trainer/global_step‚1849 TDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 13035530240 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚22143838 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0^ÿ Y'åãO PCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚38357795 VBDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated‚152001345602560 R>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚152001097317376 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 248285184 WGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 13035530240 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 TGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚38357795 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚605 RBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 13807648768 PCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚38357793 H?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚1321 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 YIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 13207863296 QDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚33694856 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚3689 QCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 243631708 M=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 13568115712 \HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚146908670991536 PCDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated‚46456498 N@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 248285184 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 599785472 L?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚46430410 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚11550411 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 TFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 599785472 D>DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 YEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚148731712392284 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚17039360 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚22143840 eQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚152371884401664 E8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚84788203 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 586907136 ]IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚150168109107712 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚31931 PCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚46430410 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 ^LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 1823024361388 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚24903680 UADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚148731468760576 P@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 13118524033 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 IDeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚3689 [IDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 1833236494848 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 152061440 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 WGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 12589930496 @8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚895 C;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚895 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚286 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 599785472 MFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚33 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚26088 P@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 13568115712 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 231245824 ZFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚154288805624320 ^LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 1833236494848 TDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 13807648768 trainer/global_step‚1849 TDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 13035530240 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚22143838 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 231245824 M@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚33694249 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚2 WEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 1833005249024 \HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚150168092068352 TGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚46456498 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 JBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚286 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚31686 YEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚150168092068352 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚2 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 UGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 582240777 O?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 13807648768 % _timestamp‚1.7885041167166755e+09 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚26090 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚31686 YEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚152001345602560 `LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚150168109107712 OFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚1299 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 BDeviceStatsMonitor.on_train_batch_start/allocation.all.current‚26090 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚607 ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 265779200 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 320831488 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 [KDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 13207863296 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚609 IDeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚84858861 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 582240777 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚609 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚26116 RBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 13807648768 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 C6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚84832745 NADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚38377750 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1321 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 599785472 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚26114 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚33 _KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚152451454358016 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚286 NADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚38377752 REDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚46481109 OBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚33712199 aODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 1918223400960 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚17039360 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚22155333 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚31931 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 320831488 XFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 1833962720256 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 231259136 N>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 13568115712 NADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚46454995 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 BDeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚33711577 L>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 248298496 G:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚84858861 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚38377750 PBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 586907136 P6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚895 [GDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚150246620316672 \HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚154369702662656 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 152061440 WCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚148809511174430 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚895 F6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚895 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 UCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 1833962720256 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚3689 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚11556866 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 trainer/global_step‚1849 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚31686 WCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚150246603277312 UEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 12589930496 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚895 REDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚38377752 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 WGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 13207863296 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚286 J=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚46454995 K;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 13568115712 S?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚152080565997568 cODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚152451479261696 ZFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚146985517726080 EDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 13118524033 TDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 13207863296 XFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 1823749816434 \JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 1834193979392 T@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚152080814296064 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚17039360 YGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 1834193979392 REDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚22155331 REDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚11556246 WIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 599785472 _runtime‚3193.960438859 OADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 248298496 ^JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚150246620316672 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚31931 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 599785472hD\ó&èíM REDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚38377752 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 WGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 13207863296 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚286 J=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚46454995 K;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 13568115712 S?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚152080565997568 cODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚152451479261696 ZFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚146985517726080 EDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 13118524033 TDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 13207863296 XFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 1823749816434 \JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 1834193979392 T@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚152080814296064 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚17039360 YGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 1834193979392 REDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚22155331 REDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚11556246 WIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 599785472 _runtime‚3193.960438859 OADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 248298496 ^JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚150246620316672 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚31931 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 599785472 RBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 13035530240 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 231259136 K>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚84858861 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 582240777 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚609 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚26116 RBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 13807648768 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 C6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚84832745 NADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚38377750 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1321 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 599785472 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚26114 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚33 _KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚152451454358016 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚286 NADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚38377752 REDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚46481109 OBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚33712199 aODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 1918223400960 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚17039360 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚22155333 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚31931 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 320831488 XFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 1833962720256 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 231259136 N>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 13568115712 NADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚46454995 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 BDeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚33711577 L>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 248298496 G:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚84858861 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚38377750 PBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 586907136 P6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚895 [GDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚150246620316672 \HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚154369702662656 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 152061440 WCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚148809511174430 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚895 F6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚895 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 UCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 1833962720256 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚3689 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚11556866 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 trainer/global_step‚1849 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚31686 WCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚150246603277312 UEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 12589930496 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚895•‰[–2é- _wandbruntime‚3193  _runtime‚3193&¨gZêÔ -loss/FBt4/detection_loss‚2.51478865891695 /loss/FRt3/assignment_loss‚2.249708388745785 1loss/SRqqt4/detection_loss‚0.8488387199955792 .loss/FBt1/detection_loss‚2.468990167746177 &loss/total_loss‚43.468376030144945 $ _timestamp‚1.788504117882655e+09 0loss/FRt2/assignment_loss‚3.6245689663020046 /loss/FRt4/detection_loss‚1.6368201808961294 1loss/SRqqt1/assignment_loss‚1.924896259352846 2loss/SRqqt3/assignment_loss‚1.9314159368249504 1loss/FBt3/assignment_loss‚0.15135702625479755 /loss/FRt3/detection_loss‚2.1426791423931717 1loss/SRqqt2/detection_loss‚2.1817625294893217 0loss/FRt1/assignment_loss‚3.5161196818718543 0loss/FRt4/assignment_loss‚1.5032148625104482 1loss/FBt4/assignment_loss‚0.17737793270998736 1loss/FBt2/assignment_loss‚0.28879956424400594 epoch‚3 trainer/global_step‚1849 _runtime‚ 3193.96182937 /loss/FRt2/detection_loss‚2.3480767321276974 2loss/SRqqt2/assignment_loss‚1.6479869492065449 /loss/FRt1/detection_loss‚2.7765948158044083 /loss/FBt2/detection_loss‚2.2696809935219147 1loss/SRqqt1/detection_loss‚2.6608159238437437 1loss/SRqqt3/detection_loss‚2.4687042236328125 .loss/FBt3/detection_loss‚1.008233210599396 _step‚150 2loss/SRqqt4/assignment_loss‚0.8729747324646545 /loss/FBt1/assignment_loss‚0.253970430689763Jþ _ëÙ /loss/FRt3/assignment_loss‚2.249708388745785 1loss/SRqqt4/detection_loss‚0.8488387199955792 .loss/FBt1/detection_loss‚2.468990167746177 &loss/total_loss‚43.468376030144945 $ _timestamp‚1.788504117882655e+09 0loss/FRt2/assignment_loss‚3.6245689663020046 /loss/FRt4/detection_loss‚1.6368201808961294 1loss/SRqqt1/assignment_loss‚1.924896259352846 2loss/SRqqt3/assignment_loss‚1.9314159368249504 1loss/FBt3/assignment_loss‚0.15135702625479755 /loss/FRt3/detection_loss‚2.1426791423931717 1loss/SRqqt2/detection_loss‚2.1817625294893217 0loss/FRt1/assignment_loss‚3.5161196818718543 0loss/FRt4/assignment_loss‚1.5032148625104482 1loss/FBt4/assignment_loss‚0.17737793270998736 1loss/FBt2/assignment_loss‚0.28879956424400594 epoch‚3 trainer/global_step‚1849 _runtime‚ 3193.96182937 /loss/FRt2/detection_loss‚2.3480767321276974 2loss/SRqqt2/assignment_loss‚1.6479869492065449 /loss/FRt1/detection_loss‚2.7765948158044083 /loss/FBt2/detection_loss‚2.2696809935219147 1loss/SRqqt1/detection_loss‚2.6608159238437437 1loss/SRqqt3/detection_loss‚2.4687042236328125 .loss/FBt3/detection_loss‚1.008233210599396 _step‚150 2loss/SRqqt4/assignment_loss‚0.8729747324646545 /loss/FBt1/assignment_loss‚0.253970430689763 -loss/FBt4/detection_loss‚2.51478865891695–Uœ¬ žì‚(:“ ¶ÐéÔý϶» gpu.0.gpu‚29.0 gpu.0.memory‚4, gpu.0.memoryAllocated‚16.718826293945312* gpu.0.memoryAllocatedBytes‚ 14361362432 gpu.0.temp‚34.0 gpu.0.powerWatts‚115.416' gpu.0.enforcedPowerLimitWatts‚500.0) gpu.0.powerPercent‚23.083199999999998 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0±aì–í‚(:‹ ¶ÐéÔ†œ½ disk.drbd1260.out‚ 0.44140625 disk.sda1.in‚ 0.1015625 proc.cpu.threads‚17 network.recv‚ 1.069557e+06% disk./.usageGB‚1174.2164726257324 disk.sda2.in‚0 disk.sda3.out‚ 227.59765625" disk.nvme0c0n1.in‚ 7653.4609375 disk.sda.out‚ 227.59765625# proc.memory.rssMB‚ 3016.53515625) disk./.usagePercent‚78.80265271115665 disk.sda3.in‚ 0.6953125& disk.nvme0c0n1.out‚72405.306640625$ disk.nvme0n1.out‚72405.306640625! disk.dm-0.out‚72405.306640625 disk.sda1.out‚0$ memory_percent‚98.94390106201172% proc.memory.availableMB‚ 173.03125 network.sent‚ 4.338829e+06 disk.sda2.out‚0 disk.dm-0.in‚ 7653.4609375 disk.dm-1.in‚ 0.6953125 disk.sda.in‚0.796875) proc.memory.percent‚18.41146945953369 cpu‚25.768949491471165 disk.nvme0n1.in‚ 7653.4609375 disk.dm-1.out‚ 227.59765625 disk.drbd1260.in‚0ÞñßÁî‚(:’ ÅÐéÔÜšõ¹ gpu.0.gpu‚40.0 gpu.0.memory‚3, gpu.0.memoryAllocated‚16.718826293945312* gpu.0.memoryAllocatedBytes‚ 14361362432 gpu.0.temp‚35.0 gpu.0.powerWatts‚147.63' gpu.0.enforcedPowerLimitWatts‚500.0) gpu.0.powerPercent‚29.525999999999996 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0íCëï‚(:’ ÅÐéÔÑÌþ¼ disk.dm-1.out‚ 227.94140625 disk.drbd1260.in‚0% disk./.usageGB‚1174.5552139282227 disk.sda1.out‚0' proc.memory.availableMB‚ 173.2265625 cpu‚25.768405839324327 proc.cpu.threads‚17 disk.dm-1.in‚ 0.6953125 disk.sda1.in‚ 0.1015625 disk.sda.in‚0.796875 disk.sda.out‚ 227.94140625$ memory_percent‚98.94270896911621# proc.memory.rssMB‚ 3016.53515625) proc.memory.percent‚18.41146945953369 disk.sda2.out‚0" disk.nvme0c0n1.in‚ 7653.4609375( disk.nvme0c0n1.out‚72784.99658203125 disk.nvme0n1.in‚ 7653.4609375& disk.nvme0n1.out‚72784.99658203125 disk.dm-0.in‚ 7653.4609375# disk.dm-0.out‚72784.99658203125 disk.drbd1260.out‚ 0.4453125 network.sent‚ 4.425469e+06 network.recv‚ 1.075527e+06) disk./.usagePercent‚78.82538592418966 disk.sda2.in‚0 disk.sda3.in‚ 0.6953125 disk.sda3.out‚ 227.94140625$$8+žð‚(:“ ÔÐéÔñ܈¹ gpu.0.gpu‚36.0 gpu.0.memory‚3, gpu.0.memoryAllocated‚16.718826293945312* gpu.0.memoryAllocatedBytes‚ 14361362432 gpu.0.temp‚33.0 gpu.0.powerWatts‚123.037' gpu.0.enforcedPowerLimitWatts‚500.0) gpu.0.powerPercent‚24.607400000000002 gpu.0.smClock‚1320 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0Ý3}™ñ‚(:Ž ÔÐéÔ‹‘¨¼) proc.memory.percent‚18.41146945953369 cpu‚25.76787424307716 proc.cpu.threads‚17 network.recv‚ 1.079186e+06) disk./.usagePercent‚78.83131915480321 disk.sda2.out‚0( disk.nvme0c0n1.out‚73171.53173828125 disk.dm-1.out‚ 228.3828125 disk.drbd1260.in‚0 disk.sda.in‚0.796875$ memory_percent‚98.94270896911621% disk./.usageGB‚1174.6436233520508 disk.sda3.out‚ 228.3828125 disk.dm-0.in‚ 7653.4609375# disk.dm-0.out‚73171.53173828125# proc.memory.rssMB‚ 3016.53515625 network.sent‚ 4.430651e+06 disk.sda2.in‚0" disk.nvme0c0n1.in‚ 7653.4609375& disk.nvme0n1.out‚73171.53173828125 disk.dm-1.in‚ 0.6953125 disk.drbd1260.out‚ 0.4453125 disk.sda1.in‚ 0.1015625 disk.sda.out‚ 228.3828125 disk.sda3.in‚ 0.6953125 disk.nvme0n1.in‚ 7653.4609375 disk.sda1.out‚0' proc.memory.availableMB‚ 173.2265625V=‰žò‚(:“ ãÐéÔü¸Ý¹ gpu.0.gpu‚34.0 gpu.0.memory‚3, gpu.0.memoryAllocated‚16.718826293945312* gpu.0.memoryAllocatedBytes‚ 14361362432 gpu.0.temp‚31.0 gpu.0.powerWatts‚114.594' gpu.0.enforcedPowerLimitWatts‚500.0) gpu.0.powerPercent‚22.918799999999997 gpu.0.smClock‚1275 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0·Ýþšó‚(: ãÐéÔýÒ´¼ disk.sda.in‚0.796875 disk.sda.out‚ 228.86328125 disk.sda3.in‚ 0.6953125! disk.nvme0c0n1.in‚ 7840.796875& disk.nvme0n1.out‚73219.87939453125' proc.memory.availableMB‚ 173.2265625 network.sent‚ 4.436751e+06( disk.nvme0c0n1.out‚73219.87939453125 cpu‚25.766726401818858 proc.cpu.threads‚17 network.recv‚ 1.083559e+06) disk./.usagePercent‚78.83132325092376 disk.dm-1.in‚ 0.6953125 disk.sda3.out‚ 228.86328125 disk.nvme0n1.in‚ 7840.796875$ memory_percent‚98.94270896911621# proc.memory.rssMB‚ 3016.53515625) proc.memory.percent‚18.41146945953369$ disk./.usageGB‚1174.643684387207 disk.dm-0.in‚ 7840.796875 disk.dm-1.out‚ 228.86328125 disk.drbd1260.in‚0 disk.drbd1260.out‚ 0.44921875 disk.sda1.in‚ 0.1015625 disk.sda2.in‚0 disk.sda2.out‚0# disk.dm-0.out‚73219.87939453125 disk.sda1.out‚0ŽÂ×?žô‚(:“ òÐéÔ¸­¯º gpu.0.gpu‚27.0 gpu.0.memory‚3, gpu.0.memoryAllocated‚16.718826293945312* gpu.0.memoryAllocatedBytes‚ 14361362432 gpu.0.temp‚34.0 gpu.0.powerWatts‚141.115' gpu.0.enforcedPowerLimitWatts‚500.0) gpu.0.powerPercent‚28.223000000000003 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0XˆÒ•õ‚(:Š òÐéÔÝãܽ) disk./.usagePercent‚78.83161100339224 disk.sda3.in‚ 0.6953125 disk.drbd1260.out‚ 0.44921875# proc.memory.rssMB‚ 3016.53515625 cpu‚25.76549341467141 proc.cpu.threads‚17 disk.dm-1.out‚ 229.359375 network.sent‚ 4.442982e+06% disk./.usageGB‚1174.6479721069336 disk.sda3.out‚ 229.359375# disk.dm-0.out‚73606.34423828125 disk.drbd1260.in‚0 disk.sda1.out‚0 disk.sda.in‚0.796875( disk.nvme0c0n1.out‚73606.34423828125 disk.nvme0n1.in‚ 7840.796875 disk.sda1.in‚ 0.1015625 disk.sda.out‚ 229.359375 disk.sda2.in‚0 disk.sda2.out‚0$ memory_percent‚98.94330501556396( proc.memory.availableMB‚ 173.12890625! disk.nvme0c0n1.in‚ 7840.796875& disk.nvme0n1.out‚73606.34423828125 disk.dm-0.in‚ 7840.796875 disk.dm-1.in‚ 0.6953125) proc.memory.percent‚18.41146945953369 network.recv‚ 1.088261e+06 “W£2ö- _wandbruntime‚3265  _runtime‚3265Œ¼ÉöÇ÷Á trainer/global_step‚1899 % _timestamp‚1.7885041898327506e+09 _runtime‚3265.912178069 _step‚151 & lr-AdamW/pg1‚0.0011301815470377374 & lr-AdamW/pg2‚0.0011301815470377374*&Ø2ÌøÆ trainer/global_step‚1899 % _timestamp‚1.7885041898327506e+09 _runtime‚3265.912178069 _step‚151 & lr-AdamW/pg1‚0.0011301815470377374 & lr-AdamW/pg2‚0.0011301815470377374—ì?Ú2ù- _wandbruntime‚3265  _runtime‚3265év8 Ñ úÞO MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 PCDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated‚47689100 L?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚47663012 WEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 1880906638848 aMDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚156351602499584 _MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 1967027011072 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 248285184 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 320831488 @8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚895 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚17039360 PCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚47663012 \HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚150751007718736 E8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚87018655 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 231245824 YEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚152621678661192 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚26088 MFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚33 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 586907136 P@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 13568115712 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚3689 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚31686 D>DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 PCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚39355645 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚17039360 IDeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚155974559155200 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 VBDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated‚155974807440384 M=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 13568115712 YEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚155974807440384 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 152061440 P@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 13118524033 [IDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 1881137884672 M@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚87044745 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚26090 ^JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚158318930679296 SEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 384795648 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 UADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚152621435029484 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚31686 ]IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚154093669555712 \HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚154093652516352 TGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚47689100 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 301168640 ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 276264960 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚286 `LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚154093669555712 XJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 226592348 O?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 13807648768 trainer/global_step‚1899 _runtime‚3265.912579196 UGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 582240777 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚26088 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚2 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚24903680 IDeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚3689 ZFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚158318629510656 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 TDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 13807648768 VFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 13207863296 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚31931 `LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚150751024758096 TFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 599785472 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 UADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚155974559155200 WGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 13035530240 ZHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 1880906638848 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚22718488 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚2 PCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚39355643 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚22718490 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 QCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 243631708 G?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚286 TDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 13035530240 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 231245824 M@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚34571163 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚11853294 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 TGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚39355645 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 RBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 13807648768 N@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 248285184 QDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚34571784 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚619 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 _step‚152 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚26090 ^LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 1881137884672 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 586907136 eQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚156351627403264?`5¹é'ûãO IDeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚155974559155200 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 VBDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated‚155974807440384 M=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 13568115712 YEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚155974807440384 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 152061440 P@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 13118524033 [IDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 1881137884672 M@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚87044745 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚26090 ^JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚158318930679296 SEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 384795648 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 UADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚152621435029484 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚31686 ]IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚154093669555712 \HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚154093652516352 TGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚47689100 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 301168640 ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 276264960 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚286 `LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚154093669555712 XJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 226592348 O?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 13807648768 trainer/global_step‚1899 _runtime‚3265.912579196 UGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 582240777 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚26088 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚2 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚24903680 IDeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚3689 ZFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚158318629510656 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 TDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 13807648768 VFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 13207863296 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚31931 `LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚150751024758096 TFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 599785472 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 UADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚155974559155200 WGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 13035530240 ZHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 1880906638848 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚22718488 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚2 PCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚39355643 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚22718490 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 QCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 243631708 G?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚286 TDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 13035530240 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 231245824 M@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚34571163 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚11853294 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 TGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚39355645 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 RBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 13807648768 N@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 248285184 QDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚34571784 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚619 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 _step‚152 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚26090 ^LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 1881137884672 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 586907136 eQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚156351627403264 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 PCDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated‚47689100 L?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚47663012 WEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 1880906638848 aMDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚156351602499584 _MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 1967027011072 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 248285184 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 320831488 @8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚895 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚17039360 PCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚47663012 \HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚150751007718736 E8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚87018655 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 231245824 YEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚152621678661192 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚26088 MFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚33 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 586907136 P@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 13568115712 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚3689 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚31686 D>DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 PCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚39355645 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚17039360˜ºÑñ2ü- _wandbruntime‚3266  _runtime‚3266#$¥Ýî&ýèM cODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚156431222263296 _KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚156431197359616 WGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 13207863296 C6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚87063197 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚31931 K>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚34588812 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 152061440 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 WCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚156054276133888 F8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 231259136 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚648 QCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 384795648 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚609 OADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 248298496 ^JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚154172180764672 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚895 L>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 248298496 UCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 1881864110080 G:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚87063197 RBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 13807648768 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 599785472 K;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 13568115712 M=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 13807648768 WIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 599785472 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚895 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚895 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚609 NADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚39375600 VHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 226592556 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 BDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 13568115712 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚3689 aODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 1968306925568 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 PDeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚87089313 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚11859479 ]KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 1968028576768 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚895 YIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 13207863296 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚650 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚33 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 303252480 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24903680 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 243631916 S?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚152699233811422 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚286 T@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚156054276133888 NADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚47687597 TDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 13207863296 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚26114 XFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 1881864110080 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚31931 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚17039360 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 599785472 PBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 586907136 _step‚153 NADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚39375602 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚39375600 FDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 13118524033 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 REDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚47713711 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 582240777 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 % _timestamp‚1.7885041907391334e+09 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17039360 ZFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚154172163725312 OBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚34589462 REDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚11858831¯rRÁþíM NADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚47687597 TDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 13207863296 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚26114 XFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 1881864110080 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚31931 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚17039360 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 599785472 PBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 586907136 _step‚153 NADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚39375602 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚39375600 FDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 13118524033 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 REDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚47713711 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 582240777 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 % _timestamp‚1.7885041907391334e+09 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17039360 ZFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚154172163725312 OBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚34589462 REDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚11858831 cODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚156431222263296 _KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚156431197359616 WGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 13207863296 C6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚87063197 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚31931 K>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚34588812 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 152061440 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 WCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚156054276133888 F8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 231259136 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚648 QCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 384795648 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚609 OADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 248298496 ^JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚154172180764672 C;DeviceStatsMonitor.on_t?:ÜF2rain_batch_end/segment.all.allocated‚895 L>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 248298496 UCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 1881864110080 G:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚87063197 RBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 13807648768 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 599785472 K;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 13568115712 M=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 13807648768 WIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 599785472 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚895 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚895 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚609 NADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚39375600 VHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 226592556 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 BDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 13568115712 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚3689 aODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 1968306925568 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 PDeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚87089313 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚11859479 ]KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 1968028576768 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚895 YIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 13207863296 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚650 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚33 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 303252480 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24903680 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 243631916 S?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚152699233811422 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚286 T@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚156054276133888™tÍH»2ÿ- _wandbruntime‚3266  _runtime‚3266q„+ÁY€Ó /loss/FRt4/detection_loss‚1.6839805941942125 %loss/total_loss‚44.86359848524505 .loss/FRt1/detection_loss‚2.628552248243426 1loss/SRqqt4/assignment_loss‚1.077263021917038 1loss/FBt3/assignment_loss‚0.17489608586309813 1loss/SRqqt2/detection_loss‚1.8790719157610185 0loss/SRqqt3/detection_loss‚2.751893102358549 0loss/FRt2/assignment_loss‚3.7514601472096567 1loss/SRqqt1/detection_loss‚2.7985983213713004 1loss/SRqqt4/detection_loss‚0.9973424925288606 _runtime‚ 3266.82007656 2loss/SRqqt1/assignment_loss‚1.7803249289823133 .loss/FRt3/detection_loss‚2.075305800578049 /loss/FBt3/detection_loss‚0.9392875610467266 1loss/SRqqt3/assignment_loss‚2.150767377547872 .loss/FBt2/detection_loss‚2.253438614308834 epoch‚3 1loss/FBt4/assignment_loss‚0.12311216017741344 /loss/FBt1/detection_loss‚2.4219615198671818 /loss/FBt4/detection_loss‚2.1613614689558744 trainer/global_step‚1899 % _timestamp‚1.7885041907415807e+09 .loss/FRt2/detection_loss‚2.683875350615917 /loss/FRt3/assignment_loss‚2.498076181520115 0loss/FRt4/assignment_loss‚1.7492929613867472 1loss/FBt1/assignment_loss‚0.24438783899451266 1loss/FBt2/assignment_loss‚0.41185416512096823 _step‚154 /loss/FRt1/assignment_loss‚3.971371797264599 2loss/SRqqt2/assignment_loss‚1.6561228294307604¼å\^Ø % _timestamp‚1.7885041907415807e+09 .loss/FRt2/detection_loss‚2.683875350615917 /loss/FRt3/assignment_loss‚2.498076181520115 0loss/FRt4/assignment_loss‚1.7492929613867472 1loss/FBt1/assignment_loss‚0.24438783899451266 1loss/FBt2/assignment_loss‚0.41185416512096823 _step‚154 /loss/FRt1/assignment_loss‚3.971371797264599 2loss/SRqqt2/assignment_loss‚1.6561228294307604 /loss/FRt4/detection_loss‚1.6839805941942125 %loss/total_loss‚44.86359848524505 .loss/FRt1/detection_loss‚2.628552248243426 1loss/SRqqt4/assignment_loss‚1.077263021917038 1loss/FBt3/assignment_loss‚0.17489608586309813 1loss/SRqqt2/detection_loss‚1.8790719157610185 0loss/SRqqt3/detection_loss‚2.751893102358549 0loss/FRt2/assignment_loss‚3.7514601472096567 1loss/SRqqt1/detection_loss‚2.7985983213713004 1loss/SRqqt4/detection_loss‚0.9973424925288606 _runtime‚ 3266.82007656 2loss/SRqqt1/assignment_loss‚1.7803249289823133 .loss/FRt3/detection_loss‚2.075305800578049 /loss/FBt3/detection_loss‚0.9392875610467266 1loss/SRqqt3/assignment_loss‚2.150767377547872 .loss/FBt2/detection_loss‚2.253438614308834 epoch‚3 1loss/FBt4/assignment_loss‚0.12311216017741344 /loss/FBt1/detection_loss‚2.4219615198671818 /loss/FBt4/detection_loss‚2.1613614689558744 trainer/global_step‚1899š̸Âlž‚‚(:“ ÑéÔ¿©¼º gpu.0.gpu‚52.0 gpu.0.memory‚4, gpu.0.memoryAllocated‚16.718826293945312* gpu.0.memoryAllocatedBytes‚ 14361362432 gpu.0.temp‚34.0 gpu.0.powerWatts‚174.886' gpu.0.enforcedPowerLimitWatts‚500.0) gpu.0.powerPercent‚34.977199999999996 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0ÛqU@šƒ‚(: ÑéԈǼ disk.sda1.out‚0 disk.sda3.out‚ 229.70703125& disk.nvme0n1.out‚74049.65283203125# disk.dm-0.out‚74049.65283203125( proc.memory.availableMB‚ 172.46484375) proc.memory.percent‚18.41146945953369 cpu‚25.764764040873107 proc.cpu.threads‚17 disk.sda3.in‚ 0.6953125 disk.drbd1260.out‚0.453125 disk.sda1.in‚ 0.1015625 disk.sda2.in‚0! disk.nvme0c0n1.in‚ 7840.796875 disk.dm-1.out‚ 229.70703125# proc.memory.rssMB‚ 3016.53515625% disk./.usageGB‚1174.6488990783691 disk.sda.in‚0.796875 disk.dm-0.in‚ 7840.796875 disk.dm-1.in‚ 0.6953125$ memory_percent‚98.94735813140869 disk.sda2.out‚0 network.sent‚ 4.448097e+06 network.recv‚ 1.091694e+06) disk./.usagePercent‚78.83167321322307 disk.drbd1260.in‚0 disk.sda.out‚ 229.70703125( disk.nvme0c0n1.out‚74049.65283203125 disk.nvme0n1.in‚ 7840.796875 ¢´¨“„‚(:ˆ ÑéÔìǃ¹ gpu.0.gpu‚11.0 gpu.0.memory‚1, gpu.0.memoryAllocated‚16.718826293945312* gpu.0.memoryAllocatedBytes‚ 14361362432 gpu.0.temp‚34.0 gpu.0.powerWatts‚155.417' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚31.0834 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0w¬Èm–…‚(:‹ ÑéÔ†œ°» proc.cpu.threads‚17 network.sent‚ 4.534611e+06 disk.sda3.out‚ 230.1328125 disk.nvme0n1.in‚ 7840.796875 disk.dm-1.out‚ 230.1328125 disk.drbd1260.out‚0.453125 disk.sda1.in‚ 0.1015625 disk.sda1.out‚0$ memory_percent‚98.94444942474365 disk.sda3.in‚ 0.6953125# disk.dm-0.out‚74242.30517578125 disk.dm-1.in‚ 0.6953125 disk.sda.in‚0.796875( proc.memory.availableMB‚ 172.94140625# proc.memory.rssMB‚ 3016.53515625) proc.memory.percent‚18.41146945953369 cpu‚25.76396651643234 network.recv‚ 1.097485e+06 disk.sda2.in‚0! disk.nvme0c0n1.in‚ 7840.796875 disk.drbd1260.in‚0 disk.sda.out‚ 230.1328125) disk./.usagePercent‚78.83588018503323% disk./.usageGB‚1174.7115859985352 disk.sda2.out‚0( disk.nvme0c0n1.out‚74242.30517578125& disk.nvme0n1.out‚74242.30517578125 disk.dm-0.in‚ 7840.796875÷ž¡ž†‚(:“ ŸÑéÔºœÞº gpu.0.gpu‚27.0 gpu.0.memory‚3, gpu.0.memoryAllocated‚16.718826293945312* gpu.0.memoryAllocatedBytes‚ 14361362432 gpu.0.temp‚34.0 gpu.0.powerWatts‚131.874' gpu.0.enforcedPowerLimitWatts‚500.0) gpu.0.powerPercent‚26.374799999999997 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0A×#¬ ‡‚(:• ŸÑéÔÜì ¾( proc.memory.availableMB‚ 172.70703125 cpu‚25.763036038214544% disk./.usageGB‚1175.2761039733887 disk.sda2.in‚0 disk.dm-0.in‚ 7993.17578125# proc.memory.rssMB‚ 3016.53515625) proc.memory.percent‚18.41146945953369 proc.cpu.threads‚17 network.recv‚ 1.10213e+06 disk.sda3.in‚ 0.6953125 disk.sda3.out‚ 230.80078125 disk.drbd1260.in‚0 disk.sda1.out‚0 disk.sda.in‚0.796875 disk.sda.out‚ 230.80078125( disk./.usagePercent‚78.8737654599878# disk.nvme0c0n1.in‚ 7993.17578125( disk.nvme0c0n1.out‚74403.71435546875! disk.nvme0n1.in‚ 7993.17578125& disk.nvme0n1.out‚74403.71435546875 disk.dm-1.in‚ 0.6953125 disk.drbd1260.out‚ 0.45703125 disk.sda1.in‚ 0.1015625 network.sent‚ 4.540609e+06 disk.sda2.out‚0# disk.dm-0.out‚74403.71435546875 disk.dm-1.out‚ 230.80078125$ memory_percent‚98.94587993621826¼HÄ+“ˆ‚(:ˆ ®ÑéÔ™¨™¹ gpu.0.gpu‚28.0 gpu.0.memory‚2, gpu.0.memoryAllocated‚16.718826293945312* gpu.0.memoryAllocatedBytes‚ 14361362432 gpu.0.temp‚33.0 gpu.0.powerWatts‚138.423' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚27.6846 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚03Ëš‰‚(: ®ÑéÔÙœµ¼! disk.nvme0n1.in‚ 7993.17578125& disk.nvme0n1.out‚74857.47607421875 disk.dm-0.in‚ 7993.17578125 disk.drbd1260.in‚0 disk./.usageGB‚ 1175.28515625 disk.drbd1260.out‚ 0.45703125$ memory_percent‚98.94611835479736 proc.cpu.threads‚17) disk./.usagePercent‚78.87437296586654 disk.sda2.in‚0 disk.sda3.in‚ 0.6953125# disk.dm-0.out‚74857.47607421875 disk.sda1.out‚0 disk.sda.out‚ 231.2109375) proc.memory.percent‚18.41146945953369 cpu‚25.762338145559227 network.sent‚ 4.546828e+06# disk.nvme0c0n1.in‚ 7993.17578125 disk.dm-1.in‚ 0.6953125 disk.dm-1.out‚ 231.2109375 disk.sda1.in‚ 0.1015625 disk.sda.in‚0.796875( proc.memory.availableMB‚ 172.66796875# proc.memory.rssMB‚ 3016.53515625 network.recv‚ 1.106777e+06 disk.sda2.out‚0 disk.sda3.out‚ 231.2109375( disk.nvme0c0n1.out‚74857.47607421875± "Ü“Š‚(:ˆ ½ÑéÔäÃι gpu.0.gpu‚42.0 gpu.0.memory‚3, gpu.0.memoryAllocated‚16.718826293945312* gpu.0.memoryAllocatedBytes‚ 14361362432 gpu.0.temp‚34.0 gpu.0.powerWatts‚127.607' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚25.5214 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0Gú B—‹‚(:Œ ½ÑéÔ¿øˆ»# disk.nvme0c0n1.in‚ 7993.17578125 disk.dm-1.out‚ 231.5859375$ memory_percent‚98.94611835479736( proc.memory.availableMB‚ 172.66796875) proc.memory.percent‚18.41146945953369 cpu‚25.761564235302 proc.cpu.threads‚17# disk.dm-0.out‚75187.09326171875 disk.sda2.out‚0 disk.sda3.out‚ 231.5859375( disk.nvme0c0n1.out‚75187.09326171875& disk.nvme0n1.out‚75187.09326171875 disk.sda.in‚0.796875 disk.sda.out‚ 231.5859375 disk./.usageGB‚ 1175.34765625 disk.sda2.in‚0! disk.nvme0n1.in‚ 7993.17578125 disk.drbd1260.in‚0 disk.sda1.in‚ 0.1015625# proc.memory.rssMB‚ 3016.53515625 network.sent‚ 4.551952e+06 network.recv‚ 1.110388e+06 disk.sda3.in‚ 0.6953125 disk.dm-0.in‚ 7993.17578125 disk.dm-1.in‚ 0.6953125 disk.drbd1260.out‚ 0.46484375 disk.sda1.out‚0) disk./.usagePercent‚78.87856739330753Â\x“Œ‚(:ˆ ÌÑéÔôÞº gpu.0.gpu‚31.0 gpu.0.memory‚4, gpu.0.memoryAllocated‚16.718826293945312* gpu.0.memoryAllocatedBytes‚ 14361362432 gpu.0.temp‚33.0 gpu.0.powerWatts‚114.882' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚22.9764 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0¶ì?z¢‚(:— ÌÑéÔšöÉ»% disk./.usageGB‚1175.4729804992676# disk.nvme0c0n1.in‚ 7993.17578125 disk.drbd1260.in‚0 disk.drbd1260.out‚ 0.46484375 disk.sda.in‚0.796875# proc.memory.rssMB‚ 3016.53515625 cpu‚25.760620586661684) disk./.usagePercent‚78.88697800882993 disk.sda3.in‚ 0.6953125( disk.nvme0c0n1.out‚75540.58935546875! disk.nvme0n1.in‚ 7993.17578125 disk.dm-1.in‚ 0.6953125 proc.cpu.threads‚17 disk.sda2.in‚0 disk.sda2.out‚0 disk.sda.out‚ 232.05859375$ memory_percent‚98.94611835479736( proc.memory.availableMB‚ 172.66796875) proc.memory.percent‚18.41146945953369# disk.dm-0.out‚75540.58935546875 network.recv‚ 1.114058e+06 disk.sda3.out‚ 232.05859375& disk.nvme0n1.out‚75540.58935546875 disk.dm-0.in‚ 7993.17578125 disk.dm-1.out‚ 232.05859375 disk.sda1.in‚ 0.1015625 disk.sda1.out‚0 network.sent‚ 4.557081e+06­û…M2Ž- _wandbruntime‚3351  _runtime‚3351Kž»»Ä¾ % lr-AdamW/pg1‚0.001117158735187488 % lr-AdamW/pg2‚0.001117158735187488 trainer/global_step‚1949 $ _timestamp‚1.788504275808695e+09 _runtime‚3351.888386994 _step‚155‡Éà % lr-AdamW/pg1‚0.001117158735187488 % lr-AdamW/pg2‚0.001117158735187488 trainer/global_step‚1949 $ _timestamp‚1.788504275808695e+09 _runtime‚3351.888386994 _step‚155›5‹·n2‘- _wandbruntime‚3351  _runtime‚3351ís÷ä'’ÞO aMDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚160331345501184 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚31931 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚2 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 586907136 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚23293138 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚17039360 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚609 JBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚286 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚26090 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 G?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚286 _step‚156 P@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 13568115712 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 231245824 IDeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚3689 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚17039360 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚31686 ZFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚162348499602432 TDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 13807648768 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 IDeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 `LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚154593361485296 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚26090 L?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚40353493 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 231245824 OFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚1299 YIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 13207863296 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 599785472 G?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚609 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚607 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 \HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚158019212964352 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚12155638 WGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 12589930496 trainer/global_step‚1949 N@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 248285184 PCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚48895614 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 152061440 [KDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 13207863296 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 586907136 QDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚35449385 UADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚156511401298392 \HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚154593344445936 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚3689 ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 284653568 E=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚895 PCDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated‚48921702 eQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚160331370404864 O?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 13807648768 VFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 13207863296 WEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 1928808028672 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚31931 MFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚33 cQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 2017438754816 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 ]IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚158019230003712 YEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚158019212964352 UADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚159948020993024 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 UGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 582240777 `LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚158019230003712 @8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚895 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 @8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚895 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚609 H?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚1321 PCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚40353495 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚26088 TDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 13035530240 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 248285184 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚2 L?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚48895614 TGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚40353495 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚23293140 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 ^LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 1918283444804 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚286 R>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚159948020993024 WGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 13035530240 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚31686 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 320831488 BDeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 `LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚154593361485296 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚26090 L?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚40353493 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 231245824 OFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚1299 YIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 13207863296 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 599785472 G?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚609 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚607 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 \HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚158019212964352 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚12155638 WGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 12589930496 trainer/global_step‚1949 N@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 248285184 PCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚48895614 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 152061440 [KDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 13207863296 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 586907136 QDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚35449385 UADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚156511401298392 \HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚154593344445936 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚3689 ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 284653568 E=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚895 PCDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated‚48921702 eQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚160331370404864 O?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 13807648768 VFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 13207863296 WEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 1928808028672 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚31931 MFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚33 cQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 2017438754816 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 ]IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚158019230003712 YEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚158019212964352 UADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚159948020993024 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 UGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 582240777 `LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚158019230003712 @8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚895 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 @8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚895 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚609 H?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚1321 PCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚40353495 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚26088 TDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 13035530240 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 248285184 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚2 L?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚48895614 TGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚40353495 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚23293140 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 ^LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 1918283444804 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚286 R>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚159948020993024 WGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 13035530240 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocat¿ùêOžed‚0 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚31686 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 320831488 BDeviceStatsMonitor.on_train_batch_start/allocation.all.current‚26090 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 G?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚286 _step‚156 P@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 13568115712 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 231245824 IDeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚3689œ Ü•2”- _wandbruntime‚3353  _runtime‚3353?ž^ûî&•èM \JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 1929996759040 WCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚156589443712246 _runtime‚3353.044580511 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24903680 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚613 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚286 NADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚48946313 SEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 586907136 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 582240777 cODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚160410965264896 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 BDeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 REDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚23304631 REDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚12161576 FDeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚35466207 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 320831488 NADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚40373452 YGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 1929996759040 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 231259136 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 N>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 13118524033 \JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 1919235492406 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚895 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚611 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚17039360 trainer/global_step‚1949 G:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚89319765 RBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 13035530240 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚26114 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 152061440 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚609 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 599785472 G:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚89293649 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚33 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 VHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 226592556 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚2 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚26114 MDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚1299 XFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 1919008899850 RBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 13807648768 _step‚157 WCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚160027737971712 OADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 248298496 UEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 13035530240 [MDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚ 280445952 UEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 12589930496 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 UCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 1929765499904 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚895 ^JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚154670208219840 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚286 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 243631916 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚286 WCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚158097724173312 P@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 13807648768 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚17039360 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 XDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚162429096206336 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 C6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚89293649 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚31931 \HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚162429401555968 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 PDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 13568115712 K>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚89319765 REDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚48946313 OBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚35466820 QCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 384795648 ZFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚154670191180480 T@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚160027737971712 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚23304633 EDeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 248298496 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17039360 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1321 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚609 J=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚48920199 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 305349632 S?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚156589200080330 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚895 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚26116 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 231259136“/Køó&–íM ^JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚158097741212672 _KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚160410940361216 ]KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 2018155845120 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 REDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚23304631 REDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚12161576 FDeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚35466207 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 320831488 NADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚40373452 YGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 1929996759040 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 231259136 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 N>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 13118524033 \JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 1919235492406 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚895 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚611 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚17039360 trainer/global_step‚1949 G:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚89319765 RBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 13035530240 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚26114 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 152061440 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚609 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 599785472 G:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚89293649 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚33 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 VHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 226592556 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚2 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚26114 MDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚1299 XFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 1919008899850 RBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 13807648768 _step‚157 WCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚160027737971712 OADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 248298496 UEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 13035530240 [MDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚ 280445952 UEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 12589930496 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 UCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 1929765499904 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚895 ^JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚154670208219840 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚286 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 243631916 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚286 WCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚158097724173312 P@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 13807648768 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚17039360 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 XDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚162429096206336 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 C6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚89293649 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚31931 \HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚162429401555968 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 PDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 13568115712 K>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚89319765 REDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚48946313 OBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚35466820 QCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 384795648 ZFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚154670191180480 T@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚160027737971712 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚23304633 EDeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 248298496 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17039360 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1321 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚609 J=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚48920199 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 305349632 S?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚156589200080330 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚895 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚26116 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 231259136 \JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 1929996759040 WCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚156589443712246 _runtime‚3353.044580511 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24903680 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚613 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚286 NADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚48946313 SEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 586907136 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 582240777 cODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚160410965264896 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 BTžž‚(:“ ùÑéÔ‰”ȹ gpu.0.gpu‚25.0 gpu.0.memory‚3, gpu.0.memoryAllocated‚16.718826293945312* gpu.0.memoryAllocatedBytes‚ 14361362432 gpu.0.temp‚33.0 gpu.0.powerWatts‚136.178' gpu.0.enforcedPowerLimitWatts‚500.0) gpu.0.powerPercent‚27.235599999999998 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0q`N–Ÿ‚(:‹ ùÑéÔÒ£–» disk.sda1.in‚ 0.1015625 disk.sda1.out‚0# memory_percent‚98.9464521408081( proc.memory.availableMB‚ 172.61328125 disk.sda.in‚0.796875 disk.sda3.out‚ 233.5078125 disk.dm-0.in‚ 8116.390625 disk.drbd1260.in‚0) proc.memory.percent‚18.41146945953369 cpu‚25.75816054713413 network.recv‚ 1.128861e+06% disk./.usageGB‚1175.6977081298828 disk.sda.out‚ 233.5078125 disk.sda3.in‚ 0.6953125# proc.memory.rssMB‚ 3016.53515625 proc.cpu.threads‚17 network.sent‚ 4.65575e+06) disk./.usagePercent‚78.90205966867956 disk.sda2.out‚0! disk.nvme0c0n1.in‚ 8116.390625 disk.nvme0n1.in‚ 8116.390625# disk.dm-0.out‚76340.22900390625 disk.dm-1.in‚ 0.6953125 disk.drbd1260.out‚ 0.47265625 disk.sda2.in‚0( disk.nvme0c0n1.out‚76340.22900390625& disk.nvme0n1.out‚76340.22900390625 disk.dm-1.out‚ 233.5078125gU_’ ‚(:‡ ˆÒéÔۜ⹠gpu.0.gpu‚27.0 gpu.0.memory‚2, gpu.0.memoryAllocated‚16.718826293945312* gpu.0.memoryAllocatedBytes‚ 14361362432 gpu.0.temp‚33.0 gpu.0.powerWatts‚113.245' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚22.649 gpu.0.smClock‚1275 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0¼HÆל¡‚(:‘ ˆÒéÔ¼ network.recv‚ 1.132578e+06 disk.sda.in‚0.796875 disk.sda.out‚ 233.89453125 disk.dm-0.in‚ 8116.390625) proc.memory.percent‚18.41146945953369 disk.sda2.in‚0 disk.sda2.out‚0 disk.sda3.out‚ 233.89453125( disk.nvme0c0n1.out‚76745.97900390625& disk.nvme0n1.out‚76745.97900390625 disk.dm-1.in‚ 0.6953125 disk.drbd1260.in‚0) disk./.usagePercent‚78.90120281146245 disk.sda1.out‚0! disk.nvme0c0n1.in‚ 8116.390625 disk.nvme0n1.in‚ 8116.390625# disk.dm-0.out‚76745.97900390625 disk.dm-1.out‚ 233.89453125 disk.drbd1260.out‚ 0.47265625( proc.memory.availableMB‚ 172.36328125% disk./.usageGB‚1175.6849403381348 disk.sda1.in‚ 0.1015625 disk.sda3.in‚ 0.6953125$ memory_percent‚98.94797801971436# proc.memory.rssMB‚ 3016.53515625 cpu‚25.757455426912628 proc.cpu.threads‚17 network.sent‚ 4.661154e+06¡°X/(¢‚(:‡ —ÒéԬ๠gpu.0.gpu‚21.0 gpu.0.memory‚1, gpu.0.memoryAllocated‚16.718826293945312* gpu.0.memoryAllocatedBytes‚ 14361362432 gpu.0.temp‚31.0 gpu.0.powerWatts‚95.151' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚19.0302 gpu.0.smClock‚±] j1275 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0€š£‚(: —ÒéÔ§‰ª½ disk.dm-0.in‚ 8116.390625 disk.dm-1.out‚ 234.30859375' proc.memory.availableMB‚ 172.6015625 proc.cpu.threads‚17 network.sent‚ 4.667089e+06 network.recv‚ 1.137183e+06& disk.nvme0n1.out‚77050.60986328125# disk.dm-0.out‚77050.60986328125 disk.dm-1.in‚ 0.6953125 disk.drbd1260.in‚0 disk.sda1.in‚ 0.1015625# proc.memory.rssMB‚ 3016.53515625! disk.nvme0c0n1.in‚ 8116.390625( disk.nvme0c0n1.out‚77050.60986328125 disk.nvme0n1.in‚ 8116.390625 disk.sda.out‚ 234.30859375) proc.memory.percent‚18.41146945953369 cpu‚25.756470657119483) disk./.usagePercent‚78.90041789236247% disk./.usageGB‚1175.6732444763184 disk.sda2.in‚0 disk.sda3.in‚ 0.6953125 disk.drbd1260.out‚ 0.4765625 disk.sda1.out‚0 disk.sda.in‚0.796875$ memory_percent‚98.94652366638184 disk.sda2.out‚0 disk.sda3.out‚ 234.30859375íÏŸ¤‚(:” ¦ÒéÔ¸æû¹ disk.sda3.out‚ 234.77734375( disk.nvme0c0n1.out‚77491.93017578125 disk.sda1.out‚0 disk.sda.out‚ 234.77734375$ memory_percent‚98.94671440124512 cpu‚25.755776617102452 disk.sda2.in‚0 disk.sda2.out‚0 disk.sda3.in‚ 0.6953125# disk.nvme0c0n1.in‚ 8262.18359375 disk.dm-0.in‚ 8262.18359375# disk.dm-0.out‚77491.93017578125 disk.dm-1.out‚ 234.77734375 disk.sda1.in‚ 0.1015625 network.recv‚ 1.14205e+06) disk./.usagePercent‚78.82175343328632! disk.nvme0n1.in‚ 8262.18359375& disk.nvme0n1.out‚77491.93017578125 disk.dm-1.in‚ 0.6953125 disk.sda.in‚0.796875' proc.memory.availableMB‚ 172.5703125# proc.memory.rssMB‚ 3016.53515625 disk.drbd1260.in‚0 disk.drbd1260.out‚ 0.4765625) proc.memory.percent‚18.41146945953369 proc.cpu.threads‚17 network.sent‚ 4.673313e+06% disk./.usageGB‚1174.5010871887207#vø’¥‚(:‡ ¦ÒéÔ¯‚ƒº gpu.0.gpu‚35.0 gpu.0.memory‚4, gpu.0.memoryAllocated‚16.718826293945312* gpu.0.memoryAllocatedBytes‚ 14361362432 gpu.0.temp‚31.0 gpu.0.powerWatts‚93.804' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚18.7608 gpu.0.smClock‚1275 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0½‘½¹2¦- _wandbruntime‚3436  _runtime‚3436zžiŠÇ§Á & lr-AdamW/pg1‚0.0011037477477292047 & lr-AdamW/pg2‚0.0011037477477292047 trainer/global_step‚1999 % _timestamp‚1.7885043608044035e+09 _runtime‚3436.883130958 _step‚159ð¼mÌ¨Æ & lr-AdamW/pg1‚0.0011037477477292047 & lr-AdamW/pg2‚0.0011037477477292047 trainer/global_step‚1999 % _timestamp‚1.7885043608044035e+09 _runtime‚3436.883130958 _step‚159ŸX=02©- _wandbruntime‚3436  _runtime‚3436hêËã'ªÝO UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 586907136 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚31686 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚31686 [IDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 1976940664320 M@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚91505649 PCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚41351343 TFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 599785472 TDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 13035530240 ZHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 1965686394164 PCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚41351345 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚2 ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 276264960 `LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚158435698212496 UGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 582240777 TDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 13807648768 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 586907136 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚17039360 ZHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 1976709418496 H?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚1321 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 YEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚160401611199008 WGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 12589930496 R>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚163921482830848 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚26088 QCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 243631708 RBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 13807648768 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚2 UADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚163921482830848 JBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚609 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚17039360 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚605 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 E=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚895 P@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 13568115712 IDeviceStatsMonitor.on_train_batch_start/allocation.all.current‚26090 QDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚36326047 SEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 384795648 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 UADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚160401367567300 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 599785472 @8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚895 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚26090 cQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 2067669543936 trainer/global_step‚1999 M@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚36325440 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚23867788 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 XJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 226592348 [KDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 13207863296 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 231245824 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 231245824 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚607 G?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚609 VBDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated‚163921731116032 N@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 248285184 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 320831488 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 BDeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 E8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚91479559 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 248285184 TGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚50154304 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚286 _runtime‚3436.888487878 OFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚1299 C;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚895 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚609 _step‚160 ^JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚166378782950400 eQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚164311113406464 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 YEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚161944773412352 ZFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚166378481781760 YIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 13207863296 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 599785472 ]IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚161944790451712 `LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚161944790451712 IDeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚3689 L?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚50128216¸ „Yè'«âO YEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚161944773412352 ZFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚166378481781760 YIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 13207863296 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 599785472 ]IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚161944790451712 `LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚161944790451712 IDeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚3689 L?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚50128216 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 586907136 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚31686 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚31686 [IDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 1976940664320 M@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚91505649 PCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚41351343 TFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 599785472 TDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 13035530240 ZHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 1965686394164 PCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚41351345 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚2 ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 276264960 `LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚158435698212496 UGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 582240777 TDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 13807648768 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 586907136 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚17039360 ZHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 1976709418496 H?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚1321 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 YEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚160401611199008 WGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 12589930496 R>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚163921482830848 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚26088 QCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 243631708 RBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 13807648768 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚2 UADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚163921482830848 JBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚609 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚17039360 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚605 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 E=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚895 P@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 13568115712 IDeviceStatsMonitor.on_train_batch_start/allocation.all.current‚26090 QDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚36326047 SEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 384795648 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 UADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚160401367567300 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 599785472 @8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚895 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚26090 cQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 2067669543936 trainer/global_step‚1999 M@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚36325440 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚23867788 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 XJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 226592348 [KDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 13207863296 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 231245824 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 231245824 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚607 G?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚609 VBDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated‚163921731116032 N@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 248285184 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 320831488 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 BDeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 E8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚91479559 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 248285184 TGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚50154304 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚286 _runtime‚3436.888487878 OFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚1299 C;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚895 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚609 _step‚160 ^JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚166378782950400 eQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚164311113406464 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 ¤ÂÛ#2¬- _wandbruntime‚3438  _runtime‚3438̹J‡ƒ$­èM N>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 13118524033 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 599785472 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚31931 YGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 1977898148864 S?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚164000951511040 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 231259136 SEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 586907136 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1321 NADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚50178915 L>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 248298496 UEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 13035530240 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚286 VHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 226592556 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚2 \JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 1977898148864 _step‚161 OADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 248298496 G:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚91524101 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚2 \HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚166459385545728 [MDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚ 282543104 WCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚160479409981154 ZFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚158512527907680 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚895 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚12464462 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚895 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚609 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚26114 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚895 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 582240777 M=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 13807648768 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚41371300 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 231259136 ZFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚162023284621312 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 307446784 WIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 599785472 _runtime‚3438.461921317 WCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚162023284621312 FDeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 C6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚91524101 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 trainer/global_step‚1999 _KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚164390683362816 K;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 13568115712 REDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚50178915 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 320831488 XFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 1966638441558 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 243631916 PDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 13568115712 QCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 384795648 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚286 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚3689 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 ^JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚158512544947040 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚286 NADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚41371302 UCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 1977666889728 K>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚36343126 J=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚50152801 YIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 13207863296 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚17039360 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚33 RBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 13807648768 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚26114 S?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚160479166349238 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 PBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 586907136 K>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚91550217 cODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚164390708266496 WGDeviceStats}C“kMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 13207863296 G:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚91550217 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚31686 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚617 REDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚41371302 REDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚23879281 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚609:UAó&®íM [GDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚162023301660672 ^JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚162023301660672 XFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 1977666889728 XDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚166459078098944 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 ]KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 2068394736128 \JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 1966865034114 RBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 13035530240 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚31931 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚895 BDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 13568115712 QCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 384795648 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚286 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚3689 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 ^JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚158512544947040 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚286 NADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚41371302 UCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 1977666889728 K>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚36343126 J=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚50152801 YIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 13207863296 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚17039360 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚33 RBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 13807648768 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚26114 S?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚160479166349238 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 PBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 586907136 K>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚91550217 cODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚164390708266496 WGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 13207863296 G:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚91550217 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚31686 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚617 REDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚41371302 REDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚23879281 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚609 N>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 13118524033 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 599785472 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚31931 YGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 1977898148864 S?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚164000951511040 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 231259136 SEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 586907136 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1321 NADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚50178915 L>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 248298496 UEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 13035530240 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚286 VHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 226592556 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚2 \JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 1977898148864 _step‚161 OADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 248298496 G:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚91524101 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚2 \HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚166459385545728 [MDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚ 282543104 WCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚160479409981154 ZFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚158512527907680 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚895 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚12464462 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚895 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚609 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚26114 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚895 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 582240777 M=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 13807648768 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚41371300 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 231259136 ZFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚162023284621312 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 307446784 WIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 599785472 _runtime‚3438.461921317 WCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚162023284621312 FDeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 C6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚91524101 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 trainer/global_step‚1999 _KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚164390683362816 K;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 13568115712 REDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚50178915 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 320831488 XFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 1966638441558 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 243631916 PDeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 \HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚162278017900336 G?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚609 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 231245824 PCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚51360818 ZFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚170408405327360 ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 282556416 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 ZHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 2013315935872 aMDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚168290831504384 YEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚164291577467916 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 599785472 JBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚609 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚31931 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚2 WGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 13035530240 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚24442440 OFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚1299 E=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚895 C;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚895 L?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚42349193 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚12759453 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 152061440 `LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚162278034939696 N@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 248285184 ^JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚170408712787456 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 UGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 582240777 [KDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 13207863296 trainer/global_step‚2049 _runtime‚3527.120730493 SEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 384795648 O?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 13807648768 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚24442438 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚26090 VBDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated‚167895192953856 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 248285184 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚31931 H?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚1321 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 320831488 TGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚42349195 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 WGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 12589930496 _step‚164 WEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 2024610808320 R>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚167894944668672 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚17039360 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 MFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚33 G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚3689 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚31686 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚31686 QCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 243631708 RBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 13807648768 TFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 599785472 \HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚165870333860352 BDeviceStatsMonitor.on_train_batch_start/allocation.all.current‚26090 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚24903680aeQGMÃãO aMDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚168290831504384 YEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚164291577467916 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 599785472 JBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚609 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚31931 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚2 WGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 13035530'Ì€Ûœ%240 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚24442440 OFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚1299 E=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚895 C;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚895 L?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚42349193 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚12759453 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 152061440 `LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚162278034939696 N@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 248285184 ^JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚170408712787456 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 UGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 582240777 [KDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 13207863296 trainer/global_step‚2049 _runtime‚3527.120730493 SEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 384795648 O?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 13807648768 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚24442438 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚26090 VBDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated‚167895192953856 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 248285184 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚31931 H?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚1321 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 320831488 TGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚42349195 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 WGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 12589930496 _step‚164 WEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 2024610808320 R>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚167894944668672 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚17039360 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 MFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚33 G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚3689 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚31686 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚31686 QCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 243631708 RBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 13807648768 TFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 599785472 \HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚165870333860352 BDeviceStatsMonitor.on_train_batch_start/allocation.all.current‚26090 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚24903680 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 TGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚51386906 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 307460096 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 @8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚895 PCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚42349195 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚613 cQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 2117856379392 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 YEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚165870333860352 TDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 13035530240 M=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 13568115712 ]IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚165870350899712 `LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚165870350899712 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 JBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚286 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚3689 VFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 13207863296 IDeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 \HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚162278017900336 G?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚609 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 231245824 PCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚51360818 ZFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚170408405327360 ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 282556416 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 ZHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 2013315935872¤Öu÷2Ä- _wandbruntime‚3528  _runtime‚3528jT‡Tí&ÅçM YGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 2025799538688 NADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚51385403 L>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 248298496 \JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 2025799538688 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 320831488 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚895 NADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚42369152 T@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚167974661647360 PBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 586907136 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚17039360 XFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 2025568279552 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚895 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚31931 QCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 384795648 [GDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚165948862108672 OADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 248298496 C6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚93754553 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚26114 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 BDeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚37219357 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚644 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 WCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚164369376250062 ^JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚162354881674240 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 582240777 TDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 13207863296 ZFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚165948845069312 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚609 \HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚170489309756928 XDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚170489004407296 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚286 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 231259136 SEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 586907136 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚31686 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚12766070 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 ZFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚162354864634880 _step‚165 G:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚93780669 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚3689 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚609 WCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚165948845069312 N>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 13568115712 NADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚51411517 MDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚1299 RBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 13035530240 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚2 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24903680 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 YIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 13207863296 WGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 13207863296 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 599785472 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚609 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 231259136 WCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚167974661647360 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚646 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚31931 G:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚93754553 REDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚51411517 XFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 2014267983266 OBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚37220003 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚42369150 PDeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚93780669 cODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚168370451268096 M=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 13807648768 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚286 ^JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚165948862108672 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 REDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚12765426 VHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 226592556 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚26114 FDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 13118524033 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1321 trainer/global_step‚2049 UEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 13035530240 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚895 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚26116 UCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 2025568279552 REDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚42369152 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 243631916 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 EDeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚286 _KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚168370426364416 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚895æÐeò&ÆìM YGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 2025799538688 NADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚51385403 L>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 248298496 \JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 2025799538688 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 320831488 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚895 NADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚42369152 T@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚167974661647360 PBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 586907136 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚17039360 XFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 2025568279552 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚895 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚31931 QCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 384795648 [GDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚165948862108672 OADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 248298496 C6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚93754553 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚26114 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 BDeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚37219357 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚644 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 WCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚164369376250062 ^JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚162354881674240 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 582240777 TDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 13207863296 ZFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚165948845069312 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚609 \HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚170489309756928 XDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚170489004407296 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚286 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 231259136 SEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 586907136 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚31686 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚12766070 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 ZFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚162354864634880 _step‚165 G:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚93780669 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚3689 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚609 WCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚165948845069312 N>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 13568115712 NADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚51411517 MDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚1299 RBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 13035530240 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚2 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24903680 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 YIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 13207863296 WGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 13207863296 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 599785472 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚609 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 231259136 WCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚167974661647360 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚646 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚31931 G:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚93754553 REDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚51411517 XFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 2014267983266 OBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚37220003 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚42369150 PDeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚93780669 cODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚168370451268096 M=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 13807648768 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚286 ^JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚165948862108672 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 REDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚12765426 VHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 226592556 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚26114 FDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 13118524033 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1321 trainer/global_step‚2049 UEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 13035530240 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚895 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚26116 UCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 2025568279552 REDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚42369152 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 243631916 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 EDeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚286 _KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚168370426364416 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚895¥j¡ì½2Ç- _wandbruntime‚3528  _runtime‚3528VðlvTÈÎ trainer/global_step‚2049 epoch‚3 # _timestamp‚1.78850445275608e+09 1loss/SRqqt3/assignment_loss‚2.174656639557059 2loss/SRqqt4/assignment_loss‚1.0235843701616356 1loss/FBt1/assignment_loss‚0.22906233478498844 .loss/FBt1/detection_loss‚2.162800403104888 .loss/FBt4/detection_loss‚2.007728170264851 %loss/total_loss‚43.29337114019773 /loss/FRt3/assignment_loss‚1.969545172106835 _runtime‚ 3528.83530489 0loss/FRt2/assignment_loss‚3.1622298447291057 .loss/FRt4/detection_loss‚1.660336294848668 /loss/FRt4/assignment_loss‚1.718818758215223 0loss/FBt2/assignment_loss‚0.3425177198367324 /loss/FRt2/detection_loss‚2.6236285090446474 /loss/FBt3/detection_loss‚0.9859021301344624 0loss/FRt1/assignment_loss‚3.5951321066402997 .loss/FRt3/detection_loss‚2.341604621389941 2loss/SRqqt1/assignment_loss‚1.9292834714421483 2loss/SRqqt2/assignment_loss‚1.4145270373905077 0loss/FBt4/assignment_loss‚0.3534897608766931 /loss/FRt1/detection_loss‚2.6828675406878113 1loss/SRqqt1/detection_loss‚2.4739254087698264 1loss/SRqqt3/detection_loss‚2.8110804630488886 /loss/FBt2/detection_loss‚2.5833733528852463 _step‚166 /loss/FBt3/assignment_loss‚0.083293632492974 1loss/SRqqt2/detection_loss‚1.9652404863387347 0loss/SRqqt4/detection_loss‚0.998742911445557±‡;YÉÓ %loss/total_loss‚43.29337114019773 /loss/FRt3/assignment_loss‚1.969545172106835 _runtime‚ 3528.83530489 0loss/FRt2/assignment_loss‚3.1622298447291057 .loss/FRt4/detection_loss‚1.660336294848668 /loss/FRt4/assignment_loss‚1.718818758215223 0loss/FBt2/assignment_loss‚0.3425177198367324 /loss/FRt2/detection_loss‚2.6236285090446474 /loss/FBt3/detection_loss‚0.9859021301344624 0loss/FRt1/assignment_loss‚3.5951321066402997 .loss/FRt3/detection_loss‚2.341604621389941 2loss/SRqqt1/assignment_loss‚1.9292834714421483 2loss/SRqqt2/assignment_loss‚1.4145270373905077 0loss/FBt4/assignment_loss‚0.3534897608766931 /loss/FRt1/detection_loss‚2.6828675406878113 1loss/SRqqt1/detection_loss‚2.4739254087698264 1loss/SRqqt3/detection_loss‚2.8110804630488886 /loss/FBt2/detection_loss‚2.5833733528852463 _step‚166 /loss/FBt3/assignment_loss‚0.083293632492974 1loss/SRqqt2/detection_loss‚1.9652404863387347 0loss/SRqqt4/detection_loss‚0.998742911445557 trainer/global_step‚2049 epoch‚3 # _timestamp‚1.78850445275608e+09 1loss/SRqqt3/assignment_loss‚2.174656639557059 2loss/SRqqt4/assignment_loss‚1.0235843701616356 1loss/FBt1/assignment_loss‚0.22906233478498844 .loss/FBt1/detection_loss‚2.162800403104888 .loss/FBt4/detection_loss‚2.007728170264851¦At.<Ê‚(:“ ÓéÔè‹Ýº gpu.0.gpu‚29.0 gpu.0.memory‚3, gpu.0.memoryAllocated‚16.718826293945312* gpu.0.memoryAllocatedBytes‚ 14361362432 gpu.0.temp‚34.0 gpu.0.powerWatts‚148.156' gpu.0.enforcedPowerLimitWatts‚500.0) gpu.0.powerPercent‚29.631200000000003 gpu.0.smClock‚1410Ûmb gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0EùÈlË‚(:„ ÓéÔ’þÖ¼ disk.sda2.in‚0! disk.nvme0c0n1.in‚ 8415.453125% disk.nvme0n1.out‚79523.4130859375$ memory_percent‚98.94661903381348' proc.memory.availableMB‚ 172.5859375 disk.dm-1.in‚ 0.6953125 disk.drbd1260.out‚ 0.4921875 disk.sda.out‚ 237.828125 disk.sda3.in‚ 0.6953125 disk.sda3.out‚ 237.828125 disk.nvme0n1.in‚ 8415.453125 proc.cpu.threads‚17 disk.sda.in‚0.796875$ disk./.usageGB‚1175.410385131836" disk.dm-0.out‚79523.4130859375 disk.drbd1260.in‚0 disk.sda1.in‚ 0.1015625 disk.sda1.out‚0 disk.sda2.out‚0' disk.nvme0c0n1.out‚79523.4130859375 network.recv‚ 1.17187e+06 disk.dm-0.in‚ 8415.453125# proc.memory.rssMB‚ 3016.53515625) proc.memory.percent‚18.41146945953369 cpu‚25.753336637725965 network.sent‚ 4.792325e+06) disk./.usagePercent‚78.88277718120058 disk.dm-1.out‚ 237.8281256oÙA“Ì‚(:ˆ žÓéÔÓõ¶¹ gpu.0.gpu‚24.0 gpu.0.memory‚3, gpu.0.memoryAllocated‚16.718826293945312* gpu.0.memoryAllocatedBytes‚ 14361362432 gpu.0.temp‚32.0 gpu.0.powerWatts‚110.208' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚22.0416 gpu.0.smClock‚1275 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚03¨ÊšÍ‚(: žÓéÔŸÒ½ disk.dm-1.out‚ 238.19921875$ memory_percent‚98.94685745239258# proc.memory.rssMB‚ 3016.53515625) disk./.usagePercent‚78.88277462112524% disk./.usageGB‚1175.4103469848633 disk.sda.in‚0.796875" disk.dm-0.out‚79591.8076171875 cpu‚25.75290014823957 proc.cpu.threads‚17 disk.drbd1260.in‚0 disk.drbd1260.out‚ 0.4921875 disk.sda1.out‚0 disk.sda2.out‚0' disk.nvme0c0n1.out‚79591.8076171875! disk.nvme0n1.in‚ 8548.82421875% disk.nvme0n1.out‚79591.8076171875 disk.dm-1.in‚ 0.6953125 network.sent‚ 4.88541e+06 disk.sda3.in‚ 0.6953125# disk.nvme0c0n1.in‚ 8548.82421875 disk.dm-0.in‚ 8548.82421875& proc.memory.availableMB‚ 172.546875) proc.memory.percent‚18.41146945953369 network.recv‚ 1.185629e+06 disk.sda1.in‚ 0.1015625 disk.sda.out‚ 238.19921875 disk.sda2.in‚0 disk.sda3.out‚ 238.19921875±µà–“΂(:ˆ ­ÓéÔ×®µ¹ gpu.0.gpu‚18.0 gpu.0.memory‚1, gpu.0.memoryAllocated‚16.718826293945312* gpu.0.memoryAllocatedBytes‚ 14361362432 gpu.0.temp‚32.0 gpu.0.powerWatts‚107.238' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚21.4476 gpu.0.smClock‚1275 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0Yæ(œÏ‚(:‘ ­Óéԛܻ disk.dm-1.in‚ 0.6953125 disk.drbd1260.out‚ 0.49609375# proc.memory.rssMB‚ 3016.53515625% disk./.usageGB‚1175.4103469848633 disk.sda2.in‚0 disk.sda3.out‚ 238.734375# disk.nvme0c0n1.in‚ 8548.82421875( disk.nvme0c0n1.out‚79656.11181640625! disk.nvme0n1.in‚ 8548.82421875 disk.sda1.in‚ 0.1015625 disk.sda.in‚0.796875 network.sent‚ 4.888565e+06 network.recv‚ 1.189151e+06) disk./.usagePercent‚78.88277462112524 disk.sda3.in‚ 0.6953125 disk.dm-0.in‚ 8548.82421875 disk.drbd1260.in‚0 disk.sda.out‚ 238.734375$ memory_percent‚98.94535541534424 disk.sda2.out‚0 disk.dm-1.out‚ 238.734375 disk.sda1.out‚0( proc.memory.availableMB‚ 172.79296875) proc.memory.percent‚18.41146945953369 cpu‚25.752710293322483 proc.cpu.threads‚17& disk.nvme0n1.out‚79656.11181640625# disk.dm-0.out‚79656.111816406250~_<‘Ђ(:† ¼ÓéÔÈäº gpu.0.gpu‚11.0 gpu.0.memory‚1, gpu.0.memoryAllocated‚16.718826293945312* gpu.0.memoryAllocatedBytes‚ 14361362432 gpu.0.temp‚34.0 gpu.0.powerWatts‚114.86' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚22.972 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0ij¢Ñ‚(:— ¼ÓéÔ¡ÿ¾»% disk./.usageGB‚1175.0706939697266 disk.sda2.in‚0# disk.nvme0c0n1.in‚ 8548.82421875( disk.nvme0c0n1.out‚80043.72802734375 disk.dm-0.in‚ 8548.82421875 disk.sda1.in‚ 0.1015625 disk.sda2.out‚0& disk.nvme0n1.out‚80043.72802734375 disk.sda.in‚0.796875$ memory_percent‚98.94535541534424( proc.memory.availableMB‚ 172.79296875 network.recv‚ 1.192674e+06 disk.dm-1.in‚ 0.6953125 disk.drbd1260.in‚0 disk.sda.out‚ 239.23046875# proc.memory.rssMB‚ 3016.53515625) proc.memory.percent‚18.41146945953369 proc.cpu.threads‚17 disk.sda3.in‚ 0.6953125 disk.sda3.out‚ 239.23046875! disk.nvme0n1.in‚ 8548.82421875# disk.dm-0.out‚80043.72802734375 disk.dm-1.out‚ 239.23046875 disk.drbd1260.out‚ 0.49609375 disk.sda1.out‚0 cpu‚25.752555771049547 network.sent‚ 4.893802e+06) disk./.usagePercent‚78.859980222291554Ø…‚’Ò‚(:‡ ËÓéÔ¹¢ä¹ gpu.0.gpu‚15.0 gpu.0.memory‚2, gpu.0.memoryAllocated‚16.718826293945312* gpu.0.memoryAllocatedBytes‚ 14361362432 gpu.0.temp‚36.0 gpu.0.powerWatts‚165.505' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚33.101 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0&ØW®™Ó‚(:Ž ËÓéÔßÛÁº cpu‚25.752354051212414 disk.dm-1.out‚ 239.64453125 network.recv‚ 1.197162e+06 disk.sda3.in‚ 0.6953125 disk.sda3.out‚ 239.64453125 disk.dm-0.in‚ 8548.82421875# disk.dm-0.out‚80726.84716796875 disk.sda1.out‚0# memory_percent‚98.9469051361084) disk./.usagePercent‚78.84054899443518% disk./.usageGB‚1174.7811546325684( disk.nvme0c0n1.out‚80726.84716796875 disk.sda.out‚ 239.64453125 disk.sda2.in‚0 proc.cpu.threads‚17 disk.dm-1.in‚ 0.6953125 network.sent‚ 4.899796e+06# disk.nvme0c0n1.in‚ 8548.82421875& disk.nvme0n1.out‚80726.84716796875 disk.sda2.out‚0# proc.memory.rssMB‚ 3016.53515625! disk.nvme0n1.in‚ 8548.82421875 disk.drbd1260.in‚0 disk.drbd1260.out‚0.5 disk.sda1.in‚ 0.1015625 disk.sda.in‚0.796875' proc.memory.availableMB‚ 172.5390625) proc.memory.percent‚18.41146945953369u8¦'2Ô- _wandbruntime‚3603  _runtime‚3603œüëiÆÕÀ & lr-AdamW/pg2‚0.0010758063497415596 trainer/global_step‚2099 $ _timestamp‚1.788504527767204e+09 _runtime‚3603.846348433 _step‚167 & lr-AdamW/pg1‚0.0010758063497415596†çËÖÅ _runtime‚3603.846348433 _step‚167 & lr-AdamW/pg1‚0.0010758063497415596 & lr-AdamW/pg2‚0.0010758063497415596 trainer/global_step‚2099 $ _timestamp‚1.788504527767204e+09§O?m2×- _wandbruntime‚3603  _runtime‚3603¾´‰¨ä'ØÞO C;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚895 PCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚43347045 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚25017090 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 152061440 TDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 13807648768 UGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 582240777 RBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 13807648768 TFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 599785472 IDeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚3689 ]IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚169795911347712 @8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚895 [IDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 2072743443968 QDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚38078064 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚2 P@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 13118524033 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚609 trainer/global_step‚2099 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚286 _MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 2167806612992 BDeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚171868406506496 _step‚168 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚26090 ^LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 2072743443968 WGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 12589930496 YIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 13207863296 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 _runtime‚3603.847974398 UADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚171868406506496 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚31931 % _timestamp‚1.7885045277688897e+09 VBDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated‚171868654791680 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 599785472 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 YEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚168181543736824 E8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚95940463 L?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚52593420 `LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚169795911347712 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚26090 OFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚1299 \HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚169795894308352 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 586907136 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚31686 O?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 13807648768 @8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚895 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 D>DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 [KDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 13207863296 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚26088 YEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚169795894308352 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 231245824 IDeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚31686 WGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 13035530240 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 XJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 226592348 JBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚609 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚17039360 PCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚52593420 \HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚166120354627536ðÌYHé'ÙãO SEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 384795648 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 G?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚609 E=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚895 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 586907136 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚17039360 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 299071488 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 MFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚33 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚2 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚594 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚24903680 aMDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚172270574505984 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 320831488 VFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 13207863296 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚31686 WGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 13035530240 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 XJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 226592348 JBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚609 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚17039360 PCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚52593420 \HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚166120354627536 C;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚895 PCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚43347045 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚25017090 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 152061440 TDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 13807648768 UGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 582240777 RBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 13807648768 TFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 599785472 IDeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚3689 ]IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚169795911347712 @8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚895 [IDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 2072743443968 QDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚38078064 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚2 P@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 13118524033 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚609 trainer/global_step‚2099 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚286 _MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 2167806612992 BDeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚171868406506496 _step‚168 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚26090 ^LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 2072743443968 WGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 12589930496 YIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 13207863296 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 _runtime‚3603.847974398 UADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚171868406506496 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚31931 % _timestamp‚1.7885045277688897e+09 VBDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated‚171868654791680 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 599785472 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 YEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚168181543736824 E8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚95940463 L?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚52593420 `LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚169795911347712 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚26090 OFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚1299 \HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚169795894308352 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 586907136 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚31686 O?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 13807648768 @8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚895 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 D>DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 [KDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 13207863296 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚26088 YEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚169795894308352 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 231245824 IDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 13568115712 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚13066930 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 324224000 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 WIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 599785472 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚26114 UCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 2073469669376 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚286 G:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚95985005 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24903680 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚895 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚609 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 582240777 YIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 13207863296 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 599785472 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚26114 K>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚38094908 MDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚1299 _KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚172350169366016 aODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 2169093441536 C6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚95985005 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 231259136 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚17039360 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 WCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚168259342518970 N>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 13118524033 UEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 13035530240 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚605 \HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚174519287711232 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 KEDeviceStatsMonitor.on_tram™[w in_batch_end/reserved_bytes.small_pool.freed‚0 P6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚895 K>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚96011121 ZFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚166197201362080 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚31931 WCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚171948123485184 [MDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚ 299320320 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚286 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 P@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 13807648768 T@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚171948123485184 REDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚52644119 OBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚38095513 REDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚25028581 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 UEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 12589930496 VHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 226592556 TDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 13207863296 L>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 248298496 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚609 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚31931 PBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 586907136 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 ^JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚166197218401440 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 152061440 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚17039360 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 320831488 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 E6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚895 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 _step‚169 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 BDeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 248298496 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚609 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚31931 PBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 586907136 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 ^JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚166197218401440 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 152061440 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚17039360 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 320831488 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 E6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚895 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 _step‚169 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 BDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 13568115712 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚13066930 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 324224000 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 WIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 599785472 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚26114 UCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 2073469669376 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚286 G:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚95985005 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24903680 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚895 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚609 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 582240777 YIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 13207863296 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 599785472 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚26114 K>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚38094908 MDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚1299 _KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚172350169366016 aODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 2169093441536 C6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚95985005 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 231259136 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚17039360 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 WCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚168259342518970 N>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 13118524033 UEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 13035530240 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚605 \HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚174519287711232 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 P6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚895 K>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚96011121 ZFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚166197201362080 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚31931©HI9v2Ý- _wandbruntime‚3604  _runtime‚3604žTVÞÐ /loss/FRt2/assignment_loss‚3.512601401432451 0loss/FRt4/assignment_loss‚1.7373680715012338 /loss/SRqqt3/detection_loss‚2.21918035358996 .loss/FBt3/detection_loss‚0.593447618406279 _runtime‚3604.791381736 2loss/SRqqt4/assignment_loss‚0.9096585482567532 /loss/FRt2/detection_loss‚2.6135788412697343 1loss/SRqqt4/detection_loss‚1.0379961372871473 2loss/SRqqt1/assignment_loss‚1.8828873896598817 2loss/SRqqt2/assignment_loss‚1.5906433914052813 /loss/FRt1/detection_loss‚2.4772187062673856 /loss/FRt3/detection_loss‚2.0529696158615938 /loss/FBt1/detection_loss‚2.6496068382263185 epoch‚3 0loss/FRt1/assignment_loss‚3.7149616633142744 1loss/FBt3/assignment_loss‚0.09274481469918604 .loss/FBt2/detection_loss‚2.614459970167705 % _timestamp‚1.7885045287127836e+09 ,loss/FRt4/detection_loss‚1.9811200307336 1loss/SRqqt2/detection_loss‚2.1038373493050275 &loss/total_loss‚42.990847831403016 trainer/global_step‚2099 0loss/FRt3/assignment_loss‚2.4357583070375832 /loss/FBt2/assignment_loss‚0.231467816127611 .loss/FBt4/detection_loss‚1.851413981481032 _step‚170 0loss/SRqqt3/assignment_loss‚1.61189172718976 1loss/FBt1/assignment_loss‚0.17417655823048336 1loss/FBt4/assignment_loss‚0.23940216902412265 0loss/SRqqt1/detection_loss‚2.662456530928612HIçÝ[ßÕ /loss/FRt2/assignment_loss‚3.512601401432451 0loss/FRt4/assignment_loss‚1.7373680715012338 /loss/SRqqt3/detection_loss‚2.21918035358996 .loss/FBt3/detection_loss‚0.593447618406279 _runtime‚3604.791381736 2loss/SRqqt4/assignment_loss‚0.9096585482567532 /loss/FRt2/detection_loss‚2.6135788412697343 1loss/SRqqt4/detection_loss‚1.0379961372871473 2loss/SRqqt1/assignment_loss‚1.8828873896598817 2loss/SRqqt2/assignment_loss‚1.5906433914052813 /loss/FRt1/detection_loss‚2.4772187062673856 /loss/FRt3/detection_loss‚2.0529696158615938 /loss/FBt1/detection_loss‚2.6496068382263185 epoch‚3 0loss/FRt1/assignment_loss‚3.7149616633142744 1loss/FBt3/assignment_loss‚0.09274481469918604 .loss/FBt2/detection_loss‚2.614459970167705 % _timestamp‚1.7885045287127836e+09 ,loss/FRt4/detection_loss‚1.9811200307336 1loss/SRqqt2/detection_loss‚2.1038373493050275 &loss/total_loss‚42.990847831403016 trainer/global_step‚2099 0loss/FRt3/assignment_loss‚2.4357583070375832 /loss/FBt2/assignment_loss‚0.231467816127611 .loss/FBt4/detection_loss‚1.851413981481032 _step‚170 0loss/SRqqt3/assignment_loss‚1.61189172718976 1loss/FBt1/assignment_loss‚0.17417655823048336 1loss/FBt4/assignment_loss‚0.23940216902412265 0loss/SRqqt1/detection_loss‚2.662456530928612ªy>Ã7žà‚(:“ ÚÓéÔ–ð̺ gpu.0.gpu‚48.0 gpu.0.memory‚3, gpu.0.memoryAllocated‚16.718826293945312* gpu.0.memoryAllocatedBytes‚ 14361362432 gpu.0.temp‚35.0 gpu.0.powerWatts‚161.565' gpu.0.enforcedPowerLimitWatts‚500.0) gpu.0.powerPercent‚32.312999999999995 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0hLInšá‚(: ÚÓéÔ¯ Þº disk.sda3.out‚ 239.97265625( disk.nvme0c0n1.out‚80919.37841796875# disk.dm-0.out‚80919.37841796875( proc.memory.availableMB‚ 172.76171875# proc.memory.rssMB‚ 3016.53515625 network.sent‚ 4.907716e+06% disk./.usageGB‚1174.8000259399414 disk.sda2.out‚0& disk.nvme0n1.out‚80919.37841796875 disk.dm-1.out‚ 239.97265625 disk.drbd1260.in‚0 disk.sda.in‚0.796875 cpu‚25.75206818340212) disk./.usagePercent‚78.84181546370706 disk.sda2.in‚0 disk.sda3.in‚ 0.6953125 disk.dm-0.in‚ 8712.55078125 disk.dm-1.in‚ 0.6953125 disk.drbd1260.out‚0.5 disk.sda1.out‚0$ memory_percent‚98.94554615020752 network.recv‚ 1.203608e+06# disk.nvme0c0n1.in‚ 8712.55078125! disk.nvme0n1.in‚ 8712.55078125 disk.sda1.in‚ 0.1015625 disk.sda.out‚ 239.97265625) proc.memory.percent‚18.41146945953369 proc.cpu.threads‚17Žv2’â‚(:‡ éÓéÔ Žîº gpu.0.gpu‚1.0 gpu.0.memory‚0, gpu.0.memoryAllocated‚16.718826293945312* gpu.0.memoryAllocatedBytes‚ 14361362432 gpu.0.temp‚34.0 gpu.0.powerWatts‚114.193' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚22.8386 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0Ìo6—“ã‚(:ˆ éÓéÔÂÞ¥½$ memory_percent‚98.94704818725586& proc.memory.availableMB‚ 172.515625 network.recv‚ 1.209864e+06) disk./.usagePercent‚78.84178269474268# disk.nvme0c0n1.in‚ 8712.55078125$ disk.nvme0n1.out‚81046.822265625 disk.dm-0.in‚ 8712.55078125 disk.dm-1.in‚ 0.6953125 disk.sda1.in‚ 0.1015625 disk.sda1.out‚0 disk.sda.in‚0.796875 disk.sda.out‚ 240.359375& disk.nvme0c0n1.out‚81046.822265625 disk.drbd1260.in‚0# proc.memory.rssMB‚ 3016.53515625 proc.cpu.threads‚17 disk.sda2.in‚0 disk.sda3.out‚ 240.359375! disk.dm-0.out‚81046.822265625 disk.dm-1.out‚ 240.359375 disk.drbd1260.out‚ 0.50390625) proc.memory.percent‚18.41146945953369 cpu‚25.75168245763078 network.sent‚ 4.994437e+06% disk./.usageGB‚1174.7995376586914 disk.sda2.out‚0 disk.sda3.in‚ 0.6953125! disk.nvme0n1.in‚ 8712.55078125þÐrážä‚(:“ øÓéÔ÷ùÓ» gpu.0.gpu‚26.0 gpu.0.memory‚2, gpu.0.memoryAllocated‚16.718826293945312* gpu.0.memoryAllocatedBytes‚ 14361362432 gpu.0.temp‚34.0 gpu.0.powerWatts‚133.457' gpu.0.enforcedPowerLimitWatts‚500.0) gpu.0.powerPercent‚26.691399999999998 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0¿í‹å‚(:€ øÓéÔÊž•¿ cpu‚25.75158081248092 network.recv‚ 1.219716e+06# disk.nvme0c0n1.in‚ 8712.55078125& disk.nvme0c0n1.out‚81195.435546875! disk.dm-0.out‚81195.435546875 disk.sda.out‚240.875$ memory_percent‚98.94716739654541) proc.memory.percent‚18.41146945953369 disk.sda3.in‚ 0.6953125$ disk.nvme0n1.out‚81195.435546875 disk.drbd1260.in‚0 disk.drbd1260.out‚ 0.50390625 disk.sda1.out‚0( proc.memory.availableMB‚ 172.49609375 proc.cpu.threads‚17 network.sent‚ 5.002643e+06$ disk./.usageGB‚1174.799415588379 disk.sda2.out‚0! disk.nvme0n1.in‚ 8712.55078125 disk.dm-1.in‚ 0.6953125 disk.sda.in‚0.796875) disk./.usagePercent‚78.84177450250158 disk.sda2.in‚0 disk.sda3.out‚240.875 disk.dm-0.in‚ 8712.55078125 disk.dm-1.out‚240.875 disk.sda1.in‚ 0.1015625# proc.memory.rssMB‚ 3016.53515625¶X嬒æ‚(:‡ ‡ÔéÔÑïùº gpu.0.gpu‚34.0 gpu.0.memory‚4, gpu.0.memoryAllocated‚16.718826293945312* gpu.0.memoryAllocatedBytes‚ 14361362432 gpu.0.temp‚31.0 gpu.0.powerWatts‚94.201' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚18.8402 gpu.0.smClock‚1275 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0[ R±ç‚(:‚ ‡ÔéÔù×ì½ disk.sda2.in‚0& disk.nvme0c0n1.out‚81388.423828125 disk.drbd1260.in‚0 disk.sda.in‚0.796875 disk.sda.out‚ 241.328125# memory_percent‚98.9471435546875 cpu‚25.750986583999154 network.sent‚ 5.011743e+06) disk./.usagePercent‚78.84178474280296! disk.nvme0n1.in‚ 8712.55078125! disk.dm-0.out‚81388.423828125 disk.drbd1260.out‚ 0.5078125! proc.memory.availableMB‚172.5# proc.memory.rssMB‚ 3016.53515625) proc.memory.percent‚18.41146945953369 network.recv‚ 1.225678e+06% disk./.usageGB‚1174.7995681762695 disk.sda2.out‚0 disk.sda3.in‚ 0.6953125 disk.sda3.out‚ 241.328125# disk.nvme0c0n1.in‚ 8712.55078125 disk.dm-1.out‚ 241.328125 disk.sda1.in‚ 0.1015625$ disk.nvme0n1.out‚81388.423828125 disk.dm-1.in‚ 0.6953125 disk.sda1.out‚0 proc.cpu.threads‚17 disk.dm-0.in‚ 8712.55078125½‘¥‚2è- _wandbruntime‚3673  _runtime‚3673à ±ÇéÁ trainer/global_step‚2149 % _timestamp‚1.7885045975060296e+09 _runtime‚3673.585172387 _step‚171 & lr-AdamW/pg1‚0.0010612992683543686 & lr-AdamW/pg2‚0.0010612992683543686’æV-ÌêÆ & lr-AdamW/pg1‚0.0010612992683543686 & lr-AdamW/pg2‚0.0010612992683543686 trainer/global_step‚2149 % _timestamp‚1.7885045975060296e+09 _runtime‚3673.585172387 _step‚171«Éæ<È2ë- _wandbruntime‚3673  _runtime‚3673‰¹Ø%ìÜO H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚31686 cQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 2218231675392 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 `LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚169962708394096 G?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚286 PCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚44344895 G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚3689 PCDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated‚53852110 ]IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚173721471795712 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚26090 ^JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚178468574086656 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚31931 YEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚173721454756352 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 231245824 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 231245824 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚26088 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚24903680 M@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚38953574 UGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 582240777 \HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚173721454756352 ^LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 2120644833792 YIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 13207863296 VFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 13207863296 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 D>DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 TFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 599785472 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚286 JBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚286 WEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 2120413587968 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚17039360 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚3689 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚13362437 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 XJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 226592348 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 599785472 trainer/global_step‚2149 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚26088 VBDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated‚175842116629504 N@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 248285184 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 152061440 _step‚172 $ _timestamp‚1.788504597508333e+09 ZHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 2120413587968 H?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚1321 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚609 G?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚609 IDeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚175841868344320 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 @8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚895 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 ^LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 2108801611636 ZHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 2108575019288 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 599785472 E8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚98170915 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 248285184 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚31686 OFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚1299 _runtime‚ 3673.58753489 TGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚44344895 _MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 2217949118976 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚2 L?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚44344893 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 307460096 eQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚176250342411264 ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 282556416 @8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚895 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚31931 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚601 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚13361836 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚25591738 BDeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚175841868344320 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 @8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚895 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 ^LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 2108801611636 ZHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 2108575019288 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 599785472 E8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚98170915 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 248285184 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚31686 OFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚1299 _runtime‚ 3673.58753489 TGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚44344895 _MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 2217949118976 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚2 L?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚44344893 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 307460096 eQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚176250342411264 ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 282556416 @8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚895 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚31931 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚601 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚13361836 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚25591738 BDeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚31686 cQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 2218231675392 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 `LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚169962708394096 G?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚286 PCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚44344895 G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚3689 PCDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated‚53852110 ]IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚173721471795712 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚26090 ^JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚178468574086656 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚31931 YEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚173721454756352 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 231245824 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 231245824 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚26088 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚24903680 M@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚38953574 UGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 582240777 \HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚173721454756352 ^LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 2120644833792 YIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 13207863296 VFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 13207863296 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 D>DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 TFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 599785472 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚286 JBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚286 WEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 2120413587968 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚17039360 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚3689 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚13362437 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 XJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 226592348 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 599785472 trainer/global_step‚2149 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚26088 VBDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated‚175842116629504 N@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 248285184 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 152061440 _step‚172 $ _timestamp‚1.788504597508333e+09 ZHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 2120413587968 H?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚1321 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚609 G?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚609¬eL·h2î- _wandbruntime‚3674  _runtime‚3674Žùî&ïèM C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 G:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚98215457 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 _step‚173 K;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 13568115712 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 231259136 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 UEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 12589930496 TDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 13207863296 REDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚25603231 \JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 2121602318336 WIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 599785472 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚609 REDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚53876721 RBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 13807648768 L>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 248298496 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚605 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚25603233 XFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 2109527066682 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 599785472 UCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 2121371059200 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 ]KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 2218958235648 cODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚176329937271296 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚17039360 ZFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚170039538089280 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚26114 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17039360 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 320831488 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 WCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚172149308787878 _runtime‚3674.490816084 J=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚53850607 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚31686 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚33 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚2 FDeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚38970949 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 582240777 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚895 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 152061440 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 T@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚175921585323008 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚286 UEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 13035530240 N>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 13118524033 C6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚98215457 YIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 13207863296 PDeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚31931 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚44364850 K>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚98241573 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚895 ^JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚170039555128640 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚286 SEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 586907136 REDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚44364852 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚13368321 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚603 QCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 384795648 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 VHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 226592556 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚895 N>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 13568115712 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚26114 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1321 \HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚178549169661440 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 \JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 2109753659238 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 EDeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 248298496 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚605 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚25603233 XFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 2109527066682 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 599785472 UCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 2121371059200 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 ]KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 2218958235648 cODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚176329937271296 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚17039360 ZFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚170039538089280 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚26114 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17039360 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 320831488 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 WCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚172149308787878 _runtime‚3674.490816084 J=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚53850607 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚31686 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚33 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚2 FDeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚38970949 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 582240777 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚895 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 152061440 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 T@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚175921585323008 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚286 UEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 13035530240 N>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 13118524033 C6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚98215457 YIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 13207863296 PDeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚31931 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚44364850 K>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚98241573 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚895 ^JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚170039555128640 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚286 SEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 586907136 REDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚44364852 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚13368321 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚603 QCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 384795648 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 VHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 226592556 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚895 N>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 13568115712 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚26114 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1321 \HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚178549169661440 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 \JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 2109753659238 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 E{}žü‚(:“ ÒÔéÔÜÂï¹ gpu.0.gpu‚32.0 gpu.0.memory‚3, gpu.0.memoryAllocated‚16.718826293945312* gpu.0.memoryAllocatedBytes‚ 14361362432 gpu.0.temp‚31.0 gpu.0.powerWatts‚109.517' gpu.0.enforcedPowerLimitWatts‚500.0) gpu.0.powerPercent‚21.903399999999998 gpu.0.smClock‚1275 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0§‹Šý‚(:ÿ ÒÔéÔֹȻ( disk./.usagePercent‚78.8453470876419 disk.nvme0c0n1.in‚8916.125( disk.nvme0c0n1.out‚82575.50830078125 disk.dm-0.in‚8916.125 disk.drbd1260.out‚0.515625 disk.sda1.out‚0 disk.sda.in‚0.796875 network.sent‚ 5.141534e+06 disk.nvme0n1.in‚8916.125 disk.sda.out‚ 243.3671875 disk.sda2.out‚0 disk.sda3.in‚ 0.6953125 disk.sda3.out‚ 243.3671875% proc.memory.availableMB‚ 171.90625) proc.memory.percent‚18.41146945953369& disk.nvme0n1.out‚82575.50830078125 disk.dm-1.in‚ 0.6953125 disk.dm-1.out‚ 243.3671875 disk.sda1.in‚ 0.1015625$ memory_percent‚98.95076751708984 cpu‚25.748619632780514% disk./.usageGB‚1174.8526496887207# disk.dm-0.out‚82575.50830078125 disk.drbd1260.in‚0 disk.sda2.in‚0# proc.memory.rssMB‚ 3016.53515625 proc.cpu.threads‚17 network.recv‚ 1.292674e+06fÄœþ‚(:‘ áÔéÔØõ§¹ gpu.0.gpu‚9.0 gpu.0.memory‚1, gpu.0.memoryAllocated‚16.718826293945312* gpu.0.memoryAllocatedBytes‚ 14361362432 gpu.0.temp‚31.0 gpu.0.powerWatts‚94.091' gpu.0.enforcedPowerLimitWatts‚500.0) gpu.0.powerPercent‚18.818199999999997 gpu.0.smClock‚1275 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0 œœÿ‚(:‘ áÔéÔ“×¼ network.sent‚ 5.146475e+06 network.recv‚ 1.296328e+06) disk./.usagePercent‚78.84525185283917! disk.nvme0n1.in‚ 9010.48046875 disk.dm-1.in‚ 0.6953125 disk.drbd1260.out‚ 0.51953125 disk.sda1.out‚0 disk.sda.in‚0.796875 disk.sda2.in‚0 disk.sda3.in‚ 0.6953125 disk.sda3.out‚ 243.79296875' disk.nvme0c0n1.out‚83013.7587890625% disk.nvme0n1.out‚83013.7587890625 disk.dm-0.in‚ 9010.48046875 disk.sda.out‚ 243.79296875( proc.memory.availableMB‚ 171.65234375 disk.sda2.out‚0 disk.dm-1.out‚ 243.79296875 disk.drbd1260.in‚0 disk.sda1.in‚ 0.1015625# proc.memory.rssMB‚ 3016.53515625) proc.memory.percent‚18.41146945953369 cpu‚25.748288153091334 proc.cpu.threads‚17$ disk./.usageGB‚1174.851230621338# disk.nvme0c0n1.in‚ 9010.48046875" disk.dm-0.out‚83013.7587890625" memory_percent‚98.952317237854Ðs/í2€- _wandbruntime‚3753  _runtime‚3753¥)ÇÁ _runtime‚3753.850991145 _step‚175 & lr-AdamW/pg1‚0.0010464506501652593 & lr-AdamW/pg2‚0.0010464506501652593 trainer/global_step‚2199 % _timestamp‚1.7885046777722232e+09ǘ€?Ì‚Æ & lr-AdamW/pg2‚0.0010464506501652593 trainer/global_step‚2199 % _timestamp‚1.7885046777722232e+09 _runtime‚3753.850991145 _step‚175 & lr-AdamW/pg1‚0.0010464506501652593¯¤¶§2ƒ- _wandbruntime‚3753  _runtime‚3753€^^Mæ'„àO UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 231245824 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚625 M@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚39829572 R>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚179815330182144 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 248285184 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚26090 JDeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚3689 JDeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚31686 ZFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚182498196851712 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 311654400 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 152061440 ^LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 2156431153344 YIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 13207863296 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚26088 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚26166388 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚17039360 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚26088 PCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚55058624 UADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚175961232642932 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 599785472 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 VFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 13207863296 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚2 BDeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 YEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚179815578467328 UADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚179815330182144 L?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚55058624 TDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 13035530240 P@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 13118524033 RBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 13807648768 [IDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 2168546223616 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 [KDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 13207863296 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 586907136 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚13663807 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 YEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚175961476274640 \HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚173805028081936 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚609 TGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚45342745 XJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 226592348 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚31931 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚31931 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 G?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚609 _runtime‚3753.852792591 _step‚176 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 231245824 N@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚ 100427457 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚3689 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚31686 _MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 2268136342528 `LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚177647032243712 MFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚33 O?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 13807648768 VBDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated‚179815578467328 ZHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 2156204560996 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 586907136 H?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚1321 ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 286750720 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 `LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚173805045121296 UGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 582240777 N@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 248285184 ZHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 2168314977792 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚13663184 cQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 2268423093248 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 P@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 13568115712 WGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 12589930496 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚286+Rë'…åO ]IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚177647032243712 WGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 13035530240 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 599785472 PCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚45342745 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚31686 ZFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚182498196851712 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 311654400 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 152061440 ^LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 2156431153344 YIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 13207863296 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚26088 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚26166388 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚17039360 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚26088 PCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚55058624 UADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚175961232642932 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 599785472 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 VFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 13207863296 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚2 BDeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 YEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚179815578467328 UADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚179815330182144 L?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚55058624 TDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 13035530240 P@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 13118524033 RBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 13807648768 [IDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 2168546223616 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 [KDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 13207863296 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 586907136 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚13663807 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 YEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚175961476274640 \HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚173805028081936 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚609 TGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚45342745 XJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 226592348 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚31931 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚31931 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 G?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚609 _runtime‚3753.852792591 _step‚176 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 231245824 N@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚ 100427457 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚3689 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚31686 _MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 2268136342528 `LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚177647032243712 MFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚33 O?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 13807648768 VBDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated‚179815578467328 ZHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 2156204560996 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 586907136 H?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚1321 ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 286750720 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 `LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚173805045121296 UGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 582240777 N@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 248285184 ZHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 2168314977792 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚13663184 cQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 2268423093248 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 P@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 13568115712 WGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 12589930496 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚286 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 231245824 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚625 M@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚39829572 R>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚179815330182144 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 248285184 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚26090 JDeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚3689 JDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 13118524033 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17039360 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚26114 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 NADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚55109323 \JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 2169503708160 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚33 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 REDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚45362702 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 ZFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚173881874816480 UEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 12589930496 XFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 2157156608390 trainer/global_step‚2199 _KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚180309655369216 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 J=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚55083209 K;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 13568115712 UCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 2169272449024 REDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚26177881 ]KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 2269137940992 \JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 2157383200946 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚286 WGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 13207863296 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚609 EDeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 NADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚45362700 QCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 384795648 _runtime‚3755.632354641 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1321 P@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 13807648768 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 599785472 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚609 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚31931 L>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚ 100472025 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚3689 MDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚1299 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 599785472 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚26114 RBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 13035530240 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚31931 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24903680 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 320831488 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 305349632 _step‚177 N>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 13568115712 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚13669761 REDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚13669134 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 582240777 % _timestamp‚1.7885046795528617e+09 D6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚ 100445909 L>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 248298496 ZFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚177725526413312 XFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 2169272449024 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚286 SEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 586907136 \HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚182579098659840 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚45362700 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 WCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚176039275056786 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚895 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚895 [GDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚177725543452672 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚17039360 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚31686 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 S?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚176039031424870 PDeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚39847015 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚2 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 T@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚179895047160832 B6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚895 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚2 FDeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 NADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚45362700 QCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 384795648 _runtime‚3755.632354641 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1321 P@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 13807648768 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 599785472 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚609 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚31931 L>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚ 100472025 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚3689 MDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚1299 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 599785472 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚26114 RBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 13035530240 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚31931 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24903680 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 320831488 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 305349632 _step‚177 N>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 13568115712 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚13669761 REDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚13669134 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 582240777 % _timestamp‚1.7885046795528617e+09 D6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚ 100445909 L>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 248298496 ZFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚177725526413312 XFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 2169272449024 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚286 SEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 586907136 \HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚182579098659840 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚45362700 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 WCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚176039275056786 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚895 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚895 [GDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚177725543452672 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚17039360 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚31686 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 S?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚176039031424870 PDeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚39847015 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚2 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 T@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚179895047160832 B6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚895 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚2 FDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 13118524033 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17039360 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚26114 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 NADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚55109323 \JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 2169503708160 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚33 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 REDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚45362702 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 ZFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚173881874816480±= Ю2‰- _wandbruntime‚3755  _runtime‚3755·†|{[ŠÕ 1loss/SRqqt1/assignment_loss‚1.870931090137293 2loss/SRqqt4/assignment_loss‚0.9927254646271467 1loss/SRqqt3/detection_loss‚2.7807716046060835 _step‚178 1loss/FBt2/assignment_loss‚0.26219412035150635 /loss/FRt2/detection_loss‚2.6109289740448567 1loss/SRqqt4/detection_loss‚0.8702737678438425 /loss/FBt3/detection_loss‚0.9189225361319898 % _timestamp‚1.7885046795559797e+09 2loss/SRqqt2/assignment_loss‚1.2951845979114296 1loss/FBt4/assignment_loss‚0.30928370670133315 /loss/FRt1/detection_loss‚2.4533626546462517 1loss/SRqqt1/detection_loss‚2.7089323131150977 trainer/global_step‚2199 .loss/FRt2/assignment_loss‚3.67194559236071 /loss/FRt3/assignment_loss‚2.696493223048093 2loss/SRqqt3/assignment_loss‚1.8808122422013964 .loss/FRt3/detection_loss‚2.512251506053042 _runtime‚3755.634914038 0loss/SRqqt2/detection_loss‚2.242848685745037 /loss/FBt1/detection_loss‚2.3116630323623357 epoch‚3 /loss/FRt1/assignment_loss‚3.854432220260302 0loss/FRt4/assignment_loss‚1.3768478656202796 1loss/FBt1/assignment_loss‚0.16622052626108166 &loss/total_loss‚43.851501610945206 0loss/FBt3/assignment_loss‚0.1264076734884465 .loss/FRt4/detection_loss‚1.727148512484339 /loss/FBt2/detection_loss‚2.1447728735369607 /loss/FBt4/detection_loss‚2.0661468274063535Ö7ç`‹Ú &loss/total_loss‚43.851501610945206 0loss/FBt3/assignment_loss‚0.1264076734884465 .loss/FRt4/detection_loss‚1.727148512484339 /loss/FBt2/detection_loss‚2.1447728735369607 /loss/FBt4/detection_loss‚2.0661468274063535 1loss/SRqqt1/assignment_loss‚1.870931090137293 2loss/SRqqt4/assignment_loss‚0.9927254646271467 1loss/SRqqt3/detection_loss‚2.7807716046060835 _step‚178 1loss/FBt2/assignment_loss‚0.26219412035150635 /loss/FRt2/detection_loss‚2.6109289740448567 1loss/SRqqt4/detection_loss‚0.8702737678438425 /loss/FBt3/detection_loss‚0.9189225361319898 % _timestamp‚1.7885046795559797e+09 2loss/SRqqt2/assignment_loss‚1.2951845979114296 1loss/FBt4/assignment_loss‚0.30928370670133315 /loss/FRt1/detection_loss‚2.4533626546462517 1loss/SRqqt1/detection_loss‚2.7089323131150977 trainer/global_step‚2199 .loss/FRt2/assignment_loss‚3.67194559236071 /loss/FRt3/assignment_loss‚2.696493223048093 2loss/SRqqt3/assignment_loss‚1.8808122422013964 .loss/FRt3/detection_loss‚2.512251506053042 _runtime‚3755.634914038 0loss/SRqqt2/detection_loss‚2.242848685745037 /loss/FBt1/detection_loss‚2.3116630323623357 epoch‚3 /loss/FRt1/assignment_loss‚3.854432220260302 0loss/FRt4/assignment_loss‚1.3768478656202796 1loss/FBt1/assignment_loss‚0.16622052626108166²ª>~X‘Œ‚(:† ðÔéÔìÇü¹ gpu.0.gpu‚31.0 gpu.0.memory‚3, gpu.0.memoryAllocated‚16.718826293945312* gpu.0.memoryAllocatedBytes‚ 14361362432 gpu.0.temp‚31.0 gpu.0.powerWatts‚93.455' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚18.691 gpu.0.smClock‚1275 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0Ò£)œ‚(:‘ ðÔéÔÙæ¼ disk.sda.in‚0.796875$ memory_percent‚98.95408153533936( proc.memory.availableMB‚ 171.36328125# proc.memory.rssMB‚ 3016.53515625 proc.cpu.threads‚17 network.sent‚ 5.151731e+06) disk./.usagePercent‚78.86635173780373 disk.sda2.out‚0 disk.sda3.in‚ 0.6953125% disk.nvme0n1.out‚83246.0322265625" disk.dm-0.out‚83246.0322265625 disk.dm-1.in‚ 0.6953125 disk.sda1.out‚0! disk.nvme0n1.in‚ 9010.48046875 disk.dm-0.in‚ 9010.48046875 disk.dm-1.out‚ 244.2578125 disk.sda.out‚ 244.2578125) proc.memory.percent‚18.41146945953369 cpu‚25.747826596493095 network.recv‚ 1.300084e+06% disk./.usageGB‚1175.1656341552734 disk.sda2.in‚0 disk.sda3.out‚ 244.2578125# disk.nvme0c0n1.in‚ 9010.48046875 disk.drbd1260.in‚0 disk.sda1.in‚ 0.1015625' disk.nvme0c0n1.out‚83246.0322265625 disk.drbd1260.out‚ 0.519531253Dü“Ž‚(:ˆ ÿÔéÔƒ…ƒ» gpu.0.gpu‚42.0 gpu.0.memory‚5, gpu.0.memoryAllocated‚16.718826293945312* gpu.0.memoryAllocatedBytes‚ 14361362432 gpu.0.temp‚31.0 gpu.0.powerWatts‚104.298' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚20.8596 gpu.0.smClock‚1350 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0,gŠ¨š‚(: ÿÔéÔ󺺽 disk.dm-1.out‚ 244.703125 disk.drbd1260.in‚0( proc.memory.availableMB‚ 171.60546875) disk./.usagePercent‚78.87047781123307% disk./.usageGB‚1175.2271156311035 disk.sda2.out‚0 disk.sda1.in‚ 0.1015625# proc.memory.rssMB‚ 3016.53515625 cpu‚25.74749893018036 proc.cpu.threads‚17 disk.sda3.in‚ 0.6953125 disk.sda3.out‚ 244.703125# disk.nvme0c0n1.in‚ 9010.48046875 disk.sda.out‚ 244.703125) proc.memory.percent‚18.41146945953369 disk.sda2.in‚0( disk.nvme0c0n1.out‚83599.98876953125& disk.nvme0n1.out‚83599.98876953125 disk.dm-1.in‚ 0.6953125 disk.drbd1260.out‚ 0.5234375 disk.sda1.out‚0 disk.sda.in‚0.796875$ memory_percent‚98.95260334014893 network.sent‚ 5.238242e+06 network.recv‚ 1.305332e+06! disk.nvme0n1.in‚ 9010.48046875 disk.dm-0.in‚ 9010.48046875# disk.dm-0.out‚83599.98876953125kfè“‚(:ˆ ŽÕéÔëà º gpu.0.gpu‚45.0 gpu.0.memory‚5, gpu.0.memoryAllocated‚16.718826293945312* gpu.0.memoryAllocatedBytes‚ 14361362432 gpu.0.temp‚34.0 gpu.0.powerWatts‚160.302' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚32.0604 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0òˆ‰vš‘‚(: ŽÕéÔç¿êº disk.sda1.in‚ 0.1015625 disk.sda.in‚0.796875 disk.sda.out‚ 245.109375 disk.sda2.out‚0 disk.sda3.in‚ 0.6953125( disk.nvme0c0n1.out‚83761.18017578125' proc.memory.availableMB‚ 171.7734375# proc.memory.rssMB‚ 3016.53515625) disk./.usagePercent‚78.87886666611506! disk.nvme0n1.in‚ 9010.48046875& disk.nvme0n1.out‚83761.18017578125 disk.dm-1.in‚ 0.6953125 disk.dm-1.out‚ 245.109375 disk.drbd1260.in‚0 disk.sda2.in‚0 disk.sda3.out‚ 245.109375 network.recv‚ 1.310173e+06 disk.dm-0.in‚ 9010.48046875# disk.dm-0.out‚83761.18017578125 disk.drbd1260.out‚ 0.5234375$ memory_percent‚98.95157814025879) proc.memory.percent‚18.41146945953369 disk.sda1.out‚0 network.sent‚ 5.246515e+06% disk./.usageGB‚1175.3521156311035# disk.nvme0c0n1.in‚ 9010.48046875 cpu‚25.747089947778605 proc.cpu.threads‚17¬rIù“’‚(:ˆ ÕéÔ·¡Úº gpu.0.gpu‚23.0 gpu.0.memory‚2, gpu.0.memoryAllocated‚16.718826293945312* gpu.0.memoryAllocatedBytes‚ 14361362432 gpu.0.temp‚31.0 gpu.0.powerWatts‚109.246' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚21.8492 gpu.0.smClock‚1275 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0³pÿ™“‚(:Ž ÕéÔÍ­ì» disk.sda2.in‚0 disk.drbd1260.in‚0 disk.drbd1260.out‚ 0.52734375 disk.sda1.in‚ 0.1015625 disk.sda.in‚0.796875 disk.sda2.out‚0% disk.nvme0n1.out‚84288.0966796875$ memory_percent‚98.95155429840088( proc.memory.availableMB‚ 171.77734375# proc.memory.rssMB‚ 3016.53515625) proc.memory.percent‚18.41146945953369 cpu‚25.746828115413404 proc.cpu.threads‚17) disk./.usagePercent‚78.87886666611506% disk./.usageGB‚1175.3521156311035' disk.nvme0c0n1.out‚84288.0966796875 disk.nvme0n1.in‚ 9167.9921875 disk.dm-1.in‚ 0.6953125 disk.sda1.out‚0 disk.sda.out‚ 245.4765625" disk.dm-0.out‚84288.0966796875 network.sent‚ 5.249818e+06 network.recv‚ 1.313882e+06 disk.sda3.in‚ 0.6953125 disk.sda3.out‚ 245.4765625" disk.nvme0c0n1.in‚ 9167.9921875 disk.dm-0.in‚ 9167.9921875 disk.dm-1.out‚ 245.4765625 ¸sÈž”‚(:“ ¬ÕéÔÈ‹ðº gpu.0.gpu‚32.0 gpu.0.memory‚3, gpu.0.memoryAllocated‚16.718826293945312* gpu.0.memoryAllocatedBytes‚ 14361362432 gpu.0.temp‚32.0 gpu.0.powerWatts‚117.402' gpu.0.enforcedPowerLimitWatts‚500.0) gpu.0.powerPercent‚23.480400000000003 gpu.0.smClock‚1275 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0xúv+”•‚(:‰ ¬ÕéÔÓ˜Ó») disk./.usagePercent‚78.92933522339258 disk.sda2.out‚0 disk.sda3.out‚ 245.9296875' disk.nvme0c0n1.out‚84544.3935546875 disk.nvme0n1.in‚ 9167.9921875% disk.nvme0n1.out‚84544.3935546875 disk.dm-1.out‚ 245.9296875 proc.cpu.threads‚17$ disk./.usageGB‚1176.104133605957" disk.nvme0c0n1.in‚ 9167.9921875 disk.sda1.out‚0% proc.memory.availableMB‚ 171.78125 disk.dm-0.in‚ 9167.9921875" disk.dm-0.out‚84544.3935546875 network.recv‚ 1.317346e+06 disk.sda3.in‚ 0.6953125 disk.dm-1.in‚ 0.6953125 disk.sda1.in‚ 0.1015625$ memory_percent‚98.95153045654297# proc.memory.rssMB‚ 3016.53515625 network.sent‚ 5.256851e+06 disk.sda2.in‚0 disk.drbd1260.in‚0 disk.drbd1260.out‚ 0.52734375 disk.sda.in‚0.796875 disk.sda.out‚ 245.9296875) proc.memory.percent‚18.41146945953369 cpu‚25.74639094121495Ø“–‚(:ˆ »ÕéÔﳨº gpu.0.gpu‚34.0 gpu.0.memory‚4, gpu.0.memoryAllocated‚16.718826293945312* gpu.0.memoryAllocatedBytes‚ 14361362432 gpu.0.temp‚33.0 gpu.0.powerWatts‚114.193' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚22.8386 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0{Êm——‚(:Œ »ÕéÔÄ­¼# proc.memory.rssMB‚ 3016.53515625 disk.sda2.in‚0 disk.dm-0.in‚ 9167.9921875 cpu‚25.74594516949091 proc.cpu.threads‚17 network.recv‚ 1.321616e+06) disk./.usagePercent‚78.92919979540697 disk.sda2.out‚0" disk.nvme0c0n1.in‚ 9167.9921875& disk.nvme0n1.out‚85046.28564453125 disk.dm-1.in‚ 0.6953125$ memory_percent‚98.95462989807129) proc.memory.percent‚18.41146945953369 network.sent‚ 5.260836e+06% disk./.usageGB‚1176.1021156311035 disk.sda3.in‚ 0.6953125( disk.nvme0c0n1.out‚85046.28564453125 disk.dm-1.out‚ 246.2890625 disk.drbd1260.out‚0.53125 disk.sda1.in‚ 0.1015625 disk.sda1.out‚0 disk.sda3.out‚ 246.2890625 disk.nvme0n1.in‚ 9167.9921875# disk.dm-0.out‚85046.28564453125 disk.drbd1260.in‚0 disk.sda.in‚0.796875 disk.sda.out‚ 246.2890625' proc.memory.availableMB‚ 171.2734375!Pú2˜- _wandbruntime‚3844  _runtime‚3844`ªÍÄ™¾ _step‚179 % lr-AdamW/pg1‚0.001031272892747357 % lr-AdamW/pg2‚0.001031272892747357 trainer/global_step‚2249 % _timestamp‚1.7885047685221994e+09 _runtime‚ 3844.60100836X‰d—ɚà % lr-AdamW/pg1‚0.001031272892747357 % lr-AdamW/pg2‚0.001031272892747357 trainer/global_step‚2249 % _timestamp‚1.7885047685221994e+09 _runtime‚ 3844.60100836 _step‚179³UkÉ°2›- _wandbruntime‚3844  _runtime‚3844c/lþœáO QDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚40707607 TDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 13807648768 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚31686 WGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 13035530240 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚13965941 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 ^LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 2216447613440 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 ZFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚186528064782848 cQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 2318537537024 ^LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 2204060695052 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚26741038 M=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 13568115712 WEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 2216216367616 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 231245824 JDeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 ˆl “éUGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 231245824 YEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚183789040305152 H?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚1321 eQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚184209828414464 OFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚1299 aMDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚184209803510784 `LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚177647381848496 L?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚56291226 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 586907136 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 248285184 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚26090 ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 276264960 YIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 13207863296 ]IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚181572592691712 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 586907136 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 TFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 599785472 @8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚895 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚31931 PCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚46340593 QCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 243631708 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 O?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 13807648768 E=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚895 JBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚609 UADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚179851198911840 ZHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 2203834102704 % _timestamp‚1.7885047685236042e+09 R>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚183788792019968 P@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 13568115712 PCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚56291226 VFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 13207863296 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 599785472 MFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚33 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚13966567 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 WGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 12589930496 JDeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚3689 ZHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 2216216367616 TGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚46340595 TGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚56317314 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 301168640 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚2 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 YEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚179851442543548 trainer/global_step‚2249 [IDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 2216447613440 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚26088 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 P@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 13118524033 JBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚286 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 _step‚180 PCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚46340595 L?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚46340593 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚31686 _MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 2318261272064 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚609 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚286 `LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚181572592691712 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚3689‚)Ì ì'æO JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚628 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚24903680 D>DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 231245824 YEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚183789040305152 H?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚1321 eQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚184209828414464 OFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚1299 aMDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚184209803510784 `LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚177647381848496 L?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚56291226 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 586907136 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 248285184 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚26090 ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 276264960 YIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 13207863296 ]IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚181572592691712 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 586907136 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 TFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 599785472 @8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚895 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚31931 PCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚46340593 QCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 243631708 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 O?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 13807648768 E=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚895 JBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚609 UADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚179851198911840 ZHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 2203834102704 % _timestamp‚1.7885047685236042e+09 R>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚183788792019968 P@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 13568115712 PCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚56291226 VFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 13207863296 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 599785472 MFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚33 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚13966567 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 WGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 12589930496 JDeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚3689 ZHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 2216216367616 TGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚46340595 TGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚56317314 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 301168640 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚2 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 YEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚179851442543548 trainer/global_step‚2249 [IDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 2216447613440 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚26088 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 P@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 13118524033 JBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚286 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 _step‚180 PCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚46340595 L?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚46340593 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚31686 _MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 2318261272064 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚609 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚286 `LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚181572592691712 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚3689 QDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚40707607 TDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 13807648768 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚31686 WGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 13035530240 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚13965941 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 ^LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 2216447613440 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 ZFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚186528064782848 cQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 2318537537024 ^LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 2204060695052 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚26741038 M=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 13568115712 WEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 2216216367616 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 231245824 JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 13568115712 NADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚46360552 FDeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚ 102702477 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚31931 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚46360550 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚26114 YIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 13207863296 [GDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚181651103900672 RBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 13035530240 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 231259136 REDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚56341925 _KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚184289398370816 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 SEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 586907136 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚13972619 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 T@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚183868508998656 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚895 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚26116 WCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚183868508998656 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚33 ]KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 2319260740608 BDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 13118524033 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚895 P@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 13807648768 WGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 13207863296 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚895 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚609 S?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚183868260700160 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚2 REDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚26752531 % _timestamp‚1.7885047696604528e+09 L>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 248298496 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚17039360 QCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 384795648 UEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 12589930496 trainer/global_step‚2249 D6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚ 102676361 WCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚181651086861312 ZFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚177724211543680 \JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 2205012742654 K>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚40724520 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 H:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚ 102702477 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚31686 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚895 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 152061440 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 EDeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 OADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 248298496 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 231259136 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 305349632 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 582240777 PBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 586907136 H:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚ 102676361 REDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚46360552 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 VHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 226592556 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚609 J=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚56315811 XFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 2217173838848 FDeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 K;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 13568115712 [MDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚ 280445952 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚17039360 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚609 _step‚181 ^JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚181651103900672 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚630 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24903680 S?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚179928997693778Á$DÜä ñM SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 231259136 REDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚56341925 _KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚184289398370816 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 SEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 586907136 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚13972619 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 T@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚183868508998656 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚895 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚26116 WCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚183868508998656 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚33 ]KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 2319260740608 BDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 13118524033 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚895 P@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 13807648768 WGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 13207863296 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚895 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚609 S?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚183868260700160 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚2 REDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚26752531 % _timestamp‚1.7885047696604528e+09 L>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 248298496 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚17039360 QCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 384795648 UEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 12589930496 trainer/global_step‚2249 D6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚ 102676361 WCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚181651086861312 ZFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚177724211543680 \JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 2205012742654 K>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚40724520 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 H:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚ 102702477 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚31686 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚895 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 152061440 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 EDeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 OADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 248298496 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 231259136 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 305349632 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 582240777 PBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 586907136 H:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚ 102676361 REDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚46360552 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 VHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 226592556 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚609 J=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚56315811 XFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 2217173838848 FDeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 K;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 13568115712 [MDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚ 280445952 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚17039360 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚609 _step‚181 ^JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚181651103900672 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚630 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24903680 S?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚179928997693778 N>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 13568115712 NADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚46360552 FDeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚ 102702477 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚31931 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚46360550 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚26114 YIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 13207863296 [GDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚181651103900672 RBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 13035530240µ:7>´2¡- _wandbruntime‚3845  _runtime‚3845(j}<\¢Ö /loss/FRt2/detection_loss‚2.6723098302042327 /loss/FBt3/detection_loss‚0.6823290069986667 &loss/total_loss‚42.927444561570304 trainer/global_step‚2249 _runtime‚3845.741205165 2loss/SRqqt3/assignment_loss‚1.8042641525735725 0loss/FBt2/assignment_loss‚0.3328061884164096 1loss/SRqqt1/detection_loss‚2.3371638221255804 1loss/SRqqt2/detection_loss‚2.1657238049783567 .loss/FBt2/detection_loss‚2.445987569434302 .loss/FRt3/detection_loss‚1.984472947621561 .loss/SRqqt3/detection_loss‚2.4545702861773 epoch‚3 2loss/SRqqt1/assignment_loss‚1.5774626960441218 1loss/FBt1/assignment_loss‚0.29115671204168764 0loss/FRt4/assignment_loss‚1.6210654466821437 2loss/SRqqt4/assignment_loss‚1.0519161888445263 0loss/FBt4/assignment_loss‚0.3080100499384773 0loss/SRqqt4/detection_loss‚1.000226602382675 /loss/FBt4/detection_loss‚2.1678268859783807 0loss/FRt3/assignment_loss‚2.0823437032929384 1loss/FBt3/assignment_loss‚0.16593889252998098 /loss/FRt4/detection_loss‚1.9481862497826417 % _timestamp‚1.7885047696623216e+09 0loss/FRt1/assignment_loss‚3.7316831172780787 0loss/FRt2/assignment_loss‚3.5517125779932197 /loss/FRt1/detection_loss‚2.9634269196936427 /loss/FBt1/detection_loss‚2.0091437352555137 _step‚182 2loss/SRqqt2/assignment_loss‚1.5777171753022983ÇP2a£Û 0loss/FRt4/assignment_loss‚1.6210654466821437 2loss/SRqqt4/assignment_loss‚1.0519161888445263 0loss/FBt4/assignment_loss‚0.3080100499384773 0loss/SRqqt4/detection_loss‚1.000226602382675 /loss/FBt4/detection_loss‚2.1678268859783807 0loss/FRt3/assignment_loss‚2.0823437032929384 1loss/FBt3/assignment_loss‚0.16593889252998098 /loss/FRt4/detection_loss‚1.9481862497826417 % _timestamp‚1.7885047696623216e+09 0loss/FRt1/assignment_loss‚3.7316831172780787 0loss/FRt2/assignment_loss‚3.5517125779932197 /loss/FRt1/detection_loss‚2.9634269196936427 /loss/FBt1/detection_loss‚2.0091437352555137 _step‚182 2loss/SRqqt2/assignment_loss‚1.5777171753022983 /loss/FRt2/detection_loss‚2.6723098302042327 /loss/FBt3/detection_loss‚0.6823290069986667 &loss/total_loss‚42.927444561570304 trainer/global_step‚2249 _runtime‚3845.741205165 2loss/SRqqt3/assignment_loss‚1.8042641525735725 0loss/FBt2/assignment_loss‚0.3328061884164096 1loss/SRqqt1/detection_loss‚2.3371638221255804 1loss/SRqqt2/detection_loss‚2.1657238049783567 .loss/FBt2/detection_loss‚2.445987569434302 .loss/FRt3/detection_loss‚1.984472947621561 .loss/SRqqt3/detection_loss‚2.4545702861773 epoch‚3 2loss/SRqqt1/assignment_loss‚1.5774626960441218 1loss/FBt1/assignment_loss‚0.29115671204168764¶‰6—ž¤‚(:“ ÊÕéÔ–ò¹º gpu.0.gpu‚55.0 gpu.0.memory‚5, gpu.0.memoryAllocated‚16.718826293945312* gpu.0.memoryAllocatedBytes‚ 14361362432 gpu.0.temp‚33.0 gpu.0.powerWatts‚156.181' gpu.0.enforcedPowerLimitWatts‚500.0) gpu.0.powerPercent‚31.236200000000004 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0z3pš’¥‚(:‡ ÊÕéÔݥ¼ network.sent‚ 5.267211e+06) disk./.usagePercent‚78.92919979540697 disk.sda2.in‚0 disk.sda3.out‚ 246.59375# disk.dm-0.out‚85399.79736328125 disk.dm-1.in‚ 0.6953125 disk.sda.in‚0.796875 disk.sda.out‚ 246.59375 disk.sda3.in‚ 0.6953125" disk.nvme0c0n1.in‚ 9167.9921875( disk.nvme0c0n1.out‚85399.79736328125 disk.nvme0n1.in‚ 9167.9921875& disk.nvme0n1.out‚85399.79736328125 disk.sda1.in‚ 0.1015625 disk.sda1.out‚0) proc.memory.percent‚18.41146945953369 network.recv‚ 1.326468e+06 disk.dm-0.in‚ 9167.9921875 disk.drbd1260.in‚0$ memory_percent‚98.95243644714355# proc.memory.rssMB‚ 3016.53515625 cpu‚25.745445918485142% disk./.usageGB‚1176.1021156311035 disk.sda2.out‚0 disk.dm-1.out‚ 246.59375 disk.drbd1260.out‚0.53125' proc.memory.availableMB‚ 171.6328125 proc.cpu.threads‚17Î…F¢“¦‚(:ˆ ÙÕéÔÿ°¤¹ gpu.0.gpu‚63.0 gpu.0.memory‚5, gpu.0.memoryAllocated‚16.718826293945312* gpu.0.memoryAllocatedBytes‚ 14361362432 gpu.0.temp‚33.0 gpu.0.powerWatts‚121.444' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚24.2888 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0,J\Ÿ§‚(:” ÙÕéÔÎÉû network.sent‚ 5.353601e+06 network.recv‚ 1.331502e+06# disk.nvme0c0n1.in‚ 9300.69921875! disk.nvme0n1.in‚ 9300.69921875" disk.dm-0.out‚85632.4267578125 disk.drbd1260.in‚0 disk.drbd1260.out‚ 0.53515625 cpu‚25.745030248165794 disk.sda2.out‚0 disk.sda3.in‚ 0.6953125' disk.nvme0c0n1.out‚85632.4267578125% disk.nvme0n1.out‚85632.4267578125 disk.dm-0.in‚ 9300.69921875 disk.dm-1.out‚ 247.03515625$ memory_percent‚98.95246028900146( proc.memory.availableMB‚ 171.62890625 disk.sda3.out‚ 247.03515625 disk.dm-1.in‚ 0.6953125 disk.sda1.in‚ 0.1015625 disk.sda1.out‚0# proc.memory.rssMB‚ 3016.53515625) proc.memory.percent‚18.41146945953369) disk./.usagePercent‚78.92919979540697% disk./.usageGB‚1176.1021156311035 disk.sda2.in‚0 disk.sda.in‚0.796875 disk.sda.out‚ 247.03515625 proc.cpu.threads‚17!×M¥ž¨‚(:“ èÕéÔ­†è¹ gpu.0.gpu‚55.0 gpu.0.memory‚6, gpu.0.memoryAllocated‚16.718826293945312* gpu.0.memoryAllocatedBytes‚ 14361362432 gpu.0.temp‚33.0 gpu.0.powerWatts‚125.439' gpu.0.enforcedPowerLimitWatts‚500.0) gpu.0.powerPercent‚25.087799999999998 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0™¥•™©‚(:Ž èÕéÔæ³í»% disk.nvme0n1.out‚85985.9072265625 disk.dm-0.in‚ 9300.69921875 disk.sda.in‚0.796875 disk.sda2.in‚0) proc.memory.percent‚18.41146945953369! disk.nvme0n1.in‚ 9300.69921875 disk.sda1.in‚ 0.1015625 disk.sda2.out‚0 proc.cpu.threads‚17 network.sent‚ 5.358845e+06# disk.nvme0c0n1.in‚ 9300.69921875' disk.nvme0c0n1.out‚85985.9072265625" disk.dm-0.out‚85985.9072265625 disk.dm-1.out‚ 247.43359375 disk.sda1.out‚0 disk.sda.out‚ 247.43359375 disk.sda3.in‚ 0.6953125( disk./.usagePercent‚78.9627603350856% disk./.usageGB‚1176.6021919250488 disk.dm-1.in‚ 0.6953125 disk.sda3.out‚ 247.43359375$ memory_percent‚98.95401000976562# proc.memory.availableMB‚171.375# proc.memory.rssMB‚ 3016.53515625 cpu‚25.744726752928795 disk.drbd1260.in‚0 disk.drbd1260.out‚ 0.53515625 network.recv‚ 1.335017e+06E÷’@žª‚(:“ ÷ÕéÔìÑ̹ gpu.0.gpu‚12.0 gpu.0.memory‚1, gpu.0.memoryAllocated‚16.718826293945312* gpu.0.memoryAllocatedBytes‚ 14361362432 gpu.0.temp‚33.0 gpu.0.powerWatts‚123.175' gpu.0.enforcedPowerLimitWatts‚500.0) gpu.0.powerPercent‚24.634999999999998 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0Ò1Ùž«‚(:“ ÷ÕéÔ±Ôè» disk.sda1.in‚ 0.1015625 disk.sda.in‚0.796875( proc.memory.availableMB‚ 171.37890625) proc.memory.percent‚18.41146945953369 cpu‚25.74443055560212 network.sent‚ 5.364734e+06# disk.nvme0c0n1.in‚ 9300.69921875 disk.dm-0.in‚ 9300.69921875 proc.cpu.threads‚17) disk./.usagePercent‚78.96275905504793% disk./.usageGB‚1176.6021728515625 disk.sda2.out‚0 disk.sda3.out‚ 248.2421875& disk.nvme0n1.out‚86487.81298828125 disk.dm-1.out‚ 248.2421875 disk.drbd1260.in‚0 disk.drbd1260.out‚ 0.54296875 network.recv‚ 1.339335e+06 disk.sda3.in‚ 0.6953125# disk.dm-0.out‚86487.81298828125 disk.dm-1.in‚ 0.6953125 disk.sda1.out‚0 disk.sda.out‚ 248.2421875$ memory_percent‚98.95398616790771# proc.memory.rssMB‚ 3016.53515625 disk.sda2.in‚0( disk.nvme0c0n1.out‚86487.81298828125! disk.nvme0n1.in‚ 9300.69921875¿¶@’¬‚(:‡ †ÖéÔ±úìº gpu.0.gpu‚7.0 gpu.0.memory‚1, gpu.0.memoryAllocated‚16.718826293945312* gpu.0.memoryAllocatedBytes‚ 14361362432 gpu.0.temp‚33.0 gpu.0.powerWatts‚155.473' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚31.0946 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚08Ø÷9ž­‚(:“ †ÖéÔæÆ€¿) proc.memory.percent‚18.41146945953369 cpu‚25.744093231898766) disk./.usagePercent‚78.93736873981705 disk.sda3.in‚ 0.6953125( disk.nvme0c0n1.out‚86744.10986328125& disk.nvme0n1.out‚86744.10986328125 disk.dm-1.in‚ 0.6953125$ memory_percent‚98.95405769348145 proc.cpu.threads‚17 network.sent‚ 5.372974e+06 network.recv‚ 1.344079e+06 disk.sda2.in‚0 disk.sda2.out‚0! disk.nvme0n1.in‚ 9300.69921875 disk.dm-0.in‚ 9300.69921875 disk.sda1.in‚ 0.1015625 disk.sda1.out‚0% disk./.usageGB‚1176.2238388061523 disk.sda3.out‚ 248.7109375# disk.nvme0c0n1.in‚ 9300.69921875# disk.dm-0.out‚86744.10986328125 disk.dm-1.out‚ 248.7109375 disk.drbd1260.in‚0 disk.sda.in‚0.796875 disk.sda.out‚ 248.7109375 disk.drbd1260.out‚ 0.54296875' proc.memory.availableMB‚ 171.3671875# proc.memory.rssMB‚ 3016.53515625`ßž®‚(:“ •ÖéÔúúŠº gpu.0.gpu‚44.0 gpu.0.memory‚3, gpu.0.memoryAllocated‚16.718826293945312* gpu.0.memoryAllocatedBytes‚ 14361362432 gpu.0.temp‚34.0 gpu.0.powerWatts‚147.917' gpu.0.enforcedPowerLimitWatts‚500.0) gpu.0.powerPercent‚29.583399999999997 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0"Ú°‘¯‚(:† •ÖéÔ¡íÚº disk.sda.in‚0.796875# proc.memory.rssMB‚ 3016.53515625) proc.memory.percent‚18.41146945953369# disk.nvme0c0n1.in‚ 9445.58203125 disk.dm-0.in‚ 9445.58203125 disk.dm-1.out‚ 249.0703125 disk.sda2.in‚0 disk.sda2.out‚0 disk.sda3.in‚ 0.6953125' proc.memory.availableMB‚ 171.6171875 proc.cpu.threads‚17) disk./.usagePercent‚78.93736873981705% disk./.usageGB‚1176.2238388061523% disk.nvme0c0n1.out‚87001.59765625 disk.dm-0.out‚87001.59765625 disk.dm-1.in‚ 0.6953125 disk.drbd1260.out‚0.546875 disk.sda.out‚ 249.0703125 disk.sda3.out‚ 249.0703125 network.sent‚ 5.378043e+06 network.recv‚ 1.347792e+06 disk.drbd1260.in‚0 disk.sda1.out‚0# memory_percent‚98.9525318145752 cpu‚25.74385891580648! disk.nvme0n1.in‚ 9445.58203125# disk.nvme0n1.out‚87001.59765625 disk.sda1.in‚ 0.1015625LÆ5¥2°- _wandbruntime‚3936  _runtime‚3936©UàDZÁ & lr-AdamW/pg1‚0.0010157786684823651 & lr-AdamW/pg2‚0.0010157786684823651 trainer/global_step‚2299 % _timestamp‚1.7885048600494168e+09 _runtime‚3936.128932022 _step‚183FÚÉÌ²Æ % _timestamp‚1.7885048600494168e+09 _runtime‚3936.128932022 _step‚183 & lr-AdamW/pg1‚0.0010157786684823651 & lr-AdamW/pg2‚0.0010157786684823651 trainer/global_step‚2299·8±¬ï2³- _wandbruntime‚3936  _runtime‚3936E葵è'´âO UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 586907136 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 231245824 P@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 13118524033 JBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚286 ZHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 2264117757440 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚14268834 BDeviceStatsMonitor.on_train_batch_start/allocation.all.current‚26090 JDeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚187762253857792 UGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 582240777 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚26088 ZFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚190558025753600 MFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚33 aMDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚188189546512384 D>DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚31931 G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚3689 `LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚185498153139712 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚2 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 320831488 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚17039360 YEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚185498136100352 _MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 2368479241216 F8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚ 104862271 PCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚57523828 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 290682880 SEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 384795648 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 QCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 243631708 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚609 C;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚895 WEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 2264117757440 UADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚187762253857792 eQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚188189571416064 XJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 226592348 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 \HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚185498136100352 ^LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 2264349003264 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚593 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚14268241 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 YEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚183741408812456 VFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 13207863296 trainer/global_step‚2299 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚24903680 G?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚609 _step‚184 P@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 13568115712 ZHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 2251463644412 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚595 TDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 13807648768 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 QDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚41584524 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 L?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚47338443 TDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 13035530240 [IDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 2264349003264 JDeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚31686 PCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚47338443 `LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚181489718575696 ^JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚190558316436480 O?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 13807648768 % _timestamp‚1.7885048600523949e+09 N@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚ 104888361 [KDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 13207863296 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 599785472 E=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚895 VBDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated‚187762502142976 M=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 13568115712 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚31931 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚31686 H?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚1321 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 WGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 12589930496¥æ@QµçO _MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 2368479241216 F8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚ 104862271 PCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚57523828 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 290682880 SEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 384795648 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 QCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 243631708 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚609 C;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚895 WEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 2264117757440 UADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚187762253857792 eQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚188189571416064 XJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 226592348 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 \HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚185498136100352 ^LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 2264349003264 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚593 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚14268241 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 YEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚183741408812456 VFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 13207863296 trainer/global_step‚2299 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚24903680 G?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚609 _step‚184 P@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 13568115712 ZHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 2251463644412 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚595 TDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 13807648768 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 QDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚41584524 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 L?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚47338443 TDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 13035530240 [IDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 2264349003264 JDeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚31686 PCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚47338443 `LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚181489718575696 ^JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚190558316436480 O?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 13807648768 % _timestamp‚1.7885048600523949e+09 N@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚ 104888361 [KDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 13207863296 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 599785472 E=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚895 VBDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated‚187762502142976 M=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 13568115712 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚31931 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚31686 H?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚1321 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 WGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 12589930496 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 586907136 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 231245824 P@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 13118524033 JBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚286 ZHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 2264117757440 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚14268834 BDeviceStatsMonitor.on_train_batch_start/allocation.all.current‚26090 JDeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚187762253857792 UGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 582240777 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚26088 ZFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚190558025753600 MFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚33 aMDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚188189546512384 D>DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚31931 G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚3689 `LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚185498153139712 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚2 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 320831488 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚17039360 YEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚185498136100352¸ÄNGÎ2¶- _wandbruntime‚3938  _runtime‚3938äò&·ìM HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚609 L>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 248298496 L>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚ 104932929 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚31931 cODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚188269166276096 YGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 2265306487808 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚632 XDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚190638616950784 ]KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 2369475578368 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚3689 K>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚41601393 D6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚ 104906813 OADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 248298496 SEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 586907136 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚47358400 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚895 K;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 13568115712 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚17039360 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 231259136 XFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 2265075228672 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 320831488 ZFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚181566548270880 M=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 13807648768 PDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 T@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚187841970836480 H:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚ 104906813 WGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 13207863296 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24903680 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 N>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 13118524033 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚31931 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 WIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 599785472 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17039360 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 trainer/global_step‚2299 _runtime‚3938.437269899 EDeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚26114 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 582240777 YIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 13207863296 _step‚185 NADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚57574527 PBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 586907136 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 \JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 2252642284362 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚286 WCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚187841970836480 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1321 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 152061440 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚895 REDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚57574527 OBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚41602027 REDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚14274212 ^JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚181566565310240 RBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 13807648768 NADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚57548413 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 309543936 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 J=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚57548413 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚14274844 aODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 2369760218624 P@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 13807648768 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 599785472 ^JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚185576664348672 UCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 2265075228672 F6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚895 WCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚185576647309312 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 231259136 N>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 13568115712 NADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚47358400 QCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 384795648 S?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚183818963962686 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 599785472 [GDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚185576664348672 ZFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚185576647309312 XFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 2252415691806 % _timestamp‚1.7885048623575535e+09 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 UEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 12589930496 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚609 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚26116 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚26114 FDeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚26114 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 582240777 YIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 13207863296 _step‚185 NADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚57574527 PBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 586907136 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 \JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 2252642284362 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚286 WCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚187841970836480 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1321 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 152061440 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚895 REDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚57574527 OBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚41602027 REDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚14274212 ^JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚181566565310240 RBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 13807648768 NADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚57548413 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 309543936 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 J=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚57548413 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚14274844 aODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 2369760218624 P@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 13807648768 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 599785472 ^JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚185576664348672 UCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 2265075228672 F6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚895 WCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚185576647309312 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 231259136 N>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 13568115712 NADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚47358400 QCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 384795648 S?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚183818963962686 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 599785472 [GDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚185576664348672 ZFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚185576647309312 XFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 2252415691806 % _timestamp‚1.7885048623575535e+09 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 UEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 12589930496 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚609 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚26116 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚26114 FDeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 248298496 L>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚ 104932929 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚31931 cODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚188269166276096 YGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 2265306487808 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚632 XDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚190638616950784 ]KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 2369475578368 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚3689 K>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚41601393 D6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚ 104906813 OADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 248298496 SEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 586907136 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚47358400 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚895 K;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 13568115712 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚17039360 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 231259136 XFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 2265075228672 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 320831488 ZFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚181566548270880 M=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 13807648768 PDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 T@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚187841970836480 H:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚ 104906813 WGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 13207863296 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24903680 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 N>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 13118524033 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚31931 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 WIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 599785472 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17039360 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 trainer/global_step‚2299 _runtime‚3938.437269899 E‡šÀ‚(: ÂÖéÔýËĺ$ disk.nvme0n1.out‚88017.248046875 disk.dm-1.in‚ 0.6953125 disk.drbd1260.out‚ 0.55078125 cpu‚25.742562727998276 network.recv‚ 1.362048e+06# disk.nvme0c0n1.in‚ 9445.58203125! disk.nvme0n1.in‚ 9445.58203125 disk.dm-0.in‚ 9445.58203125 disk.sda1.out‚0$ memory_percent‚98.95281791687012' proc.memory.availableMB‚ 171.5703125) proc.memory.percent‚18.41146945953369 disk.sda3.in‚ 0.6953125 disk.drbd1260.in‚0 disk.sda1.in‚ 0.1015625 disk.sda2.in‚0 disk.sda2.out‚0$ disk./.usageGB‚1176.200496673584! disk.dm-0.out‚88017.248046875 disk.dm-1.out‚ 250.55078125 disk.sda.in‚0.796875 disk.sda.out‚ 250.55078125# proc.memory.rssMB‚ 3016.53515625 proc.cpu.threads‚17 network.sent‚ 5.475305e+06) disk./.usagePercent‚78.93580222971502 disk.sda3.out‚ 250.55078125& disk.nvme0c0n1.out‚88017.248046875™Ç‚(:‹ ïÖéÔà˜¡») disk./.usagePercent‚78.93658894086775# disk.dm-0.out‚88952.66357421875 disk.drbd1260.in‚0 disk.sda.in‚0.7968û&<ý75 disk.sda.out‚ 252.15625( disk.nvme0c0n1.out‚88952.66357421875$ memory_percent‚98.95288944244385 disk.nvme0n1.in‚ 9599.2421875 disk.dm-1.in‚ 0.6953125 disk.dm-1.out‚ 252.15625 disk.drbd1260.out‚ 0.55859375( proc.memory.availableMB‚ 171.55859375# proc.memory.rssMB‚ 3016.53515625) proc.memory.percent‚18.41146945953369 cpu‚25.741590500030583% disk./.usageGB‚1176.2122192382812& disk.nvme0n1.out‚88952.66357421875 disk.sda1.out‚0 disk.sda3.in‚ 0.6953125 disk.sda3.out‚ 252.15625" disk.nvme0c0n1.in‚ 9599.2421875 proc.cpu.threads‚17 network.sent‚ 5.491414e+06 disk.dm-0.in‚ 9599.2421875 disk.sda1.in‚ 0.1015625 disk.sda2.in‚0 disk.sda2.out‚0 network.recv‚ 1.373396e+06ôÒz§2È- _wandbruntime‚4027  _runtime‚4027ãy†ÅÉ¿ % lr-AdamW/pg1‚0.000999980913980001 % lr-AdamW/pg2‚0.000999980913980001 trainer/global_step‚2349 % _timestamp‚1.7885049511946504e+09 _runtime‚4027.273817279 _step‚187ṧÊÊÄ _runtime‚4027.273817279 _step‚187 % lr-AdamW/pg1‚0.000999980913980001 % lr-AdamW/pg2‚0.000999980913980001 trainer/global_step‚2349 % _timestamp‚1.7885049511946504e+09»€¥ãí2Ë- _wandbruntime‚4027  _runtime‚40274CŒç'ÌáO GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 QCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 243631708 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 586907136 eQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚192169314417664 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚31931 MFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚33 @8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚895 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 D>DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 F8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚ 107092723 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 248285184 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚615 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚609 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 trainer/global_step‚2349 P@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 13568115712 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚26090 ]IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚189423713587712 ZHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 2312019147264 L?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚58756430 ^JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚194588236275200 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚24903680 O?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 13807648768 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚26088 UADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚187631131449656 YEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚187631375081364 TDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 13807648768 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 VFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 13207863296 `LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚189423713587712 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚17039360 SEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 384795648 WEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 2312019147264 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 % _timestamp‚1.7885049511954916e+09 ZHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 2299093186120 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚2 M@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚42460794 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 599785472 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚3689 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 PCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚48336295 M=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 13568115712 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 231245824 ^LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 2299319778468 PCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚58756430 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚31686 PCDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated‚58782518 TFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 599785472 _runtime‚ 4027.27456598 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚2 N@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 248285184 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚31931 N@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚ 107118813 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 320831488 XJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 226592348 JBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚609 TGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚48336295 TGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚58782518 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚27890340 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚613 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 UGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 582240777 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 G?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚609 G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚3689 QDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚42461409 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚14571069 BDeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚191735715695616 WGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 13035530240 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚26090 OFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚1299 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 H?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚1321 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚14570456 `LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚185332055302896 E=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚895 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 @8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚895 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚27890338 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 586907136 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 231245824 YIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 13207863296 YEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚191735963980800 \HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚185332038263536 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 _step‚188 JDeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚31686 YEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚189423696548352 PCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚48336293 aMDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚192169289513984 C;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚895 RBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 13807648768 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚286 L?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚48336293 [IDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 2312250393088 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 WGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 12589930496 [KDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 13207863296 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 cQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 2418921857536 _MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 2418639301120 JDeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚3689 QDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚42461409 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚14571069 BDeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚191735715695616 WGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 13035530240 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚26090 OFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚1299 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 H?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚1321 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚14570456 `LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚185332055302896 E=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚895 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 @8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚895 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚27890338 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 586907136 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 231245824 YIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 13207863296 YEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚191735963980800 \HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚185332038263536 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 _step‚188 JDeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚31686 YEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚189423696548352 PCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚48336293 aMDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚192169289513984 C;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚895 RBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 13807648768 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚286 L?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚48336293 [IDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 2312250393088 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 WGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 12589930496 [KDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 13207863296 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 cQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 2418921857536 _MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 2418639301120 JDeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 F8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚ 107092723 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 248285184 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚615 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚609 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 trainer/global_step‚2349 P@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 13568115712 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚26090 ]IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚189423713587712 ZHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 2312019147264¼¬QhÞ2Î- _wandbruntime‚4028  _runtime‚4028 Õ&~ò&ÏìM RBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 13807648768 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 599785472 QCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 384795648 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 NADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚48356250 MDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚1299 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚895 N>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 13568115712 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17039360 L>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚ 107163381 REDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚48356252 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 320831488 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 599785472 OADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 248298496 \HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚194668829246464 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 PBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 586907136 XDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚194668530188288 cODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚192248909277696 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚286 L>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 248298496 trainer/global_step‚2349 % _timestamp‚1.7885049522506223e+09 RBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 13035530240 YGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 2313207877632 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚17039360 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚895 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚286 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚26116 REDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚27901831 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 P@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 13807648768 TDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 13207863296 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 YIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 13207863296 FDeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚42478529 aODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 2419919968768 WGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 13207863296 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚31686 OBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚42479138 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚609 NADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚58781015 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚2 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚3689 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 299058176 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 243631916 S?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚187708930231594 M=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 13807648768 T@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚191815432674304 PDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 13118524033 ^JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚189502224796672 XFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 2312976618496 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚895 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 XFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 2300045233514 NADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚48356252 [GDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚189502224796672 D6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚ 107137265 _KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚192248884374016 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 NADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚58807129 H:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚ 107137265 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚14577305 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚607 ]KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 2419645814272 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 _runtime‚4028.330461878 H:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚ 107163381 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚31931 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 231259136 SEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 586907136 REDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚58807129 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚33 [MDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚ 274154496 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚895 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚48356250 UEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 13035530240 VHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 226592556¬ŠÛÓÐñM FDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 13118524033 ^JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚189502224796672 XFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 2312976618496 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚895 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 XFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 2300045233514 NADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚48356252 [GDeviceStatsMonitor.on_train_batch_end/active_bytes.LÛ_$#large_pool.allocated‚189502224796672 D6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚ 107137265 _KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚192248884374016 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 NADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚58807129 H:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚ 107137265 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚14577305 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚607 ]KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 2419645814272 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 _runtime‚4028.330461878 H:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚ 107163381 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚31931 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 231259136 SEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 586907136 REDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚58807129 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚33 [MDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚ 274154496 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚895 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚48356250 UEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 13035530240 VHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 226592556 RBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 13807648768 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 599785472 QCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 384795648 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 NADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚48356250 MDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚1299 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚895 N>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 13568115712 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17039360 L>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚ 107163381 REDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚48356252 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 320831488 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 599785472 OADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 248298496 \HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚194668829246464 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 PBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 586907136 XDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚194668530188288 cODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚192248909277696 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚286 L>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 248298496 trainer/global_step‚2349 % _timestamp‚1.7885049522506223e+09 RBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 13035530240 YGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 2313207877632 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚17039360 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚895 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚286 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚26116 REDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚27901831 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 P@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 13807648768 TDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 13207863296 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 YIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 13207863296 FDeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚42478529 aODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 2419919968768 WGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 13207863296 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚31686 OBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚42479138 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚609 NADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚58781015 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚2 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚3689 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 299058176 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 243631916 S?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚187708930231594 M=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 13807648768 T@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚191815432674304 PDeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 TGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚60015120 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 BDeviceStatsMonitor.on_train_batch_start/allocation.all.current‚26090 TFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 599785472 MFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚33 XJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 226592348 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚26088 H?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚1321 _MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 2468872619520 F8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚ 109323175 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚17039360 QDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚43337130 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚595 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 ^LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 2346949320176 G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚3689 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 586907136 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚28464990 ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 286750720 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 ZFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚198617905135104 YEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚193349256996352 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 UGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 582240777 TDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 13807648768 VFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 13207863296 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚2 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚31686 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚14871547 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 599785472 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 trainer/global_step‚2399 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚24903680 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 \HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚189174374990736 YIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 13207863296 _step‚192 P@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 13568115712 PCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚59989032 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚28464988 O?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 13807648768 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 599785472 _runtime‚ 4098.48830272 L?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚59989032 ^LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 2360151782912 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚31931 M@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚43336535 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 RBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 13807648768 [IDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 2360151782912 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚14872140 SEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 384795648 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 WGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 12589930496 JDeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚195709177533440 `LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚193349274035712 \HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚193349256996352 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 [KDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 13207863296 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 586907136 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 311654400 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚286 PCDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated‚60015120 VBDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated‚195709425818624 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 248285184 OFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚1299 ^JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚198618216789504 P@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 13118524033 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 C;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚895 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 231245824 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 E=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚895 N@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 248285184 UADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚191521097718564 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚593 @8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚895 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 WEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 2359920537088 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚31686 G?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚286ó¿î¯ ãæO ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 320831488 YEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚191521341350272 `LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚189174392030096 ]IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚193349274035712 N@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚ 109349265 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚26088 PCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚49334143 TDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 13035530240 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 231245824 ZHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 2359920537088 eQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚196149057419264 M=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 13568115712 D>DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 TGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚60015120 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 BDeviceStatsMonitor.on_train_batch_start/allocation.all.current‚26090 TFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 599785472 MFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚33 XJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 226592348 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚26088 H?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚1321 _MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 2468872619520 F8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚ 109323175 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚17039360 QDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚43337130 JBDeviceStatsMonitor.on_train_batch_start/inactivepc1=_split.all.current‚595 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 ^LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 2346949320176 G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚3689 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 586907136 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚28464990 ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 286750720 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 ZFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚198617905135104 YEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚193349256996352 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 UGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 582240777 TDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 13807648768 VFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 13207863296 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚2 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚31686 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚14871547 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 599785472 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 trainer/global_step‚2399 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚24903680 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 \HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚189174374990736 YIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 13207863296 _step‚192 P@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 13568115712 PCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚59989032 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚28464988 O?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 13807648768 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 599785472 _runtime‚ 4098.48830272 L?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚59989032 ^LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 2360151782912 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚31931 M@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚43336535 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 RBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 13807648768 [IDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 2360151782912 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚14872140 SEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 384795648 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 WGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 12589930496 JDeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚195709177533440 `LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚193349274035712 \HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚193349256996352 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 [KDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 13207863296 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 586907136 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 311654400 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚286 PCDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated‚60015120 VBDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated‚195709425818624 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 248285184 OFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚1299 ^JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚198618216789504 P@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 13118524033 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 C;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚895 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 231245824 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 E=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚895 N@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 248285184 UADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚191521097718564 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚593 @8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚895 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 WEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 2359920537088 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚31686 G?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚286 JDeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚43354039 RBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 13807648768 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 RBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 13035530240 WCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚191599140132418 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 582240777 F8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 WGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 13207863296 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚286 T@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚195788894512128 NADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚49354100 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 320831488 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 P@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 13807648768 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚2 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚49354100 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚895 BDeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚ 109393833 ^JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚189251238764640 H:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚ 109393833 K;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 13568115712 OADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 248298496 MDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚1299 aODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 2470149781504 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚895 D6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚ 109367717 ZFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚193427768205312 REDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚28476481 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 NADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚60039731 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1321 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚895 % _timestamp‚1.7885050233107023e+09 PDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 13118524033 UCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 2360878008320 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 288572416 S?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚191598896500502 YIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 13207863296 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚895 N>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 13568115712 XFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 2360878008320 REDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚49354102 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚2 _KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚196228627375616 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 231259136 ]KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 2469886112768 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 OBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚43354601 [MDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚ 263668736 M=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 13807648768 _runtime‚4099.390149156 L>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 248298496 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚3689 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 152061440 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 243631916 ^JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚193427785244672 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚609 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24903680 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 599785472 trainer/global_step‚2399 \JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 2361109267456 REDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚14877558 WCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚193427768205312 XFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 2347674775222 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚31931 WCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚195788894512128 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 cODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚196228652279296 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 599785472 NADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚49354102 J=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚60013617 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚28476483®Bè4÷&æñM PDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 13118524033 UCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 2360878008320 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 288572416 S?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚191598896500502 YIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 13207863296 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚895 N>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 13568115712 XFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 2360878008320 REDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚49354102 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚2 _KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚196228627375616 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 231259136 ]KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 2469886112768 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 OBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚43354601 [MDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚ 263668736 M=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 13807648768 _runtime‚4099.390149156 L>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 248298496 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚3689 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 152061440 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 243631916 ^JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚193427785244672 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚609 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24903680 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 599785472 trainer/global_step‚2399 \JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 2361109267456 REDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚14877558 WCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚193427768205312 XFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 2347674775222 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚31931 WCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚195788894512128 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 cODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚196228652279296 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 599785472 NADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚49354102 J=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚60013617 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚28476483 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 K>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚43354039 RBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 13807648768 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 RBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 13035530240 WCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚191599140132418 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 582240777 F8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 WGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 13207863296 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚286 T@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚195788894512128 NADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚49354100 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 320831488 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 P@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 13807648768 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚2 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚49354100 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚895 BDeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚ 109393833 ^JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚189251238764640 H:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚ 109393833 K;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 13568115712 OADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 248298496 MDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚1299 aODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 2470149781504 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚895 D6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚ 109367717 ZFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚193427768205312 REDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚28476481 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 NADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚60039731 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1321 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚895 % _timestamp‚1.7885050233107023e+09Á”ô¡µ2ç- _wandbruntime‚4099  _runtime‚4099ËK9¹\èÖ _runtime‚4099.391650906 0loss/FRt2/assignment_loss‚3.5175992511631398 2loss/SRqqt2/assignment_loss‚1.4452544921524313 /loss/FRt4/detection_loss‚1.8179371058940887 trainer/global_step‚2399 $ _timestamp‚1.788505023313152e+09 2loss/SRqqt3/assignment_loss‚1.7375275554453455 0loss/SRqqt4/assignment_loss‚1.03320485769779 1loss/FBt4/assignment_loss‚0.20966113245022144 1loss/SRqqt4/detection_loss‚0.9420192637759396 0loss/FRt4/assignment_loss‚1.6847707659770281 1loss/FBt2/assignment_loss‚0.37366639955989844 /loss/FRt2/detection_loss‚2.4970653458927456 1loss/FBt3/assignment_loss‚0.09142584206704342 /loss/FRt1/detection_loss‚2.5287017021023175 /loss/FBt4/detection_loss‚1.6979823195934296 epoch‚3 0loss/FRt1/assignment_loss‚3.5932487132119353 0loss/FRt3/assignment_loss‚2.4100957259739917 1loss/SRqqt1/assignment_loss‚2.120855177532543 /loss/FBt2/detection_loss‚2.4915736534378747 /loss/FBt3/detection_loss‚0.8056713827254791 1loss/FBt1/assignment_loss‚0.21152238464251613 1loss/SRqqt2/detection_loss‚2.2938061936260903 1loss/SRqqt3/detection_loss‚2.4679184621427117 -loss/FBt1/detection_loss‚1.92277537326555 _step‚194 -loss/FRt3/detection_loss‚2.16345970424479 1loss/SRqqt1/detection_loss‚2.2011604650454086 &loss/total_loss‚42.2589032696203050Çâ¬aéÛ 1loss/SRqqt1/assignment_loss‚2.120855177532543 /loss/FBt2/detection_loss‚2.4915736534378747 /loss/FBt3/detection_loss‚0.8056713827254791 1loss/FBt1/assignment_loss‚0.21152238464251613 1loss/SRqqt2/detection_loss‚2.2938061936260903 1loss/SRqqt3/detection_loss‚2.4679184621427117 -loss/FBt1/detection_loss‚1.92277537326555 _step‚194 -loss/FRt3/detection_loss‚2.16345970424479 1loss/SRqqt1/detection_loss‚2.2011604650454086 &loss/total_loss‚42.258903269620305 _runtime‚4099.391650906 0loss/FRt2/assignment_loss‚3.5175992511631398 2loss/SRqqt2/assignment_loss‚1.4452544921524313 /loss/FRt4/detection_loss‚1.8179371058940887 trainer/global_step‚2399 $ _timestamp‚1.788505023313152e+09 2loss/SRqqt3/assignment_loss‚1.7375275554453455 0loss/SRqqt4/assignment_loss‚1.03320485769779 1loss/FBt4/assignment_loss‚0.20966113245022144 1loss/SRqqt4/detection_loss‚0.9420192637759396 0loss/FRt4/assignment_loss‚1.6847707659770281 1loss/FBt2/assignment_loss‚0.37366639955989844 /loss/FRt2/detection_loss‚2.4970653458927456 1loss/FBt3/assignment_loss‚0.09142584206704342 /loss/FRt1/detection_loss‚2.5287017021023175 /loss/FBt4/detection_loss‚1.6979823195934296 epoch‚3 0loss/FRt1/assignment_loss‚3.5932487132119353 0loss/FRt3/assignment_loss‚2.4100957259739917­ [Å“ê‚(:ˆ É×éÔ‚£þ¸ gpu.0.gpu‚49.0 gpu.0.memory‚4, gpu.0.memoryAllocated‚16.718826293945312* gpu.0.memoryAllocatedBytes‚ 14361362432 gpu.0.temp‚35.0 gpu.0.powerWatts‚131.516' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚26.3032 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0øMë‚(:’ É×éÔޯм% disk./.usageGB‚1175.4800453186035 disk.sda2.in‚0 disk.sda3.in‚ 0.6953125 disk.sda3.out‚ 254.49609375" disk.nvme0c0n1.in‚ 9749.6796875 disk.drbd1260.in‚0 disk.sda1.out‚0 proc.cpu.threads‚17 network.sent‚ 5.605595e+06 disk.sda.in‚0.796875 disk.sda2.out‚0( disk.nvme0c0n1.out‚91085.47998046875 disk.dm-0.in‚ 9749.6796875 disk.dm-1.in‚ 0.6953125 disk.dm-1.out‚ 254.49609375$ memory_percent‚98.95429611206055 network.recv‚ 1.398671e+06) disk./.usagePercent‚78.88745213478335 disk.nvme0n1.in‚ 9749.6796875& disk.nvme0n1.out‚91085.47998046875 disk.sda1.in‚ 0.1015625) proc.memory.percent‚18.41146945953369 cpu‚25.738327283119972 disk.sda.out‚ 254.49609375# disk.dm-0.out‚91085.47998046875 disk.drbd1260.out‚ 0.57421875& proc.memory.availableMB‚ 171.328125# proc.memory.rssMB‚ 3016.53515625\€Ežì‚(:“ Ø×éÔ¼¹¹ gpu.0.gpu‚49.0 gpu.0.memory‚3, gpu.0.memoryAllocated‚16.718826293945312* gpu.0.memoryAllocatedBytes‚ 14361362432 gpu.0.temp‚35.0 gpu.0.powerWatts‚177.322' gpu.0.enforcedPowerLimitWatts‚500.0) gpu.0.powerPercent‚35.464400000000005 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0,ѧ›í‚(: Ø×éÔ®ËÆ» disk.dm-0.in‚ 9870.796875$ memory_percent‚98.95346164703369( proc.memory.availableMB‚ 171.46484375" proc.memory.rssMB‚ 3015.6328125 network.sent‚ 5.694101e+06 disk.sda3.out‚ 254.91796875! disk.nvme0c0n1.in‚ 9870.796875 disk.nvme0n1.in‚ 9870.796875 disk.dm-1.in‚ 0.6953125 disk.drbd1260.out‚ 0.57421875 disk.sda1.out‚0 disk.sda.in‚0.796875 disk.sda2.in‚0# disk.dm-0.out‚91375.19091796875 disk.drbd1260.in‚0 disk.sda.out‚ 254.91796875 proc.cpu.threads‚17) disk./.usagePercent‚78.88745981500936% disk./.usageGB‚1175.4801597595215( disk.nvme0c0n1.out‚91375.19091796875 disk.dm-1.out‚ 254.91796875 disk.sda1.in‚ 0.1015625* proc.memory.percent‚18.405961990356445 cpu‚25.75072626043426 network.recv‚ 1.403675e+06 disk.sda2.out‚0 disk.sda3.in‚ 0.6953125& disk.nvme0n1.out‚91375.19091796875èîŽÄ’î‚(:‡ ç×éÔ°Ïθ gpu.0.gpu‚2.0 gpu.0.memory‚0, gpu.0.memoryAllocated‚16.718826293945312* gpu.0.memoryAllocatedBytes‚ 14361362432 gpu.0.temp‚34.0 gpu.0.powerWatts‚141.769' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚28.3538 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0ùƒcï‚(:‹ ç×éÔ©¡€½ disk.sda1.in‚ 0.1015625 disk.sda2.out‚0 disk.dm-1.out‚ 255.3h›P36328125) disk./.usagePercent‚78.88745725493402 disk.sda3.in‚ 0.6953125 disk.sda3.out‚ 255.36328125! disk.nvme0c0n1.in‚ 9870.796875 disk.sda1.out‚0 disk.sda.out‚ 255.36328125 disk.sda2.in‚0( disk.nvme0c0n1.out‚91693.29638671875 disk.nvme0n1.in‚ 9870.796875 disk.drbd1260.out‚0.578125 disk.sda.in‚0.796875$ memory_percent‚98.95830154418945& proc.memory.availableMB‚ 170.671875# proc.memory.rssMB‚ 3016.49609375) proc.memory.percent‚18.41123104095459& disk.nvme0n1.out‚91693.29638671875 disk.dm-0.in‚ 9870.796875# disk.dm-0.out‚91693.29638671875 disk.dm-1.in‚ 0.6953125 disk.drbd1260.in‚0 cpu‚25.776557260169607 proc.cpu.threads‚17 network.sent‚ 5.7001e+06 network.recv‚ 1.408235e+06% disk./.usageGB‚1175.4801216125488´°$Ç‘ð‚(:† ö×éÔ‚”«¹ gpu.0.gpu‚0.0 gpu.0.memory‚0, gpu.0.memoryAllocated‚16.718826293945312* gpu.0.memoryAllocatedBytes‚ 14361362432 gpu.0.temp‚31.0 gpu.0.powerWatts‚93.277' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚18.6554 gpu.0.smClock‚1275 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0!4ðšñ‚(: ö×éÔÞä´¼ disk.sda3.in‚ 0.6953125 disk.dm-1.in‚ 0.6953125 disk.drbd1260.in‚0 disk.sda.out‚ 255.70703125$ memory_percent‚98.95870685577393 disk.sda3.out‚ 255.70703125! disk.nvme0c0n1.in‚ 9870.796875# disk.dm-0.out‚91693.64013671875 disk.sda1.in‚ 0.1015625( proc.memory.availableMB‚ 170.60546875# proc.memory.rssMB‚ 3016.49609375 cpu‚25.795864175625553 proc.cpu.threads‚17 network.recv‚ 1.412881e+06 disk.sda2.in‚0 disk.dm-0.in‚ 9870.796875 disk.drbd1260.out‚0.578125 disk.sda.in‚0.796875) disk./.usagePercent‚78.88745495086621( disk.nvme0c0n1.out‚91693.64013671875 disk.nvme0n1.in‚ 9870.796875& disk.nvme0n1.out‚91693.64013671875 disk.dm-1.out‚ 255.70703125 disk.sda1.out‚0) proc.memory.percent‚18.41123104095459 network.sent‚ 5.704481e+06% disk./.usageGB‚1175.4800872802734 disk.sda2.out‚08ǹ¡“ò‚(:ˆ …ØéÔÃǸ gpu.0.gpu‚14.0 gpu.0.memory‚1, gpu.0.memoryAllocated‚16.718826293945312* gpu.0.memoryAllocatedBytes‚ 14361362432 gpu.0.temp‚34.0 gpu.0.powerWatts‚171.041' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚34.2082 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0æ}˜Üœó‚(:‘ …ØéÔ„Ó©º disk.drbd1260.out‚ 0.58203125 disk.sda1.in‚ 0.1015625 disk.sda2.out‚0( disk.nvme0c0n1.out‚91984.84814453125# disk.dm-0.out‚91984.84814453125) proc.memory.percent‚18.41123104095459 network.recv‚ 1.416441e+06% disk./.usageGB‚1175.3711891174316 disk.drbd1260.in‚0 disk.sda.out‚ 256.06640625 disk.sda3.out‚ 256.06640625 disk.nvme0n1.in‚ 9870.796875# proc.memory.rssMB‚ 3016.49609375) disk./.usagePercent‚78.88014670378625 disk.sda.in‚0.796875 disk.sda2.in‚0 disk.sda3.in‚ 0.6953125& disk.nvme0n1.out‚91984.84814453125 disk.dm-0.in‚ 9870.796875 disk.dm-1.in‚ 0.6953125( proc.memory.availableMB‚ 170.58984375 disk.sda1.out‚0! disk.nvme0c0n1.in‚ 9870.796875 disk.dm-1.out‚ 256.06640625$ memory_percent‚98.95880222320557 cpu‚25.809776730365034 proc.cpu.threads‚17 network.sent‚ 5.709514e+06%A½2ô- _wandbruntime‚4170  _runtime‚4170!Gݳrõ‚¦ ;"Purity/*FRt2SRqqt*FBt/event_purity‚0.019272908568382263 9"Purity/1FRt0SRqqt1FBt/event_purity‚0.2547757923603058 ,&Purity/1FRt2SRqqt3FBt/event_proportion‚0 ,&Purity/4FRt0SRqqt2FBt/event_proportion‚0 *jet/accuracy_1_of_1‚0.4251188635826111 jet/accuracy_5_of_5‚0 9"Purity/1FRt*SRqqt2FBt/SRqqt_purity‚0.5328981280326843 7 Purity/1FRt1SRqqt1FBt/FRt_purity‚0.3501157760620117 ?&Purity/2FRt*SRqqt1FBt/event_proportion‚0.021169355139136314 8 Purity/2FRt*SRqqt1FBt/FRt_purity‚0.38005712628364563 @&Purity/2FRt*SRqqt3FBt/event_proportion‚6.300403038039804e-05 7 Purity/2FRt0SRqqt0FBt/FRt_purity‚0.1580076366662979 ("Purity/2FRt3SRqqt0FBt/event_purity‚0 & Purity/1FRt1SRqqt4FBt/FBt_purity‚1 7 Purity/*FRt0SRqqt2FBt/FBt_purity‚0.9910476207733154 & Purity/4FRt1SRqqt*FBt/FRt_purity‚0 ("Purity/0FRt2SRqqt3FBt/SRqqt_purity‚0 *"Purity/4FRt2SRqqt0FBt/SRqqt_purity‚0.5 >&Purity/0FRt0SRqqt1FBt/event_proportion‚0.08606351166963577 ,&Purity/1FRt3SRqqt3FBt/event_proportion‚0 ("Purity/2FRt2SRqqt*FBt/event_purity‚0 ,&Purity/4FRt*SRqqt4FBt/event_proportion‚0 particle/accuracy_3_of_4‚1 particle/accuracy_6_of_7‚1 :"Purity/2FRt*SRqqt2FBt/event_purity‚0.05833333358168602 >&Purity/2FRt1SRqqt0FBt/event_proportion‚0.03181703761219978 & Purity/2FRt1SRqqt1FBt/FBt_purity‚1 ,&Purity/2FRt3SRqqt3FBt/event_proportion‚0 ,&Purity/4FRt0SRqqt4FBt/event_proportion‚0 9"Purity/1FRt1SRqqt*FBt/SRqqt_purity‚0.4543781876564026 @&Purity/3FRt0SRqqt1FBt/event_proportion‚0.0011340725468471646 @&Purity/3FRt0SRqqt2FBt/event_proportion‚6.300403038039804e-05 ;"Purity/1FRt1SRqqt0FBt/event_purity‚0.010184351354837418 particle/accuracy_2_of_2‚1 ("Purity/*FRt0SRqqt4FBt/event_purity‚0 ("Purity/*FRt4SRqqt*FBt/event_purity‚0 ("Purity/3FRt0SRqqt2FBt/event_purity‚0 +"Purity/4FRt*SRqqt0FBt/SRqqt_purity‚0.75 9"Purity/*FRt1SRqqt2FBt/SRqqt_purity‚0.5415419936180115 ("Purity/1FRt3SRqqt*FBt/event_purity‚0 ("Purity/3FRt2SRqqt0FBt/event_purity‚0 (particle/f_score‚0.34177660942077637 ("Purity/0FRt4SRqqt1FBt/event_purity‚0 & Purity/2FRt*SRqqt3FBt/FBt_purity‚1 ("Purity/3FRt3SRqqt0FBt/event_purity‚0 & Purity/1FRt1SRqqt3FBt/FBt_purity‚1 ,&Purity/*FRt3SRqqt4FBt/event_proportion‚0 9"Purity/0FRt0SRqqt3FBt/event_purity‚0.1899999976158142 @&Purity/0FRt2SRqqt2FBt/event_proportion‚0.0010080644860863686 )loss/val_total_loss‚42.25387696344677 =&Purity/*FRt*SRqqt*FBt/event_proportion‚0.9591733813285828 ?&Purity/1FRt1SRqqt2FBt/event_proportion‚0.006111391354352236 ?&Purity/2FRt0SRqqt0FBt/event_proportion‚0.024571573361754417 @&Purity/3FRt*SRqqt2FBt/event_proportion‚0.0003150201519019902 @&Purity/4FRt1SRqqt*FBt/event_proportion‚6.300403038039804e-05 8"Purity/0FRt1SRqqt3FBt/SRqqt_purity‚0.699999988079071 ("Purity/1FRt1SRqqt4FBt/SRqqt_purity‚0 7 Purity/*FRt1SRqqt1FBt/FBt_purity‚0.9988254308700562 >&Purity/*FRt2SRqqt1FBt/event_proportion‚0.02255544438958168 ("Purity/0FRt0SRqqt0FBt/event_purity‚1 9"Purity/2FRt1SRqqt*FBt/SRqqt_purity‚0.4450591504573822 ."Purity/0FRt4SRqqt*FBt/SRqqt_purity‚0.21875 ("Purity/2FRt4SRqqt0FBt/event_purity‚0 jet/accuracy_2_of_7‚0 ( Purity/1FRt*SRqqt4FBt/FRt_purity‚0.5 8 Purity/*FRt3SRqqt0FBt/FRt_purity‚0.15494666993618011 9"Purity/1FRt2SRqqt0FBt/SRqqt_purity‚0.3806386888027191 *jet/accuracy_1_of_3‚0.8155019283294678 (particle/accuracy‚0.8391453623771667 8 Purity/2FRt4SRqqt*FBt/FRt_purity‚0.20000000298023224 8 Purity/*FRt0SRqqt2FBt/FRt_purity‚0.48945343494415283 9"Purity/1FRt2SRqqt*FBt/SRqqt_purity‚0.4069845676422119 ,&Purity/2FRt*SRqqt4FBt/event_proportion‚0 ,&Purity/3FRt3SRqqt4FBt/event_proportion‚0 *"Purity/0FRt4SRqqt1FBt/SRqqt_purity‚0.5 8 Purity/2FRt3SRqqt1FBt/FRt_purity‚0.10000000149011612 particle/accuracy_1_of_7‚1 ("Purity/*FRt3SRqqt2FBt/event_purity‚0 :"Purity/1FRt0SRqqt2FBt/event_purity‚0.12220816314220428 ?&Purity/1FRt0SRqqt2FBt/event_proportion‚0.015120967291295528 ,&Purity/*FRt3SRqqt3FBt/event_proportion‚0 :"Purity/0FRt*SRqqt3FBt/event_purity‚0.17592592537403107 ,&Purity/4FRt*SRqqt1FBt/event_proportion‚0 ,&Purity/4FRt1SRqqt2FBt/event_proportion‚0 7 Purity/*FRt2SRqqt1FBt/FBt_purity‚0.9940702319145203 ,&Purity/*FRt4SRqqt2FBt/event_proportion‚0 9"Purity/0FRt2SRqqt0FBt/SRqqt_purity‚0.3624584972858429 A&Purity/1FRt1SRqqt3FBt/event_proportion‚0.00025201612152159214 @&Purity/3FRt3SRqqt*FBt/event_proportion‚0.0005670362734235823 @&Purity/3FRt3SRqqt0FBt/event_proportion‚0.0005670362734235823 ("Purity/*FRt4SRqqt0FBt/event_purity‚0 ( Purity/*FRt*SRqqt4FBt/FRt_purity‚0.5 A&Purity/0FRt1SRqqt3FBt/event_proportion‚0.00044102821266278625 ("Purity/3FRt0SRqqt1FBt/event_purity‚0 & Purity/1FRt4SRqqt0FBt/FRt_purity‚0 & Purity/3FRt*SRqqt1FBt/FBt_purity‚1 ,&Purity/4FRt4SRqqt4FBt/event_proportion‚0 ("Purity/2FRt*SRqqt3FBt/event_purity‚0 7 Purity/1FRt1SRqqt2FBt/FRt_purity‚0.4656462073326111 & Purity/1FRt3SRqqt*FBt/FBt_purity‚1 ,&Purity/1FRt4SRqqt4FBt/event_proportion‚0 8 Purity/2FRt*SRqqt2FBt/FRt_purity‚0.49972227215766907 ,&Purity/3FRt4SRqqt3FBt/event_proportion‚0 9"Purity/*FRt3SRqqt2FBt/SRqqt_purity‚0.3333333432674408 & Purity/4FRt*SRqqt*FBt/FRt_purity‚0 @&Purity/1FRt1SRqqt4FBt/event_proportion‚6.300403038039804e-05 @&Purity/3FRt*SRqqt1FBt/event_proportion‚0.0024571572430431843 ,&Purity/4FRt2SRqqt2FBt/event_proportion‚0 ("Purity/*FRt1SRqqt4FBt/SRqqt_purity‚0 @&Purity/*FRt1SRqqt4FBt/event_proportion‚6.300403038039804e-05 ,&Purity/2FRt4SRqqt3FBt/event_proportion‚0 ;"Purity/*FRt1SRqqt2FBt/event_purity‚0.017473118379712105 >&Purity/0FRt*SRqqt1FBt/event_proportion‚0.14787046611309052 @&Purity/0FRt2SRqqt3FBt/event_proportion‚6.300403038039804e-05 7 Purity/*FRt0SRqqt3FBt/FBt_purity‚0.9271826148033142 @&Purity/*FRt4SRqqt1FBt/event_proportion‚6.300403038039804e-05 7 Purity/0FRt*SRqqt1FBt/FBt_purity‚0.9992362856864929 9"Purity/0FRt1SRqqt0FBt/SRqqt_purity‚0.3709058165550232 & Purity/2FRt2SRqqt1FBt/FBt_purity‚1 8 Purity/3FRt1SRqqt0FBt/FRt_purity‚0.10000000149011612 & Purity/*FRt4SRqqt1FBt/FBt_purity‚1 & Purity/0FRt4SRqqt1FBt/FBt_purity‚1 9"Purity/*FRt1SRqqt*FBt/event_purity‚0.1667831987142563 ,&Purity/0FRt4SRqqt4FBt/event_proportion‚0 ,jet/accuracy_3_of_3‚0.024457141757011414 particle/accuracy_4_of_4‚1 jet/accuracy_5_of_7‚0 *"Purity/4FRt2SRqqt*FBt/SRqqt_purity‚0.5 >&Purity/2FRt*SRqqt*FBt/event_proportion‚0.10244455933570862 @&Purity/4FRt2SRqqt0FBt/event_proportion‚6.300403038039804e-05 7 Purity/*FRt*SRqqt1FBt/FBt_purity‚0.9988430738449097 & Purity/0FRt2SRqqt1FBt/FBt_purity‚1 8 Purity/1FRt0SRqqt0FBt/FRt_purity‚0.15637008845806122 <"Purity/2FRt*SRqqt0FBt/event_purity‚0.0031388800125569105 ("Purity/3FRt*SRqqt1FBt/event_purity‚0 9"Purity/*FRt2SRqqt1FBt/SRqqt_purity‚0.5316393971443176 7 Purity/1FRt3SRqqt*FBt/FRt_purity‚0.1742575466632843 9"Purity/2FRt*SRqqt2FBt/SRqqt_purity‚0.7149122357368469 7"Purity/*FRt1SRqqt3FBt/SRqqt_purity‚0.69047611951828 *"Purity/1FRt*SRqqt3FBt/SRqqt_purity‚0.5 7 Purity/0FRt1SRqqt1FBt/FBt_purity‚0.9990783333778381 :"Purity/1FRt*SRqqt2FBt/event_purity‚0.08398813009262085 6 Purity/1FRt1SRqqt2FBt/FBt_purity‚0.988095223903656 7 Purity/2FRt*SRqqt2FBt/FBt_purity‚0.9911110997200012 @&Purity/2FRt4SRqqt*FBt/event_proportion‚0.0003150201519019902 & Purity/3FRt2SRqqt1FBt/FBt_purity‚1 ("Purity/1FRt0SRqqt4FBt/event_purity‚0 8 Purity/*FRt0SRqqt1FBt/FRt_purity‚0.43833544850349426 ,&Purity/*FRt4SRqqt4FBt/event_proportion‚0 8 Purity/1FRt2SRqqt0FBt/FRt_purity‚0.13502077758312225 8 Purity/3FRt0SRqqt0FBt/FRt_purity‚0.16993464529514313 7 Purity/*FRt0SRqqt4FBt/FBt_purity‚0.8500000238418579 9"Purity/3FRt3SRqqt*FBt/SRqqt_purity‚0.2380952388048172 ;"Purity/*FRt2SRqqt1FBt/event_purity‚0.010651323944330215 7 Purity/1FRt*SRqqt1FBt/FBt_purity‚0.9982349276542664 ?&Purity/1FRt3SRqqt0FBt/event_proportion‚0.007938507944345474 ,&Purity/3FRt2SRqqt3FBt/event_proportion‚0 ,&Purity/4FRt3SRqqt2FBt/event_proportion‚0 ( Purity/2FRt*SRqqt3FBt/FRt_purity‚0.5 & Purity/*FRt1SRqqt4FBt/FRt_purity‚0 ( Purity/1FRt1SRqqt3FBt/FRt_purity‚0.5 >&Purity/0FRt0SRqqt0FBt/event_proportion‚0.04082661122083664 ?&Purity/0FRt2SRqqt0FBt/event_proportion‚0.043283771723508835 9"Purity/3FRt*SRqqt1FBt/SRqqt_purity‚0.8666666746139526 @&Purity/4FRt0SRqqt*FBt/event_proportion‚6.300403038039804e-05 epoch‚3 ("Purity/*FRt3SRqqt1FBt/event_purity‚0 :"Purity/0FRt2SRqqt0FBt/event_purity‚0.05581668019294739 8 Purity/2FRt1SRqqt1FBt/FRt_purity‚0.42148616909980774 ?&Purity/2FRt1SRqqt2FBt/event_proportion‚0.001890120911411941 @&Purity/3FRt0SRqqt0FBt/event_proportion‚0.0015120967291295528 ."Purity/*FRt4SRqqt0FBt/SRqqt_purity‚0.21875 & Purity/*FRt2SRqqt3FBt/FBt_purity‚1 7 Purity/*FRt*SRqqt2FBt/FBt_purity‚0.9902760982513428 9"Purity/1FRt1SRqqt2FBt/SRqqt_purity‚0.5318027138710022 ,&Purity/2FRt1SRqqt3FBt/event_proportion‚0 & Purity/0FRt2SRqqt2FBt/FBt_purity‚1 7 Purity/1FRt0SRqqt*FBt/FBt_purity‚0.9950029253959656 @&Purity/1FRt3SRqqt1FBt/event_proportion‚0.0011340725468471646 ("Purity/2FRt3SRqqt1FBt/event_purity‚0 & Purity/4FRt*SRqqt0FBt/FRt_purity‚0 ( Purity/2FRt2SRqqt2FBt/FRt_purity‚0.5 7 Purity/*FRt*SRqqt3FBt/FRt_purity‚0.6764705777168274 ("Purity/1FRt2SRqqt0FBt/event_purity‚0 @&Purity/0FRt4SRqqt1FBt/event_proportion‚6.300403038039804e-05 @&Purity/3FRt2SRqqt0FBt/event_proportion‚0.0024571572430431843 @&Purity/4FRt2SRqqt*FBt/event_proportion‚6.300403038039804e-05 :"Purity/2FRt4SRqqt0FBt/SRqqt_purity‚0.20000000298023224 7 Purity/3FRt*SRqqt2FBt/FRt_purity‚0.3333333432674408 7 Purity/*FRt1SRqqt3FBt/FBt_purity‚0.9603174924850464 ("Purity/*FRt1SRqqt4FBt/event_purity‚0 9"Purity/*FRt3SRqqt1FBt/SRqqt_purity‚0.4193122088909149 7 Purity/1FRt*SRqqt2FBt/FBt_purity‚0.9745225310325623 7 Purity/1FRt2SRqqt*FBt/FBt_purity‚0.9832949638366699 & Purity/3FRt0SRqqt*FBt/FBt_purity‚1 +"Purity/3FRt*SRqqt2FBt/SRqqt_purity‚0.75 @&Purity/3FRt0SRqqt*FBt/event_proportion‚0.0027091733645647764 7 Purity/3FRt1SRqqt*FBt/FRt_purity‚0.1666666716337204 ,&Purity/4FRt*SRqqt2FBt/event_proportion‚0 :"Purity/3FRt2SRqqt0FBt/SRqqt_purity‚0.40942031145095825 =&Purity/*FRt0SRqqt*FBt/event_proportion‚0.3558467626571655 ?&Purity/0FRt0SRqqt2FBt/event_proportion‚0.029674898833036423 7 Purity/0FRt1SRqqt2FBt/FBt_purity‚0.9906481504440308 + Purity/1FRt*SRqqt3FBt/FRt_purity‚0.6875 :"Purity/1FRt0SRqqt0FBt/event_purity‚0.15637008845806122 7 Purity/1FRt1SRqqt1FBt/FBt_purity‚0.9987593293190002 & Purity/2FRt*SRqqt1FBt/FBt_purity‚1 >&Purity/0FRt*SRqqt2FBt/event_proportion‚0.03944052383303642 9"Purity/1FRt1SRqqt0FBt/SRqqt_purity‚0.3994210660457611 ,&Purity/2FRt1SRqqt4FBt/event_proportion‚0 *jet/accuracy_1_of_4‚0.8554442524909973 +jet/accuracy_2_of_5‚0.41363969445228577 :"Purity/2FRt3SRqqt1FBt/SRqqt_purity‚0.36666667461395264 & Purity/2FRt3SRqqt1FBt/FBt_purity‚1 9"Purity/3FRt3SRqqt0FBt/SRqqt_purity‚0.2380952388048172 ;"Purity/2FRt*SRqqt1FBt/event_purity‚0.023485952988266945 ?&Purity/2FRt0SRqqt1FBt/event_proportion‚0.008946572430431843 particle/accuracy_3_of_7‚1 ("Purity/4FRt*SRqqt0FBt/event_purity‚0 ("Purity/2FRt2SRqqt2FBt/event_purity‚0 _step‚195 9"Purity/*FRt*SRqqt1FBt/SRqqt_purity‚0.5545254945755005 8 Purity/*FRt2SRqqt1FBt/FRt_purity‚0.24101616442203522 :"Purity/0FRt*SRqqt2FBt/SRqqt_purity‚0.47768524289131165 8 Purity/1FRt3SRqqt0FBt/FRt_purity‚0.15376342833042145 ,&Purity/1FRt4SRqqt1FBt/event_proportion‚0 ,&Purity/3FRt4SRqqt1FBt/event_proportion‚0 @&Purity/4FRt*SRqqt0FBt/event_proportion‚0.0001890120911411941 & Purity/3FRt*SRqqt2FBt/FBt_purity‚1 :"Purity/*FRt*SRqqt2FBt/event_purity‚0.17380233108997345 9"Purity/*FRt0SRqqt2FBt/event_purity‚0.2371792048215866 9"Purity/*FRt1SRqqt*FBt/SRqqt_purity‚0.4341394305229187 ,&Purity/0FRt2SRqqt4FBt/event_proportion‚0 7 Purity/1FRt*SRqqt2FBt/FRt_purity‚0.4908255338668823 8 Purity/1FRt0SRqqt*FBt/FRt_purity‚0.26090046763420105 @&Purity/1FRt2SRqqt2FBt/event_proportion‚0.0010080644860863686 ,&Purity/3FRt*SRqqt3FBt/event_proportion‚0 7 Purity/0FRt*SRqqt2FBt/FBt_purity‚0.9905280470848083 8 Purity/1FRt2SRqqt*FBt/FRt_purity‚0.15288078784942627 9"Purity/2FRt*SRqqt1FBt/SRqqt_purity‚0.5870416164398193 8 Purity/3FRt0SRqqt*FBt/FRt_purity‚0.27020204067230225 A&Purity/3FRt4SRqqt*FBt/event_proportion‚0.00012600806076079607 particle/accuracy_5_of_5‚1 & Purity/2FRt2SRqqt2FBt/FBt_purity‚1 ("Purity/4FRt2SRqqt*FBt/event_purity‚0 +"Purity/1FRt0SRqqt3FBt/event_purity‚0.25 :"Purity/2FRt3SRqqt0FBt/SRqqt_purity‚0.19404762983322144 +jet/accuracy_3_of_5‚0.09457746893167496 particle/accuracy_4_of_5‚1 jet/accuracy_7_of_7‚0 ("Purity/3FRt1SRqqt2FBt/event_purity‚0 & Purity/*FRt0SRqqt4FBt/FRt_purity‚1 ("Purity/4FRt2SRqqt0FBt/event_purity‚0 8 Purity/*FRt1SRqqt1FBt/FRt_purity‚0.36592239141464233 & Purity/*FRt1SRqqt4FBt/FBt_purity‚1 ("Purity/3FRt3SRqqt*FBt/event_purity‚0 7 Purity/*FRt*SRqqt2FBt/FRt_purity‚0.4640257656574249 :"Purity/*FRt*SRqqt1FBt/event_purity‚0.40456315875053406 & Purity/3FRt1SRqqt*FBt/FBt_purity‚1 particle/accuracy_1_of_5‚1 =&Purity/*FRt1SRqqt1FBt/event_proportion‚0.1001134067773819 ?&Purity/1FRt3SRqqt*FBt/event_proportion‚0.009135584346950054 8"Purity/*FRt0SRqqt*FBt/event_purity‚0.370997816324234 8 Purity/2FRt1SRqqt0FBt/FRt_purity‚0.18965700268745422 ,&Purity/4FRt2SRqqt3FBt/event_proportion‚0 particle/accuracy_1_of_2‚1 9"Purity/0FRt3SRqqt0FBt/SRqqt_purity‚0.2536706328392029 >&Purity/*FRt1SRqqt0FBt/event_proportion‚0.33039313554763794 :"Purity/*FRt1SRqqt1FBt/event_purity‚0.10099740326404572 ?&Purity/*FRt3SRqqt*FBt/event_proportion‚0.020476309582591057 >&Purity/1FRt*SRqqt1FBt/event_proportion‚0.09929435700178146 ,&Purity/3FRt3SRqqt3FBt/event_proportion‚0 @&Purity/*FRt0SRqqt4FBt/event_proportion‚0.0003150201519019902 >&Purity/0FRt0SRqqt*FBt/event_proportion‚0.11857359111309052 :"Purity/1FRt*SRqqt1FBt/event_purity‚0.14193688333034515 ;"Purity/2FRt*SRqqt*FBt/event_purity‚0.008866189047694206 +jet/accuracy_2_of_3‚0.22971779108047485 *jet/accuracy_1_of_5‚0.8804249167442322 ("Purity/*FRt4SRqqt1FBt/event_purity‚0 7 Purity/*FRt0SRqqt*FBt/FBt_purity‚0.9925392270088196 :"Purity/*FRt1SRqqt0FBt/SRqqt_purity‚0.38503947854042053 ,&Purity/2FRt4SRqqt2FBt/event_proportion‚0 ("Purity/1FRt*SRqqt4FBt/event_purity‚0 ("Purity/*FRt*SRqqt4FBt/SRqqt_purity‚0 >&Purity/1FRt0SRqqt0FBt/event_proportion‚0.13142640888690948 8"Purity/1FRt2SRqqt2FBt/SRqqt_purity‚0.569444477558136 ("Purity/1FRt4SRqqt*FBt/event_purity‚0 -"Purity/*FRt0SRqqt3FBt/event_purity‚0.1875 >&Purity/0FRt*SRqqt*FBt/event_proportion‚0.40996724367141724 :"Purity/1FRt*SRqqt0FBt/event_purity‚0.06959802657365799 9"Purity/1FRt*SRqqt1FBt/SRqqt_purity‚0.5761092901229858 & Purity/2FRt0SRqqt1FBt/FBt_purity‚1 *"Purity/*FRt4SRqqt1FBt/SRqqt_purity‚0.5 9"Purity/*FRt1SRqqt1FBt/SRqqt_purity‚0.5738791227340698 ("Purity/4FRt1SRqqt0FBt/SRqqt_purity‚1 9"Purity/*FRt3SRqqt0FBt/SRqqt_purity‚0.2515145242214203 & Purity/*FRt3SRqqt1FBt/FBt_purity‚1 particle/accuracy_2_of_4‚1 & Purity/0FRt3SRqqt1FBt/FBt_purity‚1 >&Purity/*FRt0SRqqt0FBt/event_proportion‚0.15757308900356293 9"Purity/0FRt1SRqqt1FBt/SRqqt_purity‚0.5371935963630676 >&Purity/1FRt1SRqqt1FBt/event_proportion‚0.03944052383303642 ("Purity/2FRt2SRqqt1FBt/event_purity‚0 ,&Purity/2FRt4SRqqt1FBt/event_proportion‚0 +jet/accuracy_2_of_4‚0.35931360721588135 ("Purity/1FRt4SRqqt0FBt/event_purity‚0 ("Purity/4FRt*SRqqt*FBt/event_purity‚0 :"Purity/1FRt*SRqqt*FBt/event_purity‚0.08694488555192947 ."Purity/1FRt*SRqqt3FBt/event_purity‚0.21875 ,&Purity/2FRt3SRqqt4FBt/event_proportion‚0 ,&Purity/3FRt4SRqqt4FBt/event_proportion‚0 ("Purity/*FRt1SRqqt3FBt/event_purity‚0 & Purity/4FRt2SRqqt0FBt/FRt_purity‚0 8 Purity/*FRt2SRqqt0FBt/FRt_purity‚0.13716711103916168 :"Purity/1FRt3SRqqt0FBt/SRqqt_purity‚0.26676908135414124 7 Purity/1FRt3SRqqt1FBt/FRt_purity‚0.2888888716697693 ?&Purity/2FRt2SRqqt0FBt/event_proportion‚0.015435988083481789 ("Purity/0FRt3SRqqt*FBt/event_purity‚0 9"Purity/*FRt2SRqqt2FBt/SRqqt_purity‚0.5789474248886108 & Purity/1FRt0SRqqt4FBt/FRt_purity‚1 :"Purity/0FRt1SRqqt1FBt/event_purity‚0.16205064952373505 ."Purity/2FRt2SRqqt1FBt/SRqqt_purity‚0.59375 A&Purity/3FRt4SRqqt0FBt/event_proportion‚0.00012600806076079607 7 Purity/3FRt3SRqqt*FBt/FRt_purity‚0.1428571492433548 ("Purity/4FRt1SRqqt*FBt/event_purity‚0 7 Purity/*FRt0SRqqt*FBt/FRt_purity‚0.2572295367717743 @&Purity/0FRt3SRqqt1FBt/event_proportion‚0.0006300403038039804 ,&Purity/2FRt2SRqqt3FBt/event_proportion‚0 ("Purity/3FRt2SRqqt*FBt/event_purity‚0 ("Purity/0FRt2SRqqt2FBt/event_purity‚0 ("Purity/0FRt4SRqqt0FBt/event_purity‚0 ,&Purity/0FRt3SRqqt4FBt/event_proportion‚0 ?&Purity/1FRt2SRqqt1FBt/event_proportion‚0.008946572430431843 9"Purity/2FRt1SRqqt2FBt/SRqqt_purity‚0.7149122357368469 ,&Purity/4FRt3SRqqt4FBt/event_proportion‚0 particle/accuracy_6_of_6‚1 particle/accuracy_4_of_7‚1 ,&Purity/3FRt2SRqqt2FBt/event_proportion‚0 7 Purity/*FRt1SRqqt0FBt/FRt_purity‚0.1667749583721161 >&Purity/*FRt*SRqqt1FBt/event_proportion‚0.27079132199287415 7 Purity/0FRt0SRqqt1FBt/FBt_purity‚0.9993913769721985 :"Purity/0FRt2SRqqt*FBt/event_purity‚0.04847293719649315 ("Purity/0FRt*SRqqt4FBt/event_purity‚0 ("Purity/0FRt0SRqqt4FBt/event_purity‚0 >&Purity/1FRt0SRqqt*FBt/event_proportion‚0.19745463132858276 :"Purity/2FRt*SRqqt*FBt/SRqqt_purity‚0.42300543189048767 8 Purity/2FRt1SRqqt*FBt/FRt_purity‚0.24480102956295013 ("Purity/3FRt1SRqqt0FBt/event_purity‚0 @&Purity/4FRt*SRqqt*FBt/event_proportion‚0.0001890120911411941 7 Purity/1FRt0SRqqt3FBt/FBt_purity‚0.9523809552192688 particle/accuracy_3_of_6‚1 particle/accuracy_4_of_6‚1 particle/accuracy_5_of_6‚1 9"Purity/1FRt3SRqqt2FBt/SRqqt_purity‚0.3333333432674408 ,&Purity/0FRt4SRqqt3FBt/event_proportion‚0 =&Purity/1FRt1SRqqt0FBt/event_proportion‚0.1265120953321457 ?&Purity/2FRt2SRqqt*FBt/event_proportion‚0.019468246027827263 9"Purity/3FRt*SRqqt0FBt/SRqqt_purity‚0.3804846704006195 7 Purity/*FRt2SRqqt*FBt/FBt_purity‚0.9931789040565491 6 Purity/2FRt0SRqqt2FBt/FRt_purity‚0.523809552192688 @&Purity/4FRt0SRqqt0FBt/event_proportion‚6.300403038039804e-05 particle/accuracy_2_of_6‚1 & Purity/*FRt3SRqqt2FBt/FBt_purity‚1 =&Purity/*FRt*SRqqt0FBt/event_proportion‚0.6160534024238586 7 Purity/0FRt0SRqqt2FBt/FBt_purity‚0.9908624291419983 =&Purity/1FRt*SRqqt0FBt/event_proportion‚0.3135710656642914 >&Purity/1FRt2SRqqt*FBt/event_proportion‚0.05727066472172737 ("Purity/*FRt*SRqqt4FBt/event_purity‚0 ("Purity/1FRt*SRqqt4FBt/SRqqt_purity‚0 :"Purity/0FRt*SRqqt0FBt/event_purity‚0.29739949107170105 9"Purity/2FRt1SRqqt0FBt/SRqqt_purity‚0.3845084011554718 ,&Purity/3FRt0SRqqt4FBt/event_proportion‚0 ,jet/accuracy_4_of_5‚0.011827956885099411 particle/accuracy_7_of_7‚1 ;"Purity/2FRt0SRqqt0FBt/event_purity‚0.007564478553831577 ,&Purity/3FRt0SRqqt3FBt/event_proportion‚0 8 Purity/3FRt1SRqqt1FBt/FRt_purity‚0.25641024112701416 jet/accuracy_6_of_7‚0 9"Purity/1FRt4SRqqt*FBt/SRqqt_purity‚0.1666666716337204 7 Purity/*FRt1SRqqt*FBt/FBt_purity‚0.9955849051475525 ,&Purity/*FRt4SRqqt3FBt/event_proportion‚0 ,&Purity/4FRt1SRqqt1FBt/event_proportion‚0 ,&Purity/4FRt4SRqqt0FBt/event_proportion‚0 ("Purity/4FRt0SRqqt*FBt/event_purity‚0 9"Purity/*FRt*SRqqt0FBt/event_purity‚0.1419798880815506 @&Purity/2FRt4SRqqt0FBt/event_proportion‚0.0003150201519019902 :"Purity/0FRt3SRqqt*FBt/SRqqt_purity‚0.25449734926223755 :"Purity/0FRt2SRqqt1FBt/event_purity‚0.03238607197999954 :"Purity/1FRt*SRqqt*FBt/SRqqt_purity‚0.42218199372291565 ;"Purity/1FRt1SRqqt1FBt/event_purity‚0.041154488921165466 7 Purity/2FRt1SRqqt*FBt/FBt_purity‚0.9964157938957214 9"Purity/2FRt2SRqqt0FBt/SRqqt_purity‚0.4018875062465668 ,&Purity/3FRt3SRqqt2FBt/event_proportion‚0 ,&Purity/4FRt4SRqqt*FBt/event_proportion‚0 ,particle/sensitivity‚0.26740843057632446 8 Purity/*FRt2SRqqt*FBt/FRt_purity‚0.15563635528087616 9"Purity/1FRt2SRqqt1FBt/SRqqt_purity‚0.5060675740242004 ,&Purity/4FRt4SRqqt2FBt/event_proportion‚0 ("Purity/3FRt4SRqqt0FBt/event_purity‚0 +"Purity/4FRt*SRqqt*FBt/SRqqt_purity‚0.75 ;"Purity/0FRt1SRqqt2FBt/event_purity‚0.041111111640930176 8 Purity/1FRt0SRqqt1FBt/FRt_purity‚0.45259344577789307 & Purity/1FRt0SRqqt4FBt/FBt_purity‚1 & Purity/4FRt2SRqqt*FBt/FRt_purity‚0 7 Purity/*FRt0SRqqt0FBt/FRt_purity‚0.1573752909898758 9"Purity/*FRt2SRqqt0FBt/SRqqt_purity‚0.3782947361469269 9"Purity/0FRt2SRqqt*FBt/SRqqt_purity‚0.3860718905925751 9"Purity/0FRt2SRqqt1FBt/SRqqt_purity‚0.5032898187637329 9"Purity/1FRt1SRqqt1FBt/SRqqt_purity‚0.6145957112312317 7 Purity/1FRt2SRqqt1FBt/FRt_purity‚0.2346390336751938 7 Purity/3FRt0SRqqt1FBt/FRt_purity‚0.4038461446762085 :"Purity/2FRt0SRqqt1FBt/event_purity‚0.05296296626329422 7 Purity/2FRt0SRqqt2FBt/FBt_purity‚0.9940475821495056 +jet/accuracy_2_of_6‚0.45614030957221985 7 Purity/3FRt3SRqqt0FBt/FRt_purity‚0.1428571492433548 & Purity/*FRt3SRqqt2FBt/FRt_purity‚1 8 Purity/*FRt1SRqqt*FBt/FRt_purity‚0.22166764736175537 @&Purity/1FRt4SRqqt*FBt/event_proportion‚0.0003780241822823882 ?&Purity/2FRt1SRqqt1FBt/event_proportion‚0.007875503972172737 9"Purity/3FRt2SRqqt*FBt/SRqqt_purity‚0.4548611342906952 7validation_average_jet_accuracy‚0.39359399676322937 trainer/global_step‚2415 >&Purity/0FRt1SRqqt1FBt/event_proportion‚0.05185231938958168 ("Purity/1FRt2SRqqt1FBt/event_purity‚0 ,&Purity/1FRt4SRqqt2FBt/event_proportion‚0 ;"Purity/2FRt0SRqqt*FBt/event_purity‚0.024065010249614716 ,jet/accuracy_4_of_4‚0.005097564775496721 +jet/accuracy_3_of_6‚0.07017543911933899 ( Purity/2FRt0SRqqt3FBt/FRt_purity‚0.5 ("Purity/4FRt1SRqqt*FBt/SRqqt_purity‚1 7 Purity/0FRt0SRqqt3FBt/FBt_purity‚0.9266667366027832 7 Purity/0FRt1SRqqt*FBt/FBt_purity‚0.9953683018684387 ,&Purity/3FRt4SRqqt2FBt/event_proportion‚0 & Purity/3FRt2SRqqt*FBt/FBt_purity‚1 ,&Purity/2FRt3SRqqt2FBt/event_proportion‚0 A&Purity/3FRt1SRqqt2FBt/event_proportion‚0.00025201612152159214 7 Purity/1FRt*SRqqt*FBt/FBt_purity‚0.9945351481437683 ?&Purity/2FRt0SRqqt2FBt/event_proportion‚0.003465221729129553 8 Purity/2FRt2SRqqt*FBt/FRt_purity‚0.16461481153964996 ,&Purity/4FRt4SRqqt1FBt/event_proportion‚0 7 Purity/*FRt4SRqqt*FBt/FRt_purity‚0.0833333358168602 jet/accuracy_4_of_6‚0 9"Purity/0FRt4SRqqt0FBt/SRqqt_purity‚0.1785714328289032 7 Purity/1FRt2SRqqt2FBt/FBt_purity‚0.9583333134651184 8 Purity/1FRt1SRqqt0FBt/FRt_purity‚0.16055183112621307 @&Purity/1FRt4SRqqt0FBt/event_proportion‚0.0003780241822823882 7 Purity/2FRt*SRqqt0FBt/FRt_purity‚0.1685911864042282 8 Purity/2FRt0SRqqt1FBt/FRt_purity‚0.37792328000068665 particle/accuracy_1_of_4‚1 & Purity/3FRt0SRqqt2FBt/FBt_purity‚1 ?&Purity/*FRt2SRqqt2FBt/event_proportion‚0.002079133177176118 ?&Purity/*FRt3SRqqt1FBt/event_proportion‚0.002142137149348855 >&Purity/0FRt2SRqqt*FBt/event_proportion‚0.05361643061041832 :"Purity/1FRt0SRqqt*FBt/event_purity‚0.17848843336105347 ("Purity/1FRt3SRqqt0FBt/event_purity‚0 ?&Purity/2FRt3SRqqt0FBt/event_proportion‚0.003654233878478408 8 Purity/3FRt*SRqqt*FBt/FRt_purity‚0.17193636298179626 ("Purity/3FRt0SRqqt0FBt/event_purity‚0 =&Purity/*FRt0SRqqt2FBt/event_proportion‚0.0483240932226181 9"Purity/0FRt1SRqqt2FBt/SRqqt_purity‚0.4894576370716095 7 Purity/1FRt0SRqqt2FBt/FBt_purity‚0.9907549619674683 & Purity/4FRt0SRqqt0FBt/FRt_purity‚0 % _timestamp‚1.7885050948096626e+09 7 Purity/*FRt*SRqqt*FBt/FBt_purity‚0.9935318827629089 9"Purity/0FRt*SRqqt*FBt/SRqqt_purity‚0.3962169587612152 & Purity/0FRt2SRqqt*FBt/FBt_purity‚1 ) Purity/3FRt1SRqqt2FBt/FRt_purity‚0.25 9"Purity/1FRt4SRqqt0FBt/SRqqt_purity‚0.1666666716337204 ("Purity/*FRt3SRqqt0FBt/event_purity‚0 9"Purity/0FRt1SRqqt*FBt/SRqqt_purity‚0.4139994978904724 ,&Purity/0FRt1SRqqt4FBt/event_proportion‚0 7 Purity/1FRt0SRqqt3FBt/FRt_purity‚0.6785714030265808 >&Purity/1FRt2SRqqt0FBt/event_proportion‚0.04731602966785431 ,&Purity/3FRt1SRqqt4FBt/event_proportion‚0 particle/accuracy_1_of_6‚1 ,"Purity/3FRt4SRqqt*FBt/SRqqt_purity‚0.375 A&Purity/1FRt*SRqqt4FBt/event_proportion‚0.00012600806076079607 ,&Purity/2FRt0SRqqt4FBt/event_proportion‚0 @&Purity/2FRt3SRqqt1FBt/event_proportion‚0.0003780241822823882 ,&Purity/4FRt3SRqqt0FBt/event_proportion‚0 & Purity/3FRt1SRqqt2FBt/FBt_purity‚1 ("Purity/4FRt1SRqqt0FBt/event_purity‚0 ("Purity/*FRt2SRqqt3FBt/SRqqt_purity‚0 ("Purity/4FRt0SRqqt0FBt/event_purity‚0 :"Purity/*FRt3SRqqt*FBt/SRqqt_purity‚0.26995566487312317 9"Purity/0FRt0SRqqt2FBt/event_purity‚0.3156907558441162 :"Purity/1FRt*SRqqt0FBt/SRqqt_purity‚0.37928831577301025 ("Purity/3FRt1SRqqt*FBt/event_purity‚0 8 Purity/3FRt2SRqqt*FBt/FRt_purity‚0.11921296268701553 ,&Purity/4FRt0SRqqt3FBt/event_proportion‚0 particle/accuracy_5_of_7‚1 & Purity/2FRt0SRqqt3FBt/FBt_purity‚1 ,&Purity/4FRt*SRqqt3FBt/event_proportion‚0 particle/accuracy_1_of_1‚1 + Purity/0FRt0SRqqt4FBt/FBt_purity‚0.8125 *"Purity/1FRt1SRqqt3FBt/SRqqt_purity‚0.5 :"Purity/*FRt1SRqqt0FBt/event_purity‚0.19486361742019653 >&Purity/*FRt3SRqqt0FBt/event_proportion‚0.01827116869390011 7 Purity/0FRt*SRqqt*FBt/FBt_purity‚0.9925971031188965 ?&Purity/0FRt3SRqqt0FBt/event_proportion‚0.006111391354352236 8 Purity/1FRt*SRqqt*FBt/FRt_purity‚0.22683686017990112 8 Purity/1FRt0SRqqt2FBt/FRt_purity‚0.49626123905181885 & Purity/2FRt2SRqqt*FBt/FBt_purity‚1 :"Purity/2FRt3SRqqt*FBt/SRqqt_purity‚0.22809524834156036 ?&Purity/2FRt2SRqqt1FBt/event_proportion‚0.003969253972172737 @&Purity/2FRt2SRqqt2FBt/event_proportion‚6.300403038039804e-05 8 Purity/2FRt3SRqqt0FBt/FRt_purity‚0.12867964804172516 ("Purity/3FRt*SRqqt*FBt/event_purity‚0 @&Purity/3FRt1SRqqt1FBt/event_proportion‚0.0009450604557059705 particle/accuracy_3_of_5‚1 & Purity/*FRt4SRqqt*FBt/FBt_purity‚1 & Purity/0FRt4SRqqt*FBt/FBt_purity‚1 8 Purity/*FRt3SRqqt1FBt/FRt_purity‚0.20000000298023224 & Purity/3FRt*SRqqt*FBt/FBt_purity‚1 7 Purity/1FRt2SRqqt2FBt/FRt_purity‚0.2777777910232544 7 Purity/3FRt0SRqqt2FBt/FRt_purity‚0.6666666865348816 9"Purity/*FRt*SRqqt3FBt/SRqqt_purity‚0.5952380895614624 >&Purity/*FRt*SRqqt2FBt/event_proportion‚0.06747731566429138 & Purity/1FRt3SRqqt1FBt/FBt_purity‚1 ,&Purity/3FRt1SRqqt3FBt/event_proportion‚0 ("Purity/0FRt3SRqqt0FBt/event_purity‚0 & Purity/3FRt4SRqqt*FBt/FRt_purity‚0 jet/accuracy_4_of_7‚0 _runtime‚4170.891647165 >&Purity/*FRt2SRqqt0FBt/event_proportion‚0.10855595022439957 @&Purity/*FRt3SRqqt2FBt/event_proportion‚6.300403038039804e-05 @&Purity/*FRt4SRqqt*FBt/event_proportion‚0.0013230846961960196 =&Purity/*FRt1SRqqt*FBt/event_proportion‚0.4482736885547638 8 Purity/*FRt*SRqqt*FBt/FRt_purity‚0.22412659227848053 9"Purity/*FRt*SRqqt*FBt/SRqqt_purity‚0.4101209044456482 A&Purity/0FRt4SRqqt0FBt/event_proportion‚0.00044102821266278625 ?&Purity/1FRt*SRqqt2FBt/event_proportion‚0.022303426638245583 >&Purity/2FRt1SRqqt*FBt/event_proportion‚0.04158266261219978 ("Purity/2FRt1SRqqt2FBt/event_purity‚0 @&Purity/3FRt1SRqqt0FBt/event_proportion‚0.0025201612152159214 @&Purity/3FRt2SRqqt1FBt/event_proportion‚0.0003780241822823882 +jet/accuracy_2_of_2‚0.09671532362699509 particle/accuracy_2_of_3‚1 :"Purity/2FRt4SRqqt*FBt/SRqqt_purity‚0.20000000298023224 ("Purity/1FRt3SRqqt2FBt/event_purity‚0 ?&Purity/*FRt1SRqqt2FBt/event_proportion‚0.017011089250445366 ,&Purity/*FRt2SRqqt4FBt/event_proportion‚0 ("Purity/3FRt0SRqqt*FBt/event_purity‚0 8 Purity/1FRt1SRqqt*FBt/FRt_purity‚0.21561168134212494 8 Purity/2FRt*SRqqt*FBt/FRt_purity‚0.22618263959884644 ,&Purity/4FRt1SRqqt4FBt/event_proportion‚0 ,&Purity/4FRt2SRqqt1FBt/event_proportion‚0 ("Purity/0FRt2SRqqt3FBt/event_purity‚0 ;"Purity/*FRt2SRqqt0FBt/event_purity‚0.021510345861315727 A&Purity/0FRt*SRqqt4FBt/event_proportion‚0.00025201612152159214 @&Purity/0FRt4SRqqt*FBt/event_proportion‚0.0005040322430431843 7 Purity/2FRt2SRqqt0FBt/FRt_purity‚0.1473909467458725 ,&Purity/2FRt2SRqqt4FBt/event_proportion‚0 ?&Purity/3FRt*SRqqt*FBt/event_proportion‚0.009954636916518211 & Purity/3FRt1SRqqt1FBt/FBt_purity‚1 7 Purity/*FRt4SRqqt0FBt/FRt_purity‚0.0833333358168602 9"Purity/0FRt1SRqqt*FBt/event_purity‚0.3110085129737854 =&Purity/1FRt*SRqqt*FBt/event_proportion‚0.4366179406642914 ("Purity/3FRt2SRqqt1FBt/event_purity‚0 :"Purity/0FRt2SRqqt2FBt/SRqqt_purity‚0.48076921701431274 & Purity/2FRt3SRqqt*FBt/FBt_purity‚1 ("Purity/3FRt4SRqqt*FBt/event_purity‚0 9"Purity/0FRt*SRqqt*FBt/event_purity‚0.4150574207305908 8 Purity/3Øœ³ù FRt*SRqqt0FBt/FRt_purity‚0.11828043311834335 ("Purity/*FRt2SRqqt2FBt/event_purity‚0 & Purity/1FRt3SRqqt2FBt/FBt_purity‚1 & Purity/4FRt0SRqqt*FBt/FRt_purity‚0 ("Purity/2FRt2SRqqt0FBt/event_purity‚0 9"Purity/3FRt2SRqqt1FBt/SRqqt_purity‚0.8333333134651184 ,"Purity/3FRt4SRqqt0FBt/SRqqt_purity‚0.375 9"Purity/*FRt*SRqqt0FBt/SRqqt_purity‚0.3692529499530792 ,&Purity/4FRt3SRqqt3FBt/event_proportion‚0 ,&Purity/0FRt4SRqqt2FBt/event_proportion‚0 @&Purity/2FRt0SRqqt3FBt/event_proportion‚6.300403038039804e-05 ("Purity/2FRt3SRqqt*FBt/event_purity‚0 9"Purity/3FRt*SRqqt*FBt/SRqqt_purity‚0.4603174924850464 8 Purity/3FRt2SRqqt0FBt/FRt_purity‚0.09903382509946823 7 Purity/1FRt2SRqqt1FBt/FBt_purity‚0.9873272180557251 ?&Purity/2FRt*SRqqt2FBt/event_proportion‚0.005418346729129553 & Purity/3FRt0SRqqt1FBt/FBt_purity‚1 8"Purity/3FRt1SRqqt1FBt/SRqqt_purity‚0.807692289352417 9"Purity/3FRt1SRqqt*FBt/SRqqt_purity‚0.5617284178733826 ("Purity/*FRt3SRqqt*FBt/event_purity‚0 :"Purity/0FRt*SRqqt0FBt/SRqqt_purity‚0.36038872599601746 9"Purity/0FRt0SRqqt1FBt/event_purity‚0.9993913769721985 7 Purity/1FRt*SRqqt0FBt/FRt_purity‚0.1549118161201477 @&Purity/1FRt0SRqqt4FBt/event_proportion‚6.300403038039804e-05 ,&Purity/1FRt4SRqqt3FBt/event_proportion‚0 ;"Purity/2FRt1SRqqt0FBt/event_purity‚0.001240694778971374 7 Purity/1FRt1SRqqt*FBt/FBt_purity‚0.9968394637107849 7 Purity/2FRt*SRqqt*FBt/FBt_purity‚0.9961161017417908 ,&Purity/3FRt*SRqqt4FBt/event_proportion‚0 ("Purity/1FRt2SRqqt2FBt/event_purity‚0 & Purity/3FRt4SRqqt0FBt/FRt_purity‚0 particle/accuracy_2_of_7‚1 ("Purity/2FRt0SRqqt3FBt/event_purity‚0 ("Purity/0FRt1SRqqt3FBt/event_purity‚0 :"Purity/*FRt*SRqqt*FBt/event_purity‚0.21816574037075043 ("Purity/3FRt1SRqqt1FBt/event_purity‚0 @&Purity/3FRt2SRqqt*FBt/event_proportion‚0.0028351815417408943 particle/accuracy_3_of_3‚1 ("Purity/2FRt4SRqqt*FBt/event_purity‚0 & Purity/1FRt3SRqqt2FBt/FRt_purity‚1 8 Purity/2FRt3SRqqt*FBt/FRt_purity‚0.11561904847621918 ("Purity/3FRt*SRqqt0FBt/event_purity‚0 ,&Purity/4FRt3SRqqt1FBt/event_proportion‚0 jet/accuracy_5_of_6‚0 7 Purity/*FRt*SRqqt0FBt/FRt_purity‚0.1572863906621933 & Purity/1FRt0SRqqt1FBt/FBt_purity‚1 jet/accuracy_3_of_7‚0 =&Purity/*FRt2SRqqt*FBt/event_proportion‚0.1332535296678543 ?&Purity/2FRt0SRqqt*FBt/event_proportion‚0.037046369165182114 :"Purity/3FRt1SRqqt0FBt/SRqqt_purity‚0.43333330750465393 9"Purity/0FRt*SRqqt3FBt/SRqqt_purity‚0.5666666626930237 7 Purity/*FRt1SRqqt2FBt/FBt_purity‚0.9891833662986755 8"Purity/0FRt0SRqqt*FBt/event_purity‚0.808997392654419 ("Purity/2FRt1SRqqt1FBt/event_purity‚0 8 Purity/2FRt1SRqqt2FBt/FRt_purity‚0.46929821372032166 @&Purity/3FRt*SRqqt0FBt/event_proportion‚0.0071824598126113415 7 Purity/1FRt*SRqqt3FBt/FBt_purity‚0.9687500596046448 :"Purity/1FRt3SRqqt1FBt/SRqqt_purity‚0.49259263277053833 jet/accuracy_1_of_7‚1 9"Purity/*FRt2SRqqt*FBt/SRqqt_purity‚0.4067593514919281 =&Purity/0FRt1SRqqt0FBt/event_proportion‚0.1694808453321457 @&Purity/1FRt3SRqqt2FBt/event_proportion‚6.300403038039804e-05 9"Purity/2FRt0SRqqt2FBt/event_purity‚0.0833333358168602 8 Purity/2FRt2SRqqt1FBt/FRt_purity‚0.23125000298023224 *jet/accuracy_1_of_2‚0.6927793025970459 particle/accuracy_2_of_5‚1 + Purity/0FRt*SRqqt4FBt/FBt_purity‚0.8125 ?&Purity/*FRt*SRqqt3FBt/event_proportion‚0.004473286215215921 7 Purity/0FRt0SRqqt*FBt/FBt_purity‚0.9909959435462952 7 Purity/2FRt0SRqqt*FBt/FRt_purity‚0.2451522946357727 +"Purity/3FRt1SRqqt2FBt/SRqqt_purity‚0.75 7 Purity/0FRt1SRqqt3FBt/FBt_purity‚0.8999999761581421 & Purity/1FRt1SRqqt4FBt/FRt_purity‚0 ( Purity/*FRt1SRqqt3FBt/FRt_purity‚0.5 & Purity/4FRt1SRqqt0FBt/FRt_purity‚0 & Purity/*FRt3SRqqt*FBt/FBt_purity‚1 :"Purity/0FRt*SRqqt2FBt/event_purity‚0.24450920522212982 ("Purity/1FRt3SRqqt1FBt/event_purity‚0 8 Purity/3FRt*SRqqt1FBt/FRt_purity‚0.29671719670295715 /"Purity/*FRt4SRqqt*FBt/SRqqt_purity‚0.234375 7 Purity/*FRt2SRqqt2FBt/FBt_purity‚0.9736841917037964 & Purity/0FRt3SRqqt*FBt/FBt_purity‚1 9"Purity/*FRt*SRqqt2FBt/SRqqt_purity‚0.5442273020744324 7 Purity/*FRt*SRqqt3FBt/FBt_purity‚0.9296935796737671 ;"Purity/1FRt1SRqqt*FBt/event_purity‚0.016663115471601486 :"Purity/2FRt*SRqqt0FBt/SRqqt_purity‚0.37143418192863464 9"Purity/2FRt1SRqqt1FBt/SRqqt_purity‚0.5650538206100464 & Purity/0FRt2SRqqt3FBt/FBt_purity‚1 @&Purity/*FRt1SRqqt3FBt/event_proportion‚0.0006930443341843784 ?&Purity/0FRt1SRqqt2FBt/event_proportion‚0.008757560513913631 ,&Purity/4FRt4SRqqt3FBt/event_proportion‚0 ("Purity/2FRt2SRqqt2FBt/SRqqt_purity‚1 @&Purity/*FRt*SRqqt4FBt/event_proportion‚0.0003780241822823882 9"Purity/2FRt2SRqqt*FBt/SRqqt_purity‚0.4562370479106903 ,&Purity/4FRt2SRqqt4FBt/event_proportion‚0 >&Purity/*FRt0SRqqt1FBt/event_proportion‚0.14591734111309052 7 Purity/*FRt0SRqqt1FBt/FBt_purity‚0.9996017217636108 7 Purity/0FRt*SRqqt3FBt/FBt_purity‚0.9135802984237671 9"Purity/0FRt1SRqqt0FBt/event_purity‚0.3709058165550232 <"Purity/2FRt1SRqqt*FBt/event_purity‚0.0010405826615169644 +jet/accuracy_3_of_4‚0.06386362761259079 ("Purity/0FRt4SRqqt*FBt/event_purity‚0 9"Purity/*FRt0SRqqt1FBt/event_purity‚0.6803897023200989 @&Purity/*FRt2SRqqt3FBt/event_proportion‚6.300403038039804e-05 ,&Purity/1FRt3SRqqt4FBt/event_proportion‚0 ?&Purity/2FRt3SRqqt*FBt/event_proportion‚0.004032257944345474 7 Purity/3FRt2SRqqt1FBt/FRt_purity‚0.1666666716337204 *jet/accuracy_1_of_6‚0.9298246502876282 ("Purity/0FRt3SRqqt1FBt/event_purity‚0 ("Purity/*FRt2SRqqt3FBt/event_purity‚0 9"Purity/0FRt*SRqqt1FBt/event_purity‚0.6408859491348267 @&Purity/0FRt*SRqqt3FBt/event_proportion‚0.0030871976632624865 7 Purity/2FRt0SRqqt*FBt/FBt_purity‚0.9967741966247559 ?&Purity/3FRt1SRqqt*FBt/event_proportion‚0.003717237850651145 ,&Purity/4FRt0SRqqt1FBt/event_proportion‚0 jet/accuracy_6_of_6‚0 8 Purity/*FRt2SRqqt2FBt/FRt_purity‚0.29487180709838867 ("Purity/1FRt1SRqqt4FBt/event_purity‚0 ?&Purity/0FRt0SRqqt3FBt/event_proportion‚0.002583165420219302 ?&Purity/0FRt2SRqqt1FBt/event_proportion‚0.009261592291295528 ,&Purity/0FRt3SRqqt3FBt/event_proportion‚0 8 Purity/1FRt*SRqqt1FBt/FRt_purity‚0.39276212453842163 9"Purity/1FRt3SRqqt*FBt/SRqqt_purity‚0.2939537763595581 ("Purity/3FRt*SRqqt2FBt/event_purity‚0 & Purity/1FRt*SRqqt4FBt/FBt_purity‚1 ("Purity/1FRt1SRqqt3FBt/event_purity‚0 ,&Purity/1FRt2SRqqt4FBt/event_proportion‚0 ,&Purity/4FRt3SRqqt*FBt/event_proportion‚0 9"Purity/0FRt3SRqqt1FBt/SRqqt_purity‚0.2708333432674408 9"Purity/*FRt*SRqqt3FBt/event_purity‚0.1551724076271057 ?&Purity/*FRt0SRqqt3FBt/event_proportion‚0.003717237850651145 8 Purity/*FRt3SRqqt*FBt/FRt_purity‚0.15615354478359222 >&Purity/2FRt*SRqqt0FBt/event_proportion‚0.07579384744167328 ,&Purity/3FRt2SRqqt4FBt/event_proportion‚0 ,&Purity/3FRt3SRqqt1FBt/event_proportion‚0 @&Purity/4FRt1SRqqt0FBt/event_proportion‚6.300403038039804e-05 8 Purity/2FRt4SRqqt0FBt/FRt_purity‚0.20000000298023224 8 Purity/*FRt1SRqqt2FBt/FRt_purity‚0.44632840156555176 >&Purity/0FRt*SRqqt0FBt/event_proportion‚0.21931703388690948 A&Purity/0FRt0SRqqt4FBt/event_proportion‚0.00025201612152159214 @&Purity/1FRt0SRqqt3FBt/event_proportion‚0.0010710685746744275 ("Purity/1FRt2SRqqt*FBt/event_purity‚0 7 Purity/2FRt1SRqqt2FBt/FBt_purity‚0.9868420958518982 ,&Purity/4FRt1SRqqt3FBt/event_proportion‚0 & Purity/1FRt4SRqqt*FBt/FRt_purity‚0 @&Purity/*FRt4SRqqt0FBt/event_proportion‚0.0012600806076079607 5 Purity/*FRt*SRqqt1FBt/FRt_purity‚0.38417848944664 9"Purity/0FRt*SRqqt1FBt/SRqqt_purity‚0.5184654593467712 @&Purity/1FRt*SRqqt3FBt/event_proportion‚0.0013230846961960196 =&Purity/1FRt1SRqqt*FBt/event_proportion‚0.1723790317773819 ("Purity/1FRt1SRqqt2FBt/event_purity‚0 ,&Purity/2FRt4SRqqt4FBt/event_proportion‚0 +particle/specificity‚0.9449643492698669 9"Purity/*FRt0SRqqt0FBt/event_purity‚0.1316862255334854 =&Purity/0FRt1SRqqt*FBt/event_proportion‚0.2305317521095276 ?&Purity/0FRt3SRqqt*FBt/event_proportion‚0.006741431541740894 ,&Purity/0FRt3SRqqt2FBt/event_proportion‚0 ?&Purity/1FRt0SRqqt1FBt/event_proportion‚0.049773186445236206 particle/accuracy_1_of_3‚1 * Purity/*FRt*SRqqt4FBt/FBt_purity‚0.875 7 Purity/*FRt0SRqqt3FBt/FRt_purity‚0.6666666865348816GÚHÁù^ö‡¦ >&Purity/1FRt2SRqqt0FBt/event_proportion‚0.04731602966785431 ,&Purity/3FRt1SRqqt4FBt/event_proportion‚0 particle/accuracy_1_of_6‚1 ,"Purity/3FRt4SRqqt*FBt/SRqqt_purity‚0.375 A&Purity/1FRt*SRqqt4FBt/event_proportion‚0.00012600806076079607 ,&Purity/2FRt0SRqqt4FBt/event_proportion‚0 @&Purity/2FRt3SRqqt1FBt/event_proportion‚0.0003780241822823882 ,&Purity/4FRt3SRqqt0FBt/event_proportion‚0 & Purity/3FRt1SRqqt2FBt/FBt_purity‚1 ("Purity/4FRt1SRqqt0FBt/event_purity‚0 ("Purity/*FRt2SRqqt3FBt/SRqqt_purity‚0 ("Purity/4FRt0SRqqt0FBt/event_purity‚0 :"Purity/*FRt3SRqqt*FBt/SRqqt_purity‚0.26995566487312317 9"Purity/0FRt0SRqqt2FBt/event_purity‚0.3156907558441162 :"Purity/1FRt*SRqqt0FBt/SRqqt_purity‚0.37928831577301025 ("Purity/3FRt1SRqqt*FBt/event_purity‚0 8 Purity/3FRt2SRqqt*FBt/FRt_purity‚0.11921296268701553 ,&Purity/4FRt0SRqqt3FBt/event_proportion‚0 particle/accuracy_5_of_7‚1 & Purity/2FRt0SRqqt3FBt/FBt_purity‚1 ,&Purity/4FRt*SRqqt3FBt/event_proportion‚0 particle/accuracy_1_of_1‚1 + Purity/0FRt0SRqqt4FBt/FBt_purity‚0.8125 *"Purity/1FRt1SRqqt3FBt/SRqqt_purity‚0.5 :"Purity/*FRt1SRqqt0FBt/event_purity‚0.19486361742019653 >&Purity/*FRt3SRqqt0FBt/event_proportion‚0.01827116869390011 7 Purity/0FRt*SRqqt*FBt/FBt_purity‚0.9925971031188965 ?&Purity/0FRt3SRqqt0FBt/event_proportion‚0.006111391354352236 8 Purity/1FRt*SRqqt*FBt/FRt_purity‚0.22683686017990112 8 Purity/1FRt0SRqqt2FBt/FRt_purity‚0.49626123905181885 & Purity/2FRt2SRqqt*FBt/FBt_purity‚1 :"Purity/2FRt3SRqqt*FBt/SRqqt_purity‚0.22809524834156036 ?&Purity/2FRt2SRqqt1FBt/event_proportion‚0.003969253972172737 @&Purity/2FRt2SRqqt2FBt/event_proportion‚6.300403038039804e-05 8 Purity/2FRt3SRqqt0FBt/FRt_purity‚0.12867964804172516 ("Purity/3FRt*SRqqt*FBt/event_purity‚0 @&Purity/3FRt1SRqqt1FBt/event_proportion‚0.0009450604557059705 particle/accuracy_3_of_5‚1 & Purity/*FRt4SRqqt*FBt/FBt_purity‚1 & Purity/0FRt4SRqqt*FBt/FBt_purity‚1 8 Purity/*FRt3SRqqt1FBt/FRt_purity‚0.20000000298023224 & Purity/3FRt*SRqqt*FBt/FBt_purity‚1 7 Purity/1FRt2SRqqt2FBt/FRt_purity‚0.2777777910232544 7 Purity/3FRt0SRqqt2FBt/FRt_purity‚0.6666666865348816 9"Purity/*FRt*SRqqt3FBt/SRqqt_purity‚0.5952380895614624 >&Purity/*FRt*SRqqt2FBt/event_proportion‚0.06747731566429138 & Purity/1FRt3SRqqt1FBt/FBt_purity‚1 ,&Purity/3FRt1SRqqt3FBt/event_proportion‚0 ("Purity/0FRt3SRqqt0FBt/event_purity‚0 & Purity/3FRt4SRqqt*FBt/FRt_purity‚0 jet/accuracy_4_of_7‚0 _runtime‚4170.891647165 >&Purity/*FRt2SRqqt0FBt/event_proportion‚0.10855595022439957 @&Purity/*FRt3SRqqt2FBt/event_proportion‚6.300403038039804e-05 @&Purity/*FRt4SRqqt*FBt/event_proportion‚0.0013230846961960196 =&Purity/*FRt1SRqqt*FBt/event_proportion‚0.4482736885547638 8 Purity/*FRt*SRqqt*FBt/FRt_purity‚0.22412659227848053 9"Purity/*FRt*SRqqt*FBt/SRqqt_purity‚0.4101209044456482 A&Purity/0FRt4SRqqt0FBt/event_proportion‚0.00044102821266278625 ?&Purity/1FRt*SRqqt2FBt/event_proportion‚0.022303426638245583 >&Purity/2FRt1SRqqt*FBt/event_proportion‚0.04158266261219978 ("Purity/2FRt1SRqqt2FBt/event_purity‚0 @&Purity/3FRt1SRqqt0FBt/event_proportion‚0.0025201612152159214 @&Purity/3FRt2SRqqt1FBt/event_proportion‚0.0003780241822823882 +jet/accuracy_2_of_2‚0.09671532362699509 particle/accuracy_2_of_3‚1 :"Purity/2FRt4SRqqt*FBt/SRqqt_purity‚0.20000000298023224 ("Purity/1FRt3SRqqt2FBt/event_purity‚0 ?&Purity/*FRt1SRqqt2FBt/event_proportion‚0.017011089250445366 ,&Purity/*FRt2SRqqt4FBt/event_proportion‚0 ("Purity/3FRt0SRqqt*FBt/event_purity‚0 8 Purity/1FRt1SRqqt*FBt/FRt_purity‚0.21561168134212494 8 Purity/2FRt*SRqqt*FBt/FRt_purity‚0.22618263959884644 ,&Purity/4FRt1SRqqt4FBt/event_proportion‚0 ,&Purity/4FRt2SRqqt1FBt/event_proportion‚0 ("Purity/0FRt2SRqqt3FBt/event_purity‚0 ;"Purity/*FRt2SRqqt0FBt/event_purity‚0.021510345861315727 A&Purity/0FRt*SRqqt4FBt/event_proportion‚0.00025201612152159214 @&Purity/0FRt4SRqqt*FBt/event_proportion‚0.0005040322430431843 7 Purity/2FRt2SRqqt0FBt/FRt_purity‚0.1473909467458725 ,&Purity/2FRt2SRqqt4FBt/event_proportion‚0 ?&Purity/3FRt*SRqqt*FBt/event_proportion‚0.009954636916518211 & Purity/3FRt1SRqqt1FBt/FBt_purity‚1 7 Purity/*FRt4SRqqt0FBt/FRt_purity‚0.0833333358168602 9"Purity/0FRt1SRqqt*FBt/event_purity‚0.3110085129737854 =&Purity/1FRt*SRqqt*FBt/event_proportion‚0.4366179406642914 ("Purity/3FRt2SRqqt1FBt/event_purity‚0 :"Purity/0FRt2SRqqt2FBt/SRqqt_purity‚0.48076921701431274 & Purity/2FRt3SRqqt*FBt/FBt_purity‚1 ("Purity/3FRt4SRqqt*FBt/event_purity‚0 9"Purity/0FRt*SRqqt*FBt/event_purity‚0.4150574207305908 8 Purity/3FRt*SRqqt0FBt/FRt_purity‚0.11828043311834335 ("Purity/*FRt2SRqqt2FBt/event_purity‚0 & Purity/1FRt3SRqqt2FBt/FBt_purity‚1 & Purity/4FRt0SRqqt*FBt/FRt_purity‚0 ("Purity/2FRt2SRqqt0FBt/event_purity‚0 9"Purity/3FRt2SRqqt1FBt/SRqqt_purity‚0.8333333134651184 ,"Purity/3FRt4SRqqt0FBt/SRqqt_purity‚0.375 9"Purity/*FRt*SRqqt0FBt/SRqqt_purity‚0.3692529499530792 ,&Purity/4FRt3SRqqt3FBt/event_proportion‚0 ,&Purity/0FRt4SRqqt2FBt/event_proportion‚0 @&Purity/2FRt0SRqqt3FBt/event_proportion‚6.300403038039804e-05 ("Purity/2FRt3SRqqt*FBt/event_purity‚0 9"Purity/3FRt*SRqqt*FBt/SRqqt_purity‚0.4603174924850464 8 Purity/3FRt2SRqqt0FBt/FRt_purity‚0.09903382509946823 7 Purity/1FRt2SRqqt1FBt/FBt_purity‚0.9873272180557251 ?&Purity/2FRt*SRqqt2FBt/event_proportion‚0.005418346729129553 & Purity/3FRt0SRqqt1FBt/FBt_purity‚1 8"Purity/3FRt1SRqqt1FBt/SRqqt_purity‚0.807692289352417 9"Purity/3FRt1SRqqt*FBt/SRqqt_purity‚0.5617284178733826 ("Purity/*FRt3SRqqt*FBt/event_purity‚0 :"Purity/0FRt*SRqqt0FBt/SRqqt_purity‚0.36038872599601746 9"Purity/0FRt0SRqqt1FBt/event_purity‚0.9993913769721985 7 Purity/1FRt*SRqqt0FBt/FRt_purity‚0.1549118161201477 @&Purity/1FRt0SRqqt4FBt/event_proportion‚6.300403038039804e-05 ,&Purity/1FRt4SRqqt3FBt/event_proportion‚0 ;"Purity/2FRt1SRqqt0FBt/event_purity‚0.001240694778971374 7 Purity/1FRt1SRqqt*FBt/FBt_purity‚0.9968394637107849 7 Purity/2FRt*SRqqt*FBt/FBt_purity‚0.9961161017417908 ,&Purity/3FRt*SRqqt4FBt/event_proportion‚0 ("Purity/1FRt2SRqqt2FBt/event_purity‚0 & Purity/3FRt4SRqqt0FBt/FRt_purity‚0 particle/accuracy_2_of_7‚1 ("Purity/2FRt0SRqqt3FBt/event_purity‚0 ("Purity/0FRt1SRqqt3FBt/event_purity‚0 :"Purity/*FRt*SRqqt*FBt/event_purity‚0.21816574037075043 ("Purity/3FRt1SRqqt1FBt/event_purity‚0 @&Purity/3FRt2SRqqt*FBt/event_proportion‚0.0028351815417408943 particle/accuracy_3_of_3‚1 ("Purity/2FRt4SRqqt*FBt/event_purity‚0 & Purity/1FRt3SRqqt2FBt/FRt_purity‚1 8 Purity/2FRt3SRqqt*FBt/FRt_purity‚0.11561904847621918 ("Purity/3FRt*SRqqt0FBt/event_purity‚0 ,&Purity/4FRt3SRqqt1FBt/event_proportion‚0 jet/accuracy_5_of_6‚0 7 Purity/*FRt*SRqqt0FBt/FRt_purity‚0.1572863906621933 & Purity/1FRt0SRqqt1FBt/FBt_purity‚1 jet/accuracy_3_of_7‚0 =&Purity/*FRt2SRqqt*FBt/event_proportion‚0.1332535296678543 ?&Purity/2FRt0SRqqt*FBt/event_proportion‚0.037046369165182114 :"Purity/3FRt1SRqqt0FBt/SRqqt_purity‚0.43333330750465393 9"Purity/0FRt*SRqqt3FBt/SRqqt_purity‚0.5666666626930237 7 Purity/*FRt1SRqqt2FBt/FBt_purity‚0.9891833662986755 8"Purity/0FRt0SRqqt*FBt/event_purity‚0.808997392654419 ("Purity/2FRt1SRqqt1FBt/event_purity‚0 8 Purity/2FRt1SRqqt2FBt/FRt_purity‚0.46929821372032166 @&Purity/3FRt*SRqqt0FBt/event_proportion‚0.0071824598126113415 7 Purity/1FRt*SRqqt3FBt/FBt_purity‚0.9687500596046448 :"Purity/1FRt3SRqqt1FBt/SRqqt_purity‚0.49259263277053833 jet/accuracy_1_of_7‚1 9"Purity/*FRt2SRqqt*FBt/SRqqt_purity‚0.4067593514919281 =&Purity/0FRt1SRqqt0FBt/event_proportion‚0.1694808453321457 @&Purity/1FRt3SRqqt2FBt/event_proportion‚6.300403038039804e-05 9"Purity/2FRt0SRqqt2FBt/event_purity‚0.0833333358168602 8 Purity/2FRt2SRqqt1FBt/FRt_purity‚0.23125000298023224 *jet/accuracy_1_of_2‚0.6927793025970459 particle/accuracy_2_of_5‚1 + Purity/0FRt*SRqqt4FBt/FBt_purity‚0.8125 ?&Purity/*FRt*SRqqt3FBt/event_proportion‚0.004473286215215921 7 Purity/0FRt0SRqqt*FBt/FBt_purity‚0.9909959435462952 7 Purity/2FRt0SRqqt*FBt/FRt_purity‚0.2451522946357727 +"Purity/3FRt1SRqqt2FBt/SRqqt_purity‚0.75 7 Purity/0FRt1SRqqt3FBt/FBt_purity‚0.8999999761581421 & Purity/1FRt1SRqqt4FBt/FRt_purity‚0 ( Purity/*FRt1SRqqt3FBt/FRt_purity‚0.5 & Purity/4FRt1SRqqt0FBt/FRt_purity‚0 & Purity/*FRt3SRqqt*FBt/FBt_purity‚1 :"Purity/0FRt*SRqqt2FBt/event_purity‚0.24450920522212982 ("Purity/1FRt3SRqqt1FBt/event_purity‚0 8 Purity/3FRt*SRqqt1FBt/FRt_purity‚0.29671719670295715 /"Purity/*FRt4SRqqt*FBt/SRqqt_purity‚0.234375 7 Purity/*FRt2SRqqt2FBt/FBt_purity‚0.9736841917037964 & Purity/0FRt3SRqqt*FBt/FBt_purity‚1 9"Purity/*FRt*SRqqt2FBt/SRqqt_purity‚0.5442273020744324 7 Purity/*FRt*SRqqt3FBt/FBt_purity‚0.9296935796737671 ;"Purity/1FRt1SRqqt*FBt/event_purity‚0.016663115471601486 :"Purity/2FRt*SRqqt0FBt/SRqqt_purity‚0.37143418192863464 9"Purity/2FRt1SRqqt1FBt/SRqqt_purity‚0.5650538206100464 & Purity/0FRt2SRqqt3FBt/FBt_purity‚1 @&Purity/*FRt1SRqqt3FBt/event_proportion‚0.0006930443341843784 ?&Purity/0FRt1SRqqt2FBt/event_proportion‚0.008757560513913631 ,&Purity/4FRt4SRqqt3FBt/event_proportion‚0 ("Purity/2FRt2SRqqt2FBt/SRqqt_purity‚1 @&Purity/*FRt*SRqqt4FBt/event_proportion‚0.0003780241822823882 9"Purity/2FRt2SRqqt*FBt/SRqqt_purity‚0.4562370479106903 ,&Purity/4FRt2SRqqt4FBt/event_proportion‚0 >&Purity/*FRt0SRqqt1FBt/event_proportion‚0.14591734111309052 7 Purity/*FRt0SRqqt1FBt/FBt_purity‚0.9996017217636108 7 Purity/0FRt*SRqqt3FBt/FBt_purity‚0.9135802984237671 9"Purity/0FRt1SRqqt0FBt/event_purity‚0.3709058165550232 <"Purity/2FRt1SRqqt*FBt/event_purity‚0.0010405826615169644 +jet/accuracy_3_of_4‚0.06386362761259079 ("Purity/0FRt4SRqqt*FBt/event_purity‚0 9"Purity/*FRt0SRqqt1FBt/event_purity‚0.6803897023200989 @&Purity/*FRt2SRqqt3FBt/event_proportion‚6.300403038039804e-05 ,&Purity/1FRt3SRqqt4FBt/event_proportion‚0 ?&Purity/2FRt3SRqqt*FBt/event_proportion‚0.004032257944345474 7 Purity/3FRt2SRqqt1FBt/FRt_purity‚0.1666666716337204 *jet/accuracy_1_of_6‚0.9298246502876282 ("Purity/0FRt3SRqqt1FBt/event_purity‚0 ("Purity/*FRt2SRqqt3FBt/event_purity‚0 9"Purity/0FRt*SRqqt1FBt/event_purity‚0.6408859491348267 @&Purity/0FRt*SRqqt3FBt/event_proportion‚0.0030871976632624865 7 Purity/2FRt0SRqqt*FBt/FBt_purity‚0.9967741966247559 ?&Purity/3FRt1SRqqt*FBt/event_proportion‚0.003717237850651145 ,&Purity/4FRt0SRqqt1FBt/event_proportion‚0 jet/accuracy_6_of_6‚0 8 Purity/*FRt2SRqqt2FBt/FRt_purity‚0.29487180709838867 ("Purity/1FRt1SRqqt4FBt/event_purity‚0 ?&Purity/0FRt0SRqqt3FBt/event_proportion‚0.002583165420219302 ?&Purity/0FRt2SRqqt1FBt/event_proportion‚0.009261592291295528 ,&Purity/0FRt3SRqqt3FBt/event_proportion‚0 8 Purity/1FRt*SRqqt1FBt/FRt_purity‚0.39276212453842163 9"Purity/1FRt3SRqqt*FBt/SRqqt_purity‚0.2939537763595581 ("Purity/3FRt*SRqqt2FBt/event_purity‚0 & Purity/1FRt*SRqqt4FBt/FBt_purity‚1 ("Purity/1FRt1SRqqt3FBt/event_purity‚0 ,&Purity/1FRt2SRqqt4FBt/event_proportion‚0 ,&Purity/4FRt3SRqqt*FBt/event_proportion‚0 9"Purity/0FRt3SRqqt1FBt/SRqqt_purity‚0.2708333432674408 9"Purity/*FRt*SRqqt3FBt/event_purity‚0.1551724076271057 ?&Purity/*FRt0SRqqt3FBt/event_proportion‚0.003717237850651145 8 Purity/*FRt3SRqqt*FBt/FRt_purity‚0.15615354478359222 >&Purity/2FRt*SRqqt0FBt/event_proportion‚0.07579384744167328 ,&Purity/3FRt2SRqqt4FBt/event_proportion‚0 ,&Purity/3FRt3SRqqt1FBt/event_proportion‚0 @&Purity/4FRt1SRqqt0FBt/event_proportion‚6.300403038039804e-05 8 Purity/2FRt4SRqqt0FBt/FRt_purity‚0.20000000298023224 8 Purity/*FRt1SRqqt2FBt/FRt_purity‚0.44632840156555176 >&Purity/0FRt*SRqqt0FBt/event_proportion‚0.21931703388690948 A&Purity/0FRt0SRqqt4FBt/event_proportion‚0.00025201612152159214 @&Purity/1FRt0SRqqt3FBt/event_proportion‚0.0010710685746744275 ("Purity/1FRt2SRqqt*FBt/event_purity‚0 7 Purity/2FRt1SRqqt2FBt/FBt_purity‚0.9868420958518982 ,&Purity/4FRt1SRqqt3FBt/event_proportion‚0 & Purity/1FRt4SRqqt*FBt/FRt_purity‚0 @&Purity/*FRt4SRqqt0FBt/event_proportion‚0.0012600806076079607 5 Purity/*FRt*SRqqt1FBt/FRt_purity‚0.38417848944664 9"Purity/0FRt*SRqqt1FBt/SRqqt_purity‚0.5184654593467712 @&Purity/1FRt*SRqqt3FBt/event_proportion‚0.0013230846961960196 =&Purity/1FRt1SRqqt*FBt/event_proportion‚0.1723790317773819 ("Purity/1FRt1SRqqt2FBt/event_purity‚0 ,&Purity/2FRt4SRqqt4FBt/event_proportion‚0 +particle/specificity‚0.9449643492698669 9"Purity/*FRt0SRqqt0FBt/event_purity‚0.1316862255334854 =&Purity/0FRt1SRqqt*FBt/event_proportion‚0.2305317521095276 ?&Purity/0FRt3SRqqt*FBt/event_proportion‚0.006741431541740894 ,&Purity/0FRt3SRqqt2FBt/event_proportion‚0 ?&Purity/1FRt0SRqqt1FBt/event_proportion‚0.049773186445236206 particle/accuracy_1_of_3‚1 * Purity/*FRt*SRqqt4FBt/FBt_purity‚0.875 7 Purity/*FRt0SRqqt3FBt/FRt_purity‚0.6666666865348816 ;"Purity/*FRt2SRqqt*FBt/event_purity‚0.019272908568382263 9"Purity/1FRt0SRqqt1FBt/event_purity‚0.2547757923603058 ,&Purity/1FRt2SRqqt3FBt/event_proportion‚0 ,&Purity/4FRt0SRqqt2FBt/event_proportion‚0 *jet/accuracy_1_of_1‚0.4251188635826111 jet/accuracy_5_of_5‚0 9"Purity/1FRt*SRqqt2FBt/SRqqt_purity‚0.5328981280326843 7 Purity/1FRt1SRqqt1FBt/FRt_purity‚0.3501157760620117 ?&Purity/2FRt*SRqqt1FBt/event_proportion‚0.021169355139136314 8 Purity/2FRt*SRqqt1FBt/FRt_purity‚0.38005712628364563 @&Purity/2FRt*SRqqt3FBt/event_proportion‚6.300403038039804e-05 7 Purity/2FRt0SRqqt0FBt/FRt_purity‚0.1580076366662979 ("Purity/2FRt3SRqqt0FBt/event_purity‚0 & Purity/1FRt1SRqqt4FBt/FBt_purity‚1 7 Purity/*FRt0SRqqt2FBt/FBt_purity‚0.9910476207733154 & Purity/4FRt1SRqqt*FBt/FRt_purity‚0 ("Purity/0FRt2SRqqt3FBt/SRqqt_purity‚0 *"Purity/4FRt2SRqqt0FBt/SRqqt_purity‚0.5 >&Purity/0FRt0SRqqt1FBt/event_proportion‚0.08606351166963577 ,&Purity/1FRt3SRqqt3FBt/event_proportion‚0 ("Purity/2FRt2SRqqt*FBt/event_purity‚0 ,&Purity/4FRt*SRqqt4FBt/event_proportion‚0 particle/accuracy_3_of_4‚1 particle/accuracy_6_of_7‚1 :"Purity/2FRt*SRqqt2FBt/event_purity‚0.05833333358168602 >&Purity/2FRt1SRqqt0FBt/event_proportion‚0.03181703761219978 & Purity/2FRt1SRqqt1FBt/FBt_purity‚1 ,&Purity/2FRt3SRqqt3FBt/event_proportion‚0 ,&Purity/4FRt0SRqqt4FBt/event_proportion‚0 9"Purity/1FRt1SRqqt*FBt/SRqqt_purity‚0.4543781876564026 @&Purity/3FRt0SRqqt1FBt/event_proportion‚0.0011340725468471646 @&Purity/3FRt0SRqqt2FBt/event_proportion‚6.300403038039804e-05 ;"Purity/1FRt1SRqqt0FBt/event_purity‚0.010184351354837418 particle/accuracy_2_of_2‚1 ("Purity/*FRt0SRqqt4FBt/event_purity‚0 ("Purity/*FRt4SRqqt*FBt/event_purity‚0 ("Purity/3FRt0SRqqt2FBt/event_purity‚0 +"Purity/4FRt*SRqqt0FBt/SRqqt_purity‚0.75 9"Purity/*FRt1SRqqt2FBt/SRqqt_purity‚0.5415419936180115 ("Purity/1FRt3SRqqt*FBt/event_purity‚0 ("Purity/3FRt2SRqqt0FBt/event_purity‚0 (particle/f_score‚0.34177660942077637 ("Purity/0FRt4SRqqt1FBt/event_purity‚0 & Purity/2FRt*SRqqt3FBt/FBt_purity‚1 ("Purity/3FRt3SRqqt0FBt/event_purity‚0 & Purity/1FRt1SRqqt3FBt/FBt_purity‚1 ,&Purity/*FRt3SRqqt4FBt/event_proportion‚0 9"Purity/0FRt0SRqqt3FBt/event_purity‚0.1899999976158142 @&Purity/0FRt2SRqqt2FBt/event_proportion‚0.0010080644860863686 )loss/val_total_loss‚42.25387696344677 =&Purity/*FRt*SRqqt*FBt/event_proportion‚0.9591733813285828 ?&Purity/1FRt1SRqqt2FBt/event_proportion‚0.006111391354352236 ?&Purity/2FRt0SRqqt0FBt/event_proportion‚0.024571573361754417 @&Purity/3FRt*SRqqt2FBt/event_proportion‚0.0003150201519019902 @&Purity/4FRt1SRqqt*FBt/event_proportion‚6.300403038039804e-05 8"Purity/0FRt1SRqqt3FBt/SRqqt_purity‚0.699999988079071 ("Purity/1FRt1SRqqt4FBt/SRqqt_purity‚0 7 Purity/*FRt1SRqqt1FBt/FBt_purity‚0.9988254308700562 >&Purity/*FRt2SRqqt1FBt/event_proportion‚0.02255544438958168 ("Purity/0FRt0SRqqt0FBt/event_purity‚1 9"Purity/2FRt1SRqqt*FBt/SRqqt_purity‚0.4450591504573822 ."Purity/0FRt4SRqqt*FBt/SRqqt_purity‚0.21875 ("Purity/2FRt4SRqqt0FBt/event_purity‚0 jet/accuracy_2_of_7‚0 ( Purity/1FRt*SRqqt4FBt/FRt_purity‚0.5 8 Purity/*FRt3SRqqt0FBt/FRt_purity‚0.15494666993618011 9"Purity/1FRt2SRqqt0FBt/SRqqt_purity‚0.3806386888027191 *jet/accuracy_1_of_3‚0.8155019283294678 (particle/accuracy‚0.8391453623771667 8 Purity/2FRt4SRqqt*FBt/FRt_purity‚0.20000000298023224 8 Purity/*FRt0SRqqt2FBt/FRt_purity‚0.48945343494415283 9"Purity/1FRt2SRqqt*FBt/SRqqt_purity‚0.4069845676422119 ,&Purity/2FRt*SRqqt4FBt/event_proportion‚0 ,&Purity/3FRt3SRqqt4FBt/event_proportion‚0 *"Purity/0FRt4SRqqt1FBt/SRqqt_purity‚0.5 8 Purity/2FRt3SRqqt1FBt/FRt_purity‚0.10000000149011612 particle/accuracy_1_of_7‚1 ("Purity/*FRt3SRqqt2FBt/event_purity‚0 :"Purity/1FRt0SRqqt2FBt/event_purity‚0.12220816314220428 ?&Purity/1FRt0SRqqt2FBt/event_proportion‚0.015120967291295528 ,&Purity/*FRt3SRqqt3FBt/event_proportion‚0 :"Purity/0FRt*SRqqt3FBt/event_purity‚0.17592592537403107 ,&Purity/4FRt*SRqqt1FBt/event_proportion‚0 ,&Purity/4FRt1SRqqt2FBt/event_proportion‚0 7 Purity/*FRt2SRqqt1FBt/FBt_purity‚0.9940702319145203 ,&Purity/*FRt4SRqqt2FBt/event_proportion‚0 9"Purity/0FRt2SRqqt0FBt/SRqqt_purity‚0.3624584972858429 A&Purity/1FRt1SRqqt3FBt/event_proportion‚0.00025201612152159214 @&Purity/3FRt3SRqqt*FBt/event_proportion‚0.0005670362734235823 @&Purity/3FRt3SRqqt0FBt/event_proportion‚0.0005670362734235823 ("Purity/*FRt4SRqqt0FBt/event_purity‚0 ( Purity/*FRt*SRqqt4FBt/FRt_purity‚0.5 A&Purity/0FRt1SRqqt3FBt/event_proportion‚0.00044102821266278625 ("Purity/3FRt0SRqqt1FBt/event_purity‚0 & Purity/1FRt4SRqqt0FBt/FRt_purity‚0 & Purity/3FRt*SRqqt1FBt/FBt_purity‚1 ,&Purity/4FRt4SRqqt4FBt/event_proportion‚0 ("Purity/2FRt*SRqqt3FBt/event_purity‚0 7 Purity/1FRt1SRqqt2FBt/FRt_purity‚0.4656462073326111 & Purity/1FRt3SRqqt*FBt/FBt_purity‚1 ,&Purity/1FRt4SRqqt4FBt/event_proportion‚0 8 Purity/2FRt*SRqqt2FBt/FRt_purity‚0.49972227215766907 ,&Purity/3FRt4SRqqt3FBt/event_proportion‚0 9"Purity/*FRt3SRqqt2FBt/SRqqt_purity‚0.3333333432674408 & Purity/4FRt*SRqqt*FBt/FRt_purity‚0 @&Purity/1FRt1SRqqt4FBt/event_proportion‚6.300403038039804e-05 @&Purity/3FRt*SRqqt1FBt/event_proportion‚0.0024571572430431843 ,&Purity/4FRt2SRqqt2FBt/event_proportion‚0 ("Purity/*FRt1SRqqt4FBt/SRqqt_purity‚0 @&Purity/*FRt1SRqqt4FBt/event_proportion‚6.300403038039804e-05 ,&Purity/2FRt4SRqqt3FBt/event_proportion‚0 ;"Purity/*FRt1SRqqt2FBt/event_purity‚0.017473118379712105 >&Purity/0FRt*SRqqt1FBt/event_proportion‚0.14787046611309052 @&Purity/0FRt2SRqqt3FBt/event_proportion‚6.300403038039804e-05 7 Purity/*FRt0SRqqt3FBt/FBt_purity‚0.9271826148033142 @&Purity/*FRt4SRqqt1FBt/event_proportion‚6.300403038039804e-05 7 Purity/0FRt*SRqqt1FBt/FBt_purity‚0.9992362856864929 9"Purity/0FRt1SRqqt0FBt/SRqqt_purity‚0.3709058165550232 & Purity/2FRt2SRqqt1FBt/FBt_purity‚1 8 Purity/3FRt1SRqqt0FBt/FRt_purity‚0.10000000149011612 & Purity/*FRt4SRqqt1FBt/FBt_purity‚1 & Purity/0FRt4SRqqt1FBt/FBt_purity‚1 9"Purity/*FRt1SRqqt*FBt/event_purity‚0.1667831987142563 ,&Purity/0FRt4SRqqt4FBt/event_proportion‚0 ,jet/accuracy_3_of_3‚0.024457141757011414 particle/accuracy_4_of_4‚1 jet/accuracy_5_of_7‚0 *"Purity/4FRt2SRqqt*FBt/SRqqt_purity‚0.5 >&Purity/2FRt*SRqqt*FBt/event_proportion‚0.10244455933570862 @&Purity/4FRt2SRqqt0FBt/event_proportion‚6.300403038039804e-05 7 Purity/*FRt*SRqqt1FBt/FBt_purity‚0.9988430738449097 & Purity/0FRt2SRqqt1FBt/FBt_purity‚1 8 Purity/1FRt0SRqqt0FBt/FRt_purity‚0.15637008845806122 <"Purity/2FRt*SRqqt0FBt/event_purity‚0.0031388800125569105 ("Purity/3FRt*SRqqt1FBt/event_purity‚0 9"Purity/*FRt2SRqqt1FBt/SRqqt_purity‚0.5316393971443176 7 Purity/1FRt3SRqqt*FBt/FRt_purity‚0.1742575466632843 9"Purity/2FRt*SRqqt2FBt/SRqqt_purity‚0.7149122357368469 7"Purity/*FRt1SRqqt3FBt/SRqqt_purity‚0.69047611951828 *"Purity/1FRt*SRqqt3FBt/SRqqt_purity‚0.5 7 Purity/0FRt1SRqqt1FBt/FBt_purity‚0.9990783333778381 :"Purity/1FRt*SRqqt2FBt/event_purity‚0.08398813009262085 6 Purity/1FRt1SRqqt2FBt/FBt_purity‚0.988095223903656 7 Purity/2FRt*SRqqt2FBt/FBt_purity‚0.9911110997200012 @&Purity/2FRt4SRqqt*FBt/event_proportion‚0.0003150201519019902 & Purity/3FRt2SRqqt1FBt/FBt_purity‚1 ("Purity/1FRt0SRqqt4FBt/event_purity‚0 8 Purity/*FRt0SRqqt1FBt/FRt_purity‚0.43833544850349426 ,&Purity/*FRt4SRqqt4FBt/event_proportion‚0 8 Purity/1FRt2SRqqt0FBt/FRt_purity‚0.13502077758312225 8 Purity/3FRt0SRqqt0FBt/FRt_purity‚0.16993464529514313 7 Purity/*FRt0SRqqt4FBt/FBt_purity‚0.8500000238418579 9"Purity/3FRt3SRqqt*FBt/SRqqt_purity‚0.2380952388048172 ;"Purity/*FRt2SRqqt1FBt/event_purity‚0.010651323944330215 7 Purity/1FRt*SRqqt1FBt/FBt_purity‚0.9982349276542664 ?&Purity/1FRt3SRqqt0FBt/event_proportion‚0.007938507944345474 ,&Purity/3FRt2SRqqt3FBt/event_proportion‚0 ,&Purity/4FRt3SRqqt2FBt/event_proportion‚0 ( Purity/2FRt*SRqqt3FBt/FRt_purity‚0.5 & Purity/*FRt1SRqqt4FBt/FRt_purity‚0 ( Purity/1FRt1SRqqt3FBt/FRt_purity‚0.5 >&Purity/0FRt0SRqqt0FBt/event_proportion‚0.04082661122083664 ?&Purity/0FRt2SRqqt0FBt/event_proportion‚0.043283771723508835 9"Purity/3FRt*SRqqt1FBt/SRqqt_purity‚0.8666666746139526 @&Purity/4FRt0SRqqt*FBt/event_proportion‚6.300403038039804e-05 epoch‚3 ("Purity/*FRt3SRqqt1FBt/event_purity‚0 :"Purity/0FRt2SRqqt0FBt/event_purity‚0.05581668019294739 8 Purity/2FRt1SRqqt1FBt/FRt_purity‚0.42148616909980774 ?&Purity/2FRt1SRqqt2FBt/event_proportion‚0.001890120911411941 @&Purity/3FRt0SRqqt0FBt/event_proportion‚0.0015120967291295528 ."Purity/*FRt4SRqqt0FBt/SRqqt_purity‚0.21875 & Purity/*FRt2SRqqt3FBt/FBt_purity‚1 7 Purity/*FRt*SRqqt2FBt/FBt_purity‚0.9902760982513428 9"Purity/1FRt1SRqqt2FBt/SRqqt_purity‚0.5318027138710022 ,&Purity/2FRt1SRqqt3FBt/event_proportion‚0 & Purity/0FRt2SRqqt2FBt/FBt_purity‚1 7 Purity/1FRt0SRqqt*FBt/FBt_purity‚0.9950029253959656 @&Purity/1FRt3SRqqt1FBt/event_proportion‚0.0011340725468471646 ("Purity/2FRt3SRqqt1FBt/event_purity‚0 & Purity/4FRt*SRqqt0FBt/FRt_purity‚0 ( Purity/2FRt2SRqqt2FBt/FRt_purity‚0.5 7 Purity/*FRt*SRqqt3FBt/FRt_purity‚0.6764705777168274 ("Purity/1FRt2SRqqt0FBt/event_purity‚0 @&Purity/0FRt4SRqqt1FBt/event_proportion‚6.300403038039804e-05 @&Purity/3FRt2SRqqt0FBt/event_proportion‚0.0024571572430431843 @&Purity/4FRt2SRqqt*FBt/event_proportion‚6.300403038039804e-05 :"Purity/2FRt4SRqqt0FBt/SRqqt_purity‚0.20000000298023224 7 Purity/3FRt*SRqqt2FBt/FRt_purity‚0.3333333432674408 7 Purity/*FRt1SRqqt3FBt/FBt_purity‚0.9603174924850464 ("Purity/*FRt1SRqqt4FBt/event_purity‚0 9"Purity/*FRt3SRqqt1FBt/SRqqt_purity‚0.4193122088909149 7 Purity/1FRt*SRqqt2FBt/FBt_purity‚0.9745225310325623 7 Purity/1FRt2SRqqt*FBt/FBt_purity‚0.9832949638366699 & Purity/3FRt0SRqqt*FBt/FBt_purity‚1 +"Purity/3FRt*SRqqt2FBt/SRqqt_purity‚0.75 @&Purity/3FRt0SRqqt*FBt/event_proportion‚0.0027091733645647764 7 Purity/3FRt1SRqqt*FBt/FRt_purity‚0.1666666716337204 ,&Purity/4FRt*SRqqt2FBt/event_proportion‚0 :"Purity/3FRt2SRqqt0FBt/SRqqt_purity‚0.40942031145095825 =&Purity/*FRt0SRqqt*FBt/event_proportion‚0.3558467626571655 ?&Purity/0FRt0SRqqt2FBt/event_proportion‚0.029674898833036423 7 Purity/0FRt1SRqqt2FBt/FBt_purity‚0.9906481504440308 + Purity/1FRt*SRqqt3FBt/FRt_purity‚0.6875 :"Purity/1FRt0SRqqt0FBt/event_purity‚0.15637008845806122 7 Purity/1FRt1SRqqt1FBt/FBt_purity‚0.9987593293190002 & Purity/2FRt*SRqqt1FBt/FBt_purity‚1 >&Purity/0FRt*SRqqt2FBt/event_proportion‚0.03944052383303642 9"Purity/1FRt1SRqqt0FBt/SRqqt_purity‚0.3994210660457611 ,&Purity/2FRt1SRqqt4FBt/event_proportion‚0 *jet/accuracy_1_of_4‚0.8554442524909973 +jet/accuracy_2_of_5‚0.41363969445228577 :"Purity/2FRt3SRqqt1FBt/SRqqt_purity‚0.36666667461395264 & Purity/2FRt3SRqqt1FBt/FBt_purity‚1 9"Purity/3FRt3SRqqt0FBt/S