:W&Bá¾´ÅüM r7or6c6nª; 9 /0.29.0+7872fd9d9f86ca5c50617e19cb168122b7f959480.70.0<†d—ø‚" s0xecs192715 r7or6c6nŠÔ r7or6c6njuju11-uc-san-diegoSPAtop"— _wandb‚{}  batch_size‚512  dataset_limit‚0.2  dropout‚0.02189627882975461 epochs‚10  hidden_dim‚64 & learning_rate‚0.001874591612364422  num_attention_heads‚8  num_branch_encoder_layers‚3  num_embedding_layers‚6  num_encoder_layers‚6Bjjtttt_vanilla_r7or6c6nb88oquqqwj)jj-spatop-sweepagent-tttt-vanilla-1-k2mn8Š Ñ“äÔ ï«Äš3 (H¨¸hp"3.12.10*0.29.0j linux-x86_64øÛ‘k7 r7or6c6n&  _wandb.runtime‚0  _runtime‚0ƒœ9jÒä -Linux-6.8.0-136-generic-x86_64-with-glibc2.36CPython 3.12.10 Ñ“äÔàüѤ*-of*=/data/spatop/options_files/sweep_tttt_jj/config_r7or6c6n.json*-l* /scratch/logs*-n*jjtttt_vanilla_r7or6c6n2-m spanet.trainRjacksonjulian2014@gmail.comZ /scratch/logsb)jj-spatop-sweepagent-tttt-vanilla-1-k2mn8r/opt/conda/bin/pythonº 0zbgtzhnn2hd66hxvsfccw7rg5ylpuumïÂ#ÉÒÀ€ˆ€’NVIDIA A100-SXM4-80GB˜¢ /€€€¥È.€ ˜¥äª€àãÓ½ÂS NVIDIA A100-SXM4-80GB€€€€À€6"Ampere*(GPU-78f24734-e3a1-59a7-ce1b-bb4283f77c97Ê13.2º 0zbgtzhnn2hd66hxvsfccw7rg5ylpuumµª½Ç2  requirements.txtÖ“JW r7or6c6nZF (H¨¸ (H¨¸ (hpð"3.12.10*0.29.0j linux-x86_64ÎGó”Y r7or6c6nZH (H¨¸ (H¨¸ (8hpð"3.12.10*0.29.0j linux-x86_64¦,ë±*  r7or6c6nb trainer/global_step2@,‰ƒ\  r7or6c6nZK (H¨¸ (H¨¸(8hpð"3.12.10*0.29.0j linux-x86_64ö$,ÿ/  r7or6c6nb*"trainer/global_step2Ôá×K  r7or6c6nj: Ò“äÔ€ÝåÎ*LOCAL_RANK: 0 - CUDA_VISIBLE_DEVICES: [0] ´yÞ“  r7or6c6nj  Ò“äÔ𧊯î â”â”â”â”┳â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”┳â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”┳â”â”â”â”â”â”â”â”┳â”â”â”â”â”â”â”┳â”â”â”â”â”â”â”┓ ┃ ┃ Name ┃ Type ┃ Params ┃ Mode ┃ FLOPs ┃ ┡â”â”â”╇â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”╇â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”╇â”â”â”â”â”â”â”â”╇â”â”â”â”â”â”â”╇â”â”â”â”â”â”â”┩ │ 0 │ embedding │ MultiInputVectorEmbed… │ 220 K │ train │ 0 │ │ 1 │ encoder │ JetEncoder │ 498 K │ train │ 0 │ │ 2 │ branch_decoders │ ModuleList │ 9.6 M │ train │ 0 │ │ 3 │ regression_decoder │ RegressionDecoder │ 0 │ train │ 0 │ │ 4 │ classification_decoder │ ClassificationDecoder │ 0 │ train │ 0 │ │ │ other params │ n/a │ 48 │ n/a │ n/a │ └───┴────────────────────────┴────────────────────────┴────────┴───────┴───────┘ Ùú£æ\ r7or6c6njÊ Ò“äÔ ‰Œ°·Trainable params: 10.3 M Non-trainable params: 86 Total params: 10.3 M Total estimated model params size (MB): 41.196 Modules in train mode: 4413 Modules in eval mode: 0 Total FLOPs: 0 s[ëq r7or6c6nj` Ò“äԨ⥰Pwandb: WARNING Config item 'hidden_dim' was locked by 'sweep' (ignored update). õer}{ r7or6c6njj Ò“äÔøÁµ°Zwandb: WARNING Config item 'num_embedding_layers' was locked by 'sweep' (ignored update). ‚êàCy r7or6c6njh Ò“äÔÐ×À°Xwandb: WARNING Config item 'num_encoder_layers' was locked by 'sweep' (ignored update). f¡È€ r7or6c6njo Ò“äÔø…É°_wandb: WARNING Config item 'num_branch_encoder_layers' was locked by 'sweep' (ignored update). šðŸz r7or6c6nji Ò“äÔȀаYwandb: WARNING Config item 'num_attention_heads' was locked by 'sweep' (ignored update). QàÙt r7or6c6njc Ò“äÔè„Ù°Swandb: WARNING Config item 'dataset_limit' was locked by 'sweep' (ignored update). êeÓCq r7or6c6nj` Ò“äÔ ‡à°Pwandb: WARNING Config item 'batch_size' was locked by 'sweep' (ignored update). ¸` t r7or6c6njc Ò“äÔ˜ê°Swandb: WARNING Config item 'learning_rate' was locked by 'sweep' (ignored update). PïDeviceStatsMonitor.on_train_batch_start/oversize_segments.peak"trainer/global_step2:JP Á6¿k:bg DDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak"trainer/global_step2:JPÉ«"9p;bl IDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current"trainer/global_step2:JP6Eœùtbm JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated"trainer/global_step2:JPÂÞ®Di?be BDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current"trainer/global_step2:JPuÁ€Âh@bd ADeviceStatsMonitor.on_train_batch_start/active.small_pool.current"trainer/global_step2:JPtCSmAbi FDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed"trainer/global_step2:JPÐÄ$.cBb_ Š,gFbc @DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed"trainer/global_step2:JPUbfrGbn KDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated"trainer/global_step2:JPìùFfHbb ?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed"trainer/global_step2:JPÄ¢¼sIbo LDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak"trainer/global_step2:JPìÿCknJbj GDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed"trainer/global_step2:JP2¯SgKbc @DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed"trainer/global_step2:JP ÿhLbd ADeviceStatsMonitor.on_train_batch_start/active.large_pool.current"trainer/global_step2:JP½Ã.ÏdMb` =DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak"trainer/global_step2:JP™·µëlNbh EDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current"trainer/global_step2:JPô‰J{_Ob[ 8DeviceStatsMonitor.on_train_batch_start/num_device_alloc"trainer/global_step2:JP¬gahPbd ADeviceStatsMonitor.on_train_batch_start/oversize_segments.current"trainer/global_step2:JPS^¹âsQbo LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated"trainer/global_step2:JPëæÒ8eRba >DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak"trainer/global_step2:JPq—Ä‹gSbc @DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current"trainer/global_step2:JP‘‡ƒeTba >DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed"trainer/global_step2:JPýtìcUb_ ·vobr ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current"trainer/global_step2:JPÓÄZgnpbj GDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak"trainer/global_step2:JPJZŽÔaqb] :DeviceStatsMonitor.on_train_batch_start/active.all.current"trainer/global_step2:JPü]Olrbh EDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated"trainer/global_step2:JP¯.ʘksbg DDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated"trainer/global_step2:JP,œ’¦mtbi FDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak"trainer/global_step2:JPâyAoubk HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed"trainer/global_step2:JPx+Æqvbm JDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current"trainer/global_step2:JP+$Ýnwbj GDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak"trainer/global_step2:JP®bÑæbxb^ ;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak"trainer/global_step2:JPÿûÜÖjybf CDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed"trainer/global_step2:JPôë+òxzbt QDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated"trainer/global_step2:JP›gZ‹a{b] :DeviceStatsMonitor.on_train_batch_start/num_oom_rejections"trainer/global_step2:JP…£n|bj GDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak"trainer/global_step2:JPÿ]øAp}bl IDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current"trainer/global_step2:JPA@ñ»m~bi FDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak"trainer/global_step2:JP± [hbd ADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed"trainer/global_step2:JPú3.’h€bc @DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak"trainer/global_step2:JPfüNpbk HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed"trainer/global_step2:JPCjEÃw‚br ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current"trainer/global_step2:JP§S—cƒb^ ;DeviceStatsMonitor.on_train_batch_start/segment.all.current"trainer/global_step2:JPÇ_5\j„be BDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak"trainer/global_step2:JPðKJØj…be BDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current"trainer/global_step2:JPtyûi†bd ADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak"trainer/global_step2:JPöó»To‡bj GDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed"trainer/global_step2:JP]â»lˆbg DDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current"trainer/global_step2:JPÁâm‰bh EDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed"trainer/global_step2:JP5Ðñ'XŠbS 0DeviceStatsMonitor.on_train_batch_start/num_ooms"trainer/global_step2:JP4˜¯Ðj‹be BDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current"trainer/global_step2:JP`4¹gŒbb ?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak"trainer/global_step2:JP~涖_bZ 7DeviceStatsMonitor.on_train_batch_start/active.all.peak"trainer/global_step2:JP ¿°oŽbj GDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current"trainer/global_step2:JP@ÁTubp MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed"trainer/global_step2:JPâ®f^bY 6DeviceStatsMonitor.on_train_batch_start/max_split_size"trainer/global_step2:JP( L+`‘b[ 8DeviceStatsMonitor.on_train_batch_start/active.all.freed"trainer/global_step2:JP%°= t’bo LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated"trainer/global_step2:JP hÂ*p“bk HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed"trainer/global_step2:JPÌæI`f”ba >DeviceStatsMonitor.on_train_batch_start/allocation.all.current"trainer/global_step2:JPVP«‹n•bi FDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak"trainer/global_step2:JPn)Ö0s–bn KDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated"trainer/global_step2:JPUWÕ–g—bb ?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed"trainer/global_step2:JPË¥žøf˜ba >DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak"trainer/global_step2:JPa522q™bl IDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current"trainer/global_step2:JPð!i\jšbe BDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed"trainer/global_step2:JPlèEp›bk HDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed"trainer/global_step2:JPxõOrœbm JDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current"trainer/global_step2:JPWê+¥lbg DDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated"trainer/global_step2:JPÜ hžbc @DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed"trainer/global_step2:JP¯¡'tŸbo LDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak"trainer/global_step2:JP€²{k bf CDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current"trainer/global_step2:JPXH}t¡bo LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated"trainer/global_step2:JP»Q|‰.¢) _wandbruntime‚41  _runtime‚41ÆuUx£ùN WEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚ 2747885336742 SDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 9900654592 MFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚35 ZHDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚ 2711417534464 XJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 165386696 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 434362880 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚23105 NCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚922686 cQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚ 2817915576320 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 VGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 8980994048 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 444596224 TFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 444596224 SADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚ 2825075972608 O@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 9679236096 ZHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚ 2788594352640 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚24772608 C;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚643 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚22832 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚455 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 WEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚ 2788594352640 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 QBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 9900654592 N?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 9900654592 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚23105 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚17170432 _MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚ 2817890803712 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 D8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚1630330 \LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 36650225664 XHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 36481619968 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚18746 NFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚950 VKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚242776 J?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚707644 G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚2605 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚3 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 XHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 36285376222 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 168605696 ODDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚668220 K@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚667765 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 232494592 QCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 182426056 ^LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚ 2711434573824 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 NCDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated‚941430 NCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚707646 VKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚425444 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚431 _runtime‚ 41.874777034 L@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚1649076 XIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 9456058368 [IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚ 2788611523072 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚18744 G?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚212 TBDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated‚ 2825261748736 SDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 9283702784 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 168605696 RGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚425441 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚2605 D>DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 434362880 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚452 ]MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 38267611136 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚212 \JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚ 2856389971968 JBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚431 _step‚1 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚18744 XFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚ 2856158414848 BDeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚22832 N@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 185776128 P>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚ 2825075972608 HDeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 434362880 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚452 ]MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 38267611136 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚212 \JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚ 2856389971968 JBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚431 _step‚1 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚18744 XFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚ 2856158414848 BDeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚22832 N@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 185776128 P>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚ 2825075972608 HDeviceStatsMonitor.on_train_batch_start/allocation.all.current‚18746 NFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚950 VKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚242776 J?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚707644 G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚2605 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚3 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 XHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 36285376222 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 168605696 ODDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚668220 K@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚667765 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 232494592 QCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 182426056 ^LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚ 2711434573824 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 NCDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated‚941430 NCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚707646 VKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚425444 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚431 _runtime‚ 41.874777034 L@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚1649076 XIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 9456058368 [IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚ 2788611523072 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚18744 G?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚212 TBDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated‚ 2825261748736 SDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 9283702784 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 168605696 RGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚425441 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚2605ß„ìOw¥br ODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated"trainer/global_step2:JP¯47”`¦b[ 8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections"trainer/global_step2:JPøtƒ“m§bh EDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak"trainer/global_step2:JP¯2Ò6k¨bf CDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed"trainer/global_step2:JP†¼í/f©ba >DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak"trainer/global_step2:JPæ‚*mªbh EDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak"trainer/global_step2:JP®­Óa«b\ 9DeviceStatsMonitor.on_train_batch_end/segment.all.current"trainer/global_step2:JPAJûºd¬b_ DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed"trainer/global_step2:JPê˜<šuÃbp MDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current"trainer/global_step2:JP–ƒÐI\ÄbW 4DeviceStatsMonitor.on_train_batch_end/max_split_size"trainer/global_step2:JP¬\niÅbd ADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed"trainer/global_step2:JP {|²kÆbf CDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current"trainer/global_step2:JP~É[qÇbl IDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated"trainer/global_step2:JPÄÒÝÍbÈb] :DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams"trainer/global_step2:JPˆäµ›hÉbc @DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed"trainer/global_step2:JP³ Ì‚rÊbm JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated"trainer/global_step2:JPkeÃhËbc @DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current"trainer/global_step2:JPMÕ’ÅdÌb_ DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current"trainer/global_step2:JP¨dÐÿdÝb_ DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak"trainer/global_step2:JPµÇ£ ^òbY 6DeviceStatsMonitor.on_train_batch_end/segment.all.peak"trainer/global_step2:JPÚFcgóbb ?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current"trainer/global_step2:JP¸”6iôbd ADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated"trainer/global_step2:JPÕÐVEkõbf CDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed"trainer/global_step2:JPÎú«Íröbm JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated"trainer/global_step2:JPò@¡f÷ba >DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed"trainer/global_step2:JP8„‰óqøbl IDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated"trainer/global_step2:JP "Žoùbj GDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current"trainer/global_step2:JPb!H¸lúbg DDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed"trainer/global_step2:JPs,Ørûbm JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated"trainer/global_step2:JP°°}]düb_ DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated"trainer/global_step2:JP±a#j—be BDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated"trainer/global_step2:JPpx÷f˜ba >DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed"trainer/global_step2:JPØ>j™be BDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated"trainer/global_step2:JP7Çkfšba >DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed"trainer/global_step2:JP@ˆÒj›be BDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated"trainer/global_step2:JPU„¦VœbQ .DeviceStatsMonitor.on_train_batch_end/num_ooms"trainer/global_step2:JPoV~Xnbi FDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed"trainer/global_step2:JP8<©Žožbj GDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current"trainer/global_step2:JPJ…—hŸbc @DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak"trainer/global_step2:JPÃÖ. ) _wandbruntime‚42  _runtime‚42tP‹ ¡„M TIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚433835 [KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 39023589888 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 168619008 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 144203776 trainer/global_step‚49 SDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 9456058368 E=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚968 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 PEDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚959928 PEDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚247038 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 _step‚2 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17170432 L>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 185789440 FDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 9373784469 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚643 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚2 LADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚959928 SCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 37206973952 ZJDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 37375592960 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 WIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 444596224 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚456 VDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚ 2912848833024 ZJDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 37172209834 E6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚643 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚35 \JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚ 2843975212544 UHDe?~F£rviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚17170432 J>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚1681574 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚431 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚212 I>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚680870 MBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚681329 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 VFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 37006822930 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 444596224 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚22834 _ODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 39236652544 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 431137949 UCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚ 2843958042112 M>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 9679237120 SEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 434363904 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚643 NDeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24772608 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 LADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚721644 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚18770 TIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚247494 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 \JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚ 2765299240960 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚212 F6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚643 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚35 \JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚ 2843975212544 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚17170432 J>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚1681574 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚431 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚212 I>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚680870 MBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚681329 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 VFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 37006822930 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 444596224 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚22834 _ODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 39236652544 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 431137949 UCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚ 2843958042112 M>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 9679237120 SEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 434363904 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚643 NDeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24772608 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 LADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚721644 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚18770 TIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚247494 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 \JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚ 2765299240960 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚212 FDeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 185789440 FDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 9373784469 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚643 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚2 LADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚959928 SCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 37206973952 ZJDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 37375592960 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 WIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 444596224 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚456 VDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚ 2912848833024 ZJDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 37172209834 E loss/SRqqt2/assignment_loss"trainer/global_step2:JP5ÅA¦b< loss/FRt1/assignment_loss"trainer/global_step2:JP¹-"™A§b< loss/FRt3/assignment_loss"trainer/global_step2:JPNuJÄC¨b> loss/SRqqt4/assignment_loss"trainer/global_step2:JP{&K@©b; loss/FBt4/detection_loss"trainer/global_step2:JP´7_ï7ªb2 loss/total_loss"trainer/global_step2:JPññÛîA«b< loss/FRt2/assignment_loss"trainer/global_step2:JP loss/SRqqt1/assignment_loss"trainer/global_step2:JPÜ;C®b> loss/SRqqt3/assignment_loss"trainer/global_step2:JP•ÆÎí@¯b; loss/FRt1/detection_loss"trainer/global_step2:JPû\zrB°b= loss/SRqqt2/detection_loss"trainer/global_step2:JP<þZ=A±b< loss/FBt2/assignment_loss"trainer/global_step2:JP(”1@²b; loss/FRt2/detection_loss"trainer/global_step2:JPäÚ¬bB³b= loss/SRqqt1/detection_loss"trainer/global_step2:JP.š†ê@´b; loss/FBt1/detection_loss"trainer/global_step2:JPU`òBµb= loss/SRqqt3/detection_loss"trainer/global_step2:JPñ׿@¶b; loss/FRt4/detection_loss"trainer/global_step2:JP-}’Þ@·b; loss/FBt3/detection_loss"trainer/global_step2:JP|y7Ä-¸b( epoch"trainer/global_step2:JP@«÷ÈA¹b< loss/FBt1/assignment_loss"trainer/global_step2:JPîÊQAºb< loss/FBt3/assignment_loss"trainer/global_step2:JPÝ‚ŠôA»b< loss/FBt4/assignment_loss"trainer/global_step2:JP‘Ÿ»Ò@¼b; loss/FRt3/detection_loss"trainer/global_step2:JPJ²¡}.½) _wandbruntime‚42  _runtime‚42Ë`sßB¾¼ .loss/FBt4/detection_loss‚5.900318368027608 &loss/total_loss‚138.06188292677456 /loss/FRt2/assignment_loss‚7.084048819541931 /loss/FRt4/assignment_loss‚7.407628621512313 0loss/SRqqt1/assignment_loss‚5.44891764169716 1loss/SRqqt3/assignment_loss‚5.264211929069375 .loss/FRt1/detection_loss‚4.798102200434014 0loss/SRqqt2/detection_loss‚4.840581493603216 /loss/FBt2/assignment_loss‚2.521885931491852 .loss/FRt2/detection_loss‚6.533866300544214 0loss/SRqqt1/detection_loss‚4.071689651134503 /loss/FBt1/detection_loss‚2.0946536547026797 0loss/SRqqt3/detection_loss‚2.801540563972491 /loss/FRt4/detection_loss‚2.7193170961568027 .loss/FBt3/detection_loss‚12.15177415476905 epoch‚0 % _timestamp‚1.7884144609232516e+09 /loss/FBt1/assignment_loss‚2.363362052223899 0loss/FBt3/assignment_loss‚2.4288061018343323 /loss/FBt4/assignment_loss‚2.295633896191915 .loss/FRt3/detection_loss‚2.778439454925366 1loss/SRqqt4/detection_loss‚1.8562849438447746 _runtime‚ 42.572532217 /loss/FBt2/detection_loss‚27.274549300471943 trainer/global_step‚49 0loss/SRqqt2/assignment_loss‚5.43925411636765 ,loss/FRt1/assignment_loss‚7.362784946287 /loss/FRt3/assignment_loss‚7.140735124930357 1loss/SRqqt4/assignment_loss‚5.483496563040095 _step‚3À‰(­F¿À /loss/FRt3/assignment_loss‚7.140735124930357 1loss/SRqqt4/assignment_loss‚5.483496563040095 _step‚3 .loss/FBt4/detection_loss‚5.900318368027608 &loss/total_loss‚138.06188292677456 /loss/FRt2/assignment_loss‚7.084048819541931 /loss/FRt4/assignment_loss‚7.407628621512313 0loss/SRqqt1/assignment_loss‚5.44891764169716 1loss/SRqqt3/assignment_loss‚5.264211929069375 .loss/FRt1/detection_loss‚4.798102200434014 0loss/SRqqt2/detection_loss‚4.840581493603216 /loss/FBt2/assignment_loss‚2.521885931491852 .loss/FRt2/detection_loss‚6.533866300544214 0loss/SRqqt1/detection_loss‚4.071689651134503 /loss/FBt1/detection_loss‚2.0946536547026797 0loss/SRqqt3/detection_loss‚2.801540563972491 /loss/FRt4/detection_loss‚2.7193170961568027 .loss/FBt3/detection_loss‚12.15177415476905 epoch‚0 % _timestamp‚1.7884144609232516e+09 /loss/FBt1/assignment_loss‚2.363362052223899 0loss/FBt3/assignment_loss‚2.4288061018343323 /loss/FBt4/assignment_loss‚2.295633896191915 .loss/FRt3/detection_loss‚2.778439454925366 1loss/SRqqt4/detection_loss‚1.8562849438447746 _runtime‚ 42.572532217 /loss/FBt2/detection_loss‚27.274549300471943 trainer/global_step‚49 0loss/SRqqt2/assignment_loss‚5.43925411636765 ,loss/FRt1/assignment_loss‚7.362784946287Væ»W“À‚(:ˆ ÿ“äÔ’µŸ© gpu.0.gpu‚52.0 gpu.0.memory‚6, gpu.0.memoryAllocated‚12.163162231445312* gpu.0.memoryAllocatedBytes‚ 10448076800 gpu.0.temp‚35.0 gpu.0.powerWatts‚168.124' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚33.6248 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0[S†è;Á‚(:° ÿ“äԮÿª disk.nvme0n1.in‚ 11.80859375 disk.sda2.out‚0 disk.drbd2266.in‚0 disk.sda3.in‚0 disk.sda3.out‚ 1.42578125 disk.rbd0.in‚0 disk.drbd2266.out‚0 disk.sda.in‚0( proc.memory.availableMB‚ 2613.7421875 disk.sda2.in‚0 network.sent‚64122 network.recv‚77275 disk.drbd1716.out‚0 disk.sda1.out‚0 disk.dm-0.in‚ 11.80859375$ memory_percent‚84.04698371887207* proc.memory.percent‚16.438984870910645 disk.drbd1716.in‚0 disk.drbd2080.out‚0 disk.rbd0.out‚0 disk.dm-0.out‚ 183.474609375 disk.sda.out‚ 1.42578125 disk.drbd2265.in‚0 disk.drbd2265.out‚0 disk.rbd1.out‚0! disk.nvme0c0n1.in‚ 11.80859375 cpu‚26.242989089122258 proc.cpu.threads‚17 disk.rbd1.in‚0$ disk.nvme0c0n1.out‚ 183.349609375* disk./.usagePercent‚16.723550602080568% disk./.usageGB‚249.19298934936523 disk.dm-1.in‚0" disk.nvme0n1.out‚ 183.349609375# proc.memory.rssMB‚ 2693.36328125 disk.dm-1.out‚ 1.42578125 disk.drbd2080.in‚0 disk.sda1.in‚0&Éøv“‚(:ˆ Ž”äÔß„ª¨ gpu.0.gpu‚49.0 gpu.0.memory‚4, gpu.0.memoryAllocated‚12.163162231445312* gpu.0.memoryAllocatedBytes‚ 10448076800 gpu.0.temp‚36.0 gpu.0.powerWatts‚163.162' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚32.6324 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0ÖŠ¨`GÂ(:¼ Ž”äÔœÕÔª disk.sda1.out‚0 disk.sda2.out‚0 disk.drbd2265.out‚0 disk.sda3.out‚ 1.97265625 disk.dm-1.in‚0 disk.drbd1716.in‚0* proc.memory.percent‚16.439056396484375 network.sent‚132576% disk./.usageGB‚249.17454528808594 disk.dm-0.in‚ 11.80859375! disk.dm-0.out‚251.10498046875 disk.drbd2080.out‚0$ memory_percent‚84.06050205230713 proc.cpu.threads‚17 disk.sda.out‚ 1.97265625 disk.nvme0n1.in‚ 11.80859375 disk.drbd2266.in‚0) proc.memory.availableMB‚ 2611.52734375 disk.rbd0.out‚0 disk.drbd2266.out‚0 cpu‚26.154896763536303* disk./.usagePercent‚16.722312805652514& disk.nvme0c0n1.out‚250.97998046875 disk.sda3.in‚0 disk.sda2.in‚0 proc.memory.rssMB‚2693.375 disk.drbd2265.in‚0 disk.rbd1.in‚0! disk.nvme0c0n1.in‚ 11.80859375 disk.dm-1.out‚ 1.97265625 disk.rbd0.in‚0$ disk.nvme0n1.out‚250.97998046875 network.recv‚109472 disk.sda.in‚0 disk.sda1.in‚0 disk.rbd1.out‚ 0.1484375 disk.drbd1716.out‚0 disk.drbd2080.in‚0—êÉï’Ä‚(:‡ ”äÔï‹´¨ gpu.0.gpu‚0.0 gpu.0.memory‚0, gpu.0.memoryAllocated‚12.163162231445312* gpu.0.memoryAllocatedBytes‚ 10448076800 gpu.0.temp‚35.0 gpu.0.powerWatts‚151.084' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚30.2168 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0ºýg?Å‚(:´ ”äÔ—¶¹ª disk.sda.in‚0 disk.drbd1716.out‚0 disk.drbd2080.in‚0 disk.rbd0.out‚0 network.recv‚115052! disk.nvme0c0n1.in‚ 11.80859375 disk.dm-1.out‚2.359375 disk.nvme0n1.in‚ 11.80859375* proc.memory.percent‚16.439056396484375 network.sent‚140052 disk.drbd2265.out‚0& disk.nvme0c0n1.out‚307.88232421875$ disk.nvme0n1.out‚307.88232421875 disk.sda2.in‚0) proc.memory.availableMB‚ 2611.96484375 disk.drbd2265.in‚0 disk.rbd1.in‚0! disk.dm-0.out‚308.00732421875 disk.sda3.out‚2.359375 cpu‚26.0886337268947 disk.drbd2266.in‚0 disk.drbd2266.out‚0 disk.dm-1.in‚0 proc.memory.rssMB‚2693.375* disk./.usagePercent‚16.725899983222288% disk./.usageGB‚249.22799682617188 disk.rbd1.out‚ 0.1484375 disk.dm-0.in‚ 11.80859375 disk.rbd0.in‚0 disk.sda1.in‚0 disk.sda2.out‚0 proc.cpu.threads‚17 disk.sda.out‚2.359375 disk.sda3.in‚0 disk.drbd1716.in‚0 disk.drbd2080.out‚0 disk.sda1.out‚0$ memory_percent‚84.05783176422119V MT.Æ) _wandbruntime‚80  _runtime‚80°ï~ÞÁÇ» & lr-AdamW/pg2‚0.0003072592212319169 trainer/global_step‚99 % _timestamp‚1.7884144985600784e+09 _runtime‚ 80.209434095 _step‚4 & lr-AdamW/pg1‚0.0003072592212319169çëHÅÈ¿ & lr-AdamW/pg1‚0.0003072592212319169 & lr-AdamW/pg2‚0.0003072592212319169 trainer/global_step‚99 % _timestamp‚1.7884144985600784e+09 _runtime‚ 80.209434095 _step‚4—$«”.É) _wandbruntime‚80  _runtime‚80Ž¥mI#Ê„O ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 185776128 OCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚1407644 QCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 182426056 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚431 L@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚3276028 ZHDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚ 5404650891264 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚22834 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 144203776 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 XJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 165386696 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 168605696 ZHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚ 5556778826240 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚462 MFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚35 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 O@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 9373784469 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚2605 \LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 72945753088 SGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚1407646 XFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚ 5690603992576 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚24772608 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 _step‚5 TBDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated‚ 5629741749760 N@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 185776128 SDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 9283702784 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 434363904 VGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 8980994048 G?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚212 \JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚ 5690848132608 ]MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 75991217664 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 WEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚ 5477218200950 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚22834 XHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 72777147392 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚23107 L@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚1326315 RGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚844991 QBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 9900654592 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚212 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 168605696 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚3 C;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚643 ^LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚ 5556795996672 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 244140032 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 SEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 309121024 \LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 72550270326 P>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚ 5629555973632 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚2605 ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 219367424 BDeviceStatsMonitor.on_train_batch_start/allocation.all.current‚18746 K?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚1407644 YIDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 72945753088 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 234590720 SADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚ 5477035774894 JBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚431 _runtime‚ 80.210948272 SDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 9900654592 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 444596224 JBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚212 VGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 9283702784 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚459 aQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 76210585088 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 UGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 431137949 HDeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 @8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚643 SGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚1868382 VKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚481783ñ?ZƒŽ'ˈO WEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚ 5556778826240 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚18744 @8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚643 N?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 9900654592 XIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 9456058368 SADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚ 5629555973632 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 HDeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 @8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚643 SGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚1868382 VKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚481783 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 185776128 OCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚1407644 QCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 182426056 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚431 L@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚3276028 ZHDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚ 5404650891264 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚22834 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 144203776 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 XJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 165386696 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 168605696 ZHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚ 5556778826240 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚462 MFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚35 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 O@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 9373784469 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚2605 \LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 72945753088 SGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚1407646 XFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚ 5690603992576 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚24772608 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 _step‚5 TBDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated‚ 5629741749760 N@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 185776128 SDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 9283702784 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 434363904 VGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 8980994048 G?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚212 \JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚ 5690848132608 ]MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 75991217664 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 WEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚ 5477218200950 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚22834 XHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 72777147392 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚23107 L@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚1326315 RGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚844991 QBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 9900654592 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚212 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 168605696 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚3 C;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚643 ^LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚ 5556795996672 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 244140032 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 SEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 309121024 \LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 72550270326 P>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚ 5629555973632 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚2605 ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 219367424 BDeviceStatsMonitor.on_train_batch_start/allocation.all.current‚18746 K?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚1407644 YIDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 72945753088 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 234590720 SADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚ 5477035774894 JBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚431 _runtime‚ 80.210948272 SDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 9900654592 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 444596224 JBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚212 VGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 9283702784 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚459 aQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 76210585088 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 UGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 431137949 HDeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚1339513 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚17170432 PEDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚486131 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 144203776 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚431 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚212 trainer/global_step‚99 QBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 9283702784 E=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚987 ZJDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 73271717242 L=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 9900654592 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 UCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚ 5612142515712 MADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚1421644 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚2605 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 TEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 8980994048 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 234590720 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚212 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 MADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚1421646 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚23107 XFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚ 5458515558400 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 444596224 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚643 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 SCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 73502501376 QEDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚1886880 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 168619008 E8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 FDeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 185789440 SEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 434363904 SDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 9456058368 M>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 9679237120 VDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚ 5747298115584 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 431137949 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚441 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚2 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚18770 I=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚1868110 UCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚ 5685830806528 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚17039360 VFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 73106330338 QEDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚1421646 WIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 444596224 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚35 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 239932416 QCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 309121024 VHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 165386904 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 $ _timestamp‚1.788414499239596e+09 YGDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚ 5612159686144 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 FDeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚3308526 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚22834 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚643 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 _step‚6 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚18772 NDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 9373784469 _runtime‚ 80.889337964 F:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚3308526 VFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 73502501376 _ODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 76966061056 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚643 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚431 Q?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚ 5531621888738 Q?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚ 5685645017088 ZJDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 73671120384 TIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚853385 ]KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚ 5670547214336 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚212 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚23107 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚18770 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24772608 TEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 9283702784 [MDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚ 215159808 QBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 9900654592L´Jµ—&ΑM UCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚ 5612142515712 MADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚1421644 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚2605 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 TEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 8980994048 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 234590720 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚212 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 MADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚1421646 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚23107 XFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚ 5458515558400 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 444596224 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚643 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 SCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 73502501376 QEDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚1886880 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 168619008 E8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 FDeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 185789440 SEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 434363904 SDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 9456058368 M>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 9679237120 VDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚ 5747298115584 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 431137949 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚441 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚2 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚18770 I=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚1868110 UCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚ 5685830806528 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚17039360 VFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 73106330338 QEDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚1421646 WIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 444596224 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚35 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 239932416 QCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 309121024 VHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 165386904 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 $ _timestamp‚1.788414499239596e+09 YGDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚ 5612159686144 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 FDeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚3308526 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚22834 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚643 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 _step‚6 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚18772 NDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 9373784469 _runtime‚ 80.889337964 F:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚3308526 VFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 73502501376 _ODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 76966061056 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚643 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚431 Q?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚ 5531621888738 Q?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚ 5685645017088 ZJDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 73671120384 TIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚853385 ]KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚ 5670547214336 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚212 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚23107 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚18770 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24772608 TEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 9283702784 [MDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚ 215159808 QBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 9900654592 XIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 9456058368 UCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚ 5531804315002 I=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚1421644 R@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚ 5685830806528 J>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚1339513 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚17170432 PEDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚486131 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 144203776 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚431 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚212 trainer/global_step‚99 QBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 9283702784 E=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚987 ZJDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 73271717242 L=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 9900654592 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0½Ù=ë.Ï) _wandbruntime‚80  _runtime‚80—û‡LÐÆ 1loss/SRqqt1/assignment_loss‚3.494162678718567 2loss/SRqqt3/assignment_loss‚3.2004425937746777 /loss/FRt4/detection_loss‚3.1119716739548102 _step‚7 /loss/FRt1/assignment_loss‚7.289822136728387 0loss/FBt3/assignment_loss‚0.5625244889063118 .loss/FRt2/detection_loss‚4.782159038782121 .loss/FRt3/detection_loss‚2.916116375063177 1loss/SRqqt2/detection_loss‚4.5758028187461806 0loss/FRt3/assignment_loss‚7.1249144077301025 0loss/FBt4/assignment_loss‚0.3903941528013431 1loss/SRqqt4/assignment_loss‚3.673038941337949 /loss/FBt3/detection_loss‚10.736211526496657 epoch‚0 $ _timestamp‚1.788414499241256e+09 _runtime‚ 80.890441747 /loss/FRt2/assignment_loss‚7.266049566268921 /loss/FRt4/assignment_loss‚7.327264721904482 0loss/FBt2/assignment_loss‚0.5758577802334912 .loss/FRt1/detection_loss‚3.554408383996863 1loss/SRqqt4/detection_loss‚1.4657420737316205 .loss/FBt2/detection_loss‚19.54024927318096 .loss/FBt4/detection_loss‚1.743509426759917 &loss/total_loss‚113.48768434832766 0loss/FBt1/assignment_loss‚0.7697286284808069 1loss/SRqqt3/detection_loss‚3.7795282514007003 .loss/FBt1/detection_loss‚9.606690938264096 0loss/SRqqt1/detection_loss‚2.434643730225544 trainer/global_step‚99 1loss/SRqqt2/assignment_loss‚3.566450740839984L(S&'ÑÊ .loss/FBt4/detection_loss‚1.743509426759917 &loss/total_loss‚113.48768434832766 0loss/FBt1/assignment_loss‚0.7697286284808069 1loss/SRqqt3/detection_loss‚3.7795282514007003 .loss/FBt1/detection_loss‚9.606690938264096 0loss/SRqqt1/detection_loss‚2.434643730225544 ë>ý)trainer/global_step‚99 1loss/SRqqt2/assignment_loss‚3.566450740839984 1loss/SRqqt1/assignment_loss‚3.494162678718567 2loss/SRqqt3/assignment_loss‚3.2004425937746777 /loss/FRt4/detection_loss‚3.1119716739548102 _step‚7 /loss/FRt1/assignment_loss‚7.289822136728387 0loss/FBt3/assignment_loss‚0.5625244889063118 .loss/FRt2/detection_loss‚4.782159038782121 .loss/FRt3/detection_loss‚2.916116375063177 1loss/SRqqt2/detection_loss‚4.5758028187461806 0loss/FRt3/assignment_loss‚7.1249144077301025 0loss/FBt4/assignment_loss‚0.3903941528013431 1loss/SRqqt4/assignment_loss‚3.673038941337949 /loss/FBt3/detection_loss‚10.736211526496657 epoch‚0 $ _timestamp‚1.788414499241256e+09 _runtime‚ 80.890441747 /loss/FRt2/assignment_loss‚7.266049566268921 /loss/FRt4/assignment_loss‚7.327264721904482 0loss/FBt2/assignment_loss‚0.5758577802334912 .loss/FRt1/detection_loss‚3.554408383996863 1loss/SRqqt4/detection_loss‚1.4657420737316205 .loss/FBt2/detection_loss‚19.54024927318096 ™ß.“Ò‚(:ˆ ¬”äÔá’£¨ gpu.0.gpu‚44.0 gpu.0.memory‚5, gpu.0.memoryAllocated‚12.163162231445312* gpu.0.memoryAllocatedBytes‚ 10448076800 gpu.0.temp‚36.0 gpu.0.powerWatts‚170.014' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚34.0028 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0µ´çôDÓ‚(:¹ ¬”äÔ櫈« network.sent‚146539 network.recv‚119676 disk.sda1.out‚0 disk.sda3.out‚2.953125 disk.drbd1716.out‚0 disk.drbd2080.out‚0 disk.sda1.in‚0 disk.drbd2266.out‚0 disk.rbd0.out‚0 disk.nvme0n1.in‚ 11.80859375* disk./.usagePercent‚16.727425788126364# disk./.usageGB‚249.250732421875 disk.rbd1.in‚0 disk.dm-1.out‚2.953125 disk.drbd2080.in‚0) proc.memory.availableMB‚ 2611.83203125 disk.dm-0.in‚ 11.80859375 disk.rbd0.in‚0$ memory_percent‚84.05864238739014# proc.memory.rssMB‚ 2693.37890625* proc.memory.percent‚16.439080238342285% disk.nvme0c0n1.out‚373.3974609375 disk.drbd2265.in‚0 disk.drbd2265.out‚0 disk.drbd1716.in‚0 cpu‚26.037226122924082 proc.cpu.threads‚17 disk.dm-0.out‚373.5224609375 disk.sda.in‚0 disk.sda2.out‚0! disk.nvme0c0n1.in‚ 11.80859375 disk.rbd1.out‚ 0.1806640625 disk.dm-1.in‚0 disk.sda2.in‚0 disk.sda3.in‚0# disk.nvme0n1.out‚373.3974609375 disk.drbd2266.in‚0 disk.sda.out‚2.953125ÚÕǶ“Ô‚(:ˆ »”äÔÖÞÿ§ gpu.0.gpu‚44.0 gpu.0.memory‚5, gpu.0.memoryAllocated‚12.163162231445312* gpu.0.memoryAllocatedBytes‚ 10448076800 gpu.0.temp‚36.0 gpu.0.powerWatts‚156.578' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚31.3156 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚04‘ L<Õ‚(:± »”äÔýã‚«# disk.nvme0n1.out‚434.0654296875 disk.drbd2266.out‚0 disk.dm-0.out‚434.1904296875 disk.sda.in‚0 disk.dm-1.in‚0 disk.drbd1716.out‚0$ memory_percent‚84.05900001525879 network.sent‚196968 disk.drbd2265.in‚0 disk.sda3.in‚0 disk.sda3.out‚3.3125 disk.drbd1716.in‚0 disk.drbd2080.out‚0( proc.memory.availableMB‚ 2611.7734375 cpu‚25.999980599008225 disk.drbd2265.out‚0 disk.nvme0n1.in‚ 11.80859375" proc.memory.rssMB‚ 2693.3828125 disk.sda1.in‚0% disk.nvme0c0n1.out‚434.0654296875 disk.sda2.out‚0 disk.sda.out‚3.3125 proc.cpu.threads‚17 disk.sda1.out‚0 disk.sda2.in‚0 disk.drbd2266.in‚0 disk.rbd1.out‚ 0.1806640625 disk.drbd2080.in‚0 disk.rbd0.out‚0* proc.memory.percent‚16.439104080200195 network.recv‚126114) disk./.usagePercent‚16.72866512059962 disk.rbd1.in‚0 disk.dm-1.out‚3.3125 disk.rbd0.in‚0$ disk./.usageGB‚249.2691993713379! disk.nvme0c0n1.in‚ 11.80859375 disk.dm-0.in‚ 11.80859375½€[Î0Ö+ _wandbruntime‚118  _runtime‚118ö¯£Ã×½ % _timestamp‚1.7884145363880389e+09 _runtime‚ 118.037405171 _step‚8 & lr-AdamW/pg1‚0.0004624406460965214 & lr-AdamW/pg2‚0.0004624406460965214 trainer/global_step‚14980â¬ÇØÁ & lr-AdamW/pg1‚0.0004624406460965214 & lr-AdamW/pg2‚0.0004624406460965214 trainer/global_step‚149 % _timestamp‚1.7884145363880389e+09 _runtime‚ 118.037405171 _step‚8máoö0Ù+ _wandbruntime‚118  _runtime‚118Ö·Ù–'ÚO UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 168605696 P>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚ 8434035974656 O@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 9679237120 SGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚2107646 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚431 D8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚4884234 OCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚2107646 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚2 K?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚2107644 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 ZIDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 109241280512 [IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚ 8324980470272 WEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚ 8324963299840 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚18746 BDeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚2605 ]LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 109241280512 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚464 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 @8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚643 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚22834 N@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 185776128 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 434363904 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚212 OCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚2107644 HDeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 XIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 9456058368 TFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 444596224 E=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚643 SEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 309121024 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 234590720 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 QBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 9900654592 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚22834 NFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚969 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 144203776 SDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 9900654592 OCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚2776590 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 444596224híAóš'Û”O =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 trainer/global_step‚149 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 168605696 VKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚720734 ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 215173120 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 444596224 C;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚643 G?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚431 L@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚1984811 MFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚35 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 QCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 182426056 ]LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 108649777734 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚2605 ]LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 109241280512 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚464 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 @8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚643 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚22834 N@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 185776128 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 434363904 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚212 OCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚2107644 HDeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 XIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 9456058368 TFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 444596224 E=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚643 SEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 309121024 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 234590720 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 QBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 9900654592 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚22834 NFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚969 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 144203776 SDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 9900654592 OCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚2776590 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 444596224 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 168605696 P>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚ 8434035974656 O@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 9679237120 SGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚2107646 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚431 D8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚4884234 OCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚2107646 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚2 K?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚2107644 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 ZIDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 109241280512 [IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚ 8324980470272 WEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚ 8324963299840 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚18746 BDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 SDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 9456058368 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚17170432 TEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 8980994048 SEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 434363904 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 250418176 R@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚ 8490310807552 I=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚2795062 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 168619008 TCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 109798028800 ZHDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚ 8582003386880 Q?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚ 8260954752946 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚431 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚18772 MADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚2121646 WFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 109798028800 J>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚1997986 [MDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚ 225645568 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 FDeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚4935478 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 \JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚ 8151765954560 M>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 9679237120 EDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 9373784469 [JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 109371224650 QBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 9900654592 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 444596224 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 168619008 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚22834 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚643 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚431 $ _timestamp‚1.788414537064789e+09 _runtime‚ 118.714494049 MADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚2795062 E=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚987 \KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 114483783168 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚2605 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚212 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚3 VDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚ 8581752968704 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚17039360 L=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 9900654592 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚431 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 TIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚725519 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 MADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚2121644 `ODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 114709428736 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚643 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚1272932 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚465 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 I=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚2121644 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 182426264 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 UCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚ 8380326989312 UCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚ 8490310807552 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24772608 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚212 YGDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚ 8380344159744 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚18770 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚35 XJDeviceStatsMonCÕC•itor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 234590720 UCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚ 8261137179210 QBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 9283702784 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚643 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚643 _step‚10 L>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 185789440 PBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 434363904 LDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚969 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 XFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚ 8151748915200 TEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 9283702784 PEDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚725054 ]KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚ 8467269185536 trainer/global_step‚149 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 431137949 XIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 9456058368 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 aODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚ 8467293958144 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚18770 J;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 9679237120 [JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 109966647808 BDeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚4935478 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 \JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚ 8151765954560 M>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 9679237120 EDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 9373784469 [JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 109371224650 QBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 9900654592 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 444596224 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 168619008 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚22834 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚643 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚431 $ _timestamp‚1.788414537064789e+09 _runtime‚ 118.714494049 MADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚2795062 E=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚987 \KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 114483783168 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚2605 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚212 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚3 VDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚ 8581752968704 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚17039360 L=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 9900654592 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚431 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 TIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚725519 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 MADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚2121644 `ODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 114709428736 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚643 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚1272932 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚465 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 I=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚2121644 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 182426264 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 UCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚ 8380326989312 UCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚ 8490310807552 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24772608 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚212 YGDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚ 8380344159744 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚18770 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚35 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 234590720 UCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚ 8261137179210 QBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 9283702784 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚643 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚643 _step‚10 L>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 185789440 PBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 434363904 LDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚969 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 XFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚ 8151748915200 TEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 9283702784 PEDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚725054 ]KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚ 8467269185536 trainer/global_step‚149 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 431137949 XIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 9456058368 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 aODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚ 8467293958144 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚18770 J;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 9679237120 [JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 109966647808 BDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 SDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 9456058368 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚17170432 TEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 8980994048 SEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 434363904 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 250418176 R@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚ 8490310807552 I=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚2795062 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 168619008 TCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 109798028800 ZHDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚ 8582003386880 Q?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚ 8260954752946 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚431 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚18772 MADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚2121646 WFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 109798028800 J>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚1997986 [MDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚ 225645568 rÜZ0ß+ _wandbruntime‚118  _runtime‚118xBÀéOàÉ /loss/FRt1/assignment_loss‚6.658947366656679 2loss/SRqqt3/assignment_loss‚3.2291494881341216 1loss/SRqqt1/assignment_loss‚3.125696995365086 0loss/FBt2/assignment_loss‚0.6357107677618582 0loss/FBt3/assignment_loss‚0.4218402079359772 0loss/SRqqt3/detection_loss‚2.691002281844069 .loss/FBt3/detection_loss‚7.566692072309945 /loss/FRt2/assignment_loss‚6.823561103720414 /loss/FRt3/assignment_loss‚6.669838594353718 2loss/SRqqt4/assignment_loss‚3.4389966421349105 0loss/FBt1/assignment_loss‚0.7683003938291222 1loss/SRqqt1/detection_loss‚1.9844510802415325 -loss/FBt1/detection_loss‚9.98112948681516 /loss/FBt4/detection_loss‚1.0543729707618428 0loss/FRt4/assignment_loss‚6.6706709317993695 0loss/FBt4/assignment_loss‚0.3491803375551566 /loss/FRt3/detection_loss‚3.4599131523307576 trainer/global_step‚149 _step‚11 .loss/FRt4/detection_loss‚2.883687911849273 epoch‚0 /loss/FBt2/detection_loss‚10.462887470211301 % _timestamp‚1.7884145370663984e+09 2loss/SRqqt2/assignment_loss‚3.2629494972718067 /loss/FRt1/detection_loss‚3.3401803163566974 .loss/FRt2/detection_loss‚4.613691562100461 /loss/SRqqt2/detection_loss‚4.15658310170357 0loss/SRqqt4/detection_loss‚1.601240631976828 %loss/total_loss‚95.85067436501966 _runtime‚ 118.715590758:ÿ*SáÍ /loss/FRt3/assignment_loss‚6.669838594353718 2loss/SRqqt4/assignment_loss‚3.4389966421349105 0loss/FBt1/assignment_loss‚0.7683003938291222 1loss/SRqqt1/detection_loss‚1.9844510802415325 -loss/FBt1/detection_loss‚9.98112948681516 /loss/FBt4/detection_loss‚1.0543729707618428 0loss/FRt4/assignment_loss‚6.6706709317993695 0loss/FBt4/assignment_loss‚0.3491803375551566 /loss/FRt3/detection_loss‚3.4599131523307576 trainer/global_step‚149 _step‚11 .loss/FRt4/detection_loss‚2.883687911849273 epoch‚0 /loss/FBt2/detection_loss‚10.462887470211301 % _timestamp‚1.7884145370663984e+09 2loss/SRqqt2/assignment_loss‚3.2629494972718067 /loss/FRt1/detection_loss‚3.3401803163566974 .loss/FRt2/detection_loss‚4.613691562100461 /loss/SRqqt2/detection_loss‚4.15658310170357 0loss/SRqqt4/detection_loss‚1.601240631976828 %loss/total_loss‚95.85067436501966 _runtime‚ 118.715590758 /loss/FRt1/assignment_loss‚6.658947366656679 2loss/SRqqt3/assignment_loss‚3.2291494881341216 1loss/SRqqt1/assignment_loss‚3.125696995365086 0loss/FBt2/assignment_loss‚0.6357107677618582 0loss/FBt3/assignment_loss‚0.4218402079359772 0loss/SRqqt3/detection_loss‚2.691002281844069 .loss/FBt3/detection_loss‚7.566692072309945 /loss/FRt2/assignment_loss‚6.823561103720414 Q¦ÏGâ‚(:¼ Ê”äԲ) disk./.usagePercent‚16.73022932663383 disk.dm-1.out‚3.859375 disk.nvme0n1.in‚ 11.80859375 disk.sda1.out‚0 disk.sda2.out‚0 disk.sda1.in‚0 disk.drbd2265.out‚0 disk.rbd1.out‚ 0.24365234375! disk.dm-0.out‚509.08154296875 disk.sda.out‚3.859375# memory_percent‚84.0630292892456 disk.drbd1716.in‚0 disk.rbd0.in‚0$ disk.nvme0n1.out‚508.95654296875& disk.nvme0c0n1.out‚508.95654296875 disk.dm-0.in‚ 11.80859375 network.recv‚131281 disk.drbd2080.in‚0 disk.rbd1.in‚0) proc.memory.availableMB‚ 2611.11328125 network.sent‚203610 disk.drbd1716.out‚0 disk.sda3.out‚3.859375# proc.memory.rssMB‚ 2693.38671875% disk./.usageGB‚249.29250717163086 disk.dm-1.in‚0 disk.drbd2080.out‚0 disk.rbd0.out‚0 disk.drbd2266.out‚0 disk.sda.in‚0 cpu‚25.97079291306957 proc.cpu.threads‚17 disk.drbd2266.in‚0! disk.nvme0c0n1.in‚ 11.80859375 disk.sda2.in‚0 disk.drbd2265.in‚0 disk.sda3.in‚0* proc.memory.percent‚16.439127922058105k³Ê“ã‚(:ˆ Ê”äÔŒÆæÊ gpu.0.gpu‚50.0 gpu.0.memory‚4, gpu.0.memoryAllocated‚12.163162231445312* gpu.0.memoryAllocatedBytes‚ 10448076800 gpu.0.temp‚36.0 gpu.0.powerWatts‚135.693' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚27.1386 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0ckÛÓä‚(:ˆ Ù”äÔÚ嶨 gpu.0.gpu‚38.0 gpu.0.memory‚5, gpu.0.memoryAllocated‚12.163162231445312* gpu.0.memoryAllocatedBytes‚ 10448076800 gpu.0.temp‚36.0 gpu.0.powerWatts‚178.124' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚35.6248 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0-úk¨Må‚(: ٔäÔ¡ø©ª network.sent‚253094 disk.drbd2080.in‚0 disk.drbd2080.out‚0$ disk.nvme0n1.out‚561.51025390625# proc.memory.rssMB‚ 2693.38671875* proc.memory.percent‚16.439127922058105* disk./.usagePercent‚16.731270765283117 disk.dm-1.out‚ 4.3515625 disk.drbd1716.out‚0 disk.nvme0n1.in‚ 11.80859375 disk.rbd1.in‚0 disk.dm-1.in‚0 disk.sda1.in‚0 disk.dm-0.in‚ 11.80859375 disk.sda.in‚0 disk.sda.out‚ 4.3515625$ memory_percent‚84.06360149383545 disk.drbd1716.in‚0 disk.rbd0.in‚0 disk.drbd2266.in‚0 disk.drbd2265.in‚0 disk.sda3.out‚ 4.3515625) proc.memory.availableMB‚ 2611.01953125 proc.cpu.threads‚17 disk.rbd1.out‚ 0.24365234375! disk.nvme0c0n1.in‚ 11.80859375& disk.nvme0c0n1.out‚561.51025390625% disk./.usageGB‚249.30802536010742 disk.rbd0.out‚0 disk.drbd2265.out‚0 disk.sda3.in‚0 network.recv‚136647 disk.sda2.in‚0 disk.drbd2266.out‚0 cpu‚25.949914472812157 disk.sda1.out‚0 disk.sda2.out‚0! disk.dm-0.out‚561.63525390625¥^†“æ‚(:ˆ è”äÔï´¨ gpu.0.gpu‚47.0 gpu.0.memory‚5, gpu.0.memoryAllocated‚12.163162231445312* gpu.0.memoryAllocatedBytes‚ 10448076800 gpu.0.temp‚36.0 gpu.0.powerWatts‚170.643' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚34.1286 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0HXfNç‚(:à è”äÔÏ™ª disk.dm-0.in‚ 11.80859375 disk.sda3.out‚ 4.98828125% disk./.usageGB‚249.33053970336914 disk.nvme0n1.in‚ 11.80859375) proc.memory.availableMB‚ 2611.25390625 disk.rbd0.out‚0 disk.sda1.out‚0 disk.rbd1.in‚0& disk.nvme0c0n1.out‚630.48583984375 proc.cpu.threads‚17 disk.rbd0.in‚0 disk.sda.in‚0 disk.sda3.in‚0 disk.drbd1716.in‚0* disk./.usagePercent‚16.732781721750204 disk.sda2.in‚0 disk.drbd2265.out‚0 disk.sda2.out‚0 network.recv‚141123 disk.drbd2080.in‚0$ disk.nvme0n1.out‚630.48583984375 disk.drbd2265.in‚0! disk.nvme0c0n1.in‚ 11.80859375 disk.dm-1.in‚0 disk.dm-1.out‚ 4.98828125 disk.drbd2266.in‚0 disk.drbd2266.out‚0 disk.drbd1716.out‚0# proc.memory.rssMB‚ 2693.38671875 cpu‚25.926978143223657 disk.drbd2080.out‚0 disk.sda1.in‚0 disk.rbd1.out‚ 0.306640625* proc.memory.percent‚16.439127922058105 network.sent‚259018! disk.dm-0.out‚630.61083984375 disk.sda.out‚ 4.98828125$ memory_percent‚84.06217098236084û<¥0è+ _wandbruntime‚156  _runtime‚156(èÅYÃé½ & lr-AdamW/pg1‚0.0006176220709611258 & lr-AdamW/pg2‚0.0006176220709611258 trainer/global_step‚199 $ _timestamp‚1.788414574510234e+09 _runtime‚ 156.159585403 _step‚12¥^ÓµÇêÁ & lr-AdamW/pg1‚0.0006176220709611258 & lr-AdamW/pg2‚0.0006176220709611258 trainer/global_step‚199 $ _timestamp‚1.788414574510234e+09 _runtime‚ 156.159585403 _step‚12 T·0ë+ _wandbruntime‚156  _runtime‚156ó —#§'ì¡O _LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚11093164943872 E=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚643 OCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚2807646 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚2605 OCDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated‚3722286 O@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 9679237120 G?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚990 JBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚431 YHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 144583898446 ZKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 9456058368 XIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 9456058368 G?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚212 XEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚11238701751808 ]LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 145536807936 @8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚643 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 XJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 165386696 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 JBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚212 \IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚11093164943872 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚22834 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚458 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚1684094 `MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚11208056717312 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 [HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚10791117604864 K?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚3703542 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 QCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 182426056 _LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚10791134644224 TADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚10935701503310 ]LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 144749285142 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 TFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 444596224 D8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚6511186 SDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 9283702784 VGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 9283702784 YHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 145368202240 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 434363904 N?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 9900654592 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚23107 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚22834 XEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚11093147773440 TADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚11238515975680 OCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚2807644 L@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚2644294 VGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 8980994048 MFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚35 RGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚960203 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚24772608 UGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 431137949 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 [HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚11093147773440 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚455 OCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚3703542 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚212 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 Q>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚11238515975680 PDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚2644752 @8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚643 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 _step‚13 SGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚2807646 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚18746 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚18744 SGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚3722286 bQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 151657842688 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚431 K?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚2807644 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 168605696 SGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚1684091 SEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 309121024 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 444596224 G?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚431 _runtime‚ 156.160857453 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚18746 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚3 YFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚11359495192576 ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 219367424 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚2 UBDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated‚11238701751808 N@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 185776128 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 244140032 ^MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 151438475264 BDeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚2605 L=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 9679237120 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 168605696 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚23107 VKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚960658 NFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚972 ]JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚11359739332608 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 144203776 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 234590720 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 UFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 9456058368 VEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 145368202240 D>DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 XEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚10935883929366 HDeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚22834 XEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚11093147773440 TADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚11238515975680 OCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚2807644 L@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚2644294 VGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 8980994048 MFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚35 RGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚960203 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚24772608 UGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 431137949 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 [HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚11093147773440 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚455 OCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚3703542 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚212 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 Q>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚11238515975680 PDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚2644752 @8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚643 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 _step‚13 SGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚2807646 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚18746 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚18744 SGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚3722286 bQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 151657842688 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚431 K?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚2807644 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 168605696 SGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚1684091 SEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 309121024 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 444596224 G?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚431 _runtime‚ 156.160857453 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚18746 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚3 YFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚11359495192576 ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 219367424 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚2 UBDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated‚11238701751808 N@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 185776128 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 244140032 ^MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 151438475264 BDeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚2605 L=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 9679237120 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 168605696 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚23107 VKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚960658 NFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚972 ]JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚11359739332608 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 144203776 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 234590720 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 UFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 9456058368 VEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 145368202240 D>DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 XEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚10935883929366 H6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚643 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚431 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚17170432 QCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 309121024 [MDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚ 227742720 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 PEDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚964866 `ODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 152416529920 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 182426264 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚35 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚1692482 bODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚11264015929344 R?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚10990287617154 M>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 9373784469 O@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 9900654592 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚643 F6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚643 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚212 EDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 9679237120 [HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚11416432459264 [JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 145470732058 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 QBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 9283702784 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 234590720 QBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 9900654592 % _timestamp‚1.7884145751873112e+09 L>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 185789440 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚18770 J>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚2657348 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚444 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24772608 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚1692485 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚17039360 J;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 9679237120 PBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 434363904 ]JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚11148528633344 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚23107 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 444596224 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚18772 ZGDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚11148528633344 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 168619008 B6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚6543658 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚2 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚431 TCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 146093556224 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 168619008 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 144203776 \KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 152188787200 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 I=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚2821644 J>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚6562430 QEDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚2821646 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 QEDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚3740784 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 YFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚10844982272000 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚18770 I=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚3722014 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚3 VHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 165386904 WIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 444596224 E=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚990 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0÷âðµ&ð¯M trainer/global_step‚199 SEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 434363904 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 MADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚3722014 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚447 ^KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚11263991156736 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 _runtime‚ 156.837116261 ODeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 9679237120 [HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚11416432459264 [JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 145470732058 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 QBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 9283702784 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 234590720 QBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 9900654592 % _timestamp‚1.7884145751873112e+09 L>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 185789440 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚18770 J>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚2657348 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚444 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24772608 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚1692485 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚17039360 J;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 9679237120 PBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 434363904 ]JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚11148528633344 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚23107 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 444596224 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚18772 ZGDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚11148528633344 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 168619008 B6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚6543658 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚2 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚431 TCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 146093556224 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 168619008 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 144203776 \KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 152188787200 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 I=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚2821644 J>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚6562430 QEDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚2821646 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 QEDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚3740784 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 YFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚10844982272000 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚18770 I=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚3722014 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚3 VHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 165386904 WIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 444596224 E=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚990 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 MADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚2821646 VCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚11148511462912 [JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 146262175232 YFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚11148511462912 MADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚2821644 B6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚643 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚431 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚17170432 QCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 309121024 [MDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚ 227742720 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 PEDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚964866 `ODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 152416529920 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 182426264 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚35 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚1692482 bODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚11264015929344 R?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚10990287617154 M>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 9373784469 O@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 9900654592 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚643 F6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚643 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚212 EDeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚2605 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚18746 `MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚14004778688512 QBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 9900654592 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 TFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 444596224 % _timestamp‚1.7884146132981207e+09 TADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚14042995976704 VGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 9283702784 YFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚14193942694912 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 234590720 BDeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚22834 N@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 185776128 OCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚4630494 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 L=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 9679237120 @8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚643 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚212 JBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚212 ]LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 180848792550 K?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚4630494 HDeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚14042995976704 YHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 181663729664 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 UGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 431137949 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚447 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚24772608 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 _runtime‚ 194.94779687 _step‚17 HDeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 SGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚3507646 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚18744 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚1201457 |•¬'ý¦O F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 XEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚14043181752832 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚2103644 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚3 dQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚14004803461120 G?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚990 G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚2605 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚18746 `MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚14004778688512 QBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 9900654592 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 TFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 444596224 % _timestamp‚1.7884146132981207e+09 TADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚14042995976704 VGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 9283702784 YFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚14193942694912 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 234590720 BDeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚22834 N@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 185776128 OCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚4630494 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 L=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 9679237120 @8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚643 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚212 JBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚212 ]LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 180848792550 K?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚4630494 HDeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚14042995976704 YHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 181663729664 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 UGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 431137949 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚447 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚24772608 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 _runtime‚ 194.94779687 _step‚17 HDeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 SGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚3507646 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚18744 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚1201457 K?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚3507644 [HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚13861332247040 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 VGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 8980994048 YHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 180683405854 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 444596224 MFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚35 OCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚3507644 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 XIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 9456058368 JBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚431Ñqm—’þ‚(:‡ ••äÔÃǨ¨ gpu.0.gpu‚0.0 gpu.0.memory‚0, gpu.0.memoryAllocated‚12.163162231445312* gpu.0.memoryAllocatedBytes‚ 10448076800 gpu.0.temp‚35.0 gpu.0.powerWatts‚161.186' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚32.2372 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0×ÇCBÿ‚(:· ••äÔ·¿¶ª disk.sda2.out‚0 network.recv‚156582! disk.nvme0c0n1.in‚ 11.80859375 disk.rbd0.out‚0 disk.sda1.in‚0 disk.drbd2265.in‚0 disk.rbd0.in‚0 disk.drbd2265.out‚0 disk.sda3.out‚ 6.3359375) disk./.usagePercent‚16.73562647747068 disk.rbd1.in‚0 disk.rbd1.out‚ 0.306640625 disk.dm-1.in‚0 disk.drbd1716.in‚0 disk.drbd2080.in‚0 disk.dm-0.out‚ 780.732421875 disk.nvme0n1.in‚ 11.80859375 disk.sda1.out‚0 disk.drbd2266.in‚0* proc.memory.percent‚16.439127922058105 disk.sda.out‚ 6.3359375% disk./.usageGB‚249.37292861938477$ memory_percent‚84.06171798706055' proc.memory.availableMB‚ 2611.328125 cpu‚25.887138915936323 disk.dm-1.out‚ 6.3359375 proc.cpu.threads‚17 disk.sda2.in‚0 network.sent‚322240 disk.dm-0.in‚ 11.80859375 disk.drbd1716.out‚0 disk.drbd2080.out‚0# proc.memory.rssMB‚ 2693.38671875$ disk.nvme0c0n1.out‚ 780.607421875 disk.sda.in‚0 disk.sda3.in‚0" disk.nvme0n1.out‚ 780.607421875 disk.drbd2266.out‚0H›Þ¤0€+ _wandbruntime‚195  _runtime‚195Z$N³&­M J;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 9679237120 XGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 182557702656 NBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚3318315 SDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 9456058368 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 444596224 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚212 VCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚14099270809600 QCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 309121024 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 L>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 185789440 F6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚643 J>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚8189382 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 144203776 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 XIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 9456058368 F8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 VGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 9456058368 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚431 B6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚8170610 QEDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚3521646 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 F:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚8189382 J>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚3317861 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚18770 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 168619008 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 YFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚13538215628800 VCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚13916695936512 LDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚972 M>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 9373784469 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚17039360 trainer/global_step‚249 I=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚3521644 E6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚643 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 SEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 434363904 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚2112035 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 WFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 181404852562 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚431 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚431 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚18772 MADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚4667736 ]JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚13916713106944 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚454 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚451 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚212 _runtime‚ 195.638339134 S@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚14099270809600 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚23107 ^KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚14060713127936 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚643 ZGDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚13916713106944 M>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 9679237120 _step‚18^Y\h‚±M B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚18772 MADeviceStatsMonitor.on_traiÐÎê,O&n_batch_end/active.small_pool.allocated‚4667736 ]JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚13916713106944 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚454 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚451 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚212 _runtime‚ 195.638339134 S@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚14099270809600 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚23107 ^KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚14060713127936 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚643 ZGDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚13916713106944 M>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 9679237120 _step‚18 J;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 9679237120 XGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 182557702656 NBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚3318315 SDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 9456058368 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 444596224 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚212 VCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚14099270809600 QCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 309121024 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 L>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 185789440 F6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚643 J>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚8189382 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 144203776 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 XIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 9456058368 F8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 VGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 9456058368 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚431 B6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚8170610 QEDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚3521646 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 F:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚8189382 J>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚3317861 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚18770 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 168619008 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 YFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚13538215628800 VCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚13916695936512 LDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚972 M>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 9373784469 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚17039360 trainer/global_step‚249 I=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚3521644 E6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚643 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 SEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 434363904 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚2112035 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 WFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 181404852562 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚431 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚431ç†U0ƒ+ _wandbruntime‚195  _runtime‚195XJ"¶P„Ê /loss/FBt4/detection_loss‚2.5003378174343793 _runtime‚ 195.63973094 /loss/FBt2/assignment_loss‚0.430870653860225 1loss/SRqqt1/detection_loss‚1.4426638000757894 0loss/SRqqt2/detection_loss‚4.902456079658709 /loss/FRt4/assignment_loss‚6.679080842768104 /loss/FRt2/detection_loss‚2.7868787438973137 0loss/FRt1/assignment_loss‚4.4411639228780215 .loss/FRt1/detection_loss‚2.163616677970751 /loss/FRt4/detection_loss‚2.7017479035460834 1loss/SRqqt3/detection_loss‚3.0461433128810222 %loss/total_loss‚80.48540953509648 epoch‚0 /loss/FBt1/assignment_loss‚0.631777337537362 0loss/FBt3/assignment_loss‚0.3513098226001881 1loss/SRqqt4/detection_loss‚3.2619552023803133 $ _timestamp‚1.788414613990231e+09 1loss/SRqqt1/assignment_loss‚2.131670227256891 1loss/SRqqt3/assignment_loss‚3.319827538949472 2loss/SRqqt4/assignment_loss‚3.4922953406466712 1loss/FBt4/assignment_loss‚0.47751288126575675 .loss/FBt1/detection_loss‚5.618122782962045 /loss/FBt3/detection_loss‚6.7474210781435815 _step‚19 /loss/FRt2/assignment_loss‚6.626406451930171 /loss/FRt3/assignment_loss‚6.709555458378148 2loss/SRqqt2/assignment_loss‚3.6790578365325928 /loss/FRt3/detection_loss‚3.3112364177768296 .loss/FBt2/detection_loss‚3.032301403766061 trainer/global_step‚249c,ÍKT…Î 1loss/SRqqt1/assignment_loss‚2.131670227256891 1loss/SRqqt3/assignment_loss‚3.319827538949472 2loss/SRqqt4/assignment_loss‚3.4922953406466712 1loss/FBt4/assignment_loss‚0.47751288126575675 .loss/FBt1/detection_loss‚5.618122782962045 /loss/FBt3/detection_loss‚6.7474210781435815 _step‚19 /loss/FRt2/assignment_loss‚6.626406451930171 /loss/FRt3/assignment_loss‚6.709555458378148 2loss/SRqqt2/assignment_loss‚3.6790578365325928 /loss/FRt3/detection_loss‚3.3112364177768296 .loss/FBt2/detection_loss‚3.032301403766061 trainer/global_step‚249 /loss/FBt4/detection_loss‚2.5003378174343793 _runtime‚ 195.63973094 /loss/FBt2/assignment_loss‚0.430870653860225 1loss/SRqqt1/detection_loss‚1.4426638000757894 0loss/SRqqt2/detection_loss‚4.902456079658709 /loss/FRt4/assignment_loss‚6.679080842768104 /loss/FRt2/detection_loss‚2.7868787438973137 0loss/FRt1/assignment_loss‚4.4411639228780215 .loss/FRt1/detection_loss‚2.163616677970751 /loss/FRt4/detection_loss‚2.7017479035460834 1loss/SRqqt3/detection_loss‚3.0461433128810222 %loss/total_loss‚80.48540953509648 epoch‚0 /loss/FBt1/assignment_loss‚0.631777337537362 0loss/FBt3/assignment_loss‚0.3513098226001881 1loss/SRqqt4/detection_loss‚3.2619552023803133 $ _timestamp‚1.788414613990231e+09€;󑆂(:† ¤•äÔՈΨ gpu.0.gpu‚46.0 gpu.0.memory‚4, gpu.0.memoryAllocated‚12.163162231445312* gpu.0.memoryAllocatedBytes‚ 10448076800 gpu.0.temp‚36.0 gpu.0.powerWatts‚163.08' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚32.616 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0ºÝìÛA‡‚(:¶ ¤•äÔØ©« disk.sda2.in‚0 disk.sda.out‚ 6.8828125 disk.drbd1716.out‚0 disk.nvme0n1.in‚ 11.80859375# disk.nvme0c0n1.out‚ 839.79296875 disk.sda3.in‚0* proc.memory.percent‚16.439127922058105 proc.cpu.threads‚17 network.recv‚161629 disk.drbd1716.in‚0 disk.rbd0.in‚0 disk.drbd2266.in‚0$ memory_percent‚84.06088352203369 cpu‚25.873871155935053 disk.drbd2080.in‚0 disk.rbd1.in‚0 disk.rbd1.out‚ 0.306640625 disk.dm-0.out‚ 839.91796875 disk.sda3.out‚ 6.8828125% disk./.usageGB‚249.39036178588867 disk.rbd0.out‚0 disk.sda2.out‚0 disk.drbd2266.out‚0 disk.sda.in‚0 disk.dm-1.in‚0 disk.dm-1.out‚ 6.8828125) disk./.usagePercent‚16.73679643190216! disk.nvme0n1.out‚ 839.79296875 disk.dm-0.in‚ 11.80859375# proc.memory.rssMB‚ 2693.38671875 network.sent‚360247 disk.drbd2080.out‚0 disk.drbd2265.in‚0 disk.drbd2265.out‚0! disk.nvme0c0n1.in‚ 11.80859375) proc.memory.availableMB‚ 2611.46484375 disk.sda1.in‚0 disk.sda1.out‚0Fä Ï“ˆ‚(:ˆ ³•äԥ廊 gpu.0.gpu‚26.0 gpu.0.memory‚3, gpu.0.memoryAllocated‚12.163162231445312* gpu.0.memoryAllocatedBytes‚ 10448076800 gpu.0.temp‚36.0 gpu.0.powerWatts‚168.124' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚33.6248 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0-«T0‰‚(:¥ ³•äÔÙǘ« disk.drbd1716.in‚0 cpu‚25.864431829910107 proc.cpu.threads‚17 network.sent‚400109! disk.nvme0n1.out‚ 888.21484375 disk.sda1.out‚0 disk.sda2.in‚0 disk.rbd1.in‚0% disk./.usageGB‚249.39945602416992 disk.rbd1.out‚ 0.306640625) proc.memory.availableMB‚ 2611.70703125# disk.nvme0c0n1.out‚ 888.21484375 disk.dm-1.out‚7.25 disk.drbd2080.out‚0 disk.nvme0n1.in‚ 11.80859375 disk.sda2.out‚0 disk.drbd2080.in‚0 disk.drbd2266.out‚0 disk.sda.in‚0* proc.memory.percent‚16.439151763916016 network.recv‚167955 disk.rbd0.out‚0 disk.drbd2266.in‚0 disk.dm-0.in‚ 11.80859375 disk.dm-0.out‚ 888.33984375 disk.sda.out‚7.25 disk.dm-1.in‚0) disk./.usagePercent‚16.73740675386379 disk.drbd2265.out‚0! disk.nvme0c0n1.in‚ 11.80859375 disk.sda3.in‚0 disk.sda3.out‚7.25 disk.drbd1716.out‚0$ memory_percent‚84.05940532684326! proc.memory.rssMB‚ 2693.390625 disk.rbd0.in‚0 disk.sda1.in‚0 disk.drbd2265.in‚0+0Š+ _wandbruntime‚233  _runtime‚233õxäÄ‹¾ _runtime‚ 233.318584622 _step‚20 & lr-AdamW/pg1‚0.0009279849206903347 & lr-AdamW/pg2‚0.0009279849206903347 trainer/global_step‚299 % _timestamp‚1.7884146516692247e+09fuȌ % _timestamp‚1.7884146516692247e+09 _runtime‚ 233.318584622 _step‚20 & lr-AdamW/pg1‚0.0009279849206903347 & lr-AdamW/pg2‚0.0009279849206903347 trainer/global_step‚299z6î0+ _wandbruntime‚233  _runtime‚233†€I¨'Ž¢O H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚18746 L@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚3962903 K?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚4207644 D8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚9765090 L=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 9679237120 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚17170432 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚1440141 BDeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚16847475977728 NFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚972 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 D>DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 N?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 9900654592 ZKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 9456058368 UFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 9456058368 MFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚35 SEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 309121024 dQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚16801525432320 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚431 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚23107 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚22834 XEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚16847661753856 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚3 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚429 SGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚1439712 VGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 8980994048 K?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚5557446 \IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚16629533891072 _LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚16629533891072 G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚2605 TADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚16847475977728 VGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 9283702784 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚23107 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚24772608 OCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚4207646 ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 231950336 bQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 227076946944 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 234590720 QBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 9900654592 SDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 9283702784 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 % _timestamp‚1.7884146516703084e+09 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚18746 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚2605 G?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚990 ]JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚17028602379264 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚2 L@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚9783836 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚2523194 _LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚16177601357824 [HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚16177584318464 _step‚21 XEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚16629516720640 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 168605696 ]LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 216948299958 O@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 9679237120 PDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚3963335 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚432 YHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 216782913262 JBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚431 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚18744 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 TFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 444596224 HDeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚2605 TADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚16847475977728 VGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 9283702784 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚23107 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚24772608 OCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚4207646 ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 231950336 bQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 227076946944 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 234590720 QBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 9900654592 SDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 9283702784 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 % _timestamp‚1.7884146516703084e+09 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚18746 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚2605 G?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚990 ]JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚17028602379264 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚2 L@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚9783836 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚2523194 _LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚16177601357824 [HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚16177584318464 _step‚21 XEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚16629516720640 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 168605696 ]LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 216948299958 O@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 9679237120 PDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚3963335 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚432 YHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 216782913262 JBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚431 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚18744 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 TFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 444596224 HDeviceStatsMonitor.on_train_batch_start/allocation.all.current‚18746 L@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚3962903 K?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚4207644 D8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚9765090 L=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 9679237120 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚17170432 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚1440141 BDeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚16847475977728 NFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚972 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 D>DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 N?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 9900654592 ZKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 9456058368 UFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 9456058368 MFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚35 SEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 309121024 dQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚16801525432320 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚431 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚23107 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚22834 XEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚16847661753856 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚3 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚429 SGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚1439712 VGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 8980994048ö¼ñ0+ _wandbruntime‚234  _runtime‚234«÷^7³&‘­M FDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 9679237120 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚18770 ^KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚16857435099136 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 234590720 F:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚9816334 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚18772 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚2605 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 _runtime‚ 234.008429054 [HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚17085286101504 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 VCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚16903750810624 MADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚5575918 VCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚16449135771834 MADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚5594688 VCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚16684880410112 ]JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚16684897580544 YFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚16684880410112 NBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚3976512 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚643 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 TCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 218684611072 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚2531585 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 I=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚4221644 QEDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚4221646 [JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 217669746874 YFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚16231448985600 VGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 9456058368 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 444596224 % _timestamp‚1.7884146523585117e+09 SEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 434363904 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚431 FDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 WFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 218684611072 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚23107 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚1444497 [MDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚ 234034176 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 M>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 9373784469 J>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚3976079 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 144203776 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚431 R?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚16448953345570 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 444596224 L>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 185789440 bODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚16857459871744 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24772608 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 BDeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚9816334 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚643 EDeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚3 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚1444927 L=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 9900654592 MADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚4221646 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17170432 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚2531582 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚212 B6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚9797562 VHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 165386904 O@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 9900654592 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 168619008 [JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 218853230080 `ODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 227826229760 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚643 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚23107 I=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚5575918 R?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚16903565021184 XIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 9456058368 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚18770 ZGDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚16684897580544 QBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 9283702784 WDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚17085027294720 \KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 227592195584 QBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 9900654592 S@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚16903750810624 TEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 9283702784Ç£-W·&’±M R?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚16448953345570 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 444596224 L>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 185789440 bODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚16857459871744 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24772608 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 BDeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚9816334 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚643 EDeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚3 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚1444927 L=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 9900654592 MADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚4221646 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17170432 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚2531582 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚212 B6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚9797562 VHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 165386904 O@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 9900654592 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 168619008 [JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 218853230080 `ODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 227826229760 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚643 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚23107 I=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚5575918 R?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚16903565021184 XIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 9456058368 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚18770 ZGDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚16684897580544 QBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 9283702784 WDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚17085027294720 \KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 227592195584 QBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 9900654592 S@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚16903750810624 TEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 9283702784 FDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 9679237120 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚18770 ^KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚16857435099136 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 234590720 F:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚9816334 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚18772 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚2605 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 _runtime‚ 234.008429054 [HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚17085286101504 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 VCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚16903750810624 MADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚5575918 VCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚16449135771834 MADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚5594688 VCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚16684880410112 ]JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚16684897580544 YFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚16684880410112 NBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚3976512 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚643 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 TCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 218684611072 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚2531585 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 I=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚4221644 QEDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚4221646 [JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 217669746874 YFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚16231448985600 VGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 9456058368 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 444596224 % _timestamp‚1.7884146523585117e+09 SEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 434363904 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚431 FDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 WFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 218684611072 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚23107 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚1444497 [MDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚ 234034176 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 M>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 9373784469 J>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚3976079 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 144203776 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚4312ë7J0“+ _wandbruntime‚234  _runtime‚234Ž¼P”Ê /loss/FBt4/detection_loss‚1.7213702850052317 epoch‚0 /loss/FRt3/assignment_loss‚5.511118605732918 0loss/FBt4/assignment_loss‚0.3551549959986981 .loss/FRt3/detection_loss‚2.999670011922717 0loss/FBt3/assignment_loss‚0.5184664203310906 .loss/FRt4/detection_loss‚3.063124402245479 1loss/SRqqt1/detection_loss‚1.3372861382687036 trainer/global_step‚299 2loss/SRqqt2/assignment_loss‚3.6111162483692167 1loss/SRqqt4/detection_loss‚2.9114048610771857 $loss/total_loss‚75.0793938977417 0loss/FBt2/assignment_loss‚0.5547149482200937 /loss/FRt1/detection_loss‚1.8912055864455595 /loss/SRqqt3/detection_loss‚2.37338946742573 % _timestamp‚1.7884146523603559e+09 /loss/FRt4/assignment_loss‚6.624588994837519 2loss/SRqqt4/assignment_loss‚3.4326804068780716 .loss/FRt2/detection_loss‚2.924186999733384 1loss/SRqqt2/detection_loss‚3.4558210149407387 /loss/FBt3/detection_loss‚2.9290202445579023 _runtime‚ 234.009588241 /loss/FRt2/assignment_loss‚6.387058129181733 .loss/FBt1/detection_loss‚7.005228643233959 /loss/FRt1/assignment_loss‚4.702249285521781 2loss/SRqqt3/assignment_loss‚3.1612263273918764 2loss/SRqqt1/assignment_loss‚1.8470486976171323 0loss/FBt1/assignment_loss‚0.5358026379690239 .loss/FBt2/detection_loss‚5.226460544835954 _step‚23þJN÷T•Î 1loss/SRqqt4/detection_loss‚2.9114048610771857 $loss/total_loss‚75.0793938977417 0loss/FBt2/assignment_loss‚0.5547149482200937 /loss/FRt1/detection_loss‚1.8912055864455595 /loss/SRqqt3/detection_loss‚2.37338946742573 % _timestamp‚1.7884146523603559e+09 /loss/FRt4/assignment_loss‚6.624588994837519 2loss/SRqqt4/assignment_loss‚3.4326804068780716 .loss/FRt2/detection_loss‚2.924186999733384 1loss/SRqqt2/detection_loss‚3.4558210149407387 /loss/FBt3/detection_loss‚2.9290202445579023 _runtime‚ 234.009588241 /loss/FRt2/assignment_loss‚6.387058129181733 .loss/FBt1/detection_loss‚7.005228643233959 /loss/FRt1/assignment_loss‚4.702249285521781 2loss/SRqqt3/assignment_loss‚3.1612263273918764 2loss/SRqqt1/assignment_loss‚1.8470486976171323 0loss/FBt1/assignment_loss‚0.5358026379690239 .loss/FBt2/detection_loss‚5.226460544835954 _step‚23 /loss/FBt4/detection_loss‚1.7213702850052317 epoch‚0 /loss/FRt3/assignment_loss‚5.511118605732918 0loss/FBt4/assignment_loss‚0.3551549959986981 .loss/FRt3/detection_loss‚2.999670011922717 0loss/FBt3/assignment_loss‚0.5184664203310906 .loss/FRt4/detection_loss‚3.063124402245479 1loss/SRqqt1/detection_loss‚1.3372861382687036 trainer/global_step‚299 2loss/SRqqt2/assignment_loss‚3.6111162483692167°h®×‘–‚(:† •äÔá·Œ¨ gpu.0.gpu‚41.0 gpu.0.memory‚4, gpu.0.memoryAllocated‚12.163162231445312* gpu.0.memoryAllocatedBytes‚ 10448076800 gpu.0.temp‚36.0 gpu.0.powerWatts‚138.49' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚27.698 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0ÔÐŽtJ—‚(:¿ •äÔš™É« disk.drbd1716.out‚0 disk.rbd0.out‚0 disk.sda2.out‚0 disk.rbd1.in‚0) proc.memory.availableMB‚ 2611.64453125 disk.dm-1.in‚0 disk.sda1.out‚0 disk.sda2.in‚0 disk.drbd2266.out‚0 proc.cpu.threads‚17 disk.sda1.in‚0 network.sent‚406682 disk.drbd1716.in‚0 disk.drbd2080.out‚0 disk.drbd2266.in‚0 disk.drbd2265.out‚0& disk.nvme0c0n1.out‚942.92822265625 disk.sda.in‚0! proc.memory.rssMB‚ 2693.390625 network.recv‚172981 disk.drbd2265.in‚0! disk.dm-0.out‚943.05322265625 disk.sda3.out‚ 7.87109375* proc.memory.percent‚16.439151763916016 disk.nvme0n1.in‚ 11.80859375$ disk.nvme0n1.out‚942.92822265625 disk.rbd1.out‚ 0.306640625 disk.dm-0.in‚ 11.80859375 cpu‚25.85501368459974$ disk./.usageGB‚249.4389991760254 disk.dm-1.out‚ 7.87109375 disk.drbd2080.in‚0 disk.rbd0.in‚0 disk.sda.out‚ 7.87109375$ memory_percent‚84.05978679656982* disk./.usagePercent‚16.740060527963724! disk.nvme0c0n1.in‚ 11.80859375 disk.sda3.in‚0kdÀ“˜‚(:ˆ Ñ•äÔ·‰Øª gpu.0.gpu‚53.0 gpu.0.memory‚6, gpu.0.memoryAllocated‚12.163162231445312* gpu.0.memoryAllocatedBytes‚ 10448076800 gpu.0.temp‚35.0 gpu.0.powerWatts‚120.011' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚24.0022 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0ƒÄ¢C™‚(:¸ Ñ•äÔ–«§¬ disk.dm-0.in‚ 11.80859375 disk.sda.in‚0 disk.sda.out‚8.28125 disk.drbd1716.in‚0) proc.memory.availableMB‚ 2611.63671875 proc.cpu.threads‚17 disk.rbd0.in‚0* disk./.usagePercent‚16.742123692682163 disk.sda1.in‚0 disk.sda2.in‚0 disk.drbd2080.out‚0 disk.rbd1.out‚ 0.306640625 disk.drbd1716.out‚0* proc.memory.percent‚16.439151763916016 disk.drbd2266.out‚0 disk.rbd1.in‚0& disk.nvme0c0n1.out‚973.52978515625 disk.dm-1.out‚8.28125$ memory_percent‚84.05983448028564 network.sent‚458405 disk.drbd2080.in‚0! disk.dm-0.out‚973.65478515625 disk.dm-1.in‚0 disk.drbd2265.in‚0 disk.drbd2265.out‚0 disk.sda3.out‚8.28125% disk./.usageGB‚249.46974182128906 disk.rbd0.out‚0 disk.nvme0n1.in‚ 11.80859375$ disk.nvme0n1.out‚973.52978515625 disk.sda2.out‚0 network.recv‚178400 disk.sda1.out‚0 disk.drbd2266.in‚0 disk.sda3.in‚0! proc.memory.rssMB‚ 2693.390625 cpu‚25.847430311004892! disk.nvme0c0n1.in‚ 11.80859375ú$ë.žš‚(:“ à•äÔި gpu.0.gpu‚51.0 gpu.0.memory‚4, gpu.0.memoryAllocated‚12.163162231445312* gpu.0.memoryAllocatedBytes‚ 10448076800 gpu.0.temp‚35.0 gpu.0.powerWatts‚138.522' gpu.0.enforcedPowerLimitWatts‚500.0) gpu.0.powerPercent‚27.704399999999996 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0þŒ|J›‚(:¿ à•äÔÙ£²ª' proc.memory.availableMB‚ 2611.609375# proc.memory.rssMB‚ 2693.39453125$ disk./.usageGB‚249.5120620727539! disk.nvme0c0n1.in‚ 11.80859375' disk.nvme0c0n1.out‚1030.07568359375 disk.drbd2266.in‚0 disk.sda3.out‚ 8.9453125 disk.drbd2266.out‚0$ memory_percent‚84.06000137329102* proc.memory.percent‚16.439175605773926 cpu‚25.83941147705973 disk.rbd0.out‚0 network.sent‚461956 disk.rbd1.in‚0 disk.sda.in‚0 disk.sda.out‚ 8.9453125 disk.drbd2080.in‚0 disk.rbd0.in‚0 disk.sda1.in‚0 disk.sda1.out‚0 network.recv‚182655" disk.dm-0.out‚1030.20068359375 disk.sda2.out‚0 disk.drbd2265.in‚0 disk.drbd2265.out‚0 proc.cpu.threads‚17 disk.sda3.in‚0* disk./.usagePercent‚16.744963840267026 disk.dm-0.in‚ 11.80859375 disk.dm-1.in‚0 disk.dm-1.out‚ 8.9453125 disk.drbd1716.in‚0 disk.drbd2080.out‚0 disk.nvme0n1.in‚ 11.80859375% disk.nvme0n1.out‚1030.07568359375 disk.rbd1.out‚ 0.306640625 disk.drbd1716.out‚0 disk.sda2.in‚0™ =,0œ+ _wandbruntime‚272  _runtime‚272‚,âaľ & lr-AdamW/pg1‚0.0010831663455549393 & lr-AdamW/pg2‚0.0010831663455549393 trainer/global_step‚349 % _timestamp‚1.7884146904981568e+09 _runtime‚ 272.147510452 _step‚24|¶^Ȟ & lr-AdamW/pg1‚0.0010831663455549393 & lr-AdamW/pg2‚0.0010831663455549393 trainer/global_step‚349 % _timestamp‚1.7884146904981568e+09 _runtime‚ 272.147510452 _step‚246¶—0Ÿ+ _wandbruntime‚272  _runtime‚272N¥JÁ5 ¦O XIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 9456058368 ZIDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 254423390208 dQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚19598247403520 _LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚18870834714624 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 N?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 9900654592 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 XEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚19652141754880 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 VGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 8980994048 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚22834 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚18744 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚22834 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚3 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 XJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 165386696 [HDeviceStatsMoWqˆñw"nitor.on_train_batch_start/allocated_bytes.large_pool.freed‚19397701194240 NFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚979 ]JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚19862954192384 TADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚19123700095934 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 168605696 YFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚19862703760896 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 $ _timestamp‚1.788414690499124e+09 YHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 254254784512 `MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚19598222630912 TFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 444596224 UBDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated‚19652141754880 XEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚19123882521990 K?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚4907644 XEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚19397701194240 bQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 264706788864 SDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 9283702784 IDeviceStatsMonitor.on_train_batch_start/allocation.all.current‚18746 SGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚4907646 SGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚1678872 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 250431488 E=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚643 _LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚19397718364672 M@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚11410788 SEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 313315328 ^MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 264481129984 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 [HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚18870817675264 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 238785024 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 G?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚212 VEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 254254784512 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 434363904 PDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚4622073 D>DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 SDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 9900654592 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 E8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚11392042 O@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 9679237120 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚2942744 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 @8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚643 MFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚35 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚24772608 QCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 182426056 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚17170432 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 434363904 ]LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 254423390208 SGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚6503142 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 trainer/global_step‚349 _runtime‚ 272.151782453 UGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 431137949 L@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚4621613 SGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚2942741 ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 225658880 ]LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 253047807366 JBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚212 IDeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚2605 OCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚6484398 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚431 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚23107 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚23107 OCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚4907644 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚2605 O@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 9373784469 G?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚431 _step‚25 L=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 9679237120 OCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚4907646 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚18744 K?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚6484398 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 185776128 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 444596224 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚457 BDeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚19651955978752 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 168605696 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚17170432 VGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 9283702784>6(|°'¡ªO F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 N?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 9900654592 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 XEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚19652141754880 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 VGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 8980994048 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚22834 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚18744 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚22834 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚3 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 XJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 165386696 [HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚19397701194240 NFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚979 ]JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚19862954192384 TADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚19123700095934 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 168605696 YFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚19862703760896 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 $ _timestamp‚1.788414690499124e+09 YHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 254254784512 `MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚19598222630912 TFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 444596224 UBDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated‚19652141754880 XEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚19123882521990 K?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚4907644 XEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚19397701194240 bQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 264706788864 SDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 9283702784 IDeviceStatsMonitor.on_train_batch_start/allocation.all.current‚18746 SGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚4907646 SGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚1678872 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 250431488 E=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚643 _LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚19397718364672 M@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚11410788 SEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 313315328 ^MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 264481129984 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 [HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚18870817675264 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 238785024 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 G?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚212 VEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 254254784512 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 434363904 PDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚4622073 D>DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 SDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 9900654592 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 E8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚11392042 O@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 9679237120 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚2942744 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 @8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚643 MFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚35 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚24772608 QCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 182426056 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚17170432 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 434363904 ]LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 254423390208 SGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚6503142 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 trainer/global_step‚349 _runtime‚ 272.151782453 UGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 431137949 L@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚4621613 SGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚2942741 ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 225658880 ]LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 253047807366 JBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚212 IDeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚2605 OCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚6484398 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚431 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚23107 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚23107 OCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚4907644 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚2605 O@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 9373784469 G?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚431 _step‚25 L=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 9679237120 OCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚4907646 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚18744 K?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚6484398 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 185776128 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 444596224 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚457 BDeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚19651955978752 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 168605696 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚17170432 VGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 9283702784 XIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 9456058368 ZIDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 254423390208 dQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚19598247403520 _LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚18870834714624›0¢+ _wandbruntime‚272  _runtime‚272ô¶&£°M QEDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚4921646 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚2605 [MDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚ 227742720 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 K>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚11443286 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 ]JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚18924699381760 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚212 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚212 TCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 254980138496 TEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 9283702784 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚449 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚1683534 [JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 253769254282 VCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚19453064883712 G:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚11424514 E=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚997 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚643 SDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 9456058368 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚643 $ _timestamp‚1.788414691183599e+09 ZGDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚19453082054144 F6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚643 OADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 185789440 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 431137949 WIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 444596224 MADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚4921646 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚18770 M>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 9679237120 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚17170432 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚35 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 144203776 VGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 9456058368 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚431 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚2951135 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚2 I=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚6502870 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 182426264 _runtime‚ 272.833541449 R?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚19708045022208 YFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚19453064883712 WFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 253603867378 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚643 J>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚4634666 F8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 M>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 9373784469 YFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚18924682342400 \KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 265232488448 VHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 165386904 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚431 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚212 G:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚11443286 MADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚4921644 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 SEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 434363904 BDeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 185789440 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚18772 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚22834 QCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 313315328 QBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 9900654592 EDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 _step‚26 VCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚19708230811648 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚18770 `ODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 265460231168 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 VCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚19178468636042 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 444596224 I=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚4921644 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚23107 R?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚19178286209778W7ù}Ф´M >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚643 SDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 9456058368 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚643 $ _timestamp‚1.788414691183599e+09 ZGDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚19453082054144 F6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚643 OADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 185789440 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 431137949 WIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 444596224 MADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚4921646 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚18770 M>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 9679237120 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚17170432 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚35 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 144203776 VGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 9456058368 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚431 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚2951135 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚2 I=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚6502870 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 182426264 _runtime‚ 272.833541449 R?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚19708045022208 YFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚19453064883712 WFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 253603867378 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚643 J>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚4634666 F8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 M>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 9373784469 YFDeU£RžêviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚18924682342400 \KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 265232488448 VHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 165386904 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚431 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚212 G:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚11443286 MADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚4921644 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 SEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 434363904 BDeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 185789440 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚18772 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚22834 QCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 313315328 QBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 9900654592 EDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 _step‚26 VCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚19708230811648 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚18770 `ODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 265460231168 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 VCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚19178468636042 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 444596224 I=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚4921644 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚23107 R?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚19178286209778 QEDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚4921646 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚2605 [MDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚ 227742720 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 K>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚11443286 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 ]JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚18924699381760 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚212 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚212 TCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 254980138496 TEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 9283702784 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚449 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚1683534 [JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 253769254282 VCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚19453064883712 G:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚11424514 E=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚997i­Û0¥+ _wandbruntime‚272  _runtime‚272Œ&’N¦È /loss/FRt2/assignment_loss‚6.355231538732001 2loss/SRqqt1/assignment_loss‚1.8477337778353229 /loss/FRt3/detection_loss‚2.7340632861151413 1loss/SRqqt4/detection_loss‚2.7649160218688675 .loss/FBt1/detection_loss‚7.189053416252136 1loss/SRqqt3/assignment_loss‚3.160788743994957 /loss/FBt3/assignment_loss‚0.553112138564858 /loss/FRt3/assignment_loss‚5.820712068501641 1loss/SRqqt2/assignment_loss‚3.572460095087687 .loss/FRt2/detection_loss‚2.744549961800271 trainer/global_step‚349 # _timestamp‚1.78841469118532e+09 2loss/SRqqt4/assignment_loss‚3.2531671838940315 /loss/FBt2/assignment_loss‚0.366254612701883 0loss/SRqqt2/detection_loss‚2.831566641728083 epoch‚0 _step‚27 /loss/FRt4/assignment_loss‚6.255083983944308 1loss/SRqqt1/detection_loss‚1.2285265220200707 .loss/FRt4/detection_loss‚2.820119389603215 /loss/FBt4/detection_loss‚1.4630126377848136 _runtime‚ 272.834850629 /loss/FRt1/assignment_loss‚4.342279935536319 1loss/FBt1/assignment_loss‚0.19424657523631908 1loss/SRqqt3/detection_loss‚2.2644398552040723 0loss/FBt4/assignment_loss‚0.3738347247699403 .loss/FRt1/detection_loss‚1.994609891347689 /loss/FBt2/detection_loss‚3.1695028397074916 /loss/FBt3/detection_loss‚2.7299642868944116 %loss/total_loss‚70.02923012912552¨E%~R§Ì _step‚27 /loss/FRt4/assignment_loss‚6.255083983944308 1loss/SRqqt1/detection_loss‚1.2285265220200707 .loss/FRt4/detection_loss‚2.820119389603215 /loss/FBt4/detection_loss‚1.4630126377848136 _runtime‚ 272.834850629 /loss/FRt1/assignment_loss‚4.342279935536319 1loss/FBt1/assignment_loss‚0.19424657523631908 1loss/SRqqt3/detection_loss‚2.2644398552040723 0loss/FBt4/assignment_loss‚0.3738347247699403 .loss/FRt1/detection_loss‚1.994609891347689 /loss/FBt2/detection_loss‚3.1695028397074916 /loss/FBt3/detection_loss‚2.7299642868944116 %loss/total_loss‚70.02923012912552 /loss/FRt2/assignment_loss‚6.355231538732001 2loss/SRqqt1/assignment_loss‚1.8477337778353229 /loss/FRt3/detection_loss‚2.7340632861151413 1loss/SRqqt4/detection_loss‚2.7649160218688675 .loss/FBt1/detection_loss‚7.189053416252136 1loss/SRqqt3/assignment_loss‚3.160788743994957 /loss/FBt3/assignment_loss‚0.553112138564858 /loss/FRt3/assignment_loss‚5.820712068501641 1loss/SRqqt2/assignment_loss‚3.572460095087687 .loss/FRt2/detection_loss‚2.744549961800271 trainer/global_step‚349 # _timestamp‚1.78841469118532e+09 2loss/SRqqt4/assignment_loss‚3.2531671838940315 /loss/FBt2/assignment_loss‚0.366254612701883 0loss/SRqqt2/detection_loss‚2.831566641728083 epoch‚0Ö®ô“¨‚(:ˆ ï•äÔçÍ£¨ gpu.0.gpu‚45.0 gpu.0.memory‚4, gpu.0.memoryAllocated‚12.163162231445312* gpu.0.memoryAllocatedBytes‚ 10448076800 gpu.0.temp‚36.0 gpu.0.powerWatts‚157.421' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚31.4842 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0Û!ðý=©‚(:² ï•äÔ³®·ª network.sent‚468933 disk.drbd2266.in‚0 disk.sda.out‚9.25 cpu‚25.83263807918229 disk.drbd2080.in‚0 disk.drbd2080.out‚0 disk.rbd0.in‚0 disk.rbd1.out‚ 0.306640625' disk.nvme0c0n1.out‚1075.18505859375 disk.dm-0.in‚ 11.80859375 disk.sda.in‚0$ memory_percent‚84.06052589416504 disk.dm-1.in‚0 disk.drbd1716.out‚0 disk.sda1.in‚0 network.recv‚188086* disk./.usagePercent‚16.747087678771084% disk.nvme0n1.out‚1075.18505859375 disk.drbd2266.out‚0 disk.sda3.out‚9.25( proc.memory.availableMB‚ 2611.5234375 proc.cpu.threads‚17 disk.rbd1.in‚0! disk.nvme0c0n1.in‚ 11.80859375 disk.nvme0n1.in‚ 11.80859375 disk.sda1.out‚0 disk.drbd2265.in‚0% disk./.usageGB‚249.54370880126953" disk.dm-0.out‚1075.31005859375 disk.sda3.in‚0# proc.memory.rssMB‚ 2693.39453125* proc.memory.percent‚16.439175605773926 disk.dm-1.out‚9.25 disk.drbd1716.in‚0 disk.rbd0.out‚0 disk.sda2.in‚0 disk.sda2.out‚0 disk.drbd2265.out‚0[-žª‚(:“ þ•äÔÑ÷£¨ gpu.0.gpu‚42.0 gpu.0.memory‚5, gpu.0.memoryAllocated‚12.163162231445312* gpu.0.memoryAllocatedBytes‚ 10448076800 gpu.0.temp‚36.0 gpu.0.powerWatts‚168.945' gpu.0.enforcedPowerLimitWatts‚500.0) gpu.0.powerPercent‚33.788999999999994 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0v³I-G«‚(:¼ þ•äÔ¼¶£«* proc.memory.percent‚16.439175605773926" disk.dm-0.out‚1139.44384765625# proc.memory.rssMB‚ 2693.39453125 cpu‚25.82635991494006 disk.rbd1.in‚0 disk.drbd2080.in‚0 disk.sda2.out‚0 disk.drbd2266.out‚0 proc.cpu.threads‚17* disk./.usagePercent‚16.747185473649164' disk.nvme0c0n1.out‚1139.31884765625 disk.sda.in‚0 disk.dm-1.in‚0 disk.drbd2080.out‚0 disk.sda1.in‚0 disk.rbd0.in‚0$ memory_percent‚84.05947685241699( proc.memory.availableMB‚ 2611.6953125 network.recv‚194231 disk.dm-1.out‚9.828125 disk.drbd1716.in‚0 disk.drbd2265.out‚0 disk.drbd2265.in‚0 disk.nvme0n1.in‚ 11.80859375 disk.sda1.out‚0 disk.sda2.in‚0 network.sent‚519259# disk./.usageGB‚249.545166015625 disk.rbd1.out‚ 0.306640625! disk.nvme0c0n1.in‚ 11.80859375 disk.dm-0.in‚ 11.80859375% disk.nvme0n1.out‚1139.31884765625 disk.sda.out‚9.828125 disk.sda3.in‚0 disk.sda3.out‚9.828125 disk.drbd1716.out‚0 disk.rbd0.out‚0 disk.drbd2266.in‚0Oõm90¬+ _wandbruntime‚311  _runtime‚311òÊÄ­¾ & lr-AdamW/pg1‚0.0012383477704195438 & lr-AdamW/pg2‚0.0012383477704195438 trainer/global_step‚399 % _timestamp‚1.7884147293530588e+09 _runtime‚ 311.002415759 _step‚28 €ÖȮ & lr-AdamW/pg1‚0.0012383477704195438 & lr-AdamW/pg2‚0.0012383477704195438 trainer/global_step‚399 % _timestamp‚1.7884147293530588e+09 _runtime‚ 311.002415759 _step‚28àèæ‚0¯+ _wandbruntime‚311  _runtime‚311y5MU­'°§O VEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 290550311936 YHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 290550311936 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 O@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 9679237120 L=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 9679237120 UFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 9456058368 SGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚3362291 OCDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated‚7430094 _LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚22165902838272 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚3 TADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚21853032960142 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚22834 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 434363904 XEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚21853215386198 O@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 9373784469 QBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 9900654592 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚17170432 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚1918383 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 trainer/global_step‚399 TADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚22456435979776 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚24772608 _LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚21564068071424 C;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚643 OCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚5607644 `MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚22394944602112 ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 217270272 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 dQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚22394969374720 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 _runtime‚ 311.00685815 IDeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 VGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 8980994048 E=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚643 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 185776128 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 SEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 313315328 [HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚21564051032064 UGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 431137949 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 QCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 182426056 SDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 9900654592 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 SDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 9283702784 ^MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 302197081600 XJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 165386696 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚465 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 242042880 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 JBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚431 K?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚7411350 Q>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚22456435979776 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 168605696 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 JBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚212 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 168605696 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚18746 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚18744 N?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 9900654592 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚18746 ZIDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 290718917632 ]LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 290718917632 XEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚22165885667840 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 BDeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚2605 OCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚7411350 ]JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚22697383726592 XIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 9456058368 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 VGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 9283702784 ]LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 289147314774 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 444596224 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 434363904 YFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚22697141683712 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 444596224 PDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚5280677 SGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚1917921 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 144203776 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 238785024 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚212 UBDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated‚22456621755904 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚23107 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 TFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 444596224 N@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 185776128 SGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚7430094 YHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 288981928078T`k„–'±«O [HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚22165885667840 L@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚5280212 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚3362294 D>DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 VGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 8980994048 E=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚643 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 185776128 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 SEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 313315328 [HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚21564051032064 UGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 431137949 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 QCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 182426056 SDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 9900654592 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 SDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 9283702784 ^MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 302197081600 XJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 165386696 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚465 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 242042880 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 JBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚431 K?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚7411350 Q>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚22456435979776 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 168605696 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 JBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚212 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 168605696 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚18746 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚18744 N?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 9900654592 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚18746 ZIDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 290718917632 ]LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 290718917632 XEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚22165885667840 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 BDeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚2605 OCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚7411350 ]JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚22697383726592 XIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 9456058368 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 VGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 9283702784 ]LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 289147314774 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 444596224 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 434363904 YFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚22697141683712 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 444596224 PDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚5280677 SGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚1917921 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 144203776 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 238785024 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚212 UBDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated‚22456621755904 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚23107 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 TFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 444596224 N@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 185776128 SGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚7430094 YHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 288981928078 VEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 290550311936 YHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 290550311936 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 O@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 9679237120 L=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 9679237120 UFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 9456058368 SGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚3362291 OCDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated‚7430094 _LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚22165902838272 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚3 TADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚21853032960142 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚22834 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 434363904 XEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚21853215386198 O@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 9373784469 QBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 9900654592 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚17170432 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚1918383 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 trainer/global_step‚399 TADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚22456435979776 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚24772608 _LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚21564068071424 C;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚643 OCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚5607644 `MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚22394944602112 ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 217270272 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 dQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚22394969374720 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 _runtime‚ 311.00685815 IDeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 185789440 [JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 291444284928 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 ]JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚21617932738560 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 444596224 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚643 OADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 185789440 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚499 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 431137949 WIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 444596224 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 444596224 R?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚22512525023232 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24772608 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 [HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚22754074835968 [MDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚ 219354112 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚212 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚431 TCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 291275665920 YFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚22221249357312 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1000 R?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚21907619073986 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚17039360 M>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 9679237120 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚18770 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚18772 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17170432 QEDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚7448592 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 244126720 VHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 165386904 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 238785024 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 182426264 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚643 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚23107 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚643 M>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 9373784469 _runtime‚ 311.910203962 EDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 144203776 XIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 9456058368 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚35 _step‚30 ODeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚13070238 NBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚5293941 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚3 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚643 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚431 LDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚982 O@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 9900654592 MADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚7448592 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚18770 VCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚22221249357312 WFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 291275665920 F8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 [JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 289868761690 J>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚5293442 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 168619008 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 WFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 289703374786 QBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 9900654592 SDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 9456058368 FDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 9679237120 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚18770 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚18772 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17170432 QEDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚7448592 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 244126720 VHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 165386904 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 238785024 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 182426264 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚643 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚23107 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚643 M>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 9373784469 _runtime‚ 311.910203962 EDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 144203776 XIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 9456058368 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚35 _step‚30 ODeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚13070238 NBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚5293941 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚3 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚643 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚431 LDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚982 O@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 9900654592 MADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚7448592 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚18770 VCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚22221249357312 WFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 291275665920 F8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 [JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 289868761690 J>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚5293442 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 168619008 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 WFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 289703374786 QBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 9900654592 SDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 9456058368 FDeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 185789440 [JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 291444284928 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 ]JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚21617932738560 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 444596224 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚643 OADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 185789440 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚499 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 431137949 WIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 444596224 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 444596224•Œ0µ+ _wandbruntime‚311  _runtime‚311_|¸O¶É .loss/FRt1/detection_loss‚1.912211572163857 %loss/total_loss‚69.12818583262137 2loss/SRqqt1/assignment_loss‚1.9748327016830445 0loss/FBt3/assignment_loss‚0.6849610017190104 /loss/FBt1/detection_loss‚5.6991768678029375 /loss/FRt1/assignment_loss‚4.701876801932418 2loss/SRqqt3/assignment_loss‚3.2374570846557615 .loss/FRt2/detection_loss‚2.529106716611492 1loss/SRqqt1/detection_loss‚1.2315893471418393 /loss/SRqqt2/detection_loss‚2.83851712773431 0loss/SRqqt3/detection_loss‚2.238084806266584 /loss/FBt3/detection_loss‚2.9348273681056116 /loss/FBt4/detection_loss‚0.9769024108460619 /loss/FRt2/assignment_loss‚6.175406335005119 0loss/SRqqt4/assignment_loss‚3.20946217872001 epoch‚0 trainer/global_step‚399 _step‚31 0loss/FBt2/assignment_loss‚0.6289515434150633 .loss/FRt3/detection_loss‚2.648438863916162 .loss/FBt2/detection_loss‚4.646442589305696 2loss/SRqqt2/assignment_loss‚2.9855165616521298 1loss/FBt4/assignment_loss‚0.37701374271586224 /loss/FRt4/detection_loss‚2.5618713544560716 % _timestamp‚1.7884147302619827e+09 0loss/FBt1/assignment_loss‚0.5187310427427292 0loss/SRqqt4/detection_loss‚3.307042919300698 _runtime‚ 311.911217925 /loss/FRt3/assignment_loss‚6.114617112242145 /loss/FRt4/assignment_loss‚4.995147782486755°Ð»S·Í 0loss/FBt1/assignment_loss‚0.5187310427427292 0loss/SRqqt4/detection_loss‚3.307042919300698 _runtime‚ 311.911217925 /loss/FRt3/assignment_loss‚6.114617112242145 /loss/FRt4/assignment_loss‚4.995147782486755 .loss/FRt1/detection_loss‚1.912211572163857 %loss/total_loss‚69.12818583262137 2loss/SRqqt1/assignment_loss‚1.9748327016830445 0loss/FBt3/assignment_loss‚0.6849610017190104 /loss/FBt1/detection_loss‚5.6991768678029375 /loss/FRt1/assignment_loss‚4.701876801932418 2loss/SRqqt3/assignment_loss‚3.2374570846557615 .loss/FRt2/detection_loss‚2.529106716611492 1loss/SRqqt1/detection_loss‚1.2315893471418393 /loss/SRqqt2/detection_loss‚2.83851712773431 0loss/SRqqt3/detection_loss‚2.238084806266584 /loss/FBt3/detection_loss‚2.9348273681056116 /loss/FBt4/detection_loss‚0.9769024108460619 /loss/FRt2/assignment_loss‚6.175406335005119 0loss/SRqqt4/assignment_loss‚3.20946217872001 epoch‚0 trainer/global_step‚399 _step‚31 0loss/FBt2/assignment_loss‚0.6289515434150633 .loss/FRt3/detection_loss‚2.648438863916162 .loss/FBt2/detection_loss‚4.646442589305696 2loss/SRqqt2/assignment_loss‚2.9855165616521298 1loss/FBt4/assignment_loss‚0.37701374271586224 /loss/FRt4/detection_loss‚2.5618713544560716 % _timestamp‚1.7884147302619827e+09ÄH¸‚(:… –äÔ¹µù¨ gpu.0.gpu‚22.0 gpu.0.memory‚2, gpu.0.memoryAllocated‚12.163162231445312* gpu.0.memoryAllocatedBytes‚ 10448076800 gpu.0.temp‚36.0 gpu.0.powerWatts‚120.15' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚24.03 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0J¸·•F¹‚(:» –äÔ˜Õ¦ª disk.rbd1.out‚ 0.306640625# memory_percent‚84.0616226196289& proc.memory.availableMB‚ 2611.34375 disk.drbd2080.out‚0) disk./.usagePercent‚16.74846755938064% disk./.usageGB‚249.56427001953125 disk.dm-1.out‚ 10.234375 disk.nvme0n1.in‚ 11.80859375 disk.drbd2266.out‚0 disk.sda.out‚ 10.234375 disk.rbd0.in‚0 disk.sda1.out‚0 proc.cpu.threads‚17 disk.sda3.out‚ 10.234375! disk.nvme0c0n1.in‚ 11.80859375& disk.nvme0c0n1.out‚1195.6181640625! disk.dm-0.out‚1195.7431640625 disk.drbd1716.in‚0 disk.drbd1716.out‚0 disk.rbd0.out‚0 disk.sda1.in‚0 disk.sda2.in‚0 disk.sda2.out‚0 disk.drbd2266.in‚0 disk.sda.in‚0 disk.sda3.in‚0 disk.drbd2265.in‚0 disk.rbd1.in‚0 disk.dm-0.in‚ 11.80859375 disk.dm-1.in‚0 disk.drbd2080.in‚0# proc.memory.rssMB‚ 2693.39453125 network.sent‚525007 network.recv‚198276$ disk.nvme0n1.out‚1195.6181640625 disk.drbd2265.out‚0* proc.memory.percent‚16.439175605773926 cpu‚25.821659408417872üȆŒ’º‚(:‡ œ–äÔâö™¨ gpu.0.gpu‚0.0 gpu.0.memory‚0, gpu.0.memoryAllocated‚12.163162231445312* gpu.0.memoryAllocatedBytes‚ 10448076800 gpu.0.temp‚35.0 gpu.0.powerWatts‚172.441' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚34.4882 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0XÖ…KG»‚(:¼ œ–äÔ£†ªª disk.rbd0.out‚0 disk.nvme0n1.in‚ 11.80859375) disk./.usagePercent‚16.74945037230459$ disk.nvme0n1.out‚1257.5693359375 disk.sda2.out‚0 disk.drbd2266.out‚0 disk.drbd2265.in‚0* proc.memory.percent‚16.439223289489746 disk.dm-1.in‚0$ memory_percent‚84.06178951263428 network.sent‚574341& disk.nvme0c0n1.out‚1257.5693359375 disk.sda.in‚0 disk.drbd1716.out‚0 disk.dm-0.in‚ 11.80859375 disk.drbd2080.in‚0 disk.rbd1.out‚ 0.306640625! disk.nvme0c0n1.in‚ 11.80859375 disk.drbd1716.in‚0 disk.drbd2266.in‚0) proc.memory.availableMB‚ 2611.31640625% disk./.usageGB‚249.57891464233398 disk.sda3.out‚10.78125 disk.drbd2080.out‚0# proc.memory.rssMB‚ 2693.40234375 cpu‚25.816501181671576 proc.cpu.threads‚17 network.recv‚203458 disk.sda.out‚10.78125 disk.dm-1.out‚10.78125 disk.sda1.out‚0 disk.drbd2265.out‚0 disk.sda1.in‚0 disk.sda2.in‚0 disk.rbd1.in‚0! disk.dm-0.out‚1257.6943359375 disk.sda3.in‚0 disk.rbd0.in‚0v‰Üû“¼‚(:ˆ «–äÔÌöШ gpu.0.gpu‚44.0 gpu.0.memory‚4, gpu.0.memoryAllocated‚12.163162231445312* gpu.0.memoryAllocatedBytes‚ 10448076800 gpu.0.temp‚36.0 gpu.0.powerWatts‚163.801' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚32.7602 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0}kN–L½‚(:Á «–äÔ¹çâª* disk./.usagePercent‚16.750786731633323 disk.sda.in‚0 disk.sda3.in‚0 disk.dm-1.out‚ 11.3046875 disk.drbd2080.in‚0 disk.nvme0n1.in‚ 11.80859375 disk.sda1.out‚0$ memory_percent‚84.06386375427246 disk.rbd0.out‚0 disk.sda2.out‚0 disk.dm-0.in‚ 11.80859375 disk.sda.out‚ 11.3046875 disk.drbd2080.out‚0 disk.rbd0.in‚0# proc.memory.rssMB‚ 2693.40234375 proc.cpu.threads‚17 disk.sda2.in‚0! disk.dm-0.out‚1297.7294921875 cpu‚25.81102298410623 network.sent‚583404 disk.rbd1.in‚0* proc.memory.percent‚16.439223289489746 network.recv‚208224 disk.drbd2266.out‚0 disk.drbd1716.out‚0 disk.rbd1.out‚ 0.306640625& disk.nvme0c0n1.out‚1297.6044921875 disk.drbd1716.in‚0$ disk.nvme0n1.out‚1297.6044921875 disk.drbd2265.out‚0! disk.nvme0c0n1.in‚ 11.80859375 disk.sda3.out‚ 11.3046875 disk.dm-1.in‚0 disk.sda1.in‚0% disk./.usageGB‚249.59882736206055 disk.drbd2266.in‚0 disk.drbd2265.in‚0( proc.memory.availableMB‚ 2610.9765625}û=›0¾+ _wandbruntime‚349  _runtime‚349 "¾Ä¿¾ & lr-AdamW/pg2‚0.0013935291952841481 trainer/global_step‚449 % _timestamp‚1.7884147683141246e+09 _runtime‚ 349.963483982 _step‚32 & lr-AdamW/pg1‚0.0013935291952841481¹¨iœÈÀ trainer/global_step‚449 % _timestamp‚1.7884147683141246e+09 _runtime‚ 349.963483982 _step‚32 & lr-AdamW/pg1‚0.0013935291952841481 & lr-AdamW/pg2‚0.0013935291952841481 ÆÅñD0Á+ _wandbruntime‚349  _runtime‚349 5`!w¨O M@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚14664692 IDeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚2605 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 144203776 QCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 182426056 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚18744 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 UGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 431137949 _runtime‚ 349.967887922 _step‚33 \IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚24934087311872 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚17170432 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 444596224 ]LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 327014445056 YFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚25531598014976 XEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚24582548250406 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 434363904 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 D>DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 TFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 444596224 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚24772608 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 238785024 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 G?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚212 trainer/global_step‚449 VEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 326845839360 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 % _timestamp‚1.7884147683151157e+09 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 444596224 SGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚8357046 PDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚5941216 ]JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚25531844252160 O@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 9373784469 E=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚643 ZIDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 327014445056 TADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚25260915980800 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 168605696 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 434363904 L@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚5940766 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚3 `MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚25191666573312 MFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚35 N@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 185776128 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚17170432 OCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚6307644 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 SDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 9283702784 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚18744 N?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 9900654592 JBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚212 ZKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 9456058368 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 E8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚14645946 _LDe„â7‘7viceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚24934087311872 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚450 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚2159372 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚23107 SEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 313315328 TADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚24582365824350 K?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚6307644 [HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚24934070141440 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 C;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚643 SGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚3781841 XIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 9456058368 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 185776128 VGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 9283702784 [HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚24257284388864 @8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚643 OCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚8338302 _LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚24257301428224 H?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚1003 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 246237184 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 K?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚8338302 IDeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚25260915980800 L=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 9679237120 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚18746 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚22834 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚3781844 dQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚25191691345920 OCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚6307646 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚2 YHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 326845839360 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚23107 OCDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated‚8357046 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚22834 ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 221464576 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚212 VGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 8980994048 YHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 325081435486 UBDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated‚25261101756928 SGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚6307646 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚2605 ^MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 339931441664 BDeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 TFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 444596224 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚24772608 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 238785024 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 G?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚212 trainer/global_step‚449 VEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 326845839360 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 % _timestamp‚1.7884147683151157e+09 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 444596224 SGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚8357046 PDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚5941216 ]JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚25531844252160 O@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 9373784469 E=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚643 ZIDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 327014445056 TADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚25260915980800 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 168605696 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 434363904 L@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚5940766 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚3 `MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚25191666573312 MFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚35 N@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 185776128 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚17170432 OCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚6307644 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 SDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 9283702784 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚18744 N?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 9900654592 JBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚212 ZKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 9456058368 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 E8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚14645946 _LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚24934087311872 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚450 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚2159372 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚23107 SEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 313315328 TADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚24582365824350 K?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚6307644 [HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚24934070141440 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 C;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚643 SGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚3781841 XIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 9456058368 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 185776128 VGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 9283702784 [HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚24257284388864 @8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚643 OCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚8338302 _LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚24257301428224 H?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚1003 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 246237184 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 K?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚8338302 IDeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚25260915980800 L=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 9679237120 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚18746 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚22834 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚3781844 dQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚25191691345920 OCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚6307646 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚2 YHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 326845839360 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚23107 OCDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated‚8357046 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚22834 ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 221464576 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚212 VGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 8980994048 YHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 325081435486 UBDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated‚25261101756928 SGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚6307646 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚2605 ^MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 339931441664 BDeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚2605 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 144203776 QCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 182426056 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚18744 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 UGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 431137949 _runtime‚ 349.967887922 _step‚33!Êþˆ0Ä+ _wandbruntime‚350  _runtime‚350liP¸&ŲM TCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 327571193344 R?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚24636951938194 O@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 9900654592 SDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 9456058368 OADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 185789440 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚643 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚212 trainer/global_step‚449 VCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚25317190813696 QEDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚8375544 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚35 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 TEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 9283702784 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚3 WDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚25588281553920 \KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 340680541184 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 ZGDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚24989451001344 _runtime‚ 350.662714626 G:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚14697190 I=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚8356774 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 168619008 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚431 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 [MDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚ 225645568 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚23107 MADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚8375544 MADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚8356774 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚2 L>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 185789440 ]JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚24989451001344 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 bODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚25247625785344 QBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 9900654592 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 444596224 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚18770 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚444 ^KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚25247601012736 [JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 325968269098 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚643 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚643 ODeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚14697190 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚23107 QEDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚6321646 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 182426264 [JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 327739812352 S@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚25317190813696 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24772608 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚431 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚3790235 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚18772 MADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚6321646 EDeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚5953874 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚17039360 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚431 C6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚14678418 YFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚24989433830912 LDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚985 QCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 313315328 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 WFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 325802882194 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 168619008 NBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚5954321 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚2164086 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 238785024 XIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 9456058368 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚212 M>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 9679237120 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 VGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 9456058368 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚212 J;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 9679237120 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17170432 QBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 9283702784 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚17170432 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 250418176 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 431137949 FDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 9373784469 _step‚34 PBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 434363904 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚2605°ª§Ö ƶM I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚18770 SEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 434363904 FDeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚5953874 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚17039360 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚431 C6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚14678418 YFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚24989433830912 LDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚985 QCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 313315328 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 WFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 325802882194 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 168619008 NBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚5954321 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚2164086 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 238785024 XIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 9456058368 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚212 M>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 9679237120 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 VGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 9456058368 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚212 J;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 9679237120 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17170432 QBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 9283702784 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚17170432 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 250418176 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 431137949 FDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 9373784469 _step‚34 PBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 434363904 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚2605 TCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 327571193344 R?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚24636951938194 O@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 9900654592 SDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 9456058368 OADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 185789440 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚643 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚212 trainer/global_step‚449 VCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚25317190813696 QEDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚8375544 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚35 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 TEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 9283702784 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚3 WDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚25588281553920 \KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 340680541184 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 ZGDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚24989451001344 _runtime‚ 350.662714626 G:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚14697190 I=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚8356774 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 168619008 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚431 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 [MDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚ 225645568 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚23107 MADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚8375544 MADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚8356774 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚2 L>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 185789440 ]JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚24989451001344 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 bODeviceSt¿KÁ ° atsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚25247625785344 QBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 9900654592 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 444596224 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚18770 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚444 ^KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚25247601012736 [JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 325968269098 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚643 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚643 ODeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚14697190 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚23107 QEDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚6321646 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 182426264 [JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 327739812352 S@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚25317190813696 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24772608 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚431 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚3790235 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚18772 MADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚6321646 EDeviceStatsMonitor.on_train_batch_start/allocation.all.current‚18746 `MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚27988388544512 JBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚431 % _timestamp‚1.7884148073552265e+09 L=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 9679237120 QCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 182426056 _LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚26950534785024 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 TADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚28065395981824 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 434363904 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 SGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚4201391 XIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 9456058368 E=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚643 OCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚9265254 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 254625792 VGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 8980994048 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 168605696 YFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚28366121995776 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 144203776 O@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 9373784469 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚23107 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚2 G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚2605 PDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚6600446 SGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚2398593 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚22834 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚462 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 TFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 444596224 M@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚16291644 ZIDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 363309972480 OCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚7007644 ^MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 377733451264 _LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚27702271785472 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 238785024 N?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 9900654592 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 444596224 SGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚9283998 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 VEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 363141366784 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚2399052 ]JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚28366376621568 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚24772608 BDeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 ZKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 9456058368 G?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚431 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚17170432 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚22834 H?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚1003 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚459 SEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 313315328 UGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 431137949 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚212 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 185776128 [HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚27702254615040 VGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 9283702784 IDeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚28065395981824 O@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 9679237120 QBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 9900654592 @8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚643_é²'Ó¬O O@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 9373784469 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚23107 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚2 G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚2605 PDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚6600446 SGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚2398593 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚22834 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚462 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 TFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 444596224 M@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚16291644 ZIDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 363309972480 OCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚7007644 ^MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 377733451264 _LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚27702271785472 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 238785024 N?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 9900654592 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 444596224 SGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚9283998 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 VEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 363141366784 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚2399052 ]JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚28366376621568 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚24772608 BDeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 ZKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 9456058368 G?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚431 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚17170432 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚22834 H?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚1003 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚459 SEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 313315328 UGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 431137949 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚212 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 185776128 [HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚27702254615040 VGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 9283702784 IDeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚28065395981824 O@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 9679237120 QBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 9900654592 @8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚643 OCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚7007646 OCDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated‚9283998 \IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚27702271785472 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚18746 `MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚27988388544512 JBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚431 % _timestamp‚1.7884148073552265e+09 L=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 9679237120 QCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 182426056 _LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚26950534785024 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 TADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚28065395981824 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 434363904 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 SGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚4201391 XIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 9456058368 E=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚643 OCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚9265254 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 254625792 VGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 8980994048 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 168605696 YFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚28366121995776 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 144203776%„>» 0Ô+ _wandbruntime‚389  _runtime‚389ýäšœÔ Õ²M G:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚16305370 TCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 363866720768 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 168619008 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 242029568 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 238785024 S@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚28121670814720 TEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 8980994048 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 YFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚27757618304512 NBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚6613708 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚643 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 XIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 9456058368 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚643 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚643 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚18772 K>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚16324142 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 VCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚27366467228666 QBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 9900654592 SDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 9456058368 C6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚16305370 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚212 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1003 YFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚27004382412800 WFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 363866720768 WDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚28422816956416 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 431137949 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚212 % _timestamp‚1.7884148080394115e+09 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚18770 G:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚16324142 LDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚985 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚2 MADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚9302496 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚2605 `ODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 378711229440 \KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 378493972480 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚18770 OADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 185789440 [JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 364035339776 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚455 [HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚2~óÄä8423058985984 WFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 361902389602 VGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 9456058368 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚431 E6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚643 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚431 I=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚7021644 QEDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚7021646 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚2403471 QCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 313315328 ^KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚28044322983936 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚212 O@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 9900654592 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚431 I=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚9283726 L>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 185789440 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24772608 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 ]JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚27004399452160 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 L=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 9900654592 WIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 444596224 MADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚9283726 XGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 364035339776 VCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚28121670814720 ]JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚27757635474944 TEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 9283702784 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚452 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚4209785 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 182426264 _runtime‚ 389.689160064 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚22834 ZGDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚27757635474944 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚2403923 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚17039360 _step‚38 MADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚7021646 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 trainer/global_step‚499 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚23107 J>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚6613253 BDeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 444596224 SEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 434363904 QEDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚9302496 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 [JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 362067776506 VHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 165386904 VCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚27757618304512 M>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 9679237120 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 R?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚28121485025280 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 FDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 9373784469 MADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚7021644 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚35 O6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚643 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚431 I=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚7021644 QEDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚7021646 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚2403471 QCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 313315328 ^KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚28044322983936 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚212 O@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 9900654592 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚431 I=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚9283726 L>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 185789440 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24772608 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 ]JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚27004399452160 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 L=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 9900654592 WIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 444596224 MADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚9283726 XGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 364035339776 VCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚28121670814720 ]JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚27757635474944 TEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 9283702784 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚452 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚4209785 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 182426264 _runtime‚ 389.689160064 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚22834 ZGDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚27757635474944 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚2403923 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚17039360 _step‚38 MADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚7021646 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 trainer/global_step‚499 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚23107 J>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚6613253 BDeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 444596224 SEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 434363904 QEDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚9302496 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 [JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 362067776506 VHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 165386904 VCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚27757618304512 M>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 9679237120 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 R?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚28121485025280 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 FDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 9373784469 MADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚7021644 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚35 O6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚643 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 XIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 9456058368 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚643 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚643 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚18772 K>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚16324142 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 VCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚27366467228666 QBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 9900654592 SDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 9456058368 C6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚16305370 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚212 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1003 YFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚27004382412800 WFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 363866720768 WDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚28422816956416 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 431137949 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚212 % _timestamp‚1.7884148080394115e+09 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚18770 G:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚16324142 LDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚985 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚2 MADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚9302496 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚2605 `ODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 378711229440 \KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 378493972480 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚18770 OADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 185789440 [JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 364035339776 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚455&+#0¶0×+ _wandbruntime‚389  _runtime‚389ô¾G!QØË 1loss/SRqqt1/detection_loss‚1.8139572610014252 trainer/global_step‚499 .loss/FRt2/detection_loss‚2.601350064277649 0loss/FRt1/assignment_loss‚5.0655177774883455 0loss/FBt1/assignment_loss‚1.0557143432753426 1loss/FBt4/assignment_loss‚0.28252063561211666 /loss/FBt3/assignment_loss‚0.796242356300354 _step‚39 0loss/FBt2/assignment_loss‚0.8094071702045553 /loss/SRqqt2/detection_loss‚2.46765441793791 % _timestamp‚1.7884148080412078e+09 /loss/FRt2/assignment_loss‚5.576530415217082 2loss/SRqqt2/assignment_loss‚2.9310644512445156 /loss/FRt1/detection_loss‚2.1520752296561287 1loss/SRqqt3/detection_loss‚2.5398099541664125 /loss/FBt2/detection_loss‚2.8097507392658905 .loss/FBt3/detection_loss‚2.330601744353771 /loss/FRt3/assignment_loss‚5.626550853252411 2loss/SRqqt4/assignment_loss‚1.9564199976739474 /loss/FRt4/detection_loss‚1.8831492246439059 /loss/FBt4/detection_loss‚0.6682792651053104 %loss/total_loss‚59.54448796858876 .loss/FBt1/detection_loss‚3.831765753882272 epoch‚0 _runtime‚ 389.69041942 /loss/FRt4/assignment_loss‚2.631939391295115 1loss/SRqqt1/assignment_loss‚2.624760728648731 1loss/SRqqt4/detection_loss‚1.3569937435677275 1loss/SRqqt3/assignment_loss‚2.715114673284384 /loss/FRt3/detection_loss‚3.0173177772334645œ!¸UÙÏ /loss/SRqqt2/detection_loss‚2.46765441793791 % _timestamp‚1.7884148080412078e+09 /loss/FRt2/assignment_loss‚5.576530415217082 2loss/SRqqt2/assignment_loss‚2.9310644512445156 /loss/FRt1/detection_loss‚2.1520752296561287 1loss/SRqqt3/detection_loss‚2.5398099541664125 /loss/FBt2/detection_loss‚2.8097507392658905 .loss/FBt3/detection_loss‚2.330601744353771 /loss/FRt3/assignment_loss‚5.626550853252411 2loss/SRqqt4/assignment_loss‚1.9564199976739474 /loss/FRt4/detection_loss‚1.8831492246439059 /loss/FBt4/detection_loss‚0.6682792651053104 %loss/total_loss‚59.54448796858876 .loss/FBt1/detection_loss‚3.831765753882272 epoch‚0 _runtime‚ 389.69041942 /loss/FRt4/assignment_loss‚2.631939391295115 1loss/SRqqt1/assignment_loss‚2.624760728648731 1loss/SRqqt4/detection_loss‚1.3569937435677275 1loss/SRqqt3/assignment_loss‚2.715114673284384 /loss/FRt3/detection_loss‚3.0173177772334645 1loss/SRqqt1/detection_loss‚1.8139572610014252 trainer/global_step‚499 .loss/FRt2/detection_loss‚2.601350064277649 0loss/FRt1/assignment_loss‚5.0655177774883455 0loss/FBt1/assignment_loss‚1.0557143432753426 1loss/FBt4/assignment_loss‚0.28252063561211666 /loss/FBt3/assignment_loss‚0.796242356300354 _step‚39 0loss/FBt2/assignment_loss‚0.8094071702045553'3Å¿Õ‘Ú‚(:† Ø–äÔ½øĨ gpu.0.gpu‚56.0 gpu.0.memory‚5, gpu.0.memoryAllocated‚12.163162231445312* gpu.0.memoryAllocatedBytes‚ 10448076800 gpu.0.temp‚35.0 gpu.0.powerWatts‚122.68' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚24.536 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0Z†éMÛ‚(: ؖäÔ½© « disk.nvme0n1.in‚ 11.80859375 disk.sda2.in‚0 disk.sda2.out‚0 disk.drbd2265.in‚0 network.recv‚222651 disk.drbd2080.out‚0 disk.sda1.out‚0 disk.drbd2266.in‚0& disk.nvme0c0n1.out‚1466.7373046875! disk.dm-0.out‚1466.8623046875 disk.dm-1.in‚0 cpu‚25.798895631847266 disk.rbd0.in‚0 disk.dm-0.in‚ 11.80859375 disk.dm-1.out‚ 12.8671875 disk.drbd1716.in‚0) disk./.usagePercent‚16.75321905921614 disk.drbd2080.in‚0 disk.rbd0.out‚0 disk.rbd1.out‚ 0.4873046875 disk.sda1.in‚0 disk.sda.out‚ 12.8671875 disk.sda3.out‚ 12.8671875$ memory_percent‚84.06596183776855# proc.memory.rssMB‚ 2693.40234375 proc.cpu.threads‚17 disk.sda.in‚0 disk.drbd1716.out‚0$ disk.nvme0n1.out‚1466.7373046875 disk.drbd2266.out‚0! disk.nvme0c0n1.in‚ 11.80859375 disk.sda3.in‚0( proc.memory.availableMB‚ 2610.6328125* proc.memory.percent‚16.439223289489746 network.sent‚642882% disk./.usageGB‚249.63507080078125 disk.drbd2265.out‚0 disk.rbd1.in‚0Xµa³‘Ü‚(:† ç–äÔ„ý¼¨ gpu.0.gpu‚29.0 gpu.0.memory‚4, gpu.0.memoryAllocated‚12.163162231445312* gpu.0.memoryAllocatedBytes‚ 10448076800 gpu.0.temp‚35.0 gpu.0.powerWatts‚122.68' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚24.536 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0üu ûKÝ‚(:À ç–äÔú뢫* proc.memory.percent‚16.439223289489746 disk.sda.out‚ 13.328125 disk.drbd2080.out‚0) proc.memory.availableMB‚ 2610.80078125! disk.dm-0.out‚1515.8583984375 disk.nvme0n1.in‚ 11.80859375 disk.sda2.in‚0 disk.drbd2266.out‚0 disk.rbd1.in‚0 network.recv‚228944! disk.nvme0c0n1.in‚ 11.80859375& disk.nvme0c0n1.out‚1515.7333984375 disk.sda.in‚0 disk.sda3.out‚ 13.328125 disk.dm-1.out‚ 13.328125 disk.rbd0.out‚0 network.sent‚693370% disk./.usageGB‚249.66124725341797 disk.dm-1.in‚0 disk.drbd2080.in‚0 disk.rbd0.in‚0$ memory_percent‚84.06493663787842* disk./.usagePercent‚16.754975782916098 disk.dm-0.in‚ 11.80859375 disk.sda3.in‚0 disk.sda2.out‚0 disk.drbd2265.in‚0 disk.rbd1.out‚ 0.4873046875 proc.cpu.threads‚17 disk.drbd1716.in‚0 disk.drbd2265.out‚0 cpu‚25.79543488220031 disk.sda1.out‚0 disk.drbd2266.in‚0 disk.drbd1716.out‚0$ disk.nvme0n1.out‚1515.7333984375 disk.sda1.in‚0# proc.memory.rssMB‚ 2693.40234375®è/žÞ‚(:“ ö–äÔüêÀ¨ gpu.0.gpu‚25.0 gpu.0.memory‚3, gpu.0.memoryAllocated‚12.163162231445312* gpu.0.memoryAllocatedBytes‚ 10448076800 gpu.0.temp‚36.0 gpu.0.powerWatts‚170.255' gpu.0.enforcedPowerLimitWatts‚500.0) gpu.0.powerPercent‚34.050999999999995 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0\ÓJŸKß‚(:À ö–äÔ«Õá« network.recv‚234172 disk.drbd1716.in‚0 disk.rbd0.in‚0 disk.sda1.in‚0! disk.dm-0.out‚1560.9189453125$ memory_percent‚84.06498432159424& disk.nvme0c0n1.out‚1560.7939453125% disk./.usageGB‚249.68088912963867 disk.sda3.in‚0 disk.drbd1716.out‚0 disk.drbd2266.in‚0 proc.cpu.threads‚17 disk.sda3.out‚ 14.0390625 disk.sda2.out‚0 disk.rbd1.in‚0 disk.dm-0.in‚ 11.80859375$ disk.nvme0n1.out‚1560.7939453125 disk.sda.out‚ 14.0390625 proc.memory.rssMB‚ 2693.40625( disk./.usagePercent‚16.7562939657099 disk.rbd0.out‚0 disk.rbd1.out‚ 0.55029296875) proc.memory.availableMB‚ 2610.79296875 network.sent‚699999 disk.dm-1.in‚0 disk.drbd2080.in‚0 disk.drbd2080.out‚0 disk.nvme0n1.in‚ 11.80859375 disk.drbd2265.in‚0* proc.memory.percent‚16.439247131347656 disk.sda1.out‚0 disk.drbd2266.out‚0 disk.drbd2265.out‚0 disk.dm-1.out‚ 14.0390625 disk.sda2.in‚0! disk.nvme0c0n1.in‚ 11.80859375 disk.sda.in‚0 cpu‚25.792713359881624– ×¢0à+ _wandbruntime‚427  _runtime‚427Ð}ÉÄá¾ & lr-AdamW/pg2‚0.0017038920450133572 trainer/global_step‚549 % _timestamp‚1.7884148457089162e+09 _runtime‚ 427.358344978 _step‚40 & lr-AdamW/pg1‚0.0017038920450133572 6ÝÌÈâ % _timestamp‚1.7884148457089162e+09 _runtime‚ 427.358344978 _step‚40 & lr-AdamW/pg1‚0.0017038920450133572 & lr-AdamW/pg2‚0.0017038920450133572 trainer/global_step‚549(9\0ã+ _wandbruntime‚427  _runtime‚427F—Š ‹!ä¬O E8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚17899850 QBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 9900654592 OCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚7707646 ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 217270272 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 238785024 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 O@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 9373784469 ZKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 9456058368 Q>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚30869875982848 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 168605696 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚18746 H?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚1003 SGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚4620941 MFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚35 NFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚985 YFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚31200563700224 O@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 9679237120 dQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚30785135288320 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 TADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚30041031552766 _LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚29643768141824 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 _runtime‚ 427.361834394 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚23107 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 434363904 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 185776128 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚18744 [HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚29643751102464 XIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 9456058368 G?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚431 L?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚10192206 ]LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 399605499904 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚486 ]JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚31200805743104 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 trainer/global_step‚549 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚22834 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 168605696 L@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚7260340 QCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 182426056 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚2 M@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚17918596 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚483 ]LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 397445836998 N?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 9900654592 TFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 444596224 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚17170432 XEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚30041213978822 K?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚7707644 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚18744 VEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 399436894208 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚2605 PDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚7260826 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 VGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 8980994048 N@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 185776128 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 242042880 G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚2605 XEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚30870061758976 [HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚30470439088640 TGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚10210950 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚212 PCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚10192206 @8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚643 _step‚41 \IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚30470456259072 SDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 9283702784 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚22834 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 444596224 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚17170432 XEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚30470439088640 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 444596224 _LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚30470456259072 IDeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 % _timestamp‚1.7884148457100062e+09 IDeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚2605 XEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚30870061758976 [HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚30470439088640 TGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚10210950 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚212 PCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚10192206 @8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚643 _step‚41 \IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚30470456259072 SDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 9283702784 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚22834 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 444596224 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚17170432 XEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚30470439088640 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 444596224 _LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚30470456259072 IDeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 % _timestamp‚1.7884148457100062e+09 IDeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚30869875982848 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 168605696 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚18746 H?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚1003 SGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚4620941 MFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚35 NFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚985 YFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚31200563700224 O@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 9679237120 dQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚30785135288320 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 TADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚30041031552766 _LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚29643768141824 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 _runtime‚ 427.361834394 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚23107 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 434363904 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 185776128 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚18744 [HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚29643751102464 XIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 9456058368 G?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚431 L?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚10192206 ]LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 399605499904 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚486 ]JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚31200805743104 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 trainer/global_step‚549 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚22834 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 168605696 L@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚7260340)Ö»Ý0æ+ _wandbruntime‚428  _runtime‚428É{í¼&ç¶M [HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚31257500561408 [MDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚ 221451264 \KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 416209382400 BDeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 185789440 ODeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚431 I=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚7721644 ]JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚30525819948544 bODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚30841069727744 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 M>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 9373784469 XIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 9456058368 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚23107 M>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 9679237120 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24772608 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚643 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚212 VCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚30525802778112 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚431 S@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚30926150815744 OADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 185789440 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚643 J;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 9679237120 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚17170432 NBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚7273925 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚2644158 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 444596224 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 SEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 434363904 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 246223872 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 238785024 O@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 9900654592 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚212 QBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 9283702784 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚23107 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚435 J>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚7273490 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 WIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 444596224 ZGDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚30525819948544 PBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 434363904 YFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚30525802778112 K>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚17951094 QEDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚7721646 REDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚10229448 YFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚29697615769600 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚18772 C6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚17932322 MADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚7721646 WFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 400162248192 WFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 398001897010 trainer/global_step‚549 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1003 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚432 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 144203776 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚2 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚22834 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚4629335 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚4629332 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚2644590 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚17039360 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚212 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚3 VHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 165386904 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 431137949 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 444596224 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚431 J=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚10210678 MADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚7721644 VCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚30095800092874 [JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 398167283914 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 FDeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 TCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 400162248192 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 168619008 E6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚643 `ODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 416430833664 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 L=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 9900654592 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚643 _runtime‚ 428.053901579 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚18770 NADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚10210678 TEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 9283702784 [JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 400330867200 QCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 313315328 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 QBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 9900654592 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚35 NADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚10229448 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17170432 ^KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚30841044955136 VGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 9456058368 % _timestamp‚1.7884148464040701e+09 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 168619008 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2[‰›þÀ&èºM ZGDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚30525819948544 PBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 434363904 YFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚30525802778112 K>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚17951094 QEDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚7721646 REDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚10229448 YFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚29697615769600 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚18772 C6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚17932322 MADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚7721646 WFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 400162248192 WFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 398001897010 trainer/global_step‚549 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1003 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚432 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 144203776 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚2 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚22834 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚4629335 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚4629332 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚2644590 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚17039360 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚212 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚3 VHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 165386904 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 431137949 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 444596224 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚431 J=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚10210678 MADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚7721644 VCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚30095800092874 [JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 398167283914 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 FDeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 TCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 400162248192 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 168619008 E6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚643 `ODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 416430833664 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 L=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 9900654592 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚643 _runtime‚ 428.053901579 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚18770 NADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚10210678 TEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 9283702784 [JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 400330867200 QCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 313315328 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 QBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 9900654592 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚35 NADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚10229448 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17170432 ^KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚30841044955136 VGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 9456058368 % _timestamp‚1.7884148464040701e+09 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 168619008 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 [HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚31257500561408 [MDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚ 221451264 \KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 416209382400 BDeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 185789440 ODeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚431 I=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚7721644 ]JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚30525819948544 bODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚30841069727744 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 M>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 9373784469 XIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 9456058368 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚23107 M>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 9679237120 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24772608 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚643 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚212 VCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚30525802778112 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚431 S@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚30926150815744 OADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 185789440 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚643 J;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 9679237120 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚17170432 NBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚7273925 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚2644158 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 444596224 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 SEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 434363904 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 246223872 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 238785024 O@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 9900654592 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚212 QBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 9283702784 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚23107 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚435 J>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚7273490 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 WIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 444596224*Ú¤å0é+ _wandbruntime‚428  _runtime‚428ª°$„êÎ 2loss/SRqqt3/assignment_loss‚2.8778780585243586 0loss/FBt1/assignment_loss‚0.5575222829841854 .loss/FRt2/detection_loss‚2.530334231257439 1loss/SRqqt1/detection_loss‚1.7145483620140864 .loss/FBt3/detection_loss‚3.312866484417635 0loss/FBt3/assignment_loss‚0.5183069337816795 /loss/FRt4/assignment_loss‚3.025497902970795 1loss/SRqqt2/assignment_loss‚2.622864388426145 /loss/FRt1/detection_loss‚2.0715366769582033 /loss/FBt4/detection_loss‚0.6879564725550944 .loss/FRt3/detection_loss‚2.996407734354337 /loss/FBt1/detection_loss‚2.9035972307125726 .loss/FBt2/detection_loss‚2.881925520144011 %loss/total_loss‚57.71834232787435 _runtime‚ 428.055267236 0loss/FRt3/assignment_loss‚5.2841523090998335 2loss/SRqqt4/assignment_loss‚1.9223703771378056 0loss/FBt2/assignment_loss‚0.3849866773050867 1loss/SRqqt3/detection_loss‚2.4668449078287398 _step‚43 /loss/FRt1/assignment_loss‚4.970048116520047 0loss/FRt2/assignment_loss‚5.5553445299466455 1loss/SRqqt1~_Dí>/assignment_loss‚2.517205934593643 1loss/FBt4/assignment_loss‚0.19204292305560183 1loss/SRqqt4/detection_loss‚1.1119721192640788 /loss/FRt4/detection_loss‚1.8603937800871122 0loss/SRqqt2/detection_loss‚2.751738373935223 epoch‚0 trainer/global_step‚549 % _timestamp‚1.7884148464060302e+09wÓú(XëÒ 0loss/FBt3/assignment_loss‚0.5183069337816795 /loss/FRt4/assignment_loss‚3.025497902970795 1loss/SRqqt2/assignment_loss‚2.622864388426145 /loss/FRt1/detection_loss‚2.0715366769582033 /loss/FBt4/detection_loss‚0.6879564725550944 .loss/FRt3/detection_loss‚2.996407734354337 /loss/FBt1/detection_loss‚2.9035972307125726 .loss/FBt2/detection_loss‚2.881925520144011 %loss/total_loss‚57.71834232787435 _runtime‚ 428.055267236 0loss/FRt3/assignment_loss‚5.2841523090998335 2loss/SRqqt4/assignment_loss‚1.9223703771378056 0loss/FBt2/assignment_loss‚0.3849866773050867 1loss/SRqqt3/detection_loss‚2.4668449078287398 _step‚43 /loss/FRt1/assignment_loss‚4.970048116520047 0loss/FRt2/assignment_loss‚5.5553445299466455 1loss/SRqqt1/assignment_loss‚2.517205934593643 1loss/FBt4/assignment_loss‚0.19204292305560183 1loss/SRqqt4/detection_loss‚1.1119721192640788 /loss/FRt4/detection_loss‚1.8603937800871122 0loss/SRqqt2/detection_loss‚2.751738373935223 epoch‚0 trainer/global_step‚549 % _timestamp‚1.7884148464060302e+09 2loss/SRqqt3/assignment_loss‚2.8778780585243586 0loss/FBt1/assignment_loss‚0.5575222829841854 .loss/FRt2/detection_loss‚2.530334231257439 1loss/SRqqt1/detection_loss‚1.7145483620140864 .loss/FBt3/detection_loss‚3.312866484417635+“jË&“ì‚(:ˆ …—äÔ—š©¨ gpu.0.gpu‚42.0 gpu.0.memory‚4, gpu.0.memoryAllocated‚12.163162231445312* gpu.0.memoryAllocatedBytes‚ 10448076800 gpu.0.temp‚36.0 gpu.0.powerWatts‚158.946' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚31.7892 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0²ÂÎNí‚(:à …—äÔÖçñª disk.sda3.in‚0 disk.drbd1716.out‚0 disk.drbd2265.out‚0) proc.memory.availableMB‚ 2610.69140625 disk.rbd0.out‚0 disk.drbd2266.out‚0 network.recv‚238003 disk.sda.out‚ 14.41796875 disk.dm-1.in‚0$ disk.nvme0n1.out‚1599.6103515625# memory_percent‚84.0656042098999 proc.memory.rssMB‚ 2693.40625 proc.cpu.threads‚17 disk.rbd1.in‚0 disk.rbd1.out‚ 0.55029296875 disk.dm-0.in‚ 11.80859375 disk.dm-1.out‚ 14.41796875 disk.rbd0.in‚0* proc.memory.percent‚16.439247131347656& disk.nvme0c0n1.out‚1599.6103515625 disk.drbd2080.out‚0 disk.nvme0n1.in‚ 11.80859375 disk.drbd2266.in‚0 cpu‚25.789243631706817 network.sent‚705691% disk./.usageGB‚249.69685745239258 disk.sda3.out‚ 14.41796875 disk.drbd2080.in‚0 disk.sda1.in‚0 disk.sda2.out‚0 disk.drbd2265.in‚0) disk./.usagePercent‚16.75736561324823! disk.dm-0.out‚1599.7353515625 disk.drbd1716.in‚0 disk.sda1.out‚0! disk.nvme0c0n1.in‚ 11.80859375 disk.sda.in‚0 disk.sda2.in‚0ÿy¯“î‚(:ˆ ”—äÔ†ƒÁ¨ gpu.0.gpu‚53.0 gpu.0.memory‚6, gpu.0.memoryAllocated‚12.163162231445312* gpu.0.memoryAllocatedBytes‚ 10448076800 gpu.0.temp‚36.0 gpu.0.powerWatts‚174.048' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚34.8096 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0§ågIï‚(:¾ ”—äÔ‡£ìª network.sent‚754990 disk.drbd2266.out‚0 disk.drbd1716.in‚0 disk.rbd0.in‚0 disk.sda1.in‚0 disk.sda2.out‚0) proc.memory.availableMB‚ 2610.69140625 proc.memory.rssMB‚ 2693.40625* disk./.usagePercent‚16.758599313555735 disk.drbd2266.in‚0 disk.sda.in‚0 disk.sda3.in‚0 disk.sda3.out‚ 15.19921875 disk.dm-1.out‚ 15.19921875 disk.sda2.in‚0 disk.drbd2080.in‚0* proc.memory.percent‚16.439247131347656% disk./.usageGB‚249.71524047851562! disk.nvme0c0n1.in‚ 11.80859375 disk.drbd2265.out‚0 disk.sda.out‚ 15.19921875 network.recv‚243150 disk.rbd1.in‚0 disk.rbd1.out‚ 0.61328125 disk.drbd1716.out‚0 disk.drbd2080.out‚0# disk.nvme0n1.out‚1655.787109375 disk.drbd2265.in‚0 disk.dm-0.in‚ 11.80859375 disk.dm-0.out‚1655.912109375 disk.nvme0n1.in‚ 11.80859375 disk.sda1.out‚0 cpu‚25.786951070669147 proc.cpu.threads‚17% disk.nvme0c0n1.out‚1655.787109375 disk.dm-1.in‚0 disk.rbd0.out‚0# memory_percent‚84.0656042098999%…S!ð‚(:’ £—äÔ¬™œ¨ gpu.0.gpu‚50.0 gpu.0.memory‚4, gpu.0.memoryAllocated‚12.163162231445312* gpu.0.memoryAllocatedBytes‚ 10448076800 gpu.0.temp‚36.0 gpu.0.powerWatts‚156.76' gpu.0.enforcedPowerLimitWatts‚500.0) gpu.0.powerPercent‚31.351999999999997 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0í<1·Qñ‚(:Æ £—äÔáÚ–«# proc.memory.rssMB‚ 2693.41015625 disk.sda3.out‚ 15.66796875 disk.rbd0.in‚0 disk.sda1.out‚0 disk.drbd2265.out‚0 disk.rbd1.in‚0 disk.drbd2266.in‚0 disk.dm-0.in‚ 11.80859375" disk.dm-0.out‚1685.92626953125 disk.sda.out‚ 15.66796875 disk.dm-1.in‚0 disk.nvme0n1.in‚ 11.80859375% disk.nvme0n1.out‚1685.80126953125 network.sent‚761951 network.recv‚248304! disk.nvme0c0n1.in‚ 11.80859375' disk.nvme0c0n1.out‚1685.80126953125 disk.sda.in‚0 disk.drbd1716.in‚0* disk./.usagePercent‚16.759782836386528 disk.sda2.in‚0 cpu‚25.783737380804737 disk.drbd2265.in‚0 disk.rbd1.out‚ 0.61328125 disk.dm-1.out‚ 15.66796875 disk.drbd1716.out‚0* proc.memory.percent‚16.439270973205566$ disk./.usageGB‚249.7328758239746 disk.sda3.in‚0 disk.drbd2080.out‚0 disk.rbd0.out‚0 disk.sda1.in‚0 disk.sda2.out‚0 proc.cpu.threads‚17 disk.drbd2080.in‚0 disk.drbd2266.out‚0$ memory_percent‚84.06567573547363( proc.memory.availableMB‚ 2610.6796875Éû;á0ò+ _wandbruntime‚465  _runtime‚4656ç*Âó¼ & lr-AdamW/pg1‚0.0018590734698779616 & lr-AdamW/pg2‚0.0018590734698779616 trainer/global_step‚599 $ _timestamp‚1.788414884344228e+09 _runtime‚ 465.99359494 _step‚44öÙ¬ÆôÀ _step‚44 & lr-AdamW/pg1‚0.0018590734698779616 & lr-AdamW/pg2‚0.0018590734698779616 trainer/global_step‚599 $ _timestamp‚1.788414884344228e+09 _runtime‚ 465.99359494,³Í%$0õ+ _wandbruntime‚465  _runtime‚465¤Æ-²'ö¬O TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚17170432 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚18746 L=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 9679237120 L@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚7919637 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚2879567 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚18746 SDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 9283702784 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚22834 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 Q>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚33674355983872 XEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚33238623562240 PDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚7920061 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 UGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 431137949 SDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 9900654592 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚23107 O@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 9679237120 TGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚11137902 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 E=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚643 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 IDeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚22834 NFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚985 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 VGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 9283702784 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 G?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚212 G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚2605 XEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚33674541760000 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚17170432 H?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚1003 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 444596224 JBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚212 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 168605696 ]JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚34035283640832 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 C;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚643 TADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚33674355983872 TADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚32770364416974 XIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 9456058368 SGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚8407646 OCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚8407644 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚431 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 O@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 9373784469 M@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚19545548 MFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚35 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 238785024 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 168605696 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 185776128 PCDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated‚11137902 ]LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 435901027328 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚5040494 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚3 BDeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 QBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 9900654592 % _timestamp‚1.7884148843453023e+09 \IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚33238640732672 VEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 435732421632 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 434363904 YHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 435732421632 ^MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 453202819584 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 @8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚643 [HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚32336984459264 ]LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 433545344406 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 $×Q¶'÷°O F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 G?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚212 G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚2605 XEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚33674541760000 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚17170432 H?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚1003 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 444596224 JBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚212 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 168605696 ]JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚34035283640832 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 C;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚643 TADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚33674355983872 TADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚32770364416974 XIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 9456058368 SGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚8407646 OCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚8407644 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚431 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 O@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 9373784469 M@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚19545548 MFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚35 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 238785024 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 168605696 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 185776128 PCDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated‚11137902 ]LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 435901027328 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚5040494 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚3 BDeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 QBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 9900654592 % _timestamp‚1.7884148843453023e+09 \IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚33238640732672 VEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 435732421632 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 434363904 YHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 435732421632 ^MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 453202819584 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 @8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚643 [HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚32336984459264 ]LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 433545344406 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚17170432 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚18746 L=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 9679237120 L@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚7919637 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚2879567 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚18746 SDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 9283702784 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚22834 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 Q>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚33674355983872 XEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚33238623562240 PDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚7920061 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 UGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 431137949 SDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 9900654592 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚23107 O@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 9679237120 TGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚11137902 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 E=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚643 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 IDeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚22834 NFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚985 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 VGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 9283702784-ºm¼ô0ø+ _wandbruntime‚466  _runtime‚466|&/'Ïù¶M XIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 9456058368 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚454 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 R?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚33730445027328 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚18770 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚5048882 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚643 XGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 436626394624 PBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 434363904 VGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 9456058368 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚212 ZGDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚33294004422144 QEDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚8421646 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1003 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚5048885 VCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚33293987251712 [JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 434266791322 QBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 9900654592 NADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚11137630 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚22834 WDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚34091727608832 O@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 9900654592 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚18770 TEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 9283702784 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚2884238 QCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 313315328 VHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 165386904 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 `ODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 454177939456 R?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚32824950530818 M>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 9373784469 _step‚46 J;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 9679237120 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 168619008 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚431 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚23107 M>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 9679237120 REDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚11156400 \KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 453960682496 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 OADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 185789440 L>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 185789440 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚18772 K>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚19578046 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚457 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 SDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 9456058368 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 444596224 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17170432 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 242029568 bODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚33637791698944 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 B6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚643 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚212 LDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚985 L=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 9900654592 WFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 434101404418 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 168619008 ]JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocaöKíted‚33294004422144 WFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 436457775616 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 144203776 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 ]JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚32390866165760 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚2605 ^KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚33637766926336 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚212 TEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 8980994048 MADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚8421646 S@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚33730630816768 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚35 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 238785024 FDeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚7932666 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚2883784 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24772608 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 trainer/global_step‚599 % _timestamp‚1.7884148850228877e+09 VCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚33730630816768 SEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 434363904 G:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚19559274 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚17039360 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚643 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚2 [MDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚ 217256960 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚643 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚431 NADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚11156400 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚23107 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 182426264 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 C6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚19559274 I=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚8421644 EDeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 O6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚643 XGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 436626394624 PBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 434363904 VGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 9456058368 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚212 ZGDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚33294004422144 QEDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚8421646 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1003 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚5048885 VCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚33293987251712 [JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 434266791322 QBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 9900654592 NADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚11137630 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚22834 WDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚34091727608832 O@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 9900654592 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚18770 TEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 9283702784 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚2884238 QCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 313315328 VHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 165386904 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 `ODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 454177939456 R?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚32824950530818 M>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 9373784469 _step‚46 J;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 9679237120 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 168619008 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚431 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚23107 M>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 9679237120 REDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚11156400 \KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 453960682496 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 OADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 185789440 L>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 185789440 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚18772 K>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚19578046 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚457 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 SDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 9456058368 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 444596224 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17170432 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 242029568 bODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚33637791698944 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 B6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚643 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚212 LDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚985 L=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 9900654592 WFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 434101404418 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 168619008 ]JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚33294004422144 WFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 436457775616 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 144203776 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 ]JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚32390866165760 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚2605 ^KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚33637766926336 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚212 TEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 8980994048 MADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚8421646 S@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚33730630816768 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚35 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 238785024 FDeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚7932666 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚2883784 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24772608 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 trainer/global_step‚599 % _timestamp‚1.7884148850228877e+09 VCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚33730630816768 SEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 434363904 G:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚19559274 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚17039360 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚643 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚2 [MDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚ 217256960 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚643 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚431 NADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚11156400 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚23107 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 182426264 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 C6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚19559274 I=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚8421644 EDeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 O8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0.p7O0û+ _wandbruntime‚466  _runtime‚466^TAÉVüÐ 0loss/FBt1/assignment_loss‚0.2381772081583525 1loss/FBt4/assignment_loss‚0.24530517926467715 1loss/SRqqt3/assignment_loss‚2.571048711426556 2loss/SRqqt4/assignment_loss‚1.7370292370700662 0loss/FBt2/assignment_loss‚0.5655249181650909 .loss/FRt1/detection_loss‚2.323213592171669 &loss/total_loss‚56.619148388474954 /loss/FRt1/assignment_loss‚4.164550452278211 /loss/FRt3/assignment_loss‚4.591069526359683 /loss/FBt4/detection_loss‚0.7313876184149205 .loss/FRt4/detection_loss‚2.163892337303052 0loss/SRqqt1/detection_loss‚1.816444946186883 _runtime‚ 466.673898676 2loss/SRqqt1/assignment_loss‚2.3079766861530913 1loss/SRqqt2/detection_loss‚2.5046699101274665 /loss/FBt2/detection_loss‚3.0720183183165157 trainer/global_step‚599 % _timestamp‚1.7884148850246685e+09 0loss/FBt3/assignment_loss‚0.7428901733500933 .loss/FRt2/detection_loss‚2.610633441380092 /loss/FBt1/detection_loss‚3.1541591386000314 /loss/FBt3/detection_loss‚3.0348820896709667 0loss/FRt4/assignment_loss‚3.1672329471029084 2loss/SRqqt2/assignment_loss‚2.8003971847620877 1loss/SRqqt3/detection_loss‚2.5649296762421727 1loss/SRqqt4/detection_loss‚1.0296366487574928 epoch‚0 /loss/FRt2/assignment_loss‚5.539404377104744 /loss/FRt3/detection_loss‚2.9426740701081324 _step‚47¸ÞŠ¾ZýÔ /loss/FRt3/detection_loss‚2.9426740701081324 _step‚47 0loss/FBt1/assignment_loss‚0.2381772081583525 1loss/FBt4/assignment_loss‚0.24530517926467715 1loss/SRqqt3/assignment_loss‚2.571048711426556 2loss/SRqqt4/assignment_loss‚1.7370292370700662 0loss/FBt2/assignment_loss‚0.5655249181650909 .loss/FRt1/detection_loss‚2.323213592171669 &loss/total_loss‚56.619148388474954 /loss/FRt1/assignment_loss‚4.164550452278211 /loss/FRt3/assignment_loss‚4.591069526359683 /loss/FBt4/detection_loss‚0.7313876184149205 .loss/FRt4/detection_loss‚2.163892337303052 0loss/SRqqt1/detection_loss‚1.816444946186883 _runtime‚ 466.673898676 2loss/SRqqt1/assignment_loss‚2.3079766861530913 1loss/SRqqt2/detection_loss‚2.5046699101274665 /loss/FBt2/detection_loss‚3.0720183183165157 trainer/global_step‚599 % _timestamp‚1.7884148850246685e+09 0loss/FBt3/assignment_loss‚0.7428901733500933 .loss/FRt2/detection_loss‚2.610633441380092 /loss/FBt1/detection_loss‚3.1541591386000314 /loss/FBt3/detection_loss‚3.0348820896709667 0loss/FRt4/assignment_loss‚3.1672329471029084 2loss/SRqqt2/assignment_loss‚2.8003971847620877 1loss/SRqqt3/detection_loss‚2.5649296762421727 1loss/SRqqt4/detection_loss‚1.0296366487574928 epoch‚0 /loss/FRt2/assignment_loss‚5.539404377104744/¦Üh“þ‚(:ˆ ²—äÔΊͨ gpu.0.gpu‚61.0 gpu.0.memory‚3, gpu.0.memoryAllocated‚12.163162231445312* gpu.0.memoryAllocatedBytes‚ 10448076800 gpu.0.temp‚34.0 gpu.0.powerWatts‚119.619' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚23.9238 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0’ùøšMÿ‚(: ²—äÔ…ïÆ« disk.drbd2266.in‚0 disk.sda.in‚0 disk.drbd2080.out‚0 disk.rbd0.in‚0 disk.sda2.out‚0 disk.drbd2265.in‚0 disk.dm-0.in‚ 11.80859375 disk.sda.out‚ 16.18359375 disk.dm-1.out‚ 16.18359375) proc.memory.availableMB‚ 2533.78515625 network.sent‚768507 disk.sda1.out‚0 disk.sda2.in‚0 disk.sda3.out‚ 16.18359375* proc.memory.percent‚16.904473304748535 proc.cpu.threads‚17% disk./.usageGB‚249.75862884521484# disk.nvme0n1.out‚1761.123046875 disk.rbd1.out‚ 0.61328125 disk.dm-0.out‚1761.248046875# proc.memory.rssMB‚ 2769.62890625* disk./.usagePercent‚16.761511143250182 disk.rbd1.in‚0! disk.nvme0c0n1.in‚ 11.80859375 disk.sda3.in‚0$ memory_percent‚84.53500270843506 network.recv‚253212 disk.rbd0.out‚0% disk.nvme0c0n1.out‚1761.123046875 disk.dm-1.in‚0 cpu‚25.931490940833232 disk.drbd1716.in‚0 disk.nvme0n1.in‚ 11.80859375 disk.sda1.in‚0 disk.drbd2265.out‚0 disk.drbd1716.out‚0 disk.drbd2266.out‚0 disk.drbd2080.in‚0 ­¾Qœ€‚(:‘ Á—äÔÉ¢ ¨ gpu.0.gpu‚6.0 gpu.0.memory‚0, gpu.0.memoryAllocated‚12.163162231445312* gpu.0.memoryAllocatedBytes‚ 10448076800 gpu.0.temp‚35.0 gpu.0.powerWatts‚137.83' gpu.0.enforcedPowerLimitWatts‚500.0) gpu.0.powerPercent‚27.566000000000003 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0Q–9üM‚(: Á—äÔ¤–ÿª% disk./.usageGB‚249.76581192016602 disk.sda1.out‚0 disk.sda3.out‚ 16.8828125) proc.memory.availableMB‚ 2536.82421875 network.sent‚817878 disk.drbd2266.in‚0 disk.drbd2265.out‚0 disk.drbd1716.out‚0 network.recv‚258289 disk.rbd0.in‚0 disk.sda1.in‚0 disk.drbd2266.out‚0 disk.rbd1.in‚0 disk.sda.in‚0 disk.dm-1.in‚0 disk.drbd2080.in‚0 disk.nvme0n1.in‚ 11.80859375 disk.drbd1716.in‚0! proc.memory.rssMB‚ 2769.796875 proc.cpu.threads‚17* disk./.usagePercent‚16.761993205437157 disk.sda2.in‚0 disk.sda2.out‚0! disk.nvme0c0n1.in‚ 11.80859375 disk.sda.out‚ 16.8828125 disk.dm-1.out‚ 16.8828125* proc.memory.percent‚16.905498504638672 disk.drbd2265.in‚0" disk.dm-0.out‚1806.61865234375 disk.sda3.in‚0 disk.drbd2080.out‚0 cpu‚26.11415075625519% disk.nvme0n1.out‚1806.49365234375' disk.nvme0c0n1.out‚1806.49365234375 disk.dm-0.in‚ 11.80859375$ memory_percent‚84.51645374298096 disk.rbd0.out‚0 disk.rbd1.out‚ 0.69921875¡hŒJ‚bE "Purity/*FRt3SRqqt0FBt/event_purity"trainer/global_step2:JPºHƒbC Purity/0FRt2SRqqt1FBt/FBt_purity"trainer/global_step2:JPƒ&$J„bE "Purity/1FRt*SRqqt*FBt/SRqqt_purity"trainer/global_step2:JPÑÆMH…bC Purity/1FRt1SRqqt*FBt/FRt_purity"trainer/global_step2:JP}…„?N†bI &Purity/3FRt2SRqqt*FBt/event_proportion"trainer/global_step2:JPú‚@‡b; particle/accuracy_3_of_3"trainer/global_step2:JP„=!‡NˆbI &Purity/*FRt0SRqqt1FBt/event_proportion"trainer/global_step2:JP'3ÕrN‰bI &Purity/0FRt1SRqqt4FBt/event_proportion"trainer/global_step2:JP¹Ú vNŠbI &Purity/1FRt1SRqqt*FBt/event_proportion"trainer/global_step2:JPåóDH‹bC Purity/2FRt1SRqqt*FBt/FRt_purity"trainer/global_step2:JPä66ŒJŒbE "Purity/3FRt1SRqqt1FBt/event_purity"trainer/global_step2:JP•œuGJbE "Purity/0FRt3SRqqt*FBt/event_purity"trainer/global_step2:JPF»¹\HŽbC Purity/1FRt*SRqqt4FBt/FRt_purity"trainer/global_step2:JP–”€JbE "Purity/0FRt*SRqqt*FBt/event_purity"trainer/global_step2:JPRZ|‚JbE "Purity/0FRt*SRqqt0FBt/event_purity"trainer/global_step2:JPB¸X•N‘bI &Purity/1FRt*SRqqt0FBt/event_proportion"trainer/global_step2:JP W8H’bC Purity/2FRt0SRqqt1FBt/FBt_purity"trainer/global_step2:JPÑ+X©H“bC Purity/4FRt0SRqqt0FBt/FRt_purity"trainer/global_step2:JP,íõÂH”bC Purity/*FRt*SRqqt1FBt/FRt_purity"trainer/global_step2:JPJ -ãH•bC Purity/1FRt*SRqqt3FBt/FBt_purity"trainer/global_step2:JP×XÚüH–bC Purity/2FRt*SRqqt*FBt/FRt_purity"trainer/global_step2:JP&uÁÀ<—b7 particle/sensitivity"trainer/global_step2:JP3Ž¦z@˜b; particle/accuracy_7_of_7"trainer/global_step2:JP]t[ÕJ™bE "Purity/2FRt2SRqqt2FBt/event_purity"trainer/global_step2:JP¦¿ÂøJšbE "Purity/1FRt*SRqqt3FBt/event_purity"trainer/global_step2:JP!½ ÃN›bI &Purity/1FRt0SRqqt3FBt/event_proportion"trainer/global_step2:JP§OHœbC Purity/1FRt1SRqqt0FBt/FRt_purity"trainer/global_step2:JPÚÚÃNbI &Purity/2FRt*SRqqt*FBt/event_proportion"trainer/global_step2:JP.‘©@žb; particle/accuracy_2_of_3"trainer/global_step2:JPıÒJŸbE "Purity/*FRt*SRqqt3FBt/event_purity"trainer/global_step2:JPìc~=N bI &Purity/2FRt*SRqqt1FBt/event_proportion"trainer/global_step2:JP‚hu*H¡bC Purity/2FRt1SRqqt0FBt/FRt_purity"trainer/global_step2:JP#¦•N¢bI &Purity/2FRt1SRqqt3FBt/event_proportion"trainer/global_step2:JPF† J£bE "Purity/2FRt3SRqqt0FBt/SRqqt_purity"trainer/global_step2:JPßâ¬KH¤bC Purity/3FRt0SRqqt1FBt/FBt_purity"trainer/global_step2:JPp°r­J¥bE "Purity/3FRt1SRqqt*FBt/SRqqt_purity"trainer/global_step2:JPŒnú£N¦bI &Purity/3FRt2SRqqt3FBt/event_proportion"trainer/global_step2:JP¥yˆ/N§bI &Purity/3FRt0SRqqt1FBt/event_proportion"trainer/global_step2:JPa¸nçJ¨bE "Purity/0FRt1SRqqt1FBt/SRqqt_purity"trainer/global_step2:JPÕÉJ©bE "Purity/1FRt2SRqqt0FBt/event_purity"trainer/global_step2:JPÚ@¾NªbI &Purity/1FRt4SRqqt1FBt/event_proportion"trainer/global_step2:JP ^]ÜJ«bE "Purity/2FRt*SRqqt1FBt/event_purity"trainer/global_step2:JPý–ÈwH¬bC Purity/2FRt3SRqqt1FBt/FRt_purity"trainer/global_step2:JPX÷ÿÞH­bC Purity/3FRt4SRqqt0FBt/FRt_purity"trainer/global_step2:JPï®.N®bI &Purity/*FRt1SRqqt1FBt/event_proportion"trainer/global_step2:JPyóJ¯bE "Purity/0FRt*SRqqt*FBt/SRqqt_purity"trainer/global_step2:JPVÎÓH°bC Purity/1FRt0SRqqt*FBt/FBt_purity"trainer/global_step2:JP½UÇ9N±bI &Purity/4FRt2SRqqt2FBt/event_proportion"trainer/global_step2:JPŽÐN²bI &Purity/2FRt0SRqqt4FBt/event_proportion"trainer/global_step2:JP€ ÞÔH³bC Purity/3FRt*SRqqt0FBt/FRt_purity"trainer/global_step2:JPj–uXJ´bE "Purity/3FRt1SRqqt*FBt/event_purity"trainer/global_step2:JPå¦_¡;µb6 jet/accuracy_3_of_5"trainer/global_step2:JPlÊ,o@¶b; particle/accuracy_4_of_7"trainer/global_step2:JPúÿ¸®J·bE "Purity/*FRt1SRqqt4FBt/SRqqt_purity"trainer/global_step2:JPIͪ™H¸bC Purity/0FRt1SRqqt3FBt/FBt_purity"trainer/global_step2:JP--*˜N¹bI &Purity/3FRt2SRqqt0FBt/event_proportion"trainer/global_step2:JP¦z™WJºbE "Purity/1FRt*SRqqt0FBt/event_purity"trainer/global_step2:JPM„GôH»bC Purity/*FRt0SRqqt3FBt/FBt_purity"trainer/global_step2:JPÜ\ N¼bI &Purity/4FRt*SRqqt1FBt/event_proportion"trainer/global_step2:JPeM·¯;½b6 jet/accuracy_5_of_7"trainer/global_step2:JPÞ°¯H¾bC Purity/2FRt0SRqqt3FBt/FRt_purity"trainer/global_step2:JPe¯¦H¿bC Purity/*FRt0SRqqt0FBt/FRt_purity"trainer/global_step2:JP=NÀbI &Purity/1FRt4SRqqt*FBt/event_proportion"trainer/global_step2:JP…B.“NÁbI &Purity/2FRt4SRqqt3FBt/event_proportion"trainer/global_step2:JPâ.„;Âb6 jet/accuracy_2_of_3"trainer/global_step2:JP$¡‚JÃbE "Purity/2FRt4SRqqt*FBt/SRqqt_purity"trainer/global_step2:JP8’šÐHÄbC Purity/*FRt2SRqqt2FBt/FRt_purity"trainer/global_step2:JPçÈÔJÅbE "Purity/0FRt1SRqqt3FBt/event_purity"trainer/global_step2:JP‹µ%bJÆbE "Purity/1FRt3SRqqt2FBt/event_purity"trainer/global_step2:JP‚J´ýJÇbE "Purity/0FRt0SRqqt3FBt/event_purity"trainer/global_step2:JPÝÍÁ·NÈbI &Purity/0FRt0SRqqt4FBt/event_proportion"trainer/global_step2:JPúQÎzHÉbC Purity/1FRt3SRqqt*FBt/FRt_purity"trainer/global_step2:JP‚·dHÊbC Purity/2FRt0SRqqt*FBt/FBt_purity"trainer/global_step2:JP‘ iéNËbI &Purity/4FRt3SRqqt*FBt/event_proportion"trainer/global_step2:JPä ù@;Ìb6 jet/accuracy_1_of_1"trainer/global_step2:JPúu[uJÍbE "Purity/2FRt4SRqqt0FBt/event_purity"trainer/global_step2:JP¨ÕËHÎbC Purity/*FRt2SRqqt2FBt/FBt_purity"trainer/global_step2:JPˆ$ð`JÏbE "Purity/1FRt0SRqqt2FBt/event_purity"trainer/global_step2:JP«ÞHÐbC Purity/1FRt2SRqqt*FBt/FRt_purity"trainer/global_step2:JPør½JÑbE "Purity/2FRt*SRqqt2FBt/SRqqt_purity"trainer/global_step2:JPÔÍô˜HÒbC Purity/2FRt2SRqqt0FBt/FRt_purity"trainer/global_step2:JPö^8HÓbC Purity/3FRt0SRqqt*FBt/FBt_purity"trainer/global_step2:JPS?¦JÔbE "Purity/*FRt*SRqqt4FBt/event_purity"trainer/global_step2:JP}ÐhGJÕbE "Purity/3FRt3SRqqt*FBt/event_purity"trainer/global_step2:JPcá+ýJÖbE "Purity/3FRt3SRqqt0FBt/SRqqt_purity"trainer/global_step2:JP~ÒiˆN×bI &Purity/0FRt0SRqqt0FBt/event_proportion"trainer/global_step2:JPýè] HØbC Purity/1FRt0SRqqt3FBt/FRt_purity"trainer/global_step2:JPsóNÙbI &Purity/2FRt0SRqqt2FBt/event_proportion"trainer/global_step2:JPÛðK;Úb6 jet/accuracy_4_of_4"trainer/global_step2:JPW[ÈÍJÛbE "Purity/4FRt2SRqqt*FBt/event_purity"trainer/global_step2:JPl<‘\JÜbE "Purity/*FRt*SRqqt1FBt/SRqqt_purity"trainer/global_step2:JPœ>™àNÝbI &Purity/0FRt4SRqqt*FBt/event_proportion"trainer/global_step2:JP˜”ÓHÞbC Purity/1FRt0SRqqt1FBt/FBt_purity"trainer/global_step2:JP47œ;ßb6 jet/accuracy_4_of_7"trainer/global_step2:JP9XõàNàbI &Purity/*FRt4SRqqt4FBt/event_proportion"trainer/global_step2:JPøb;âNábI &Purity/0FRt*SRqqt*FBt/event_proportion"trainer/global_step2:JPCpNJâbE "Purity/0FRt*SRqqt3FBt/event_purity"trainer/global_step2:JP-GDNãbI &Purity/1FRt3SRqqt0FBt/event_proportion"trainer/global_step2:JP­nÔHäbC Purity/3FRt1SRqqt1FBt/FRt_purity"trainer/global_step2:JP,8YJåbE "Purity/2FRt4SRqqt*FBt/event_purity"trainer/global_step2:JPÌzÛFHæbC Purity/1FRt1SRqqt3FBt/FBt_purity"trainer/global_step2:JPak þHçbC Purity/0FRt*SRqqt2FBt/FBt_purity"trainer/global_step2:JP˜½NèbI &Purity/0FRt3SRqqt3FBt/event_proportion"trainer/global_step2:JP’ðã HébC Purity/1FRt*SRqqt0FBt/FRt_purity"trainer/global_step2:JPpôeJêbE "Purity/1FRt1SRqqt2FBt/SRqqt_purity"trainer/global_step2:JPå*/9NëbI &Purity/4FRt1SRqqt0FBt/event_proportion"trainer/global_step2:JPõÿJìbE "Purity/*FRt4SRqqt0FBt/SRqqt_purity"trainer/global_step2:JPðçºWHíbC Purity/3FRt3SRqqt0FBt/FRt_purity"trainer/global_step2:JP¥èe©JîbE "Purity/4FRt2SRqqt*FBt/SRqqt_purity"trainer/global_step2:JP—‰ìNïbI &Purity/*FRt0SRqqt4FBt/event_proportion"trainer/global_step2:JPc)dNðbI &Purity/*FRt1SRqqt4FBt/event_proportion"trainer/global_step2:JP§¬\JñbE "Purity/0FRt*SRqqt1FBt/SRqqt_purity"trainer/global_step2:JP N¹NòbI &Purity/0FRt1SRqqt1FBt/event_proportion"trainer/global_step2:JPÉyrNóbI &Purity/1FRt2SRqqt0FBt/event_proportion"trainer/global_step2:JP'‚ £JôbE "Purity/2FRt0SRqqt0FBt/event_purity"trainer/global_step2:JP;.˜¥NõbI &Purity/3FRt*SRqqt0FBt/event_proportion"trainer/global_step2:JPÁUÃJöbE "Purity/3FRt3SRqqt*FBt/SRqqt_purity"trainer/global_step2:JP‚ÞÇN÷bI &Purity/2FRt0SRqqt0FBt/event_proportion"trainer/global_step2:JPHT%NøbI &Purity/*FRt2SRqqt1FBt/event_proportion"trainer/global_step2:JP™M¶iNùbI &Purity/1FRt*SRqqt1FBt/event_proportion"trainer/global_step2:JPh]rÄNúbI &Purity/2FRt1SRqqt0FBt/event_proportion"trainer/global_step2:JP&Â~UJûbE "Purity/3FRt2SRqqt0FBt/SRqqt_purity"trainer/global_step2:JPGM;NübI &Purity/4FRt2SRqqt4FBt/event_proportion"trainer/global_step2:JPÖtRNýbI &Purity/*FRt2SRqqt4FBt/event_proportion"trainer/global_step2:JPÂEÿNþbI &Purity/0FRt3SRqqt1FBt/event_proportion"trainer/global_step2:JPp”rUHÿbC Purity/1FRt*SRqqt*FBt/FRt_purity"trainer/global_step2:JPhj`J€bE "Purity/2FRt*SRqqt0FBt/SRqqt_purity"trainer/global_step2:JPY¢uNbI &Purity/2FRt2SRqqt1FBt/event_proportion"trainer/global_step2:JP€z(H‚bC Purity/*FRt0SRqqt4FBt/FBt_purity"trainer/global_step2:JP-=òHƒbC Purity/2FRt*SRqqt3FBt/FBt_purity"trainer/global_step2:JPÒ{\XH„bC Purity/3FRt3SRqqt*FBt/FRt_purity"trainer/global_step2:JP‡”ç9N…bI &Purity/0FRt0SRqqt1FBt/event_proportion"trainer/global_step2:JP ®?iN†bI &Purity/0FRt1SRqqt3FBt/event_proportion"trainer/global_step2:JPïþâN‡bI &Purity/3FRt1SRqqt3FBt/event_proportion"trainer/global_step2:JPËÖsÀëJÿbE "Purity/3FRt2SRqqt*FBt/event_purity"trainer/global_step2:JP¬°ñ˜J€bE "Purity/3FRt1SRqqt2FBt/SRqqt_purity"trainer/global_step2:JPì$Ú’HbC Purity/*FRt1SRqqt0FBt/FRt_purity"trainer/global_step2:JPý>ˆ H‚bC Purity/*FRt2SRqqt1FBt/FBt_purity"trainer/global_step2:JP'•ÿNƒbI &Purity/*FRt4SRqqt2FBt/event_proportion"trainer/global_step2:JPúÒü.N„bI &Purity/0FRt4SRqqt2FBt/event_proportion"trainer/global_step2:JPüôo•N…bI &Purity/4FRt*SRqqt0FBt/event_proportion"trainer/global_step2:JP£ÀŒ H†bC Purity/*FRt*SRqqt4FBt/FBt_purity"trainer/global_step2:JPIŽ H‡bC Purity/3FRt2SRqqt1FBt/FBt_purity"trainer/global_step2:JPþf>#JˆbE "Purity/1FRt2SRqqt2FBt/SRqqt_purity"trainer/global_step2:JPĤjJ‰bE "Purity/*FRt0SRqqt3FBt/event_purity"trainer/global_step2:JPœÊÊKJŠbE "Purity/0FRt0SRqqt0FBt/event_purity"trainer/global_step2:JPìäòJ‹bE "Purity/1FRt0SRqqt1FBt/event_purity"trainer/global_step2:JPùÔJžHŒbC Purity/2FRt3SRqqt*FBt/FRt_purity"trainer/global_step2:JP¸“GNbI &Purity/4FRt1SRqqt*FBt/event_proportion"trainer/global_step2:JP#Y‘áJŽbE "Purity/*FRt4SRqqt0FBt/event_purity"trainer/global_step2:JPŠgoiNbI &Purity/*FRt0SRqqt0FBt/event_proportion"trainer/global_step2:JP~P‚šNbI &Purity/*FRt1SRqqt0FBt/event_proportion"trainer/global_step2:JPÕÁóN‘bI &Purity/0FRt4SRqqt4FBt/event_proportion"trainer/global_step2:JP§RH’bC Purity/1FRt0SRqqt2FBt/FBt_purity"trainer/global_step2:JP´pMÀN“bI &Purity/4FRt*SRqqt*FBt/event_proportion"trainer/global_step2:JPSÀƒy@”b; particle/accuracy_3_of_6"trainer/global_step2:JPâ,yéN•bI &Purity/*FRt1SRqqt2FBt/event_proportion"trainer/global_step2:JPR ßN–bI &Purity/0FRt4SRqqt3FBt/event_proportion"trainer/global_step2:JPÂXH—bC Purity/1FRt0SRqqt2FBt/FRt_purity"trainer/global_step2:JPUî\N˜bI &Purity/2FRt4SRqqt*FBt/event_proportion"trainer/global_step2:JP`2N™bI &Purity/3FRt0SRqqt4FBt/event_proportion"trainer/global_step2:JP_¿ñJšbE "Purity/3FRt2SRqqt*FBt/SRqqt_purity"trainer/global_step2:JP¥á_N›bI &Purity/3FRt4SRqqt1FBt/event_proportion"trainer/global_step2:JPl&¹,JœbE "Purity/2FRt4SRqqt0FBt/SRqqt_purity"trainer/global_step2:JP²G>HbC Purity/0FRt*SRqqt1FBt/FBt_purity"trainer/global_step2:JPð‘g“JžbE "Purity/1FRt0SRqqt*FBt/event_purity"trainer/global_step2:JP.X®àJŸbE "Purity/2FRt0SRqqt*FBt/event_purity"trainer/global_step2:JPNÅFöJ bE "Purity/3FRt3SRqqt0FBt/event_purity"trainer/global_step2:JPèÖ¼áH¡bC Purity/*FRt*SRqqt*FBt/FRt_purity"trainer/global_step2:JPû½ªH¢bC Purity/2FRt*SRqqt1FBt/FRt_purity"trainer/global_step2:JPÄĆXN£bI &Purity/3FRt0SRqqt0FBt/event_proportion"trainer/global_step2:JP y^J¤bE "Purity/3FRt1SRqqt1FBt/SRqqt_purity"trainer/global_step2:JPчcN¥bI &Purity/4FRt1SRqqt2FBt/event_proportion"trainer/global_step2:JP|¡€J¦bE "Purity/*FRt2SRqqt*FBt/event_purity"trainer/global_step2:JPOT*N§bI &Purity/1FRt0SRqqt4FBt/event_proportion"trainer/global_step2:JPÐÒÇH¨bC Purity/1FRt1SRqqt1FBt/FRt_purity"trainer/global_step2:JP4WëzN©bI &Purity/2FRt2SRqqt3FBt/event_proportion"trainer/global_step2:JPôyHwHªbC Purity/2FRt1SRqqt1FBt/FRt_purity"trainer/global_step2:JP¨˜JN«bI &Purity/*FRt2SRqqt0FBt/event_proportion"trainer/global_step2:JPù({zJ¬bE "Purity/*FRt3SRqqt*FBt/event_purity"trainer/global_step2:JPo€A N­bI &Purity/0FRt2SRqqt0FBt/event_proportion"trainer/global_step2:JP¤“Û‰N®bI &Purity/4FRt0SRqqt2FBt/event_proportion"trainer/global_step2:JPô|CJJ¯bE "Purity/*FRt*SRqqt*FBt/event_purity"trainer/global_step2:JPQ ÕH°bC Purity/*FRt*SRqqt0FBt/FRt_purity"trainer/global_step2:JPY [J±bE "Purity/3FRt0SRqqt0FBt/event_purity"trainer/global_step2:JPo¨i3N²bI &Purity/3FRt0SRqqt3FBt/event_proportion"trainer/global_step2:JPni‹9³b4 particle/accuracy"trainer/global_step2:JPg!~ûH´bC Purity/*FRt1SRqqt4FBt/FBt_purity"trainer/global_step2:JPûF!åNµbI &Purity/0FRt*SRqqt2FBt/event_proportion"trainer/global_step2:JPX^ìJ¶bE "Purity/0FRt1SRqqt1FBt/event_purity"trainer/global_step2:JP“„ãJ·bE "Purity/2FRt*SRqqt1FBt/SRqqt_purity"trainer/global_step2:JPö1ã‰H¸bC Purity/3FRt*SRqqt1FBt/FRt_purity"trainer/global_step2:JPý>J¹bE "Purity/3FRt0SRqqt*FBt/event_purity"trainer/global_step2:JP_Dä%HºbC Purity/0FRt4SRqqt*FBt/FBt_purity"trainer/global_step2:JPyå3r@»b; particle/accuracy_1_of_6"trainer/global_step2:JPÿŽß@¼b; particle/accuracy_5_of_7"trainer/global_step2:JPϵ¬ƒH½bC Purity/*FRt1SRqqt*FBt/FRt_purity"trainer/global_step2:JPgÿÌQJ¾bE "Purity/*FRt2SRqqt*FBt/SRqqt_purity"trainer/global_step2:JP­E¿îJ¿bE "Purity/3FRt2SRqqt0FBt/event_purity"trainer/global_step2:JPEnó;Àb6 jet/accuracy_1_of_3"trainer/global_step2:JPHª‡@Áb; particle/accuracy_5_of_5"trainer/global_step2:JPŒXyM;Âb6 jet/accuracy_3_of_6"trainer/global_step2:JP0J:]JÃbE "Purity/*FRt*SRqqt*FBt/SRqqt_purity"trainer/global_step2:JP³…[JÄbE "Purity/*FRt0SRqqt0FBt/event_purity"trainer/global_step2:JPRΫJÅbE "Purity/1FRt*SRqqt2FBt/SRqqt_purity"trainer/global_step2:JPÏIAJÆbE "Purity/2FRt*SRqqt0FBt/event_purity"trainer/global_step2:JPtmVáHÇbC Purity/2FRt3SRqqt0FBt/FRt_purity"trainer/global_step2:JPAÂÍtNÈbI &Purity/3FRt2SRqqt4FBt/event_proportion"trainer/global_step2:JPt1ækJÉbE "Purity/*FRt0SRqqt4FBt/event_purity"trainer/global_step2:JPYMŠJÊbE "Purity/*FRt3SRqqt*FBt/SRqqt_purity"trainer/global_step2:JP¥K¬šHËbC Purity/0FRt0SRqqt3FBt/FBt_purity"trainer/global_step2:JP~u¿ƒNÌbI &Purity/1FRt1SRqqt1FBt/event_proportion"trainer/global_step2:JP;NÍbI &Purity/1FRt4SRqqt4FBt/event_proportion"trainer/global_step2:JPUÑ<÷JÎbE "Purity/2FRt1SRqqt2FBt/SRqqt_purity"trainer/global_step2:JP @NÏbI &Purity/2FRt4SRqqt2FBt/event_proportion"trainer/global_step2:JPÀ²NÐbI &Purity/4FRt3SRqqt0FBt/event_proportion"trainer/global_step2:JPO®NÑbI &Purity/*FRt2SRqqt3FBt/event_proportion"trainer/global_step2:JP\T‹NÒbI &Purity/0FRt*SRqqt1FBt/event_proportion"trainer/global_step2:JP `ÇNÓbI &Purity/1FRt*SRqqt2FBt/event_proportion"trainer/global_step2:JPîT­•NÔbI &Purity/1FRt1SRqqt4FBt/event_proportion"trainer/global_step2:JPÙ®­HÕbC Purity/1FRt2SRqqt*FBt/FBt_purity"trainer/global_step2:JP “€NÖbI &Purity/2FRt1SRqqt1FBt/event_proportion"trainer/global_step2:JPCN×bI &Purity/3FRt0SRqqt*FBt/event_proportion"trainer/global_step2:JP2ʬKHØbC Purity/3FRt0SRqqt*FBt/FRt_purity"trainer/global_step2:JPþ@ RNÙbI &Purity/*FRt*SRqqt1FBt/event_proportion"trainer/global_step2:JP®¬³JÚbE "Purity/*FRt1SRqqt*FBt/event_purity"trainer/global_step2:JPüxNÛbI &Purity/1FRt1SRqqt3FBt/event_proportion"trainer/global_step2:JP0®ÞØJÜbE "Purity/3FRt1SRqqt0FBt/event_purity"trainer/global_step2:JP-èq;Ýb6 jet/accuracy_4_of_5"trainer/global_step2:JP¦'¡f@Þb; particle/accuracy_4_of_4"trainer/global_step2:JP¡ÐQ@ßb; particle/accuracy_2_of_6"trainer/global_step2:JP=\þHàbC Purity/3FRt1SRqqt2FBt/FRt_purity"trainer/global_step2:JP2ZHábC Purity/*FRt3SRqqt0FBt/FRt_purity"trainer/global_step2:JPŸQj™NâbI &Purity/0FRt3SRqqt0FBt/event_proportion"trainer/global_step2:JP×F˜AHãbC Purity/2FRt*SRqqt1FBt/FBt_purity"trainer/global_step2:JPÁuÈ NäbI &Purity/3FRt*SRqqt1FBt/event_proportion"trainer/global_step2:JPæF¹áNåbI &Purity/3FRt4SRqqt2FBt/event_proportion"trainer/global_step2:JPm_Ê€JæbE "Purity/0FRt3SRqqt1FBt/event_purity"trainer/global_step2:JP>’¨ŒHçbC Purity/1FRt0SRqqt4FBt/FRt_purity"trainer/global_step2:JP¢âRJèbE "Purity/4FRt1SRqqt0FBt/SRqqt_purity"trainer/global_step2:JP‚€Û=JébE "Purity/*FRt0SRqqt2FBt/event_purity"trainer/global_step2:JPn‹‘(HêbC Purity/1FRt3SRqqt*FBt/FBt_purity"trainer/global_step2:JP‡Ö9ÌÒJ¤bE "Purity/*FRt2SRqqt0FBt/event_purity"trainer/global_step2:JPY¹H¥bC Purity/0FRt1SRqqt*FBt/FBt_purity"trainer/global_step2:JP#ò}=N¦bI &Purity/2FRt0SRqqt*FBt/event_proportion"trainer/global_step2:JP:E}qH§bC Purity/2FRt0SRqqt0FBt/FRt_purity"trainer/global_step2:JP û2H¨bC Purity/*FRt4SRqqt*FBt/FRt_purity"trainer/global_step2:JPÎ.­H©bC Purity/*FRt0SRqqt3FBt/FRt_purity"trainer/global_step2:JP7N‡JªbE "Purity/1FRt2SRqqt1FBt/SRqqt_purity"trainer/global_step2:JPA¯¥H«bC Purity/3FRt*SRqqt*FBt/FBt_purity"trainer/global_step2:JPĆ=fJ¬bE "Purity/0FRt3SRqqt0FBt/event_purity"trainer/global_step2:JPÏiŽBJ­bE "Purity/3FRt4SRqqt*FBt/event_purity"trainer/global_step2:JPÝ/Û™H®bC Purity/2FRt0SRqqt3FBt/FBt_purity"trainer/global_step2:JP‡¹8ÌJ¯bE "Purity/4FRt*SRqqt*FBt/event_purity"trainer/global_step2:JPЗF1H°bC Purity/1FRt*SRqqt1FBt/FBt_purity"trainer/global_step2:JP ®]Á;±b6 jet/accuracy_1_of_4"trainer/global_step2:JPÉ,úJ²bE "Purity/4FRt*SRqqt0FBt/event_purity"trainer/global_step2:JP´FÍzN³bI &Purity/*FRt*SRqqt3FBt/event_proportion"trainer/global_step2:JPbåaN´bI &Purity/*FRt3SRqqt0FBt/event_proportion"trainer/global_step2:JPô#'|JµbE "Purity/0FRt1SRqqt*FBt/SRqqt_purity"trainer/global_step2:JP (F;¶b6 jet/accuracy_5_of_6"trainer/global_step2:JPîgO¢N·bI &Purity/*FRt1SRqqt*FBt/event_proportion"trainer/global_step2:JPLŠ”J¸bE "Purity/0FRt2SRqqt1FBt/SRqqt_purity"trainer/global_step2:JP/èrhJ¹bE "Purity/1FRt0SRqqt0FBt/event_purity"trainer/global_step2:JPJÄ„HºbC Purity/1FRt0SRqqt0FBt/FRt_purity"trainer/global_step2:JPËù?J»bE "Purity/1FRt1SRqqt0FBt/SRqqt_purity"trainer/global_step2:JP_r%®J¼bE "Purity/1FRt3SRqqt0FBt/SRqqt_purity"trainer/global_step2:JPðìì¼N½bI &Purity/4FRt4SRqqt*FBt/event_proportion"trainer/global_step2:JPE-:³@¾b; particle/accuracy_2_of_5"trainer/global_step2:JP„áJ¿bE "Purity/*FRt*SRqqt0FBt/SRqqt_purity"trainer/global_step2:JP¨[THÊbC Purity/1FRt*SRqqt3FBt/FRt_purity"trainer/global_step2:JPËq†NËbI &Purity/4FRt*SRqqt2FBt/event_proportion"trainer/global_step2:JPút³«NÌbI &Purity/4FRt*SRqqt4FBt/event_proportion"trainer/global_step2:JP ÁŒŒJÍbE "Purity/0FRt2SRqqt2FBt/event_purity"trainer/global_step2:JPw†æeHÎbC Purity/*FRt0SRqqt1FBt/FBt_purity"trainer/global_step2:JP#­žëHÏbC Purity/*FRt1SRqqt2FBt/FRt_purity"trainer/global_step2:JPýöPJÐbE "Purity/*FRt1SRqqt2FBt/SRqqt_purity"trainer/global_step2:JPDìø¸JÑbE "Purity/0FRt1SRqqt*FBt/event_purity"trainer/global_step2:JP7ã„ JÒbE "Purity/1FRt3SRqqt1FBt/event_purity"trainer/global_step2:JPZ AäNÓbI &Purity/3FRt1SRqqt*FBt/event_proportion"trainer/global_step2:JPƒ0aJÔbE "Purity/*FRt2SRqqt2FBt/event_purity"trainer/global_step2:JP8ŒÃJÕbE "Purity/0FRt4SRqqt0FBt/SRqqt_purity"trainer/global_step2:JPDÜ ŽNÖbI &Purity/1FRt0SRqqt1FBt/event_proportion"trainer/global_step2:JP¨lh™N×bI &Purity/2FRt4SRqqt0FBt/event_proportion"trainer/global_step2:JP *HØbC Purity/4FRt1SRqqt0FBt/FRt_purity"trainer/global_step2:JP~yÖJÙbE "Purity/*FRt2SRqqt3FBt/SRqqt_purity"trainer/global_step2:JPü3ŠNÚbI &Purity/0FRt3SRqqt4FBt/event_proportion"trainer/global_step2:JPƒ&Ò?HÛbC Purity/1FRt2SRqqt1FBt/FBt_purity"trainer/global_step2:JP8ÑœHÜbC Purity/1FRt0SRqqt4FBt/FBt_purity"trainer/global_step2:JPE&YJÝbE "Purity/1FRt3SRqqt2FBt/SRqqt_purity"trainer/global_step2:JPø[kÑJÞbE "Purity/*FRt2SRqqt1FBt/event_purity"trainer/global_step2:JPƒxvÌNßbI &Purity/*FRt2SRqqt2FBt/event_proportion"trainer/global_step2:JP?g‚JàbE "Purity/1FRt1SRqqt1FBt/event_purity"trainer/global_step2:JPþªg¾HábC Purity/1FRt3SRqqt1FBt/FBt_purity"trainer/global_step2:JP7V”‰JâbE "Purity/0FRt2SRqqt3FBt/event_purity"trainer/global_step2:JPhž!ÊJãbE "Purity/4FRt2SRqqt0FBt/SRqqt_purity"trainer/global_step2:JP¾^µNäbI &Purity/*FRt3SRqqt4FBt/event_proportion"trainer/global_step2:JP|¹ÑsJåbE "Purity/0FRt2SRqqt*FBt/event_purity"trainer/global_step2:JP¦”gNæbI &Purity/1FRt1SRqqt0FBt/event_proportion"trainer/global_step2:JP+ÛHçbC Purity/2FRt*SRqqt*FBt/FBt_purity"trainer/global_step2:JP9¶´JJèbE "Purity/2FRt*SRqqt2FBt/event_purity"trainer/global_step2:JP(Ùš\JébE "Purity/2FRt1SRqqt1FBt/SRqqt_purity"trainer/global_step2:JPiÖÂYJêbE "Purity/2FRt3SRqqt*FBt/event_purity"trainer/global_step2:JPq`HëbC Purity/4FRt1SRqqt*FBt/FRt_purity"trainer/global_step2:JPk+”HìbC Purity/*FRt*SRqqt2FBt/FBt_purity"trainer/global_step2:JP¸ÓLHNíbI &Purity/*FRt*SRqqt4FBt/event_proportion"trainer/global_step2:JP€ ªí;îb6 jet/accuracy_1_of_6"trainer/global_step2:JP:Ѓ$JïbE "Purity/3FRt4SRqqt0FBt/SRqqt_purity"trainer/global_step2:JP( +zHðbC Purity/*FRt*SRqqt2FBt/FRt_purity"trainer/global_step2:JPêÔWqNñbI &Purity/4FRt3SRqqt4FBt/event_proportion"trainer/global_step2:JPrq2JòbE "Purity/0FRt4SRqqt1FBt/SRqqt_purity"trainer/global_step2:JPæ{žNóbI &Purity/*FRt4SRqqt0FBt/event_proportion"trainer/global_step2:JPfŸÖ†JôbE "Purity/0FRt2SRqqt*FBt/SRqqt_purity"trainer/global_step2:JPàËY"JõbE "Purity/2FRt3SRqqt*FBt/SRqqt_purity"trainer/global_step2:JPùYGðJöbE "Purity/3FRt0SRqqt1FBt/event_purity"trainer/global_step2:JPY"¡NžbI &Purity/3FRt*SRqqt3FBt/event_proportion"trainer/global_step2:JPéÆ$OJŸbE "Purity/*FRt4SRqqt*FBt/event_purity"trainer/global_step2:JPÖíJ bE "Purity/0FRt4SRqqt*FBt/event_purity"trainer/global_step2:JPøxÃC;¡b6 jet/accuracy_6_of_6"trainer/global_step2:JP´ˆÞN¢bI &Purity/*FRt0SRqqt*FBt/event_proportion"trainer/global_step2:JPÿwžäJ£bE "Purity/0FRt*SRqqt1FBt/event_purity"trainer/global_step2:JP­àÅ|H¤bC Purity/3FRt1SRqqt0FBt/FRt_purity"trainer/global_step2:JPWvùª;¥b6 jet/accuracy_3_of_3"trainer/global_step2:JPÉÚ¶Ä;¦b6 jet/accuracy_2_of_4"trainer/global_step2:JPmß |H§bC Purity/0FRt2SRqqt2FBt/FBt_purity"trainer/global_step2:JP-k˜J¨bE "Purity/1FRt4SRqqt*FBt/event_purity"trainer/global_step2:JP[ )H©bC Purity/4FRt2SRqqt*FBt/FRt_purity"trainer/global_step2:JPŠº:HªbC Purity/*FRt2SRqqt*FBt/FBt_purity"trainer/global_step2:JPÁ2¥éJ«bE "Purity/1FRt2SRqqt0FBt/SRqqt_purity"trainer/global_step2:JP[‰8H¬bC Purity/2FRt0SRqqt2FBt/FBt_purity"trainer/global_step2:JP]ûN­bI &Purity/3FRt1SRqqt2FBt/event_proportion"trainer/global_step2:JPmüé,N®bI &Purity/3FRt3SRqqt2FBt/event_proportion"trainer/global_step2:JPŸìN¯bI &Purity/4FRt4SRqqt0FBt/event_proportion"trainer/global_step2:JPë•Þ;°b6 jet/accuracy_1_of_2"trainer/global_step2:JP‚r·‡H±bC Purity/3FRt2SRqqt*FBt/FBt_purity"trainer/global_step2:JP‰ƒ43H²bC Purity/2FRt0SRqqt2FBt/FRt_purity"trainer/global_step2:JP5­mfN³bI &Purity/4FRt*SRqqt3FBt/event_proportion"trainer/global_step2:JP(8qJ´bE "Purity/0FRt2SRqqt3FBt/SRqqt_purity"trainer/global_step2:JPÚ¸0“<µbE "Purity/4FRt0SRqqt*FBt/event_purity"trainer/global_stl¤3‚ep2:JP äO$N¶bI &Purity/4FRt3SRqqt1FBt/event_proportion"trainer/global_step2:JPÃ6{äH“ bC Purity/3FRt2SRqqt*FBt/FRt_purity"trainer/global_step2:JP;´©N” bI &Purity/3FRt2SRqqt1FBt/event_proportion"trainer/global_step2:JP#×@• b; particle/accuracy_4_of_6"trainer/global_step2:JP”éX fÑJÄ bE "Purity/*FRt*SRqqt2FBt/event_purity"trainer/global_step2:JPV)ÍJÅ bE "Purity/1FRt3SRqqt*FBt/SRqqt_purity"trainer/global_step2:JPÆ÷0JÆ bE "Purity/2FRt1SRqqt*FBt/event_purity"trainer/global_step2:JP)â¾NÇ bI &Purity/4FRt0SRqqt4FBt/event_proportion"trainer/global_step2:JP?Z$;È b6 jet/accuracy_3_of_4"trainer/global_step2:JPTlÉ¢HÉ bC Purity/*FRt4SRqqt1FBt/FBt_purity"trainer/global_step2:JPb‰Y¶;Ê b6 jet/accuracy_2_of_6"trainer/global_step2:JPjÇšJË bE "Purity/*FRt1SRqqt3FBt/SRqqt_purity"trainer/global_step2:JPÒõˆNÌ bI &Purity/*FRt0SRqqt3FBt/event_proportion"trainer/global_step2:JPtG´xHÍ bC Purity/*FRt3SRqqt1FBt/FBt_purity"trainer/global_step2:JP4B'¹HÎ bC Purity/2FRt*SRqqt0FBt/FRt_purity"trainer/global_step2:JPÖèYnJÏ bE "Purity/1FRt*SRqqt4FBt/SRqqt_purity"trainer/global_step2:JPDlðHÐ bC Purity/4FRt0SRqqt*FBt/FRt_purity"trainer/global_step2:JP™WøÙ0Ñ + _wandbruntime‚506  _runtime‚506òÕŠžÝNÒ ”¥ =&Purity/1FRt*SRqqt*FBt/event_proportion‚0.4366179406642914 7 Purity/1FRt*SRqqt2FBt/FBt_purity‚0.9632810950279236 :"Purity/1FRt2SRqqt*FBt/SRqqt_purity‚0.15548980236053467 ,&Purity/4FRt3SRqqt3FBt/event_proportion‚0 ) Purity/*FRt1SRqqt3FBt/FRt_purity‚0.25 ?&Purity/1FRt0SRqqt2FBt/event_proportion‚0.015120967291295528 ("Purity/1FRt0SRqqt3FBt/event_purity‚0 ?&Purity/2FRt*SRqqt2FBt/event_proportion‚0.005418346729129553 9"Purity/0FRt3SRqqt1FBt/SRqqt_purity‚0.3749999701976776 ,&Purity/2FRt3SRqqt3FBt/event_proportion‚0 ("Purity/0FRt4SRqqt0FBt/event_purity‚0 A&Purity/0FRt*SRqqt4FBt/event_proportion‚0.00025201612152159214 ?&Purity/0FRt0SRqqt2FBt/event_proportion‚0.029674898833036423 ("Purity/1FRt2SRqqt*FBt/event_purity‚0 ,&Purity/4FRt4SRqqt3FBt/event_proportion‚0 ("Purity/1FRt*SRqqt4FBt/event_purity‚0 & Purity/2FRt2SRqqt2FBt/FRt_purity‚0 @&Purity/0FRt*SRqqt3FBt/event_proportion‚0.0030871976632624865 >&Purity/1FRt0SRqqt0FBt/event_proportion‚0.13142640888690948 9"Purity/2FRt2SRqqt1FBt/SRqqt_purity‚0.2874999940395355 ("Purity/4FRt*SRqqt0FBt/SRqqt_purity‚0 & Purity/2FRt2SRqqt2FBt/FBt_purity‚1 7 Purity/*FRt0SRqqt2FBt/FRt_purity‚0.2667907178401947 7 Purity/*FRt1SRqqt1FBt/FBt_purity‚0.9953466653823853 8 Purity/*FRt2SRqqt0FBt/FRt_purity‚0.11591441929340363 ?&Purity/0FRt0SRqqt3FBt/event_proportion‚0.002583165420219302 6 Purity/0FRt1SRqqt2FBt/FBt_purity‚0.961388885974884 @&Purity/1FRt2SRqqt2FBt/event_proportion‚0.0010080644860863686 ("Purity/2FRt1SRqqt2FBt/event_purity‚0 ,&Purity/2FRt3SRqqt2FBt/event_proportion‚0 7 Purity/*FRt0SRqqt2FBt/FBt_purity‚0.9855976104736328 :"Purity/1FRt*SRqqt2FBt/event_purity‚0.06994287669658661 <"Purity/1FRt1SRqqt0FBt/event_purity‚0.0025529316626489162 ("Purity/1FRt3SRqqt0FBt/event_purity‚0 ,&Purity/1FRt3SRqqt4FBt/event_proportion‚0 8 Purity/3FRt2SRqqt0FBt/FRt_purity‚0.06763284653425217 particle/accuracy_3_of_5‚1 ("Purity/*FRt1SRqqt3FBt/event_purity‚0 ("Purity/1FRt2SRqqt2FBt/event_purity‚0 7 Purity/1FRt2SRqqt2FBt/FRt_purity‚0.2222222238779068 ("Purity/3FRt4SRqqt0FBt/event_purity‚0 9"Purity/*FRt*SRqqt3FBt/SRqqt_purity‚0.4523809552192688 A&Purity/1FRt*SRqqt4FBt/event_proportion‚0.00012600806076079607 >&Purity/1FRt0SRqqt*FBt/event_proportion‚0.19745463132858276 :"Purity/2FRt2SRqqt*FBt/SRqqt_purity‚0.15444688498973846 & Purity/3FRt1SRqqt*FBt/FBt_purity‚1 7 Purity/1FRt2SRqqt2FBt/FBt_purity‚0.9583333134651184 & Purity/2FRt3SRqqt1FBt/FBt_purity‚1 ("Purity/3FRt1SRqqt2FBt/event_purity‚0 +"Purity/1FRt*SRqqt3FBt/SRqqt_purity‚0.25 8 Purity/*FRt2SRqqt*FBt/FRt_purity‚0.11843079328536987 >&Purity/0FRt2SRqqt*FBt/event_proportion‚0.05361643061041832 7 Purity/3FRt2SRqqt*FBt/FRt_purity‚0.0659722238779068 @&Purity/3FRt2SRqqt1FBt/event_proportion‚0.0003780241822823882 particle/accuracy_4_of_6‚1 & Purity/1FRt3SRqqt2FBt/FBt_purity‚1 =&Purity/*FRt*SRqqt0FBt/event_proportion‚0.6160534024238586 >&Purity/*FRt*SRqqt2FBt/event_proportion‚0.06747731566429138 8 Purity/*FRt*SRqqt3FBt/FRt_purity‚0.11764705926179886 :"Purity/*FRt1SRqqt0FBt/event_purity‚0.07913762331008911 * Purity/0FRt*SRqqt4FBt/FBt_purity‚0.875 & Purity/1FRt3SRqqt2FBt/FRt_purity‚0 7 Purity/*FRt*SRqqt1FBt/FBt_purity‚0.9975016117095947 ("Purity/2FRt2SRqqt*FBt/event_purity‚0 ("Purity/3FRt*SRqqt1FBt/event_purity‚0 ,&Purity/3FRt*SRqqt4FBt/event_proportion‚0 7 Purity/2FRt0SRqqt1FBt/FRt_purity‚0.2100132405757904 ?&Purity/3FRt*SRqqt*FBt/event_proportion‚0.009954636916518211 6validation_average_jet_accuracy‚0.2880053222179413 :"Purity/*FRt1SRqqt1FBt/SRqqt_purity‚0.25053146481513977 7 Purity/0FRt0SRqqt*FBt/FBt_purity‚0.9896841645240784 7 Purity/3FRt0SRqqt1FBt/FRt_purity‚0.2499999850988388 :"Purity/2FRt3SRqqt1FBt/SRqqt_purity‚0.06666667014360428 particle/accuracy_6_of_7‚1 ("Purity/4FRt2SRqqt0FBt/event_purity‚0 :"Purity/1FRt*SRqqt1FBt/SRqqt_purity‚0.24542465806007385 ?&Purity/2FRt3SRqqt*FBt/event_proportion‚0.004032257944345474 @&Purity/3FRt1SRqqt1FBt/event_proportion‚0.0009450604557059705 @&Purity/4FRt2SRqqt0FBt/event_proportion‚6.300403038039804e-05 ("Purity/*FRt1SRqqt4FBt/event_purity‚0 ?&Purity/0FRt1SRqqt2FBt/event_proportion‚0.008757560513913631 particle/accuracy_1_of_2‚1 & Purity/3FRt4SRqqt*FBt/FRt_purity‚0 9"Purity/1FRt4SRqqt0FBt/SRqqt_purity‚0.0833333358168602 ) Purity/1FRt1SRqqt4FBt/FBt_purity‚0.75 ("Purity/1FRt3SRqqt*FBt/event_purity‚0 :"Purity/2FRt1SRqqt*FBt/SRqqt_purity‚0.17319801449775696 jet/accuracy_2_of_7‚0 & Purity/*FRt0SRqqt4FBt/FRt_purity‚0 :"Purity/2FRt2SRqqt0FBt/SRqqt_purity‚0.11015220731496811 8 Purity/3FRt*SRqqt*FBt/FRt_purity‚0.07696706056594849 @&Purity/3FRt1SRqqt0FBt/event_proportion‚0.0025201612152159214 9"Purity/0FRt3SRqqt*FBt/SRqqt_purity‚0.1863001137971878 * Purity/0FRt3SRqqt*FBt/FBt_purity‚0.875 ("Purity/1FRt0SRqqt4FBt/event_purity‚0 8 Purity/*FRt1SRqqt1FBt/FRt_purity‚0.19032436609268188 >&Purity/0FRt0SRqqt*FBt/event_proportion‚0.11857359111309052 ;"Purity/1FRt*SRqqt*FBt/event_purity‚0.031498633325099945 @&Purity/2FRt2SRqqt2FBt/event_proportion‚6.300403038039804e-05 ,&Purity/3FRt4SRqqt3FBt/event_proportion‚0 ,&Purity/4FRt2SRqqt3FBt/event_proportion‚0 :"Purity/*FRt*SRqqt2FBt/event_purity‚0.16306354105472565 :"Purity/1FRt3SRqqt*FBt/SRqqt_purity‚0.14480715990066528 ("Purity/2FRt1SRqqt*FBt/event_purity‚0 ,&Purity/4FRt0SRqqt4FBt/event_proportion‚0 ,jet/accuracy_3_of_4‚0.025847364217042923 & Purity/*FRt4SRqqt1FBt/FBt_purity‚1 +jet/accuracy_2_of_6‚0.25438597798347473 *"Purity/*FRt1SRqqt3FBt/SRqqt_purity‚0.5 ?&Purity/*FRt0SRqqt3FBt/event_proportion‚0.003717237850651145 6 Purity/*FRt3SRqqt1FBt/FBt_purity‚0.976190447807312 8 Purity/2FRt*SRqqt0FBt/FRt_purity‚0.09636475890874863 ("Purity/1FRt*SRqqt4FBt/SRqqt_purity‚0 & Purity/4FRt0SRqqt*FBt/FRt_purity‚0 ("Purity/*FRt3SRqqt0FBt/event_purity‚0 7 Purity/0FRt2SRqqt1FBt/FBt_purity‚0.9953917264938354 :"Purity/1FRt*SRqqt*FBt/SRqqt_purity‚0.16174755990505219 8 Purity/1FRt1SRqqt*FBt/FRt_purity‚0.14117175340652466 @&Purity/3FRt2SRqqt*FBt/event_proportion‚0.0028351815417408943 particle/accuracy_3_of_3‚1 >&Purity/*FRt0SRqqt1FBt/event_proportion‚0.14591734111309052 ,&Purity/0FRt1SRqqt4FBt/event_proportion‚0 =&Purity/1FRt1SRqqt*FBt/event_proportion‚0.1723790317773819 8 Purity/2FRt1SRqqt*FBt/FRt_purity‚0.12405509501695633 ("Purity/3FRt1SRqqt1FBt/event_purity‚0 ("Purity/0FRt3SRqqt*FBt/event_purity‚0 & Purity/1FRt*SRqqt4FBt/FRt_purity‚0 :"Purity/0FRt*SRqqt*FBt/event_purity‚0.30606314539909363 :"Purity/0FRt*SRqqt0FBt/event_purity‚0.11835914105176926 =&Purity/1FRt*SRqqt0FBt/event_proportion‚0.3135710656642914 & Purity/2FRt0SRqqt1FBt/FBt_purity‚1 & Purity/4FRt0SRqqt0FBt/FRt_purity‚0 7 Purity/*FRt*SRqqt1FBt/FRt_purity‚0.1892215609550476 7 Purity/1FRt*SRqqt3FBt/FBt_purity‚0.9166667461395264 7 Purity/2FRt*SRqqt*FBt/FRt_purity‚0.1228276863694191 ,particle/sensitivity‚0.18784886598587036 particle/accuracy_7_of_7‚0.5 ("Purity/2FRt2SRqqt2FBt/event_purity‚0 epoch‚0 ("Purity/1FRt*SRqqt3FBt/event_purity‚0 @&Purity/1FRt0SRqqt3FBt/event_proportion‚0.0010710685746744275 8 Purity/1FRt1SRqqt0FBt/FRt_purity‚0.13157279789447784 >&Purity/2FRt*SRqqt*FBt/event_proportion‚0.10244455933570862 particle/accuracy_2_of_3‚1 :"Purity/*FRt*SRqqt3FBt/event_purity‚0.05689655616879463 ?&Purity/2FRt*SRqqt1FBt/event_proportion‚0.021169355139136314 8 Purity/2FRt1SRqqt0FBt/FRt_purity‚0.10072343051433563 ,&Purity/2FRt1SRqqt3FBt/event_proportion‚0 9"Purity/2FRt3SRqqt0FBt/SRqqt_purity‚0.1170634999871254 & Purity/3FRt0SRqqt1FBt/FBt_purity‚1 :"Purity/3FRt1SRqqt*FBt/SRqqt_purity‚0.22839507460594177 ,&Purity/3FRt2SRqqt3FBt/event_proportion‚0 @&Purity/3FRt0SRqqt1FBt/event_proportion‚0.0011340725468471646 :"Purity/0FRt1SRqqt1FBt/SRqqt_purity‚0.24180883169174194 ("Purity/1FRt2SRqqt0FBt/event_purity‚0 ,&Purity/1FRt4SRqqt1FBt/event_proportion‚0 :"Purity/2FRt*SRqqt1FBt/event_purity‚0.00358422938734293 8 Purity/2FRt3SRqqt1FBt/FRt_purity‚0.15000000596046448 & Purity/3FRt4SRqqt0FBt/FRt_purity‚0 =&Purity/*FRt1SRqqt1FBt/event_proportion‚0.1001134067773819 :"Purity/0FRt*SRqqt*FBt/SRqqt_purity‚0.17750997841358185 7 Purity/1FRt0SRqqt*FBt/FBt_purity‚0.9918087124824524 ,&Purity/4FRt2SRqqt2FBt/event_proportion‚0 ,&Purity/2FRt0SRqqt4FBt/event_proportion‚0 9 Purity/3FRt*SRqqt0FBt/FRt_purity‚0.041507940739393234 ("Purity/3FRt1SRqqt*FBt/event_purity‚0 )jet/accuracy_3_of_5‚0.024527532979846 particle/accuracy_4_of_7‚1 ("Purity/*FRt1SRqqt4FBt/SRqqt_purity‚0 7 Purity/0FRt1SRqqt3FBt/FBt_purity‚0.8999999761581421 trainer/global_step‚603 @&Purity/3FRt2SRqqt0FBt/event_proportion‚0.0024571572430431843 ;"Purity/1FRt*SRqqt0FBt/event_purity‚0.024264488369226456 7 Purity/*FRt0SRqqt3FBt/FBt_purity‚0.9216270446777344 ,&Purity/4FRt*SRqqt1FBt/event_proportion‚0 jet/accuracy_5_of_7‚0 & Purity/2FRt0SRqqt3FBt/FRt_purity‚0 9 Purity/*FRt0SRqqt0FBt/FRt_purity‚0.058934058994054794 @&Purity/1FRt4SRqqt*FBt/event_proportion‚0.0003780241822823882 ,&Purity/2FRt4SRqqt3FBt/event_proportion‚0 *jet/accuracy_2_of_3‚0.1241641491651535 :"Purity/2FRt4SRqqt*FBt/SRqqt_purity‚0.05000000074505806 8 Purity/*FRt2SRqqt2FBt/FRt_purity‚0.20512820780277252 ("Purity/0FRt1SRqqt3FBt/event_purity‚0 ("Purity/1FRt3SRqqt2FBt/event_purity‚0 :"Purity/0FRt0SRqqt3FBt/event_purity‚0.10999999940395355 A&Purity/0FRt0SRqqt4FBt/event_proportion‚0.00025201612152159214 8 Purity/1FRt3SRqqt*FBt/FRt_purity‚0.14444445073604584 7 Purity/2FRt0SRqqt*FBt/FBt_purity‚0.9931899905204773 ,&Purity/4FRt3SRqqt*FBt/event_proportion‚0 +jet/accuracy_1_of_1‚0.30278247594833374 ("Purity/2FRt4SRqqt0FBt/event_purity‚0 7 Purity/*FRt2SRqqt2FBt/FBt_purity‚0.9736841917037964 :"Purity/1FRt0SRqqt2FBt/event_purity‚0.10420432686805725 8 Purity/1FRt2SRqqt*FBt/FRt_purity‚0.12346053868532181 9"Purity/2FRt*SRqqt2FBt/SRqqt_purity‚0.3114035129547119 8 Purity/2FRt2SRqqt0FBt/FRt_purity‚0.12143034487962723 & Purity/3FRt0SRqqt*FBt/FBt_purity‚1 ("Purity/*FRt*SRqqt4FBt/event_purity‚0 ("Purity/3FRt3SRqqt*FBt/event_purity‚0 9"Purity/3FRt3SRqqt0FBt/SRqqt_purity‚0.1428571492433548 >&Purity/0FRt0SRqqt0FBt/event_proportion‚0.04082661122083664 7 Purity/1FRt0SRqqt3FBt/FRt_purity‚0.1428571492433548 ?&Purity/2FRt0SRqqt2FBt/event_proportion‚0.003465221729129553 jet/accuracy_4_of_4‚0 ("Purity/4FRt2SRqqt*FBt/event_purity‚0 :"Purity/*FRt*SRqqt1FBt/SRqqt_purity‚0.24577125906944275 @&Purity/0FRt4SRqqt*FBt/event_proportion‚0.0005040322430431843 & Purity/1FRt0SRqqt1FBt/FBt_purity‚1 jet/accuracy_4_of_7‚0 ,&Purity/*FRt4SRqqt4FBt/event_proportion‚0 >&Purity/0FRt*SRqqt*FBt/event_proportion‚0.40996724367141724 :"Purity/0FRt*SRqqt3FBt/event_purity‚0.10185185074806213 ?&Purity/1FRt3SRqqt0FBt/event_proportion‚0.007938507944345474 8 Purity/3FRt1SRqqt1FBt/FRt_purity‚0.14102564752101898 ("Purity/2FRt4SRqqt*FBt/event_purity‚0 7 Purity/1FRt1SRqqt3FBt/FBt_purity‚0.9166666865348816 6 Purity/0FRt*SRqqt2FBt/FBt_purity‚0.980598509311676 ,&Purity/0FRt3SRqqt3FBt/event_proportion‚0 8 Purity/1FRt*SRqqt0FBt/FRt_purity‚0.09786342084407806 :"Purity/1FRt1SRqqt2FBt/SRqqt_purity‚0.24447278678417206 @&Purity/4FRt1SRqqt0FBt/event_proportion‚6.300403038039804e-05 /"Purity/*FRt4SRqqt0FBt/SRqqt_purity‚0.109375 & Purity/3FRt3SRqqt0FBt/FRt_purity‚0 ("Purity/4FRt2SRqqt*FBt/SRqqt_purity‚0 @&Purity/*FRt0SRqqt4FBt/event_proportion‚0.0003150201519019902 @&Purity/*FRt1SRqqt4FBt/event_proportion‚6.300403038039804e-05 :"Purity/0FRt*SRqqt1FBt/SRqqt_purity‚0.24334624409675598 >&Purity/0FRt1SRqqt1FBt/event_proportion‚0.05185231938958168 >&Purity/1FRt2SRqqt0FBt/event_proportion‚0.04731602966785431 ("Purity/2FRt0SRqqt0FBt/event_purity‚0 @&Purity/3FRt*SRqqt0FBt/event_proportion‚0.0071824598126113415 9"Purity/3FRt3SRqqt*FBt/SRqqt_purity‚0.1428571492433548 ?&Purity/2FRt0SRqqt0FBt/event_proportion‚0.024571573361754417 >&Purity/*FRt2SRqqt1FBt/event_proportion‚0.02255544438958168 >&Purity/1FRt*SRqqt1FBt/event_proportion‚0.09929435700178146 >&Purity/2FRt1SRqqt0FBt/event_proportion‚0.03181703761219978 :"Purity/3FRt2SRqqt0FBt/SRqqt_purity‚0.12318839877843857 ,&Purity/4FRt2SRqqt4FBt/event_proportion‚0 ,&Purity/*FRt2SRqqt4FBt/event_proportion‚0 @&Purity/0FRt3SRqqt1FBt/event_proportion‚0.0006300403038039804 8 Purity/1FRt*SRqqt*FBt/FRt_purity‚0.12459518760442734 :"Purity/2FRt*SRqqt0FBt/SRqqt_purity‚0.13546496629714966 ?&Purity/2FRt2SRqqt1FBt/event_proportion‚0.003969253972172737 7 Purity/*FRt0SRqqt4FBt/FBt_purity‚0.8999999761581421 & Purity/2FRt*SRqqt3FBt/FBt_purity‚1 & Purity/3FRt3SRqqt*FBt/FRt_purity‚0 >&Purity/0FRt0SRqqt1FBt/event_proportion‚0.08606351166963577 A&Purity/0FRt1SRqqt3FBt/event_proportion‚0.00044102821266278625 ,&Purity/3FRt1SRqqt3FBt/event_proportion‚0 ,&Purity/4FRt4SRqqt1FBt/event_proportion‚0 particle/accuracy_1_of_7‚1 * Purity/0FRt3SRqqt1FBt/FBt_purity‚0.875 & Purity/1FRt1SRqqt4FBt/FRt_purity‚0 7 Purity/*FRt0SRqqt*FBt/FRt_purity‚0.1122928187251091 7 Purity/*FRt3SRqqt*FBt/FBt_purity‚0.9772727489471436 ,&Purity/1FRt3SRqqt3FBt/event_proportion‚0 ,&Purity/4FRt4SRqqt4FBt/event_proportion‚0 * Purity/1FRt*SRqqt4FBt/FBt_purity‚0.875 ?&Purity/0FRt3SRqqt*FBt/event_proportion‚0.006741431541740894 8 Purity/1FRt3SRqqt0FBt/FRt_purity‚0.13437020778656006 ,&Purity/2FRt*SRqqt4FBt/event_proportion‚0 & Purity/*FRt4SRqqt*FBt/FBt_purity‚1 =&Purity/*FRt0SRqqt2FBt/event_proportion‚0.0483240932226181 ("Purity/*FRt1SRqqt2FBt/event_purity‚0 ,&Purity/0FRt2SRqqt4FBt/event_proportion‚0 8 Purity/1FRt2SRqqt0FBt/FRt_purity‚0.11887060850858688 ,&Purity/1FRt2SRqqt4FBt/event_proportion‚0 8 Purity/2FRt2SRqqt*FBt/FRt_purity‚0.12317753583192825 ,&Purity/3FRt3SRqqt4FBt/event_proportion‚0 ("Purity/*FRt4SRqqt1FBt/event_purity‚0 =&Purity/*FRt2SRqqt*FBt/event_proportion‚0.1332535296678543 7 Purity/0FRt2SRqqt*FBt/FBt_purity‚0.9900153875350952 @&Purity/1FRt*SRqqt3FBt/event_proportion‚0.0013230846961960196 :"Purity/*FRt2SRqqt2FBt/SRqqt_purity‚0.19736842811107635 :"Purity/0FRt2SRqqt2FBt/SRqqt_purity‚0.13461539149284363 >&Purity/0FRt*SRqqt0FBt/event_proportion‚0.21931703388690948 ;"Purity/0FRt2SRqqt0FBt/event_purity‚0.006425026338547468 9"Purity/1FRt1SRqqt*FBt/SRqqt_purity‚0.1717623621225357 ("Purity/1FRt2SRqqt1FBt/event_purity‚0 9"Purity/2FRt*SRqqt*FBt/SRqqt_purity‚0.1614358127117157 :"Purity/3FRt*SRqqt0FBt/SRqqt_purity‚0.13167516887187958 & Purity/3FRt1SRqqt1FBt/FBt_purity‚1 @&Purity/3FRt3SRqqt*FBt/event_proportion‚0.0005670362734235823 8 Purity/1FRt0SRqqt1FBt/FRt_purity‚0.20196278393268585 ?&Purity/1FRt2SRqqt1FBt/event_proportion‚0.008946572430431843 @&Purity/1FRt4SRqqt0FBt/event_proportion‚0.0003780241822823882 & Purity/2FRt3SRqqt*FBt/FBt_purity‚1 7 Purity/2FRt*SRqqt2FBt/FBt_purity‚0.9855555295944214 8 Purity/2FRt4SRqqt*FBt/FRt_purity‚0.30000001192092896 & Purity/1FRt4SRqqt0FBt/FRt_purity‚0 =&Purity/0FRt1SRqqt*FBt/event_proportion‚0.2305317521095276 ;"Purity/0FRt2SRqqt1FBt/event_purity‚0.008064515888690948 8 Purity/2FRt*SRqqt2FBt/FRt_purity‚0.18111109733581543 ("Purity/*FRt*SRqqt4FBt/SRqqt_purity‚0 *"Purity/0FRt1SRqqt3FBt/SRqqt_purity‚0.5 & Purity/4FRt*SRqqt*FBt/FRt_purity‚0 :"Purity/3FRt1SRqqt0FBt/SRqqt_purity‚0.23333334922790527 A&Purity/3FRt4SRqqt*FBt/event_proportion‚0.00012600806076079607 particle/accuracy_6_of_6‚1 ("Purity/*FRt3SRqqt2FBt/event_purity‚0 7 Purity/*FRt*SRqqt*FBt/FBt_purity‚0.9891412258148193 <"Purity/1FRt1SRqqt*FBt/event_purity‚0.0032553812488913536 ,&Purity/1FRt2SRqqt3FBt/event_proportion‚0 <"Purity/2FRt*SRqqt*FBt/event_purity‚0.0005973715451546013 ?&Purity/2FRt3SRqqt0FBt/event_proportion‚0.003654233878478408 +jet/accuracy_2_of_2‚0.05297669395804405 jet/accuracy_5_of_5‚0 ("Purity/3FRt*SRqqt2FBt/SRqqt_purity‚0 :"Purity/*FRt3SRqqt1FBt/SRqqt_purity‚0.20105817914009094 @&Purity/3FRt*SRqqt2FBt/event_proportion‚0.0003150201519019902 particle/accuracy_3_of_4‚1 9"Purity/0FRt*SRqqt3FBt/SRqqt_purity‚0.4333333373069763 ("Purity/4FRt1SRqqt0FBt/event_purity‚0 :"Purity/0FRt*SRqqt2FBt/SRqqt_purity‚0.24472223222255707 7 Purity/0FRt0SRqqt1FBt/FBt_purity‚0.9993136525154114 ,&Purity/0FRt3SRqqt2FBt/event_proportion‚0 :"Purity/1FRt3SRqqt1FBt/SRqqt_purity‚0.13333334028720856 ,&Purity/4FRt0SRqqt1FBt/event_proportion‚0 ,&Purity/4FRt3SRqqt2FBt/event_proportion‚0 *jet/accuracy_1_of_5‚0.7825754880905151 :"Purity/3FRt*SRqqt*FBt/SRqqt_purity‚0.14980843663215637 ,&Purity/3FRt3SRqqt3FBt/event_proportion‚0 8 Purity/3FRt*SRqqt2FBt/FRt_purity‚0.06666667014360428 ("Purity/2FRt*SRqqt3FBt/event_purity‚0 ("Purity/4FRt1SRqqt*FBt/event_purity‚0 _runtime‚ 506.297242008 )loss/val_total_loss‚57.12536543837079 9"Purity/0FRt0SRqqt2FBt/event_purity‚0.3242494463920593 & Purity/3FRt*SRqqt2FBt/FBt_purity‚1 7 Purity/*FRt2SRqqt3FBt/FBt_purity‚0.6666666865348816 :"Purity/*FRt2SRqqt1FBt/SRqqt_purity‚0.24108685553073883 :"Purity/1FRt1SRqqt1FBt/SRqqt_purity‚0.26845529675483704 ,&Purity/3FRt3SRqqt1FBt/event_proportion‚0 jet/accuracy_4_of_6‚0 9"Purity/*FRt2SRqqt0FBt/SRqqt_purity‚0.1480746865272522 ("Purity/0FRt1SRqqt2FBt/event_purity‚0 ("Purity/2FRt1SRqqt0FBt/event_purity‚0 ("Purity/2FRt1SRqqt1FBt/event_purity‚0 ?&Purity/2FRt2SRqqt0FBt/event_proportion‚0.015435988083481789 ("Purity/0FRt0SRqqt4FBt/event_purity‚0 :"Purity/0FRt3SRqqt0FBt/SRqqt_purity‚0.17668652534484863 particle/accuracy_2_of_7‚1 8 Purity/1FRt1SRqqt2FBt/FRt_purity‚0.12738095223903656 7 Purity/2FRt1SRqqt2FBt/FBt_purity‚0.9868420958518982 ,&Purity/2FRt4SRqqt1FBt/event_proportion‚0 ,&Purity/3FRt2SRqqt2FBt/event_proportion‚0 7 Purity/1FRt1SRqqt2FBt/FBt_purity‚0.9666666984558105 ;"Purity/2FRt0SRqqt1FBt/event_purity‚0.008333333767950535 8 Purity/2FRt1SRqqt2FBt/FRt_purity‚0.08771929889917374 ,&Purity/2FRt1SRqqt4FBt/event_proportion‚0 ("Purity/3FRt*SRqqt*FBt/event_purity‚0 @&Purity/3FRt0SRqqt2FBt/event_proportion‚6.300403038039804e-05 ("Purity/0FRt4SRqqt1FBt/event_purity‚0 ("Purity/4FRt1SRqqt*FBt/SRqqt_purity‚0 7 Purity/0FRt*SRqqt3FBt/FBt_purity‚0.9238684177398682 8 Purity/2FRt4SRqqt0FBt/FRt_purity‚0.30000001192092896 +"Purity/3FRt2SRqqt1FBt/SRqqt_purity‚0.25 jet/accuracy_1_of_7‚0 & Purity/1FRt4SRqqt*FBt/FRt_purity‚0 ,&Purity/*FRt3SRqqt3FBt/event_proportion‚0 7 Purity/*FRt1SRqqt*FBt/FBt_purity‚0.9857835173606873 ?&Purity/*FRt3SRqqt*FBt/event_proportion‚0.020476309582591057 :"Purity/0FRt1SRqqt0FBt/event_purity‚0.15242965519428253 ,&Purity/1FRt4SRqqt2FBt/event_proportion‚0 @&Purity/2FRt3SRqqt1FBt/event_proportion‚0.0003780241822823882 & Purity/4FRt*SRqqt0FBt/FRt_purity‚0 ?&Purity/0FRt2SRqqt1FBt/event_proportion‚0.009261592291295528 A&Purity/0FRt4SRqqt0FBt/event_proportion‚0.00044102821266278625 >&Purity/2FRt1SRqqt*FBt/event_proportion‚0.04158266261219978 ("Purity/3FRt2SRqqt*FBt/event_purity‚0 ("Purity/3FRt1SRqqt2FBt/SRqqt_purity‚0 8 Purity/*FRt1SRqqt0FBt/FRt_purity‚0.11775355786085129 7 Purity/*FRt2SRqqt1FBt/FBt_purity‚0.9975185990333557 ,&Purity/*FRt4SRqqt2FBt/event_proportion‚0 ,&Purity/0FRt4SRqqt2FBt/event_proportion‚0 @&Purity/4FRt*SRqqt0FBt/event_proportion‚0.0001890120911411941 * Purity/*FRt*SRqqt4FBt/FBt_purity‚0.875 & Purity/3FRt2SRqqt1FBt/FBt_purity‚1 9"Purity/1FRt2SRqqt2FBt/SRqqt_purity‚0.2152777910232544 :"Purity/*FRt0SRqqt3FBt/event_purity‚0.05892857536673546 ("Purity/0FRt0SRqqt0FBt/event_purity‚1 :"Purity/1FRt0SRqqt1FBt/event_purity‚0.08975519239902496 8 Purity/2FRt3SRqqt*FBt/FRt_purity‚0.13123808801174164 @&Purity/4FRt1SRqqt*FBt/event_proportion‚6.300403038039804e-05 ("Purity/*FRt4SRqqt0FBt/event_purity‚0 >&Purity/*FRt0SRqqt0FBt/event_proportion‚0.1575730890?n¾C0356293 >&Purity/*FRt1SRqqt0FBt/event_proportion‚0.33039313554763794 ,&Purity/0FRt4SRqqt4FBt/event_proportion‚0 7 Purity/1FRt0SRqqt2FBt/FBt_purity‚0.9850406646728516 @&Purity/4FRt*SRqqt*FBt/event_proportion‚0.0001890120911411941 particle/accuracy_3_of_6‚1 ?&Purity/*FRt1SRqqt2FBt/event_proportion‚0.017011089250445366 ,&Purity/0FRt4SRqqt3FBt/event_proportion‚0 7 Purity/1FRt0SRqqt2FBt/FRt_purity‚0.2968064248561859 @&Purity/2FRt4SRqqt*FBt/event_proportion‚0.0003150201519019902 ,&Purity/3FRt0SRqqt4FBt/event_proportion‚0 9"Purity/3FRt2SRqqt*FBt/SRqqt_purity‚0.1440972089767456 ,&Purity/3FRt4SRqqt1FBt/event_proportion‚0 :"Purity/2FRt4SRqqt0FBt/SRqqt_purity‚0.05000000074505806 7 Purity/0FRt*SRqqt1FBt/FBt_purity‚0.9969359636306763 :"Purity/1FRt0SRqqt*FBt/event_purity‚0.06716331094503403 <"Purity/2FRt0SRqqt*FBt/event_purity‚0.0015360983088612556 ("Purity/3FRt3SRqqt0FBt/event_purity‚0 8 Purity/*FRt*SRqqt*FBt/FRt_purity‚0.12208934128284454 8 Purity/2FRt*SRqqt1FBt/FRt_purity‚0.20347507297992706 @&Purity/3FRt0SRqqt0FBt/event_proportion‚0.0015120967291295528 :"Purity/3FRt1SRqqt1FBt/SRqqt_purity‚0.42307692766189575 ,&Purity/4FRt1SRqqt2FBt/event_proportion‚0 ;"Purity/*FRt2SRqqt*FBt/event_purity‚0.002497996436432004 @&Purity/1FRt0SRqqt4FBt/event_proportion‚6.300403038039804e-05 8 Purity/1FRt1SRqqt1FBt/FRt_purity‚0.18075275421142578 ,&Purity/2FRt2SRqqt3FBt/event_proportion‚0 8 Purity/2FRt1SRqqt1FBt/FRt_purity‚0.22561442852020264 >&Purity/*FRt2SRqqt0FBt/event_proportion‚0.10855595022439957 ("Purity/*FRt3SRqqt*FBt/event_purity‚0 ?&Purity/0FRt2SRqqt0FBt/event_proportion‚0.043283771723508835 ,&Purity/4FRt0SRqqt2FBt/event_proportion‚0 :"Purity/*FRt*SRqqt*FBt/event_purity‚0.14523905515670776 8 Purity/*FRt*SRqqt0FBt/FRt_purity‚0.09495631605386734 ("Purity/3FRt0SRqqt0FBt/event_purity‚0 ,&Purity/3FRt0SRqqt3FBt/event_proportion‚0 (particle/accuracy‚0.8221973180770874 ) Purity/*FRt1SRqqt4FBt/FBt_purity‚0.75 >&Purity/0FRt*SRqqt2FBt/event_proportion‚0.03944052383303642 :"Purity/0FRt1SRqqt1FBt/event_purity‚0.07174625247716904 9"Purity/2FRt*SRqqt1FBt/SRqqt_purity‚0.2293599247932434 7 Purity/3FRt*SRqqt1FBt/FRt_purity‚0.1674242466688156 ("Purity/3FRt0SRqqt*FBt/event_purity‚0 & Purity/0FRt4SRqqt*FBt/FBt_purity‚1 particle/accuracy_1_of_6‚1 particle/accuracy_5_of_7‚1 8 Purity/*FRt1SRqqt*FBt/FRt_purity‚0.13301745057106018 :"Purity/*FRt2SRqqt*FBt/SRqqt_purity‚0.16571961343288422 ("Purity/3FRt2SRqqt0FBt/event_purity‚0 *jet/accuracy_1_of_3‚0.6566240787506104 particle/accuracy_5_of_5‚1 +jet/accuracy_3_of_6‚0.02631578966975212 :"Purity/*FRt*SRqqt*FBt/SRqqt_purity‚0.16882193088531494 ;"Purity/*FRt0SRqqt0FBt/event_purity‚0.046057987958192825 9"Purity/1FRt*SRqqt2FBt/SRqqt_purity‚0.2625192105770111 ("Purity/2FRt*SRqqt0FBt/event_purity‚0 8 Purity/2FRt3SRqqt0FBt/FRt_purity‚0.13658009469509125 ,&Purity/3FRt2SRqqt4FBt/event_proportion‚0 ("Purity/*FRt0SRqqt4FBt/event_purity‚0 :"Purity/*FRt3SRqqt*FBt/SRqqt_purity‚0.14834100008010864 7 Purity/0FRt0SRqqt3FBt/FBt_purity‚0.9355555772781372 >&Purity/1FRt1SRqqt1FBt/event_proportion‚0.03944052383303642 ,&Purity/1FRt4SRqqt4FBt/event_proportion‚0 9"Purity/2FRt1SRqqt2FBt/SRqqt_purity‚0.3114035129547119 ,&Purity/2FRt4SRqqt2FBt/event_proportion‚0 ,&Purity/4FRt3SRqqt0FBt/event_proportion‚0 @&Purity/*FRt2SRqqt3FBt/event_proportion‚6.300403038039804e-05 >&Purity/0FRt*SRqqt1FBt/event_proportion‚0.14787046611309052 ?&Purity/1FRt*SRqqt2FBt/event_proportion‚0.022303426638245583 @&Purity/1FRt1SRqqt4FBt/event_proportion‚6.300403038039804e-05 7 Purity/1FRt2SRqqt*FBt/FBt_purity‚0.9959677457809448 ?&Purity/2FRt1SRqqt1FBt/event_proportion‚0.007875503972172737 @&Purity/3FRt0SRqqt*FBt/event_proportion‚0.0027091733645647764 7 Purity/3FRt0SRqqt*FBt/FRt_purity‚0.1262626349925995 >&Purity/*FRt*SRqqt1FBt/event_proportion‚0.27079132199287415 9"Purity/*FRt1SRqqt*FBt/event_purity‚0.0671287402510643 A&Purity/1FRt1SRqqt3FBt/event_proportion‚0.00025201612152159214 ("Purity/3FRt1SRqqt0FBt/event_purity‚0 -jet/accuracy_4_of_5‚0.0029325513169169426 particle/accuracy_4_of_4‚1 particle/accuracy_2_of_6‚1 7 Purity/3FRt1SRqqt2FBt/FRt_purity‚0.0833333358168602 8 Purity/*FRt3SRqqt0FBt/FRt_purity‚0.14047223329544067 ?&Purity/0FRt3SRqqt0FBt/event_proportion‚0.006111391354352236 & Purity/2FRt*SRqqt1FBt/FBt_purity‚1 @&Purity/3FRt*SRqqt1FBt/event_proportion‚0.0024571572430431843 ,&Purity/3FRt4SRqqt2FBt/event_proportion‚0 ("Purity/0FRt3SRqqt1FBt/event_purity‚0 & Purity/1FRt0SRqqt4FBt/FRt_purity‚0 ("Purity/4FRt1SRqqt0FBt/SRqqt_purity‚0 :"Purity/*FRt0SRqqt2FBt/event_purity‚0.22911956906318665 & Purity/1FRt3SRqqt*FBt/FBt_purity‚1 8 Purity/2FRt0SRqqt*FBt/FRt_purity‚0.11734628677368164 ("Purity/2FRt0SRqqt2FBt/event_purity‚0 7 Purity/*FRt4SRqqt0FBt/FRt_purity‚0.0972222313284874 jet/accuracy_6_of_7‚0 & Purity/3FRt1SRqqt2FBt/FBt_purity‚1 ("Purity/2FRt2SRqqt2FBt/SRqqt_purity‚0 9"Purity/*FRt1SRqqt*FBt/SRqqt_purity‚0.1749628484249115 8 Purity/*FRt2SRqqt1FBt/FRt_purity‚0.11085624992847443 @&Purity/*FRt4SRqqt*FBt/event_proportion‚0.0013230846961960196 :"Purity/0FRt2SRqqt0FBt/SRqqt_purity‚0.16971705853939056 ,&Purity/2FRt4SRqqt4FBt/event_proportion‚0 ("Purity/3FRt*SRqqt0FBt/event_purity‚0 7 Purity/3FRt2SRqqt1FBt/FRt_purity‚0.1111111119389534 :"Purity/*FRt0SRqqt*FBt/event_purity‚0.30659642815589905 @&Purity/1FRt3SRqqt2FBt/event_proportion‚6.300403038039804e-05 ("Purity/2FRt2SRqqt1FBt/event_purity‚0 @&Purity/4FRt0SRqqt0FBt/event_proportion‚6.300403038039804e-05 particle/accuracy_4_of_5‚1 jet/accuracy_3_of_7‚0 & Purity/2FRt*SRqqt3FBt/FRt_purity‚0 _step‚48 @&Purity/0FRt4SRqqt1FBt/event_proportion‚6.300403038039804e-05 ,&Purity/4FRt0SRqqt3FBt/event_proportion‚0 @&Purity/*FRt4SRqqt1FBt/event_proportion‚6.300403038039804e-05 7 Purity/0FRt0SRqqt2FBt/FBt_purity‚0.9857633709907532 @&Purity/0FRt2SRqqt2FBt/event_proportion‚0.0010080644860863686 ?&Purity/1FRt1SRqqt2FBt/event_proportion‚0.006111391354352236 7 Purity/1FRt*SRqqt*FBt/FBt_purity‚0.9890235066413879 ?&Purity/2FRt0SRqqt1FBt/event_proportion‚0.008946572430431843 ?&Purity/2FRt2SRqqt*FBt/event_proportion‚0.019468246027827263 & Purity/3FRt*SRqqt1FBt/FBt_purity‚1 particle/accuracy_1_of_3‚1 ,&Purity/1FRt4SRqqt3FBt/event_proportion‚0 ,&Purity/2FRt2SRqqt4FBt/event_proportion‚0 ("Purity/3FRt4SRqqt*FBt/SRqqt_purity‚0 ) Purity/1FRt1SRqqt3FBt/FRt_purity‚0.25 :"Purity/0FRt1SRqqt0FBt/SRqqt_purity‚0.15242965519428253 @&Purity/3FRt3SRqqt0FBt/event_proportion‚0.0005670362734235823 particle/accuracy_2_of_4‚1 9 Purity/3FRt0SRqqt0FBt/FRt_purity‚0.016339870169758797 9"Purity/0FRt0SRqqt1FBt/event_purity‚0.9993136525154114 7 Purity/1FRt0SRqqt3FBt/FBt_purity‚0.9047619104385376 & Purity/2FRt1SRqqt1FBt/FBt_purity‚1 & Purity/2FRt2SRqqt*FBt/FBt_purity‚1 :"Purity/3FRt*SRqqt1FBt/SRqqt_purity‚0.39444446563720703 9"Purity/*FRt*SRqqt2FBt/SRqqt_purity‚0.2613834738731384 @&Purity/*FRt3SRqqt2FBt/event_proportion‚6.300403038039804e-05 7 Purity/1FRt1SRqqt1FBt/FBt_purity‚0.9938275218009949 ,&Purity/2FRt3SRqqt4FBt/event_proportion‚0 ,&Purity/3FRt4SRqqt4FBt/event_proportion‚0 9"Purity/*FRt1SRqqt1FBt/event_purity‚0.0399739108979702 7 Purity/*FRt0SRqqt*FBt/FBt_purity‚0.9905406832695007 8 Purity/*FRt3SRqqt*FBt/FRt_purity‚0.13986510038375854 particle/accuracy_1_of_1‚1 ("Purity/2FRt3SRqqt1FBt/event_purity‚0 9"Purity/2FRt1SRqqt0FBt/SRqqt_purity‚0.1530253142118454 @&Purity/0FRt2SRqqt3FBt/event_proportion‚6.300403038039804e-05 :"Purity/1FRt*SRqqt1FBt/event_purity‚0.04602519050240517 <"Purity/*FRt2SRqqt0FBt/event_purity‚0.0023567776661366224 7 Purity/0FRt1SRqqt*FBt/FBt_purity‚0.9856379628181458 ?&Purity/2FRt0SRqqt*FBt/event_proportion‚0.037046369165182114 8 Purity/2FRt0SRqqt0FBt/FRt_purity‚0.07232969999313354 7 Purity/*FRt4SRqqt*FBt/FRt_purity‚0.0972222313284874 8 Purity/*FRt0SRqqt3FBt/FRt_purity‚0.13333334028720856 9"Purity/1FRt2SRqqt1FBt/SRqqt_purity‚0.2312595695257187 & Purity/3FRt*SRqqt*FBt/FBt_purity‚1 ("Purity/0FRt3SRqqt0FBt/event_purity‚0 ("Purity/3FRt4SRqqt*FBt/event_purity‚0 & Purity/2FRt0SRqqt3FBt/FBt_purity‚1 ("Purity/4FRt*SRqqt*FBt/event_purity‚0 7 Purity/1FRt*SRqqt1FBt/FBt_purity‚0.9973198175430298 *jet/accuracy_1_of_4‚0.7101631164550781 ("Purity/4FRt*SRqqt0FBt/event_purity‚0 ?&Purity/*FRt*SRqqt3FBt/event_proportion‚0.004473286215215921 >&Purity/*FRt3SRqqt0FBt/event_proportion‚0.01827116869390011 9"Purity/0FRt1SRqqt*FBt/SRqqt_purity‚0.1764245182275772 jet/accuracy_5_of_6‚0 % _timestamp‚1.7884149246458404e+09 =&Purity/*FRt1SRqqt*FBt/event_proportion‚0.4482736885547638 :"Purity/0FRt2SRqqt1FBt/SRqqt_purity‚0.22992832958698273 :"Purity/1FRt0SRqqt0FBt/event_purity‚0.05533648654818535 8 Purity/1FRt0SRqqt0FBt/FRt_purity‚0.05533648654818535 :"Purity/1FRt1SRqqt0FBt/SRqqt_purity‚0.13646410405635834 8"Purity/1FRt3SRqqt0FBt/SRqqt_purity‚0.143638014793396 ,&Purity/4FRt4SRqqt*FBt/event_proportion‚0 particle/accuracy_2_of_5‚1 :"Purity/*FRt*SRqqt0FBt/SRqqt_purity‚0.14628589153289795 7 Purity/1FRt1SRqqt*FBt/FBt_purity‚0.9837210178375244 @&Purity/1FRt3SRqqt1FBt/event_proportion‚0.0011340725468471646 ("Purity/0FRt*SRqqt4FBt/event_purity‚0 7 Purity/2FRt1SRqqt*FBt/FBt_purity‚0.9964157938957214 & Purity/2FRt2SRqqt1FBt/FBt_purity‚1 ("Purity/1FRt1SRqqt4FBt/event_purity‚0 ("Purity/4FRt*SRqqt*FBt/SRqqt_purity‚0 7 Purity/*FRt1SRqqt2FBt/FBt_purity‚0.9631656408309937 ("Purity/*FRt3SRqqt1FBt/event_purity‚0 7 Purity/0FRt1SRqqt1FBt/FBt_purity‚0.9956509470939636 * Purity/1FRt*SRqqt3FBt/FRt_purity‚0.125 ,&Purity/4FRt*SRqqt2FBt/event_proportion‚0 ,&Purity/4FRt*SRqqt4FBt/event_proportion‚0 ("Purity/0FRt2SRqqt2FBt/event_purity‚0 7 Purity/*FRt0SRqqt1FBt/FBt_purity‚0.9995640516281128 8 Purity/*FRt1SRqqt2FBt/FRt_purity‚0.12085485458374023 9"Purity/*FRt1SRqqt2FBt/SRqqt_purity‚0.2665006220340729 9"Purity/0FRt1SRqqt*FBt/event_purity‚0.1278877705335617 ("Purity/1FRt3SRqqt1FBt/event_purity‚0 ?&Purity/3FRt1SRqqt*FBt/event_proportion‚0.003717237850651145 ("Purity/*FRt2SRqqt2FBt/event_purity‚0 9"Purity/0FRt4SRqqt0FBt/SRqqt_purity‚0.2142857164144516 ?&Purity/1FRt0SRqqt1FBt/event_proportion‚0.049773186445236206 @&Purity/2FRt4SRqqt0FBt/event_proportion‚0.0003150201519019902 & Purity/4FRt1SRqqt0FBt/FRt_purity‚0 *"Purity/*FRt2SRqqt3FBt/SRqqt_purity‚0.5 ,&Purity/0FRt3SRqqt4FBt/event_proportion‚0 & Purity/1FRt2SRqqt1FBt/FBt_purity‚1 & Purity/1FRt0SRqqt4FBt/FBt_purity‚1 9"Purity/1FRt3SRqqt2FBt/SRqqt_purity‚0.3333333432674408 ;"Purity/*FRt2SRqqt1FBt/event_purity‚0.004032257944345474 ?&Purity/*FRt2SRqqt2FBt/event_proportion‚0.002079133177176118 :"Purity/1FRt1SRqqt1FBt/event_purity‚0.00601592194288969 & Purity/1FRt3SRqqt1FBt/FBt_purity‚1 ("Purity/0FRt2SRqqt3FBt/event_purity‚0 ("Purity/4FRt2SRqqt0FBt/SRqqt_purity‚0 ,&Purity/*FRt3SRqqt4FBt/event_proportion‚0 ;"Purity/0FRt2SRqqt*FBt/event_purity‚0.006569046527147293 =&Purity/1FRt1SRqqt0FBt/event_proportion‚0.1265120953321457 7 Purity/2FRt*SRqqt*FBt/FBt_purity‚0.9939655065536499 ("Purity/2FRt*SRqqt2FBt/event_purity‚0 9"Purity/2FRt1SRqqt1FBt/SRqqt_purity‚0.2003072202205658 ("Purity/2FRt3SRqqt*FBt/event_purity‚0 & Purity/4FRt1SRqqt*FBt/FRt_purity‚0 7 Purity/*FRt*SRqqt2FBt/FBt_purity‚0.9802989363670349 @&Purity/*FRt*SRqqt4FBt/event_proportion‚0.0003780241822823882 *jet/accuracy_1_of_6‚0.7280701398849487 ("Purity/3FRt4SRqqt0FBt/SRqqt_purity‚0 8 Purity/*FRt*SRqqt2FBt/FRt_purity‚0.20880936086177826 ,&Purity/4FRt3SRqqt4FBt/event_proportion‚0 ("Purity/0FRt4SRqqt1FBt/SRqqt_purity‚0 @&Purity/*FRt4SRqqt0FBt/event_proportion‚0.0012600806076079607 9"Purity/0FRt2SRqqt*FBt/SRqqt_purity‚0.1811172217130661 9"Purity/2FRt3SRqqt*FBt/SRqqt_purity‚0.1133016049861908 ("Purity/3FRt0SRqqt1FBt/event_purity‚0 ("Purity/3FRt2SRqqt1FBt/event_purity‚0 +"Purity/1FRt1SRqqt3FBt/SRqqt_purity‚0.25 7 Purity/0FRt2SRqqt3FBt/FBt_purity‚0.6666666865348816 :"Purity/*FRt1SRqqt0FBt/SRqqt_purity‚0.14680232107639313 @&Purity/*FRt1SRqqt3FBt/event_proportion‚0.0006930443341843784 8 Purity/1FRt0SRqqt*FBt/FRt_purity‚0.10913842916488647 ("Purity/1FRt1SRqqt2FBt/event_purity‚0 particle/accuracy_1_of_4‚1 8 Purity/2FRt2SRqqt1FBt/FRt_purity‚0.10982142388820648 @&Purity/4FRt0SRqqt*FBt/event_proportion‚6.300403038039804e-05 & Purity/*FRt1SRqqt4FBt/FRt_purity‚0 ,&Purity/*FRt4SRqqt3FBt/event_proportion‚0 @&Purity/2FRt*SRqqt3FBt/event_proportion‚6.300403038039804e-05 (particle/f_score‚0.24809974431991577 particle/accuracy_5_of_6‚1 8 Purity/*FRt0SRqqt1FBt/FRt_purity‚0.20683792233467102 :"Purity/*FRt3SRqqt0FBt/SRqqt_purity‚0.14315751194953918 :"Purity/0FRt1SRqqt2FBt/SRqqt_purity‚0.25503969192504883 9"Purity/1FRt4SRqqt*FBt/SRqqt_purity‚0.0833333358168602 ?&Purity/1FRt3SRqqt*FBt/event_proportion‚0.009135584346950054 /"Purity/*FRt4SRqqt*FBt/SRqqt_purity‚0.109375 -"Purity/0FRt4SRqqt*FBt/SRqqt_purity‚0.1875 jet/accuracy_7_of_7‚0 ("Purity/4FRt0SRqqt0FBt/event_purity‚0 :"Purity/*FRt*SRqqt1FBt/event_purity‚0.34903016686439514 6 Purity/*FRt*SRqqt3FBt/FBt_purity‚0.921455979347229 9"Purity/*FRt0SRqqt1FBt/event_purity‚0.6212626099586487 ?&Purity/*FRt3SRqqt1FBt/event_proportion‚0.002142137149348855 8 Purity/1FRt*SRqqt1FBt/FRt_purity‚0.18393653631210327 >&Purity/2FRt*SRqqt0FBt/event_proportion‚0.07579384744167328 ("Purity/1FRt4SRqqt0FBt/event_purity‚0 ("Purity/1FRt1SRqqt4FBt/SRqqt_purity‚0 & Purity/0FRt4SRqqt1FBt/FBt_purity‚1 & Purity/*FRt3SRqqt2FBt/FRt_purity‚0 & Purity/*FRt3SRqqt2FBt/FBt_purity‚1 8"Purity/0FRt0SRqqt*FBt/event_purity‚0.809985339641571 @&Purity/2FRt0SRqqt3FBt/event_proportion‚6.300403038039804e-05 ?&Purity/2FRt1SRqqt2FBt/event_proportion‚0.001890120911411941 ("Purity/2FRt2SRqqt0FBt/event_purity‚0 ,&Purity/3FRt*SRqqt3FBt/event_proportion‚0 ("Purity/*FRt4SRqqt*FBt/event_purity‚0 ("Purity/0FRt4SRqqt*FBt/event_purity‚0 jet/accuracy_6_of_6‚0 =&Purity/*FRt0SRqqt*FBt/event_proportion‚0.3558467626571655 9"Purity/0FRt*SRqqt1FBt/event_purity‚0.6072275638580322 8 Purity/3FRt1SRqqt0FBt/FRt_purity‚0.05833332985639572 ,jet/accuracy_3_of_3‚0.010062775574624538 *jet/accuracy_2_of_4‚0.1807018667459488 & Purity/0FRt2SRqqt2FBt/FBt_purity‚1 ("Purity/1FRt4SRqqt*FBt/event_purity‚0 & Purity/4FRt2SRqqt*FBt/FRt_purity‚0 7 Purity/*FRt2SRqqt*FBt/FBt_purity‚0.9936702251434326 :"Purity/1FRt2SRqqt0FBt/SRqqt_purity‚0.14020393788814545 6 Purity/2FRt0SRqqt2FBt/FBt_purity‚0.988095223903656 A&Purity/3FRt1SRqqt2FBt/event_proportion‚0.00025201612152159214 ,&Purity/3FRt3SRqqt2FBt/event_proportion‚0 ,&Purity/4FRt4SRqqt0FBt/event_proportion‚0 *jet/accuracy_1_of_2‚0.5453732013702393 & Purity/3FRt2SRqqt*FBt/FBt_purity‚1 7 Purity/2FRt0SRqqt2FBt/FRt_purity‚0.2422618865966797 ,&Purity/4FRt*SRqqt3FBt/event_proportion‚0 *"Purity/0FRt2SRqqt3FBt/SRqqt_purity‚0.5 ("Purity/4FRt0SRqqt*FBt/event_purity‚0 ,&Purity/4FRt3SRqqt1FBt/event_proportion‚0 * Purity/0FRt0SRqqt4FBt/FBt_purity‚0.875 & Purity/*FRt*SRqqt4FBt/FRt_purity‚0 & Purity/3FRt0SRqqt2FBt/FRt_purity‚0 =&Purity/0FRt1SRqqt0FBt/event_proportion‚0.1694808453321457 9"Purity/1FRt*SRqqt0FBt/SRqqt_purity‚0.1371915489435196 8 Purity/3FRt1SRqqt*FBt/FRt_purity‚0.09362141042947769 ,&Purity/4FRt1SRqqt1FBt/event_proportion‚0 ,&Purity/4FRt1SRqqt3FBt/event_proportion‚0 particle/accuracy_2_of_2‚1 particle/accuracy_1_of_5‚1 ("Purity/3FRt0SRqqt2FBt/event_purity‚0 =&Purity/*FRt*SRqqt*FBt/event_proportion‚0.9591733813285828 :"Purity/0FRt*SRqqt0FBt/SRqqt_purity‚0.15784071385860443 8 Purity/1FRt2SRqqt1FBt/FRt_purity‚0.13072197139263153 @&Purity/4FRt2SRqqt*FBt/event_proportion‚6.300403038039804e-05 particle/accuracy_3_of_7‚1 ("Purity/3FRt*SRqqt2FBt/event_purity‚0 & Purity/3FRt0SRqqt2FBt/FBt_purity‚1 ("Purity/*FRt2SRqqt3FBt/event_purity‚0 ,&Purity/4FRt2SRqqt1FBt/event_proportion‚0 9"Purity/*FRt3SRqqt2FBt/SRqqt_purity‚0.3333333432674408 ("Purity/1FRt1SRqqt3FBt/event_purity‚0 9"Purity/0FRt*SRqqt2FBt/event_purity‚0.2407081425189972 ,&Purity/4FRt4SRqqt2FBt/event_proportion‚0 )particle/specificity‚0.93961101770401 ("Purity/*FRt4SRqqt1FBt/SRqqt_purity‚0 ("Purity/2FRt0SRqqt3FBt/event_purity‚0 6 Purity/*FRt1SRqqt3FBt/FBt_purity‚0.920634925365448 & Purity/4FRt2SRqqt0FBt/FRt_purity‚0 6 Purity/0FRt*SRqqt*FBt/FBt_purity‚0.988025426864624 >&Purity/1FRt2SRqqt*FBt/event_proportion‚0.05727066472172737 8 Purity/1FRt3SRqqt1FBt/FRt_purity‚0.32222220301628113 ("Purity/2FRt3SRqqt0FBt/event_purity‚0 ,&Purity/4FRt1SRqqt4FBt/event_proportion‚0 *jet/accuracy_2_of_5‚0.2317090779542923 ;"Purity/*FRt*SRqqt0FBt/event_purity‚0.054504506289958954 ) Purity/*FRt3SRqqt1FBt/FRt_purity‚0.25 7 Purity/1FRt*SRqqt2FBt/FRt_purity‚0.2651996612548828 ,&Purity/3FRt1SRqqt4FBt/event_proportion‚0 A&Purity/3FRt4SRqqt0FBt/event_proportion‚0.00012600806076079607ã³s4<Ó ˜¥ jet/accuracy_3_of_7‚0 & Purity/2FRt*SRqqt3FBt/FRt_purity‚0 _step‚48 @&Purity/0FRt4SRqqt1FBt/event_proportion‚6.300403038039804e-05 ,&Purity/4FRt0SRqqt3FBt/event_proportion‚0 @&Purity/*FRt4SRqqt1FBt/event_proportion‚6.300403038039804e-05 7 Purity/0FRt0SRqqt2FBt/FBt_purity‚0.9857633709907532 @&Purity/0FRt2SRqqt2FBt/event_proportion‚0.0010080644860863686 ?&Purity/1FRt1SRqqt2FBt/event_proportion‚0.006111391354352236 7 Purity/1FRt*SRqqt*FBt/FBt_purity‚0.9890235066413879 ?&Purity/2FRt0SRqqt1FBt/event_proportion‚0.008946572430431843 ?&Purity/2FRt2SRqqt*FBt/event_proportion‚0.019468246027827263 & Purity/3FRt*SRqqt1FBt/FBt_purity‚1 particle/accuracy_1_of_3‚1 ,&Purity/1FRt4SRqqt3FBt/event_proportion‚0 ,&Purity/2FRt2SRqqt4FBt/event_proportion‚0 ("Purity/3FRt4SRqqt*FBt/SRqqt_purity‚0 ) Purity/1FRt1SRqqt3FBt/FRt_purity‚0.25 :"Purity/0FRt1SRqqt0FBt/SRqqt_purity‚0.15242965519428253 @&Purity/3FRt3SRqqt0FBt/event_proportion‚0.0005670362734235823 particle/accuracy_2_of_4‚1 9 Purity/3FRt0SRqqt0FBt/FRt_purity‚0.016339870169758797 9"Purity/0FRt0SRqqt1FBt/event_purity‚0.9993136525154114 7 Purity/1FRt0SRqqt3FBt/FBt_purity‚0.9047619104385376 & Purity/2FRt1SRqqt1FBt/FBt_purity‚1 & Purity/2FRt2SRqqt*FBt/FBt_purity‚1 :"Purity/3FRt*SRqqt1FBt/SRqqt_purity‚0.39444446563720703 9"Purity/*FRt*SRqqt2FBt/SRqqt_purity‚0.2613834738731384 @&Purity/*FRt3SRqqt2FBt/event_proportion‚6.300403038039804e-05 7 Purity/1FRt1SRqqt1FBt/FBt_purity‚0.9938275218009949 ,&Purity/2FRt3SRqqt4FBt/event_proportion‚0 ,&Purity/3FRt4SRqqt4FBt/event_proportion‚0 9"Purity/*FRt1SRqqt1FBt/event_purity‚0.0399739108979702 7 Purity/*FRt0SRqqt*FBt/FBt_purity‚0.9905406832695007 8 Purity/*FRt3SRqqt*FBt/FRt_purity‚0.13986510038375854 particle/accuracy_1_of_1‚1 ("Purity/2FRt3SRqqt1FBt/event_purity‚0 9"Purity/2FRt1SRqqt0FBt/SRqqt_purity‚0.1530253142118454 @&Purity/0FRt2SRqqt3FBt/event_proportion‚6.300403038039804e-05 :"Purity/1FRt*SRqqt1FBt/event_purity‚0.04602519050240517 <"Purity/*FRt2SRqqt0FBt/event_purity‚0.0023567776661366224 7 Purity/0FRt1SRqqt*FBt/FBt_purity‚0.9856379628181458 ?&Purity/2FRt0SRqqt*FBt/event_proportion‚0.037046369165182114 8 Purity/2FRt0SRqqt0FBt/FRt_purity‚0.07232969999313354 7 Purity/*FRt4SRqqt*FBt/FRt_purity‚0.0972222313284874 8 Purity/*FRt0SRqqt3FBt/FRt_purity‚0.13333334028720856 9"Purity/1FRt2SRqqt1FBt/SRqqt_purity‚0.2312595695257187 & Purity/3FRt*SRqqt*FBt/FBt_purity‚1 ("Purity/0FRt3SRqqt0FBt/event_purity‚0 ("Purity/3FRt4SRqqt*FBt/event_purity‚0 & Purity/2FRt0SRqqt3FBt/FBt_purity‚1 ("Purity/4FRt*SRqqt*FBt/event_purity‚0 7 Purity/1FRt*SRqqt1FBt/FBt_purity‚0.9973198175430298 *jet/accuracy_1_of_4‚0.7101631164550781 ("Purity/4FRt*SRqqt0FBt/event_purity‚0 ?&Purity/*FRt*SRqqt3FBt/event_proportion‚0.004473286215215921 >&Purity/*FRt3SRqqt0FBt/event_proportion‚0.01827116869390011 9"Purity/0FRt1SRqqt*FBt/SRqqt_purity‚0.1764245182275772 jet/accuracy_5_of_6‚0 % _timestamp‚1.7884149246458404e+09 =&Purity/*FRt1SRqqt*FBt/event_proportion‚0.4482736885547638 :"Purity/0FRt2SRqqt1FBt/SRqqt_purity‚0.22992832958698273 :"Purity/1FRt0SRqqt0FBt/event_purity‚0.05533648654818535 8 Purity/1FRt0SRqqt0FBt/FRt_purity‚0.05533648654818535 :"Purity/1FRt1SRqqt0FBt/SRqqt_purity‚0.13646410405635834 8"Purity/1FRt3SRqqt0FBt/SRqqt_purity‚0.143638014793396 ,&Purity/4FRt4SRqqt*FBt/event_proportion‚0 particle/accuracy_2_of_5‚1 :"Purity/*FRt*SRqqt0FBt/SRqqt_purity‚0.14628589153289795 7 Purity/1FRt1SRqqt*FBt/FBt_purity‚0.9837210178375244 @&Purity/1FRt3SRqqt1FBt/event_proportion‚0.0011340725468471646 ("Purity/0FRt*SRqqt4FBt/event_purity‚0 7 Purity/2FRt1SRqqt*FBt/FBt_purity‚0.9964157938957214 & Purity/2FRt2SRqqt1FBt/FBt_purity‚1 ("Purity/1FRt1SRqqt4FBt/event_purity‚0 ("Purity/4FRt*SRqqt*FBt/SRqqt_purity‚0 7 Purity/*FRt1SRqqt2FBt/FBt_purity‚0.9631656408309937 ("Purity/*FRt3SRqqt1FBt/event_purity‚0 7 Purity/0FRt1SRqqt1FBt/FBt_purity‚0.9956509470939636 * Purity/1FRt*SRqqt3FBt/FRt_purity‚0.125 ,&Purity/4FRt*SRqqt2FBt/event_proportion‚0 ,&Purity/4FRt*SRqqt4FBt/event_proportion‚0 ("Purity/0FRt2SRqqt2FBt/event_purity‚0 7 Purity/*FRt0SRqqt1FBt/FBt_purity‚0.9995640516281128 8 Purity/*FRt1SRqqt2FBt/FRt_purity‚0.12085485458374023 9"Purity/*FRt1SRqqt2FBt/SRqqt_purity‚0.2665006220340729 9"Purity/0FRt1SRqqt*FBt/event_purity‚0.1278877705335617 ("Purity/1FRt3SRqqt1FBt/event_purity‚0 ?&Purity/3FRt1SRqqt*FBt/event_proportion‚0.003717237850651145 ("Purity/*FRt2SRqqt2FBt/event_purity‚0 9"Purity/0FRt4SRqqt0FBt/SRqqt_purity‚0.2142857164144516 ?&Purity/1FRt0SRqqt1FBt/event_proportion‚0.049773186445236206 @&Purity/2FRt4SRqqt0FBt/event_proportion‚0.0003150201519019902 & Purity/4FRt1SRqqt0FBt/FRt_purity‚0 *"Purity/*FRt2SRqqt3FBt/SRqqt_purity‚0.5 ,&Purity/0FRt3SRqqt4FBt/event_proportion‚0 & Purity/1FRt2SRqqt1FBt/FBt_purity‚1 & Purity/1FRt0SRqqt4FBt/FBt_purity‚1 9"Purity/1FRt3SRqqt2FBt/SRqqt_purity‚0.3333333432674408 ;"Purity/*FRt2SRqqt1FBt/event_purity‚0.004032257944345474 ?&Purity/*FRt2SRqqt2FBt/event_proportion‚0.002079133177176118 :"Purity/1FRt1SRqqt1FBt/event_purity‚0.00601592194288969 & Purity/1FRt3SRqqt1FBt/FBt_purity‚1 ("Purity/0FRt2SRqqt3FBt/event_purity‚0 ("Purity/4FRt2SRqqt0FBt/SRqqt_purity‚0 ,&Purity/*FRt3SRqqt4FBt/event_proportion‚0 ;"Purity/0FRt2SRqqt*FBt/event_purity‚0.006569046527147293 =&Purity/1FRt1SRqqt0FBt/event_proportion‚0.1265120953321457 7 Purity/2FRt*SRqqt*FBt/FBt_purity‚0.9939655065536499 ("Purity/2FRt*SRqqt2FBt/event_purity‚0 9"Purity/2FRt1SRqqt1FBt/SRqqt_purity‚0.2003072202205658 ("Purity/2FRt3SRqqt*FBt/event_purity‚0 & Purity/4FRt1SRqqt*FBt/FRt_purity‚0 7 Purity/*FRt*SRqqt2FBt/FBt_purity‚0.9802989363670349 @&Purity/*FRt*SRqqt4FBt/event_proportion‚0.0003780241822823882 *jet/accuracy_1_of_6‚0.7280701398849487 ("Purity/3FRt4SRqqt0FBt/SRqqt_purity‚0 8 Purity/*FRt*SRqqt2FBt/FRt_purity‚0.20880936086177826 ,&Purity/4FRt3SRqqt4FBt/event_proportion‚0 ("Purity/0FRt4SRqqt1FBt/SRqqt_purity‚0 @&Purity/*FRt4SRqqt0FBt/event_proportion‚0.0012600806076079607 9"Purity/0FRt2SRqqt*FBt/SRqqt_purity‚0.1811172217130661 9"Purity/2FRt3SRqqt*FBt/SRqqt_purity‚0.1133016049861908 ("Purity/3FRt0SRqqt1FBt/event_purity‚0 ("Purity/3FRt2SRqqt1FBt/event_purity‚0 +"Purity/1FRt1SRqqt3FBt/SRqqt_purity‚0.25 7 Purity/0FRt2SRqqt3FBt/FBt_purity‚0.6666666865348816 :"Purity/*FRt1SRqqt0FBt/SRqqt_purity‚0.14680232107639313 @&Purity/*FRt1SRqqt3FBt/event_proportion‚0.0006930443341843784 8 Purity/1FRt0SRqqt*FBt/FRt_purity‚0.10913842916488647 ("Purity/1FRt1SRqqt2FBt/event_purity‚0 particle/accuracy_1_of_4‚1 8 Purity/2FRt2SRqqt1FBt/FRt_purity‚0.10982142388820648 @&Purity/4FRt0SRqqt*FBt/event_proportion‚6.300403038039804e-05 & Purity/*FRt1SRqqt4FBt/FRt_purity‚0 ,&Purity/*FRt4SRqqt3FBt/event_proportion‚0 @&Purity/2FRt*SRqqt3FBt/event_proportion‚6.300403038039804e-05 (particle/f_score‚0.24809974431991577 particle/accuracy_5_of_6‚1 8 Purity/*FRt0SRqqt1FBt/FRt_purity‚0.20683792233467102 :"Purity/*FRt3SRqqt0FBt/SRqqt_purity‚0.14315751194953918 :"Purity/0FRt1SRqqt2FBt/SRqqt_purity‚0.25503969192504883 9"Purity/1FRt4SRqqt*FBt/SRqqt_purity‚0.0833333358168602 ?&Purity/1FRt3SRqqt*FBt/event_proportion‚0.009135584346950054 /"Purity/*FRt4SRqqt*FBt/SRqqt_purity‚0.109375 -"Purity/0FRt4SRqqt*FBt/SRqqt_purity‚0.1875 jet/accuracy_7_of_7‚0 ("Purity/4FRt0SRqqt0FBt/event_purity‚0 :"Purity/*FRt*SRqqt1FBt/event_purity‚0.34903016686439514 6 Purity/*FRt*SRqqt3FBt/FBt_purity‚0.921455979347229 9"Purity/*FRt0SRqqt1FBt/event_purity‚0.6212626099586487 ?&Purity/*FRt3SRqqt1FBt/event_proportion‚0.002142137149348855 8 Purity/1FRt*SRqqt1FBt/FRt_purity‚0.18393653631210327 >&Purity/2FRt*SRqqt0FBt/event_proportion‚0.07579384744167328 ("Purity/1FRt4SRqqt0FBt/event_purity‚0 ("Purity/1FRt1SRqqt4FBt/SRqqt_purity‚0 & Purity/0FRt4SRqqt1FBt/FBt_purity‚1 & Purity/*FRt3SRqqt2FBt/FRt_purity‚0 & Purity/*FRt3SRqqt2FBt/FBt_purity‚1 8"Purity/0FRt0SRqqt*FBt/event_purity‚0.809985339641571 @&Purity/2FRt0SRqqt3FBt/event_proportion‚6.300403038039804e-05 ?&Purity/2FRt1SRqqt2FBt/event_proportion‚0.001890120911411941 ("Purity/2FRt2SRqqt0FBt/event_purity‚0 ,&Purity/3FRt*SRqqt3FBt/event_proportion‚0 ("Purity/*FRt4SRqqt*FBt/event_purity‚0 ("Purity/0FRt4SRqqt*FBt/event_purity‚0 jet/accuracy_6_of_6‚0 =&Purity/*FRt0SRqqt*FBt/event_proportion‚0.3558467626571655 9"Purity/0FRt*SRqqt1FBt/event_purity‚0.6072275638580322 8 Purity/3FRt1SRqqt0FBt/FRt_purity‚0.05833332985639572 ,jet/accuracy_3_of_3‚0.010062775574624538 *jet/accuracy_2_of_4‚0.1807018667459488 & Purity/0FRt2SRqqt2FBt/FBt_purity‚1 ("Purity/1FRt4SRqqt*FBt/event_purity‚0 & Purity/4FRt2SRqqt*FBt/FRt_purity‚0 7 Purity/*FRt2SRqqt*FBt/FBt_purity‚0.9936702251434326 :"Purity/1FRt2SRqqt0FBt/SRqqt_purity‚0.14020393788814545 6 Purity/2FRt0SRqqt2FBt/FBt_purity‚0.988095223903656 A&Purity/3FRt1SRqqt2FBt/event_proportion‚0.00025201612152159214 ,&Purity/3FRt3SRqqt2FBt/event_proportion‚0 ,&Purity/4FRt4SRqqt0FBt/event_proportion‚0 *jet/accuracy_1_of_2‚0.5453732013702393 & Purity/3FRt2SRqqt*FBt/FBt_purity‚1 7 Purity/2FRt0SRqqt2FBt/FRt_purity‚0.2422618865966797 ,&Purity/4FRt*SRqqt3FBt/event_proportion‚0 *"Purity/0FRt2SRqqt3FBt/SRqqt_purity‚0.5 ("Purity/4FRt0SRqqt*FBt/event_purity‚0 ,&Purity/4FRt3SRqqt1FBt/event_proportion‚0 * Purity/0FRt0SRqqt4FBt/FBt_purity‚0.875 & Purity/*FRt*SRqqt4FBt/FRt_purity‚0 & Purity/3FRt0SRqqt2FBt/FRt_purity‚0 =&Purity/0FRt1SRqqt0FBt/event_proportion‚0.1694808453321457 9"Purity/1FRt*SRqqt0FBt/SRqqt_purity‚0.1371915489435196 8 Purity/3FRt1SRqqt*FBt/FRt_purity‚0.09362141042947769 ,&Purity/4FRt1SRqqt1FBt/event_proportion‚0 ,&Purity/4FRt1SRqqt3FBt/event_proportion‚0 particle/accuracy_2_of_2‚1 particle/accuracy_1_of_5‚1 ("Purity/3FRt0SRqqt2FBt/event_purity‚0 =&Purity/*FRt*SRqqt*FBt/event_proportion‚0.9591733813285828 :"Purity/0FRt*SRqqt0FBt/SRqqt_purity‚0.15784071385860443 8 Purity/1FRt2SRqqt1FBt/FRt_purity‚0.13072197139263153 @&Purity/4FRt2SRqqt*FBt/event_proportion‚6.300403038039804e-05 particle/accuracy_3_of_7‚1 ("Purity/3FRt*SRqqt2FBt/event_purity‚0 & Purity/3FRt0SRqqt2FBt/FBt_purity‚1 ("Purity/*FRt2SRqqt3FBt/event_purity‚0 ,&Purity/4FRt2SRqqt1FBt/event_proportion‚0 9"Purity/*FRt3SRqqt2FBt/SRqqt_purity‚0.3333333432674408 ("Purity/1FRt1SRqqt3FBt/event_purity‚0 9"Purity/0FRt*SRqqt2FBt/event_purity‚0.2407081425189972 ,&Purity/4FRt4SRqqt2FBt/event_proportion‚0 )particle/specificity‚0.93961101770401 ("Purity/*FRt4SRqqt1FBt/SRqqt_purity‚0 ("Purity/2FRt0SRqqt3FBt/event_purity‚0 6 Purity/*FRt1SRqqt3FBt/FBt_purity‚0.920634925365448 & Purity/4FRt2SRqqt0FBt/FRt_purity‚0 6 Purity/0FRt*SRqqt*FBt/FBt_purity‚0.988025426864624 >&Purity/1FRt2SRqqt*FBt/event_proportion‚0.05727066472172737 8 Purity/1FRt3SRqqt1FBt/FRt_purity‚0.32222220301628113 ("Purity/2FRt3SRqqt0FBt/event_purity‚0 ,&Purity/4FRt1SRqqt4FBt/event_proportion‚0 *jet/accuracy_2_of_5‚0.2317090779542923 ;"Purity/*FRt*SRqqt0FBt/event_purity‚0.054504506289958954 ) Purity/*FRt3SRqqt1FBt/FRt_purity‚0.25 7 Purity/1FRt*SRqqt2FBt/FRt_purity‚0.2651996612548828 ,&Purity/3FRt1SRqqt4FBt/event_proportion‚0 A&Purity/3FRt4SRqqt0FBt/event_proportion‚0.00012600806076079607 =&Purity/1FRt*SRqqt*FBt/event_proportion‚0.4366179406642914 7 Purity/1FRt*SRqqt2FBt/FBt_purity‚0.9632810950279236 :"Purity/1FRt2SRqqt*FBt/SRqqt_purity‚0.15548980236053467 ,&Purity/4FRt3SRqqt3FBt/event_proportion‚0 ) Purity/*FRt1SRqqt3FBt/FRt_purity‚0.25 ?&Purity/1FRt0SRqqt2FBt/event_proportion‚0.015120967291295528 ("Purity/1FRt0SRqqt3FBt/event_purity‚0 ?&Purity/2FRt*SRqqt2FBt/event_proportion‚0.005418346729129553 9"Purity/0FRt3SRqqt1FBt/SRqqt_purity‚0.3749999701976776 ,&Purity/2FRt3SRqqt3FBt/event_proportion‚0 ("Purity/0FRt4SRqqt0FBt/event_purity‚0 A&Purity/0FRt*SRqqt4FBt/event_proportion‚0.00025201612152159214 ?&Purity/0FRt0SRqqt2FBt/event_proportion‚0.029674898833036423 ("Purity/1FRt2SRqqt*FBt/event_purity‚0 ,&Purity/4FRt4SRqqt3FBt/event_proportion‚0 ("Purity/1FRt*SRqqt4FBt/event_purity‚0 & Purity/2FRt2SRqqt2FBt/FRt_purity‚0 @&Purity/0FRt*SRqqt3FBt/event_proportion‚0.0030871976632624865 >&Purity/1FRt0SRqqt0FBt/event_proportion‚0.13142640888690948 9"Purity/2FRt2SRqqt1FBt/SRqqt_purity‚0.2874999940395355 ("Purity/4FRt*SRqqt0FBt/SRqqt_purity‚0 & Purity/2FRt2SRqqt2FBt/FBt_purity‚1 7 Purity/*FRt0SRqqt2FBt/FRt_purity‚0.2667907178401947 7 Purity/*FRt1SRqqt1FBt/FBt_purity‚0.9953466653823853 8 Purity/*FRt2SRqqt0FBt/FRt_purity‚0.11591441929340363 ?&Purity/0FRt0SRqqt3FBt/event_proportion‚0.002583165420219302 6 Purity/0FRt1SRqqt2FBt/FBt_purity‚0.961388885974884 @&Purity/1FRt2SRqqt2FBt/event_proportion‚0.0010080644860863686 ("Purity/2FRt1SRqqt2FBt/event_purity‚0 ,&Purity/2FRt3SRqqt2FBt/event_proportion‚0 7 Purity/*FRt0SRqqt2FBt/FBt_purity‚0.9855976104736328 :"Purity/1FRt*SRqqt2FBt/event_purity‚0.06994287669658661 <"Purity/1FRt1SRqqt0FBt/event_purity‚0.0025529316626489162 ("Purity/1FRt3SRqqt0FBt/event_purity‚0 ,&Purity/1FRt3SRqqt4FBt/event_proportion‚0 8 Purity/3FRt2SRqqt0FBt/FRt_purity‚0.06763284653425217 particle/accuracy_3_of_5‚1 ("Purity/*FRt1SRqqt3FBt/event_purity‚0 ("Purity/1FRt2SRqqt2FBt/event_purity‚0 7 Purity/1FRt2SRqqt2FBt/FRt_purity‚0.2222222238779068 ("Purity/3FRt4SRqqt0FBt/event_purity‚0 9"Purity/*FRt*SRqqt3FBt/SRqqt_purity‚0.4523809552192688 A&Purity/1FRt*SRqqt4FBt/event_proportion‚0.00012600806076079607 >&Purity/1FRt0SRqqt*FBt/event_proportion‚0.19745463132858276 :"Purity/2FRt2SRqqt*FBt/SRqqt_purity‚0.15444688498973846 & Purity/3FRt1SRqqt*FBt/FBt_purity‚1 7 Purity/1FRt2SRqqt2FBt/FBt_purity‚0.9583333134651184 & Purity/2FRt3SRqqt1FBt/FBt_purity‚1 ("Purity/3FRt1SRqqt2FBt/event_purity‚0 +"Purity/1FRt*SRqqt3FBt/SRqqt_purity‚0.25 8 Purity/*FRt2SRqqt*FBt/FRt_purity‚0.11843079328536987 >&Purity/0FRt2SRqqt*FBt/event_proportion‚0.05361643061041832 7 Purity/3FRt2SRqqt*FBt/FRt_purity‚0.0659722238779068 @&Purity/3FRt2SRqqt1FBt/event_proportion‚0.0003780241822823882 particle/accuracy_4_of_6‚1 & Purity/1FRt3SRqqt2FBt/FBt_purity‚1 =&Purity/*FRt*SRqqt0FBt/event_proportion‚0.6160534024238586 >&Purity/*FRt*SRqqt2FBt/event_proportion‚0.06747731566429138 8 Purity/*FRt*SRqqt3FBt/FRt_purity‚0.11764705926179886 :"Purity/*FRt1SRqqt0FBt/event_purity‚0.07913762331008911 * Purity/0FRt*SRqqt4FBt/FBt_purity‚0.875 & Purity/1FRt3SRqqt2FBt/FRt_purity‚0 7 Purity/*FRt*SRqqt1FBt/FBt_purity‚0.9975016117095947 ("Purity/2FRt2SRqqt*FBt/event_purity‚0 ("Purity/3FRt*SRqqt1FBt/event_purity‚0 ,&Purity/3FRt*SRqqt4FBt/event_proportion‚0 7 Purity/2FRt0SRqqt1FBt/FRt_purity‚0.2100132405757904 ?&Purity/3FRt*SRqqt*FBt/event_proportion‚0.009954636916518211 6validation_average_jet_accuracy‚0.2880053222179413 :"Purity/*FRt1SRqqt1FBt/SRqqt_purity‚0.25053146481513977 7 Purity/0FRt0SRqqt*FBt/FBt_purity‚0.9896841645240784 7 Purity/3FRt0SRqqt1FBt/FRt_purity‚0.2499999850988388 :"Purity/2FRt3S‡ †CkVRqqt1FBt/SRqqt_purity‚0.06666667014360428 particle/accuracy_6_of_7‚1 ("Purity/4FRt2SRqqt0FBt/event_purity‚0 :"Purity/1FRt*SRqqt1FBt/SRqqt_purity‚0.24542465806007385 ?&Purity/2FRt3SRqqt*FBt/event_proportion‚0.004032257944345474 @&Purity/3FRt1SRqqt1FBt/event_proportion‚0.0009450604557059705 @&Purity/4FRt2SRqqt0FBt/event_proportion‚6.300403038039804e-05 ("Purity/*FRt1SRqqt4FBt/event_purity‚0 ?&Purity/0FRt1SRqqt2FBt/event_proportion‚0.008757560513913631 particle/accuracy_1_of_2‚1 & Purity/3FRt4SRqqt*FBt/FRt_purity‚0 9"Purity/1FRt4SRqqt0FBt/SRqqt_purity‚0.0833333358168602 ) Purity/1FRt1SRqqt4FBt/FBt_purity‚0.75 ("Purity/1FRt3SRqqt*FBt/event_purity‚0 :"Purity/2FRt1SRqqt*FBt/SRqqt_purity‚0.17319801449775696 jet/accuracy_2_of_7‚0 & Purity/*FRt0SRqqt4FBt/FRt_purity‚0 :"Purity/2FRt2SRqqt0FBt/SRqqt_purity‚0.11015220731496811 8 Purity/3FRt*SRqqt*FBt/FRt_purity‚0.07696706056594849 @&Purity/3FRt1SRqqt0FBt/event_proportion‚0.0025201612152159214 9"Purity/0FRt3SRqqt*FBt/SRqqt_purity‚0.1863001137971878 * Purity/0FRt3SRqqt*FBt/FBt_purity‚0.875 ("Purity/1FRt0SRqqt4FBt/event_purity‚0 8 Purity/*FRt1SRqqt1FBt/FRt_purity‚0.19032436609268188 >&Purity/0FRt0SRqqt*FBt/event_proportion‚0.11857359111309052 ;"Purity/1FRt*SRqqt*FBt/event_purity‚0.031498633325099945 @&Purity/2FRt2SRqqt2FBt/event_proportion‚6.300403038039804e-05 ,&Purity/3FRt4SRqqt3FBt/event_proportion‚0 ,&Purity/4FRt2SRqqt3FBt/event_proportion‚0 :"Purity/*FRt*SRqqt2FBt/event_purity‚0.16306354105472565 :"Purity/1FRt3SRqqt*FBt/SRqqt_purity‚0.14480715990066528 ("Purity/2FRt1SRqqt*FBt/event_purity‚0 ,&Purity/4FRt0SRqqt4FBt/event_proportion‚0 ,jet/accuracy_3_of_4‚0.025847364217042923 & Purity/*FRt4SRqqt1FBt/FBt_purity‚1 +jet/accuracy_2_of_6‚0.25438597798347473 *"Purity/*FRt1SRqqt3FBt/SRqqt_purity‚0.5 ?&Purity/*FRt0SRqqt3FBt/event_proportion‚0.003717237850651145 6 Purity/*FRt3SRqqt1FBt/FBt_purity‚0.976190447807312 8 Purity/2FRt*SRqqt0FBt/FRt_purity‚0.09636475890874863 ("Purity/1FRt*SRqqt4FBt/SRqqt_purity‚0 & Purity/4FRt0SRqqt*FBt/FRt_purity‚0 ("Purity/*FRt3SRqqt0FBt/event_purity‚0 7 Purity/0FRt2SRqqt1FBt/FBt_purity‚0.9953917264938354 :"Purity/1FRt*SRqqt*FBt/SRqqt_purity‚0.16174755990505219 8 Purity/1FRt1SRqqt*FBt/FRt_purity‚0.14117175340652466 @&Purity/3FRt2SRqqt*FBt/event_proportion‚0.0028351815417408943 particle/accuracy_3_of_3‚1 >&Purity/*FRt0SRqqt1FBt/event_proportion‚0.14591734111309052 ,&Purity/0FRt1SRqqt4FBt/event_proportion‚0 =&Purity/1FRt1SRqqt*FBt/event_proportion‚0.1723790317773819 8 Purity/2FRt1SRqqt*FBt/FRt_purity‚0.12405509501695633 ("Purity/3FRt1SRqqt1FBt/event_purity‚0 ("Purity/0FRt3SRqqt*FBt/event_purity‚0 & Purity/1FRt*SRqqt4FBt/FRt_purity‚0 :"Purity/0FRt*SRqqt*FBt/event_purity‚0.30606314539909363 :"Purity/0FRt*SRqqt0FBt/event_purity‚0.11835914105176926 =&Purity/1FRt*SRqqt0FBt/event_proportion‚0.3135710656642914 & Purity/2FRt0SRqqt1FBt/FBt_purity‚1 & Purity/4FRt0SRqqt0FBt/FRt_purity‚0 7 Purity/*FRt*SRqqt1FBt/FRt_purity‚0.1892215609550476 7 Purity/1FRt*SRqqt3FBt/FBt_purity‚0.9166667461395264 7 Purity/2FRt*SRqqt*FBt/FRt_purity‚0.1228276863694191 ,particle/sensitivity‚0.18784886598587036 particle/accuracy_7_of_7‚0.5 ("Purity/2FRt2SRqqt2FBt/event_purity‚0 epoch‚0 ("Purity/1FRt*SRqqt3FBt/event_purity‚0 @&Purity/1FRt0SRqqt3FBt/event_proportion‚0.0010710685746744275 8 Purity/1FRt1SRqqt0FBt/FRt_purity‚0.13157279789447784 >&Purity/2FRt*SRqqt*FBt/event_proportion‚0.10244455933570862 particle/accuracy_2_of_3‚1 :"Purity/*FRt*SRqqt3FBt/event_purity‚0.05689655616879463 ?&Purity/2FRt*SRqqt1FBt/event_proportion‚0.021169355139136314 8 Purity/2FRt1SRqqt0FBt/FRt_purity‚0.10072343051433563 ,&Purity/2FRt1SRqqt3FBt/event_proportion‚0 9"Purity/2FRt3SRqqt0FBt/SRqqt_purity‚0.1170634999871254 & Purity/3FRt0SRqqt1FBt/FBt_purity‚1 :"Purity/3FRt1SRqqt*FBt/SRqqt_purity‚0.22839507460594177 ,&Purity/3FRt2SRqqt3FBt/event_proportion‚0 @&Purity/3FRt0SRqqt1FBt/event_proportion‚0.0011340725468471646 :"Purity/0FRt1SRqqt1FBt/SRqqt_purity‚0.24180883169174194 ("Purity/1FRt2SRqqt0FBt/event_purity‚0 ,&Purity/1FRt4SRqqt1FBt/event_proportion‚0 :"Purity/2FRt*SRqqt1FBt/event_purity‚0.00358422938734293 8 Purity/2FRt3SRqqt1FBt/FRt_purity‚0.15000000596046448 & Purity/3FRt4SRqqt0FBt/FRt_purity‚0 =&Purity/*FRt1SRqqt1FBt/event_proportion‚0.1001134067773819 :"Purity/0FRt*SRqqt*FBt/SRqqt_purity‚0.17750997841358185 7 Purity/1FRt0SRqqt*FBt/FBt_purity‚0.9918087124824524 ,&Purity/4FRt2SRqqt2FBt/event_proportion‚0 ,&Purity/2FRt0SRqqt4FBt/event_proportion‚0 9 Purity/3FRt*SRqqt0FBt/FRt_purity‚0.041507940739393234 ("Purity/3FRt1SRqqt*FBt/event_purity‚0 )jet/accuracy_3_of_5‚0.024527532979846 particle/accuracy_4_of_7‚1 ("Purity/*FRt1SRqqt4FBt/SRqqt_purity‚0 7 Purity/0FRt1SRqqt3FBt/FBt_purity‚0.8999999761581421 trainer/global_step‚603 @&Purity/3FRt2SRqqt0FBt/event_proportion‚0.0024571572430431843 ;"Purity/1FRt*SRqqt0FBt/event_purity‚0.024264488369226456 7 Purity/*FRt0SRqqt3FBt/FBt_purity‚0.9216270446777344 ,&Purity/4FRt*SRqqt1FBt/event_proportion‚0 jet/accuracy_5_of_7‚0 & Purity/2FRt0SRqqt3FBt/FRt_purity‚0 9 Purity/*FRt0SRqqt0FBt/FRt_purity‚0.058934058994054794 @&Purity/1FRt4SRqqt*FBt/event_proportion‚0.0003780241822823882 ,&Purity/2FRt4SRqqt3FBt/event_proportion‚0 *jet/accuracy_2_of_3‚0.1241641491651535 :"Purity/2FRt4SRqqt*FBt/SRqqt_purity‚0.05000000074505806 8 Purity/*FRt2SRqqt2FBt/FRt_purity‚0.20512820780277252 ("Purity/0FRt1SRqqt3FBt/event_purity‚0 ("Purity/1FRt3SRqqt2FBt/event_purity‚0 :"Purity/0FRt0SRqqt3FBt/event_purity‚0.10999999940395355 A&Purity/0FRt0SRqqt4FBt/event_proportion‚0.00025201612152159214 8 Purity/1FRt3SRqqt*FBt/FRt_purity‚0.14444445073604584 7 Purity/2FRt0SRqqt*FBt/FBt_purity‚0.9931899905204773 ,&Purity/4FRt3SRqqt*FBt/event_proportion‚0 +jet/accuracy_1_of_1‚0.30278247594833374 ("Purity/2FRt4SRqqt0FBt/event_purity‚0 7 Purity/*FRt2SRqqt2FBt/FBt_purity‚0.9736841917037964 :"Purity/1FRt0SRqqt2FBt/event_purity‚0.10420432686805725 8 Purity/1FRt2SRqqt*FBt/FRt_purity‚0.12346053868532181 9"Purity/2FRt*SRqqt2FBt/SRqqt_purity‚0.3114035129547119 8 Purity/2FRt2SRqqt0FBt/FRt_purity‚0.12143034487962723 & Purity/3FRt0SRqqt*FBt/FBt_purity‚1 ("Purity/*FRt*SRqqt4FBt/event_purity‚0 ("Purity/3FRt3SRqqt*FBt/event_purity‚0 9"Purity/3FRt3SRqqt0FBt/SRqqt_purity‚0.1428571492433548 >&Purity/0FRt0SRqqt0FBt/event_proportion‚0.04082661122083664 7 Purity/1FRt0SRqqt3FBt/FRt_purity‚0.1428571492433548 ?&Purity/2FRt0SRqqt2FBt/event_proportion‚0.003465221729129553 jet/accuracy_4_of_4‚0 ("Purity/4FRt2SRqqt*FBt/event_purity‚0 :"Purity/*FRt*SRqqt1FBt/SRqqt_purity‚0.24577125906944275 @&Purity/0FRt4SRqqt*FBt/event_proportion‚0.0005040322430431843 & Purity/1FRt0SRqqt1FBt/FBt_purity‚1 jet/accuracy_4_of_7‚0 ,&Purity/*FRt4SRqqt4FBt/event_proportion‚0 >&Purity/0FRt*SRqqt*FBt/event_proportion‚0.40996724367141724 :"Purity/0FRt*SRqqt3FBt/event_purity‚0.10185185074806213 ?&Purity/1FRt3SRqqt0FBt/event_proportion‚0.007938507944345474 8 Purity/3FRt1SRqqt1FBt/FRt_purity‚0.14102564752101898 ("Purity/2FRt4SRqqt*FBt/event_purity‚0 7 Purity/1FRt1SRqqt3FBt/FBt_purity‚0.9166666865348816 6 Purity/0FRt*SRqqt2FBt/FBt_purity‚0.980598509311676 ,&Purity/0FRt3SRqqt3FBt/event_proportion‚0 8 Purity/1FRt*SRqqt0FBt/FRt_purity‚0.09786342084407806 :"Purity/1FRt1SRqqt2FBt/SRqqt_purity‚0.24447278678417206 @&Purity/4FRt1SRqqt0FBt/event_proportion‚6.300403038039804e-05 /"Purity/*FRt4SRqqt0FBt/SRqqt_purity‚0.109375 & Purity/3FRt3SRqqt0FBt/FRt_purity‚0 ("Purity/4FRt2SRqqt*FBt/SRqqt_purity‚0 @&Purity/*FRt0SRqqt4FBt/event_proportion‚0.0003150201519019902 @&Purity/*FRt1SRqqt4FBt/event_proportion‚6.300403038039804e-05 :"Purity/0FRt*SRqqt1FBt/SRqqt_purity‚0.24334624409675598 >&Purity/0FRt1SRqqt1FBt/event_proportion‚0.05185231938958168 >&Purity/1FRt2SRqqt0FBt/event_proportion‚0.04731602966785431 ("Purity/2FRt0SRqqt0FBt/event_purity‚0 @&Purity/3FRt*SRqqt0FBt/event_proportion‚0.0071824598126113415 9"Purity/3FRt3SRqqt*FBt/SRqqt_purity‚0.1428571492433548 ?&Purity/2FRt0SRqqt0FBt/event_proportion‚0.024571573361754417 >&Purity/*FRt2SRqqt1FBt/event_proportion‚0.02255544438958168 >&Purity/1FRt*SRqqt1FBt/event_proportion‚0.09929435700178146 >&Purity/2FRt1SRqqt0FBt/event_proportion‚0.03181703761219978 :"Purity/3FRt2SRqqt0FBt/SRqqt_purity‚0.12318839877843857 ,&Purity/4FRt2SRqqt4FBt/event_proportion‚0 ,&Purity/*FRt2SRqqt4FBt/event_proportion‚0 @&Purity/0FRt3SRqqt1FBt/event_proportion‚0.0006300403038039804 8 Purity/1FRt*SRqqt*FBt/FRt_purity‚0.12459518760442734 :"Purity/2FRt*SRqqt0FBt/SRqqt_purity‚0.13546496629714966 ?&Purity/2FRt2SRqqt1FBt/event_proportion‚0.003969253972172737 7 Purity/*FRt0SRqqt4FBt/FBt_purity‚0.8999999761581421 & Purity/2FRt*SRqqt3FBt/FBt_purity‚1 & Purity/3FRt3SRqqt*FBt/FRt_purity‚0 >&Purity/0FRt0SRqqt1FBt/event_proportion‚0.08606351166963577 A&Purity/0FRt1SRqqt3FBt/event_proportion‚0.00044102821266278625 ,&Purity/3FRt1SRqqt3FBt/event_proportion‚0 ,&Purity/4FRt4SRqqt1FBt/event_proportion‚0 particle/accuracy_1_of_7‚1 * Purity/0FRt3SRqqt1FBt/FBt_purity‚0.875 & Purity/1FRt1SRqqt4FBt/FRt_purity‚0 7 Purity/*FRt0SRqqt*FBt/FRt_purity‚0.1122928187251091 7 Purity/*FRt3SRqqt*FBt/FBt_purity‚0.9772727489471436 ,&Purity/1FRt3SRqqt3FBt/event_proportion‚0 ,&Purity/4FRt4SRqqt4FBt/event_proportion‚0 * Purity/1FRt*SRqqt4FBt/FBt_purity‚0.875 ?&Purity/0FRt3SRqqt*FBt/event_proportion‚0.006741431541740894 8 Purity/1FRt3SRqqt0FBt/FRt_purity‚0.13437020778656006 ,&Purity/2FRt*SRqqt4FBt/event_proportion‚0 & Purity/*FRt4SRqqt*FBt/FBt_purity‚1 =&Purity/*FRt0SRqqt2FBt/event_proportion‚0.0483240932226181 ("Purity/*FRt1SRqqt2FBt/event_purity‚0 ,&Purity/0FRt2SRqqt4FBt/event_proportion‚0 8 Purity/1FRt2SRqqt0FBt/FRt_purity‚0.11887060850858688 ,&Purity/1FRt2SRqqt4FBt/event_proportion‚0 8 Purity/2FRt2SRqqt*FBt/FRt_purity‚0.12317753583192825 ,&Purity/3FRt3SRqqt4FBt/event_proportion‚0 ("Purity/*FRt4SRqqt1FBt/event_purity‚0 =&Purity/*FRt2SRqqt*FBt/event_proportion‚0.1332535296678543 7 Purity/0FRt2SRqqt*FBt/FBt_purity‚0.9900153875350952 @&Purity/1FRt*SRqqt3FBt/event_proportion‚0.0013230846961960196 :"Purity/*FRt2SRqqt2FBt/SRqqt_purity‚0.19736842811107635 :"Purity/0FRt2SRqqt2FBt/SRqqt_purity‚0.13461539149284363 >&Purity/0FRt*SRqqt0FBt/event_proportion‚0.21931703388690948 ;"Purity/0FRt2SRqqt0FBt/event_purity‚0.006425026338547468 9"Purity/1FRt1SRqqt*FBt/SRqqt_purity‚0.1717623621225357 ("Purity/1FRt2SRqqt1FBt/event_purity‚0 9"Purity/2FRt*SRqqt*FBt/SRqqt_purity‚0.1614358127117157 :"Purity/3FRt*SRqqt0FBt/SRqqt_purity‚0.13167516887187958 & Purity/3FRt1SRqqt1FBt/FBt_purity‚1 @&Purity/3FRt3SRqqt*FBt/event_proportion‚0.0005670362734235823 8 Purity/1FRt0SRqqt1FBt/FRt_purity‚0.20196278393268585 ?&Purity/1FRt2SRqqt1FBt/event_proportion‚0.008946572430431843 @&Purity/1FRt4SRqqt0FBt/event_proportion‚0.0003780241822823882 & Purity/2FRt3SRqqt*FBt/FBt_purity‚1 7 Purity/2FRt*SRqqt2FBt/FBt_purity‚0.9855555295944214 8 Purity/2FRt4SRqqt*FBt/FRt_purity‚0.30000001192092896 & Purity/1FRt4SRqqt0FBt/FRt_purity‚0 =&Purity/0FRt1SRqqt*FBt/event_proportion‚0.2305317521095276 ;"Purity/0FRt2SRqqt1FBt/event_purity‚0.008064515888690948 8 Purity/2FRt*SRqqt2FBt/FRt_purity‚0.18111109733581543 ("Purity/*FRt*SRqqt4FBt/SRqqt_purity‚0 *"Purity/0FRt1SRqqt3FBt/SRqqt_purity‚0.5 & Purity/4FRt*SRqqt*FBt/FRt_purity‚0 :"Purity/3FRt1SRqqt0FBt/SRqqt_purity‚0.23333334922790527 A&Purity/3FRt4SRqqt*FBt/event_proportion‚0.00012600806076079607 particle/accuracy_6_of_6‚1 ("Purity/*FRt3SRqqt2FBt/event_purity‚0 7 Purity/*FRt*SRqqt*FBt/FBt_purity‚0.9891412258148193 <"Purity/1FRt1SRqqt*FBt/event_purity‚0.0032553812488913536 ,&Purity/1FRt2SRqqt3FBt/event_proportion‚0 <"Purity/2FRt*SRqqt*FBt/event_purity‚0.0005973715451546013 ?&Purity/2FRt3SRqqt0FBt/event_proportion‚0.003654233878478408 +jet/accuracy_2_of_2‚0.05297669395804405 jet/accuracy_5_of_5‚0 ("Purity/3FRt*SRqqt2FBt/SRqqt_purity‚0 :"Purity/*FRt3SRqqt1FBt/SRqqt_purity‚0.20105817914009094 @&Purity/3FRt*SRqqt2FBt/event_proportion‚0.0003150201519019902 particle/accuracy_3_of_4‚1 9"Purity/0FRt*SRqqt3FBt/SRqqt_purity‚0.4333333373069763 ("Purity/4FRt1SRqqt0FBt/event_purity‚0 :"Purity/0FRt*SRqqt2FBt/SRqqt_purity‚0.24472223222255707 7 Purity/0FRt0SRqqt1FBt/FBt_purity‚0.9993136525154114 ,&Purity/0FRt3SRqqt2FBt/event_proportion‚0 :"Purity/1FRt3SRqqt1FBt/SRqqt_purity‚0.13333334028720856 ,&Purity/4FRt0SRqqt1FBt/event_proportion‚0 ,&Purity/4FRt3SRqqt2FBt/event_proportion‚0 *jet/accuracy_1_of_5‚0.7825754880905151 :"Purity/3FRt*SRqqt*FBt/SRqqt_purity‚0.14980843663215637 ,&Purity/3FRt3SRqqt3FBt/event_proportion‚0 8 Purity/3FRt*SRqqt2FBt/FRt_purity‚0.06666667014360428 ("Purity/2FRt*SRqqt3FBt/event_purity‚0 ("Purity/4FRt1SRqqt*FBt/event_purity‚0 _runtime‚ 506.297242008 )loss/val_total_loss‚57.12536543837079 9"Purity/0FRt0SRqqt2FBt/event_purity‚0.3242494463920593 & Purity/3FRt*SRqqt2FBt/FBt_purity‚1 7 Purity/*FRt2SRqqt3FBt/FBt_purity‚0.6666666865348816 :"Purity/*FRt2SRqqt1FBt/SRqqt_purity‚0.24108685553073883 :"Purity/1FRt1SRqqt1FBt/SRqqt_purity‚0.26845529675483704 ,&Purity/3FRt3SRqqt1FBt/event_proportion‚0 jet/accuracy_4_of_6‚0 9"Purity/*FRt2SRqqt0FBt/SRqqt_purity‚0.1480746865272522 ("Purity/0FRt1SRqqt2FBt/event_purity‚0 ("Purity/2FRt1SRqqt0FBt/event_purity‚0 ("Purity/2FRt1SRqqt1FBt/event_purity‚0 ?&Purity/2FRt2SRqqt0FBt/event_proportion‚0.015435988083481789 ("Purity/0FRt0SRqqt4FBt/event_purity‚0 :"Purity/0FRt3SRqqt0FBt/SRqqt_purity‚0.17668652534484863 particle/accuracy_2_of_7‚1 8 Purity/1FRt1SRqqt2FBt/FRt_purity‚0.12738095223903656 7 Purity/2FRt1SRqqt2FBt/FBt_purity‚0.9868420958518982 ,&Purity/2FRt4SRqqt1FBt/event_proportion‚0 ,&Purity/3FRt2SRqqt2FBt/event_proportion‚0 7 Purity/1FRt1SRqqt2FBt/FBt_purity‚0.9666666984558105 ;"Purity/2FRt0SRqqt1FBt/event_purity‚0.008333333767950535 8 Purity/2FRt1SRqqt2FBt/FRt_purity‚0.08771929889917374 ,&Purity/2FRt1SRqqt4FBt/event_proportion‚0 ("Purity/3FRt*SRqqt*FBt/event_purity‚0 @&Purity/3FRt0SRqqt2FBt/event_proportion‚6.300403038039804e-05 ("Purity/0FRt4SRqqt1FBt/event_purity‚0 ("Purity/4FRt1SRqqt*FBt/SRqqt_purity‚0 7 Purity/0FRt*SRqqt3FBt/FBt_purity‚0.9238684177398682 8 Purity/2FRt4SRqqt0FBt/FRt_purity‚0.30000001192092896 +"Purity/3FRt2SRqqt1FBt/SRqqt_purity‚0.25 jet/accuracy_1_of_7‚0 & Purity/1FRt4SRqqt*FBt/FRt_purity‚0 ,&Purity/*FRt3SRqqt3FBt/event_proportion‚0 7 Purity/*FRt1SRqqt*FBt/FBt_purity‚0.9857835173606873 ?&Purity/*FRt3SRqqt*FBt/event_proportion‚0.020476309582591057 :"Purity/0FRt1SRqqt0FBt/event_purity‚0.15242965519428253 ,&Purity/1FRt4SRqqt2FBt/event_proportion‚0 @&Purity/2FRt3SRqqt1FBt/event_proportion‚0.0003780241822823882 & Purity/4FRt*SRqqt0FBt/FRt_purity‚0 ?&Purity/0FRt2SRqqt1FBt/event_proportion‚0.009261592291295528 A&Purity/0FRt4SRqqt0FBt/event_proportion‚0.00044102821266278625 >&Purity/2FRt1SRqqt*FBt/event_proportion‚0.04158266261219978 ("Purity/3FRt2SRqqt*FBt/event_purity‚0 ("Purity/3FRt1SRqqt2FBt/SRqqt_purity‚0 8 Purity/*FRt1SRqqt0FBt/FRt_purity‚0.11775355786085129 7 Purity/*FRt2SRqqt1FBt/FBt_purity‚0.9975185990333557 ,&Purity/*FRt4SRqqt2FBt/event_proportion‚0 ,&Purity/0FRt4SRqqt2FBt/event_proportion‚0 @&Purity/4FRt*SRqqt0FBt/event_proportion‚0.0001890120911411941 * Purity/*FRt*SRqqt4FBt/FBt_purity‚0.875 & Purity/3FRt2SRqqt1FBt/FBt_purity‚1 9"Purity/1FRt2SRqqt2FBt/SRqqt_purity‚0.2152777910232544 :"Purity/*FRt0SRqqt3FBt/event_purity‚0.05892857536673546 ("Purity/0FRt0SRqqt0FBt/event_purity‚1 :"Purity/1FRt0SRqqt1FBt/event_purity‚0.08975519239902496 8 Purity/2FRt3SRqqt*FBt/FRt_purity‚0.13123808801174164 @&Purity/4FRt1SRqqt*FBt/event_proportion‚6.300403038039804e-05 ("Purity/*FRt4SRqqt0FBt/event_purity‚0 >&Purity/*FRt0SRqqt0FBt/event_proportion‚0.15757308900356293 >&Purity/*FRt1SRqqt0FBt/event_proportion‚0.33039313554763794 ,&Purity/0FRt4SRqqt4FBt/event_proportion‚0 7 Purity/1FRt0SRqqt2FBt/FBt_purity‚0.9850406646728516 @&Purity/4FRt*SRqqt*FBt/event_proportion‚0.0001890120911411941 particle/accuracy_3_of_6‚1 ?&Purity/*FRt1SRqqt2FBt/event_proportion‚0.017011089250445366 ,&Purity/0FRt4SRqqt3FBt/event_proportion‚0 7 Purity/1FRt0SRqqt2FBt/FRt_purity‚0.2968064248561859 @&Purity/2FRt4SRqqt*FBt/event_proportion‚0.0003150201519019902 ,&Purity/3FRt0SRqqt4FBt/event_proportion‚0 9"Purity/3FRt2SRqqt*FBt/SRqqt_purity‚0.1440972089767456 ,&Purity/3FRt4SRqqt1FBt/event_proportion‚0 :"Purity/2FRt4SRqqt0FBt/SRqqt_purity‚0.05000000074505806 7 Purity/0FRt*SRqqt1FBt/FBt_purity‚0.9969359636306763 :"Purity/1FRt0SRqqt*FBt/event_purity‚0.06716331094503403 <"Purity/2FRt0SRqqt*FBt/event_purity‚0.0015360983088612556 ("Purity/3FRt3SRqqt0FBt/event_purity‚0 8 Purity/*FRt*SRqqt*FBt/FRt_purity‚0.12208934128284454 8 Purity/2FRt*SRqqt1FBt/FRt_purity‚0.20347507297992706 @&Purity/3FRt0SRqqt0FBt/event_proportion‚0.0015120967291295528 :"Purity/3FRt1SRqqt1FBt/SRqqt_purity‚0.42307692766189575 ,&Purity/4FRt1SRqqt2FBt/event_proportion‚0 ;"Purity/*FRt2SRqqt*FBt/event_purity‚0.002497996436432004 @&Purity/1FRt0SRqqt4FBt/event_proportion‚6.300403038039804e-05 8 Purity/1FRt1SRqqt1FBt/FRt_purity‚0.18075275421142578 ,&Purity/2FRt2SRqqt3FBt/event_proportion‚0 8 Purity/2FRt1SRqqt1FBt/FRt_purity‚0.22561442852020264 >&Purity/*FRt2SRqqt0FBt/event_proportion‚0.10855595022439957 ("Purity/*FRt3SRqqt*FBt/event_purity‚0 ?&Purity/0FRt2SRqqt0FBt/event_proportion‚0.043283771723508835 ,&Purity/4FRt0SRqqt2FBt/event_proportion‚0 :"Purity/*FRt*SRqqt*FBt/event_purity‚0.14523905515670776 8 Purity/*FRt*SRqqt0FBt/FRt_purity‚0.09495631605386734 ("Purity/3FRt0SRqqt0FBt/event_purity‚0 ,&Purity/3FRt0SRqqt3FBt/event_proportion‚0 (particle/accuracy‚0.8221973180770874 ) Purity/*FRt1SRqqt4FBt/FBt_purity‚0.75 >&Purity/0FRt*SRqqt2FBt/event_proportion‚0.03944052383303642 :"Purity/0FRt1SRqqt1FBt/event_purity‚0.07174625247716904 9"Purity/2FRt*SRqqt1FBt/SRqqt_purity‚0.2293599247932434 7 Purity/3FRt*SRqqt1FBt/FRt_purity‚0.1674242466688156 ("Purity/3FRt0SRqqt*FBt/event_purity‚0 & Purity/0FRt4SRqqt*FBt/FBt_purity‚1 particle/accuracy_1_of_6‚1 particle/accuracy_5_of_7‚1 8 Purity/*FRt1SRqqt*FBt/FRt_purity‚0.13301745057106018 :"Purity/*FRt2SRqqt*FBt/SRqqt_purity‚0.16571961343288422 ("Purity/3FRt2SRqqt0FBt/event_purity‚0 *jet/accuracy_1_of_3‚0.6566240787506104 particle/accuracy_5_of_5‚1 +jet/accuracy_3_of_6‚0.02631578966975212 :"Purity/*FRt*SRqqt*FBt/SRqqt_purity‚0.16882193088531494 ;"Purity/*FRt0SRqqt0FBt/event_purity‚0.046057987958192825 9"Purity/1FRt*SRqqt2FBt/SRqqt_purity‚0.2625192105770111 ("Purity/2FRt*SRqqt0FBt/event_purity‚0 8 Purity/2FRt3SRqqt0FBt/FRt_purity‚0.13658009469509125 ,&Purity/3FRt2SRqqt4FBt/event_proportion‚0 ("Purity/*FRt0SRqqt4FBt/event_purity‚0 :"Purity/*FRt3SRqqt*FBt/SRqqt_purity‚0.14834100008010864 7 Purity/0FRt0SRqqt3FBt/FBt_purity‚0.9355555772781372 >&Purity/1FRt1SRqqt1FBt/event_proportion‚0.03944052383303642 ,&Purity/1FRt4SRqqt4FBt/event_proportion‚0 9"Purity/2FRt1SRqqt2FBt/SRqqt_purity‚0.3114035129547119 ,&Purity/2FRt4SRqqt2FBt/event_proportion‚0 ,&Purity/4FRt3SRqqt0FBt/event_proportion‚0 @&Purity/*FRt2SRqqt3FBt/event_proportion‚6.300403038039804e-05 >&Purity/0FRt*SRqqt1FBt/event_proportion‚0.14787046611309052 ?&Purity/1FRt*SRqqt2FBt/event_proportion‚0.022303426638245583 @&Purity/1FRt1SRqqt4FBt/event_proportion‚6.300403038039804e-05 7 Purity/1FRt2SRqqt*FBt/FBt_purity‚0.9959677457809448 ?&Purity/2FRt1SRqqt1FBt/event_proportion‚0.007875503972172737 @&Purity/3FRt0SRqqt*FBt/event_proportion‚0.0027091733645647764 7 Purity/3FRt0SRqqt*FBt/FRt_purity‚0.1262626349925995 >&Purity/*FRt*SRqqt1FBt/event_proportion‚0.27079132199287415 9"Purity/*FRt1SRqqt*FBt/event_purity‚0.0671287402510643 A&Purity/1FRt1SRqqt3FBt/event_proportion‚0.00025201612152159214 ("Purity/3FRt1SRqqt0FBt/event_purity‚0 -jet/accuracy_4_of_5‚0.0029325513169169426 particle/accuracy_4_of_4‚1 particle/accuracy_2_of_6‚1 7 Purity/3FRt1SRqqt2FBt/FRt_purity‚0.0833333358168602 8 Purity/*FRt3SRqqt0FBt/FRt_purity‚0.14047223329544067 ?&Purity/0FRt3SRqqt0FBt/event_proportion‚0.006111391354352236 & Purity/2FRt*SRqqt1FBt/FBt_purity‚1 @&Purity/3FRt*SRqqt1FBt/event_proportion‚0.0024571572430431843 ,&Purity/3FRt4SRqqt2FBt/event_proportion‚0 ("Purity/0FRt3SRqqt1FBt/event_purity‚0 & Purity/1FRt0SRqqt4FBt/FRt_purity‚0 ("Purity/4FRt1SRqqt0FBt/SRqqt_purity‚0 :"Purity/*FRt0SRqqt2FBt/event_purity‚0.22911956906318665 & Purity/1FRt3SRqqt*FBt/FBt_purity‚1 8 Purity/2FRt0SRqqt*FBt/FRt_purity‚0.11734628677368164 ("Purity/2FRt0SRqqt2FBt/event_purity‚0 7 Purity/*FRt4SRqqt0FBt/FRt_purity‚0.0972222313284874 jet/accuracy_6_of_7‚0 & Purity/3FRt1SRqqt2FBt/FBt_purity‚1 ("Purity/2FRt2SRqqt2FBt/SRqqt_purity‚0 9"Purity/*FRt1SRqqt*FBt/SRqqt_purity‚0.1749628484249115 8 Purity/*FRt2SRqqt1FBt/FRt_purity‚0.11085624992847443 @&Purity/*FRt4SRqqt*FBt/event_proportion‚0.0013230846961960196 :"Purity/0FRt2SRqqt0FBt/SRqqt_purity‚0.16971705853939056 ,&Purity/2FRt4SRqqt4FBt/event_proportion‚0 ("Purity/3FRt*SRqqt0FBt/event_purity‚0 7 Purity/3FRt2SRqqt1FBt/FRt_purity‚0.1111111119389534 :"Purity/*FRt0SRqqt*FBt/event_purity‚0.30659642815589905 @&Purity/1FRt3SRqqt2FBt/event_proportion‚6.300403038039804e-05 ("Purity/2FRt2SRqqt1FBt/event_purity‚0 @&Purity/4FRt0SRqqt0FBt/event_proportion‚6.300403038039804e-05 particle/accuracy_4_of_5‚10`ƒg¿“Ô ‚(:ˆ ЗäÔõ®µ¨ gpu.0.gpu‚25.0 gpu.0.memory‚3, gpu.0.memoryAllocated‚12.163162231445312* gpu.0.memoryAllocatedBytes‚ 10448076800 gpu.0.temp‚34.0 gpu.0.powerWatts‚122.822' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚24.5644 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0#5HÕ ‚(:½ ЗäÔÙ«« disk.sda2.in‚0 disk.drbd2266.out‚0 disk.rbd1.out‚ 0.76171875 disk.sda.in‚0 disk.dm-1.out‚ 17.62890625 disk.sda1.in‚0 proc.cpu.threads‚17 network.recv‚262411* disk./.usagePercent‚16.779004138064824# disk./.usageGB‚250.019287109375! disk.nvme0c0n1.in‚ 11.80859375* proc.memory.percent‚16.943836212158203 disk.sda3.out‚ 17.62890625 disk.rbd0.in‚0 disk.drbd2265.out‚0! proc.memory.rssMB‚ 2776.078125 network.sent‚823576& disk.nvme0c0n1.out‚1877.1494140625 disk.drbd2265.in‚0 disk.dm-0.in‚ 11.80859375$ disk.nvme0n1.out‚1877.1494140625 disk.sda2.out‚0 disk.drbd2266.in‚0 disk.rbd1.in‚0% proc.memory.availableMB‚ 2488.3125 disk.sda3.in‚0 disk.dm-1.in‚0 disk.drbd1716.in‚0 disk.sda1.out‚0 cpu‚26.254158011932663! disk.dm-0.out‚1877.2744140625 disk.sda.out‚ 17.62890625 disk.nvme0n1.in‚ 11.80859375$ memory_percent‚84.81254577636719 disk.drbd1716.out‚0 disk.drbd2080.in‚0 disk.drbd2080.out‚0 disk.rbd0.out‚0Ìú2‰“Ö ‚(:ˆ ß—äÔË×™ª gpu.0.gpu‚54.0 gpu.0.memory‚7, gpu.0.memoryAllocated‚12.163162231445312* gpu.0.memoryAllocatedBytes‚ 10448076800 gpu.0.temp‚35.0 gpu.0.powerWatts‚122.049' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚24.4098 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0¡ìF× ‚(:» ß—äÔ¥§Ï« disk.rbd1.out‚ 0.76171875 disk.dm-0.in‚ 11.80859375! disk.dm-0.out‚2165.8212890625 disk.sda3.in‚0 disk.drbd2080.in‚0 disk.rbd0.out‚0) proc.memory.availableMB‚ 2488.70703125 disk.sda1.out‚0 cpu‚26.23990146478301& disk.nvme0c0n1.out‚2165.6962890625 disk.sda1.in‚0 disk.sda2.out‚0 disk.drbd2266.out‚0 proc.cpu.threads‚17 disk.sda.in‚0 disk.rbd0.in‚0$ disk.nvme0n1.out‚2165.6962890625 network.sent‚927168% disk./.usageGB‚250.01859664916992 disk.drbd2265.in‚0 disk.dm-1.in‚0 disk.sda2.in‚0 disk.drbd2266.in‚0 disk.sda3.out‚18.03125 disk.drbd1716.in‚0 disk.drbd1716.out‚0 disk.drbd2080.out‚0$ memory_percent‚84.81013774871826 disk.rbd1.in‚0! disk.nvme0c0n1.in‚ 11.80859375 disk.sda.out‚18.03125 disk.nvme0n1.in‚ 11.80859375# proc.memory.rssMB‚ 2776.08203125* proc.memory.percent‚16.943860054016113 network.recv‚269715* disk./.usagePercent‚16.778957800701125 disk.drbd2265.out‚0 disk.dm-1.out‚18.03125eÿ·û“Ø ‚(:ˆ î—äԮʿ¨ gpu.0.gpu‚51.0 gpu.0.memory‚4, gpu.0.memoryAllocated‚12.165603637695312* gpu.0.memoryAllocatedBytes‚ 10450173952 gpu.0.temp‚36.0 gpu.0.powerWatts‚164.042' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚32.8084 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0ªº¸GÙ ‚(:¼ î—äÔ¹Šåª! disk.dm-0.out‚2208.5595703125 disk.sda3.out‚ 18.671875 disk.dm-1.out‚ 18.671875 disk.drbd1716.out‚0 disk.rbd0.out‚0 disk.sda2.out‚0% disk./.usageGB‚250.04101943969727 disk.dm-1.in‚0 disk.drbd1716.in‚0 disk.drbd2080.in‚0 disk.drbd2080.out‚0 disk.drbd2265.out‚0 disk.rbd0.in‚0 disk.sda2.in‚0 disk.drbd2266.in‚0 disk.drbd2266.out‚0 disk.drbd2265.in‚0% proc.memory.availableMB‚ 2488.1875* proc.memory.percent‚16.943860054016113$ memory_percent‚84.81330871582031 disk.nvme0n1.in‚ 11.80859375 disk.sda1.out‚0 disk.rbd1.in‚0 disk.sda.out‚ 18.671875 disk.sda3.in‚0# proc.memory.rssMB‚ 2776.08203125 proc.cpu.threads‚17 network.recv‚274550 disk.rbd1.out‚ 0.7939453125& disk.nvme0c0n1.out‚2208.4345703125$ disk.nvme0n1.out‚2208.4345703125 network.sent‚933827) disk./.usagePercent‚16.78046261298739 disk.dm-0.in‚ 11.80859375 disk.sda.in‚0 disk.sda1.in‚0 cpu‚26.225178982986275! disk.nvme0c0n1.in‚ 11.80859375ôJ8û0Ú + _wandbruntime‚543  _runtime‚543ãÈNÄÛ ¾ & lr-AdamW/pg1‚0.0018742746643194642 & lr-AdamW/pg2‚0.0018742746643194642 trainer/global_step‚649 % _timestamp‚1.7884149617299883e+09 _runtime‚ 543.379382688 _step‚49.ÓuÈÜ Â & lr-AdamW/pg1‚0.0018742746643194642 & lr-AdamW/pg2‚0.0018742746643194642 trainer/global_step‚649 % _timestamp‚1.7884149617299883e+09 _runtime‚ 543.379382688 _step‚491Ž|&>0Ý + _wandbruntime‚543  _runtime‚543…ôÅÇÞ ¬O UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 435025408 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚20038 E8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚21722146 UBDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated‚37672153886208 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚24126 YHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 486778326016 ]JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚38086403044352 ^MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 506142670848 QCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 182432372 _step‚50 SGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚3254762 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 245575680 dQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚37580039570432 TADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚36631158552143 @8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚644 % _timestamp‚1.7884149617310598e+09 XEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚37185189122560 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 446693376 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 G?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚213 trainer/global_step‚649 N@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 186437632 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 VGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 8980994048 _runtime‚ 543.383836755 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚5676920 ]LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 486947593216 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 YHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 484183641679 C;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚644 JBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚213 IDeviceStatsMonitor.on_train_batch_start/allocation.all.current‚20038 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 ]LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 484349034691 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 ZKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 9456058368 D>DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚431 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚213 TGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚12299056 PDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚8932141 BDeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚37671967448576 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 186437632 O@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 9373790785 PCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚12279020 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚459 NFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚985 @8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚644 G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚2605 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚20036 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 169267200 L@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚8931679 UGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 431144265 SDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 9902751744 UFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 9456058368 OCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚9443126 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 XEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚36631340984515 OCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚9443128 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚2605 ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 220803072 E=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚644 PCDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated‚12299056 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚462 SEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 313315328 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 TFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 446693376 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 242306048 JBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚431 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚2 L=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 9679898624 _LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚37185206292992 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚24126 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 [HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚37185189122560^i}e¶'ß °O D>DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚431 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚213 TGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚12299056 PDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚8932141 BDeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚37671967448576 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 186437632 O@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 9373790785 PCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚12279020 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚459 NFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚985 @8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚644 G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚2605 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚20036 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 169267200 L@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚8931679 UGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 431144265 SDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 9902751744 UFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 9456058368 OCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚9443126 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 XEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚36631340984515 OCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚9443128 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚2605 ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 220803072 E=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚644 PCDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated‚12299056 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚462 SEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 313315328 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 TFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 446693376 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 242306048 JBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚431 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚2 L=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 9679898624 _LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚37185206292992 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚24126 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 [HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚37185189122560 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 435025408 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚20038 E8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚21722146 UBDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated‚37672153886208 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚24126 YHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 486778326016 ]JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚38086403044352 ^MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 506142670848 QCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 182432372 _step‚50 SGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚3254762 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 245575680 dQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚37580039570432 TADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚36631158552143 @8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚644 % _timestamp‚1.7884149617310598e+09 XEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚37185189122560 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 446693376 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 G?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚213 trainer/global_step‚649 N@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 186437632 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 VGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 8980994048 _runtime‚ 543.383836755 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚5676920 ]LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 486947593216 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 YHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 484183641679 C;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚644 JBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚213 IDeviceStatsMonitor.on_train_batch_start/allocation.all.current‚20038 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 ]LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 484349034691 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 ZKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 94560583682CîÔc0à + _wandbruntime‚544  _runtime‚544 uîI¼&á ¶M UIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚5685311 ]JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚37240569982464 L=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 9902751744 I=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚9457126 S@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚37728242942976 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚2605 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚24126 XIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 9456058368 PBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 435025408 R?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚37728056492032 [JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 487672960512 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚24399 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚644 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 QBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 9283702784 WFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 487503680000 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 [JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 485070481607 O@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 9902751744 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚431 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚213 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚3 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 241368064 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 182432580 % _timestamp‚1.7884149624225194e+09 FDeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 186450944 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 BDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 O8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚213 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚644 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚20062 REDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚12317554 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚467 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 144203776 ]JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚36200856616960 VGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 9456058368 VCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚37240552812032 WDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚38142853066240 ^KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚37635949237248 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 169280512 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 169280512 SEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 435025408 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 M>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 9373790785 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 MADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚9457128 ZGDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚37240569982464 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚17039360 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚644 QEDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚9457128 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 431144265 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚431 QCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 313315328 bODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚37635974009856 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚644 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚17170432 K>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚21774682 YFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚36200839577600 _runtime‚ 544.072436188 TCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 487503680000 G:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚21754618 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚5685308 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚3260062 R?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚36685744665987 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 OADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 186450944 MADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚9457126 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚35 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 WIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 446693376 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 446693376 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚24399 M>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 9679898624 VCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚37728242942976 VCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚36685927098567 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚431 TEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 9283702784 NADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚12297492 G:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚21774682 trainer/global_step‚649 YFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚37240552812032 TEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 8980994048 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 446693376 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚20064 NADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚12317554 NBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚8945373 [HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚38143094434304 _step‚51 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 SDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 9456058368 FDeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚8944903uþÙÙPâ ºM JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚431 QCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 313315328 bODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚37635974009856 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚644 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚17170432 K>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚21774682 YFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚36200839577600 _runtime‚ 544.072436188 TCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 487503680000 G:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚21754618 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚5685308 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚3260062 R?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚36685744665987 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 OADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 186450944 MADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚9457126 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚35 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 WIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 446693376 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 446693376 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚24399 M>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 9679898624 VCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚37728242942976 VCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚36685927098567 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚431 TEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 9283702784 NADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚12297492 G:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚21774682 trainer/global_step‚649 YFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚37240552812032 TEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 8980994048 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 446693376 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚20064 NADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚12317554 NBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚8945373 [HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚38143094434304 _step‚51 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 SDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 9456058368 FDeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚8944903 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚5685311 ]JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚37240569982464 L=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 9902751744 I=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚9457126 S@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚37728242942976 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚2605 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚24126 XIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 9456058368 PBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 435025408 R?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚37728056492032 [JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 487672960512 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚24399 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚644 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 QBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 9283702784 WFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 487503680000 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 [JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 485070481607 O@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 9902751744 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚431 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚213 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚3 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 241368064 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 182432580 % _timestamp‚1.7884149624225194e+09 FDeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 186450944 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 BDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 O8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 H@DeviceStatsMonitor.on_train_batch_end/segment.small_poÙ^Œ pol.current‚213 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚644 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚20062 REDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚12317554 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚467 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 144203776 ]JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚36200856616960 VGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 9456058368 VCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚37240552812032 WDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚38142853066240 ^KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚37635949237248 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 169280512 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 169280512 SEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 435025408 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 M>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 9373790785 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 MADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚9457128 ZGDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚37240569982464 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚17039360 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚644 QEDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚9457128 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 4311442653ìó_Ø0ã + _wandbruntime‚544  _runtime‚544#¸(Vä Ð /loss/FRt3/assignment_loss‚4.551190470204209 1loss/SRqqt2/detection_loss‚2.4229505096163066 1loss/SRqqt3/detection_loss‚1.8668566262610604 .loss/FBt3/detection_loss‚2.688468042016029 trainer/global_step‚649 /loss/FRt2/assignment_loss‚5.486409087438841 /loss/FRt1/detection_loss‚2.3481951863790047 1loss/SRqqt4/detection_loss‚1.1933943738297719 0loss/FRt4/assignment_loss‚2.7180128300443607 0loss/FBt1/assignment_loss‚0.7836704909801483 .loss/FRt3/detection_loss‚2.190489683789436 %loss/total_loss‚54.58885548928741 % _timestamp‚1.7884149624243767e+09 _runtime‚ 544.073675518 /loss/FRt1/assignment_loss‚4.842994970934732 0loss/FBt4/assignment_loss‚0.2800435270706815 _step‚52 2loss/SRqqt1/assignment_loss‚2.3347241883355427 /loss/FRt4/detection_loss‚2.0265795292055353 /loss/FBt2/detection_loss‚3.0767635594714773 1loss/SRqqt3/assignment_loss‚2.633694111186767 /loss/FRt2/detection_loss‚2.4581055351205774 1loss/SRqqt1/detection_loss‚1.7721249027465418 /loss/FBt4/detection_loss‚0.7873835228837844 epoch‚1 2loss/SRqqt2/assignment_loss‚2.7523964626448496 2loss/SRqqt4/assignment_loss‚1.9114387553276084 0loss/FBt3/assignment_loss‚0.6266024054144494 0loss/FBt2/assignment_loss‚0.8518872567753781 /loss/FBt1/detection_loss‚1.9844794616103172têuZå Ô 2loss/SRqqt1/assignment_loss‚2.3347241883355427 /loss/FRt4/detection_loss‚2.0265795292055353 /loss/FBt2/detection_loss‚3.0767635594714773 1loss/SRqqt3/assignment_loss‚2.633694111186767 /loss/FRt2/detection_loss‚2.4581055351205774 1loss/SRqqt1/detection_loss‚1.7721249027465418 /loss/FBt4/detection_loss‚0.7873835228837844 epoch‚1 2loss/SRqqt2/assignment_loss‚2.7523964626448496 2loss/SRqqt4/assignment_loss‚1.9114387553276084 0loss/FBt3/assignment_loss‚0.6266024054144494 0loss/FBt2/assignment_loss‚0.8518872567753781 /loss/FBt1/detection_loss‚1.9844794616103172 /loss/FRt3/assignment_loss‚4.551190470204209 1loss/SRqqt2/detection_loss‚2.4229505096163066 1loss/SRqqt3/detection_loss‚1.8668566262610604 .loss/FBt3/detection_loss‚2.688468042016029 trainer/global_step‚649 /loss/FRt2/assignment_loss‚5.486409087438841 /loss/FRt1/detection_loss‚2.3481951863790047 1loss/SRqqt4/detection_loss‚1.1933943738297719 0loss/FRt4/assignment_loss‚2.7180128300443607 0loss/FBt1/assignment_loss‚0.7836704909801483 .loss/FRt3/detection_loss‚2.190489683789436 %loss/total_loss‚54.58885548928741 % _timestamp‚1.7884149624243767e+09 _runtime‚ 544.073675518 /loss/FRt1/assignment_loss‚4.842994970934732 0loss/FBt4/assignment_loss‚0.2800435270706815 _step‚524²2zæ ‚(:’ ý—äÔ–Þå¨ gpu.0.gpu‚40.0 gpu.0.memory‚4, gpu.0.memoryAllocated‚12.165603637695312* gpu.0.memoryAllocatedBytes‚ 10450173952 gpu.0.temp‚36.0 gpu.0.powerWatts‚168.414' gpu.0.enforcedPowerLimitWatts‚500.0( gpu.0.powerPercent‚33.68279999999999 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0öûFç ‚(:» ý—äÔÌÞºª disk.sda2.out‚0 network.sent‚939475 disk.rbd1.in‚0 disk.drbd2265.in‚0 cpu‚26.211605206602304) disk./.usagePercent‚16.77959321140111& disk.nvme0c0n1.out‚2248.0126953125! disk.dm-0.out‚2248.1376953125 disk.sda3.in‚0 disk.dm-1.in‚0 disk.dm-1.out‚ 19.1484375 disk.drbd1716.out‚0 disk.rbd0.in‚0 disk.sda3.out‚ 19.1484375 disk.drbd2080.out‚0 disk.sda2.in‚0 proc.cpu.threads‚17$ disk./.usageGB‚250.0280647277832! disk.nvme0c0n1.in‚ 11.80859375 disk.sda1.in‚0 disk.drbd2265.out‚0# proc.memory.availableMB‚2487.75" proc.memory.rssMB‚ 2776.0859375 disk.rbd1.out‚ 0.7939453125 disk.drbd1716.in‚0$ disk.nvme0n1.out‚2248.0126953125 disk.drbd2266.in‚0* proc.memory.percent‚16.943883895874023 network.recv‚278431 disk.dm-0.in‚ 11.80859375 disk.sda.in‚0 disk.sda.out‚ 19.1484375 disk.nvme0n1.in‚ 11.80859375 disk.sda1.out‚0 disk.drbd2080.in‚0 disk.rbd0.out‚0 disk.drbd2266.out‚0$ memory_percent‚84.81597900390625Âw\Ì“è ‚(:ˆ Œ˜äÔ‘ȉ¨ gpu.0.gpu‚54.0 gpu.0.memory‚7, gpu.0.memoryAllocated‚12.165603637695312* gpu.0.memoryAllocatedBytes‚ 10450173952 gpu.0.temp‚36.0 gpu.0.powerWatts‚167.297' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚33.4594 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0ù¹"ßSé ‚(:È Œ˜äÔ˜úɪ disk.dm-1.in‚0 disk.dm-1.out‚ 19.7265625 disk.drbd1716.out‚0 network.recv‚283602 disk.sda2.in‚0 disk.drbd2080.out‚0& proc.memory.availableMB‚ 2487.96875% disk.nvme0n1.out‚2295.09130859375 disk.drbd2265.out‚0 disk.dm-0.in‚ 11.80859375" disk.dm-0.out‚2295.21630859375 disk.sda3.in‚0$ memory_percent‚84.81464385986328* disk./.usagePercent‚16.781013797208608 disk.sda1.in‚0 disk.sda.in‚0 disk.sda.out‚ 19.7265625% disk./.usageGB‚250.04923248291016 disk.rbd1.in‚0 disk.rbd0.in‚0 disk.rbd0.out‚0* proc.memory.percent‚16.943883895874023 cpu‚26.1999410054795 network.sent‚ 1.025196e+06 disk.sda1.out‚0 disk.rbd1.out‚ 0.85693359375' disk.nvme0c0n1.out‚2295.09130859375 disk.sda3.out‚ 19.7265625 disk.sda2.out‚0 disk.drbd2265.in‚0" proc.memory.rssMB‚ 2776.0859375 disk.drbd2266.out‚0! disk.nvme0c0n1.in‚ 11.80859375 disk.drbd1716.in‚0 disk.drbd2080.in‚0 proc.cpu.threads‚17 disk.nvme0n1.in‚ 11.80859375 disk.drbd2266.in‚0âs0G0ê + _wandbruntime‚582  _runtime‚582ÖÚ²Äë ¾ & lr-AdamW/pg2‚0.0018731793166419997 trainer/global_step‚699 % _timestamp‚1.7884150005463705e+09 _runtime‚ 582.196833961 _step‚53 & lr-AdamW/pg1‚0.0018731793166419997¹ÉêñÈì  & lr-AdamW/pg1‚0.0018731793166419997 & lr-AdamW/pg2‚0.0018731793166419997 trainer/global_step‚699 % _timestamp‚1.7884150005463705e+09 _runtime‚ 582.196833961 _step‚535˜E.‚0í + _wandbruntime‚582  _runtime‚582=˜”¸´'î ®O H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚24126 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 E8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚23349098 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚17170432 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 169267200 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚3 MFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚35 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 XEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚39360673848723 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚2 YHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 523073853440 O@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 9373790785 [HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚38840208267264 L?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚13205972 Q>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚40476447449600 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 446693376 L?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚10143126 G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚2605 \IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚39953390766592 ^MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 544215005184 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 _LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚38840225306624 C;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚644 SDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 9283702784 _LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚39953390766592 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 435025408 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚2605 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚24126 PDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚9587485 L@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚9587014 SGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚3490547 TADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚40476447449600 NFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚985 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 TADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚39360491416351 @8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚644 ZIDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 523243120640 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 M@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚23369136 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚431 trainer/global_step‚699 ]LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 523243120640 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚20036 UBDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated‚40476633887232 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 144203776 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 QCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 182432372 VGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 8980994048 XJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 165393012 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚213 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚17170432 IDeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 TFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 446693376 G?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚431 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚24772608 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 242306048 N?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 9902751744 UFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 9456058368 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚20038 _runtime‚ 582.196937055 XEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚40476633887232 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 YHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 520283149087 [HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚39953373596160 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚24399 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚20036 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 435025408 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 169267200 SEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 313315328ÓA–Û¸'ï ²O L?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚13205972 Q>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚40476447449600 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 446693376 L?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚10143126 G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚2605 \IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚39953390766592 ^MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 544215005184 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 _LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚38840225306624 C;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚644 SDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 9283702784 _LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚39953390766592 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 435025408 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚2605 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚24126 PDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚9587485 L@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚9587014 SGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚3490547 TADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚40476447449600 NFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚985 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 TADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚39360491416351 @8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚644 ZIDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 523243120640 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 M@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚23369136 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚431 trainer/global_step‚699 ]LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 523243120640 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚20036 UBDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated‚40476633887232 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 144203776 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 QCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 182432372 VGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 8980994048 XJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 165393012 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚213 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚17170432 IDeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 TFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 446693376 G?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚431 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚24772608 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 242306048 N?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 9902751744 UFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 9456058368 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚20038 _runtime‚ 582.196937055 XEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚40476633887232 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 YHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 520283149087 [HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚39953373596160 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚24399 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚20036 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 435025408 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 169267200 SEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 313315328 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚24126 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 E8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚23349098 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚17170432 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 169267200 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚3 MFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚35 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 XEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚39360673848723 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚2 YHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 523073853440 O@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 9373790785 [HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚388402082672646—›ÍI0ð + _wandbruntime‚582  _runtime‚582#&\ïñ ¹M [JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 523968487936 SEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 435025408 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚3495380 [MDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚ 212401152 O@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 9902751744 $ _timestamp‚1.788415001233834e+09 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚17170432 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚6104861 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚3 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚461 ^KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚40432671208448 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 _step‚55 XGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 523968487936 OADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 186450944 REDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚10157128 QCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 313315328 R?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚39415077530195 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚2 VCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚39415259962775 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚431 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚213 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚644 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 M>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 9373790785 TEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 8980994048 ]JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚40008754456064 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 237173760 WDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚40977646861824 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 WFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 521004595795 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚431 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚431 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚35 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 trainer/global_step‚699 _runtime‚ 582.883726575 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 182432580 QBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 9283702784 J>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚9600238 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚20064 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚464 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚644 C6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚23381570 TEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 9283702784 G:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚23381570 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚âù­)Ð24126 S@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚40532722944000 G:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚23401634 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17170432 FDeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚23401634 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 NADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚10157126 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚20062 [JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 521169989015 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚644 ZGDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚40008754456064 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1003 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚3495841 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 XIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 9456058368 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚24399 `ODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 545188054528 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 NADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚10157128 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚10157126 NBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚9600702 bODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚40432695981056 SDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 9456058368 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 169280512 PBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 435025408 M>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 9679898624 ODeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 186450944 J;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 9679898624 FDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 _step‚55 XGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 523968487936 OADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 186450944 REDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚10157128 QCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 313315328 R?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚39415077530195 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚2 VCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚39415259962775 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚431 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚213 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚644 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 M>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 9373790785 TEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 8980994048 ]JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚40008754456064 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 237173760 WDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚40977646861824 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 WFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 521004595795 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚431 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚431 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚35 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 trainer/global_step‚699 _runtime‚ 582.883726575 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 182432580 QBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 9283702784 J>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚9600238 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚20064 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚464 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚644 C6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚23381570 TEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 9283702784 G:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚23381570 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚24126 S@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚40532722944000 G:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚23401634 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17170432 FDeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚23401634 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 NADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚10157126 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚20062 [JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 521169989015 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚644 ZGDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚40008754456064 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1003 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚3495841 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 XIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 9456058368 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚24399 `ODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 545188054528 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 NADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚10157128 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚10157126 NBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚9600702 bODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚40432695981056 SDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 9456058368 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 169280512 PBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 435025408 M>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 9679898624 ODeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 186450944 J;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 967989862478†Fò0ó + _wandbruntime‚582  _runtime‚582Gn«Tô Î _runtime‚ 582.884817504 1loss/SRqqt4/assignment_loss‚1.957379815184464 /loss/FRt3/detection_loss‚2.4558923590457304 1loss/SRqqt4/detection_loss‚1.4337348513052626 % _timestamp‚1.7884150012356062e+09 /loss/FRt1/assignment_loss‚4.021445587277412 1loss/SRqqt1/assignment_loss‚2.188039691824662 1loss/FBt4/assignment_loss‚0.20289617395829082 -loss/FRt1/detection_loss‚2.25728179483364 trainer/global_step‚699 0loss/FBt2/assignment_loss‚0.3754204812861038 /loss/FRt4/detection_loss‚2.1644522438562195 1loss/SRqqt2/detection_loss‚2.7021700195197402 &loss/total_loss‚55.592032338228236 epoch‚1 _step‚56 2loss/SRqqt2/assignment_loss‚2.9562868229274093 /loss/FBt4/detection_loss‚0.7937245200677164 /loss/FRt3/assignment_loss‚4.648770218608023 2loss/SRqqt3/assignment_loss‚2.2789980235553924 .loss/FBt1/assignment_loss‚0.22418242143467 /loss/FBt3/detection_loss‚3.0252654949824014 1loss/FBt3/assignment_loss‚0.49574130473451283 /loss/FRt2/detection_loss‚2.5722049686216537 1loss/SRqqt1/detection_loss‚2.0963077024409644 /loss/FRt2/assignment_loss‚5.659351502695391 0loss/FRt4/assignment_loss‚2.9288624933985776 1loss/SRqqt3/detection_loss‚1.9344014247258505 .loss/FBt1/detection_loss‚3.472058284282684 .loss/FBt2/detection_loss‚2.747164137661457*aY^Xõ Ò % _timestamp‚1.7884150012356062e+09 /loss/FRt1/assignment_loss‚4.021445587277412 1loss/SRqqt1/assignment_loss‚2.188039691824662 1loss/FBt4/assignment_loss‚0.20289617395829082 -loss/FRt1/detection_loss‚2.25728179483364 trainer/global_step‚699 0loss/FBt2/assignment_loss‚0.3754204812861038 /loss/FRt4/detection_loss‚2.1644522438562195 1loss/SRqqt2/detection_loss‚2.7021700195197402 &loss/total_loss‚55.592032338228236 epoch‚1 _step‚56 2loss/SRqqt2/assignment_loss‚2.9562868229274093 /loss/FBt4/detection_loss‚0.7937245200677164 /loss/FRt3/assignment_loss‚4.648770218608023 2loss/SRqqt3/assignment_loss‚2.2789980235553924 .loss/FBt1/assignment_loss‚0.22418242143467 /loss/FBt3/detection_loss‚3.0252654949824014 1loss/FBt3/assignment_loss‚0.49574130473451283 /loss/FRt2/detection_loss‚2.5722049686216537 1loss/SRqqt1/detection_loss‚2.0963077024409644 /loss/FRt2/assignment_loss‚5.659351502695391 0loss/FRt4/assignment_loss‚2.9288624933985776 1loss/SRqqt3/detection_loss‚1.9344014247258505 .loss/FBt1/detection_loss‚3.472058284282684 .loss/FBt2/detection_loss‚2.747164137661457 _runtime‚ 582.884817504 1loss/SRqqt4/assignment_loss‚1.957379815184464 /loss/FRt3/detection_loss‚2.4558923590457304 1loss/SRqqt4/detection_loss‚1.43373485130526268Žæöö ‚(:’ ›˜äÔº÷‚© gpu.0.gpu‚44.0 gpu.0.memory‚4, gpu.0.memoryAllocated‚12.165603637695312* gpu.0.memoryAllocatedBytes‚ 10450173952 gpu.0.temp‚36.0 gpu.0.powerWatts‚168.749' gpu.0.enforcedPowerLimitWatts‚500.0( gpu.0.powerPercent‚33.74979999999999 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0NBtS÷ ‚(:È ›˜äÔáÖΪ disk.drbd2080.out‚0 network.sent‚ 1.032387e+06* disk./.usagePercent‚16.779967494416166 disk.rbd1.in‚0! disk.nvme0c0n1.in‚ 11.80859375 disk.drbd2080.in‚0$ proc.memory.availableMB‚2487.875 disk.sda3.out‚ 20.2421875 disk.drbd1716.in‚0 disk.rbd0.in‚0 disk.rbd0.out‚0 disk.nvme0n1.in‚ 11.80859375 disk.drbd2266.in‚0 disk.drbd2266.out‚0 disk.drbd2265.out‚0 disk.sda3.in‚0 disk.drbd2265.in‚0 disk.sda.out‚ 20.2421875 disk.dm-1.in‚0 disk.drbd1716.out‚0% disk.nvme0n1.out‚2338.17138671875 network.recv‚289112" disk.dm-0.out‚2338.29638671875 disk.dm-1.out‚ 20.2421875 disk.sda2.out‚0$ memory_percent‚84.81521606445312% disk./.usageGB‚250.03364181518555 disk.rbd1.out‚ 0.85693359375 disk.dm-0.in‚ 11.80859375" proc.memory.rssMB‚ 2776.0859375 cpu‚26.187211634278857 disk.sda.in‚0' disk.nvme0c0n1.out‚2338.17138671875 disk.sda1.out‚0 disk.sda2.in‚0* proc.memory.percent‚16.943883895874023 disk.sda1.in‚0 proc.cpu.threads‚17òºÁø ‚(:’ ª˜äÔ›¼›¨ gpu.0.gpu‚30.0 gpu.0.memory‚4, gpu.0.memoryAllocated‚12.165603637695312* gpu.0.memoryAllocatedBytes‚ 10450173952 gpu.0.temp‚36.0 gpu.0.powerWatts‚171.91' gpu.0.enforcedPowerLimitWatts‚500.0) gpu.0.powerPercent‚34.382000000000005 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0,}Jù ‚(:¿ ª˜äÔö±§ª$ disk./.usageGB‚250.0622100830078 disk.sda.out‚20.78125 disk.sda2.in‚0 disk.rbd1.in‚0 disk.sda.in‚0 disk.sda3.in‚0 disk.rbd1.out‚ 0.919921875 proc.cpu.threads‚17 disk.drbd1716.out‚0 disk.drbd2080.out‚0 disk.sda1.in‚0! disk.nvme0c0n1.in‚ 11.80859375 disk.dm-0.in‚ 11.80859375" proc.memory.rssMB‚ 2776.0859375 disk.drbd2080.in‚0 disk.sda2.out‚0 disk.drbd2266.in‚0 network.sent‚ 1.120015e+06 disk.dm-1.out‚20.78125 disk.rbd0.out‚0 disk.nvme0n1.in‚ 11.80859375 disk.drbd2266.out‚0 disk.drbd1716.in‚0 disk.drbd2265.out‚0& disk.nvme0c0n1.out‚2391.7529296875! disk.dm-0.out‚2391.8779296875$ memory_percent‚84.81531143188477 network.recv‚295993 disk.sda3.out‚20.78125$ disk.nvme0n1.out‚2391.7529296875 disk.drbd2265.in‚0* proc.memory.percent‚16.943883895874023* disk./.usagePercent‚16.781884734840094 disk.dm-1.in‚0 disk.rbd0.in‚0 disk.sda1.out‚0' proc.memory.availableMB‚ 2487.859375 cpu‚26.176328727922478w½ø ’ú ‚(:‡ ¹˜äÔ ƒ®¨ gpu.0.gpu‚0.0 gpu.0.memory‚0, gpu.0.memoryAllocated‚12.165603637695312* gpu.0.memoryAllocatedBytes‚ 10450173952 gpu.0.temp‚35.0 gpu.0.powerWatts‚167.828' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚33.5656 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0ÿšb©Tû ‚(:É ¹˜äÔ™Ö³ª$ disk./.usageGB‚250.0644302368164" disk.dm-0.out‚2443.39599609375 disk.sda.in‚0 disk.dm-1.in‚0 disk.drbd2080.in‚0 disk.sda1.in‚0 disk.sda2.in‚0 network.sent‚ 1.125975e+06! disk.nvme0c0n1.in‚ 11.80859375' disk.nvme0c0n1.out‚2443.27099609375 disk.dm-0.in‚ 11.80859375 disk.rbd0.in‚0 disk.nvme0n1.in‚ 11.80859375 disk.drbd2266.in‚0 disk.drbd2265.in‚0 disk.sda3.out‚ 21.2890625 disk.dm-1.out‚ 21.2890625 disk.drbd1716.in‚0 disk.sda1.out‚0 disk.rbd1.out‚ 0.919921875# memory_percent‚84.8153829574585 network.recv‚300265) disk./.usagePercent‚16.78203373122502 disk.sda2.out‚0 disk.drbd2266.out‚0 disk.drbd2265.out‚0 disk.sda3.in‚0) proc.memory.availableMB‚ 2487.84765625 disk.rbd0.out‚0% disk.nvme0n1.out‚2443.27099609375 cpu‚26.165139659943982 disk.sda.out‚ 21.2890625 disk.drbd1716.out‚0 disk.drbd2080.out‚0# proc.memory.rssMB‚ 2776.08984375 disk.rbd1.in‚0* proc.memory.percent‚16.943907737731934 proc.cpu.threads‚17;ôj„0ü + _wandbruntime‚620  _runtime‚620 ü?ÞÃý ½ & lr-AdamW/pg1‚0.0018713025707152193 & lr-AdamW/pg2‚0.0018713025707152193 trainer/global_step‚749 $ _timestamp‚1.788415039137644e+09 _runtime‚ 620.787766088 _step‚57Gl9:Çþ Á trainer/global_step‚749 $ _timestamp‚1.788415039137644e+09 _runtime‚ 620.787766088 _step‚57 & lr-AdamW/pg1‚0.0018713025707152193 & lr-AdamW/pg2‚0.00187130257071521939”éá?0ÿ + _wandbruntime‚620  _runtime‚620Þz#€ ²O `MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚43173458740224 QCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 182432372 TFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 446693376 PCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚10843126 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 446693376 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚20036 \IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚42721575240192 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚2605 QDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚10245800 SEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 313315328 Q>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚43280927450624 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚455 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚20036 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚6516020 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 446693376 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 169267200 ]JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚43755900545536 XJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 165393012 PCDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated‚14152960 XEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚42090006712931 ZKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 9456058368 G?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚431 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚213 UFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 9456058368 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚24399 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 435025408 TADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚43280927450624 IDeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚24126 TADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚42089824280559 ZIDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 559538648064 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚24399 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 SDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 9283702784 M@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚10245342 NFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚985 @8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚644 D>DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 186437632 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚17170432 YHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 559369380864 SDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 9902751744 trainer/global_step‚749 _runtime‚ 620.788041187 L?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚14132924 UBDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated‚43281113888256 SGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚6516017 M@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚24996088 bQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 582417032704 N?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 9902751744 N@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 186437632 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 TGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚14152960 SGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚3729325 O@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 9373790785 VGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 8980994048 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚2 TGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚10843128 ]LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 556548049507 C;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚644 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚20038 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 IDeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚2605 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 [HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚41533441624064 QBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 9902751744 ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 212414464 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 G?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚213 B69380864 ]LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 559538648064 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 435025408 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚458 _LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚41533458663424 YHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 556382656495 XIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 9456058368 JBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚213 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚20038 L?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚10843126 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚17170432çƒi¦¼' ¶O G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚2605 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 [HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚41533441624064 QBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 9902751744 ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 212414464 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 G?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚213 BDeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚43280927450624 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚455 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚20036 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚6516020 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 446693376 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 169267200 ]JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚43755900545536 XJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 165393012 PCDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated‚14152960 XEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚42090006712931 ZKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 9456058368 G?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚431 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚213 UFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 9456058368 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚24399 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 435025408 TADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚43280927450624 IDeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚24126 TADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚42089824280559 ZIDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 559538648064 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚24399 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 SDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 9283702784 M@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚10245342 NFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚985 @8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚644 D>DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 186437632 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚17170432 YHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 559369380864 SDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 9902751744 trainer/global_step‚749 _runtime‚ 620.788041187 L?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚14132924 UBDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated‚43281113888256 SGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚6516017 M@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚24996088 bQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 582417032704 N?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 9902751744 N@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 186437632 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 TGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚14152960 SGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚3729325 O@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 9373790785 VGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 8980994048 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚2 TGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚10843128 ]LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 556548049507 C;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚644 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚20038 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 IDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 9373790785 [HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚43812598095872 ^KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚43229393179648 R?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚42144410394403 VHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 165393220 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚24399 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚2 J;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 9679898624 SEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 435025408 REDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚14171458 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚477 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚3 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 431144265 TCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 560094734848 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚2605 WDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚43812358824960 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚10857126 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 ]JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚42776938929664 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚6524408 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 YFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚41587306291200 WFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 557104103203 QBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 9902751744 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚213 S@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚43337202945024 TEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 9283702784 REDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚10857128 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 TEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 8980994048 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚644 trainer/global_step‚749 J=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚14151396 FDeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚25028586 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚24126 \KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 582965645312 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 SDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 9456058368 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚20064 NADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚14171458 [JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 560264015360 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚644 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 169280512 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚17170432 FDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 L>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 186450944 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 VCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚42144592826983 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚431 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚431 NADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚10857128 VCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚43337202945024 `ODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 583180143616 VGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 9456058368 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚35 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚474 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 446693376 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚644 K>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚10258425 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 239270912 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚17039360 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚431 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 % _timestamp‚1.7884150398252757e+09 PBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 435025408 NADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚14151396 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 C6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚25008522 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17170432 XGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 560264015360 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚3734017 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24772608 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 [JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 557269496423 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚20062 R?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚43337016494080 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1003 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 446693376 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚213 _step‚59 ]JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚41587323330560 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 E6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚644 L=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 9902751744 WIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 446693376 M>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 9679898624 OBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚10258902 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 QCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 313315328 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚213 YFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚42776921759232 WFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 560094734848 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 242306048 G:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚25028586 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚20062 BDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 9373790785 [HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚43812598095872 ^KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚43229393179648 R?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚42144410394403 VHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 165393220 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚24399 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚2 J;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 9679898624 SEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 435025408 REDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚14171458 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚477 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚3 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 431144265 TCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 560094734848 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚2605 WDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚43812358824960 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚10857126 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 ]JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚42776938929664 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚6524408 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 YFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚41587306291200 WFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 557104103203 QBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 9902751744 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚213 S@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚43337202945024 TEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 9283702784 REDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚10857128 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 TEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 8980994048 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚644 trainer/global_step‚749 J=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚14151396 FDeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚25028586 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚24126 \KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 582965645312 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 SDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 9456058368 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚20064 NADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚14171458 [JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 560264015360 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚644 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 169280512 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚17170432 FDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 L>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 186450944 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 VCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚42144592826983 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚431 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚431 NADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚10857128 VCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚43337202945024 `ODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 583180143616 VGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 9456058368 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚35 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚474 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 446693376 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚644 K>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚10258425 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 239270912 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚17039360 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚431 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 % _timestamp‚1.7884150398252757e+09 PBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 435025408 NADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚14151396 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 C6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚25008522 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17170432 XGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 560264015360 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚3734017 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24772608 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 [JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 557269496423 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚20062 R?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚43337016494080 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1003 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 446693376 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚213 _step‚59 ]JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚41587323330560 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 E6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚644 L=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 9902751744 WIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 446693376 M>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 9679898624 OBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚10258902 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 QCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 313315328 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚213 YFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚42776921759232 WFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 560094734848 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 242306048 G:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚25028586 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚20062 BDeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚46085407451648 O@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 9679898624 NFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚985 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚24772608 TADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚44819157144767 _step‚62 IDeviceStatsMonitor.on_train_batch_start/allocation.all.current‚20038 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚24399 SGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚6935567 UGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 431144265 SDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 9902751744 XEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚46085593889280 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 249769984 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 YHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 592482163903 PCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚11543128 XEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚45489742543360 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 169267200 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 435025408 D>DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 L?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚11543126 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚3 bQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 620392350208 UFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 9456058368 BDeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚24126 N@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 186437632 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 435025408 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚17170432 M@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚26623040 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 O@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 9373790785 ]LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 592647556915 TGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚11543128 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 _LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚44226692020224 QBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 9902751744 TFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 446693376 IDeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚2605 PCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚15059876 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 144203776 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 SDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 9283702784z<X»'‘ µO IDeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚2605 PCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚15059876 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 144203776 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 SDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 9283702784 QCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 182432372 VGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 8980994048 $ _timestamp‚1.788415077487035e+09 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚17170432 ]LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 595834175488 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚498 PCDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated‚15079912 _LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚45489759713792 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 TGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚15079912 M@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚10903120 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚20038 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 [HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚44226674980864 JBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚431 L?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚15059876 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚20036 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚24126 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 446693376 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚213 Q>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚46085407451648 O@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 9679898624 NFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚985 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚24772608 TADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚44819157144767 _step‚62 IDeviceStatsMonitor.on_train_batch_start/allocation.all.current‚20038 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚24399 SGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚6935567 UGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 431144265 SDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 9902751744 XEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚46085593889280 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 249769984 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 YHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 592482163903 PCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚11543128 XEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚45489742543360 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 169267200 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 435025408 D>DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 L?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚11543126 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚3 bQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 620392350208 UFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 9456058368 BDeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚24126 N@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 186437632 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 435025408 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚17170432 M@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚26623040 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 O@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 9373790785 ]LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 592647556915 TGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚11543128 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 _LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚44226692020224 QBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 9902751744 TFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 446693376>hÑ:š0’ + _wandbruntime‚659  _runtime‚659Üi¦—“ »M >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 PBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 435025408 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 [JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 593369003831 S@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚46141682946048 [JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 596559542784 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚445 YFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚44280539648000 L>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 186450944 REDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚11557128 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚35 VGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 9456058368 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 M>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 9373790785 R?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚46141496495104 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚20062 \KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 620933293056 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚644 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚431 J=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚15078348 FDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 trainer/global_step‚799 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚644 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚2 QBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 9283702784 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 169280512 K>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚26655538 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 NADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚15078348 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚213 _runtime‚ 659.81178415 M>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 9679898624 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚17170432 REDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚15098410 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚644 O@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 9902751744 SEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 435025408 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚6943961 J;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 9679898624 G:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚26635474 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚20062 WDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚46647048443904 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24772608 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 SDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 9456058368 OADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 186450944 TEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 9283702784 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚24126 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1003 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚442 ]JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚44280556687360 XIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 9456058368 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚213 VCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚45545106232832 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 169280512 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚24399 ^KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚46026115150848 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 WIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 446693376 FDeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 K>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚10916249 WFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 593203610611 L=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 9902751744 G:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚26655538 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 242306048 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚17039360 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚431 EDeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 K>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚10916249 WFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 593203610611 L=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 9902751744 G:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚26655538 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 242306048 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚17039360 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚431 E8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 PBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 435025408 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 [JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 593369003831 S@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚46141682946048 [JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 596559542784 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚445 YFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚44280539648000 L>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 186450944 REDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚11557128 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚35 VGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 9456058368 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 M>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 9373790785 R?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚46141496495104 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚20062 \KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 620933293056 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚644 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚431 J=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚15078348 FDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 trainer/global_step‚799 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚644 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚2 QBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 9283702784 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 169280512 K>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚26655538 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 NADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚15078348 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚213 _runtime‚ 659.81178415 M>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 9679898624 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚17170432 REDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚15098410 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚644 O@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 9902751744 SEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 435025408 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚6943961 J;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 9679898624 G:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚26635474 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚20062 WDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚46647048443904 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24772608 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 SDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 9456058368 OADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 186450944 TEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 9283702784 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚24126 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1003 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚442 ]JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚44280556687360 XIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 9456058368 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚213 VCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚45545106232832 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 169280512 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚24399 ^KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚46026115150848 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 WIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 446693376 FDeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚2605 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 169267200 ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 218705920 PCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚12243126 _LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚46919925377024 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 N@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 186437632 Q>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚48889887452672 TGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚16006864 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚488 @8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚644 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚213 PCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚15986828 [HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚46919908337664 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚24399 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚17170432 XEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚48257927016960 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚491 SGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚7355117 D>DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 UFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 9456058368 L?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚12243126 TGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚12243128 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚24126 YFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚49425070158848 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚20036 QCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 182432372 XEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚48890073890304 QDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚11559677 M@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚11559186 bQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 658386182144 O@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 9373790785 TADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚48889887452672 ]JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚49425313637376 SGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚4204069 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 % _timestamp‚1.7884151161572053e+09 [HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚48257927016960 ]LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 632129702912 _step‚66 PCDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated‚16006864 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚24772608 ^MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 658167476224 QBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 9902751744 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 XIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 9456058368 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 SDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 9283702784 O@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 9679898624 YHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 631960435712 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 UGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 431144265 N?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 9902751744 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚2605 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 G?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚213 H?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚1003 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 JBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚431 JBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚213 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚17170432 MFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚35 XEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚47548672441347 TADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚47548490008975 ZKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 9456058368 TFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 446693376 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚431 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 186437632 VGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 9283702784 M@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚28249992 NFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚985 YHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 628581671311 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 446693376 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚20036 @8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚644 E=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚644 VEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 631960435712 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 169267200 IDeviceStatsMonitor.on_train_batch_start/allocation.all.current‚20038 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚3 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 144203776 BDeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚24126 _LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚48257944187392 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 _runtime‚ 697.806923497 IDeviceStatsMonitor.on_train_batch_start/allocation.all.current‚20038 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚3 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 144203776 BDeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚24126 _LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚48257944187392 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 WJDeviceStatsMonitor.on_train_batch_start/requested_b°5y:ytes.large_pool.current‚17039360 _runtime‚ 697.806923497 IDeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚2605 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 169267200 ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 218705920 PCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚12243126 _LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚46919925377024 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 N@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 186437632 Q>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚48889887452672 TGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚16006864 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚488 @8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚644 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚213 PCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚15986828 [HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚46919908337664 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚24399 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚17170432 XEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚48257927016960 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚491 SGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚7355117 D>DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 UFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 9456058368 L?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚12243126 TGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚12243128 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚24126 YFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚49425070158848 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚20036 QCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 182432372 XEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚48890073890304 QDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚11559677 M@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚11559186 bQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 658386182144 O@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 9373790785 TADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚48889887452672 ]JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚49425313637376 SGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚4204069 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 % _timestamp‚1.7884151161572053e+09 [HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚48257927016960 ]LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 632129702912 _step‚66 PCDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated‚16006864 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚24772608 ^MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 658167476224 QBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 9902751744 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 XIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 9456058368 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 SDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 9283702784 O@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 9679898624 YHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 631960435712 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 UGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 431144265 N?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 9902751744 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚2605 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 G?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚213 H?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚1003 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 JBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚431 JBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚213 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚17170432 MFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚35 XEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚47548672441347 TADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚47548490008975 ZKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 9456058368 TFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 446693376 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚431 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 186437632 VGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 9283702784 M@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚28249992 NFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚985 YHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 628581671311 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 446693376 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚20036 @8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚644BaÕUŠ0¤ + _wandbruntime‚698  _runtime‚698×½5Â&¥ ¼M `ODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 659141934080 K>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚11572474 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 242306048 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 WFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 629303118019 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 169280512 G:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚28262426 LDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚985 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 182432580 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 446693376 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚213 J=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚16005300 S@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚48946162947072 ZGDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚48313307876864 [JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 632855070208 WDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚49481764557824 BDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 9679898624 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚17170432 K>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚28282490 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24772608 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 L=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 9902751744 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚644 TCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 632685789696 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 169280512 REDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚16025362 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚3 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚35 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚213 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 J;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 9679898624 PBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 435025408 [MDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚ 214498304 O@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 9902751744 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚644 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚7363511 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚17039360 % _timestamp‚1.7884151168419626e+09 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚7363508 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚24126 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 431144265 SEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 435025408 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 NADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚12257126 OBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚11572962 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 239270912 XIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 9456058368 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚20062 QBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 9283702784 FDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 9373790785 QBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 9902751744 YFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚48313290706432 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚2605 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚213 _runtime‚ 698.491786848 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 C6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚28262426 \KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 658927435776 WIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 446693376 _step‚67 NADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚12257128 EDeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 186450944 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚24399 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚644 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚431 [JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 629468511239 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚12257126 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚20062 YFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚46973773004800 NADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚16005300 QCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 313315328 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 ODeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚11572474 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 242306048 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 WFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 629303118019 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 169280512 G:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚28262426 LDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚985 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 182432580 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 446693376 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚213 J=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚16005300 S@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚48946162947072 ZGDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚48313307876864 [JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 632855070208 WDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚49481764557824 BDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 9679898624 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚17170432 K>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚28282490 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24772608 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 L=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 9902751744 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚644 TCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 632685789696 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 169280512 REDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚16025362 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚3 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚35 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚213 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 J;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 9679898624 PBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 435025408 [MDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚ 214498304 O@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 9902751744 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚644 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚7363511 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚17039360 % _timestamp‚1.7884151168419626e+09 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚7363508 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚24126 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 431144265 SEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 435025408 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 NADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚12257126 OBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚11572962 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 239270912 XIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 9456058368 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚20062 QBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 9283702784 FDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 9373790785 QBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 9902751744 YFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚48313290706432 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚2605 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚213 _runtime‚ 698.491786848 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 C6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚28262426 \KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 658927435776 WIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 446693376 _step‚67 NADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚12257128 EDeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 186450944 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚24399 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚644 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚431 [JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 629468511239 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚12257126 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚20062 YFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚46973773004800 NADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚16005300 QCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 313315328 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 OÔ“ª ‚(:ˆ “™äÔ™Ù‚¨ gpu.0.gpu‚44.0 gpu.0.memory‚4, gpu.0.memoryAllocated‚12.165603637695312* gpu.0.memoryAllocatedBytes‚ 10450173952 gpu.0.temp‚36.0 gpu.0.powerWatts‚122.049' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚24.4098 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0o9âæT« ‚(:É “™äÔ¸‡¥ª proc.cpu.threads‚17 network.sent‚ 1.323566e+06 network.recv‚329683 disk.sda3.in‚0 disk.rbd0.out‚0 disk.drbd2266.out‚0 disk.sda3.out‚ 24.31640625 disk.drbd2265.in‚0* disk./.usagePercent‚16.788062452648848 disk.rbd1.in‚0 disk.dm-0.in‚ 11.80859375 disk.rbd1.out‚ 1.1005859375 disk.sda.in‚0 disk.dm-1.out‚ 24.31640625( proc.memory.availableMB‚ 2358.9609375 cpu‚26.10944122064507! disk.nvme0c0n1.in‚ 11.80859375" disk.dm-0.out‚2732.83544921875 disk.drbd1716.in‚0 disk.sda1.in‚0 disk.sda1.out‚0 disk.sda2.in‚0 disk.sda2.out‚0 disk.drbd2265.out‚0$ disk./.usageGB‚250.1542625427246' disk.nvme0c0n1.out‚2732.71044921875 disk.sda.out‚ 24.31640625 disk.dm-1.in‚0 disk.drbd2080.in‚0% disk.nvme0n1.out‚2732.71044921875 disk.drbd2266.in‚0# memory_percent‚85.6020450592041 disk.drbd1716.out‚0 disk.drbd2080.out‚0 disk.rbd0.in‚0 disk.nvme0n1.in‚ 11.80859375 proc.memory.rssMB‚ 2776.09375* proc.memory.percent‚16.943931579589844¸èkw‘¬ ‚(:† ¢™äÔÅ›¨ gpu.0.gpu‚55.0 gpu.0.memory‚7, gpu.0.memoryAllocated‚12.165603637695312* gpu.0.memoryAllocatedBytes‚ 10450173952 gpu.0.temp‚35.0 gpu.0.powerWatts‚121.51' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚24.302 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0ÓX]ºk­ ‚(:Ê ¢™äÔÌÒ¨« disk.dm-1.out‚ 28.02734375 disk.drbd1716.in‚0 disk.drbd2080.in‚0$ disk.nvme0n1.out‚2786.3134765625 proc.cpu.threads‚17! disk.nvme0c0n1.in‚ 11.80859375 disk.dm-0.in‚ 11.80859375 disk.sda3.out‚ 28.02734375 disk.drbd2080.out‚0 disk.rbd0.out‚0 disk.drbd2266.in‚0 proc.memory.rssMB‚ 2776.0鹺Êê9375 disk.sda.in‚0 disk.drbd1716.out‚0 disk.nvme0n1.in‚ 11.80859375 disk.drbd2265.out‚0 disk.sda.out‚ 28.02734375 disk.rbd0.in‚0 disk.sda2.in‚0 disk.sda2.out‚0$ memory_percent‚85.60206890106201 network.sent‚ 1.410815e+06 disk.rbd1.out‚ 1.16357421875! disk.dm-0.out‚2786.4384765625 network.recv‚336141 disk.sda3.in‚0 disk.sda1.out‚0) proc.memory.availableMB‚ 2358.95703125* proc.memory.percent‚16.943931579589844 cpu‚26.102120544494237) disk./.usagePercent‚16.78829157939199 disk.rbd1.in‚0 disk.dm-1.in‚0 disk.sda1.in‚0 disk.drbd2266.out‚0% disk./.usageGB‚250.15767669677734 disk.drbd2265.in‚0& disk.nvme0c0n1.out‚2786.3134765625q7tØž® ‚(:“ ±™äÔ‘Œ’¨ gpu.0.gpu‚27.0 gpu.0.memory‚3, gpu.0.memoryAllocated‚12.165603637695312* gpu.0.memoryAllocatedBytes‚ 10450173952 gpu.0.temp‚36.0 gpu.0.powerWatts‚156.229' gpu.0.enforcedPowerLimitWatts‚500.0) gpu.0.powerPercent‚31.245800000000003 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0yGc‘Q¯ ‚(:Æ ±™äÔ¡˜«! disk.dm-0.out‚2835.3212890625 disk.sda.in‚0 disk.nvme0n1.in‚ 11.80859375 disk.sda1.in‚0 disk.drbd2265.in‚0 cpu‚26.093853507798343& disk.nvme0c0n1.out‚2835.1962890625( proc.memory.availableMB‚ 2358.9140625 network.recv‚339899* disk./.usagePercent‚16.790188851228248 disk.drbd2080.out‚0 disk.drbd2266.in‚0 disk.rbd1.out‚ 1.16357421875! disk.nvme0c0n1.in‚ 11.80859375$ memory_percent‚85.60233116149902* proc.memory.percent‚16.943931579589844 disk.rbd0.out‚0 disk.sda1.out‚0 disk.sda2.out‚0 disk.drbd2266.out‚0 proc.memory.rssMB‚ 2776.09375$ disk./.usageGB‚250.1859474182129 disk.dm-1.out‚ 28.9296875 disk.drbd2080.in‚0 disk.sda2.in‚0 disk.rbd1.in‚0 disk.sda3.in‚0 disk.sda3.out‚ 28.9296875 disk.dm-1.in‚0 disk.drbd1716.out‚0 disk.rbd0.in‚0$ disk.nvme0n1.out‚2835.1962890625 disk.drbd2265.out‚0 proc.cpu.threads‚17 disk.dm-0.in‚ 11.80859375 disk.sda.out‚ 28.9296875 disk.drbd1716.in‚0 network.sent‚ 1.416225e+06-Í•0° + _wandbruntime‚736  _runtime‚736úÎهı ¾ & lr-AdamW/pg1‚0.0018610028666494847 & lr-AdamW/pg2‚0.0018610028666494847 trainer/global_step‚899 % _timestamp‚1.7884151546622114e+09 _runtime‚ 736.311755157 _step‚69ü¶SjȲ  trainer/global_step‚899 % _timestamp‚1.7884151546622114e+09 _runtime‚ 736.311755157 _step‚69 & lr-AdamW/pg1‚0.0018610028666494847 & lr-AdamW/pg2‚0.0018610028666494847E‚ÐI.0³ + _wandbruntime‚736  _runtime‚736Û‡Q=·'´ ±O QCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 182432372 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 169267200 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 C;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚644 JBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚431 trainer/global_step‚899 N@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 186437632 IDeviceStatsMonitor.on_train_batch_start/allocation.all.current‚20038 SGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚7774667 ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 222900224 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 D>DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 L=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 9679898624 BDeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚24126 H?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚1003 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚24772608 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 G?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚213 Q>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚51694367453696 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 186437632 YHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 664681178719 UGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 431144265 YHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 668255963136 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚3 NFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚985 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 247672832 JBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚213 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚17170432 ZIDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 668425230336 TADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚51694367453696 `MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚51563624653824 QBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 9902751744 UFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 9456058368 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 O@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 9679898624 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚4442957 SDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 9902751744 G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚2605 PCDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated‚16933816 VEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 668255963136 XEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚51694553891328 QDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚12217627 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚448 N?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 9902751744 _runtime‚ 736.316859611 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚20038 PCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚16913780 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 446693376 E=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚644 _step‚70 E8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚29856906 L?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚12943126Š7p7»'µ µO XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 169267200 SGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚4442512 SEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 313315328 TADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚50277822873183 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚213 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚24399 \IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚51026128660992 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚20038 SGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚7774667 ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 222900224 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 D>DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 L=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 9679898624 BDeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚24126 H?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚1003 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚24772608 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 G?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚213 Q>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚51694367453696 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 186437632 YHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 664681178719 UGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 431144265 YHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 668255963136 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚3 NFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚985 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 247672832 JBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚213 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚17170432 ZIDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 668425230336 TADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚51694367453696 `MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚51563624653824 QBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 9902751744 UFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 9456058368 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 O@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 9679898624 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚4442957 SDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 9902751744 G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚2605 PCDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated‚16933816 VEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 668255963136 XEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚51694553891328 QDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚12217627 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚448 N?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 9902751744 _runtime‚ 736.316859611 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚20038 PCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚16913780 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 446693376 E=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚644 _step‚70 E8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚29856906 L?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚12943126 QCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 182432372 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 169267200 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 C;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚644 JBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚431 trainer/global_step‚899 N@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 186437632 IDeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚12230304 ^KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚51619559093248 REDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚12957128 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 ]JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚49667023400960 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚213 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 TEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 8980994048 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚431 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚213 EDeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 TEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 9283702784 TCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 668981317120 VCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚51081475180032 SEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 435025408 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 182432580 YFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚49667006361600 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 446693376 trainer/global_step‚899 ODeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 9679898624 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚644 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚213 ]JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚51081492350464 [JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 669150597632 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1003 QCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 313315328 VHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 165393220 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 431144265 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚24399 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚24126 WIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 446693376 ZGDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚51081492350464 XGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 669150597632 PBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 435025408 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚2605 O@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 9902751744 G:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚29889378 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚644 SDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 9456058368 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 446693376 G:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚29909442 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 169280512 _step‚71 R?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚50332408987027 NADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚12957126 REDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚16952314 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚7783058 FDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 VGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 9456058368 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚12957126 [MDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚ 222886912 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚20064 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24772608 VCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚50332591419607 L=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 9902751744 J;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 9679898624 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚470 M>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 9373790785 R?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚51750456497152 QBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 9283702784 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚4447713 `ODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 697084650496 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 [JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 665568018647 XIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 9456058368 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚4447246 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 144203776 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚431 FDeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 186450944 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚7783061 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 242306048 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚2 S@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚51750642948096 [HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚52316668516352 \KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 696861763584 OADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 186450944 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚20062 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚3 QBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 9902751744 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚644 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 169280512 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚467 WDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚52316420856832 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 BDeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚29909442 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17170432 NADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚16932252 NADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚12957128 NADeviceStatsMonitor.on_train_batch_end/active.small_pool.a(¹xullocated‚16952314 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚20062 % _timestamp‚1.7884151553452303e+09 J=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚16932252 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚17170432 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚35Z‹Å&¸ ¿M % _timestamp‚1.7884151553452303e+09 J=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚16932252 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚17170432 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚35 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 _runtime‚ 736.99534308 K>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚12230304 ^KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚51619559093248 REDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚12957128 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 ]JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚49667023400960 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚213 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 TEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 8980994048 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚431 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚213 EDeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 TEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 9283702784 TCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 668981317120 VCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚51081475180032 SEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 435025408 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 182432580 YFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚49667006361600 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 446693376 trainer/global_step‚899 ODeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 9679898624 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚644 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚213 ]JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚51081492350464 [JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 669150597632 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1003 QCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 313315328 VHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 165393220 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 431144265 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚24399 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚24126 WIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 446693376 ZGDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚51081492350464 XGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 669150597632 PBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 435025408 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚2605 O@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 9902751744 G:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚29889378 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚644 SDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 9456058368 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 446693376 G:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚29909442 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 169280512 _step‚71 R?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚50332408987027 NADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚12957126 REDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚16952314 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚7783058 FDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 VGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 9456058368 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚12957126 [MDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚ 222886912 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚20064 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24772608 VCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚50332591419607 L=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 9902751744 J;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 9679898624 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚470 M>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 9373790785 R?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚51750456497152 QBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 9283702784 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚4447713 `ODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 697084650496 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 [JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 665568018647 XIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 9456058368 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚4447246 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 144203776 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚431 FDeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 186450944 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚7783061 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 242306048 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚2 S@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚51750642948096 [HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚52316668516352 \KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 696861763584 OADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 186450944 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚20062 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚3 QBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 9902751744 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚644 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 169280512 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚467 WDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚52316420856832 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 BDeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚29909442 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17170432 NADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚16932252 NADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚12957128 NADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚16952314 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚20062Gâ‘‘0¹ + _wandbruntime‚736  _runtime‚736úXoPº Ê /loss/FRt3/assignment_loss‚4.393213542201851 1loss/FBt1/assignment_loss‚0.31499481557690956 -loss/FRt1/detection_loss‚2.47990624809807 .loss/FRt3/detection_loss‚2.558810943855515 /loss/FRt4/detection_loss‚1.7726810596233935 .loss/FBt1/detection_loss‚2.380266420543194 trainer/global_step‚899 0loss/FBt3/assignment_loss‚0.3520001633558024 1loss/SRqqt2/detection_loss‚2.4317801069857468 0loss/SRqqt3/detection_loss‚2.074210226719762 % _timestamp‚1.7884151553477437e+09 /loss/FRt2/assignment_loss‚4.765301056810327 1loss/SRqqt4/assignment_loss‚1.729557321865612 .loss/FRt2/detection_loss‚2.613694979935079 .loss/FBt3/detection_loss‚2.659705662727356 0loss/FRt4/assignment_loss‚2.4441940877415718 1loss/SRqqt2/assignment_loss‚2.390648438859342 2loss/SRqqt3/assignment_loss‚2.0997067735886867 /loss/FBt2/assignment_loss‚0.575591187970711 epoch‚1 _runtime‚ 736.997052185 1loss/SRqqt1/detection_loss‚1.8047579034515049 .loss/FRt1/assignment_loss‚4.16295146129348 1loss/SRqqt4/detection_loss‚1.1516851622110873 /loss/FBt2/detection_loss‚2.6750038295984266 /loss/FBt4/detection_loss‚0.8713201392056613 2loss/SRqqt1/assignment_loss‚1.9519235274066096 1loss/FBt4/assignment_loss‚0.10586689405401095 %loss/total_loss‚50.75977195367972 _step‚72DìÁ«T» Î /loss/FRt2/assignment_loss‚4.765301056810327 1loss/SRqqt4/assignment_loss‚1.729557321865612 .loss/FRt2/detection_loss‚2.613694979935079 .loss/FBt3/detection_loss‚2.659705662727356 0loss/FRt4/assignment_loss‚2.4441940877415718 1loss/SRqqt2/assignment_loss‚2.390648438859342 2loss/SRqqt3/assignment_loss‚2.0997067735886867 /loss/FBt2/assignment_loss‚0.575591187970711 epoch‚1 _runtime‚ 736.997052185 1loss/SRqqt1/detection_loss‚1.8047579034515049 .loss/FRt1/assignment_loss‚4.16295146129348 1loss/SRqqt4/detection_loss‚1.1516851622110873 /loss/FBt2/detection_loss‚2.6750038295984266 /loss/FBt4/detection_loss‚0.8713201392056613 2loss/SRqqt1/assignment_loss‚1.9519235274066096 1loss/FBt4/assignment_loss‚0.10586689405401095 %loss/total_loss‚50.75977195367972 _step‚72 /loss/FRt3/assignment_loss‚4.393213542201851 1loss/FBt1/assignment_loss‚0.31499481557690956 -loss/FRt1/detection_loss‚2.47990624809807 .loss/FRt3/detection_loss‚2.558810943855515 /loss/FRt4/detection_loss‚1.7726810596233935 .loss/FBt1/detection_loss‚2.380266420543194 trainer/global_step‚899 0loss/FBt3/assignment_loss‚0.3520001633558024 1loss/SRqqt2/detection_loss‚2.4317801069857468 0loss/SRqqt3/detection_loss‚2.074210226719762 % _timestamp‚1.7884151553477437e+09HÙÓ­“¼ ‚(:ˆ À™äÔñ·‡¨ gpu.0.gpu‚43.0 gpu.0.memory‚4, gpu.0.memoryAllocated‚12.165603637695312* gpu.0.memoryAllocatedBytes‚ 10450173952 gpu.0.temp‚36.0 gpu.0.powerWatts‚163.852' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚32.7704 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0¦!ŽšS½ ‚(:È À™äÔÍͪª$ memory_percent‚85.60285568237305 proc.memory.rssMB‚ 2776.09375* proc.memory.percent‚16.943931579589844% disk./.usageGB‚250.21303176879883 disk.drbd1716.in‚0 disk.rbd0.in‚0 proc.cpu.threads‚17 disk.sda3.in‚0 disk.rbd0.out‚0 disk.nvme0n1.in‚ 11.80859375% disk.nvme0n1.out‚2890.69189453125 disk.rbd1.out‚ 1.2265625! disk.nvme0c0n1.in‚ 11.80859375 disk.sda3.out‚ 29.68359375 disk.sda1.in‚0 disk.drbd2266.out‚0* disk./.usagePercent‚16.792006504721353" disk.dm-0.out‚2890.81689453125 disk.sda.in‚0 disk.dm-1.in‚0 disk.drbd2080.in‚0 disk.drbd2080.out‚0 disk.sda2.out‚0 cpu‚26.085957370061745' disk.nvme0c0n1.out‚2890.69189453125 disk.drbd2265.in‚0 network.recv‚343541 disk.rbd1.in‚0 disk.dm-0.in‚ 11.80859375 network.sent‚ 1.421533e+06 disk.sda.out‚ 29.68359375 disk.drbd2265.out‚0' proc.memory.availableMB‚ 2358.828125 disk.dm-1.out‚ 29.68359375 disk.drbd1716.out‚0 disk.sda1.out‚0 disk.sda2.in‚0 disk.drbd2266.in‚0{@“¾ ‚(:ˆ Ï™äÔÌŽ¨¨ gpu.0.gpu‚34.0 gpu.0.memory‚4, gpu.0.memoryAllocated‚12.165603637695312* gpu.0.memoryAllocatedBytes‚ 10450173952 gpu.0.temp‚36.0 gpu.0.powerWatts‚169.724' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚33.9448 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0¢KþùR¿ ‚(:Ç Ï™äÔÝý½ª disk.sda.in‚0 disk.sda.out‚ 30.1171875 disk.nvme0n1.in‚ 11.80859375 network.sent‚ 1.509063e+06* disk./.usagePercent‚16.792689532822706 disk.dm-0.in‚ 11.80859375 disk.drbd2080.out‚0 disk.drbd2265.out‚0% disk.nvme0n1.out‚2929.86376953125 disk.drbd2266.in‚0$ memory_percent‚85.60287952423096 cpu‚26.079243374013895 disk.rbd1.out‚ 1.2265625! disk.nvme0c0n1.in‚ 11.80859375" disk.dm-0.out‚2929.98876953125 disk.sda3.out‚ 30.1171875 disk.dm-1.out‚ 30.1171875 disk.drbd2266.out‚0 proc.cpu.threads‚17' disk.nvme0c0n1.out‚2929.86376953125 disk.drbd1716.in‚0 disk.rbd0.in‚0 disk.sda2.in‚0% disk./.usageGB‚250.22320938110352 disk.sda1.in‚0 disk.sda1.out‚0 disk.sda2.out‚0) proc.memory.availableMB‚ 2358.82421875 disk.rbd1.in‚0 disk.dm-1.in‚0 disk.rbd0.out‚0 proc.memory.rssMB‚ 2776.09375 disk.sda3.in‚0 disk.drbd1716.out‚0 disk.drbd2080.in‚0 disk.drbd2265.in‚0* proc.memory.percent‚16.943931579589844 network.recv‚350279ÆÛÂ0À + _wandbruntime‚774  _runtime‚774dà ®ÄÁ ¾ & lr-AdamW/pg1‚0.0018560226985131659 & lr-AdamW/pg2‚0.0018560226985131659 trainer/global_step‚949 % _timestamp‚1.7884151929364123e+09 _runtime‚ 774.586819749 _step‚73ä?^>È Â & lr-AdamW/pg2‚0.0018560226985131659 trainer/global_step‚949 % _timestamp‚1.7884151929364123e+09 _runtime‚ 774.586819749 _step‚73 & lr-AdamW/pg1‚0.0018560226985131659IiÆŠy0à + _wandbruntime‚774  _runtime‚774¢y]‰¸'Ä ²O PCDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated‚17860768 SDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 9283702784 [HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚53794295964160 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 BDeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 M@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚12873428 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 _LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚52306392090624 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 446693376 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚24126 _LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚53794313134592 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚8194220 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚213 trainer/global_step‚949 PCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚13643128 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚24126 NFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚985 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 PCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚13643126 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚511 SGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚8194217 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚4679719 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 242306048 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 169267200 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 435025408 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 435025408 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 O@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 9373790785 G?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚431 Q>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚54498847454720 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚2605 XJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 165393012 QBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 9902751744 L?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚13643126 SEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 313315328 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚2 M@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚31503896 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚20036 ]JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚55094596467712 VEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 704551490560 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚20038 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚24399 PCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚17840732 QCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 182432372 O@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 9679898624 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 TADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚53007155737391 XEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚53794295964160 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 N?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 9902751744 JBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚431 _runtime‚ 774.586916983 G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚2605 VGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 9283702784 % _timestamp‚1.7884151929371557e+09 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚20038 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚431 YFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚55094357183488 G?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚213 IDeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚54498847454720 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚2605 XJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 165393012 QBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 9902751744 L?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚13643126 SEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 313315328 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚2 M@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚31503896 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚20036 ]JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚55094596467712 VEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 704551490560 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚20038 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚24399 PCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚17840732 QCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 182432372 O@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 9679898624 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 TADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚53007155737391 XEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚53794295964160 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 N?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 9902751744 JBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚431 _runtime‚ 774.586916983 G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚2605 VGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 9283702784 % _timestamp‚1.7884151929371557e+09 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚20038 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚431 YFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚55094357183488 G?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚213 IDeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 M@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚12873428 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 _LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚52306392090624 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 446693376 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚24126 _LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚53794313134592 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚8194220 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚213 trainer/global_step‚949JVƒÎ0Æ + _wandbruntime‚775  _runtime‚775çºïÂ&Ç ¼M UIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚4684437 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚213 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 446693376 ODeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 ]JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚53849676824064 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚2605 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 431144265 XIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 9456058368 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚13657126 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1003 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚4683949 [MDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚ 224984064 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 169280512 TCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 705276844544 R?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚54554936498176 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚8202608 trainer/global_step‚949 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚488 YFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚52360239718400 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚20064 G:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚31516330 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚8202611 bODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚54416305837056 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚35 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 VGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 9456058368 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚20062 SDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 9456058368 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 [JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 705446125056 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚3 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 144203776 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚431 REDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚13657128 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚213 C6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚31516330 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17170432 QBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 9283702784 F8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚644 S@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚54555122949120 QCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 313315328 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 182432580 EDeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚31536394 K>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚12886557 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 QBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 9902751744 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 446693376 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚431 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚24399 TEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 8980994048 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 XGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 705446125056 [JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 701667526055 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚431 J=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚17859204 J;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 9679898624 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 249756672 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 242306048 L=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 9902751744 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚2 NADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚17879266 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚20062 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚24126 WFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 701502132835 SEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 435025408 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚24399 REDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚17879266 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 M>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 9679898624 BDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚17170432 OBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚12887048 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 _runtime‚ 775.261518487 WDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚55151043210752 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24772608 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚644 PBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 435025408 [HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚55151292967424 ^KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚54416281064448 M>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 9373790785 O@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 9902751744 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚644 _step‚75 L>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 186450944 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 NADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚13657126 LDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚985 `ODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 734987130368 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 VHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 165393220 NADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚13657128 F6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚644 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 VCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚54555122949120 OADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 186450944 NADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚17859204 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚491û†¼^Æ&È ÀM ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 QBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 9902751744 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 446693376 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚431 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚24399 TEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 8980994048 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 XGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 705446125056 [JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 701667526055 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚431 J=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚17859204 J;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 9679898624 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 249756672 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 242306048 L=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 9902751744 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚2 NADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚17879266 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚20062 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚24126 WFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 701502132835 SEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 435025408 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚24399 REDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚17879266 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 M>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 9679898624 BDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚17170432 OBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚12887048 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 _runtime‚ 775.261518487 WDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚55151043210752 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24772608 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚644 PBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 435025408 [HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚55151292967424 ^KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚54416281064448 M>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 9373790785 O@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 9902751744 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚644 _step‚75 L>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 186450944 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 NADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚13657126 LDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚985 `ODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 734987130368 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 VHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 165393220 NADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚13657128 F6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚644 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 VCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚54555122949120 OADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 186450944 NADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚17859204 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚491 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚4684437 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚213 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 446693376 ODeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 ]JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚53849676824064 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚2605 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 431144265 XIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 9456058368 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚13657126 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1003 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚4683949 [MDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚ 224984064 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 169280512 TCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 705276844544 R?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚54554936498176 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚8202608 trainer/global_step‚949 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚488 YFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚52360239718400 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚20064 G:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚31516330 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚8202611 bODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚54416305837056 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚35 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 VGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 9456058368 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚20062 SDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 9456058368 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 [JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 705446125056 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚3 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 144203776 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚431 REDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚13657128 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚213 C6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚31516330 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17170432 QBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 9283702784 F8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚644 S@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚54555122949120 QCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 313315328 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 182432580 EDeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚31536394 K>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚12886557KÇ7·ö0É + _wandbruntime‚775  _runtime‚775ÞŽ+×VÊ Ð 0loss/FRt3/assignment_loss‚3.8322943933308125 0loss/FBt3/assignment_loss‚0.4748636140506177 /loss/FBt1/detection_loss‚2.7349443866146936 2loss/SRqqt3/assignment_loss‚1.8344401976754587 0loss/FBt1/assignment_loss‚0.2984649630736587 0loss/FBt4/assignment_loss‚0.1438403636310464 .loss/FBt2/detection_loss‚2.783912115626865 -loss/FBt3/detection_loss‚2.71638452510039 /loss/FBt4/detection_loss‚0.8882601225195685 2loss/SRqqt2/assignment_loss‚2.7133385089405797 1loss/SRqqt1/detection_loss‚1.7874144627047437 1loss/SRqqt3/detection_loss‚1.8989606188830508 %loss/total_loss‚50.57975958616822 /loss/FRt2/detection_loss‚2.5651111622988165 .loss/FRt4/detection_loss‚2.093270526049619 1loss/SRqqt4/detection_loss‚1.0737388505538306 epoch‚1 /loss/FRt2/assignment_loss‚4.985449617191897 0loss/FBt2/assignment_loss‚0.4280219541651099 /loss/FRt3/detection_loss‚2.8455628086812794 1loss/SRqqt2/detection_loss‚2.7639535723025337 0loss/FRt1/assignment_loss‚3.6619961106476664 0loss/FRt4/assignment_loss‚2.3252031987017774 .loss/FRt1/detection_loss‚2.050749509149239 trainer/global_step‚949 _step‚76 2loss/SRqqt1/assignment_loss‚1.9728397653172058 _runtime‚ 775.262848016 2loss/SRqqt4/assignment_loss‚1.7067442389577627 % _timestamp‚1.7884151936133385e+09²ŒTËZË Ô 2loss/SRqqt3/assignment_loss‚1.8344401976754587 0loss/FBt1/assignment_loss‚0.2984649630736587 0loss/FBt4/assignment_loss‚0.1438403636310464 .loss/FBt2/detection_loss‚2.783912115626865 -loss/FBt3/detection_loss‚2.71638452510039 /loss/FBt4/detection_loss‚0.8882601225195685 2loss/SRqqt2/assignment_loss‚2.7133385089405797 1loss/SRqqt1/detection_loss‚1.7874144627047437 1loss/SRqqt3/detection_loss‚1.8989606188830508 %loss/total_loss‚50.57975958616822 /loss/FRt2/detection_loss‚2.5651111622988165 .loss/FRt4/detection_loss‚2.093270526049619 1loss/SRqqt4/detection_loss‚1.0737388505538306 epoch‚1 /loss/FRt2/assignment_loss‚4.985449617191897 0loss/FBt2/assignment_loss‚0.4280219541651099 /loss/FRt3/detection_loss‚2.8455628086812794 1loss/SRqqt2/detection_loss‚2.7639535723025337 0loss/FRt1/assignment_loss‚3.6619961106476664 0loss/FRt4/assignment_loss‚2.3252031987017774 .loss/FRt1/detection_loss‚2.050749509149239 trainer/global_step‚949 _step‚76 2loss/SRqqt1/assignment_loss‚1.9728397653172058 _runtime‚ 775.262848016 2loss/SRqqt4/assignment_loss‚1.7067442389577627 % _timestamp‚1.7884151936133385e+09 0loss/FRt3/assignment_loss‚3.8322943933308125 0loss/FBt3/assignment_loss‚0.4748636140506177 /loss/FBt1/detection_loss‚2.7349443866146936L¦.¼ážÌ ‚(:“ Þ™äÔÿÈ¥© gpu.0.gpu‚54.0 gpu.0.memory‚6, gpu.0.memoryAllocated‚12.165603637695312* gpu.0.memoryAllocatedBytes‚ 10450173952 gpu.0.temp‚35.0 gpu.0.powerWatts‚125.935' gpu.0.enforcedPowerLimitWatts‚500.0) gpu.0.powerPercent‚25.186999999999998 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0"ÐQÍ ‚(:Æ Þ™äÔÃú†« disk.drbd2265.in‚0' disk.nvme0c0n1.out‚2991.24658203125 disk.dm-0.in‚ 11.80859375 disk.dm-1.out‚ 30.859375* proc.memory.percent‚16.943955421447754! disk.nvme0c0n1.in‚ 11.80859375 disk.dm-1.in‚0 disk.drbd1716.out‚0# memory_percent‚85.6032133102417% disk./.usageGB‚250.24217224121094 disk.sda1.out‚0 disk.rbd1.out‚ 1.2265625 disk.sda.in‚0 disk.drbd2266.out‚0 cpu‚26.072440794197295 network.sent‚ 1.515333e+06% disk.nvme0n1.out‚2991.24658203125 disk.drbd2266.in‚0 disk.sda.out‚ 30.859375 disk.sda3.out‚ 30.859375 disk.drbd2080.out‚0 disk.rbd0.out‚0# proc.memory.rssMB‚ 2776.09765625 disk.nvme0n1.in‚ 11.80859375 disk.sda1.in‚0 disk.sda2.in‚0 disk.drbd2080.in‚0 disk.sda3.in‚0* disk./.usagePercent‚16.793962146275415 disk.drbd2265.out‚0 disk.rbd1.in‚0" disk.dm-0.out‚2991.37158203125 proc.cpu.threads‚17 network.recv‚354974 disk.sda2.out‚0 disk.drbd1716.in‚0 disk.rbd0.in‚0) proc.memory.availableMB‚ 2358.76953125¸¹KŸ’Î ‚(:‡ í™äÔ¸£¸¨ gpu.0.gpu‚8.0 gpu.0.memory‚1, gpu.0.memoryAllocated‚12.165603637695312* gpu.0.memoryAllocatedBytes‚ 10450173952 gpu.0.temp‚36.0 gpu.0.powerWatts‚171.621' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚34.3242 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0ܤ¿ KÏ ‚(:À í™äÔ±¨¸ª network.recv‚360057* disk./.usagePercent‚16.795068610838403 disk.sda3.out‚31.3125 disk.rbd0.out‚0 disk.sda2.in‚0) proc.memory.availableMB‚ 2358.76171875" proc.memory.rssMB‚ 2776.1015625 proc.cpu.threads‚17 disk.sda.out‚31.3125 disk.dm-1.out‚31.3125 disk.drbd2080.in‚0$ memory_percent‚85.60326099395752% disk./.usageGB‚250.25865936279297 disk.sda3.in‚0 disk.nvme0n1.in‚ 11.80859375 disk.sda1.out‚0 disk.drbd2266.in‚0 disk.drbd2265.in‚0 disk.drbd2265.out‚0 cpu‚26.065660404865586 network.sent‚ 1.601643e+06 disk.rbd1.out‚ 1.2265625" disk.dm-0.out‚3034.66455078125! disk.nvme0c0n1.in‚ 11.80859375' disk.nvme0c0n1.out‚3034.53955078125 disk.dm-0.in‚ 11.80859375 disk.dm-1.in‚0 disk.drbd1716.out‚0 disk.rbd0.in‚0 disk.sda1.in‚0% disk.nvme0n1.out‚3034.53955078125* proc.memory.percent‚16.943979263305664 disk.rbd1.in‚0 disk.sda.in‚0 disk.drbd1716.in‚0 disk.drbd2080.out‚0 disk.sda2.out‚0 disk.drbd2266.out‚0[ô-§Ð ‚(:’ ü™äÔ󖔨 gpu.0.gpu‚29.0 gpu.0.memory‚3, gpu.0.memoryAllocated‚12.165603637695312* gpu.0.memoryAllocatedBytes‚ 10450173952 gpu.0.temp‚36.0 gpu.0.powerWatts‚155.05' gpu.0.enforcedPowerLimitWatts‚500.0) gpu.0.powerPercent‚31.010000000000005 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0¨HFQÑ ‚(:Æ ü™äÔÈé“«) disk./.usagePercent‚16.79628925476166 disk.drbd1716.in‚0 disk.sda1.in‚0) proc.memory.availableMB‚ 2358.76953125 disk.drbd2266.in‚0 disk.rbd1.out‚ 1.2265625 disk.sda.in‚0 disk.sda.out‚ 31.90234375 disk.sda3.out‚ 31.90234375 disk.dm-1.in‚0 disk.drbd2080.in‚0 disk.sda2.out‚0 network.sent‚ 1.606957e+06% disk./.usageGB‚250.27684783935547! disk.nvme0c0n1.in‚ 11.80859375 disk.rbd0.out‚0" proc.memory.rssMB‚ 2776.1015625 disk.drbd2266.out‚0 disk.drbd2265.out‚0 disk.drbd2080.out‚0 disk.rbd0.in‚0 disk.nvme0n1.in‚ 11.80859375 proc.cpu.threads‚17& disk.nvme0c0n1.out‚3080.0029296875 disk.sda3.in‚0* proc.memory.percent‚16.943979263305664 disk.drbd2265.in‚0 disk.rbd1.in‚0 disk.dm-0.in‚ 11.80859375 disk.drbd1716.out‚0$ disk.nvme0n1.out‚3080.0029296875 cpu‚26.05903838459168 network.recv‚363493! disk.dm-0.out‚3080.1279296875 disk.dm-1.out‚ 31.90234375 disk.sda1.out‚0 disk.sda2.in‚0# memory_percent‚85.6032133102417±ˆ¼p0Ò + _wandbruntime‚813  _runtime‚813׿…ùÃÓ ½ & lr-AdamW/pg1‚0.0018502754567215854 & lr-AdamW/pg2‚0.0018502754567215854 trainer/global_step‚999 % _timestamp‚1.7884152318394012e+09 _runtime‚ 813.48874792 _step‚77!LÝpÇÔ Á & lr-AdamW/pg1‚0.0018502754567215854 & lr-AdamW/pg2‚0.0018502754567215854 trainer/global_step‚999 % _timestamp‚1.7884152318394012e+09 _runtime‚ 813.48874792 _step‚77M˾¢µ0Õ + _wandbruntime‚813  _runtime‚813ù:÷¤Ö ²O LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚431 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚2605 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 XEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚56562480437760 VEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 740847017984 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 186437632 VGDeviceStatsMonitor.on_train_batch_start/allocat}Nˆv%ed_bytes.large_pool.peak‚ 9283702784 PCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚18767684 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚486 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚24126 ZIDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 741016285184 TADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚57303327455744 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 247672832 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 UBDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated‚57303513893376 bQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 772132957696 TFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 446693376 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚20038 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚20036 `MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚57157068596224 XIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 9456058368 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚213 trainer/global_step‚999 IDeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚57303327455744 ]LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 741016285184 M@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚13530034 SGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚4916267 dQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚57157093368832 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 242306048 @8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚644 C;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚644 @8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚644 _runtime‚ 813.493849358 PCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚14343128 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚483 QCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 182432372 TADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚55736488601599 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 YHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 740847017984 TGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚14343128 QDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚13530520 D>DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 435025408 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚20038 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 ]LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 737045586547 XJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 165393012 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 446693376 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚2 L?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚14343126 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 _LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚54999625447424 PCDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated‚18787720 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚3 YFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚57928978653696 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 XEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚55736671033971 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 169267200 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 435025408 ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 222900224 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 % _timestamp‚1.7884152318403914e+09 N@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 186437632 \IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚56562497608192 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚17170432 ^MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 771910057472 YHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 736880193535 UGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 431144265 E8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚33110810 UFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 9456058368 G?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚213 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 QBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 9902751744 G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚2605 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 NFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚985 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 [HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚56562480437760 TGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚18787720 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 XEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚57303513893376 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 169267200 IDeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚2605 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 NFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚985 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 [HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚56562480437760 TGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚18787720 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 XEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚57303513893376 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 169267200 IDeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚24126 ZIDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 741016285184 TADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚57303327455744 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 247672832 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 UBDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated‚57303513893376 bQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 772132957696 TFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 446693376 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚20038 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚20036 `MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚57157068596224 XIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 9456058368 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚213 trainer/global_step‚999 IDeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚57303327455744 ]LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 741016285184 M@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚13530034 SGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚4916267 dQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚57157093368832 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 242306048 @8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚644 C;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚644 @8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚644 _runtime‚ 813.493849358 PCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚14343128 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚483 QCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 182432372 TADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚55736488601599 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 YHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 740847017984 TGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚14343128 QDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚13530520 D>DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 435025408 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚20038 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 ]LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 737045586547 XJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 165393012 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 446693376 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚2 L?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚14343126 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 _LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚54999625447424 PCDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated‚18787720 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚3 YFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚57928978653696 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 XEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚55736671033971 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 169267200 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 435025408 ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 222900224 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 % _timestamp‚1.7884152318403914e+09 N@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 186437632 \IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚56562497608192 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚17170432 ^MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 771910057472 YHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 736880193535N9×>‹0Ø + _wandbruntime‚814  _runtime‚814}|(ëÂ&Ù ¼M UIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚8622161 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚8622158 ^KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚57213003035648 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 VHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 165393220 C6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚33143282 J;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 9679898624 NADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚18786156 _runtime‚ 814.173592022 G:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚33163346 OADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 186450944 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚4921497 LDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚985 WDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚57985675910144 FDeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚13543187 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚24126 QCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 313315328 J=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚18786156 REDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚18806218 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚3 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚644 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚213 L=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 9902751744 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚644 NADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚18806218 VCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚57359602950144 TEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 8980994048 _step‚79 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 QBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 9283702784 TEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 9283702784 G:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚33143282 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚2605 [HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚57985917278208 \KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 772672874496 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 242306048 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚431 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 BDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 9373790785 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚213 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 169280512 NADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚14357126 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚644 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 QBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 9902751744 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 VGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 9456058368 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚431 XGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 741741652480 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚24399 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚471 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 YFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚56617844127232 K>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚33163346 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚468 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 431144265 [JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 737767033463 S@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚57359602950144 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1003 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 YFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚55053473075200 % _timestamp‚1.7884152325237339e+09 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚14357126 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚17170432 REDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚14357128 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚431 WFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 741572371968 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 241368064 `ODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 772889469952 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 trainer/global_step‚999 L>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 186450944 ZGDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚56617861297664 PBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 435025408 ]JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚56617861297664 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24772608 WFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 737601640243 SEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 435025408 FDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 9679898624 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚24399 R?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚57359416499200 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 WIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 446693376 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚20062 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 NADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚14357128 VCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚56617844127232 [JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 741741652480;»&  Ú ÀM REDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚14357128 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚431 WFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 741572371968 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 241368064 `ODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 772889469952 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 trainer/global_step‚999 L>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 186450944 ZGDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚56617861297664 PBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 435025408 ]JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚56617861297664 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24772608 WFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 737601640243 SEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 435025408 FDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 9679898624 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚24399 R?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚57359416499200 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 WIDeviceStatsMonitSÒñ«or.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 446693376 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚20062 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 NADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚14357128 VCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚56617844127232 [JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 741741652480 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚8622161 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚8622158 ^KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚57213003035648 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 VHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 165393220 C6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚33143282 J;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 9679898624 NADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚18786156 _runtime‚ 814.173592022 G:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚33163346 OADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 186450944 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚4921497 LDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚985 WDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚57985675910144 FDeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚13543187 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚24126 QCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 313315328 J=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚18786156 REDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚18806218 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚3 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚644 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚213 L=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 9902751744 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚644 NADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚18806218 VCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚57359602950144 TEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 8980994048 _step‚79 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 QBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 9283702784 TEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 9283702784 G:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚33143282 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚2605 [HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚57985917278208 \KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 772672874496 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 242306048 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚431 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 BDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 9373790785 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚213 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 169280512 NADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚14357126 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚644 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 QBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 9902751744 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 VGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 9456058368 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚431 XGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 741741652480 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚24399 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚471 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 YFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚56617844127232 K>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚33163346 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚468 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 431144265 [JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 737767033463 S@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚57359602950144 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1003 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 YFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚55053473075200 % _timestamp‚1.7884152325237339e+09 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚14357126 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚17170432O–ʵ00Û + _wandbruntime‚814  _runtime‚814‹£’ XÜ Ò 0loss/FRt4/assignment_loss‚2.5355722683447377 2loss/SRqqt1/assignment_loss‚2.1281185845534005 2loss/SRqqt3/assignment_loss‚2.0194763066817303 .loss/FBt3/detection_loss‚2.439836111664772 /loss/FBt4/detection_loss‚0.6919370053701697 _runtime‚ 814.175137078 1loss/FBt1/assignment_loss‚0.19361880815122276 1loss/FBt4/assignment_loss‚0.11289840563797215 _step‚80 /loss/FRt2/assignment_loss‚4.318361819544925 0loss/SRqqt2/assignment_loss‚2.33267279955057 1loss/FBt3/assignment_loss‚0.36076015532016753 /loss/FRt2/detection_loss‚2.3796789178365394 .loss/FBt1/detection_loss‚2.573741841316223 % _timestamp‚1.7884152325259407e+09 1loss/FBt2/assignment_loss‚0.36821980023333956 .loss/FRt4/detection_loss‚2.071046350914755 1loss/SRqqt4/detection_loss‚1.1892850161840518 /loss/FRt1/assignment_loss‚3.859536829541941 /loss/FRt1/detection_loss‚2.0955103159439368 1loss/SRqqt1/detection_loss‚1.6083419259750482 1loss/SRqqt2/detection_loss‚2.4297459033819346 .loss/FRt3/detection_loss‚2.931335125118494 1loss/SRqqt3/detection_loss‚2.0032212341926536 0loss/FRt3/assignment_loss‚3.8193193344133243 2loss/SRqqt4/assignment_loss‚1.5699023854297895 /loss/FBt2/detection_loss‚2.6194577657658122 %loss/total_loss‚48.65159501106751 epoch‚1 trainer/global_step‚999§Ñ]€\Ý Ö 1loss/FBt4/assignment_loss‚0.11289840563797215 _step‚80 /loss/FRt2/assignment_loss‚4.318361819544925 0loss/SRqqt2/assignment_loss‚2.33267279955057 1loss/FBt3/assignment_loss‚0.36076015532016753 /loss/FRt2/detection_loss‚2.3796789178365394 .loss/FBt1/detection_loss‚2.573741841316223 % _timestamp‚1.7884152325259407e+09 1loss/FBt2/assignment_loss‚0.36821980023333956 .loss/FRt4/detection_loss‚2.071046350914755 1loss/SRqqt4/detection_loss‚1.1892850161840518 /loss/FRt1/assignment_loss‚3.859536829541941 /loss/FRt1/detection_loss‚2.0955103159439368 1loss/SRqqt1/detection_loss‚1.6083419259750482 1loss/SRqqt2/detection_loss‚2.4297459033819346 .loss/FRt3/detection_loss‚2.931335125118494 1loss/SRqqt3/detection_loss‚2.0032212341926536 0loss/FRt3/assignment_loss‚3.8193193344133243 2loss/SRqqt4/assignment_loss‚1.5699023854297895 /loss/FBt2/detection_loss‚2.6194577657658122 %loss/total_loss‚48.65159501106751 epoch‚1 trainer/global_step‚999 0loss/FRt4/assignment_loss‚2.5355722683447377 2loss/SRqqt1/assignment_loss‚2.1281185845534005 2loss/SRqqt3/assignment_loss‚2.0194763066817303 .loss/FBt3/detection_loss‚2.439836111664772 /loss/FBt4/detection_loss‚0.6919370053701697 _runtime‚ 814.175137078 1loss/FBt1/assignment_loss‚0.19361880815122276PsÝo£Þ ‚(:’ ‹šäÔËÙŠ© gpu.0.gpu‚54.0 gpu.0.memory‚7, gpu.0.memoryAllocated‚12.165603637695312* gpu.0.memoryAllocatedBytes‚ 10450173952 gpu.0.temp‚35.0 gpu.0.powerWatts‚119.86' gpu.0.enforcedPowerLimitWatts‚500.0) gpu.0.powerPercent‚23.971999999999998 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0ôªËûLß ‚(:Á ‹šäԞΖª network.sent‚ 1.613414e+06 disk.dm-1.in‚0 disk.drbd2080.in‚0 disk.rbd0.out‚0$ disk.nvme0n1.out‚3124.0615234375 disk.sda1.in‚0 disk.drbd2265.in‚0 disk.rbd1.out‚ 1.2265625! disk.nvme0c0n1.in‚ 11.80859375! disk.dm-0.out‚3124.1865234375 proc.cpu.threads‚17) disk./.usagePercent‚16.79738394297807% disk./.usageGB‚250.29315948486328 disk.sda3.in‚0 disk.drbd2080.out‚0 disk.rbd0.in‚0" proc.memory.rssMB‚ 2776.1015625 disk.sda2.in‚0 disk.drbd2266.in‚0 disk.drbd2266.out‚0 disk.sda.in‚0 disk.dm-1.out‚ 32.5390625 disk.nvme0n1.in‚ 11.80859375 cpu‚26.05334601475336 network.recv‚368260 disk.sda2.out‚0 disk.drbd2265.out‚0 disk.dm-0.in‚ 11.80859375 disk.sda.out‚ 32.5390625 disk.sda3.out‚ 32.5390625# memory_percent‚85.6022834777832& disk.nvme0c0n1.out‚3124.0615234375 disk.drbd1716.in‚0 disk.rbd1.in‚0 disk.drbd1716.out‚0' proc.memory.availableMB‚ 2358.921875* proc.memory.percent‚16.943979263305664 disk.sda1.out‚0…23à ‚(:’ ššäÔæ踨 gpu.0.gpu‚46.0 gpu.0.memory‚4, gpu.0.memoryAllocated‚12.165603637695312* gpu.0.memoryAllocatedBytes‚ 10450173952 gpu.0.temp‚36.0 gpu.0.powerWatts‚156.76' gpu.0.enforcedPowerLimitWatts‚500.0) gpu.0.powerPercent‚31.351999999999997 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0ç<{Bá ‚(:· ššäÔÇҪ disk.drbd2266.out‚0 disk.rbd1.out‚ 1.2265625 disk.drbd2080.out‚0 network.sent‚ 1.700532e+06% disk./.usageGB‚250.30422592163086 disk.sda.in‚0 disk.sda2.out‚0 proc.cpu.threads‚17 disk.sda.out‚33.1875 disk.sda3.in‚0 disk.dm-1.in‚0% disk.nvme0c0n1.out‚3176.498046875 disk.dm-1.out‚33.1875 disk.drbd1716.out‚0 disk.rbd0.in‚0 disk.sda2.in‚0" proc.memory.rssMB‚ 2776.1015625 cpu‚26.047182578101964 disk.sda1.out‚0! disk.nvme0c0n1.in‚ 11.80859375 disk.dm-0.out‚3176.623046875 disk.drbd2266.in‚0 disk.rbd1.in‚0 disk.drbd2080.in‚0 disk.rbd0.out‚0 disk.sda1.in‚0( proc.memory.availableMB‚ 2358.6640625 disk.nvme0n1.in‚ 11.80859375# disk.nvme0n1.out‚3176.498046875 disk.drbd2265.out‚0 disk.dm-0.in‚ 11.80859375$ memory_percent‚85.60385704040527 network.recv‚374413 disk.sda3.out‚33.1875( disk./.usagePercent‚16.7981266208349 disk.drbd1716.in‚0 disk.drbd2265.in‚0* proc.memory.percent‚16.943979263305664í¶™0â + _wandbruntime‚852  _runtime‚852]5[þÂã ¼ _runtime‚ 852.143477576 _step‚81 % lr-AdamW/pg1‚0.001843765939825658 % lr-AdamW/pg2‚0.001843765939825658 trainer/global_step‚1049 $ _timestamp‚1.788415270494118e+09ÝÇ–Æä À $ _timestamp‚1.788415270494118e+09 _runtime‚ 852.143477576 _step‚81 % lr-AdamW/pg1‚0.001843765939825658 % lr-AdamW/pg2‚0.001843765939825658 trainer/global_step‚1049QgÛ¨\0å + _wandbruntime‚852  _runtime‚852¥(¦ê¸'æ ²O H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚24126 L=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 9679898624 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚2605 SGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚9033317 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚2 SEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 313315328 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 ]LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 773145093955 QBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 9902751744 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 IDeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚2605 MFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚35 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚5155864 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 243478528 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 XIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 9456058368 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 169267200 O@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 9373790785 YHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 772979700943 ZKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 9456058368 JBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚213 SDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 9283702784 YHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 777142545408 VGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 8980994048 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 _step‚82 XEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚60107993894400 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 169267200 PCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚15043126 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚24772608 N?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 9902751744 bQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 810019874304 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 446693376 XEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚59330664911360 _LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚59330682081792 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 242306048 QCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 182432372 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚17170432 TGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚19714672 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚24126 YFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚60763591735808 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 C;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚644 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚24399 PCDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated‚19714672 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚9033320 XEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚58466003898179 XJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 165393012 VEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 777142545408 L?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚15043126 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚17170432 D>DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 $ _timestamp‚1.788415270495099e+09 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚20038 NFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚985 UGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 431144265 SDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 9902751744 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 M@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚34757800 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚24399 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 TADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚58465821465807 G?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚213 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚20038 Q>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚60107807456768 ZIDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 777311812608 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 186437632 @8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚644 JBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚431 TADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚60107807456768 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 435025408 QDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚14189184 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚470 ]JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚60763835214336 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 L?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚19694636 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 435025408 UBDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated‚60107993894400 N@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 186437632 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚20036 ]LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 777311812608 [HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚57692841764864 G?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚431 TFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 446693376 [HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚59330664911360 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚3 UFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 9456058368 \IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚59330682081792 TGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚15043128 ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 218705920 ^MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 809801168384 _LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚57692858804224 E8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚34737762 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 446693376 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚467 SGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚5155397 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚431 O@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 9679898624 `MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚59953790567424 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 PCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚15043128 VGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 9283702784 IDeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 $ _timestamp‚1.788415270495099e+09 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚20038 NFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚985 UGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 431144265 SDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 9902751744 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 M@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚34757800 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚24399 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 TADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚58465821465807 G?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚213 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚20038 Q>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚60107807456768 ZIDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 777311812608 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 186437632 @8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚644 JBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚431 TADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚60107807456768 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 435025408 QDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚14189184 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚470 ]JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚60763835214336 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 L?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚19694636 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 435025408 UBDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated‚60107993894400 N@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 186437632 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚20036 ]LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 777311812608 [HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚57692841764864 G?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚431 TFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 446693376 [HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚59330664911360 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚3 UFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 9456058368 \IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚59330682081792 TGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚15043128 ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 218705920 ^MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 809801168384 _LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚57692858804224 E8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚34737762 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 446693376 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚467 SGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚5155397 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚431 O@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 9679898624 `MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚59953790567424 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 PCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚15043128 VGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 9283702784 IDeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚24126 L=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 9679898624 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚2605 SGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚9033317 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚2 SEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 313315328 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 ]LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 773145093955 QBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 9902751744 MGD½RJeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 IDeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚2605 MFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚35 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚5155864 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0R}¬n¶0è + _wandbruntime‚852  _runtime‚852ÀYÃÃ&é ½M I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚20062 REDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚15057128 NADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚15057126 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚20062 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 431144265 REDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚19733170 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 249756672 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚431 J=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚19713108 ^KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚60009725006848 [JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 773866540871 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚20064 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚24399 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 446693376 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚15057126 TCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 777867899392 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚9041711 L=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 9902751744 SDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 9456058368 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚644 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚644 R?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚60163896500224 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚9041708 BDeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 G:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚34770234 QCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 313315328 OBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚14202284 [HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚60820531552256 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 QBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 9283702784 SEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 435025408 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 NADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚19713108 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚5160093 LDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚985 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 TEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 8980994048 VGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 9456058368 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 169280512 PBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 435025408 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 242306048 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 L>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 186450944 \KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 810556788736 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚213 % _timestamp‚1.7884152712092109e+09 NADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚19733170 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17170432 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚24126 WFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 773701147651 J;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 9679898624 XGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 778037179904 VCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚60164082951168 OADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 186450944 K>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚14201801 M>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 9373790785 VHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 165393220 _step‚83 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚213 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚2605 WDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚60820281795584 WIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 446693376 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚35 C6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚34770234 ZGDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚59386045771264 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚17170432 [JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 778037179904 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚17039360 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 E6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚644 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 O@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 9902751744 XIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 9456058368 K>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚34790298 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 M>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 9679898624 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 169280512 WFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 777867899392 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 trainer/global_step‚1049 TEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 9283702784 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚431 G:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚34790298 FDeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 186450944 \KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 810556788736 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚213 % _timestamp‚1.7884152712092109e+09 NADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚19733170 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17170432 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚24126 WFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 773701147651 J;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 9679898624 XGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 778037179904 VCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚60164082951168 OADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 186450944 K>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚14201801 M>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 9373790785 VHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 165393220 _step‚83 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚213 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚2605 WDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚60820281795584 WIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 446693376 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚35 C6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚34770234 ZGDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚59386045771264 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚17170432 [JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 778037179904 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚17039360 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 E6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚644 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 O@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 9902751744 XIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 9456058368 K>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚34790298 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 M>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 9679898624 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 169280512 WFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 777867899392 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 trainer/global_step‚1049 TEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 9283702784 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚431 G:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚34790298 FDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 446693376 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚15057126 TCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 777867899392 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚9041711 L=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 9902751744 SDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 9456058368 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚644 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚644 R?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚60163896500224 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚9041708 BDeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 G:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚34770234 QCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 313315328 OBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚14202284 [HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚60820531552256 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 QBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 9283702784 SEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 435025408 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0SÒ±å 0ë + _wandbruntime‚852  _runtime‚852ñ(0Sì Í % _timestamp‚1.7884152712112248e+09 /loss/FRt1/assignment_loss‚4.031711272716522 2loss/SRqqt3/assignment_loss‚1.8497594728206213 2loss/SRqqt4/assignment_loss‚1.5292440639521048 /loss/FRt2/detection_loss‚2.3865980920038727 /loss/SRqqt1/detection_loss‚1.90608447238251 .loss/FBt1/detection_loss‚2.775128021836281 /loss/FBt4/detection_loss‚0.8431073467003977 .loss/FRt1/detection_loss‚2.032310696363449 0loss/SRqqt4/detection_loss‚1.229378464442443 /loss/FRt4/detection_loss‚1.8719325103693538 /loss/FRt4/assignment_loss‚2.465764497386085 .loss/FRt3/detection_loss‚2.584201436903742 1loss/SRqqt2/detection_loss‚2.5997180098028325 trainer/global_step‚1049 1loss/SRqqt1/assignment_loss‚2.177936796374895 0loss/FRt3/assignment_loss‚3.7559401210811405 1loss/SRqqt2/assignment_loss‚2.529864664842833 0loss/FBt2/assignment_loss‚0.5372711988621588 0loss/FBt3/assignment_loss‚0.3083396969252969 1loss/SRqqt3/detection_loss‚1.9278032533251321 &loss/total_loss‚50.113957449309005 1loss/FBt4/assignment_loss‚0.12135392588234631 /loss/FBt3/detection_loss‚2.7160586826503277 epoch‚1 _step‚84 /loss/FRt2/assignment_loss‚4.544881883420442 0loss/FBt1/assignment_loss‚0.2694745206964342 .loss/FBt2/detection_loss‚3.120094347567785 _runtime‚ 852.860458153a²ÃöWí Ñ 1loss/SRqqt2/detection_loss‚2.5997180098028325 trainer/global_step‚1049 1loss/SRqqt1/assignment_loss‚2.177936796374895 0loss/FRt3/assignment_loss‚3.7559401210811405 1loss/SRqqt2/assignment_loss‚2.529864664842833 0loss/FBt2/assignment_loss‚0.5372711988621588 0loss/FBt3/assignment_loss‚0.3083396969252969 1loss/SRqqt3/detection_loss‚1.9278032533251321 &loss/total_loss‚50.113957449309005 1loss/FBt4/assignment_loss‚0.12135392588234631 /loss/FBt3/detection_loss‚2.7160586826503277 epoch‚1 _step‚84 /loss/FRt2/assignment_loss‚4.544881883420442 0loss/FBt1/assignment_loss‚0.2694745206964342 .loss/FBt2/detection_loss‚3.120094347567785 _runtime‚ 852.860458153 % _timestamp‚1.7884152712112248e+09 /loss/FRt1/assignment_loss‚4.031711272716522 2loss/SRqqt3/assignment_loss‚1.8497594728206213 2loss/SRqqt4/assignment_loss‚1.5292440639521048 /loss/FRt2/detection_loss‚2.3865980920038727 /loss/SRqqt1/detection_loss‚1.90608447238251 .loss/FBt1/detection_loss‚2.775128021836281 /loss/FBt4/detection_loss‚0.8431073467003977 .loss/FRt1/detection_loss‚2.032310696363449 0loss/SRqqt4/detection_loss‚1.229378464442443 /loss/FRt4/detection_loss‚1.8719325103693538 /loss/FRt4/assignment_loss‚2.465764497386085 .loss/FRt3/detection_loss‚2.584201436903742TûT<˜žî ‚(:“ ©šäÔÏž´¨ gpu.0.gpu‚43.0 gpu.0.memory‚4, gpu.0.memoryAllocated‚12.165603637695312* gpu.0.memoryAllocatedBytes‚ 10450173952 gpu.0.temp‚36.0 gpu.0.powerWatts‚170.255' gpu.0.enforcedPowerLimitWatts‚500.0) gpu.0.powerPercent‚34.050999999999995 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0ž^ùgBï ‚(:· ©šäÔ³éäª disk.rbd0.in‚0 disk.rbd1.out‚ 1.2265625 disk.dm-0.out‚3217.259765625* proc.memory.percent‚16.943979263305664 disk.drbd2080.out‚0 disk.nvme0n1.in‚ 11.80859375$ disk./.usageGB‚250.3157501220703 disk.drbd2265.in‚0 disk.sda.out‚33.65625 disk.dm-1.in‚0 cpu‚26.04124902389638 disk.drbd2266.in‚0 disk.sda.in‚0 disk.dm-1.out‚33.65625 disk.drbd2266.out‚0 disk.drbd2265.out‚0 disk.rbd1.in‚0 disk.sda3.in‚0 disk.drbd2080.in‚0 disk.rbd0.out‚0 network.recv‚378358% disk.nvme0c0n1.out‚3217.134765625 disk.sda3.out‚33.65625# disk.nvme0n1.out‚3217.134765625 disk.sda1.in‚0' proc.memory.availableMB‚ 2358.359375 network.sent‚ 1.70591e+06! disk.nvme0c0n1.in‚ 11.80859375 disk.drbd1716.in‚0 disk.drbd1716.out‚0 disk.sda1.out‚0 disk.sda2.in‚0" proc.memory.rssMB‚ 2776.1015625 proc.cpu.threads‚17) disk./.usagePercent‚16.79890001959584 disk.dm-0.in‚ 11.80859375 disk.sda2.out‚0$ memory_percent‚85.60571670532227x¯×“ð ‚(:ˆ ¸šäÔ𳨠gpu.0.gpu‚54.0 gpu.0.memory‚6, gpu.0.memoryAllocated‚12.165603637695312* gpu.0.memoryAllocatedBytes‚ 10450173952 gpu.0.temp‚35.0 gpu.0.powerWatts‚130.252' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚26.0504 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0<^J¸Pñ ‚(:Å ¸šäÔüõ˜ª disk.drbd2080.in‚0 disk.rbd0.out‚0 disk.nvme0n1.in‚ 11.80859375 disk.sda2.in‚0* proc.memory.percent‚16.943979263305664 disk.drbd1716.in‚0 disk.drbd1716.out‚0& proc.memory.availableMB‚ 2358.59375 network.sent‚ 1.791582e+06% disk./.usageGB‚250.33837509155273 disk.rbd1.out‚ 1.2265625 disk.sda3.out‚ 34.2890625 disk.drbd2266.out‚0 disk.rbd1.in‚0" disk.dm-0.out‚3280.03955078125 disk.dm-1.out‚ 34.2890625 disk.rbd0.in‚0 disk.sda1.out‚0) disk./.usagePercent‚16.80041840028142 disk.drbd2265.out‚0! disk.nvme0c0n1.in‚ 11.80859375 disk.dm-1.in‚0 disk.sda2.out‚0 disk.drbd2266.in‚0 cpu‚26.035839702076494 proc.cpu.threads‚17 disk.drbd2265.in‚0 disk.sda.in‚0 disk.sda.out‚ 34.2890625 disk.sda1.in‚0" proc.memory.rssMB‚ 2776.1015625 network.recv‚383208' disk.nvme0c0n1.out‚3279.91455078125 disk.dm-0.in‚ 11.80859375 disk.drbd2080.out‚0% disk.nvme0n1.out‚3279.91455078125$ memory_percent‚85.60428619384766 disk.sda3.in‚0%¼š§“ò ‚(:ˆ ÇšäÔÿБ¨ gpu.0.gpu‚53.0 gpu.0.memory‚6, gpu.0.memoryAllocated‚12.165603637695312* gpu.0.memoryAllocatedBytes‚ 10450173952 gpu.0.temp‚36.0 gpu.0.powerWatts‚174.136' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚34.8272 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0êý4Uó ‚(:Ê ÇšäÔƒ…ª disk.drbd1716.out‚0 disk.drbd2266.in‚0 disk.drbd2265.out‚0 disk.rbd0.in‚0 disk.nvme0n1.in‚ 11.80859375% disk.nvme0n1.out‚3318.59423828125 disk.rbd1.in‚0$ memory_percent‚85.60583591461182" proc.memory.rssMB‚ 2776.1015625 disk.sda3.in‚0 disk.sda2.out‚0 disk.drbd2266.out‚0 disk.dm-0.in‚ 11.80859375 disk.dm-1.in‚0 disk.sda1.out‚0 disk.drbd2265.in‚0 network.sent‚ 1.798185e+06 disk.sda.out‚ 34.87109375 disk.drbd2080.out‚0 disk.sda2.in‚0 cpu‚26.030570809620944 network.recv‚388057 disk.drbd2080.in‚0 disk.rbd0.out‚0 disk.sda1.in‚0! disk.nvme0c0n1.in‚ 11.80859375* proc.memory.percent‚16.943979263305664) disk./.usagePercent‚16.80058915730676 disk.dm-1.out‚ 34.87109375 disk.rbd1.out‚ 1.2265625 proc.cpu.threads‚17 disk.sda.in‚0' disk.nvme0c0n1.out‚3318.59423828125) proc.memory.availableMB‚ 2358.33984375$ disk./.usageGB‚250.3409194946289" disk.dm-0.out‚3318.71923828125 disk.sda3.out‚ 34.87109375 disk.drbd1716.in‚0-lÍñ0ô + _wandbruntime‚890  _runtime‚890~¨"»Âõ ¼ % lr-AdamW/pg2‚0.001836499582823489 trainer/global_step‚1099 $ _timestamp‚1.788415309273361e+09 _runtime‚ 890.922807147 _step‚85 % lr-AdamW/pg1‚0.001836499582823489|¶­Æö À _step‚85 % lr-AdamW/pg1‚0.001836499582823489 % lr-AdamW/pg2‚0.001836499582823489 trainer/global_step‚1099 $ _timestamp‚1.788415309273361e+09 _runtime‚ 890.922807147U‚qFJ0÷ + _wandbruntime‚890  _runtime‚890¯5Fø ³O RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 435025408 L?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚20621588 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚17170432 XEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚62098849384960 MFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚35 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚498 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 253964288 BDeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚2605 H>DeviceStatsœq˜sMonitor.on_train_batch_start/active.small_pool.peak‚24126 ]LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 813607340032 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 XJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 165393012 O@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 9679898624 _LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚62098866555392 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚20038 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚24126 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚2 M@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚14846150 M@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚36384752 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚501 bQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 847905672704 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 PCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚15743128 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚24399 ]JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚63598442983936 ZKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 9456058368 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 VEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 813438072832 SDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 9283702784 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 169267200 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 242306048 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 D>DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 JBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚213 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚17170432 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 SDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 9902751744 UFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 9456058368 JBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚431 L=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 9679898624 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 186437632 ^MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 847676481024 O@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 9373790785 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚20038 Q>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚62912287457792 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 435025408 XIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 9456058368 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚20036 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 169267200 PCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚15743126 H?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚1003 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚9452870 _LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚60386092161024 XEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚62912473895424 YHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 813438072832 `MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚62750512538624 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 XEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚61195336762387 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚431 PCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚20621588 _step‚86 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 PCDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated‚20641624 TGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚20641624 NFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚985 ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 229191680 YHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 809079208351 UGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 431144265 N?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 9902751744 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 SEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 313315328 dQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚62750537311232 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 ]LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 809244601363 VGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 9283702784 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚3/<¼½'ù ·O ]LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 809244601363 VGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 9283702784 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚3 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 435025408 L?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚20621588 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚17170432 XEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚62098849384960 MFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚35 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚498 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 253964288 BDeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚2605 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚24126 ]LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 813607340032 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 XJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 165393012 O@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 9679898624 _LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚62098866555392 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚20038 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚24126 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚2 M@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚14846150 M@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚36384752 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚501 bQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 847905672704 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 PCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚15743128 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚24399 ]JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚63598442983936 ZKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 9456058368 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 VEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 813438072832 SDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 9283702784 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 169267200 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 242306048 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 D>DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 JBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚213 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚17170432 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 SDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 9902751744 UFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 9456058368 JBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚431 L=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 9679898624 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 186437632 ^MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 847676481024 O@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 9373790785 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚20038 Q>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚62912287457792 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 435025408 XIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 9456058368 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚20036 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 169267200 PCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚15743126 H?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚1003 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚9452870 _LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚60386092161024 XEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚62912473895424 YHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 813438072832 `MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚62750512538624 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 XEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚61195336762387 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚431 PCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚20621588 _step‚86 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 PCDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated‚20641624 TGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚20641624 NFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚985 ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 229191680 YHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 809079208351 UGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 431144265 N?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 9902751744 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 SEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 313315328 dQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚62750537311232 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0VV¶e0ú + _wandbruntime‚891  _runtime‚891S)“ªÃ&û ½M [JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 814332707328 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 NADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚15757126 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 446693376 SEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 435025408 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚5398106 R?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚61249740443859 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚213 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚24399 J;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 9679898624 PBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 435025408 NADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚20640060 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 249756672 NADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚20660122 OBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚14859832 bODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚62806471750656 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚644 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚17170432 \KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 848426694656 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 BDeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚36417250 REDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚20660122 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24772608 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 VCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚62154213074432 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 144203776 O@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 9902751744 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚431 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚9461258 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 VHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 165393220 _runtime‚ 891.596988143 VCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚62968562952192 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚5398571 OADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 186450944 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1003 L=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 9902751744 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚213 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚213 R?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚62968376501248 K>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚14859364 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚9461261 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚644 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 182432580 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚24126 WDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚63654873672704 QBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 9902751744 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚2 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚35 EDeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 [HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚63655123429376 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 431144265 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚644 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚431 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚20064 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 169280512 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 169280512 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 VCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚61249922876439 % _timestamp‚1.7884153099471676e+09 G:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚36417250 M>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 9679898624 trainer/global_step‚1099 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17170432 WFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 814163426816 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚24399 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚2605 WFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 809800655059 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚644 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚431 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 XIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 9456058368 VGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 9456058368 [MDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚ 224984064 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 242306048 [JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 809966048279 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚17039360 TEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 8980994048 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 L>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 186450944 G:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚36397186 LDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚985 ]JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚60439956828160 SDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 9456058368 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚20062 ]JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚62154230244864 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚468 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚3 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 ZGDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚62154230244864 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚465 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 REDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚15757128 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 446693376 _step‚87 C6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚36397186 QBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 9283702784 TCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 814163426816 YFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚62154213074432 FDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 9373790785 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚15757126 FDeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚17170432 \KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 848426694656 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 BDeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚36417250 REDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚20660122 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24772608 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 VCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚62154213074432 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 144203776 O@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 9902751744 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚431 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚9461258 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 VHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 165393220 _runtime‚ 891.596988143 VCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚62968562952192 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚5398571 OADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 186450944 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1003 L=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 9902751744 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚213 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚213 R?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚62968376501248 K>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚14859364 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚9461261 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚644 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 182432580 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚24126 WDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚63654873672704 QBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 9902751744 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚2 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚35 EDeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 [HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚63655123429376 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 431144265 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚644 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚431 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚20064 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 169280512 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 169280512 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 VCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚61249922876439 % _timestamp‚1.7884153099471676e+09 G:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚36417250 M>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 9679898624 trainer/global_step‚1099 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17170432 WFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 814163426816 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚24399 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚2605 WFÖÜ DeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 809800655059 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚644 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚431 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 XIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 9456058368 VGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 9456058368 [MDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚ 224984064 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 242306048 [JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 809966048279 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚17039360 TEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 8980994048 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 L>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 186450944 G:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚36397186 LDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚985 ]JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚60439956828160 SDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 9456058368 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚20062 ]JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚62154230244864 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚468 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚3 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 ZGDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚62154230244864 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚465 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 REDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚15757128 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 446693376 _step‚87 C6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚36397186 QBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 9283702784 TCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 814163426816 YFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚62154213074432 FDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 9373790785 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚15757126 FDeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚24126 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚471 @8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚644 ]LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 845344108771 JBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚213 O@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 9679898624 UBDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated‚65716953896448 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚24126 SGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚9872417 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 \IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚64867051028992 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚9872420 SGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚5632437 YHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 845178715759 _step‚90 VGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 8980994048 E=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚644 G?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚213 PCDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated‚21568576 ZKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 9456058368 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 Q>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚65716767458816 XEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚65716953896448 TGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚16443128 XEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚63924669626595 XJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 165393012 G?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚431 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚213 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 L?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚21548540 [HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚63079308478464 UGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 431144265 TADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚65716767458816 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 242306048 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 N?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 9902751744 D>DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 O@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 9373790785 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 PCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚16443128 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 TADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚63924487194223 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚468 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 446693376 L?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚16443126 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 435025408 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 230895616 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 144203776 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚17170432 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚20038 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚24399 PCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚16443126 M@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚15504854 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 186437632 _LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚64867051028992 MFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚35 C;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚644 % _timestamp‚1.7884153477087147e+09 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚24399 [HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚64867033858560 IDeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚2605 BDeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚2605 BDeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚24126 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚471 @8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚644 ]LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 845344108771 JBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚213 O@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 9679898624 UBDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated‚65716953896448 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚24126 SGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚9872417 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 \IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚64867051028992 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚9872420 SGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚5632437 YHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 845178715759 _step‚90 VGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 8980994048 E=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚644 G?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚213 PCDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated‚21568576 ZKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 9456058368 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 Q>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚65716767458816 XEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚65716953896448 TGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚16443128 XEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚63924669626595 XJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 165393012 G?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚431 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚213 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 L?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚21548540 [HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚63079308478464 UGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 431144265 TADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚65716767458816 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 242306048 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 N?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 9902751744 D>DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 O@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 9373790785 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 PCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚16443128 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 TADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚63924487194223 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚468 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 446693376 L?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚16443126 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 435025408 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 230895616 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 144203776 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚17170432 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚20038 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚24399 PCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚16443126 M@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚15504854 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 186437632 _LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚64867051028992 MFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚35 C;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚644 % _timestamp‚1.7884153477087147e+09 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚24399 [HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚64867033858560 I6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚644 L>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 186450944 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚470 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 431144265 OBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚15518410 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚644 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚213 S@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚65773042953216 QCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 313315328 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17170432 VCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚64922397548032 [JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 850628234752 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 241368064 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1003 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 182432580 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚17170432 ^KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚65603168949248 O@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 9902751744 ]JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚63133190184960 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚17039360 ]JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚64922414718464 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚473 TEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 8980994048 WIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 446693376 trainer/global_step‚1149 ZGDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚64922414718464 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚9880811 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 NADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚21587074 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚24399 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 VHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 165393220 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚5637129 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24772608 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 C6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚38024138 ODeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚15517937 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 _runtime‚ 930.033546636 R?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚65772856502272 G:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚38044202 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚16457126 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚5637599 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 M>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 9373790785 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚24399 EDeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 % _timestamp‚1.7884153483838565e+09 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚20062 PBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 435025408 K>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚38044202 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚35 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 SDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 9456058368 NADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚16457128 J;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 9679898624 QBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 9283702784 YFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚64922397548032 VGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 9456058368 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚431 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚213 REDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚21587074 NADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚21567012 VCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚63979255740647 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚644 OADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 186450944 M>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 9679898624 TEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 9283702784  ¹Ç&Ž ÁM D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 % _timestamp‚1.7884153483838565e+09 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚20062 PBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 435025408 K>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚38044202 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚35 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 SDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 9456058368 NADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚16457128 J;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 9679898624 QBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 9283702784 YFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚64922397548032 VGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 9456058368 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚431 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚213 REDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚21587074 NADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚21567012 VCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚63979255740647 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚644 OADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 186450944 M>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 9679898624 TEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 9283702784 G:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚38024138 REDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚16457128 `ODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 886589369856 B6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚644 L>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 186450944 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚470 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 431144265 OBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚15518410 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚644 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚213 S@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚65773042953216 QCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 313315328 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17170432 VCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚64922397548032 [JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 850628234752 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 241368064 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1003 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 182432580 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚17170432 ^KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚65603168949248 O@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 9902751744 ]JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚63133190184960 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚17039360 ]JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚64922414718464 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚473 TEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 8980994048 WIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 446693376 trainer/global_step‚1149 ZGDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚64922414718464 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚9880811 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 NADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚21587074 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚24399 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 VHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 165393220 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚5637129 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24772608 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 C6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚38024138 ODeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚15517937 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 _runtime‚ 930.033546636 R?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚65772856502272 G:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚38044202 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚16457126 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚5637599 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 M>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 9373790785 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚24399 EDeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚68521247459840 \IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚67635235502592 VEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 886029127680 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 UBDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated‚68521433897472 NFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚985 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 242306048 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 XJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 165393012 `MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚68343956481024 QCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 182432372 _LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚65772558874624 TADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚68521247459840 H?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚1003 BDeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚24126 IDeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 ZKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 9456058368 ]LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 886198394880 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚24772608 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 _runtime‚ 968.060176015 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 247672832 XEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚68521433897472 [HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚65772541835264 YHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 881278223167 XEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚67635218332160 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 169267200 ]JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚69267749566976 ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 222900224 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 VGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 8980994048 ]LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 881443616179 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚17170432 [HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚67635218332160 YHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 886029127680 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚10291967 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 144203776 UGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 431144265 N?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 9902751744 UFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 9456058368 O@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 9679898624 L?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚22475492 % _timestamp‚1.7884153864104881e+09 G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚2605 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 JBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚213 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 435025408 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 VGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 9283702784 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚2605 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 QBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 9902751744 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚431 trainer/global_step‚1199 TGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚17143128 M@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚16162197 O@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 9373790785 E=DeviceStatsMonitor.on_train_batch_start/segment.a æ¬ô ll.allocated‚644 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚10291970 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 XIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 9456058368 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 446693376 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 446693376 G?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚213 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚20036 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 169267200 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚20038 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚492 E8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚39618618 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚17170432 XEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚66654002490803 _step‚94 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚24399 IDeviceStatsMonitor.on_train_batch_start/allocation.all.current‚20038 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚492 E8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚39618618 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚17170432 XEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚66654002490803 _step‚94 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚24399 IDeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚68521247459840 \IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚67635235502592 VEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 886029127680 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 UBDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated‚68521433897472 NFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚985 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 242306048 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 XJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 165393012 `MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚68343956481024 QCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 182432372 _LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚65772558874624 TADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚68521247459840 H?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚1003 BDeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚24126 IDeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 ZKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 9456058368 ]LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 886198394880 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚24772608 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 _runtime‚ 968.060176015 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 247672832 XEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚68521433897472 [HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚65772541835264 YHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 881278223167 XEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚67635218332160 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 169267200 ]JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚69267749566976 ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 222900224 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 VGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 8980994048 ]LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 881443616179 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚17170432 [HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚67635218332160 YHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 886029127680 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚10291967 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 144203776 UGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 431144265 N?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 9902751744 UFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 9456058368 O@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 9679898624 L?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚22475492 % _timestamp‚1.7884153864104881e+09 G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚2605 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 JBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚213 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 435025408 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 VGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 9283702784 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚2605 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 QBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 9902751744 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚431 trainer/global_step‚1199 TGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚17143128 M@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚16162197 O@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 9373790785 E=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚644 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚10291970^šF–×0œ + _wandbruntime‚968  _runtime‚968š×UÅ& ¿M TEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 8980994048 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚35 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚20062 QBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 9283702784 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚2605 % _timestamp‚1.7884153871188724e+09 REDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚10300358 \KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 924313303040 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 J=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚22493964 L>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 186450944 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚24399 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚24126 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 M>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 9679898624 C6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚39651090 ]JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚65826423541760 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚644 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚431 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚213 WDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚69324204223488 K>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚39671154 SEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 435025408 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 S@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚68577522954240 ZGDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚67690599192064 FDeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚16175224 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚457 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17170432 BDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 XIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 9456058368 bODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚68399915693056 VCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚66708588604855 NADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚17157128 TCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 886754481664 R?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚68577336503296 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚10300361 trainer/global_step‚1199 E6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚644 [JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 882165063095 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚24399 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚2 R?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚66708406172275 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚431 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚213 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 169280512 VGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 9456058368 F6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚644 _runtime‚ 968.769061479 M>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 9373790785 O8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 WIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 446693376 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 182432580 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24772608\›ËÉ&ž ÃM J=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚22493964 L>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 186450944 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚24399 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚24126 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 M>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 9679898624 C6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚39651090 ]JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚65826423541760 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚644 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚431 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚213 WDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚69324204223488 K>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚39671154 SEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 435025408 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 S@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚68577522954240 ZGDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚67690599192064 FDeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚16175224 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚457 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17170432 BDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 XIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 9456058368 bODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚68399915693056 VCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚66708588604855 NADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚17157128 TCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 886754481664 R?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚68577336503296 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚10300361 trainer/global_step‚1199 E6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚644 [JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 882165063095 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚24399 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚2 R?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚66708406172275 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚431 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚213 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 169280512 VGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 9456058368 F6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚644 _runtime‚ 968.769061479 M>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 9373790785 O8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 WIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 446693376 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 182432580 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24772608 TEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 8980994048 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚35 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚20062 QBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 9283702784 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚2605 % _timestamp‚1.7884153871188724e+09 REDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚10300358 \KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 924313303040 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0_5[l0Ÿ + _wandbruntime‚968  _runtime‚968ÆÈóë.  Ì /loss/FBt4/detection_loss‚0.84820894pì8$380015541 epoch‚1 _runtime‚ 968.770424532 0loss/FBt2/assignment_loss‚0.2908254607658212 1loss/FBt4/assignment_loss‚0.17743628228934497 /loss/FRt4/detection_loss‚1.8985638451419378 0loss/SRqqt2/detection_loss‚2.750392343977402 1loss/SRqqt4/detection_loss‚0.9882599456316751 0loss/FRt4/assignment_loss‚2.2738850612389414 1loss/SRqqt2/assignment_loss‚2.494888002245591 1loss/FBt3/assignment_loss‚0.28300878284909975 1loss/SRqqt3/detection_loss‚2.1050978196991816 .loss/FBt1/detection_loss‚2.656340320001949 2loss/SRqqt3/assignment_loss‚1.8397142498857446 $loss/total_loss‚49.9574129744547 /loss/FRt2/assignment_loss‚4.392976681951066 /loss/FRt3/assignment_loss‚4.381443610867938 .loss/FBt3/detection_loss‚2.901786408641122 0loss/FRt1/assignment_loss‚3.9907140986485916 1loss/SRqqt1/assignment_loss‚2.061273753708392 /loss/FRt1/detection_loss‚2.0804613952609627 /loss/FBt2/detection_loss‚2.9086955785751343 _step‚96 2loss/SRqqt4/assignment_loss‚1.3860537249028035 0loss/FBt1/assignment_loss‚0.1636116407202773 -loss/FRt2/detection_loss‚2.55578227110312 1loss/SRqqt1/detection_loss‚1.8418821462413721 trainer/global_step‚1199 # _timestamp‚1.78841538712118e+09 .loss/FRt3/detection_loss‚2.686110612105679.xeÁV¡ Ð 1loss/SRqqt4/detection_loss‚0.9882599456316751 0loss/FRt4/assignment_loss‚2.2738850612389414 1loss/SRqqt2/assignment_loss‚2.494888002245591 1loss/FBt3/assignment_loss‚0.28300878284909975 1loss/SRqqt3/detection_loss‚2.1050978196991816 .loss/FBt1/detection_loss‚2.656340320001949 2loss/SRqqt3/assignment_loss‚1.8397142498857446 $loss/total_loss‚49.9574129744547 /loss/FRt2/assignment_loss‚4.392976681951066 /loss/FRt3/assignment_loss‚4.381443610867938 .loss/FBt3/detection_loss‚2.901786408641122 0loss/FRt1/assignment_loss‚3.9907140986485916 1loss/SRqqt1/assignment_loss‚2.061273753708392 /loss/FRt1/detection_loss‚2.0804613952609627 /loss/FBt2/detection_loss‚2.9086955785751343 _step‚96 2loss/SRqqt4/assignment_loss‚1.3860537249028035 0loss/FBt1/assignment_loss‚0.1636116407202773 -loss/FRt2/detection_loss‚2.55578227110312 1loss/SRqqt1/detection_loss‚1.8418821462413721 trainer/global_step‚1199 # _timestamp‚1.78841538712118e+09 .loss/FRt3/detection_loss‚2.686110612105679 /loss/FBt4/detection_loss‚0.8482089380015541 epoch‚1 _runtime‚ 968.770424532 0loss/FBt2/assignment_loss‚0.2908254607658212 1loss/FBt4/assignment_loss‚0.17743628228934497 /loss/FRt4/detection_loss‚1.8985638451419378 0loss/SRqqt2/detection_loss‚2.750392343977402`õ°4„‘¢ ‚(:† ¡›äÔÀÒ»¨ gpu.0.gpu‚39.0 gpu.0.memory‚4, gpu.0.memoryAllocated‚12.165603637695312* gpu.0.memoryAllocatedBytes‚ 10450173952 gpu.0.temp‚36.0 gpu.0.powerWatts‚134.49' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚26.898 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0Õý¦óM£ ‚(: ¡›äÔ½¬âª disk.sda2.in‚0 disk.rbd1.out‚ 1.4072265625 disk.sda.in‚0 disk.drbd2265.in‚0 disk.sda.out‚38.34375 disk.drbd1716.out‚0% disk.nvme0n1.out‚3589.27490234375' disk.nvme0c0n1.out‚3589.27490234375 disk.sda3.out‚38.34375 disk.dm-1.out‚38.34375$ memory_percent‚85.60810089111328 cpu‚26.001762270719883 disk.sda2.out‚0 disk.drbd2266.in‚0 disk.dm-1.in‚0 disk.rbd0.in‚0* disk./.usagePercent‚16.805826559442245 disk.sda1.in‚0 disk.drbd2266.out‚0 disk.drbd2265.out‚0! disk.nvme0c0n1.in‚ 11.80859375 disk.drbd1716.in‚0& proc.memory.availableMB‚ 2357.96875 network.recv‚417550 disk.rbd1.in‚0" disk.dm-0.out‚3589.39990234375 disk.rbd0.out‚0! proc.memory.rssMB‚ 2776.109375 proc.cpu.threads‚17% disk./.usageGB‚250.41896057128906 disk.sda3.in‚0 disk.drbd2080.out‚0 network.sent‚ 2.054212e+06 disk.nvme0n1.in‚ 11.80859375 disk.sda1.out‚0 disk.dm-0.in‚ 11.80859375 disk.drbd2080.in‚0* proc.memory.percent‚16.9440269470214844 #“¤ ‚(:ˆ °›äÔ§Š¼¨ gpu.0.gpu‚40.0 gpu.0.memory‚3, gpu.0.memoryAllocated‚12.165603637695312* gpu.0.memoryAllocatedBytes‚ 10450173952 gpu.0.temp‚35.0 gpu.0.powerWatts‚176.849' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚35.3698 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0Hë—ÁY¥ ‚(:Î °›äÔñÉ´« disk.drbd2266.in‚0! disk.nvme0c0n1.in‚ 11.80859375 proc.cpu.threads‚17 disk.drbd1716.out‚0 disk.sda1.in‚0) proc.memory.percent‚17.41640567779541) disk./.usagePercent‚16.80749035240727 disk.dm-1.in‚0 disk.rbd0.in‚0% disk.nvme0n1.out‚3637.55517578125 disk.sda1.out‚0 disk.drbd2265.in‚0 disk.sda.out‚ 38.86328125$ memory_percent‚84.85188484191895 disk.dm-1.out‚ 38.86328125 disk.sda2.out‚0 disk.rbd1.out‚ 1.47021484375 disk.sda.in‚0 disk.sda3.in‚0 disk.nvme0n1.in‚ 11.80859375 disk.drbd2265.out‚0 disk.rbd1.in‚0 disk.dm-0.in‚ 11.80859375" disk.dm-0.out‚3637.68017578125# proc.memory.rssMB‚ 2853.50390625 network.sent‚ 2.140451e+06 disk.drbd1716.in‚0 disk.drbd2266.out‚0 cpu‚26.097852478591243 disk.sda3.out‚ 38.86328125 disk.sda2.in‚0' disk.nvme0c0n1.out‚3637.55517578125 network.recv‚422524% disk./.usageGB‚250.44375228881836 disk.drbd2080.in‚0( proc.memory.availableMB‚ 2481.8671875 disk.drbd2080.out‚0 disk.rbd0.out‚03q¦ ‚(:’ ¿›äÔ¢óþ§ gpu.0.gpu‚2.0 gpu.0.memory‚0, gpu.0.memoryAllocated‚12.165603637695312* gpu.0.memoryAllocatedBytes‚ 10450173952 gpu.0.temp‚33.0 gpu.0.powerWatts‚118.653' gpu.0.enforcedPowerLimitWatts‚500.0) gpu.0.powerPercent‚23.730600000000003 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0c 9Z§ ‚(:Ï ¿›äÔªù”ª disk.drbd2080.in‚0 disk.rbd0.in‚0 disk.sda1.in‚0$ memory_percent‚84.50026512145996 cpu‚26.20095820606408' disk.nvme0c0n1.out‚3677.54345703125* disk./.usagePercent‚16.809023325522304 disk.drbd2080.out‚0 disk.rbd0.out‚0 disk.drbd2265.out‚0 disk.sda3.in‚0% disk./.usageGB‚250.46659469604492% disk.nvme0n1.out‚3677.54345703125 disk.sda2.in‚0 disk.rbd1.out‚ 1.47021484375! disk.nvme0c0n1.in‚ 11.80859375 disk.sda.in‚0 disk.dm-1.in‚0 disk.dm-1.out‚ 39.40234375 disk.dm-0.in‚ 11.80859375 disk.sda3.out‚ 39.40234375# proc.memory.rssMB‚ 2795.91796875 network.recv‚428076" disk.dm-0.out‚3677.66845703125* proc.memory.percent‚17.064929008483887 disk.nvme0n1.in‚ 11.80859375 disk.sda2.out‚0 disk.drbd2266.out‚0 disk.drbd2265.in‚0 disk.drbd1716.in‚0 disk.drbd2266.in‚0 network.sent‚ 2.147316e+06 disk.drbd1716.out‚0 disk.sda1.out‚0 disk.rbd1.in‚0 disk.sda.out‚ 39.40234375( proc.memory.availableMB‚ 2539.4765625 proc.cpu.threads‚17JÑJÎ2¨ - _wandbruntime‚1011  _runtime‚1011“ƒTÉMc© Š§ >&Purity/2FRt1SRqqt*FBt/event_proportion‚0.04158266261219978 ,&Purity/2FRt4SRqqt2FBt/event_proportion‚0 ("Purity/3FRt1SRqqt*FBt/event_purity‚0 7"Purity/3FRt1SRqqt*FBt/SRqqt_purity‚0.34876549243927 @&Purity/4FRt1SRqqt0FBt/event_proportion‚6.300403038039804e-05 ,&Purity/4FRt3SRqqt4FBt/event_proportion‚0 jet/accuracy_5_of_5‚0 (particle/accuracy‚0.8184067010879517 +particle/sensitivity‚0.2510537803173065 ("Purity/2FRt3SRqqt1FBt/event_purity‚0 :"Purity/*FRt1SRqqt1FBt/SRqqt_purity‚0.40868526697158813 ?&Purity/2FRt*SRqqt2FBt/event_proportion‚0.005418346729129553 ?&Purity/2FRt0SRqqt0FBt/event_proportion‚0.024571573361754417 @&Purity/4FRt*SRqqt0FBt/event_proportion‚0.0001890120911411941 7 Purity/*FRt4SRqqt0FBt/FRt_purity‚0.1666666716337204 :"Purity/0FRt3SRqqt0FBt/SRqqt_purity‚0.23849208652973175 particle/accuracy_1_of_7‚1 ("Purity/*FRt*SRqqt4FBt/SRqqt_purity‚0 ?&Purity/2FRt0SRqqt*FBt/event_proportion‚0.037046369165182114 6"Purity/2FRt1SRqqt1FBt/SRqqt_purity‚0.3913594186306 ("Purity/3FRt1SRqqt2FBt/event_purity‚0 ("Purity/4FRt2SRqqt0FBt/event_purity‚0 8"Purity/0FRt0SRqqt*FBt/event_purity‚0.818209707736969 @&Purity/3FRt0SRqqt0FBt/event_proportion‚0.0015120967291295528 7 Purity/0FRt1SRqqt3FBt/FBt_purity‚0.8333333730697632 ?&Purity/0FRt1SRqqt2FBt/event_proportion‚0.008757560513913631 <"Purity/1FRt1SRqqt2FBt/event_purity‚0.0071428571827709675 :"Purity/0FRt2SRqqt2FBt/SRqqt_purity‚0.36538460850715637 ("Purity/0FRt2SRqqt3FBt/SRqqt_purity‚1 9"Purity/*FRt1SRqqt2FBt/SRqqt_purity‚0.4530316889286041 7 Purity/0FRt0SRqqt3FBt/FBt_purity‚0.9066666960716248 & Purity/2FRt0SRqqt1FBt/FBt_purity‚1 +"Purity/0FRt*SRqqt4FBt/event_purity‚0.25 ) Purity/1FRt1SRqqt4FBt/FBt_purity‚0.75 trainer/global_step‚1207 @&Purity/1FRt3SRqqt1FBt/event_proportion‚0.0011340725468471646 ?&Purity/2FRt2SRqqt1FBt/event_proportion‚0.003969253972172737 ("Purity/0FRt3SRqqt0FBt/event_purity‚0 @&Purity/0FRt2SRqqt3FBt/event_proportion‚6.300403038039804e-05 7 Purity/2FRt0SRqqt*FBt/FBt_purity‚0.9931899905204773 7 Purity/*FRt0SRqqt4FBt/FBt_purity‚0.8999999761581421 9"Purity/1FRt1SRqqt1FBt/SRqqt_purity‚0.4204671382904053 @&Purity/1FRt1SRqqt4FBt/event_proportion‚6.300403038039804e-05 @&Purity/4FRt1SRqqt*FBt/event_proportion‚6.300403038039804e-05 @&Purity/4FRt2SRqqt0FBt/event_proportion‚6.300403038039804e-05 +jet/accuracy_2_of_4‚0.26249203085899353 :"Purity/*FRt2SRqqt0FBt/event_purity‚0.00566460145637393 ,&Purity/*FRt4SRqqt4FBt/event_proportion‚0 ,&Purity/0FRt3SRqqt4FBt/event_proportion‚0 ,&Purity/3FRt4SRqqt1FBt/event_proportion‚0 ,jet/accuracy_3_of_4‚0.034572161734104156 ("Purity/1FRt4SRqqt0FBt/event_purity‚0 ("Purity/1FRt3SRqqt2FBt/event_purity‚0 ("Purity/1FRt3SRqqt0FBt/event_purity‚0 ,&Purity/3FRt3SRqqt2FBt/event_proportion‚0 particle/accuracy_3_of_5‚1 +particle/specificity‚0.9234230518341064 & Purity/3FRt2SRqqt1FBt/FBt_purity‚1 8 Purity/*FRt2SRqqt*FBt/FRt_purity‚0.16492612659931183 >&Purity/0FRt0SRqqt1FBt/event_proportion‚0.08606351166963577 ?&Purity/1FRt1SRqqt2FBt/event_proportion‚0.006111391354352236 ("Purity/2FRt1SRqqt*FBt/event_purity‚0 :"Purity/2FRt1SRqqt*FBt/SRqqt_purity‚0.26460006833076477 particle/accuracy_2_of_6‚1 jet/accuracy_3_of_7‚0 _runtime‚1011.953445378 ,&Purity/3FRt0SRqqt3FBt/event_proportion‚0 & Purity/3FRt2SRqqt*FBt/FBt_purity‚1 jet/accuracy_6_of_6‚0 7 Purity/*FRt2SRqqt3FBt/FBt_purity‚0.6666666865348816 ("Purity/0FRt2SRqqt3FBt/event_purity‚0 *"Purity/4FRt2SRqqt0FBt/SRqqt_purity‚0.5 :"Purity/*FRt0SRqqt2FBt/event_purity‚0.25009292364120483 :"Purity/*FRt1SRqqt0FBt/SRqqt_purity‚0.23820847272872925 >&Purity/0FRt*SRqqt2FBt/event_proportion‚0.03944052383303642 ("Purity/1FRt3SRqqt*FBt/event_purity‚0 9"Purity/1FRt3SRqqt*FBt/SRqqt_purity‚0.2423536628484726 7 Purity/3FRt1SRqqt*FBt/FRt_purity‚0.1419752985239029 @&Purity/4FRt2SRqqt*FBt/event_proportion‚6.300403038039804e-05 ,"Purity/3FRt4SRqqt0FBt/SRqqt_purity‚0.125 ("Purity/*FRt3SRqqt0FBt/event_purity‚0 ?&Purity/1FRt0SRqqt1FBt/event_proportion‚0.049773186445236206 & Purity/1FRt4SRqqt0FBt/FRt_purity‚0 >&Purity/1FRt1SRqqt1FBt/event_proportion‚0.03944052383303642 jet/accuracy_5_of_6‚0 9"Purity/3FRt*SRqqt0FBt/SRqqt_purity‚0.2241213172674179 ?&Purity/3FRt1SRqqt*FBt/event_proportion‚0.003717237850651145 particle/accuracy_1_of_5‚1 7 Purity/1FRt1SRqqt2FBt/FBt_purity‚0.9556547403335571 ?&Purity/2FRt2SRqqt0FBt/event_proportion‚0.015435988083481789 @&Purity/3FRt1SRqqt1FBt/event_proportion‚0.0009450604557059705 9"Purity/*FRt2SRqqt2FBt/SRqqt_purity‚0.2850877046585083 @&Purity/*FRt4SRqqt0FBt/event_proportion‚0.0012600806076079607 @&Purity/1FRt2SRqqt2FBt/event_proportion‚0.0010080644860863686 +jet/accuracy_2_of_2‚0.07190760225057602 particle/accuracy_6_of_6‚1 7 Purity/*FRt0SRqqt2FBt/FBt_purity‚0.9832113981246948 A&Purity/0FRt1SRqqt3FBt/event_proportion‚0.00044102821266278625 + Purity/1FRt*SRqqt3FBt/FBt_purity‚0.9375 8 Purity/2FRt1SRqqt*FBt/FRt_purity‚0.17398162186145782 ("Purity/2FRt4SRqqt*FBt/event_purity‚0 :"Purity/2FRt4SRqqt*FBt/SRqqt_purity‚0.20000000298023224 ("Purity/0FRt4SRqqt0FBt/event_purity‚0 7 Purity/3FRt4SRqqt0FBt/FRt_purity‚0.1666666716337204 8 Purity/1FRt*SRqqt*FBt/FRt_purity‚0.17591306567192078 =&Purity/1FRt1SRqqt0FBt/event_proportion‚0.1265120953321457 ;"Purity/2FRt*SRqqt1FBt/event_purity‚0.010880696587264538 ("Purity/2FRt3SRqqt*FBt/event_purity‚0 9"Purity/2FRt3SRqqt*FBt/SRqqt_purity‚0.2129206508398056 particle/accuracy_3_of_7‚1 ("Purity/1FRt1SRqqt4FBt/SRqqt_purity‚0 7 Purity/3FRt3SRqqt0FBt/FRt_purity‚0.1666666567325592 :"Purity/*FRt*SRqqt1FBt/SRqqt_purity‚0.38676396012306213 >&Purity/*FRt0SRqqt1FBt/event_proportion‚0.14591734111309052 9"Purity/0FRt*SRqqt2FBt/event_purity‚0.2662772834300995 8 Purity/1FRt0SRqqt*FBt/FRt_purity‚0.16800153255462646 :"Purity/1FRt0SRqqt0FBt/event_purity‚0.09607168287038803 ,&Purity/1FRt4SRqqt4FBt/event_proportion‚0 ,&Purity/4FRt1SRqqt3FBt/event_proportion‚0 >&Purity/*FRt0SRqqt0FBt/event_proportion‚0.15757308900356293 7 Purity/1FRt0SRqqt3FBt/FBt_purity‚0.9523809552192688 ("Purity/1FRt3SRqqt1FBt/event_purity‚0 ("Purity/3FRt*SRqqt1FBt/event_purity‚0 ("Purity/3FRt1SRqqt1FBt/event_purity‚0 *jet/accuracy_1_of_3‚0.7335270643234253 ,&Purity/*FRt4SRqqt2FBt/event_proportion‚0 >&Purity/0FRt0SRqqt*FBt/event_proportion‚0.11857359111309052 ("Purity/3FRt0SRqqt0FBt/event_purity‚0 ("Purity/0FRt4SRqqt*FBt/event_purity‚0 ."Purity/0FRt4SRqqt*FBt/SRqqt_purity‚0.15625 particle/accuracy_4_of_6‚1 )loss/val_total_loss‚49.49597158565765 >&Purity/1FRt0SRqqt0FBt/event_proportion‚0.13142640888690948 9"Purity/0FRt4SRqqt0FBt/SRqqt_purity‚0.1785714328289032 ("Purity/*FRt1SRqqt4FBt/event_purity‚0 8 Purity/*FRt*SRqqt*FBt/FRt_purity‚0.17455314099788666 :"Purity/*FRt3SRqqt0FBt/SRqqt_purity‚0.22708311676979065 @&Purity/*FRt4SRqqt1FBt/event_proportion‚6.300403038039804e-05 =&Purity/1FRt*SRqqt0FBt/event_proportion‚0.3135710656642914 8 Purity/1FRt2SRqqt*FBt/FRt_purity‚0.15966638922691345 6 Purity/2FRt0SRqqt2FBt/FBt_purity‚0.988095223903656 ,jet/accuracy_4_of_4‚0.003514922922477126 6 Purity/*FRt*SRqqt3FBt/FBt_purity‚0.904789388179779 :"Purity/*FRt1SRqqt*FBt/event_purity‚0.11439140886068344 :"Purity/*FRt1SRqqt*FBt/SRqqt_purity‚0.28498902916908264 :"Purity/*FRt1SRqqt0FBt/event_purity‚0.13481950759887695 :"Purity/2FRt0SRqqt1FBt/event_purity‚0.01944444514811039 ("Purity/0FRt3SRqqt*FBt/event_purity‚0 :"Purity/0FRt3SRqqt*FBt/SRqqt_purity‚0.24469955265522003 particle/accuracy_5_of_6‚1 ("Purity/3FRt*SRqqt0FBt/event_purity‚0 particle/accuracy_1_of_1‚1 ("Purity/3FRt4SRqqt0FBt/event_purity‚0 ?&Purity/3FRt*SRqqt*FBt/event_proportion‚0.009954636916518211 particle/accuracy_2_of_7‚1 =&Purity/0FRt1SRqqt0FBt/event_proportion‚0.1694808453321457 & Purity/3FRt*SRqqt1FBt/FBt_purity‚1 ,&Purity/4FRt3SRqqt1FBt/event_proportion‚0 9"Purity/*FRt3SRqqt2FBt/SRqqt_purity‚0.6666666865348816 =&Purity/*FRt0SRqqt*FBt/event_proportion‚0.3558467626571655 9"Purity/2FRt*SRqqt2FBt/SRqqt_purity‚0.6359649300575256 & Purity/3FRt*SRqqt*FBt/FBt_purity‚1 @&Purity/3FRt*SRqqt1FBt/event_proportion‚0.0024571572430431843 ,&Purity/4FRt2SRqqt1FBt/event_proportion‚0 7 Purity/*FRt4SRqqt*FBt/FRt_purity‚0.1666666716337204 ,&Purity/0FRt4SRqqt2FBt/event_proportion‚0 ,&Purity/3FRt4SRqqt3FBt/event_proportion‚0 *jet/accuracy_1_of_2‚0.6064389944076538 <"Purity/2FRt0SRqqt0FBt/event_purity‚0.0026881720405071974 & Purity/2FRt2SRqqt*FBt/FBt_purity‚1 7validation_average_jet_accuracy‚0.33515453338623047 *"Purity/3FRt2SRqqt1FBt/SRqqt_purity‚0.5 particle/accuracy_5_of_7‚1 +"Purity/3FRt*SRqqt2FBt/SRqqt_purity‚0.25 ("Purity/4FRt0SRqqt*FBt/event_purity‚0 7 Purity/*FRt1SRqqt*FBt/FBt_purity‚0.9809271693229675 7 Purity/*FRt2SRqqt1FBt/FRt_purity‚0.1919959932565689 & Purity/2FRt*SRqqt1FBt/FBt_purity‚1 ;"Purity/1FRt1SRqqt0FBt/event_purity‚0.006698790472000837 A&Purity/1FRt1SRqqt3FBt/event_proportion‚0.00025201612152159214 & Purity/2FRt2SRqqt1FBt/FBt_purity‚1 ,&Purity/2FRt4SRqqt3FBt/event_proportion‚0 ,&Purity/3FRt*SRqqt4FBt/event_proportion‚0 8 Purity/3FRt0SRqqt0FBt/FRt_purity‚0.10457516461610794 7 Purity/3FRt0SRqqt2FBt/FRt_purity‚0.3333333432674408 & Purity/*FRt3SRqqt2FBt/FBt_purity‚1 7 Purity/*FRt1SRqqt1FBt/FBt_purity‚0.9916524291038513 @&Purity/1FRt0SRqqt3FBt/event_proportion‚0.0010710685746744275 7 Purity/2FRt*SRqqt*FBt/FBt_purity‚0.9940999746322632 ,&Purity/2FRt0SRqqt4FBt/event_proportion‚0 @&Purity/3FRt*SRqqt2FBt/event_proportion‚0.0003150201519019902 ,&Purity/4FRt2SRqqt4FBt/event_proportion‚0 +jet/accuracy_1_of_1‚0.35932639241218567 -"Purity/*FRt4SRqqt0FBt/SRqqt_purity‚0.1875 =&Purity/*FRt*SRqqt0FBt/event_proportion‚0.6160534024238586 8 Purity/*FRt2SRqqt0FBt/FRt_purity‚0.15943489968776703 ,&Purity/0FRt1SRqqt4FBt/event_proportion‚0 ,&Purity/1FRt4SRqqt1FBt/event_proportion‚0 <"Purity/2FRt*SRqqt*FBt/event_purity‚0.0037686729338020086 :"Purity/2FRt*SRqqt*FBt/SRqqt_purity‚0.24550746381282806 @&Purity/3FRt2SRqqt0FBt/event_proportion‚0.0024571572430431843 8 Purity/*FRt2SRqqt2FBt/FRt_purity‚0.24358974397182465 ,&Purity/1FRt3SRqqt3FBt/event_proportion‚0 ,&Purity/4FRt3SRqqt0FBt/event_proportion‚0 +jet/accuracy_2_of_3‚0.15870389342308044 ("Purity/0FRt4SRqqt1FBt/event_purity‚0 7 Purity/0FRt2SRqqt2FBt/FBt_purity‚0.9615384340286255 =&Purity/*FRt1SRqqt1FBt/event_proportion‚0.1001134067773819 :"Purity/0FRt1SRqqt2FBt/SRqqt_purity‚0.37794971466064453 :"Purity/0FRt2SRqqt1FBt/event_purity‚0.04121863842010498 :"Purity/1FRt0SRqqt1FBt/event_purity‚0.14109766483306885 ,&Purity/4FRt2SRqqt2FBt/event_proportion‚0 ("Purity/*FRt4SRqqt0FBt/event_purity‚0 ("Purity/*FRt4SRqqt1FBt/SRqqt_purity‚0 ("Purity/0FRt3SRqqt1FBt/event_purity‚0 :"Purity/1FRt1SRqqt0FBt/SRqqt_purity‚0.21862050890922546 8 Purity/3FRt0SRqqt1FBt/FRt_purity‚0.33974358439445496 ,&Purity/4FRt1SRqqt4FBt/event_proportion‚0 ,&Purity/4FRt4SRqqt2FBt/event_proportion‚0 particle/accuracy_4_of_5‚1 +"Purity/4FRt*SRqqt0FBt/SRqqt_purity‚0.25 *"Purity/2FRt2SRqqt2FBt/SRqqt_purity‚0.5 ("Purity/1FRt*SRqqt4FBt/event_purity‚0 @&Purity/*FRt4SRqqt*FBt/event_proportion‚0.0013230846961960196 :"Purity/0FRt*SRqqt1FBt/SRqqt_purity‚0.37620046734809875 7 Purity/0FRt*SRqqt3FBt/FBt_purity‚0.8847736716270447 :"Purity/0FRt0SRqqt3FBt/event_purity‚0.07333333790302277 & Purity/1FRt3SRqqt1FBt/FBt_purity‚1 @&Purity/3FRt2SRqqt1FBt/event_proportion‚0.0003780241822823882 ("Purity/4FRt*SRqqt0FBt/event_purity‚0 & Purity/4FRt1SRqqt*FBt/FRt_purity‚0 ,&Purity/0FRt4SRqqt3FBt/event_proportion‚0 @&Purity/1FRt4SRqqt0FBt/event_proportion‚0.0003780241822823882 *jet/accuracy_2_of_5‚0.3339018225669861 epoch‚1 & Purity/1FRt3SRqqt*FBt/FBt_purity‚1 ,&Purity/4FRt3SRqqt*FBt/event_proportion‚0 ) Purity/*FRt1SRqqt4FBt/FBt_purity‚0.75 =&Purity/*FRt1SRqqt*FBt/event_proportion‚0.4482736885547638 ?&Purity/*FRt2SRqqt2FBt/event_proportion‚0.002079133177176118 @&Purity/1FRt4SRqqt*FBt/event_proportion‚0.0003780241822823882 @&Purity/4FRt*SRqqt*FBt/event_proportion‚0.0001890120911411941 & Purity/3FRt1SRqqt2FBt/FBt_purity‚1 :"Purity/*FRt0SRqqt*FBt/event_purity‚0.33299311995506287 ?&Purity/*FRt0SRqqt3FBt/event_proportion‚0.003717237850651145 ("Purity/1FRt0SRqqt4FBt/event_purity‚0 <"Purity/*FRt2SRqqt*FBt/event_purity‚0.0072134328074753284 9"Purity/*FRt2SRqqt*FBt/SRqqt_purity‚0.2629992365837097 9"Purity/0FRt*SRqqt*FBt/event_purity‚0.3600033223628998 9"Purity/0FRt*SRqqt*FBt/SRqqt_purity‚0.2823258340358734 :"Purity/1FRt*SRqqt1FBt/SRqqt_purity‚0.40322521328926086 8 Purity/1FRt3SRqqt1FBt/FRt_purity‚0.42222222685813904 ,&Purity/2FRt2SRqqt4FBt/event_proportion‚0 8 Purity/3FRt*SRqqt0FBt/FRt_purity‚0.11875661462545395 ?&Purity/0FRt3SRqqt*FBt/event_proportion‚0.006741431541740894 ?&Purity/2FRt0SRqqt1FBt/event_proportion‚0.008946572430431843 @&Purity/2FRt3SRqqt1FBt/event_proportion‚0.0003780241822823882 ,&Purity/4FRt1SRqqt2FBt/event_proportion‚0 ,&Purity/4FRt4SRqqt3FBt/event_proportion‚0 particle/accuracy_4_of_4‚1 7 Purity/2FRt4SRqqt*FBt/FRt_purity‚0.4000000059604645 8 Purity/3FRt*SRqqt2FBt/FRt_purity‚0.13333334028720856 >&Purity/*FRt2SRqqt0FBt/event_proportion‚0.10855595022439957 7 Purity/0FRt1SRqqt*FBt/FBt_purity‚0.9794159531593323 8 Purity/1FRt1SRqqt*FBt/FRt_purity‚0.18765881657600403 ;"Purity/2FRt0SRqqt*FBt/event_purity‚0.008695573545992374 8 Purity/2FRt2SRqqt0FBt/FRt_purity‚0.16001896560192108 & Purity/3FRt0SRqqt1FBt/FBt_purity‚1 ,&Purity/4FRt0SRqqt2FBt/event_proportion‚0 ("Purity/3FRt0SRqqt2FBt/event_purity‚0 8 Purity/*FRt1SRqqt0FBt/FRt_purity‚0.15339669585227966 8 Purity/*FRt1SRqqt2FBt/FRt_purity‚0.20927010476589203 :"Purity/0FRt1SRqqt1FBt/SRqqt_purity‚0.39746198058128357 9"Purity/1FRt*SRqqt2FBt/SRqqt_purity‚0.4495134949684143 ("Purity/3FRt2SRqqt*FBt/event_purity‚0 9"Purity/3FRt2SRqqt*FBt/SRqqt_purity‚0.3020833432674408 ,&Purity/4FRt*SRqqt2FBt/event_proportion‚0 * Purity/0FRt0SRqqt4FBt/FBt_purity‚0.875 >&Purity/*FRt*SRqqt1FBt/event_proportion‚0.27079132199287415 :"Purity/*FRt*SRqqt2FBt/event_purity‚0.18058794736862183 8 Purity/*FRt0SRqqt*FBt/FRt_purity‚0.16882309317588806 6 Purity/0FRt1SRqqt1FBt/FBt_purity‚0.987925112247467 7 Purity/2FRt1SRqqt2FBt/FBt_purity‚0.9868420958518982 & Purity/3FRt0SRqqt*FBt/FBt_purity‚1 ,&Purity/4FRt*SRqqt3FBt/event_proportion‚0 9"Purity/1FRt2SRqqt2FBt/SRqqt_purity‚0.2083333283662796 6 Purity/*FRt0SRqqt3FBt/FBt_purity‚0.915277898311615 7 Purity/1FRt*SRqqt2FBt/FBt_purity‚0.9738635420799255 ,&Purity/1FRt2SRqqt4FBt/event_proportion‚0 8 Purity/2FRt*SRqqt0FBt/FRt_purity‚0.13177835941314697 7 Purity/2FRt*SRqqt2FBt/FRt_purity‚0.3083333373069763 9"Purity/2FRt2SRqqt1FBt/SRqqt_purity‚0.3366071581840515 & Purity/*FRt1SRqqt4FBt/FRt_purity‚0 ("Purity/1FRt1SRqqt3FBt/event_purity‚0 8 Purity/*FRt1SRqqt1FBt/FRt_purity‚0.26268014311790466 7 Purity/*FRt2SRqqt*FBt/FBt_purity‚0.9855287075042725 @&Purity/0FRt2SRqqt2FBt/event_proportion‚0.0010080644860863686 7 Purity/1FRt0SRqqt2FBt/FBt_purity‚0.9810459017753601 @&Purity/2FRt2SRqqt2FBt/event_proportion‚6.300403038039804e-05 +jet/accuracy_3_of_5‚0.06469175964593887 7 Purity/1FRt1SRqqt3FBt/FBt_purity‚0.8333333730697632 ?&Purity/2FRt0SRqqt2FBt/event_proportion‚0.003465221729129553 7 Purity/2FRt2SRqqt1FBt/FRt_purity‚0.2440476268529892 7 Purity/3FRt2SRqqt*FBt/FRt_purity‚0.1597222089767456 ) Purity/*FRt1SRqqt3FBt/FRt_purity‚0.25 7 Purity/*FRt2SRqqt1FBt/FBt_purity‚0.9858918190002441 ("Purity/*FRt3SRqqt1FBt/event_purity‚0 ("Purity/0FRt0SRqqt1FBt/event_purity‚1 @&Purity/0FRt4SRqqt1FBt/event_proportion‚6.300403038039804e-05 7 Purity/2FRt*SRqqt1FBt/FRt_purity‚0.2925833761692047 (particle/f_score‚0.30156266689300537 ( Purity/2FRt2SRqqt2FBt/FRt_purity‚0.5 ?&Purity/*FRt1SRqqt2FBt/event_proportion‚0.017011089250445366 =&Purity/1FRt1SRqqt*FBt/event_proportion‚0.1723790317773819 ("Purity/3FRt0SRqqt*FBt/event_purity‚0 ,&Purity/3FRt2SRqqt2FBt/event_proportion‚0 ("Purity/*FRt4SRqqt*FBt/event_purity‚0 -"Purity/*FRt4SRqqt*FBt/SRqqt_purity‚0.1875 & Purity/2FRt*SRqqt3FBt/FRt_purity‚0 7 Purity/*FRt*SRqqt2FBt/FBt_purity‚0.9779426455497742 ,&Purity/*FRt2SRqqt4FBt/event_proportion‚0 >&Purity/1FRt*SRqqt1FBt/event_proportion‚0.09929435700178146 A&Purity/1FRt*SRqqt4FBt/event_proportion‚0.00012600806076079607 7 Purity/2FRt0SRqqt*FBt/FRt_purity‚0.1668718308210373 & Purity/1FRt2SRqqt2FBt/FBt_purity‚1 *"Purity/1FRt*SRqqt3FBt/SRqqt_purity‚0.5 % _timestamp‚1.7884154303018427e+09 & Purity/0FRt0SRqqt1FBt/FBt_purity‚1 ("Purity/1FRt2SRqqt1FBt/event_purity‚0 ,&Purity/2FRt4SRqqt4FBt/event_proportion‚0 @&Purity/3FRt2SRqqt*FBt/event_proportion‚0.0028351815417408943 ,&Purity/4FRt2SRqqt3FBt/event_proportion‚0 & Purity/2FRt0SRqqt3FBt/FBt_purity‚1 8"Purity/0FRt1SRqqt3FBt/SRqqt_purity‚0.800000011920929 ;"Purity/*FRt2SRqqt1FBt/event_purity‚0.016856711357831955 ("Purity/2FRt2SRqqt*FBt/event_purity‚0 :"Purity/2FRt2SRqqt*FBt/SRqqt_purity‚0.23546189069747925 ,&Purity/4FRt*SRqqt1FBt/event_proportion‚0 7 Purity/0FRt0SRqqt*FBt/FBt_purity‚0.9879888296127319 8 Purity/1FRt3SRqqt0FBt/FRt_purity‚0.18740400671958923 ,&Purity/2FRt4SRqqt1FBt/event_proportion‚0 8 Purity/3FRt*SRqqt1FBt/FRt_purity‚0.23106060922145844 particle/accuracy_1_of_3‚1 +jet/accuracy_3_of_6‚0.12280701100826263 9"Purity/*FRt*SRqqt3FBt/SRqqt_purity‚0.7142857313156128 & Purity/1FRt3SRqqt2FBt/FRt_purity‚0 ,&Purity/*FRt3SRqqt4FBt/event_proportion‚0 @&Purity/0FRt*SRqqt3FBt/event_proportion‚0.0030871976632624865 9"Purity/0FRt1SRqqt0FBt/SRqqt_purity‚0.2576642334461212 8 Purity/2FRt3SRqqt*FBt/FRt_purity‚0.20514285564422607 particle/accuracy_2_of_4‚1 jet/accuracy_4_of_6‚0 ("Purity/*FRt2SRqqt3FBt/SRqqt_purity‚1 & Purity/4FRt2SRqqt0FBt/FRt_purity‚0 >&Purity/1FRt2SRqqt0FBt/event_proportion‚0.04731602966785431 :"Purity/2FRt1SRqqt0FBt/SRqqt_purity‚0.20714537799358368 9"Purity/1FRt3SRqqt2FBt/SRqqt_purity‚0.6666666865348816 ,&Purity/2FRt1SRqqt4FBt/event_proportion‚0 ,&Purity/3FRt1SRqqt3FBt/event_proportion‚0 9"Purity/*FRt*SRqqt4FBt/event_purity‚0.1666666716337204 ("Purity/3FRt3SRqqt0FBt/event_purity‚0 _step‚97 ?&Purity/1FRt*SRqqt2FBt/event_proportion‚0.022303426638245583 ;"Purity/1FRt1SRqqt1FBt/event_purity‚0.013380973599851131 *jet/accuracy_1_of_5‚0.8874148726463318 *"Purity/1FRt1SRqqt3FBt/SRqqt_purity‚0.5 & Purity/4FRt*SRqqt0FBt/FRt_purity‚0 ;"Purity/*FRt1SRqqt2FBt/event_purity‚0.009677419438958168 ?&Purity/0FRt0SRqqt3FBt/event_proportion‚0.002583165420219302 ?&Purity/2FRt1SRqqt2FBt/event_proportion‚0.001890120911411941 @&Purity/3FRt3SRqqt*FBt/event_proportion‚0.0005670362734235823 @&Purity/4FRt0SRqqt0FBt/event_proportion‚6.300403038039804e-05 8 Purity/*FRt3SRqqt*FBt/FRt_purity‚0.20521655678749084 ,&Purity/4FRt0SRqqt1FBt/event_proportion‚0 :"Purity/*FRt0SRqqt4FBt/event_purity‚0.20000000298023224 :"Purity/0FRt2SRqqt0FBt/event_purity‚0.01410691812634468 ,&Purity/1FRt2SRqqt3FBt/event_proportion‚0 ,&Purity/2FRt2SRqqt3FBt/event_proportion‚0 ,&Purity/3FRt2SRqqt4FBt/event_proportion‚0 ("Purity/4FRt*SRqqt*FBt/event_purity‚0 +"Purity/4FRt*SRqqt*FBt/SRqqt_purity‚0.25 >&Purity/0FRt1SRqqt1FBt/event_proportion‚0.05185231938958168 ,&Purity/2FRt3SRqqt3FBt/event_proportion‚0 A&Purity/3FRt1SRqqt2FBt/event_proportion‚0.00025201612152159214 7 Purity/0FRt2SRqqt3FBt/FBt_purity‚0.6666666865348816 9"Purity/0FRt2SRqqt0FBt/SRqqt_purity‚0.2546229660511017 9"Purity/1FRt1SRqqt2FBt/SRqqt_purity‚0.4793367087841034 ("Purity/0FRt2SRqqt2FBt/event_purity‚0 >&Purity/0FRt*SRqqt1FBt/event_proportion‚0.14787046611309052 :"Purity/2FRt3SRqqt1FBt/SRqqt_purity‚0.20000000298023224 :"Purity/*FRt1SRqqt1FBt/event_purity‚0.06666264683008194 ?&Purity/*FRt3SRqqt1FBt/event_proportion‚0.002142137149348855 7 Purity/0FRt*SRqqt2FBt/FBt_purity‚0.9786064624786377 >&Purity/1FRt0SRqqt*FBt/event_proportion‚0.19745463132858276 ("Purity/2FRt1SRqqt0FBt/event_purity‚0 ,&Purity/2FRt1SRqqt3FBt/event_proportion‚0 ,&Purity/3FRt3SRqqt3FBt/event_proportion‚0 :"Purity/*FRt*SRqqt3FBt/event_purity‚0.05287356302142143 9"Purity/0FRt1SRqqt0FBt/event_purity‚0.2576642334461212 ,&Purity/1FRt4SRqqt3FBt/event_proportion‚0 ("Purity/2FRt1SRqqt1FBt/event_purity‚0 ("Purity/2FRt1SRqqt2FBt/event_purity‚0 ("Purity/*FRt2SRqqt3FBt/event_purity‚0 :"Purity/*FRt0SRqqt3FBt/event_purity‚0.05476190522313118 ,&Purity/0FRt3SRqqt2FBt/event_proportion‚0 ("Purity/1FRt*SRqqt3FBt/event_purity‚0 jet/accuracy_1_of_7‚0.5 ("Purity/*FRt3SRqqt*FBt/event_purity‚0 :"Purity/*FRt3SRqqt*FBt/SRqqt_purity‚0.23684664070606232 particle/accuracy_3_of_6‚1 +"Purity/3FRt1SRqqt2FBt/SRqqt_purity‚0.25 ("Purity/0FRt1SRqqt3FBt/event_purity‚0 8"Purity/3FRt3SRqqt0FBt/SRqqt_purity‚0.261904776096344 & Purity/4FRt0SRqqt0FBt/FRt_purity‚0 :"Purity/*FRt*SRqqt1FBt/event_purity‚0.37020379304885864 7 Purity/*FRt0SRqqt0FBt/FRt_purity‚0.0945061668753624 8 Purity/*FRt0SRqqt2FBt/FRt_purity‚0.39325767755508423 ?&Purity/2FRt3SRqqt*FBt/event_proportion‚0.004032257944345474 * Purity/0FRt*SRqqt4FBt/FBt_purity‚0.875 & Purity/0FRt4SRqqt*FBt/FBt_purity‚1 & Purity/2FRt*SRqqt3FBt/FBt_purity‚1 & Purity/1FRt1SRqqt4FBt/FRt_purity‚0 ?&Purity/0FRt3SRqqt0FBt/event_proportion‚0.006111391354352236 ("Purity/1FRt2SRqqt*FBt/event_purity‚0 :"Purity/1FRt2SRqqt*FBt/SRqqt_purity‚0.26454105973243713 :"Purity/1FRt3SRqqt1FBt/SRqqt_purity‚0.24814817309379578 8 Purity/2FRt*SRqqt*FBt/FRt_purity‚0.17533613741397858 9"Purity/3FRt*SRqqt1FBt/SRqqt_purity‚0.5444444417953491 *"Purity/3FRt1SRqqt1FBt/SRqqt_purity‚0.5 *jet/accuracy_1_of_4‚0.7918799519538879 ;"Purity/1FRt1SRqqt*FBt/event_purity‚0.008347955532371998 8"Purity/1FRt1SRqqt*FBt/SRqqt_purity‚0.276454895734787 8 Purity/1FRt1SRqqt0FBt/FRt_purity‚0.16414634883403778 8 Purity/1FRt1SRqqt2FBt/FRt_purity‚0.23937073349952698 8 Purity/2FRt2SRqqt*FBt/FRt_purity‚0.17266826331615448 9"Purity/2FRt3SRqqt0FBt/SRqqt_purity‚0.2149711698293686 & Purity/0FRt4SRqqt1FBt/FBt_purity‚1 7 Purity/3FRt2SRqqt1FBt/FRt_purity‚0.0555555559694767 8 Purity/*FRt1SRqqt*FBt/FRt_purity‚0.18116876482963562 ,&Purity/0FRt2SRqqt4FBt/event_proportion‚0 & Purity/*FRt0SRqqt4FBt/FRt_purity‚0 7 Purity/*FRt1SRqqt3FBt/FBt_purity‚0.8730158805847168 ("Purity/4FRt1SRqqt*FBt/event_purity‚0 ("Purity/4FRt1SRqqt*FBt/SRqqt_purity‚0 7 Purity/3FRt*SRqqt*FBt/FRt_purity‚0.1525217443704605 & Purity/*FRt4SRqqt*FBt/FBt_purity‚1 7 Purity/2FRt4SRqqt0FBt/FRt_purity‚0.4000000059604645 @&Purity/*FRt1SRqqt3FBt/event_proportion‚0.0006930443341843784 :"Purity/0FRt*SRqqt2FBt/SRqqt_purity‚0.36314815282821655 ?&Purity/0FRt2SRqqt0FBt/event_proportion‚0.043283771723508835 ,&Purity/4FRt4SRqqt*FBt/event_proportion‚0 & Purity/*FRt4SRqqt1FBt/FBt_purity‚1 & Purity/2FRt0SRqqt3FBt/FRt_purity‚0 ("Purity/1FRt1SRqqt4FBt/event_purity‚0 8"Purity/*FRt0SRqqt1FBt/event_purity‚0.640150785446167 >&Purity/*FRt1SRqqt0FBt/event_proportion‚0.33039313554763794 >&Purity/0FRt2SRqqt*FBt/event_proportion‚0.05361643061041832 8 Purity/2FRt0SRqqt1FBt/FRt_purity‚0.30209001898765564 ,&Purity/3FRt3SRqqt1FBt/event_proportion‚0 7 Purity/0FRt*SRqqt1FBt/FBt_purity‚0.9946838617324829 * Purity/0FRt3SRqqt*FBt/FBt_purity‚0.875 ("Purity/2FRt*SRqqt3FBt/event_purity‚0 :"Purity/1FRt0SRqqt*FBt/event_purity‚0.10721910744905472 >&Purity/1FRt2SRqqt*FBt/event_proportion‚0.05727066472172737 8 Purity/1FRt3SRqqt*FBt/FRt_purity‚0.18877367675304413 ?&Purity/2FRt3SRqqt0FBt/event_proportion‚0.003654233878478408 ,&Purity/3FRt*SRqqt3FBt/event_proportion‚0 ("Purity/*FRt2SRqqt2FBt/event_purity‚0 9"Purity/*FRt1SRqqt3FBt/SRqqt_purity‚0.7142857313156128 >&Purity/*FRt2SRqqt1FBt/event_proportion‚0.02255544438958168 >&Purity/*FRt3SRqqt0FBt/event_proportion‚0.01827116869390011 7 Purity/0FRt*SRqqt*FBt/FBt_purity‚0.9847093224525452 ,&Purity/0FRt3SRqqt3FBt/event_proportion‚0 7 Purity/2FRt0SRqqt0FBt/FRt_purity‚0.0897231251001358 8 Purity/2FRt0SRqqt2FBt/FRt_purity‚0.36249998211860657 * Purity/0FRt3SRqqt1FBt/FBt_purity‚0.875 ("Purity/*FRt1SRqqt4FBt/SRqqt_purity‚0 =&Purity/*FRt0SRqqt2FBt/event_proportion‚0.0483240932226181 >&Purity/0FRt0SRqqt0FBt/event_proportion‚0.04082661122083664 ,&Purity/3FRt4SRqqt4FBt/event_proportion‚0 7 Purity/*FRt0SRqqt3FBt/FRt_purity‚0.1666666716337204 :"Purity/1FRt*SRqqt*FBt/event_purity‚0.05186711996793747 :"Purity/1FRt*SRqqt*FBt/SRqqt_purity‚0.26915037631988525 8 Purity/*FRt0SRqqt1FBt/FRt_purity‚0.29427090287208557 ,&Purity/*FRt3SRqqt3FBt/event_proportion‚0 jet/accuracy_5_of_7‚0 ("Purity/4FRt2SRqqt*FBt/event_purity‚0 *"Purity/4FRt2SRqqt*FBt/SRqqt_purity‚0.5 particle/accuracy_7_of_7‚1 ) Purity/1FRt1SRqqt3FBt/FRt_purity‚0.25 @&Puri0wÇD0ty/0FRt4SRqqt*FBt/event_proportion‚0.0005040322430431843 ,&Purity/0FRt4SRqqt4FBt/event_proportion‚0 7 Purity/1FRt1SRqqt1FBt/FRt_purity‚0.2546731233596802 9"Purity/2FRt*SRqqt1FBt/SRqqt_purity‚0.3635369837284088 ("Purity/2FRt3SRqqt0FBt/event_purity‚0 8 Purity/3FRt2SRqqt0FBt/FRt_purity‚0.17874395847320557 A&Purity/3FRt4SRqqt0FBt/event_proportion‚0.00012600806076079607 jet/accuracy_6_of_7‚0 >&Purity/*FRt*SRqqt2FBt/event_proportion‚0.06747731566429138 8 Purity/*FRt3SRqqt0FBt/FRt_purity‚0.22105243802070618 7 Purity/2FRt1SRqqt*FBt/FBt_purity‚0.9964157938957214 particle/accuracy_2_of_5‚1 * Purity/*FRt*SRqqt4FBt/FBt_purity‚0.875 & Purity/3FRt0SRqqt2FBt/FBt_purity‚1 & Purity/*FRt3SRqqt2FBt/FRt_purity‚0 ("Purity/4FRt1SRqqt0FBt/event_purity‚0 @&Purity/*FRt*SRqqt4FBt/event_proportion‚0.0003780241822823882 7 Purity/1FRt*SRqqt*FBt/FBt_purity‚0.9855114817619324 ?&Purity/1FRt2SRqqt1FBt/event_proportion‚0.008946572430431843 ?&Purity/2FRt2SRqqt*FBt/event_proportion‚0.019468246027827263 ("Purity/0FRt4SRqqt1FBt/SRqqt_purity‚0 7 Purity/0FRt1SRqqt2FBt/FBt_purity‚0.9622685313224792 9"Purity/0FRt2SRqqt1FBt/SRqqt_purity‚0.3607654571533203 & Purity/2FRt1SRqqt1FBt/FBt_purity‚1 :"Purity/2FRt2SRqqt0FBt/SRqqt_purity‚0.21066135168075562 ("Purity/*FRt4SRqqt1FBt/event_purity‚0 jet/accuracy_7_of_7‚0 9"Purity/0FRt3SRqqt1FBt/SRqqt_purity‚0.3541666567325592 & Purity/4FRt*SRqqt*FBt/FRt_purity‚0 A&Purity/0FRt4SRqqt0FBt/event_proportion‚0.00044102821266278625 7 Purity/1FRt*SRqqt1FBt/FBt_purity‚0.9935746788978577 @&Purity/1FRt3SRqqt2FBt/event_proportion‚6.300403038039804e-05 ("Purity/3FRt1SRqqt0FBt/event_purity‚0 :"Purity/3FRt2SRqqt0FBt/SRqqt_purity‚0.26449277997016907 ,&Purity/3FRt2SRqqt3FBt/event_proportion‚0 particle/accuracy_2_of_3‚1 ("Purity/4FRt0SRqqt0FBt/event_purity‚0 ?&Purity/*FRt3SRqqt*FBt/event_proportion‚0.020476309582591057 ,&Purity/2FRt3SRqqt4FBt/event_proportion‚0 & Purity/3FRt1SRqqt1FBt/FBt_purity‚1 & Purity/1FRt0SRqqt4FBt/FBt_purity‚1 :"Purity/0FRt*SRqqt0FBt/event_purity‚0.20215217769145966 8"Purity/0FRt*SRqqt1FBt/event_purity‚0.626227080821991 @&Purity/1FRt*SRqqt3FBt/event_proportion‚0.0013230846961960196 ?&Purity/1FRt0SRqqt2FBt/event_proportion‚0.015120967291295528 @&Purity/2FRt4SRqqt0FBt/event_proportion‚0.0003150201519019902 *jet/accuracy_2_of_6‚0.5438596606254578 9"Purity/1FRt2SRqqt0FBt/SRqqt_purity‚0.2355533242225647 :"Purity/2FRt0SRqqt2FBt/event_purity‚0.02500000037252903 8 Purity/2FRt3SRqqt0FBt/FRt_purity‚0.22867964208126068 & Purity/3FRt1SRqqt*FBt/FBt_purity‚1 ,&Purity/3FRt3SRqqt4FBt/event_proportion‚0 ("Purity/2FRt2SRqqt2FBt/event_purity‚0 & Purity/4FRt2SRqqt*FBt/FRt_purity‚0 >&Purity/0FRt*SRqqt*FBt/event_proportion‚0.40996724367141724 ;"Purity/0FRt1SRqqt2FBt/event_purity‚0.011111111380159855 @&Purity/0FRt3SRqqt1FBt/event_proportion‚0.0006300403038039804 :"Purity/*FRt*SRqqt*FBt/event_purity‚0.17793890833854675 :"Purity/*FRt*SRqqt*FBt/SRqqt_purity‚0.27279314398765564 7 Purity/*FRt*SRqqt1FBt/FBt_purity‚0.9948082566261292 9"Purity/0FRt*SRqqt0FBt/SRqqt_purity‚0.2539779543876648 7 Purity/1FRt2SRqqt1FBt/FBt_purity‚0.9760368466377258 +"Purity/0FRt0SRqqt4FBt/event_purity‚0.25 6 Purity/0FRt0SRqqt2FBt/FBt_purity‚0.982708215713501 ("Purity/3FRt*SRqqt2FBt/event_purity‚0 & Purity/4FRt0SRqqt*FBt/FRt_purity‚0 6 Purity/*FRt*SRqqt*FBt/FBt_purity‚0.985958993434906 7 Purity/1FRt2SRqqt*FBt/FBt_purity‚0.9813225269317627 ?&Purity/2FRt*SRqqt1FBt/event_proportion‚0.021169355139136314 particle/accuracy_5_of_5‚1 particle/accuracy_4_of_7‚1 ,&Purity/*FRt4SRqqt3FBt/event_proportion‚0 ("Purity/1FRt2SRqqt0FBt/event_purity‚0 ,jet/accuracy_3_of_3‚0.012774068862199783 8 Purity/2FRt3SRqqt1FBt/FRt_purity‚0.10000000149011612 jet/accuracy_4_of_7‚0 * Purity/1FRt*SRqqt4FBt/FBt_purity‚0.875 ("Purity/*FRt1SRqqt3FBt/event_purity‚0 :"Purity/*FRt0SRqqt0FBt/event_purity‚0.08057259768247604 7 Purity/1FRt0SRqqt1FBt/FBt_purity‚0.9973816871643066 ("Purity/3FRt*SRqqt*FBt/event_purity‚0 9"Purity/3FRt*SRqqt*FBt/SRqqt_purity‚0.2571975886821747 A&Purity/3FRt4SRqqt*FBt/event_proportion‚0.00012600806076079607 ,jet/accuracy_4_of_5‚0.007526882458478212 particle/accuracy_1_of_2‚1 ("Purity/3FRt2SRqqt1FBt/event_purity‚0 8 Purity/*FRt3SRqqt1FBt/FRt_purity‚0.36666667461395264 >&Purity/2FRt1SRqqt0FBt/event_proportion‚0.03181703761219978 ,&Purity/4FRt4SRqqt4FBt/event_proportion‚0 particle/accuracy_3_of_4‚1 )jet/accuracy_1_of_6‚0.956140398979187 ("Purity/1FRt*SRqqt4FBt/SRqqt_purity‚0 ("Purity/*FRt3SRqqt2FBt/event_purity‚0 ("Purity/4FRt1SRqqt0FBt/SRqqt_purity‚0 A&Purity/0FRt*SRqqt4FBt/event_proportion‚0.00025201612152159214 6 Purity/1FRt0SRqqt*FBt/FBt_purity‚0.990094780921936 @&Purity/3FRt*SRqqt0FBt/event_proportion‚0.0071824598126113415 9"Purity/3FRt1SRqqt0FBt/SRqqt_purity‚0.3499999940395355 ("Purity/3FRt2SRqqt0FBt/event_purity‚0 7 Purity/*FRt2SRqqt2FBt/FBt_purity‚0.9868420958518982 ?&Purity/*FRt*SRqqt3FBt/event_proportion‚0.004473286215215921 ;"Purity/2FRt*SRqqt2FBt/event_purity‚0.023333333432674408 ("Purity/2FRt2SRqqt0FBt/event_purity‚0 =&Purity/*FRt*SRqqt*FBt/event_proportion‚0.9591733813285828 7 Purity/*FRt3SRqqt*FBt/FBt_purity‚0.9772727489471436 ("Purity/1FRt0SRqqt3FBt/event_purity‚0 8 Purity/2FRt1SRqqt0FBt/FRt_purity‚0.13957072794437408 7 Purity/2FRt1SRqqt2FBt/FRt_purity‚0.2017543911933899 7 Purity/3FRt4SRqqt*FBt/FRt_purity‚0.1666666716337204 9"Purity/*FRt*SRqqt0FBt/event_purity‚0.0941033735871315 7 Purity/1FRt*SRqqt0FBt/FRt_purity‚0.1357344686985016 7 Purity/1FRt*SRqqt2FBt/FRt_purity‚0.3855123221874237 <"Purity/2FRt*SRqqt0FBt/event_purity‚0.0008488964522257447 7 Purity/2FRt*SRqqt2FBt/FBt_purity‚0.9869444370269775 ?&Purity/2FRt1SRqqt1FBt/event_proportion‚0.007875503972172737 ("Purity/3FRt0SRqqt1FBt/event_purity‚0 7 Purity/3FRt3SRqqt*FBt/FRt_purity‚0.1666666567325592 =&Purity/*FRt2SRqqt*FBt/event_proportion‚0.1332535296678543 :"Purity/*FRt3SRqqt1FBt/SRqqt_purity‚0.29629626870155334 6 Purity/*FRt3SRqqt1FBt/FBt_purity‚0.976190447807312 :"Purity/0FRt*SRqqt3FBt/event_purity‚0.06790123879909515 9"Purity/0FRt0SRqqt2FBt/event_purity‚0.3540874421596527 7 Purity/1FRt0SRqqt3FBt/FRt_purity‚0.1785714328289032 @&Purity/1FRt0SRqqt4FBt/event_proportion‚6.300403038039804e-05 ,&Purity/1FRt4SRqqt2FBt/event_proportion‚0 7 Purity/*FRt1SRqqt2FBt/FBt_purity‚0.9622837901115417 7 Purity/1FRt0SRqqt1FBt/FRt_purity‚0.2889437973499298 @&Purity/3FRt0SRqqt1FBt/event_proportion‚0.0011340725468471646 ,&Purity/4FRt*SRqqt4FBt/event_proportion‚0 ,&Purity/4FRt4SRqqt0FBt/event_proportion‚0 & Purity/1FRt*SRqqt4FBt/FRt_purity‚0 & Purity/2FRt2SRqqt2FBt/FBt_purity‚1 >&Purity/2FRt*SRqqt*FBt/event_proportion‚0.10244455933570862 >&Purity/2FRt*SRqqt0FBt/event_proportion‚0.07579384744167328 particle/accuracy_2_of_2‚1 particle/accuracy_3_of_3‚1 particle/accuracy_1_of_4‚1 & Purity/2FRt3SRqqt1FBt/FBt_purity‚1 & Purity/3FRt*SRqqt2FBt/FBt_purity‚1 @&Purity/*FRt1SRqqt4FBt/event_proportion‚6.300403038039804e-05 9"Purity/1FRt2SRqqt1FBt/SRqqt_purity‚0.3832949101924896 :"Purity/2FRt4SRqqt0FBt/SRqqt_purity‚0.20000000298023224 7 Purity/*FRt*SRqqt3FBt/FRt_purity‚0.1764705926179886 9"Purity/*FRt2SRqqt1FBt/SRqqt_purity‚0.3590773642063141 ?&Purity/1FRt3SRqqt0FBt/event_proportion‚0.007938507944345474 @&Purity/2FRt4SRqqt*FBt/event_proportion‚0.0003150201519019902 8 Purity/3FRt1SRqqt0FBt/FRt_purity‚0.08750000596046448 ,&Purity/4FRt1SRqqt1FBt/event_proportion‚0 & Purity/2FRt3SRqqt*FBt/FBt_purity‚1 ("Purity/3FRt4SRqqt*FBt/event_purity‚0 7 Purity/*FRt*SRqqt1FBt/FRt_purity‚0.2706339359283447 :"Purity/1FRt*SRqqt0FBt/event_purity‚0.04324929416179657 8 Purity/1FRt2SRqqt1FBt/FRt_purity‚0.16620585322380066 ,&Purity/4FRt0SRqqt4FBt/event_proportion‚0 ,&Purity/4FRt4SRqqt1FBt/event_proportion‚0 ,"Purity/3FRt4SRqqt*FBt/SRqqt_purity‚0.125 7 Purity/3FRt1SRqqt2FBt/FRt_purity‚0.0833333358168602 & Purity/1FRt4SRqqt*FBt/FRt_purity‚0 9"Purity/1FRt*SRqqt2FBt/event_purity‚0.0672810897231102 ("Purity/2FRt4SRqqt0FBt/event_purity‚0 & Purity/*FRt*SRqqt4FBt/FRt_purity‚0 ;"Purity/0FRt2SRqqt*FBt/event_purity‚0.017582062631845474 9"Purity/0FRt2SRqqt*FBt/SRqqt_purity‚0.2691006064414978 ,&Purity/1FRt3SRqqt4FBt/event_proportion‚0 @&Purity/2FRt*SRqqt3FBt/event_proportion‚6.300403038039804e-05 @&Purity/3FRt0SRqqt*FBt/event_proportion‚0.0027091733645647764 ("Purity/3FRt3SRqqt*FBt/event_purity‚0 8"Purity/3FRt3SRqqt*FBt/SRqqt_purity‚0.261904776096344 & Purity/1FRt3SRqqt2FBt/FBt_purity‚1 8 Purity/*FRt*SRqqt0FBt/FRt_purity‚0.13375385105609894 8 Purity/*FRt*SRqqt2FBt/FRt_purity‚0.33620476722717285 @&Purity/*FRt3SRqqt2FBt/event_proportion‚6.300403038039804e-05 8"Purity/1FRt*SRqqt0FBt/SRqqt_purity‚0.228170245885849 :"Purity/1FRt*SRqqt1FBt/event_purity‚0.07492631673812866 8 Purity/1FRt2SRqqt0FBt/FRt_purity‚0.15981340408325195 ,&Purity/3FRt1SRqqt4FBt/event_proportion‚0 7 Purity/1FRt2SRqqt2FBt/FRt_purity‚0.2222222238779068 :"Purity/*FRt*SRqqt2FBt/SRqqt_purity‚0.42830130457878113 @&Purity/3FRt0SRqqt2FBt/event_proportion‚6.300403038039804e-05 8 Purity/3FRt1SRqqt1FBt/FRt_purity‚0.23076923191547394 ,&Purity/4FRt3SRqqt2FBt/event_proportion‚0 ("Purity/1FRt2SRqqt2FBt/event_purity‚0 jet/accuracy_2_of_7‚0 ("Purity/2FRt0SRqqt3FBt/event_purity‚0 :"Purity/*FRt*SRqqt0FBt/SRqqt_purity‚0.23849324882030487 ?&Purity/0FRt0SRqqt2FBt/event_proportion‚0.029674898833036423 A&Purity/0FRt0SRqqt4FBt/event_proportion‚0.00025201612152159214 7 Purity/0FRt2SRqqt1FBt/FBt_purity‚0.9838709831237793 8 Purity/1FRt*SRqqt1FBt/FRt_purity‚0.26295503973960876 :"Purity/2FRt*SRqqt0FBt/SRqqt_purity‚0.21174632012844086 & Purity/1FRt0SRqqt4FBt/FRt_purity‚0 ("Purity/1FRt4SRqqt*FBt/event_purity‚0 9"Purity/0FRt1SRqqt*FBt/event_purity‚0.2162652462720871 :"Purity/0FRt1SRqqt*FBt/SRqqt_purity‚0.29414936900138855 :"Purity/0FRt1SRqqt1FBt/event_purity‚0.12045029550790787 + Purity/1FRt*SRqqt3FBt/FRt_purity‚0.1875 :"Purity/1FRt0SRqqt2FBt/event_purity‚0.09752562642097473 8 Purity/2FRt1SRqqt1FBt/FRt_purity‚0.28041476011276245 @&Purity/3FRt1SRqqt0FBt/event_proportion‚0.0025201612152159214 +"Purity/1FRt4SRqqt*FBt/SRqqt_purity‚0.25 7 Purity/0FRt2SRqqt*FBt/FBt_purity‚0.9731183052062988 =&Purity/1FRt*SRqqt*FBt/event_proportion‚0.4366179406642914 8 Purity/1FRt0SRqqt0FBt/FRt_purity‚0.09607168287038803 6 Purity/1FRt0SRqqt2FBt/FRt_purity‚0.424804151058197 ?&Purity/1FRt3SRqqt*FBt/event_proportion‚0.009135584346950054 @&Purity/3FRt3SRqqt0FBt/event_proportion‚0.0005670362734235823 @&Purity/4FRt0SRqqt*FBt/event_proportion‚6.300403038039804e-05 ,&Purity/4FRt3SRqqt3FBt/event_proportion‚0 @&Purity/*FRt0SRqqt4FBt/event_proportion‚0.0003150201519019902 @&Purity/*FRt2SRqqt3FBt/event_proportion‚6.300403038039804e-05 ?&Purity/0FRt2SRqqt1FBt/event_proportion‚0.009261592291295528 ("Purity/2FRt2SRqqt1FBt/event_purity‚0 ,&Purity/4FRt0SRqqt3FBt/event_proportion‚0 particle/accuracy_1_of_6‚1 7 Purity/*FRt0SRqqt*FBt/FBt_purity‚0.9891176819801331 7 Purity/1FRt1SRqqt1FBt/FBt_purity‚0.9943307042121887 9"Purity/1FRt3SRqqt0FBt/SRqqt_purity‚0.2332821488380432 9"Purity/2FRt1SRqqt2FBt/SRqqt_purity‚0.6184210777282715 ,&Purity/2FRt3SRqqt2FBt/event_proportion‚0 8 Purity/3FRt0SRqqt*FBt/FRt_purity‚0.21212120354175568 ,&Purity/3FRt0SRqqt4FBt/event_proportion‚0 ,&Purity/3FRt4SRqqt2FBt/event_proportion‚0 :"Purity/*FRt2SRqqt0FBt/SRqqt_purity‚0.24156102538108826 @&Purity/2FRt0SRqqt3FBt/event_proportion‚6.300403038039804e-05 particle/accuracy_6_of_7‚1 +"Purity/1FRt4SRqqt0FBt/SRqqt_purity‚0.25 8"Purity/0FRt*SRqqt3FBt/SRqqt_purity‚0.800000011920929 & Purity/4FRt1SRqqt0FBt/FRt_purity‚0 7 Purity/*FRt0SRqqt1FBt/FBt_purity‚0.9990976452827454 >&Purity/0FRt*SRqqt0FBt/event_proportion‚0.21931703388690948 ("Purity/0FRt0SRqqt0FBt/event_purity‚1 =&Purity/0FRt1SRqqt*FBt/event_proportion‚0.2305317521095276 7 Purity/1FRt1SRqqt*FBt/FBt_purity‚0.9791141748428345 ,&Purity/2FRt*SRqqt4FBt/event_proportion‚0¢‘0³®Oª Ž§ & Purity/4FRt0SRqqt0FBt/FRt_purity‚0 :"Purity/*FRt*SRqqt1FBt/event_purity‚0.37020379304885864 7 Purity/*FRt0SRqqt0FBt/FRt_purity‚0.0945061668753624 8 Purity/*FRt0SRqqt2FBt/FRt_purity‚0.39325767755508423 ?&Purity/2FRt3SRqqt*FBt/event_proportion‚0.004032257944345474 * Purity/0FRt*SRqqt4FBt/FBt_purity‚0.875 & Purity/0FRt4SRqqt*FBt/FBt_purity‚1 & Purity/2FRt*SRqqt3FBt/FBt_purity‚1 & Purity/1FRt1SRqqt4FBt/FRt_purity‚0 ?&Purity/0FRt3SRqqt0FBt/event_proportion‚0.006111391354352236 ("Purity/1FRt2SRqqt*FBt/event_purity‚0 :"Purity/1FRt2SRqqt*FBt/SRqqt_purity‚0.26454105973243713 :"Purity/1FRt3SRqqt1FBt/SRqqt_purity‚0.24814817309379578 8 Purity/2FRt*SRqqt*FBt/FRt_purity‚0.17533613741397858 9"Purity/3FRt*SRqqt1FBt/SRqqt_purity‚0.5444444417953491 *"Purity/3FRt1SRqqt1FBt/SRqqt_purity‚0.5 *jet/accuracy_1_of_4‚0.7918799519538879 ;"Purity/1FRt1SRqqt*FBt/event_purity‚0.008347955532371998 8"Purity/1FRt1SRqqt*FBt/SRqqt_purity‚0.276454895734787 8 Purity/1FRt1SRqqt0FBt/FRt_purity‚0.16414634883403778 8 Purity/1FRt1SRqqt2FBt/FRt_purity‚0.23937073349952698 8 Purity/2FRt2SRqqt*FBt/FRt_purity‚0.17266826331615448 9"Purity/2FRt3SRqqt0FBt/SRqqt_purity‚0.2149711698293686 & Purity/0FRt4SRqqt1FBt/FBt_purity‚1 7 Purity/3FRt2SRqqt1FBt/FRt_purity‚0.0555555559694767 8 Purity/*FRt1SRqqt*FBt/FRt_purity‚0.18116876482963562 ,&Purity/0FRt2SRqqt4FBt/event_proportion‚0 & Purity/*FRt0SRqqt4FBt/FRt_purity‚0 7 Purity/*FRt1SRqqt3FBt/FBt_purity‚0.8730158805847168 ("Purity/4FRt1SRqqt*FBt/event_purity‚0 ("Purity/4FRt1SRqqt*FBt/SRqqt_purity‚0 7 Purity/3FRt*SRqqt*FBt/FRt_purity‚0.1525217443704605 & Purity/*FRt4SRqqt*FBt/FBt_purity‚1 7 Purity/2FRt4SRqqt0FBt/FRt_purity‚0.4000000059604645 @&Purity/*FRt1SRqqt3FBt/event_proportion‚0.0006930443341843784 :"Purity/0FRt*SRqqt2FBt/SRqqt_purity‚0.36314815282821655 ?&Purity/0FRt2SRqqt0FBt/event_proportion‚0.043283771723508835 ,&Purity/4FRt4SRqqt*FBt/event_proportion‚0 & Purity/*FRt4SRqqt1FBt/FBt_purity‚1 & Purity/2FRt0SRqqt3FBt/FRt_purity‚0 ("Purity/1FRt1SRqqt4FBt/event_purity‚0 8"Purity/*FRt0SRqqt1FBt/event_purity‚0.640150785446167 >&Purity/*FRt1SRqqt0FBt/event_proportion‚0.33039313554763794 >&Purity/0FRt2SRqqt*FBt/event_proportion‚0.05361643061041832 8 Purity/2FRt0SRqqt1FBt/FRt_purity‚0.30209001898765564 ,&Purity/3FRt3SRqqt1FBt/event_proportion‚0 7 Purity/0FRt*SRqqt1FBt/FBt_purity‚0.9946838617324829 * Purity/0FRt3SRqqt*FBt/FBt_purity‚0.875 ("Purity/2FRt*SRqqt3FBt/event_purity‚0 :"Purity/1FRt0SRqqt*FBt/event_purity‚0.10721910744905472 >&Purity/1FRt2SRqqt*FBt/event_proportion‚0.05727066472172737 8 Purity/1FRt3SRqqt*FBt/FRt_purity‚0.18877367675304413 ?&Purity/2FRt3SRqqt0FBt/event_proportion‚0.003654233878478408 ,&Purity/3FRt*SRqqt3FBt/event_proportion‚0 ("Purity/*FRt2SRqqt2FBt/event_purity‚0 9"Purity/*FRt1SRqqt3FBt/SRqqt_purity‚0.7142857313156128 >&Purity/*FRt2SRqqt1FBt/event_proportion‚0.02255544438958168 >&Purity/*FRt3SRqqt0FBt/event_proportion‚0.01827116869390011 7 Purity/0FRt*SRqqt*FBt/FBt_purity‚0.9847093224525452 ,&Purity/0FRt3SRqqt3FBt/event_proportion‚0 7 Purity/2FRt0SRqqt0FBt/FRt_purity‚0.0897231251001358 8 Purity/2FRt0SRqqt2FBt/FRt_purity‚0.36249998211860657 * Purity/0FRt3SRqqt1FBt/FBt_purity‚0.875 ("Purity/*FRt1SRqqt4FBt/SRqqt_purity‚0 =&Purity/*FRt0SRqqt2FBt/event_proportion‚0.0483240932226181 >&Purity/0FRt0SRqqt0FBt/event_proportion‚0.04082661122083664 ,&Purity/3FRt4SRqqt4FBt/event_proportion‚0 7 Purity/*FRt0SRqqt3FBt/FRt_purity‚0.1666666716337204 :"Purity/1FRt*SRqqt*FBt/event_purity‚0.05186711996793747 :"Purity/1FRt*SRqqt*FBt/SRqqt_purity‚0.26915037631988525 8 Purity/*FRt0SRqqt1FBt/FRt_purity‚0.29427090287208557 ,&Purity/*FRt3SRqqt3FBt/event_proportion‚0 jet/accuracy_5_of_7‚0 ("Purity/4FRt2SRqqt*FBt/event_purity‚0 *"Purity/4FRt2SRqqt*FBt/SRqqt_purity‚0.5 particle/accuracy_7_of_7‚1 ) Purity/1FRt1SRqqt3FBt/FRt_purity‚0.25 @&Purity/0FRt4SRqqt*FBt/event_proportion‚0.0005040322430431843 ,&Purity/0FRt4SRqqt4FBt/event_proportion‚0 7 Purity/1FRt1SRqqt1FBt/FRt_purity‚0.2546731233596802 9"Purity/2FRt*SRqqt1FBt/SRqqt_purity‚0.3635369837284088 ("Purity/2FRt3SRqqt0FBt/event_purity‚0 8 Purity/3FRt2SRqqt0FBt/FRt_purity‚0.17874395847320557 A&Purity/3FRt4SRqqt0FBt/event_proportion‚0.00012600806076079607 jet/accuracy_6_of_7‚0 >&Purity/*FRt*SRqqt2FBt/event_proportion‚0.06747731566429138 8 Purity/*FRt3SRqqt0FBt/FRt_purity‚0.22105243802070618 7 Purity/2FRt1SRqqt*FBt/FBt_purity‚0.9964157938957214 particle/accuracy_2_of_5‚1 * Purity/*FRt*SRqqt4FBt/FBt_purity‚0.875 & Purity/3FRt0SRqqt2FBt/FBt_purity‚1 & Purity/*FRt3SRqqt2FBt/FRt_purity‚0 ("Purity/4FRt1SRqqt0FBt/event_purity‚0 @&Purity/*FRt*SRqqt4FBt/event_proportion‚0.0003780241822823882 7 Purity/1FRt*SRqqt*FBt/FBt_purity‚0.9855114817619324 ?&Purity/1FRt2SRqqt1FBt/event_proportion‚0.008946572430431843 ?&Purity/2FRt2SRqqt*FBt/event_proportion‚0.019468246027827263 ("Purity/0FRt4SRqqt1FBt/SRqqt_purity‚0 7 Purity/0FRt1SRqqt2FBt/FBt_purity‚0.9622685313224792 9"Purity/0FRt2SRqqt1FBt/SRqqt_purity‚0.3607654571533203 & Purity/2FRt1SRqqt1FBt/FBt_purity‚1 :"Purity/2FRt2SRqqt0FBt/SRqqt_purity‚0.21066135168075562 ("Purity/*FRt4SRqqt1FBt/event_purity‚0 jet/accuracy_7_of_7‚0 9"Purity/0FRt3SRqqt1FBt/SRqqt_purity‚0.3541666567325592 & Purity/4FRt*SRqqt*FBt/FRt_purity‚0 A&Purity/0FRt4SRqqt0FBt/event_proportion‚0.00044102821266278625 7 Purity/1FRt*SRqqt1FBt/FBt_purity‚0.9935746788978577 @&Purity/1FRt3SRqqt2FBt/event_proportion‚6.300403038039804e-05 ("Purity/3FRt1SRqqt0FBt/event_purity‚0 :"Purity/3FRt2SRqqt0FBt/SRqqt_purity‚0.26449277997016907 ,&Purity/3FRt2SRqqt3FBt/event_proportion‚0 particle/accuracy_2_of_3‚1 ("Purity/4FRt0SRqqt0FBt/event_purity‚0 ?&Purity/*FRt3SRqqt*FBt/event_proportion‚0.020476309582591057 ,&Purity/2FRt3SRqqt4FBt/event_proportion‚0 & Purity/3FRt1SRqqt1FBt/FBt_purity‚1 & Purity/1FRt0SRqqt4FBt/FBt_purity‚1 :"Purity/0FRt*SRqqt0FBt/event_purity‚0.20215217769145966 8"Purity/0FRt*SRqqt1FBt/event_purity‚0.626227080821991 @&Purity/1FRt*SRqqt3FBt/event_proportion‚0.0013230846961960196 ?&Purity/1FRt0SRqqt2FBt/event_proportion‚0.015120967291295528 @&Purity/2FRt4SRqqt0FBt/event_proportion‚0.0003150201519019902 *jet/accuracy_2_of_6‚0.5438596606254578 9"Purity/1FRt2SRqqt0FBt/SRqqt_purity‚0.2355533242225647 :"Purity/2FRt0SRqqt2FBt/event_purity‚0.02500000037252903 8 Purity/2FRt3SRqqt0FBt/FRt_purity‚0.22867964208126068 & Purity/3FRt1SRqqt*FBt/FBt_purity‚1 ,&Purity/3FRt3SRqqt4FBt/event_proportion‚0 ("Purity/2FRt2SRqqt2FBt/event_purity‚0 & Purity/4FRt2SRqqt*FBt/FRt_purity‚0 >&Purity/0FRt*SRqqt*FBt/event_proportion‚0.40996724367141724 ;"Purity/0FRt1SRqqt2FBt/event_purity‚0.011111111380159855 @&Purity/0FRt3SRqqt1FBt/event_proportion‚0.0006300403038039804 :"Purity/*FRt*SRqqt*FBt/event_purity‚0.17793890833854675 :"Purity/*FRt*SRqqt*FBt/SRqqt_purity‚0.27279314398765564 7 Purity/*FRt*SRqqt1FBt/FBt_purity‚0.9948082566261292 9"Purity/0FRt*SRqqt0FBt/SRqqt_purity‚0.2539779543876648 7 Purity/1FRt2SRqqt1FBt/FBt_purity‚0.9760368466377258 +"Purity/0FRt0SRqqt4FBt/event_purity‚0.25 6 Purity/0FRt0SRqqt2FBt/FBt_purity‚0.982708215713501 ("Purity/3FRt*SRqqt2FBt/event_purity‚0 & Purity/4FRt0SRqqt*FBt/FRt_purity‚0 6 Purity/*FRt*SRqqt*FBt/FBt_purity‚0.985958993434906 7 Purity/1FRt2SRqqt*FBt/FBt_purity‚0.9813225269317627 ?&Purity/2FRt*SRqqt1FBt/event_proportion‚0.021169355139136314 particle/accuracy_5_of_5‚1 particle/accuracy_4_of_7‚1 ,&Purity/*FRt4SRqqt3FBt/event_proportion‚0 ("Purity/1FRt2SRqqt0FBt/event_purity‚0 ,jet/accuracy_3_of_3‚0.012774068862199783 8 Purity/2FRt3SRqqt1FBt/FRt_purity‚0.10000000149011612 jet/accuracy_4_of_7‚0 * Purity/1FRt*SRqqt4FBt/FBt_purity‚0.875 ("Purity/*FRt1SRqqt3FBt/event_purity‚0 :"Purity/*FRt0SRqqt0FBt/event_purity‚0.08057259768247604 7 Purity/1FRt0SRqqt1FBt/FBt_purity‚0.9973816871643066 ("Purity/3FRt*SRqqt*FBt/event_purity‚0 9"Purity/3FRt*SRqqt*FBt/SRqqt_purity‚0.2571975886821747 A&Purity/3FRt4SRqqt*FBt/event_proportion‚0.00012600806076079607 ,jet/accuracy_4_of_5‚0.007526882458478212 particle/accuracy_1_of_2‚1 ("Purity/3FRt2SRqqt1FBt/event_purity‚0 8 Purity/*FRt3SRqqt1FBt/FRt_purity‚0.36666667461395264 >&Purity/2FRt1SRqqt0FBt/event_proportion‚0.03181703761219978 ,&Purity/4FRt4SRqqt4FBt/event_proportion‚0 particle/accuracy_3_of_4‚1 )jet/accuracy_1_of_6‚0.956140398979187 ("Purity/1FRt*SRqqt4FBt/SRqqt_purity‚0 ("Purity/*FRt3SRqqt2FBt/event_purity‚0 ("Purity/4FRt1SRqqt0FBt/SRqqt_purity‚0 A&Purity/0FRt*SRqqt4FBt/event_proportion‚0.00025201612152159214 6 Purity/1FRt0SRqqt*FBt/FBt_purity‚0.990094780921936 @&Purity/3FRt*SRqqt0FBt/event_proportion‚0.0071824598126113415 9"Purity/3FRt1SRqqt0FBt/SRqqt_purity‚0.3499999940395355 ("Purity/3FRt2SRqqt0FBt/event_purity‚0 7 Purity/*FRt2SRqqt2FBt/FBt_purity‚0.9868420958518982 ?&Purity/*FRt*SRqqt3FBt/event_proportion‚0.004473286215215921 ;"Purity/2FRt*SRqqt2FBt/event_purity‚0.023333333432674408 ("Purity/2FRt2SRqqt0FBt/event_purity‚0 =&Purity/*FRt*SRqqt*FBt/event_proportion‚0.9591733813285828 7 Purity/*FRt3SRqqt*FBt/FBt_purity‚0.9772727489471436 ("Purity/1FRt0SRqqt3FBt/event_purity‚0 8 Purity/2FRt1SRqqt0FBt/FRt_purity‚0.13957072794437408 7 Purity/2FRt1SRqqt2FBt/FRt_purity‚0.2017543911933899 7 Purity/3FRt4SRqqt*FBt/FRt_purity‚0.1666666716337204 9"Purity/*FRt*SRqqt0FBt/event_purity‚0.0941033735871315 7 Purity/1FRt*SRqqt0FBt/FRt_purity‚0.1357344686985016 7 Purity/1FRt*SRqqt2FBt/FRt_purity‚0.3855123221874237 <"Purity/2FRt*SRqqt0FBt/event_purity‚0.0008488964522257447 7 Purity/2FRt*SRqqt2FBt/FBt_purity‚0.9869444370269775 ?&Purity/2FRt1SRqqt1FBt/event_proportion‚0.007875503972172737 ("Purity/3FRt0SRqqt1FBt/event_purity‚0 7 Purity/3FRt3SRqqt*FBt/FRt_purity‚0.1666666567325592 =&Purity/*FRt2SRqqt*FBt/event_proportion‚0.1332535296678543 :"Purity/*FRt3SRqqt1FBt/SRqqt_purity‚0.29629626870155334 6 Purity/*FRt3SRqqt1FBt/FBt_purity‚0.976190447807312 :"Purity/0FRt*SRqqt3FBt/event_purity‚0.06790123879909515 9"Purity/0FRt0SRqqt2FBt/event_purity‚0.3540874421596527 7 Purity/1FRt0SRqqt3FBt/FRt_purity‚0.1785714328289032 @&Purity/1FRt0SRqqt4FBt/event_proportion‚6.300403038039804e-05 ,&Purity/1FRt4SRqqt2FBt/event_proportion‚0 7 Purity/*FRt1SRqqt2FBt/FBt_purity‚0.9622837901115417 7 Purity/1FRt0SRqqt1FBt/FRt_purity‚0.2889437973499298 @&Purity/3FRt0SRqqt1FBt/event_proportion‚0.0011340725468471646 ,&Purity/4FRt*SRqqt4FBt/event_proportion‚0 ,&Purity/4FRt4SRqqt0FBt/event_proportion‚0 & Purity/1FRt*SRqqt4FBt/FRt_purity‚0 & Purity/2FRt2SRqqt2FBt/FBt_purity‚1 >&Purity/2FRt*SRqqt*FBt/event_proportion‚0.10244455933570862 >&Purity/2FRt*SRqqt0FBt/event_proportion‚0.07579384744167328 particle/accuracy_2_of_2‚1 particle/accuracy_3_of_3‚1 particle/accuracy_1_of_4‚1 & Purity/2FRt3SRqqt1FBt/FBt_purity‚1 & Purity/3FRt*SRqqt2FBt/FBt_purity‚1 @&Purity/*FRt1SRqqt4FBt/event_proportion‚6.300403038039804e-05 9"Purity/1FRt2SRqqt1FBt/SRqqt_purity‚0.3832949101924896 :"Purity/2FRt4SRqqt0FBt/SRqqt_purity‚0.20000000298023224 7 Purity/*FRt*SRqqt3FBt/FRt_purity‚0.1764705926179886 9"Purity/*FRt2SRqqt1FBt/SRqqt_purity‚0.3590773642063141 ?&Purity/1FRt3SRqqt0FBt/event_proportion‚0.007938507944345474 @&Purity/2FRt4SRqqt*FBt/event_proportion‚0.0003150201519019902 8 Purity/3FRt1SRqqt0FBt/FRt_purity‚0.08750000596046448 ,&Purity/4FRt1SRqqt1FBt/event_proportion‚0 & Purity/2FRt3SRqqt*FBt/FBt_purity‚1 ("Purity/3FRt4SRqqt*FBt/event_purity‚0 7 Purity/*FRt*SRqqt1FBt/FRt_purity‚0.2706339359283447 :"Purity/1FRt*SRqqt0FBt/event_purity‚0.04324929416179657 8 Purity/1FRt2SRqqt1FBt/FRt_purity‚0.16620585322380066 ,&Purity/4FRt0SRqqt4FBt/event_proportion‚0 ,&Purity/4FRt4SRqqt1FBt/event_proportion‚0 ,"Purity/3FRt4SRqqt*FBt/SRqqt_purity‚0.125 7 Purity/3FRt1SRqqt2FBt/FRt_purity‚0.0833333358168602 & Purity/1FRt4SRqqt*FBt/FRt_purity‚0 9"Purity/1FRt*SRqqt2FBt/event_purity‚0.0672810897231102 ("Purity/2FRt4SRqqt0FBt/event_purity‚0 & Purity/*FRt*SRqqt4FBt/FRt_purity‚0 ;"Purity/0FRt2SRqqt*FBt/event_purity‚0.017582062631845474 9"Purity/0FRt2SRqqt*FBt/SRqqt_purity‚0.2691006064414978 ,&Purity/1FRt3SRqqt4FBt/event_proportion‚0 @&Purity/2FRt*SRqqt3FBt/event_proportion‚6.300403038039804e-05 @&Purity/3FRt0SRqqt*FBt/event_proportion‚0.0027091733645647764 ("Purity/3FRt3SRqqt*FBt/event_purity‚0 8"Purity/3FRt3SRqqt*FBt/SRqqt_purity‚0.261904776096344 & Purity/1FRt3SRqqt2FBt/FBt_purity‚1 8 Purity/*FRt*SRqqt0FBt/FRt_purity‚0.13375385105609894 8 Purity/*FRt*SRqqt2FBt/FRt_purity‚0.33620476722717285 @&Purity/*FRt3SRqqt2FBt/event_proportion‚6.300403038039804e-05 8"Purity/1FRt*SRqqt0FBt/SRqqt_purity‚0.228170245885849 :"Purity/1FRt*SRqqt1FBt/event_purity‚0.07492631673812866 8 Purity/1FRt2SRqqt0FBt/FRt_purity‚0.15981340408325195 ,&Purity/3FRt1SRqqt4FBt/event_proportion‚0 7 Purity/1FRt2SRqqt2FBt/FRt_purity‚0.2222222238779068 :"Purity/*FRt*SRqqt2FBt/SRqqt_purity‚0.42830130457878113 @&Purity/3FRt0SRqqt2FBt/event_proportion‚6.300403038039804e-05 8 Purity/3FRt1SRqqt1FBt/FRt_purity‚0.23076923191547394 ,&Purity/4FRt3SRqqt2FBt/event_proportion‚0 ("Purity/1FRt2SRqqt2FBt/event_purity‚0 jet/accuracy_2_of_7‚0 ("Purity/2FRt0SRqqt3FBt/event_purity‚0 :"Purity/*FRt*SRqqt0FBt/SRqqt_purity‚0.23849324882030487 ?&Purity/0FRt0SRqqt2FBt/event_proportion‚0.029674898833036423 A&Purity/0FRt0SRqqt4FBt/event_proportion‚0.00025201612152159214 7 Purity/0FRt2SRqqt1FBt/FBt_purity‚0.9838709831237793 8 Purity/1FRt*SRqqt1FBt/FRt_purity‚0.26295503973960876 :"Purity/2FRt*SRqqt0FBt/SRqqt_purity‚0.21174632012844086 & Purity/1FRt0SRqqt4FBt/FRt_purity‚0 ("Purity/1FRt4SRqqt*FBt/event_purity‚0 9"Purity/0FRt1SRqqt*FBt/event_purity‚0.2162652462720871 :"Purity/0FRt1SRqqt*FBt/SRqqt_purity‚0.29414936900138855 :"Purity/0FRt1SRqqt1FBt/event_purity‚0.12045029550790787 + Purity/1FRt*SRqqt3FBt/FRt_purity‚0.1875 :"Purity/1FRt0SRqqt2FBt/event_purity‚0.09752562642097473 8 Purity/2FRt1SRqqt1FBt/FRt_purity‚0.28041476011276245 @&Purity/3FRt1SRqqt0FBt/event_proportion‚0.0025201612152159214 +"Purity/1FRt4SRqqt*FBt/SRqqt_purity‚0.25 7 Purity/0FRt2SRqqt*FBt/FBt_purity‚0.9731183052062988 =&Purity/1FRt*SRqqt*FBt/event_proportion‚0.4366179406642914 8 Purity/1FRt0SRqqt0FBt/FRt_purity‚0.09607168287038803 6 Purity/1FRt0SRqqt2FBt/FRt_purity‚0.424804151058197 ?&Purity/1FRt3SRqqt*FBt/event_proportion‚0.009135584346950054 @&Purity/3FRt3SRqqt0FBt/event_proportion‚0.0005670362734235823 @&Purity/4FRt0SRqqt*FBt/event_proportion‚6.300403038039804e-05 ,&Purity/4FRt3SRqqt3FBt/event_proportion‚0 @&Purity/*FRt0SRqqt4FBt/event_proportion‚0.0003150201519019902 @&Purity/*FRt2SRqqt3FBt/event_proportion‚6.300403038039804e-05 ?&Purity/0FRt2SRqqt1FBt/event_proportion‚0.009261592291295528 ("Purity/2FRt2SRqqt1FBt/event_purity‚0 ,&Purity/4FRt0SRqqt3FBt/event_proportion‚0 particle/accuracy_1_of_6‚1 7 Purity/*FRt0SRqqt*FBt/FBt_purity‚0.9891176819801331 7 Purity/1FRt1SRqqt1FBt/FBt_purity‚0.9943307042121887 9"Purity/1FRt3SRqqt0FBt/SRqqt_purity‚0.2332821488380432 9"Purity/2FRt1SRqqt2FBt/SRqqt_purity‚0.6184210777282715 ,&Purity/2FRt3SRqqt2FBt/event_proportion‚0 8 Purity/3FRt0SRqqt*FBt/FRt_purity‚0.21212120354175568 ,&Purity/3FRt0SRqqt4FBt/event_proportion‚0 ,&Purity/3FRt4SRqqt2FBt/event_proportion‚0 :"Purity/*FRt2SRqqt0FBt/SRqqt_purity‚0.24156102538108826 @&Purity/2FRt0SRqqt3FBt/event_proportion‚6.300403038039804e-05 particle/accuracy_6_of_7‚1 +"Purity/1FRt4SRqqt0FBt/SRqqt_purity‚0.25 8"Purity/0FRt*SRqqt3FBt/SRqqt_purity‚0.800000011920929 & Purity/4FRt1SRqqt0FBt/FRt_purity‚0 7 Purity/*FRt0SRqqt1FBt/FBt_purity‚0.9990976452827454 >&Purity/0FRt*SRqqt0FBt/event_proportion‚0.21931703388690948 ("Purity/0FRt0SRqqt0FBt/event_purity‚1 =&Purity/0FRt1SRqqt*FBt/event_proportion‚0.2305317521095276 7 Purity/1FRt1SRqqt*FBt/FBt_purity‚0.9791141748428345 ,&Purity/2FRt*SRqqt4FBt/event_proportion‚0 >&Purity/2FRt1SRqqt*FBt/event_proportion‚0.04158266261219978 ,&Purity/2FRt4SRqqt2FBt/event_proportion‚0 ("Purity/3FRt1SRqqt*FBt/event_purity‚0 7"Purity/3FRt1SRqqt*FBt/SRqqt_purity‚0.34876549243927 @&Purity/4FRt1SRqqt0FBt/event_proportion‚6.300403038039804e-05 ,&Purity/4FRt3SRqqt4FBt/event_proportion‚0 jet/accuracy_5_of_5‚0 (particle/accuracy‚0.8184067010879517 +particle/sensitivity‚0.2510537803173065 ("Purity/2FRt3SRqqt1FBt/event_purity‚0 :"Purity/*FRt1SRqqt1FBt/SRqqt_purity‚0.40868526697158813 ?&Purity/2FRt*SRqqt2FBt/event_proportion‚0.005418346729129553 ?&Purity/2FRt0SRqqt0FBt/event_proportion‚0.024571573361754417 @&Purity/4FRt*SRqqt0FBt/event_proportion‚0.0001890120911411941 7 Purity/*FRt4SRqqt0FBt/FRt_purity‚0.1666666716337204 :"Purity/0FRt3SRqqt0FBt/SRqqt_purity‚0.23849208652973175 particle/accuracy_1_of_7‚1 ("Purity/*FRt*SRqqt4FBt/SRqqt_purity‚0 ?&Purity/2FRt0SRqqt*FBt/event_proportion‚0.037046369165182114 6"Purity/2FRt1SRqqt1FBt/SRqqt_purity‚0.3913594186306 ("Purity/3FRt1SRqqt2FBt/event_purity‚0 ("Purity/4FRt2SRqqt0FBt/event_purity‚0 8"Purity/0FRt0SRqqt*FBt/event_purity‚0.818209707736969 @&Purity/3FRt0SRqqt0FBt/event_proportion‚0.0015120967291295528 7 Purity/0FRt1SRqqt3FBt/FBt_purity‚0.8333333730697632 ?&Purity/0FRt1SRqqt2FBt/event_proportion‚0.008757560513913631 <"Purity/1FRt1SRqqt2FBt/event_purity‚0.0071428571827709675 :"Purity/0FRt2SRqqt2FBt/SRqqt_purity‚0.36538460850715637 ("Purity/0FRt2SRqqt3FBt/SRqqt_purity‚1 9"Purity/*FRt1SRqqt2FBt/SRqqt_purity‚0.4530316889286041 7 Purity/0FRt0SRqqt3FBt/FBt_purity‚0.9066666960716248 & Purity/2FRt0SRqqt1FBt/FBt_purity‚1 +"Purity/0FRt*SRqqt4FBt/event_purity‚0.25 ) Purity/1FRt1SRqqt4FBt/FBt_purity‚0.75 trainer/global_step‚1207 @&Purity/1FRt3SRqqt1FBt/event_proportion‚0.0011340725468471646 ?&Purity/2FRt2SRqqt1FBt/event_proportion‚0.003969253972172737 ("Purity/0FRt3SRqqt0FBt/event_purity‚0 @&Purity/0FRt2SRqqt3FBt/event_proportion‚6.300403038039804e-05 7 Purity/2FRt0SRqqt*FBt/FBt_purity‚0.9931899905204773 7 Purity/*FRt0SRqqt4FBt/FBt_purity‚0.8999999761581421 9"Purity/1FRt1SRqqt1FBt/SRqqt_purity‚0.4204671382904053 @&Purity/1FRt1SRqqt4FBt/event_proportion‚6.300403038039804e-05 @&Purity/4FRt1SRqqt*FBt/event_proportion‚6.300403038039804e-05 @&Purity/4FRt2SRqqt0FBt/event_proportion‚6.300403038039804e-05 +jet/accuracy_2_of_4‚0.26249203085899353 :"Purity/*FRt2SRqqt0FBt/event_purity‚0.00566460145637393 ,&Purity/*FRt4SRqqt4FBt/event_proportion‚0 ,&Purity/0FRt3SRqqt4FBt/event_proportion‚0 ,&Purity/3FRt4SRqqt1FBt/event_proportion‚0 ,jet/accuracy_3_of_4‚0.034572161734104156 ("Purity/1FRt4SRqqt0FBt/event_purity‚0 ("Purity/1FRt3SRqqt2FBt/event_purity‚0 ("Purity/1FRt3SRqqt0FBt/event_purity‚0 ,&Purity/3FRt3SRqqt2FBt/event_proportion‚0 particle/accuracy_3_of_5‚1 +particle/specificity‚0.9234230518341064 & Purity/3FRt2SRqqt1FBt/FBt_purity‚1 8 Purity/*FRt2SRqqt*FBt/FRt_purity‚0.16492612659931183 >&Purity/0FRt0SRqqt1FBt/event_proportion‚0.08606351166963577 ?&Purity/1FRt1SRqqt2FBt/event_proportion‚0.006111391354352236 ("Purity/2FRt1SRqqt*FBt/event_purity‚0 :"Purity/2FRt1SRqqt*FBt/SRqqt_purity‚0.26460006833076477 particle/accuracy_2_of_6‚1 jet/accuracy_3_of_7‚0 _runtime‚1011.953445378 ,&Purity/3FRt0SRqqt3FBt/event_proportion‚0 & Purity/3FRt2SRqqt*FBt/FBt_purity‚1 jet/accuracy_6_of_6‚0 7 Purity/*FRt2SRqqt3FBt/FBt_purity‚0.6666666865348816 ("Purity/0FRt2SRqqt3FBt/event_purity‚0 *"Purity/4FRt2SRqqt0FBt/SRqqt_purity‚0.5 :"Purity/*FRt0SRqqt2FBt/event_purity‚0.25009292364120483 :"Purity/*FRt1SRqqt0FBt/SRqqt_purity‚0.23820847272872925 >&Purity/0FRt*SRqqt2FBt/event_proportion‚0.03944052383303642 ("Purity/1FRt3SRqqt*FBt/event_purity‚0 9"Purity/1FRt3SRqqt*FBt/SRqqt_purity‚0.2423536628484726 7 Purity/3FRt1SRqqt*FBt/FRt_purity‚0.1419752985239029 @&Purity/4FRt2SRqqt*FBt/e?=ÏUçCvent_proportion‚6.300403038039804e-05 ,"Purity/3FRt4SRqqt0FBt/SRqqt_purity‚0.125 ("Purity/*FRt3SRqqt0FBt/event_purity‚0 ?&Purity/1FRt0SRqqt1FBt/event_proportion‚0.049773186445236206 & Purity/1FRt4SRqqt0FBt/FRt_purity‚0 >&Purity/1FRt1SRqqt1FBt/event_proportion‚0.03944052383303642 jet/accuracy_5_of_6‚0 9"Purity/3FRt*SRqqt0FBt/SRqqt_purity‚0.2241213172674179 ?&Purity/3FRt1SRqqt*FBt/event_proportion‚0.003717237850651145 particle/accuracy_1_of_5‚1 7 Purity/1FRt1SRqqt2FBt/FBt_purity‚0.9556547403335571 ?&Purity/2FRt2SRqqt0FBt/event_proportion‚0.015435988083481789 @&Purity/3FRt1SRqqt1FBt/event_proportion‚0.0009450604557059705 9"Purity/*FRt2SRqqt2FBt/SRqqt_purity‚0.2850877046585083 @&Purity/*FRt4SRqqt0FBt/event_proportion‚0.0012600806076079607 @&Purity/1FRt2SRqqt2FBt/event_proportion‚0.0010080644860863686 +jet/accuracy_2_of_2‚0.07190760225057602 particle/accuracy_6_of_6‚1 7 Purity/*FRt0SRqqt2FBt/FBt_purity‚0.9832113981246948 A&Purity/0FRt1SRqqt3FBt/event_proportion‚0.00044102821266278625 + Purity/1FRt*SRqqt3FBt/FBt_purity‚0.9375 8 Purity/2FRt1SRqqt*FBt/FRt_purity‚0.17398162186145782 ("Purity/2FRt4SRqqt*FBt/event_purity‚0 :"Purity/2FRt4SRqqt*FBt/SRqqt_purity‚0.20000000298023224 ("Purity/0FRt4SRqqt0FBt/event_purity‚0 7 Purity/3FRt4SRqqt0FBt/FRt_purity‚0.1666666716337204 8 Purity/1FRt*SRqqt*FBt/FRt_purity‚0.17591306567192078 =&Purity/1FRt1SRqqt0FBt/event_proportion‚0.1265120953321457 ;"Purity/2FRt*SRqqt1FBt/event_purity‚0.010880696587264538 ("Purity/2FRt3SRqqt*FBt/event_purity‚0 9"Purity/2FRt3SRqqt*FBt/SRqqt_purity‚0.2129206508398056 particle/accuracy_3_of_7‚1 ("Purity/1FRt1SRqqt4FBt/SRqqt_purity‚0 7 Purity/3FRt3SRqqt0FBt/FRt_purity‚0.1666666567325592 :"Purity/*FRt*SRqqt1FBt/SRqqt_purity‚0.38676396012306213 >&Purity/*FRt0SRqqt1FBt/event_proportion‚0.14591734111309052 9"Purity/0FRt*SRqqt2FBt/event_purity‚0.2662772834300995 8 Purity/1FRt0SRqqt*FBt/FRt_purity‚0.16800153255462646 :"Purity/1FRt0SRqqt0FBt/event_purity‚0.09607168287038803 ,&Purity/1FRt4SRqqt4FBt/event_proportion‚0 ,&Purity/4FRt1SRqqt3FBt/event_proportion‚0 >&Purity/*FRt0SRqqt0FBt/event_proportion‚0.15757308900356293 7 Purity/1FRt0SRqqt3FBt/FBt_purity‚0.9523809552192688 ("Purity/1FRt3SRqqt1FBt/event_purity‚0 ("Purity/3FRt*SRqqt1FBt/event_purity‚0 ("Purity/3FRt1SRqqt1FBt/event_purity‚0 *jet/accuracy_1_of_3‚0.7335270643234253 ,&Purity/*FRt4SRqqt2FBt/event_proportion‚0 >&Purity/0FRt0SRqqt*FBt/event_proportion‚0.11857359111309052 ("Purity/3FRt0SRqqt0FBt/event_purity‚0 ("Purity/0FRt4SRqqt*FBt/event_purity‚0 ."Purity/0FRt4SRqqt*FBt/SRqqt_purity‚0.15625 particle/accuracy_4_of_6‚1 )loss/val_total_loss‚49.49597158565765 >&Purity/1FRt0SRqqt0FBt/event_proportion‚0.13142640888690948 9"Purity/0FRt4SRqqt0FBt/SRqqt_purity‚0.1785714328289032 ("Purity/*FRt1SRqqt4FBt/event_purity‚0 8 Purity/*FRt*SRqqt*FBt/FRt_purity‚0.17455314099788666 :"Purity/*FRt3SRqqt0FBt/SRqqt_purity‚0.22708311676979065 @&Purity/*FRt4SRqqt1FBt/event_proportion‚6.300403038039804e-05 =&Purity/1FRt*SRqqt0FBt/event_proportion‚0.3135710656642914 8 Purity/1FRt2SRqqt*FBt/FRt_purity‚0.15966638922691345 6 Purity/2FRt0SRqqt2FBt/FBt_purity‚0.988095223903656 ,jet/accuracy_4_of_4‚0.003514922922477126 6 Purity/*FRt*SRqqt3FBt/FBt_purity‚0.904789388179779 :"Purity/*FRt1SRqqt*FBt/event_purity‚0.11439140886068344 :"Purity/*FRt1SRqqt*FBt/SRqqt_purity‚0.28498902916908264 :"Purity/*FRt1SRqqt0FBt/event_purity‚0.13481950759887695 :"Purity/2FRt0SRqqt1FBt/event_purity‚0.01944444514811039 ("Purity/0FRt3SRqqt*FBt/event_purity‚0 :"Purity/0FRt3SRqqt*FBt/SRqqt_purity‚0.24469955265522003 particle/accuracy_5_of_6‚1 ("Purity/3FRt*SRqqt0FBt/event_purity‚0 particle/accuracy_1_of_1‚1 ("Purity/3FRt4SRqqt0FBt/event_purity‚0 ?&Purity/3FRt*SRqqt*FBt/event_proportion‚0.009954636916518211 particle/accuracy_2_of_7‚1 =&Purity/0FRt1SRqqt0FBt/event_proportion‚0.1694808453321457 & Purity/3FRt*SRqqt1FBt/FBt_purity‚1 ,&Purity/4FRt3SRqqt1FBt/event_proportion‚0 9"Purity/*FRt3SRqqt2FBt/SRqqt_purity‚0.6666666865348816 =&Purity/*FRt0SRqqt*FBt/event_proportion‚0.3558467626571655 9"Purity/2FRt*SRqqt2FBt/SRqqt_purity‚0.6359649300575256 & Purity/3FRt*SRqqt*FBt/FBt_purity‚1 @&Purity/3FRt*SRqqt1FBt/event_proportion‚0.0024571572430431843 ,&Purity/4FRt2SRqqt1FBt/event_proportion‚0 7 Purity/*FRt4SRqqt*FBt/FRt_purity‚0.1666666716337204 ,&Purity/0FRt4SRqqt2FBt/event_proportion‚0 ,&Purity/3FRt4SRqqt3FBt/event_proportion‚0 *jet/accuracy_1_of_2‚0.6064389944076538 <"Purity/2FRt0SRqqt0FBt/event_purity‚0.0026881720405071974 & Purity/2FRt2SRqqt*FBt/FBt_purity‚1 7validation_average_jet_accuracy‚0.33515453338623047 *"Purity/3FRt2SRqqt1FBt/SRqqt_purity‚0.5 particle/accuracy_5_of_7‚1 +"Purity/3FRt*SRqqt2FBt/SRqqt_purity‚0.25 ("Purity/4FRt0SRqqt*FBt/event_purity‚0 7 Purity/*FRt1SRqqt*FBt/FBt_purity‚0.9809271693229675 7 Purity/*FRt2SRqqt1FBt/FRt_purity‚0.1919959932565689 & Purity/2FRt*SRqqt1FBt/FBt_purity‚1 ;"Purity/1FRt1SRqqt0FBt/event_purity‚0.006698790472000837 A&Purity/1FRt1SRqqt3FBt/event_proportion‚0.00025201612152159214 & Purity/2FRt2SRqqt1FBt/FBt_purity‚1 ,&Purity/2FRt4SRqqt3FBt/event_proportion‚0 ,&Purity/3FRt*SRqqt4FBt/event_proportion‚0 8 Purity/3FRt0SRqqt0FBt/FRt_purity‚0.10457516461610794 7 Purity/3FRt0SRqqt2FBt/FRt_purity‚0.3333333432674408 & Purity/*FRt3SRqqt2FBt/FBt_purity‚1 7 Purity/*FRt1SRqqt1FBt/FBt_purity‚0.9916524291038513 @&Purity/1FRt0SRqqt3FBt/event_proportion‚0.0010710685746744275 7 Purity/2FRt*SRqqt*FBt/FBt_purity‚0.9940999746322632 ,&Purity/2FRt0SRqqt4FBt/event_proportion‚0 @&Purity/3FRt*SRqqt2FBt/event_proportion‚0.0003150201519019902 ,&Purity/4FRt2SRqqt4FBt/event_proportion‚0 +jet/accuracy_1_of_1‚0.35932639241218567 -"Purity/*FRt4SRqqt0FBt/SRqqt_purity‚0.1875 =&Purity/*FRt*SRqqt0FBt/event_proportion‚0.6160534024238586 8 Purity/*FRt2SRqqt0FBt/FRt_purity‚0.15943489968776703 ,&Purity/0FRt1SRqqt4FBt/event_proportion‚0 ,&Purity/1FRt4SRqqt1FBt/event_proportion‚0 <"Purity/2FRt*SRqqt*FBt/event_purity‚0.0037686729338020086 :"Purity/2FRt*SRqqt*FBt/SRqqt_purity‚0.24550746381282806 @&Purity/3FRt2SRqqt0FBt/event_proportion‚0.0024571572430431843 8 Purity/*FRt2SRqqt2FBt/FRt_purity‚0.24358974397182465 ,&Purity/1FRt3SRqqt3FBt/event_proportion‚0 ,&Purity/4FRt3SRqqt0FBt/event_proportion‚0 +jet/accuracy_2_of_3‚0.15870389342308044 ("Purity/0FRt4SRqqt1FBt/event_purity‚0 7 Purity/0FRt2SRqqt2FBt/FBt_purity‚0.9615384340286255 =&Purity/*FRt1SRqqt1FBt/event_proportion‚0.1001134067773819 :"Purity/0FRt1SRqqt2FBt/SRqqt_purity‚0.37794971466064453 :"Purity/0FRt2SRqqt1FBt/event_purity‚0.04121863842010498 :"Purity/1FRt0SRqqt1FBt/event_purity‚0.14109766483306885 ,&Purity/4FRt2SRqqt2FBt/event_proportion‚0 ("Purity/*FRt4SRqqt0FBt/event_purity‚0 ("Purity/*FRt4SRqqt1FBt/SRqqt_purity‚0 ("Purity/0FRt3SRqqt1FBt/event_purity‚0 :"Purity/1FRt1SRqqt0FBt/SRqqt_purity‚0.21862050890922546 8 Purity/3FRt0SRqqt1FBt/FRt_purity‚0.33974358439445496 ,&Purity/4FRt1SRqqt4FBt/event_proportion‚0 ,&Purity/4FRt4SRqqt2FBt/event_proportion‚0 particle/accuracy_4_of_5‚1 +"Purity/4FRt*SRqqt0FBt/SRqqt_purity‚0.25 *"Purity/2FRt2SRqqt2FBt/SRqqt_purity‚0.5 ("Purity/1FRt*SRqqt4FBt/event_purity‚0 @&Purity/*FRt4SRqqt*FBt/event_proportion‚0.0013230846961960196 :"Purity/0FRt*SRqqt1FBt/SRqqt_purity‚0.37620046734809875 7 Purity/0FRt*SRqqt3FBt/FBt_purity‚0.8847736716270447 :"Purity/0FRt0SRqqt3FBt/event_purity‚0.07333333790302277 & Purity/1FRt3SRqqt1FBt/FBt_purity‚1 @&Purity/3FRt2SRqqt1FBt/event_proportion‚0.0003780241822823882 ("Purity/4FRt*SRqqt0FBt/event_purity‚0 & Purity/4FRt1SRqqt*FBt/FRt_purity‚0 ,&Purity/0FRt4SRqqt3FBt/event_proportion‚0 @&Purity/1FRt4SRqqt0FBt/event_proportion‚0.0003780241822823882 *jet/accuracy_2_of_5‚0.3339018225669861 epoch‚1 & Purity/1FRt3SRqqt*FBt/FBt_purity‚1 ,&Purity/4FRt3SRqqt*FBt/event_proportion‚0 ) Purity/*FRt1SRqqt4FBt/FBt_purity‚0.75 =&Purity/*FRt1SRqqt*FBt/event_proportion‚0.4482736885547638 ?&Purity/*FRt2SRqqt2FBt/event_proportion‚0.002079133177176118 @&Purity/1FRt4SRqqt*FBt/event_proportion‚0.0003780241822823882 @&Purity/4FRt*SRqqt*FBt/event_proportion‚0.0001890120911411941 & Purity/3FRt1SRqqt2FBt/FBt_purity‚1 :"Purity/*FRt0SRqqt*FBt/event_purity‚0.33299311995506287 ?&Purity/*FRt0SRqqt3FBt/event_proportion‚0.003717237850651145 ("Purity/1FRt0SRqqt4FBt/event_purity‚0 <"Purity/*FRt2SRqqt*FBt/event_purity‚0.0072134328074753284 9"Purity/*FRt2SRqqt*FBt/SRqqt_purity‚0.2629992365837097 9"Purity/0FRt*SRqqt*FBt/event_purity‚0.3600033223628998 9"Purity/0FRt*SRqqt*FBt/SRqqt_purity‚0.2823258340358734 :"Purity/1FRt*SRqqt1FBt/SRqqt_purity‚0.40322521328926086 8 Purity/1FRt3SRqqt1FBt/FRt_purity‚0.42222222685813904 ,&Purity/2FRt2SRqqt4FBt/event_proportion‚0 8 Purity/3FRt*SRqqt0FBt/FRt_purity‚0.11875661462545395 ?&Purity/0FRt3SRqqt*FBt/event_proportion‚0.006741431541740894 ?&Purity/2FRt0SRqqt1FBt/event_proportion‚0.008946572430431843 @&Purity/2FRt3SRqqt1FBt/event_proportion‚0.0003780241822823882 ,&Purity/4FRt1SRqqt2FBt/event_proportion‚0 ,&Purity/4FRt4SRqqt3FBt/event_proportion‚0 particle/accuracy_4_of_4‚1 7 Purity/2FRt4SRqqt*FBt/FRt_purity‚0.4000000059604645 8 Purity/3FRt*SRqqt2FBt/FRt_purity‚0.13333334028720856 >&Purity/*FRt2SRqqt0FBt/event_proportion‚0.10855595022439957 7 Purity/0FRt1SRqqt*FBt/FBt_purity‚0.9794159531593323 8 Purity/1FRt1SRqqt*FBt/FRt_purity‚0.18765881657600403 ;"Purity/2FRt0SRqqt*FBt/event_purity‚0.008695573545992374 8 Purity/2FRt2SRqqt0FBt/FRt_purity‚0.16001896560192108 & Purity/3FRt0SRqqt1FBt/FBt_purity‚1 ,&Purity/4FRt0SRqqt2FBt/event_proportion‚0 ("Purity/3FRt0SRqqt2FBt/event_purity‚0 8 Purity/*FRt1SRqqt0FBt/FRt_purity‚0.15339669585227966 8 Purity/*FRt1SRqqt2FBt/FRt_purity‚0.20927010476589203 :"Purity/0FRt1SRqqt1FBt/SRqqt_purity‚0.39746198058128357 9"Purity/1FRt*SRqqt2FBt/SRqqt_purity‚0.4495134949684143 ("Purity/3FRt2SRqqt*FBt/event_purity‚0 9"Purity/3FRt2SRqqt*FBt/SRqqt_purity‚0.3020833432674408 ,&Purity/4FRt*SRqqt2FBt/event_proportion‚0 * Purity/0FRt0SRqqt4FBt/FBt_purity‚0.875 >&Purity/*FRt*SRqqt1FBt/event_proportion‚0.27079132199287415 :"Purity/*FRt*SRqqt2FBt/event_purity‚0.18058794736862183 8 Purity/*FRt0SRqqt*FBt/FRt_purity‚0.16882309317588806 6 Purity/0FRt1SRqqt1FBt/FBt_purity‚0.987925112247467 7 Purity/2FRt1SRqqt2FBt/FBt_purity‚0.9868420958518982 & Purity/3FRt0SRqqt*FBt/FBt_purity‚1 ,&Purity/4FRt*SRqqt3FBt/event_proportion‚0 9"Purity/1FRt2SRqqt2FBt/SRqqt_purity‚0.2083333283662796 6 Purity/*FRt0SRqqt3FBt/FBt_purity‚0.915277898311615 7 Purity/1FRt*SRqqt2FBt/FBt_purity‚0.9738635420799255 ,&Purity/1FRt2SRqqt4FBt/event_proportion‚0 8 Purity/2FRt*SRqqt0FBt/FRt_purity‚0.13177835941314697 7 Purity/2FRt*SRqqt2FBt/FRt_purity‚0.3083333373069763 9"Purity/2FRt2SRqqt1FBt/SRqqt_purity‚0.3366071581840515 & Purity/*FRt1SRqqt4FBt/FRt_purity‚0 ("Purity/1FRt1SRqqt3FBt/event_purity‚0 8 Purity/*FRt1SRqqt1FBt/FRt_purity‚0.26268014311790466 7 Purity/*FRt2SRqqt*FBt/FBt_purity‚0.9855287075042725 @&Purity/0FRt2SRqqt2FBt/event_proportion‚0.0010080644860863686 7 Purity/1FRt0SRqqt2FBt/FBt_purity‚0.9810459017753601 @&Purity/2FRt2SRqqt2FBt/event_proportion‚6.300403038039804e-05 +jet/accuracy_3_of_5‚0.06469175964593887 7 Purity/1FRt1SRqqt3FBt/FBt_purity‚0.8333333730697632 ?&Purity/2FRt0SRqqt2FBt/event_proportion‚0.003465221729129553 7 Purity/2FRt2SRqqt1FBt/FRt_purity‚0.2440476268529892 7 Purity/3FRt2SRqqt*FBt/FRt_purity‚0.1597222089767456 ) Purity/*FRt1SRqqt3FBt/FRt_purity‚0.25 7 Purity/*FRt2SRqqt1FBt/FBt_purity‚0.9858918190002441 ("Purity/*FRt3SRqqt1FBt/event_purity‚0 ("Purity/0FRt0SRqqt1FBt/event_purity‚1 @&Purity/0FRt4SRqqt1FBt/event_proportion‚6.300403038039804e-05 7 Purity/2FRt*SRqqt1FBt/FRt_purity‚0.2925833761692047 (particle/f_score‚0.30156266689300537 ( Purity/2FRt2SRqqt2FBt/FRt_purity‚0.5 ?&Purity/*FRt1SRqqt2FBt/event_proportion‚0.017011089250445366 =&Purity/1FRt1SRqqt*FBt/event_proportion‚0.1723790317773819 ("Purity/3FRt0SRqqt*FBt/event_purity‚0 ,&Purity/3FRt2SRqqt2FBt/event_proportion‚0 ("Purity/*FRt4SRqqt*FBt/event_purity‚0 -"Purity/*FRt4SRqqt*FBt/SRqqt_purity‚0.1875 & Purity/2FRt*SRqqt3FBt/FRt_purity‚0 7 Purity/*FRt*SRqqt2FBt/FBt_purity‚0.9779426455497742 ,&Purity/*FRt2SRqqt4FBt/event_proportion‚0 >&Purity/1FRt*SRqqt1FBt/event_proportion‚0.09929435700178146 A&Purity/1FRt*SRqqt4FBt/event_proportion‚0.00012600806076079607 7 Purity/2FRt0SRqqt*FBt/FRt_purity‚0.1668718308210373 & Purity/1FRt2SRqqt2FBt/FBt_purity‚1 *"Purity/1FRt*SRqqt3FBt/SRqqt_purity‚0.5 % _timestamp‚1.7884154303018427e+09 & Purity/0FRt0SRqqt1FBt/FBt_purity‚1 ("Purity/1FRt2SRqqt1FBt/event_purity‚0 ,&Purity/2FRt4SRqqt4FBt/event_proportion‚0 @&Purity/3FRt2SRqqt*FBt/event_proportion‚0.0028351815417408943 ,&Purity/4FRt2SRqqt3FBt/event_proportion‚0 & Purity/2FRt0SRqqt3FBt/FBt_purity‚1 8"Purity/0FRt1SRqqt3FBt/SRqqt_purity‚0.800000011920929 ;"Purity/*FRt2SRqqt1FBt/event_purity‚0.016856711357831955 ("Purity/2FRt2SRqqt*FBt/event_purity‚0 :"Purity/2FRt2SRqqt*FBt/SRqqt_purity‚0.23546189069747925 ,&Purity/4FRt*SRqqt1FBt/event_proportion‚0 7 Purity/0FRt0SRqqt*FBt/FBt_purity‚0.9879888296127319 8 Purity/1FRt3SRqqt0FBt/FRt_purity‚0.18740400671958923 ,&Purity/2FRt4SRqqt1FBt/event_proportion‚0 8 Purity/3FRt*SRqqt1FBt/FRt_purity‚0.23106060922145844 particle/accuracy_1_of_3‚1 +jet/accuracy_3_of_6‚0.12280701100826263 9"Purity/*FRt*SRqqt3FBt/SRqqt_purity‚0.7142857313156128 & Purity/1FRt3SRqqt2FBt/FRt_purity‚0 ,&Purity/*FRt3SRqqt4FBt/event_proportion‚0 @&Purity/0FRt*SRqqt3FBt/event_proportion‚0.0030871976632624865 9"Purity/0FRt1SRqqt0FBt/SRqqt_purity‚0.2576642334461212 8 Purity/2FRt3SRqqt*FBt/FRt_purity‚0.20514285564422607 particle/accuracy_2_of_4‚1 jet/accuracy_4_of_6‚0 ("Purity/*FRt2SRqqt3FBt/SRqqt_purity‚1 & Purity/4FRt2SRqqt0FBt/FRt_purity‚0 >&Purity/1FRt2SRqqt0FBt/event_proportion‚0.04731602966785431 :"Purity/2FRt1SRqqt0FBt/SRqqt_purity‚0.20714537799358368 9"Purity/1FRt3SRqqt2FBt/SRqqt_purity‚0.6666666865348816 ,&Purity/2FRt1SRqqt4FBt/event_proportion‚0 ,&Purity/3FRt1SRqqt3FBt/event_proportion‚0 9"Purity/*FRt*SRqqt4FBt/event_purity‚0.1666666716337204 ("Purity/3FRt3SRqqt0FBt/event_purity‚0 _step‚97 ?&Purity/1FRt*SRqqt2FBt/event_proportion‚0.022303426638245583 ;"Purity/1FRt1SRqqt1FBt/event_purity‚0.013380973599851131 *jet/accuracy_1_of_5‚0.8874148726463318 *"Purity/1FRt1SRqqt3FBt/SRqqt_purity‚0.5 & Purity/4FRt*SRqqt0FBt/FRt_purity‚0 ;"Purity/*FRt1SRqqt2FBt/event_purity‚0.009677419438958168 ?&Purity/0FRt0SRqqt3FBt/event_proportion‚0.002583165420219302 ?&Purity/2FRt1SRqqt2FBt/event_proportion‚0.001890120911411941 @&Purity/3FRt3SRqqt*FBt/event_proportion‚0.0005670362734235823 @&Purity/4FRt0SRqqt0FBt/event_proportion‚6.300403038039804e-05 8 Purity/*FRt3SRqqt*FBt/FRt_purity‚0.20521655678749084 ,&Purity/4FRt0SRqqt1FBt/event_proportion‚0 :"Purity/*FRt0SRqqt4FBt/event_purity‚0.20000000298023224 :"Purity/0FRt2SRqqt0FBt/event_purity‚0.01410691812634468 ,&Purity/1FRt2SRqqt3FBt/event_proportion‚0 ,&Purity/2FRt2SRqqt3FBt/event_proportion‚0 ,&Purity/3FRt2SRqqt4FBt/event_proportion‚0 ("Purity/4FRt*SRqqt*FBt/event_purity‚0 +"Purity/4FRt*SRqqt*FBt/SRqqt_purity‚0.25 >&Purity/0FRt1SRqqt1FBt/event_proportion‚0.05185231938958168 ,&Purity/2FRt3SRqqt3FBt/event_proportion‚0 A&Purity/3FRt1SRqqt2FBt/event_proportion‚0.00025201612152159214 7 Purity/0FRt2SRqqt3FBt/FBt_purity‚0.6666666865348816 9"Purity/0FRt2SRqqt0FBt/SRqqt_purity‚0.2546229660511017 9"Purity/1FRt1SRqqt2FBt/SRqqt_purity‚0.4793367087841034 ("Purity/0FRt2SRqqt2FBt/event_purity‚0 >&Purity/0FRt*SRqqt1FBt/event_proportion‚0.14787046611309052 :"Purity/2FRt3SRqqt1FBt/SRqqt_purity‚0.20000000298023224 :"Purity/*FRt1SRqqt1FBt/event_purity‚0.06666264683008194 ?&Purity/*FRt3SRqqt1FBt/event_proportion‚0.002142137149348855 7 Purity/0FRt*SRqqt2FBt/FBt_purity‚0.9786064624786377 >&Purity/1FRt0SRqqt*FBt/event_proportion‚0.19745463132858276 ("Purity/2FRt1SRqqt0FBt/event_purity‚0 ,&Purity/2FRt1SRqqt3FBt/event_proportion‚0 ,&Purity/3FRt3SRqqt3FBt/event_proportion‚0 :"Purity/*FRt*SRqqt3FBt/event_purity‚0.05287356302142143 9"Purity/0FRt1SRqqt0FBt/event_purity‚0.2576642334461212 ,&Purity/1FRt4SRqqt3FBt/event_proportion‚0 ("Purity/2FRt1SRqqt1FBt/event_purity‚0 ("Purity/2FRt1SRqqt2FBt/event_purity‚0 ("Purity/*FRt2SRqqt3FBt/event_purity‚0 :"Purity/*FRt0SRqqt3FBt/event_purity‚0.05476190522313118 ,&Purity/0FRt3SRqqt2FBt/event_proportion‚0 ("Purity/1FRt*SRqqt3FBt/event_purity‚0 jet/accuracy_1_of_7‚0.5 ("Purity/*FRt3SRqqt*FBt/event_purity‚0 :"Purity/*FRt3SRqqt*FBt/SRqqt_purity‚0.23684664070606232 particle/accuracy_3_of_6‚1 +"Purity/3FRt1SRqqt2FBt/SRqqt_purity‚0.25 ("Purity/0FRt1SRqqt3FBt/event_purity‚0 8"Purity/3FRt3SRqqt0FBt/SRqqt_purity‚0.261904776096344a9ë6÷ž« ‚(:“ ΛäÔϯ© gpu.0.gpu‚44.0 gpu.0.memory‚4, gpu.0.memoryAllocated‚12.165603637695312* gpu.0.memoryAllocatedBytes‚ 10450173952 gpu.0.temp‚35.0 gpu.0.powerWatts‚161.457' gpu.0.enforcedPowerLimitWatts‚500.0) gpu.0.powerPercent‚32.291399999999996 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0½ôõM¬ ‚(: ΛäÔä¾Ôª disk.drbd2265.out‚0) disk./.usagePercent‚16.81807447189537 disk.sda.out‚39.96875 disk.rbd0.in‚0 disk.drbd2266.out‚0! disk.nvme0c0n1.in‚ 11.80859375 disk.dm-0.in‚ 11.80859375 disk.sda3.in‚0 disk.nvme0n1.in‚ 11.80859375 disk.sda2.out‚0 disk.drbd2265.in‚0 disk.rbd1.in‚0' disk.nvme0c0n1.out‚3863.64892578125 disk.drbd2080.in‚0) proc.memory.availableMB‚ 2411.23828125! proc.memory.rssMB‚ 2801.046875 network.recv‚432430 disk.sda.in‚0 disk.sda3.out‚39.96875 disk.dm-1.out‚39.96875 disk.drbd2266.in‚0" disk.dm-0.out‚3863.77392578125$ memory_percent‚85.28296947479248 disk.rbd1.out‚ 1.533203125 disk.drbd2080.out‚0$ disk./.usageGB‚250.6014633178711 disk.sda1.out‚0 network.sent‚ 2.153497e+06 disk.drbd1716.out‚0 disk.rbd0.out‚0% disk.nvme0n1.out‚3863.64892578125 disk.sda1.in‚0* proc.memory.percent‚17.096233367919922 cpu‚26.240188991973344 proc.cpu.threads‚17 disk.dm-1.in‚0 disk.drbd1716.in‚0 disk.sda2.in‚0]^"˜ž­ ‚(:“ Ý›äÔþ°­¨ gpu.0.gpu‚53.0 gpu.0.memory‚6, gpu.0.memoryAllocated‚12.165603637695312* gpu.0.memoryAllocatedBytes‚ 10450173952 gpu.0.temp‚35.0 gpu.0.powerWatts‚119.721' gpu.0.enforcedPowerLimitWatts‚500.0) gpu.0.powerPercent‚23.944200000000002 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0á52‚V® ‚(:Ë Ý›äÔ顯« network.sent‚ 2.25602e+06 disk.rbd0.out‚0) proc.memory.availableMB‚ 2411.21484375$ disk./.usageGB‚250.6096420288086 disk.dm-0.in‚ 11.80859375 disk.sda.out‚ 40.45703125% disk.nvme0n1.out‚4027.01611328125 disk.sda1.out‚0! disk.nvme0c0n1.in‚ 11.80859375 disk.dm-1.out‚ 40.45703125$ memory_percent‚85.28311252593994* disk./.usagePercent‚16.818623352048782 disk.drbd2080.in‚0 disk.nvme0n1.in‚ 11.80859375 disk.sda2.in‚0 disk.drbd2266.in‚0 disk.drbd2266.out‚0 disk.rbd1.in‚0 disk.sda.in‚0 disk.drbd2265.in‚0 disk.drbd1716.out‚0 proc.cpu.threads‚17 network.recv‚438215 disk.drbd2080.out‚0 disk.drbd2265.out‚0' disk.nvme0c0n1.out‚4027.01611328125* proc.memory.percent‚17.096233367919922 disk.rbd0.in‚0 disk.sda1.in‚0" disk.dm-0.out‚4027.14111328125 disk.sda3.in‚0 cpu‚26.232973988705094 disk.sda2.out‚0 disk.rbd1.out‚ 1.533203125 disk.sda3.out‚ 40.45703125 disk.dm-1.in‚0 disk.drbd1716.in‚0! proc.memory.rssMB‚ 2801.046875áPT[2¯ - _wandbruntime‚1046  _runtime‚1046xzMâÆ° À trainer/global_step‚1249 % _timestamp‚1.7884154652061017e+09 _runtime‚1046.855463473 _step‚98 & lr-AdamW/pg1‚0.0018102232688846586 & lr-AdamW/pg2‚0.0018102232688846586ÆðAˆÊ± Ä & lr-AdamW/pg1‚0.0018102232688846586 & lr-AdamW/pg2‚0.0018102232688846586 trainer/global_step‚1249 % _timestamp‚1.7884154652061017e+09 _runtime‚1046.855463473 _step‚98bx§½2² - _wandbruntime‚1046  _runtime‚1046zd³ª¼'³ ¶O OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚20037 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 \IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚71581801062912 IDeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚2605 VGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 9283702784 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚20039 M@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚17172331 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚10928396 XEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚70514796628857 VGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 8980994048 JBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚431 XJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 165393016 YHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 937075400704 PCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚18178608 TGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚23656111 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚24127 UFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 9456058368 N@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 186438144 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 186438144 `MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚72342138791936 TADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚70514614196481 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚20039 PCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚18178610 SGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚6243938 % _timestamp‚1.7884154652072554e+09 dQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚72342163564544 JBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚213 PCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚23636074 ]JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚73318783588864 BDeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 XEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚71581783892480 XEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚72519045731328 O@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 9373790789 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 E=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚644 VEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 937075400704 TADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚72518859293184 _LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚71581801062912 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚24772608 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 QCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 182432376 N?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 9902751744 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 PCDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated‚23656111 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚24127 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 226700800 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 144203776 YHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 932081910017 ZIDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 937244668416 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 169267712 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚10928393 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚6244402 L?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚18178608 Q>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚72518859293184 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 435025920 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 1692677128=š¦=´ ºO XIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 9456058368 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚17170432 M@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚41834721 TGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚18178610 ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 201928192 @8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚644 _LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚69582549325824 TFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 446693376 L=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 9679899136 H?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚1003 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 D>DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 XEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚71581783892480 XEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚72519045731328 O@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 9373790789 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 E=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚644 VEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 937075400704 TADeviceStatsMonitor.on_train_batch_sSÕÔ?ƒ!tart/allocated_bytes.all.freed‚72518859293184 _LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚71581801062912 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚24772608 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 QCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 182432376 N?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 9902751744 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 PCDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated‚23656111 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚24127 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 226700800 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 144203776 YHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 932081910017 ZIDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 937244668416 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 169267712 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚10928393 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚6244402 L?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚18178608 Q>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚72518859293184 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 435025920 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 169267712 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚20037 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 \IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚71581801062912 IDeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚2605 VGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 9283702784 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚20039 M@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚17172331 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚10928396 XEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚70514796628857 VGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 8980994048 JBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚431 XJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 165393016 YHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 937075400704 PCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚18178608 TGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚23656111 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚24127 UFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 9456058368 N@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 186438144 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 186438144 `MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚72342138791936 TADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚70514614196481 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚20039 PCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚18178610 SGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚6243938 % _timestamp‚1.7884154652072554e+09 dQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚72342163564544 JBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚213 PCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚23636074 ]JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚73318783588864 BDeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚17185518 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚213 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚20065 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚17170432 [HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚73375488784384 bODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚72398098003968 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚17039360 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24772608 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 R?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚70569200310325 YFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚69636396953600 EDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 9679899136 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚644 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 VHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 165393224 ]JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚69636413992960 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 169281024 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚24400 ]JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚71637164752384 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 237173248 % _timestamp‚1.7884154659194314e+09 QBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 9283702784 OBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚17185999 TEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 8980994048 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 431144269 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚431 VCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚71637147581952 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚431 _step‚100 XIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 9456058368 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚213 S@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚72575134788096 R?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚72574948336640 WFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 937800754688 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 182432584 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17170432 VCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚72575134788096 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚24127 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚35 VCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚70569382742909 SDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 9456058368 C6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚41847154 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 144203776 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 J;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 9679899136 ZGDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚71637164752384 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚6249212 BDeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 186451456 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 NADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚23674609 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚213 _runtime‚1047.569279822 L=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 9902751744 J=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚23654546 YFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚71637147581952 G:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚41847154 REDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚23674609 NADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚23654546 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚10936787 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚431 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 169281024 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1003 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 O@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 9902751744 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚644 NADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚18192610 OADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 186451456 TEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 9283702784 SEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 435025920 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 K>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚41867219 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚24400 REDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚18192610 WDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚73375251611136 XGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 937970035712 [JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 937970035712 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚481 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚6248734 M>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 9373790789 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 FDeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 186451456 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 NADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚23674609 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚213 _runtime‚1047.569279822 L=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 9902751744 J=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚23654546 YFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚71637147581952 G:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚41847154 REDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚23674609 NADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚23654546 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚10936787 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚431 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 169281024 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1003 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 O@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 9902751744 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚644 NADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚18192610 OADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 186451456 TEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 9283702784 SEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 435025920 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 K>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚41867219 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚24400 REDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚18192610 WDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚73375251611136 XGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 937970035712 [JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 937970035712 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚481 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚6248734 M>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 9373790789 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 FDeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚17185518 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚213 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚20065 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚17170432 [HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚73375488784384 bODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚72398098003968 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚17039360 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24772608 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 R?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚70569200310325 YFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚69636396953600 EDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 9679899136 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚644 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 VHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 165393224 ]JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚69636413992960 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 169281024 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚24400 ]JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚71637164752384 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 237173248 % _timestamp‚1.7884154659194314e+09 QBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 9283702784 OBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚17185999 TEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 8980994048 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 431144269 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚431 VCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚71637147581952 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚431 _step‚100 XIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 9456058368 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚213 S@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚72575134788096 R?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚72574948336640 WFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 937800754688d·FáË2¸ - _wandbruntime‚1047  _runtime‚1047¹Vð‹U¹ Ï .loss/FRt3/assignment_loss‚4.03755929349344 2loss/SRqqt1/assignment_loss‚1.8731811031839758 1loss/SRqqt1/detection_loss‚1.9259133545576401 .loss/FBt2/detection_loss‚2.747490224085356 _runtime‚1047.570892335 /loss/FRt2/assignment_loss‚4.092305050769323 %loss/total_loss‚48.35155959929419 _step‚101 1loss/FBt4/assignment_loss‚0.09395789958304501 .loss/FRt1/detection_loss‚2.148662648951778 /loss/FRt4/detection_loss‚1.7965436627467473 .loss/FBt1/detection_loss‚2.458869156512347 2loss/SRqqt3/assignment_loss‚2.0917275312874053 1loss/FBt3/assignment_loss‚0.15581089450756735 0loss/SRqqt3/detection_loss‚2.135873701506191 1loss/SRqqt4/detection_loss‚1.2153088838086092 trainer/global_step‚1249 /loss/FBt1/assignment_loss‚0.536768156033813 0loss/FBt2/assignment_loss‚0.1832534014120986 /loss/FRt2/detection_loss‚2.5017990268452066 /loss/FRt3/detection_loss‚2.7339224309860906 /loss/FBt4/detection_loss‚0.6604436434360232 2loss/SRqqt4/assignment_loss‚1.4973472844725264 .loss/FBt3/detection_loss‚2.726257361471653 1loss/SRqqt2/assignment_loss‚2.235279903761924 0loss/SRqqt2/detection_loss‚2.587527320025459 epoch‚2 /loss/FRt1/assignment_loss‚3.770785724887481 0loss/FRt4/assignment_loss‚2.1449719409684875 % _timestamp‚1.7884154659216592e+095ÌoàYº Ó %loss/total_loss‚48.35155959929419 _step‚101 1loss/FBt4/assignment_loss‚0.09395789958304501 .loss/FRt1/detection_loss‚2.148662648951778 /loss/FRt4/detection_loss‚1.7965436627467473 .loss/FBt1/detection_loss‚2.458869156512347 2loss/SRqqt3/assignment_loss‚2.0917275312874053 1loss/FBt3/assignment_loss‚0.15581089450756735 0loss/SRqqt3/detection_loss‚2.135873701506191 1loss/SRqqt4/detection_loss‚1.2153088838086092 trainer/global_step‚1249 /loss/FBt1/assignment_loss‚0.536768156033813 0loss/FBt2/assignment_loss‚0.1832534014120986 /loss/FRt2/detection_loss‚2.5017990268452066 /loss/FRt3/detection_loss‚2.7339224309860906 /loss/FBt4/detection_loss‚0.6604436434360232 2loss/SRqqt4/assignment_loss‚1.4973472844725264 .loss/FBt3/detection_loss‚2.726257361471653 1loss/SRqqt2/assignment_loss‚2.235279903761924 0loss/SRqqt2/detection_loss‚2.587527320025459 epoch‚2 /loss/FRt1/assignment_loss‚3.770785724887481 0loss/FRt4/assignment_loss‚2.1449719409684875 % _timestamp‚1.7884154659216592e+09 .loss/FRt3/assignment_loss‚4.03755929349344 2loss/SRqqt1/assignment_loss‚1.8731811031839758 1loss/SRqqt1/detection_loss‚1.9259133545576401 .loss/FBt2/detection_loss‚2.747490224085356 _runtime‚1047.570892335 /loss/FRt2/assignment_loss‚4.092305050769323eS ϵ“» ‚(:ˆ ì›äÔßÙ¹¨ gpu.0.gpu‚22.0 gpu.0.memory‚3, gpu.0.memoryAllocated‚12.165603637695312* gpu.0.memoryAllocatedBytes‚ 10450173952 gpu.0.temp‚36.0 gpu.0.powerWatts‚178.447' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚35.6894 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0\LH¼ ‚(:É ì›äԙͶ«( proc.memory.availableMB‚ 2411.1640625! proc.memory.rssMB‚ 2801.046875 disk.drbd2080.out‚0 disk.rbd0.in‚0 disk.nvme0n1.in‚ 11.80859375" disk.dm-0.out‚4074.25732421875 disk.sda.in‚0 disk.sda2.out‚0 disk.dm-1.in‚0 proc.cpu.threads‚17 disk.sda3.out‚ 41.1953125! disk.nvme0c0n1.in‚ 11.80859375 disk.drbd2265.out‚0 disk.rbd1.out‚ 1.533203125 disk.dm-1.out‚ 41.1953125 disk.drbd2080.in‚0 disk.drbd2266.in‚0 disk.drbd2265.in‚0' disk.nvme0c0n1.out‚4074.13232421875 disk.sda.out‚ 41.1953125$ memory_percent‚85.28342247009277* disk./.usagePercent‚16.818045031028934% disk./.usageGB‚250.60102462768555 disk.drbd2266.out‚0 disk.rbd1.in‚0 disk.sda3.in‚0 disk.drbd1716.out‚0* proc.memory.percent‚17.096233367919922 cpu‚26.225760449723392 disk.sda1.in‚0 disk.sda1.out‚0 disk.sda2.in‚0 network.sent‚ 2.261438e+06 network.recv‚441607 disk.rbd0.out‚0% disk.nvme0n19û`¨M.out‚4074.13232421875 disk.drbd1716.in‚0 disk.dm-0.in‚ 11.80859375=LòMž½ ‚(:“ û›äÔ˜÷§ gpu.0.gpu‚52.0 gpu.0.memory‚6, gpu.0.memoryAllocated‚12.165603637695312* gpu.0.memoryAllocatedBytes‚ 10450173952 gpu.0.temp‚36.0 gpu.0.powerWatts‚148.001' gpu.0.enforcedPowerLimitWatts‚500.0) gpu.0.powerPercent‚29.600199999999997 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0d“Í=S¾ ‚(:È û›äÔ´—« disk.drbd1716.out‚0 disk.dm-0.in‚ 11.80859375 disk.sda.in‚0 disk.sda3.in‚0 disk.dm-1.in‚0 network.sent‚ 2.349158e+06* disk./.usagePercent‚16.818939521353567% disk.nvme0n1.out‚4114.21435546875 disk.sda1.in‚0$ memory_percent‚85.28366088867188 proc.cpu.threads‚17% disk./.usageGB‚250.61435317993164 disk.drbd2265.in‚0" disk.dm-0.out‚4114.33935546875 disk.sda2.out‚0 disk.dm-1.out‚ 41.62890625 disk.rbd0.in‚0! proc.memory.rssMB‚ 2801.046875 disk.drbd2080.out‚0 disk.drbd2266.out‚0 disk.rbd1.in‚0 disk.rbd1.out‚ 1.533203125! disk.nvme0c0n1.in‚ 11.80859375 disk.sda.out‚ 41.62890625 cpu‚26.218533910392686 network.recv‚448200 disk.sda2.in‚0 disk.drbd2266.in‚0 disk.drbd2265.out‚0' disk.nvme0c0n1.out‚4114.21435546875 disk.drbd2080.in‚0 disk.rbd0.out‚0$ proc.memory.availableMB‚2411.125 disk.sda3.out‚ 41.62890625 disk.drbd1716.in‚0 disk.nvme0n1.in‚ 11.80859375 disk.sda1.out‚0* proc.memory.percent‚17.096233367919922”Ãç“¿ ‚(:ˆ ŠœäÔŸˆ¨ gpu.0.gpu‚52.0 gpu.0.memory‚6, gpu.0.memoryAllocated‚12.165603637695312* gpu.0.memoryAllocatedBytes‚ 10450173952 gpu.0.temp‚35.0 gpu.0.powerWatts‚169.193' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚33.8386 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0Œ”äDUÀ ‚(:Ê ŠœäÔ놪 network.sent‚ 2.35539e+06 disk.rbd0.in‚0 disk.sda1.out‚0 disk.sda2.in‚0 disk.drbd2265.in‚0 network.recv‚452807 disk.dm-0.in‚ 11.80859375 disk.sda3.in‚0) proc.memory.availableMB‚ 2411.37109375* proc.memory.percent‚17.096233367919922 cpu‚26.21131152040641 disk.sda3.out‚ 42.23828125 disk.drbd1716.out‚0 disk.drbd2266.in‚0% disk./.usageGB‚250.63279724121094 disk.dm-1.out‚ 42.23828125 disk.drbd1716.in‚0 disk.rbd0.out‚0) disk./.usagePercent‚16.82017731778162" disk.dm-0.out‚4184.22412109375 disk.sda.out‚ 42.23828125 disk.dm-1.in‚0 disk.nvme0n1.in‚ 11.80859375 disk.sda1.in‚0 disk.drbd2080.in‚0 disk.drbd2265.out‚0 disk.rbd1.in‚0 disk.drbd2080.out‚0% disk.nvme0n1.out‚4184.09912109375 disk.drbd2266.out‚0 disk.rbd1.out‚ 1.533203125! disk.nvme0c0n1.in‚ 11.80859375' disk.nvme0c0n1.out‚4184.09912109375$ memory_percent‚85.28215885162354 proc.cpu.threads‚17 disk.sda.in‚0 disk.sda2.out‚0! proc.memory.rssMB‚ 2801.046875ZT]2Á - _wandbruntime‚1086  _runtime‚1086rÑ_Ç Á & lr-AdamW/pg1‚0.0017999964605081257 & lr-AdamW/pg2‚0.0017999964605081257 trainer/global_step‚1299 % _timestamp‚1.7884155045684114e+09 _runtime‚1086.217787333 _step‚102w SËà Å % _timestamp‚1.7884155045684114e+09 _runtime‚1086.217787333 _step‚102 & lr-AdamW/pg1‚0.0017999964605081257 & lr-AdamW/pg2‚0.0017999964605081257 trainer/global_step‚1299fÆÍöÎ2Ä - _wandbruntime‚1086  _runtime‚1086Ü­=¿'Å ¹O SGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚6481061 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚24772608 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 PCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚18878610 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 435025920 TADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚73243947060689 O@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 9373790789 SEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 313315328 XIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 9456058368 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚20037 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚17170432 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚463 YHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 973370928128 M@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚17829004 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 H?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚1003 TADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚75323339294208 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 ZKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 9456058368 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚213 trainer/global_step‚1299 G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚2605 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 144203776 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 E8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚43441634 PCDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated‚24583063 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 UFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 9456058368 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 G?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚213 QCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 182432376 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚431 NFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚985 _LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚72275782682624 SDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 9902751744 M@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚43461673 @8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚644 BDeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 YHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 968181417425 N?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 9902751744 L?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚24563026 \IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚74349985536512 VEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 973370928128 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚24400 _MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 1014560324608 C;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚644 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 UGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 431144269 MFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚35 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 235089408 XEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚75323525732352 [HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚74349968366080 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚460 YFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚76153421087744 dQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚75138885535744 ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 210316800 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 248172032 ]LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 968346810441 L=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 9679899136 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 186438144 VGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 9283702784 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚3 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 E=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚644 JBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚431 _runtime‚1086.222863564 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚24127 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 169267712 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 446693376 TFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 446693376 L?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚18878608 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚20039 TGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚18878610 XEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚73244129493065 PCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚18878608 PCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚24563026 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 G?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚431 `MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚75138860763136 N@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 186438144 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚17170432 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚6481521 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 169267712 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚2 UBDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated‚75323525732352 _LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚74349985536512 [HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚72275765643264 XJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 165393016 % _timestamp‚1.7884155045694106e+09 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 435025920 ]JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚76153656177152 JBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚213 SDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 9283702784 ZIDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 973540195840 QDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚17829467 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 Q>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚75323339294208 ]LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 973540195840 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚2605 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 IDeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚2605 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 144203776 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 E8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚43441634 PCDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated‚24583063 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 UFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 9456058368 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 G?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚213 QCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 182432376 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚431 NFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚985 _LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚72275782682624 SDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 9902751744 M@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚43461673 @8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚644 BDeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 YHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 968181417425 N?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 9902751744 L?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚24563026 \IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚74349985536512 VEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 973370928128 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚24400 _MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 1014560324608 C;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚644 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 UGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 431144269 MFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚35 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 235089408 XEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚75323525732352 [HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚74349968366080 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚460 YFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚76153421087744 dQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚75138885535744 ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 210316800 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 248172032 ]LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 968346810441 L=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 9679899136 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 186438144 VGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 9283702784 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚3 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 E=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚644 JBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚431 _runtime‚1086.222863564 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚24127 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 169267712 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 446693376 TFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 446693376 L?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚18878608 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚20039 TGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚18878610 XEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚73244129493065 PCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚18878608 PCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚24563026 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 G?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚431 `MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚75138860763136 N@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 186438144 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚17170432 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚6481521 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 169267712 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚2 UBDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated‚75323525732352 _LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚74349985536512 [HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚72275765643264 XJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 165393016 % _timestamp‚1.7884155045694106e+09 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 435025920 ]JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚76153656177152 JBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚213 SDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 9283702784 ZIDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 973540195840 QDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚17829467 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 Q>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚75323339294208 ]LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 973540195840 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚2605 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 IDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 9373790789 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24772608 FDeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 K>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚43494171 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1003 BDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 NADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚18892610 SEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 435025920 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚213 OADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 186451456 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 169281024 ]KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 1015318795776 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 J=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚24581498 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚35 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 [JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 969068257357 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚20063 REDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚18892610 _step‚104 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚20065 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚2 [MDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚ 218692096 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 VHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 165393224 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚213 trainer/global_step‚1299 G:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚43494171 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17170432 S@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚75379614789120 R?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚75379428337664 YFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚74405332055552 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚17039360 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 NADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚24601561 J;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 9679899136 K>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚17842171 XIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 9456058368 TCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 974096282112 SDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 9456058368 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚644 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚24127 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚644 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 VGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 9456058368 % _timestamp‚1.7884155052529163e+09 PBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 435025920 M>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 9679899136 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 O6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚644 WFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 974096282112 G:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚43474106 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚24400 [HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚76210357463040 VCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚73298715607117 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚18892608 bODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚75194819975168 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚11356337 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 WIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 446693376 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚476 O@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 9902751744 ZGDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚74405349225984 QBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 9283702784 FDeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 186451456 [JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 974265563136 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 243464704 YFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚72329630310400 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚431 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚431 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚3 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚17170432 TEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 9283702784 C6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚43474106 ]JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚74405349225984 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚431 _runtime‚1086.902694254 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚24400 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 169281024 TEDeviceStálJôatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 8980994048 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 R?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚73298533174533 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚6486313 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 446693376 VCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚75379614789120 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚2605 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚6485837 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 446693376 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 182432584 ]JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚72329647349760 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚213 NADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚24581498 QBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 9902751744hG/GÍ&É ÇM O6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚644 WFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 974096282112 G:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚43474106 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚24400 [HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚76210357463040 VCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚73298715607117 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚18892608 bODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚75194819975168 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚11356337 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 WIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 446693376 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚476 O@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 9902751744 ZGDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚74405349225984 QBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 9283702784 FDeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 186451456 [JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 974265563136 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 243464704 YFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚72329630310400 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚431 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚431 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚3 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚17170432 TEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 9283702784 C6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚43474106 ]JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚74405349225984 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚431 _runtime‚1086.902694254 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚24400 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 169281024 TEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 8980994048 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 R?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚73298533174533 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚6486313 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 446693376 VCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚75379614789120 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚2605 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚6485837 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 446693376 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 182432584 ]JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚72329647349760 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚213 NADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚24581498 QBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 9902751744 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚479 VCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚74405332055552 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚20063 M>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 9373790789 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24772608 FDeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 K>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚43494171 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1003 BDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 NADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚18892610 SEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 435025920 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚213 OADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 186451456 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 169281024 ]KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 1015318795776 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 J=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚24581498 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚35 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 [JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 969068257357 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚20063 REDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚18892610 _step‚104 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚20065 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚2 [MDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚ 218692096 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 VHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 165393224 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚213 trainer/global_step‚1299 G:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚43494171 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17170432 S@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚75379614789120 R?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚75379428337664 YFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚74405332055552 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚17039360 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 NADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚24601561 J;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 9679899136 K>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚17842171 XIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 9456058368 TCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 974096282112 SDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 9456058368 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚644 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚24127 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚644 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 VGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 9456058368 % _timestamp‚1.7884155052529163e+09 PBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 435025920 M>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 9679899136 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0h0N#ú2Ê - _wandbruntime‚1086  _runtime‚1086x²_Ë Ù 1loss/SRqqt1/detection_loss‚1.9949384853967185 % _timestamp‚1.7884155052544668e+09 _step‚105 2loss/SRqqt1/assignment_loss‚2.2199869158777217 2loss/SRqqt2/assignment_loss‚2.2889813538910686 2loss/FBt4/assignment_loss‚0.058843087249482305 /loss/FRt1/detection_loss‚2.2747080117622587 /loss/FBt3/detection_loss‚2.0354560797031107 epoch‚2 /loss/FRt2/assignment_loss‚4.323300685201373 1loss/FBt3/assignment_loss‚0.17901777940638708 .loss/FRt4/detection_loss‚1.925733466394419 1loss/SRqqt2/detection_loss‚2.4755003849665322 1loss/SRqqt3/detection_loss‚2.1143696150001214 0loss/SRqqt4/detection_loss‚1.191651904762819 1loss/FBt1/assignment_loss‚0.25376852659974247 /loss/FBt1/detection_loss‚2.3124047617117562 /loss/FBt4/detection_loss‚0.8071864844294068 0loss/FRt1/assignment_loss‚3.9733617331119295 /loss/FRt3/detection_loss‚2.7770085378487903 /loss/FRt2/detection_loss‚2.5912199128757822 /loss/FBt2/detection_loss‚3.2066082805395126 trainer/global_step‚1299 _runtime‚1086.903687348 0loss/FRt4/assignment_loss‚2.3481212299923566 %loss/total_loss‚48.97552523748315 .loss/FRt3/assignment_loss‚3.91070637067159 2loss/SRqqt3/assignment_loss‚1.8508858603178238 2loss/SRqqt4/assignment_loss‚1.5843770744214798 1loss/FBt2/assignment_loss‚0.27738869535096455?ÆûcÌ Ý _runtime‚1086.903687348 0loss/FRt4/assignment_loss‚2.3481212299923566 %loss/total_loss‚48.97552523748315 .loss/FRt3/assignment_loss‚3.91070637067159 2loss/SRqqt3/assignment_loss‚1.8508858603178238 2loss/SRqqt4/assignment_loss‚1.5843770744214798 1loss/FBt2/assignment_loss‚0.27738869535096455 1loss/SRqqt1/detection_loss‚1.9949384853967185 % _timestamp‚1.7884155052544668e+09 _step‚105 2loss/SRqqt1/assignment_loss‚2.2199869158777217 2loss/SRqqt2/assignment_loss‚2.2889813538910686 2loss/FBt4/assignment_loss‚0.058843087249482305 /loss/FRt1/detection_loss‚2.2747080117622587 /loss/FBt3/detection_loss‚2.0354560797031107 epoch‚2 /loss/FRt2/assignment_loss‚4.323300685201373 1loss/FBt3/assignment_loss‚0.17901777940638708 .loss/FRt4/detection_loss‚1.925733466394419 1loss/SRqqt2/detection_loss‚2.4755003849665322 1loss/SRqqt3/detection_loss‚2.1143696150001214 0loss/SRqqt4/detection_loss‚1.191651904762819 1loss/FBt1/assignment_loss‚0.25376852659974247 /loss/FBt1/detection_loss‚2.3124047617117562 /loss/FBt4/detection_loss‚0.8071864844294068 0loss/FRt1/assignment_loss‚3.9733617331119295 /loss/FRt3/detection_loss‚2.7770085378487903 /loss/FRt2/detection_loss‚2.5912199128757822 /loss/FBt2/detection_loss‚3.2066082805395126 trainer/global_step‚1299i1­Í ‚(:’ ™œäÔàϦ¨ gpu.0.gpu‚49.0 gpu.0.memory‚5, gpu.0.memoryAllocated‚12.165603637695312* gpu.0.memoryAllocatedBytes‚ 10450173952 gpu.0.temp‚35.0 gpu.0.powerWatts‚119.86' gpu.0.enforcedPowerLimitWatts‚500.0) gpu.0.powerPercent‚23.971999999999998 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚02…9OVÎ ‚(:Ë ™œäÔÙŠüª% disk./.usageGB‚250.66438674926758 disk.rbd0.out‚0% disk.nvme0n1.out‚4247.25146484375 disk.sda1.in‚0 disk.sda1.out‚0 disk.drbd2266.in‚0' disk.nvme0c0n1.out‚4247.25146484375* disk./.usagePercent‚16.822297316172662 disk.rbd1.in‚0 network.sent‚ 2.360817e+06 disk.drbd2265.in‚0 disk.dm-1.in‚0 disk.drbd1716.in‚0 disk.drbd1716.out‚0$ memory_percent‚85.28547286987305 disk.sda2.out‚0 disk.rbd1.out‚ 1.619140625 disk.sda.in‚0' proc.memory.availableMB‚ 2410.828125 disk.drbd2080.in‚0 disk.rbd0.in‚0 disk.nvme0n1.in‚ 11.80859375 disk.sda2.in‚0! disk.nvme0c0n1.in‚ 11.80859375 disk.sda3.out‚ 42.91015625 cpu‚26.205086567390705 disk.drbd2080.out‚0 disk.drbd2265.out‚0 disk.sda.out‚ 42.91015625 disk.sda3.in‚0! proc.memory.rssMB‚ 2801.046875* proc.memory.percent‚17.096233367919922 disk.dm-1.out‚ 42.91015625 network.recv‚456921 disk.drbd2266.out‚0 disk.dm-0.in‚ 11.80859375" disk.dm-0.out‚4247.37646484375 proc.cpu.threads‚17’8˜“Ï ‚(:ˆ ¨œäÔùÚ¥¨ gpu.0.gpu‚51.0 gpu.0.memory‚6, gpu.0.memoryAllocated‚12.165603637695312* gpu.0.memoryAllocatedBytes‚ 10450173952 gpu.0.temp‚35.0 gpu.0.powerWatts‚120.671' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚24.1342 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0üˆ¬ûVÐ ‚(:Ë ¨œäÔ¬¾©« network.sent‚ 2.447355e+06 disk.drbd2265.out‚0 disk.sda.out‚ 43.64453125 disk.drbd2080.in‚0% disk.nvme0n1.out‚4294.36376953125 disk.sda1.out‚0' disk.nvme0c0n1.out‚4294.36376953125" disk.dm-0.out‚4294.48876953125 proc.cpu.threads‚17% disk./.usageGB‚250.67963409423828 disk.sda.in‚0$ memory_percent‚85.00039577484131) disk./.usagePercent‚16.82332057828702 disk.drbd1716.in‚0 disk.sda2.out‚0 disk.drbd2265.in‚0! proc.memory.rssMB‚ 2801.046875 disk.sda3.in‚0 disk.sda3.out‚ 43.64453125 disk.dm-1.out‚ 43.64453125 disk.drbd1716.out‚0 disk.nvme0n1.in‚ 11.80859375! disk.nvme0c0n1.in‚ 11.80859375 disk.dm-0.in‚ 11.80859375) proc.memory.availableMB‚ 2457.53515625 disk.drbd2080.out‚0 disk.sda2.in‚0 disk.drbd2266.out‚0 disk.rbd1.in‚0 network.recv‚462201 disk.rbd0.out‚0 disk.sda1.in‚0 disk.drbd2266.in‚0 disk.rbd1.out‚ 1.681640625* proc.memory.percent‚17.096233367919922 cpu‚26.19838919930491 disk.dm-1.in‚0 disk.rbd0.in‚0ƒî­{“Ñ ‚(:ˆ ·œäÔÝŤ¨ gpu.0.gpu‚51.0 gpu.0.memory‚6, gpu.0.memoryAllocated‚12.165603637695312* gpu.0.memoryAllocatedBytes‚ 10450173952 gpu.0.temp‚36.0 gpu.0.powerWatts‚165.209' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚33.0418 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0 Q°}QÒ ‚(:Æ ·œäÔ¢Ýê disk.drbd2266.out‚0 disk.rbd1.in‚0! disk.nvme0c0n1.in‚ 11.80859375' disk.nvme0c0n1.out‚4344.27685546875 disk.sda3.in‚0! proc.memory.rssMB‚ 2801.046875* disk./.usagePercent‚16.824270878254122 disk.sda3.out‚ 44.140625 disk.rbd0.in‚0 disk.drbd2266.in‚0 disk.dm-0.in‚ 11.80859375 proc.cpu.threads‚17 disk.drbd1716.in‚0 disk.drbd2080.in‚0% disk.nvme0n1.out‚4344.27685546875 disk.drbd2265.out‚0 network.recv‚466861 disk.sda1.out‚0 disk.sda2.in‚0 disk.drbd2265.in‚0" disk.dm-0.out‚4344.40185546875 network.sent‚ 2.45384e+06 disk.drbd2080.out‚0$ memory_percent‚85.00044345855713) proc.memory.availableMB‚ 2457.52734375% disk./.usageGB‚250.69379425048828 disk.rbd1.out‚ 1.681640625* proc.memory.percent‚17.096233367919922 disk.sda.in‚0 disk.sda.out‚ 44.140625 disk.dm-1.in‚0 disk.drbd1716.out‚0 disk.nvme0n1.in‚ 11.80859375 disk.sda2.out‚0 cpu‚26.191485143722307 disk.dm-1.out‚ 44.140625 disk.rbd0.out‚0 disk.sda1.in‚0F"°W2Ó - _wandbruntime‚1125  _runtime‚1125º…¬¥ÇÔ Á trainer/global_step‚1349 % _timestamp‚1.7884155442186363e+09 _runtime‚1125.868036205 _step‚106 & lr-AdamW/pg1‚0.0017890493565248854 & lr-AdamW/pg2‚0.0017890493565248854‰MËÕ Å _step‚106 & lr-AdamW/pg1‚0.0017890493565248854 & lr-AdamW/pg2‚0.0017890493565248854 trainer/global_step‚1349 % _timestamp‚1.7884155442186363e+09 _runtime‚1125.868036205jÚ»ˆ2Ö - _wandbruntime‚1125  _runtime‚1125‡‡×€Å'× ¿O HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 _step‚107 Q>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚78127819295232 QCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 182432376 O@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 9373790789 VGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 8980994048 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚431 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚213 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚2 VGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 9283702784 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 169267712 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚11767496 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 435025920 XEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚78128005733376 NFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚985 _MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 1052495763456 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 _LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚74969016039424 L=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 9679899136 [IDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 1009835723264 M@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚18487944 SEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 313315328 cQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 1052703983104 TFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 446693376 E=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚644 @8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚644 G?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚431 UFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 9456058368 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 446693376 WEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 1009666455552 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 186438144 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 JBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚431 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚20039 PCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚19578610 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚20037 L?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚25489978 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚2605 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚3 TADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚75973279924897 ZHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 1004280924833 N@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 186438144 SDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 9283702784 [HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚77118152839680 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚20039 ZKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 9456058368 C;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚644 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚24127 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚24400 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 IDeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚2605 \IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚77118170010112 TGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚25510015 PCDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated‚25510015 QDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚18488433 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚24772608 BDeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 446693376 _runtime‚1125.872878484 L?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚19578608 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 @8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚644 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0ðþY Ø ÃO GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚2 VGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 9283702784 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 169267712 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚11767496 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 435025920 XEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚78128005733376 NFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚985 _MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 1052495763456 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 _LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚74969016039424 L=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 9679899136 [IDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 1009835723264 M@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚18487944 SEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 313315328 cQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 1052703983104 TFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 446693376 E=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚644 @8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚644 G?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚431 UFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 9456058368 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 446693376 WEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 1009666455552 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 186438144 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 JBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚431 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚20039 PCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚19578610 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚20037 L?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚25489978 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚2605 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚3 TADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚75973279924897 ZHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 1004280924833 N@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 186438144 SDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 9283702784 [HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚77118152839680 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚20039 ZKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 9456058368 C;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚644 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚24127 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚24400 WJDeviceStatsMonitor.on_train_batcȆ×ph_start/requested_bytes.large_pool.current‚17039360 IDeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚2605 \IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚77118170010112 TGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚25510015 PCDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated‚25510015 QDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚18488433 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚24772608 BDeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 446693376 _runtime‚1125.872878484 L?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚19578608 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 @8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚644 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 _step‚107 Q>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚78127819295232 QCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 182432376 O@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 9373790789 VGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 8980994048 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚431 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚213kO 62Ù - _wandbruntime‚1126  _runtime‚1126œV2IÎ&Ú ÈM >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚644 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 169281024 K>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚45121123 REDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚19592610 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚17039360 J=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚25508450 OADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 186451456 C6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚45101058 NADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚25528513 TEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 9283702784 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚469 trainer/global_step‚1349 ODeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 _step‚108 M>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 9679899136 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚466 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 239270400 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚644 J;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 9679899136 FDeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚20065 QCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 313315328 [MDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚ 214497792 M>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 9373790789 XFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 1005002371541 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 REDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚25528513 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚11775887 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 SEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 435025920 [HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚79045010613248 QBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 9902751744 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚213 E8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚431 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 NADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚19592608 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚3 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚35 bODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚77991541946368 WIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 446693376 ]JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚77173533699584 \JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 1010561090560 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚644 BDeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 186451456 VCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚77173516529152 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 YFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚75022863667200 SDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 9456058368 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 K>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚18500999 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 144203776 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚213 VCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚76028048471325 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚19592608 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17170432 XFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 1010391809536 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚2605 ^KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚77991517173760MÁFÒ&Û ÌM L=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 9902751744 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 _runtime‚ 1126.57493812 G:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚45101058 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚431 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 NADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚19592608 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚3 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚35 bODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚77991541946368 WIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 446693376 ]JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚77173533699584 \JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 1010561090560 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚644 BDeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 186451456 VCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚77173516529152 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 YFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚75022863667200 SDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 9456058368 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 K>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚18500999 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 144203776 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚213 VCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚76028048471325 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚19592608 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17170432 XFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 1010391809536 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚2605 ^KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚77991517173760 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚644 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 169281024 K>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚45121123 REDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚19592610 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚17039360 J=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚25508450 OADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 186451456 C6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚45101058 NADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚25528513 TEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 9283702784 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚469 trainer/global_step‚1349 ODeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 _step‚108 M>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 9679899136 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚466 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 239270400 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚644 J;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 9679899136 FDeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚20065 QCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 313315328 [MDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚ 214497792 M>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 9373790789 XFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 1005002371541 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 REDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚25528513 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚11775887 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 SEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 435025920 [HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚79045010613248 QBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 9902751744 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚213 EDeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚24127 BDeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚2605 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚17170432 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚3 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 251866624 O@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 9373790789 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚17170432 VGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 9283702784 ZHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 1045961982976 IDeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚80932299296256 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 435025920 ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 227094016 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 N?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 9902751744 XIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 9456058368 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 446693376 [IDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 1046131250688 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚20039 _MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 1090307424256 QDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚19147327 SGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚6959796 trainer/global_step‚1399 TADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚80932299296256 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚24400 PCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚20278610 TGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚20278610 dQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚80732329478144 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚24772608 D>DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 NFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚985 ]JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚81822863996416 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 ^LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 1046131250688 PCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚20278608 TGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚26436967 SEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 313315328 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 446693376 L?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚26416930jXKÌÉ'è ÃO M@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚46715577 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚2605 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 E8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚46695538 SDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 9283702784 [HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚79886337313280 XEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚78702795221481 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚485 XEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚79886337313280 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 _step‚111 \IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚79886354483712 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚12187043 UGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 431144269 QBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 9902751744 % _timestamp‚1.7884155842561097e+09 L?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚20278608 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 169267712 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 PCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚26416930 M@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚19146839 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 @8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚644 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚213 JBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚213 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 TADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚78702612789105 SDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 9902751744 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚2 ZHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 1040380432241 N@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 186438144 TFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 446693376 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚20039 G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚2605 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚17170432 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚3 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 251866624 O@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 9373790789 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚17170432 VGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 9283702784 ZHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 1045961982976 IDeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚80932299296256 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 435025920 ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 227094016 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 N?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 9902751744 XIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 9456058368 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 446693376 [IDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 1046131250688 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚20039 _MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 1090307424256 QDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚19147327 SGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚6959796 trainer/global_step‚1399 TADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚80932299296256 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚24400 PCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚20278610 TGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚20278610 dQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚80732329478144 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚24772608 D>DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 NFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚985 ]JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚81822863996416 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 ^LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 1046131250688 PCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚20278608 TGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚26436967 SEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 313315328 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 446693376 L?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚26416930 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 435025920 `MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚80732304705536 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 [HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚77662232356864 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 WEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 1045961982976 XEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚80932485734400 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 O@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 9679899136 _LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚79886354483712 cQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 1090534518272 XJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 165393016 IDeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚24127 B6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚644 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚213 YGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 1046856617984 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚478 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 245561856 VCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚78757381335533 L=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 9902751744 ODeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 9679899136 REDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚12195434 K>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚19159847 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚6964413 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚20065 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 \JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 1041267272173 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 446693376 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚644 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 431144269 TEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 9283702784 K>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚46748075 WDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚81879304483328 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 WIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 446693376 OADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 186451456 R?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚80988388339712 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚20063 ]KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 1091065338368 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 ]JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚77716114063360 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 _runtime‚1166.632807305 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚2 ]JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚79941718173184 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 169281024 G:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚46728010 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚213 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17170432 REDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚26455465 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 182432584 G:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚46748075 C6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚46728010 J=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚26435402 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 QBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 9902751744 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 169281024 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚24400 L>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 186451456 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚2605 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 S@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚80988574791168 J;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 9679899136 \JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 1046856617984 FDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 9373790789 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚644 NADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚20292608 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚24127 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 VCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚80988574791168 EDeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 trainer/global_step‚1399 UCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 1046687336960 XFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 1046687336960 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚24400 REDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚20292610 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚213 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚35Ò³•Îòë ÍM RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 446693376 VCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚79941701002752 OBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚19160325 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 248172032 SDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 9456058368 aODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 1091286127616 XIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 9456058368 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚431 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚431 _step‚112 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1003 NADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚26435402 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚475 LDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚985 [HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚81879550045184 [MDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚ 220789248 QBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 9283702784 YFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚77716097024000 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚644 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚17170432 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚17039360 VHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 165393224 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 trainer/global_step‚1âc9á399 UCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 1046687336960 XFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 1046687336960 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚24400 REDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚20292610 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚213 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚35 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24772608 B6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚644 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚213 YGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 1046856617984 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚478 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 245561856 VCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚78757381335533 L=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 9902751744 ODeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 9679899136 REDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚12195434 K>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚19159847 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚6964413 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚20065 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 \JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 1041267272173 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 446693376 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚644 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 431144269 TEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 9283702784 K>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚46748075 WDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚81879304483328 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 WIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 446693376 OADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 186451456 R?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚80988388339712 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚20063 ]KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 1091065338368 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 ]JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚77716114063360 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 _runtime‚1166.632807305 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚2 ]JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚79941718173184 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 169281024 G:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚46728010 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚213 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17170432 REDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚26455465 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 182432584 G:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚46748075 C6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚46728010 J=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚26435402 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 QBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 9902751744 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 169281024 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚24400 L>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 186451456 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚2605 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 S@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚80988574791168 J;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 9679899136 \JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 1046856617984 FDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 9373790789 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚644 NADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚20292608 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚24127 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 VCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚80988574791168 EDeviceStatsMonitor.on_train_batch_start/allocation.all.current‚20039 ^LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 1082426778112 MFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚35 _MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 1128258016256 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 UGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 431144269 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 trainer/global_step‚1449 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 YFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚84657284692992 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚431 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚2 SDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 9283702784 N?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 9902751744 G?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚213 SEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 313315328 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 ZHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 1076479939649 @8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚644 XEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚82654521786880 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 144203776 PCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚20978610 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 435025920 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 @8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚644 JBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚213 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚459 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 ZKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 9456058368 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 446693376 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 ZHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 1082257510400 TGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚27363919 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 248172032 [IDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 1082426778112 H?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚1003 SGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚7200042 JBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚431 % _timestamp‚1.7884156238120701e+09 XEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚83736965735424 VGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 9283702784 QDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚19807094 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 [HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚80355465713664 ^LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 1076645332665 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 446693376 IDeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚2605 L?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚27343882 N@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 186438144 L=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 9679899136 \IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚82654538957312 _LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚82654538957312 cQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 1128470430208 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚24127 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 PCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚20978608 UFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 9456058368 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 XIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 9456058368 [HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚82654521786880 VGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 8980994048 QBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 9902751744 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚17170432 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 186438144 TADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚83736779297280 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 169267712 BDeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚83736779297280 IDeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 M@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚19806635 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚20039 WEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 1082257510400 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚17170432y—[ú ÃO L=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 9679899136 \IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚82654538957312 _LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚82654538957312 cQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 1128470430208 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚24127 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 PCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚20978608 UFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 9456058368 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 XIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 9456058368 [HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚82654521786880 VGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 8980994048 QBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 9902751744 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚17170432 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 186438144 TADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚83736779297280 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 169267712 BDeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚83736779297280 IDeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 M@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚19806635 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚20039 WEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 1082257510400 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚17170432 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚20039 ^LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 1082426778112 MFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚35 _MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 1128258016256 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 UGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 431144269 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 trainer/global_step‚144À£n 9 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 YFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚84657284692992 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚431 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚2 SDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 9283702784 N?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 9902751744 G?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚213 SEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 313315328 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 ZHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 1076479939649 @8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚644 XEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚82654521786880 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 144203776 PCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚20978610 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 435025920 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 @8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚644 JBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚213 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚459 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 ZKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 9456058368 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 446693376 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 ZHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 1082257510400 TGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚27363919 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 248172032 [IDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 1082426778112 H?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚1003 SGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚7200042 JBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚431 % _timestamp‚1.7884156238120701e+09 XEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚83736965735424 VGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 9283702784 QDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚19807094 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 [HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚80355465713664 ^LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 1076645332665 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 446693376 IDeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚2605 L?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚27343882 N@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 186438144su5Ç2û - _wandbruntime‚1206  _runtime‚1206Óû1ÞÏ&ü ÉM M>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 9679899136 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 169281024 ^KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚83584961116160 VCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚81486714199741 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚431 L>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 186451456 UCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 1082982864384 XFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 1082982864384 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚12614987 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚644 \JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 1083152145408 [HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚84714222313472 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 % _timestamp‚1.7884156245148466e+09 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 248172032 R?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚81486531767157 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚431 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 VGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 9456058368 REDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚27382417 BDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 9373790789 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 169281024 WDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚84713974654464 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17170432 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚24127 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 TEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 8980994048 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚213 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 OBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚19820241 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚644 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 446693376 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 144203776 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚644 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚431 E8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 XFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 1077201386357 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 431144269 S@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚83793054792192 K>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚19819770 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1003 bODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚83584985888768 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚213 OADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 186451456 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚7205254 G:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚48375027 J=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚27362354 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚2605 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚468 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 247659008 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24772608 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚20065 REDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚20992610 QCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 313315328 VHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 165393224 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 VCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚82709885476352 SEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 435025920 ]JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚80409347420160 YFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚80409330380800 QBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 9902751744 FDeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚48375027 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 182432584 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚644 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚213 NADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚20992610 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚17170432 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚3 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚17039360 O@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 9902751744 QBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 9283702784 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚20992608 PBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 435025920 NADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚27362354 aODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 1129236424704 [MDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚ 222886400 trainer/global_step‚1449Åœž Ó&ý ÍM D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 446693376 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 144203776 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚644 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚431 E8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 XFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 1077201386357 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 431144269 S@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚83793054792192 K>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚19819770 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1003 bODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚83584985888768 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚213 OADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 186451456 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚7205254 G:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚48375027 J=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚27362354 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚2605 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚468 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 247659008 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24772608 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚20065 REDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚20992610 QCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 313315328 VHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 165393224 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 VCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚82709885476352 SEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 435025920 ]JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚80409347420160 YFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚80409330380800 QBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 9902751744 FDeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚48375027 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 182432584 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚644 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚213 NADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚20992610 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚17170432 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚3 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚17039360 O@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 9902751744 QBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 9283702784 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚20992608 PBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 435025920 NADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚27362354 aODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 1129236424704 [MDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚ 222886400 trainer/global_step‚1449 M>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 9679899136 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 169281024 ^KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚83584961116160 VCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚81486714199741 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚431 L>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 186451456 UCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 1082982864384 XFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 1082982864384 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚12614987 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚644 \JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 1083152145408 [HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚84714222313472 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 % _timestamp‚1.7884156245148466e+09 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 248172032 R?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚81486531767157 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚431 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 VGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 9456058368 REDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚27382417 BDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 9373790789 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 169281024 WDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚84713974654464 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17170432 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚24127 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 TEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 8980994048 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚213 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 OBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚19820241 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚644též2þ - _wandbruntime‚1206  _runtime‚1206%]° ]ÿ × 1loss/FBt2/assignment_loss‚0.07368171963364982 2loss/FBt4/assignment_loss‚0.031164841783048087 /loss/FBt1/detection_loss‚2.3095211187998452 _step‚117 .loss/FRt2/detection_loss‚2.479144018731619 /loss/FRt4/detection_loss‚1.6713507699720356 /loss/SRqqt2/detection_loss‚2.36292304207639 /loss/FRt1/assignment_loss‚3.690330903890521 /loss/FRt2/assignment_loss‚4.279918825940082 1loss/SRqqt1/detection_loss‚1.8379723436332174 0loss/FRt3/assignment_loss‚3.8611894387465258 1loss/FBt1/assignment_loss‚0.21081548185999613 % _timestamp‚1.7884156245166461e+09 _runtime‚1206.165858539 0loss/FRt4/assignment_loss‚2.3369283353516814 /loss/FRt1/detection_loss‚2.5403940779438545 /loss/FRt3/detection_loss‚2.8654629258009106 0loss/SRqqt4/detection_loss‚1.100548883826714 2loss/SRqqt1/assignment_loss‚2.0471105350048413 1loss/SRqqt2/assignment_loss‚2.079728992610443 2loss/SRqqt3/assignment_loss‚2.1062747753846147 /loss/FBt3/detection_loss‚1.7982361209392548 2loss/SRqqt4/assignment_loss‚1.4381547699733748 1loss/FBt3/assignment_loss‚0.44399832309645715 /loss/FBt4/detection_loss‚0.7186567424232982 %loss/total_loss‚47.14872264637765 epoch‚2 trainer/global_step‚1449 1loss/SRqqt3/detection_loss‚1.9166920858745773 /loss/FBt2/detection_loss‚2.9485235730806987]lV×a€ Û /loss/FRt1/assignment_loss‚3.690330903890521 /loss/FRt2/assignment_loss‚4.279918825940082 1loss/SRqqt1/detection_loss‚1.8379723436332174 0loss/FRt3/assignment_loss‚3.8611894387465258 1loss/FBt1/assignment_loss‚0.21081548185999613 % _timestamp‚1.7884156245166461e+09 _runtime‚1206.165858539 0loss/FRt4/assignment_loss‚2.3369283353516814 /loss/FRt1/detection_loss‚2.5403940779438545 /loss/FRt3/detection_loss‚2.8654629258009106 0loss/SRqqt4/detection_loss‚1.100548883826714 2loss/SRqqt1/assignment_loss‚2.0471105350048413 1loss/SRqqt2/assignment_loss‚2.079728992610443 2loss/SRqqt3/assignment_loss‚2.1062747753846147 /loss/FBt3/detection_loss‚1.7982361209392548 2loss/SRqqt4/assignment_loss‚1.4381547699733748 1loss/FBt3/assignment_loss‚0.44399832309645715 /loss/FBt4/detection_loss‚0.7186567424232982 %loss/total_loss‚47.14872264637765 epoch‚2 trainer/global_step‚1449 1loss/SRqqt3/detection_loss‚1.9166920858745773 /loss/FBt2/detection_loss‚2.9485235730806987 1loss/FBt2/assignment_loss‚0.07368171963364982 2loss/FBt4/assignment_loss‚0.031164841783048087 /loss/FBt1/detection_loss‚2.3095211187998452 _step‚117 .loss/FRt2/detection_loss‚2.479144018731619 /loss/FRt4/detection_loss‚1.6713507699720356 /loss/SRqqt2/detection_loss‚2.36292304207639um<5È“ ‚(:ˆ ‘äÔØ£ ¨ gpu.0.gpu‚54.0 gpu.0.memory‚7, gpu.0.memoryAllocated‚12.165603637695312* gpu.0.memoryAllocatedBytes‚ 10450173952 gpu.0.temp‚36.0 gpu.0.powerWatts‚149.188' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚29.8376 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0–ÿ,xN‚ ‚(:à ‘äÔˆÙ»ª proc.cpu.threads‚17 disk.sda1.in‚0 disk.drbd2266.in‚0 disk.sda.out‚ 47.578125 disk.dm-1.out‚ 47.578125 disk.drbd1716.out‚0 disk.drbd2080.in‚0' disk.nvme0c0n1.out‚4686.87744140625 disk.rbd0.out‚0% disk.nvme0n1.out‚4686.87744140625# memory_percent‚85.0013256072998 disk.drbd2080.out‚0 network.recv‚494699% disk./.usageGB‚250.76229095458984 disk.drbd2266.out‚0) disk./.usagePercent‚16.82886774953894 disk.sda2.out‚0" disk.dm-0.out‚4687.00244140625 disk.sda.in‚0* proc.memory.percent‚17.096233367919922 disk.drbd2265.in‚0 disk.rbd1.out‚ 1.83984375 network.sent‚ 2.656256e+06 disk.sda1.out‚0 disk.sda3.out‚ 47.578125( proc.memory.availableMB‚ 2457.3828125 disk.drbd2265.out‚0 disk.rbd0.in‚0 disk.nvme0n1.in‚ 11.80859375 disk.rbd1.in‚0 disk.sda2.in‚0 disk.dm-0.in‚ 11.80859375 disk.sda3.in‚0 disk.dm-1.in‚0! disk.nvme0c0n1.in‚ 11.80859375 disk.drbd1716.in‚0! proc.memory.rssMB‚ 2801.046875 cpu‚26.156796781874817R~´C“ƒ ‚(:ˆ  äÔ©³“© gpu.0.gpu‚20.0 gpu.0.memory‚3, gpu.0.memoryAllocated‚12.165603637695312* gpu.0.memoryAllocatedBytes‚ 10450173952 gpu.0.temp‚36.0 gpu.0.powerWatts‚167.929' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚33.5858 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0…¤ñŽA„ ‚(:¶  äÔµ¦æª disk.sda1.in‚0 disk.drbd2265.in‚0 disk.sda.in‚0 disk.drbd2080.in‚0 disk.nvme0n1.in‚ 11.80859375) proc.memory.availableMB‚ 2457.37109375% disk./.usageGB‚250.78418731689453 disk.sda1.out‚0 disk.rbd1.out‚ 1.83984375! disk.nvme0c0n1.out‚ 4752.96875 disk.dm-0.in‚ 11.80859375 disk.sda.out‚ 48.1796875! proc.memory.rssMB‚ 2801.046875 cpu‚26.151517592878854 network.sent‚ 2.742897e+06 disk.drbd2266.in‚0! disk.nvme0c0n1.in‚ 11.80859375 disk.sda3.in‚0 disk.sda3.out‚ 48.1796875 disk.dm-1.in‚0 disk.rbd0.out‚0* proc.memory.percent‚17.096233367919922 network.recv‚500119 disk.dm-0.out‚ 4753.09375 disk.rbd0.in‚0$ memory_percent‚85.00139713287354) disk./.usagePercent‚16.83033723278548 disk.sda2.out‚0 disk.drbd1716.out‚0 disk.drbd2080.out‚0 proc.cpu.threads‚17 disk.sda2.in‚0 disk.dm-1.out‚ 48.1796875 disk.nvme0n1.out‚ 4752.96875 disk.drbd2265.out‚0 disk.rbd1.in‚0 disk.drbd1716.in‚0 disk.drbd2266.out‚0yW}l2… - _wandbruntime‚1244  _runtime‚1244ÇôšÅ† ¿ % _timestamp‚1.7884156629144146e+09 _runtime‚1244.564814817 _step‚118 % lr-AdamW/pg1‚0.001751980632412933 % lr-AdamW/pg2‚0.001751980632412933 trainer/global_step‚1499ƒ,`;ɇ à % lr-AdamW/pg1‚0.001751980632412933 % lr-AdamW/pg2‚0.001751980632412933 trainer/global_step‚1499 % _timestamp‚1.7884156629144146e+09 _runtime‚1244.564814817 _step‚118vû£12ˆ - _wandbruntime‚1244  _runtime‚1244ãj÷ ‰ ¾O \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚24772608 @8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚644 _step‚119 TGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚28290871 trainer/global_step‚1499 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚24127 SDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 9283702784 TGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚21678610 YFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚87491935732736 TADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚84161278517521 E8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚49949442 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 186438144 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚24400 cQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 1166403693056 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 C;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚644 UGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 431144269 N?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 9902751744 L?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚28270834 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚2605 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 @8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚644 G?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚431 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚460 H?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚1003 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 dQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚86325773420544 [HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚85422706260480 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 241380864 `MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚86325748647936 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 XJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 165393016 SDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 9902751744 SGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚7439297 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.all%»d ocated‚213 D>DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 446693376 JBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚431 Q>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚86541259298304 O@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 9373790789 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚2 TADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚86541259298304 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 169267712 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚457 _LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚83048716109824 ZHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 1112579447057 TFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 446693376 $ _timestamp‚1.788415662915168e+09 [IDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 1118722305536 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 144203776 JBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚213 IDeviceStatsMonitor.on_train_batch_start/allocation.all.current‚20039 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚7439754 XEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚84161460949897 _runtime‚1244.564904125 XIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 9456058368 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚17170432 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 169267712 M@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚20465440 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 248172032 G?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚213 XEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚86541445736448 _LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚85422723430912 [HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚83048699070464 VGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 8980994048 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚431 ^LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 1112744840073 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚2605 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚3 XEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚85422706260480 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 UBDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated‚86541445736448 \IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚85422723430912 ^LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 1118722305536 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 UFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 9456058368 PCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚21678610 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 WEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 1118553037824 O@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 9679899136 MFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚35 QBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 9902751744 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 446693376 M@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚49969481 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 L=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 9679899136 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚24400 PCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚28270834 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚13026143 N@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 186438144 VGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 9283702784 PCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚21678608 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚24127 NFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚985 QCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 182432376 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚20037?–ÝÈ'Š ÂO G?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚431 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚460 H?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚1003 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 dQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚86325773420544 [HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚85422706260480 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 241380864 `MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚86325748647936 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 XJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 165393016 SDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 9902751744 SGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚7439297 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚213 D>DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 446693376 JBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚431 Q>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚86541259298304 O@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 9373790789 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚2 TADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚86541259298304 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 169267712 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚457 _LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚83048716109824 ZHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 1112579447057 TFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 446693376 $ _timestamp‚1.788415662915168e+09 [IDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 1118722305536 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 144203776 JBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚213 IDeviceStatsMonitor.on_train_batch_start/allocation.all.current‚20039 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚7439754 XEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚84161460949897 _runtime‚1244.564904125 XIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 9456058368 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚17170432 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 169267712 M@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚20465440 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 248172032 G?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚213 XEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚86541445736448 _LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚85422723430912 [HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚83048699070464 VGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 8980994048 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚431 ^LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 1112744840073 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚2605 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚3 XEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚85422706260480 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 UBDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated‚86541445736448 \IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚85422723430912 ^LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 1118722305536 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 UFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 9456058368 PCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚21678610 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 WEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 1118553037824 O@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 9679899136 MFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚35 QBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 9902751744 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 446693376 M@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚49969481 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 L=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 9679899136 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚24400 PCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚28270834 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚13026143 N@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 186438144 VGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 9283702784 PCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚21678608 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚24127 NFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚985 QCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 182432376 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚20037 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚24772608 @8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚644 _step‚119 TGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚28290871 trainer/global_step‚1499 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚24127 SDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 9283702784 TGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚21678610 YFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚87491935732736 TADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚84161278517521 E8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚49949442 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 186438144 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚24400 cQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 1166403693056 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 C;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚644 UGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 431144269 N?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 9902751744 L?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚28270834 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚2605 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 @8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚644w©~;“‹ ‚(:ˆ ¯äÔ§Ôè gpu.0.gpu‚45.0 gpu.0.memory‚4, gpu.0.memoryAllocated‚12.165603637695312* gpu.0.memoryAllocatedBytes‚ 10450173952 gpu.0.temp‚36.0 gpu.0.powerWatts‚165.252' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚33.0504 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0ÏHf?Œ ‚(:´ ¯äÔº¾Í« disk.drbd1716.in‚0 disk.drbd2080.in‚0 disk.drbd2266.in‚0 disk.drbd2265.out‚0 disk.sda.in‚0 disk.sda3.in‚0* proc.memory.percent‚17.096233367919922 disk.drbd2265.in‚0 disk.nvme0c0n1.out‚ 4800.9375! proc.memory.rssMB‚ 2801.046875 cpu‚26.14634350378197 proc.cpu.threads‚17 disk.drbd2080.out‚0 disk.nvme0n1.in‚ 11.80859375 disk.nvme0n1.out‚ 4800.9375 disk.sda1.out‚0 disk.drbd2266.out‚0 disk.rbd1.out‚ 1.92578125 disk.sda3.out‚ 48.52734375 disk.drbd1716.out‚0 network.recv‚505235 disk.rbd0.in‚0 disk.sda2.out‚0 disk.rbd1.in‚0! disk.nvme0c0n1.in‚ 11.80859375$ memory_percent‚85.00285148620605 disk.sda.out‚ 48.52734375( proc.memory.availableMB‚ 2457.1328125 disk.sda1.in‚0 disk.sda2.in‚0 disk.dm-0.in‚ 11.80859375 network.sent‚ 2.749362e+06) disk./.usagePercent‚16.82847836207936 disk.rbd0.out‚0 disk.dm-0.out‚ 4801.0625 disk.dm-1.in‚0 disk.dm-1.out‚ 48.52734375% disk./.usageGB‚250.75648880004883·1úß2 - _wandbruntime‚1245  _runtime‚1245åKMÎÎ&Ž ÈM VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 169281024 \JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 1113466286989 XIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 9456058368 F8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 EDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 9679899136 \JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 1119447672832 REDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚28309369 LDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚985 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚644 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 O@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 9902751744 TEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 9283702784 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚20063 WDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚87548630883840 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 ODeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 trainer/global_step‚1499 _step‚120 S@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚86597534793216 F6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚644 _runtime‚ 1245.26196205 XFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 1119278391808 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 431144269 YFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚85478069949952 K>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚50001979 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 VHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 165393224 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚13034537 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 BDeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 186451456 VCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚85478069949952 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚3 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 % _timestamp‚1.7884156636121008e+09 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚20063 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17170432 K>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚20478464 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24772608 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 144203776 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 ]JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚83102580776960 NADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚28309369 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 446693376 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 446693376 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚213 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚24127 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚35 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 YFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚83102563737600 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚644 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚431 G:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚50001979 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚449 bODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚86381707859968 M>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 9373790789 WIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 446693376 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚644 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚24400 R?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚86597348341760 XFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 1113300893765 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚213 YGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 1119447672832 REDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚21692610 NADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚28289306 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 248172032 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 NADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚21692610 SEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 435025920=ós]( ÌM VCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚84216047063949 OBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚20478913 [HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚87548872251392 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚20065 QBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 9283702784 VCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚86597534793216 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚2605 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 182432584 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 trainer/global_step‚1499 _step‚120 S@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚86597534793216 F6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚644 _runtime‚ 1245.26196205 XFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 1119278391808 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 431144269 YFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚85478069949952 K>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚50001979 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 VHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 165393224 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚13034537 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 BDeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 186451456 VCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚85478069949952 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚3 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 % _timestamp‚1.7884156636121008e+09 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚20063 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17170432 K>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚20478464 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24772608 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 144203776 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 ]JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚83102580776960 NADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚28309369 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 446693376 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 446693376 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚213 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚24127 KDDevi`ïÕªceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚35 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 YFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚83102563737600 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚644 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚431 G:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚50001979 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚449 bODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚86381707859968 M>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 9373790789 WIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 446693376 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚644 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚24400 R?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚86597348341760 XFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 1113300893765 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚213 YGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 1119447672832 REDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚21692610 NADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚28289306 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 248172032 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 NADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚21692610 SEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 435025920 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 169281024 \JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 1113466286989 XIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 9456058368 F8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 EDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 9679899136 \JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 1119447672832 REDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚28309369 LDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚985 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚644 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 O@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 9902751744 TEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 9283702784 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚20063 WDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚87548630883840 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 ODeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚24127 JBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚213 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚17170432 XEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚88190890734080 TGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚29217823 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚466 C;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚644 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚2 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 435025920 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚24127 YFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚90326577691136 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 @8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚644 O@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 9373790789 O@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 9679899136 ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 214511104 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 PCDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated‚29217823 ]JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚90326816974848 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚24772608 _LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚85741949466624 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚17170432 PCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚29197786 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚7679183 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 BDeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚13445693 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚24400 UGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 431144269 QBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 9902751744 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 169267712 WEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 1154848565248 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚13445696 MFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚35 cQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 1204321583104 XEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚86890793814105 VGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 8980994048 trainer/global_step‚1549 _LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚88190907904512 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 144203776 QCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 182432376 @8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚644 % _timestamp‚1.7884157026581123e+09 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 186438144 SDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 9902751744 Q>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚89345739299328 TGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚22378610 XEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚89345925737472 M@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚51596433 QDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚21124879 H?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚1003 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 N?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 9902751744 XIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 9456058368 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 G?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚213 VGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 9283702784 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚20039 M@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚21124413 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 ZHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 1154848565248 PCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚22378608 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 435025920 SGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚7678720 NFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚985 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 446693376 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚20039 L=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 9679899136 [IDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 1155017832960 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 IDeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚2605 [HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚88190890734080 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 169267712 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚20037 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 248172032«VIÎ'œ ÃO D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚20039 L=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 9679899136 [IDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 1155017832960 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 IDeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚2605 [HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚88190890734080 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 169267712 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚20037 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 248172032 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 L?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚29197786 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚3 _MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 1204107072000 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 JBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚431 UBDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated‚89345925737472 IDeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚24127 JBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚213 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚17170432 XEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚88190890734080 TGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚29217823 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚466 C;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚644 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚2 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 435025920 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚24127 YFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚90326577691136 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 @8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚644 O@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 9373790789 O@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 9679899136 ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 214511104 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 PCDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated‚29217823 ]JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚90326816974848 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚24772608 _LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚85741949466624 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚17170432 PCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚29197786 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚7679183 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 BDeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚13445693 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚24400 UGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 431144269 QBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 9902751744 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 169267712 WEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 1154848565248 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚13445696 MFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚35 cQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 1204321583104 XEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚86890793814105 VGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 8980994048 trainer/global_step‚1549 _LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚88190907904512 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 144203776 QCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 182432376 @8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚644 % _timestamp‚1.7884157026581123e+09 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 186438144 SDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 9902751744 Q>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚89345739299328 TGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚22378610 XEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚89345925737472 M@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚51596433 QDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚21124879 H?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚1003 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 N?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 9902751744 XIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 9456058368 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 G?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚213 VGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 9283702784 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚20039 M@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚21124413 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 ZHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 1154848565248 PCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚22378608 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 435025920 SGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚7678720 NFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚985 WIDeviceStatsMoniÁ8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 VGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 9456058368 WIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 446693376 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚2 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚35 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 SEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 435025920 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚470 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17170432 [MDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚ 222886400 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚213 TEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 9283702784 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 OBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚21137986 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 446693376 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 169281024 K>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚21137513 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚20063 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 UCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 1155573919232 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚24400 J=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚29216258 K>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚51628931 M>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 9373790789 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚644 REDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚29236321 bODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚89178429831168 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 182432584 _step‚124 O@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 9902751744 L=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 9902751744 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚431 NADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚29236321 L>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 186451456 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 YGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 1155743200256 PBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 435025920 G:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚51608866 NADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚29216258 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1003 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 248172032 TEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 8980994048 FDeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 NADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚22392608 S@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚89402014794240 VCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚88246254423552 YFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚88246254423552 XFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 1155573919232 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚3 SDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 9456058368 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 169281024 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 B6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚644 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 trainer/global_step‚1549 OADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 186451456 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚2605 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24772608 QBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 9902751744 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 M>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 9679899136 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 144203776 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚431 QBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 9283702784 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚22392608 ZGDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚88246271593984 ]JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚88246271593984 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚644 VCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚89402014794240 LDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚985 [HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚90383514000896 WDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚90383266341888 aODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 1205084169728 YFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚85795797094400 VHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 165393224 XFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 1149400401173 G:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚51628931 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚20065 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 247659008 \JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 1149565794397 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 431144269 J;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 9679899136 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚17039360 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚213 _runtime‚1285.007378659 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚17170432 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚13454087 QCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 313315328Ü•‚RÓ&Ÿ ÍM \JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 1155743200256 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 NADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚22392608 S@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚89402014794240 VCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚88246254423552 YFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚88246254423552 XFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 1155573919232 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚3 SDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 9456058368 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 169281024 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 B6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚644 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 trainer/global_step‚1549 OADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 186451456 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚2605 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24772608 QBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 9902751744 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 M>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 9679899136 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 144203776 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚431 QBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 9283702784 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚22392608 ZGDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚88246271593984 ]JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚88246271593984 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚644 VCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚89402014794240 LDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚985 [HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚90383514000896 WDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚90383266341888 aODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 1205084169728 YFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚85795797094400 VHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 165393224 XFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 1149400401173 G:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚51628931 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚20065 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 247659008 \JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 1149565794397 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 431144269 J;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 9679899136 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚17039360 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚213 _runtime‚1285.007378659 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚17170432 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚13454087 QCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 313315328 C6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚51608866 NADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚22392610 R?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚89401828342784 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚473 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 R?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚86945197495573 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚431 F8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 VGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 9456058368 WIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 446693376 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚2 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚35 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 SEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 435025920 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚470 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17170432 [MDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚ 222886400 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚213 TEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 9283702784 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 OBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚21137986 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 446693376 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 169281024 K>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚21137513 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚20063 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 UCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 1155573919232 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚24400 J=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚29216258 K>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚51628931 M>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 9373790789 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚644 REDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚29236321 bODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚89178429831168 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 182432584 _step‚124 O@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 9902751744 L=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 9902751744 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚431 NADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚29236321 L>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 186451456 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 YGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 1155743200256 PBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 435025920 G:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚51608866 NADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚29216258 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1003 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 248172032 TEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 8980994048 F<“¥ ‚(:ˆ ëäÔØù¡¨ gpu.0.gpu‚52.0 gpu.0.memory‚6, gpu.0.memoryAllocated‚12.165603637695312* gpu.0.memoryAllocatedBytes‚ 10450173952 gpu.0.temp‚36.0 gpu.0.powerWatts‚171.711' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚34.3422 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0ðŠ¨pX¦ ‚(:Í ëäÔ•Œ«« disk.dm-0.in‚ 11.80859375" disk.dm-0.out‚5046.50048828125 disk.drbd1716.in‚0 disk.drbd1716.out‚0 disk.sda2.in‚0 network.sent‚ 2.995002e+06) disk./.usagePercent‚16.83326851905253 disk.sda.in‚0 disk.sda2.out‚0! disk.nvme0c0n1.in‚ 11.80859375 disk.sda3.out‚ 50.80859375 disk.sda1.out‚0 disk.sda3.in‚0 disk.dm-1.in‚0 disk.drbd2080.out‚0 disk.rbd0.out‚0 disk.nvme0n1.in‚ 11.80859375 disk.drbd2265.in‚0 disk.drbd2265.out‚0* proc.memory.percent‚17.096257209777832% disk./.usageGB‚250.82786560058594 disk.rbd1.in‚0# proc.memory.rssMB‚ 2801.05078125 cpu‚26.12714482077661 disk.sda.out‚ 50.80859375% disk.nvme0n1.out‚5046.37548828125 disk.sda1.in‚0 disk.rbd1.out‚ 2.0205078125 disk.dm-1.out‚ 50.80859375 disk.drbd2266.in‚0$ memory_percent‚84.94839668273926( proc.memory.availableMB‚ 2466.0546875 proc.cpu.threads‚17 network.recv‚526297' disk.nvme0c0n1.out‚5046.37548828125 disk.drbd2080.in‚0 disk.rbd0.in‚0 disk.drbd2266.out‚0|Ť‘ž§ ‚(:“ úäÔü«¢¨ gpu.0.gpu‚29.0 gpu.0.memory‚4, gpu.0.memoryAllocated‚12.165603637695312* gpu.0.memoryAllocatedBytes‚ 10450173952 gpu.0.temp‚36.0 gpu.0.powerWatts‚172.979' gpu.0.enforcedPowerLimitWatts‚500.0) gpu.0.powerPercent‚34.595800000000004 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0KøðH¨ ‚(:½ úäÔÿÖ±ª disk.drbd2266.in‚0* proc.memory.percent‚17.096257209777832 disk.sda.out‚51.375 disk.drbd2265.in‚0 disk.rbd1.in‚0 disk.dm-1.out‚51.375 disk.sda2.in‚0' proc.memory.availableMB‚ 2466.296875 proc.cpu.threads‚17) disk./.usagePercent‚16.83440903261757 disk.sda3.out‚51.375 disk.dm-1.in‚0 disk.sda1.in‚0 disk.sda2.out‚0 disk.nvme0n1.in‚ 11.80859375% disk.nvme0n1.out‚5109.93212890625 disk.sda.in‚0 network.recv‚530691 disk.drbd1716.in‚0 disk.rbd0.out‚0 disk.drbd2266.out‚0 network.sent‚ 3.001185e+06 disk.rbd0.in‚0# proc.memory.rssMB‚ 2801.05078125$ disk./.usageGB‚250.8448600769043 disk.dm-0.in‚ 11.80859375$ memory_percent‚84.94691848754883 cpu‚26.12241569202098 disk.rbd1.out‚ 2.08349609375 disk.sda3.in‚0 disk.drbd2080.out‚0! disk.nvme0c0n1.in‚ 11.80859375' disk.nvme0c0n1.out‚5109.93212890625" disk.dm-0.out‚5110.05712890625 disk.drbd2080.in‚0 disk.drbd2265.out‚0 disk.drbd1716.out‚0 disk.sda1.out‚0ԭʾ2© - _wandbruntime‚1323  _runtime‚1323wä}‰Çª Á & lr-AdamW/pg1‚0.0017238499129441846 & lr-AdamW/pg2‚0.0017238499129441846 trainer/global_step‚1599 % _timestamp‚1.7884157422422843e+09 _runtime‚1323.891747412 _step‚126|•'Ë« Å & lr-AdamW/pg1‚0.0017238499129441846 & lr-AdamW/pg2‚0.0017238499129441846 trainer/global_step‚1599 % _timestamp‚1.7884157422422843e+09 _runtime‚1323.891747412 _step‚126~H4aa2¬ - _wandbruntime‚1323  _runtime‚1323y6Î]Ê­ ¿O ZHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 1191144092672 O@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 9373790789 TFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 446693376 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚24400 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚20037 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 TADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚92150219300352 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚496 XEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚89620126678313 ^LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 1184943854889 SDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 9902751744 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 G?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚213 IDeviceStatsMonitor.on_train_batch_start/allocation.all.current‚20039 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚24772608 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚24127 VGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 8980994048 UGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 431144269 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 L?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚23078608 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚24127 @8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚644 JBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚431 [IDevi½×)ûceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 1191313360384 ^LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 1191313360384 D>DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 G?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚431 trainer/global_step‚1599 _step‚127 \IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚90959092378112 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 251866624 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 WEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 1191144092672 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 186438144 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 169267712 ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 227094016 XIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 9456058368 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚2 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 169267712 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚20037 MFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚35 _LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚88435182823424 QBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 9902751744 G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚2605 L?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚30124738 O@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 9679899136 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 446693376 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 ]JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚93161448417280 UFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 9456058368 PCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚23078610 XEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚90959075207680 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚3 SGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚7918679 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 248172032 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 [HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚88435165784064 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 XEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚92150405738496 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚24400 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚499 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚13865243 NFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚985 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 _LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚90959092378112 M@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚53223385 YFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚93161196550656 ZKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 9456058368 C;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚644 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚17170432 dQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚91919217362944 TADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚89619944245937 XJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 165393016 Q>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚92150219300352 cQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 1242231054336 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚20039 _MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 1242003960320 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 435025920 QDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚21784421 BDeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚2605 L?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚30124738 O@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 9679899136 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 446693376 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 ]JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚93161448417280 UFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 9456058368 PCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚23078610 XEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚90959075207680 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚3 SGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚7918679 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 248172032 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 [HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚88435165784064 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 XEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚92150405738496 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚24400 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚499 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚13865243 NFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚985 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 _LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚90959092378112 M@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚53223385 YFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚93161196550656 ZKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 9456058368 C;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚644 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚17170432 dQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚91919217362944 TADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚89619944245937 XJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 165393016 Q>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚92150219300352 cQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 1242231054336 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚20039 _MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 1242003960320 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 435025920 QDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚21784421 BDeviceStatsMonitor.on_train_batch_start/allocation.all.current‚20039 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚24772608 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚24127 VGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 8980994048 UGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 431144269 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 L?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚23078608 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚24127 @8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚644 JBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚431 [IDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 1191313360384 ^LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 1191313360384 D>DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 G?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚431 trainer/global_step‚1599 _step‚127 \IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚90959092378112 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 251866624 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 WEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 1191144092672 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 186438144 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 169267712 ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 227094016 XIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 9456058368 pØT2¯ - _wandbruntime‚1324  _runtime‚1324k2·ÚÏ&° ÉM HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 S@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚92206494795264 K>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚21797123 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 182432584 \JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 1192038727680 LDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚985 aODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 1242982716928 ]KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 1242755636224 TEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 8980994048 R?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚89674530359781 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚24400 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17170432 UCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 1191869446656 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚3 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 169281024 NADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚23092608 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚431 VCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚91014438897152 VCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚92206494795264 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚24127 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 144203776 C6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚53235818 YGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 1192038727680 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 251853312 [MDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚ 227080704 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚644 B6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚644 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚431 M>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 9679899136 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚7923489 VGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 9456058368 L>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 186451456 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚7923977 QCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 313315328 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 NADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚30143210 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 J=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚30143210 SEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 435025920 XFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 1191869446656 bODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚91975151802368 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 VCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚89674712792365 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚213 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 446693376 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚491 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24772608 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 G:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚53255883 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚20065 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚35 XFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 1185499908581 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚644 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 169281024 SDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 9456058368 FDeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚53255883 \JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 1185665301805 O@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 9902751744 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚23092608 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚20063 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚431 EDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 9373790789 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 431144269 QBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 9902751744 trainer/global_step‚1599 QBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 9283702784 OADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 186451456 R?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚92206308343808 FDeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚213 _runtime‚1324.627922386 YFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚88489030451200 REDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚30163273 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1003 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚2 ZGDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚91014456067584 OBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚21797614 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚488 ]JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚88489047490560 ODeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 9679899136 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚7923489 VGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 9456058368 L>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 186451456 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚7923977 QCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 313315328 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 NADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚30143210 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 J=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚30143210 SEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 435025920 XFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 1191869446656 bODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚91975151802368 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 VCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚89674712792365 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚213 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 446693376 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚491 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24772608 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 G:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚53255883 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚20065 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚35 XFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 1185499908581 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚644 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 169281024 SDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 9456058368 FDeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚53255883 \JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 1185665301805 O@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 9902751744 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚23092608 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚20063 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚431 EDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 9373790789 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 431144269 QBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 9902751744 trainer/global_step‚1599 QBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 9283702784 OADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 186451456 R?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚92206308343808 FDeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚213 _runtime‚1324.627922386 YFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚88489030451200 REDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚30163273 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1003 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚2 ZGDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚91014456067584 OBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚21797614 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚488 ]JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚88489047490560 ODeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 S@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocate^Üå¼ d‚92206494795264 K>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚21797123 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 182432584 \JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 1192038727680 LDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚985 aODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 1242982716928 ]KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 1242755636224 TEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 8980994048 R?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚89674530359781 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚24400 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17170432 UCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 1191869446656 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚3 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 169281024 NADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚23092608 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚431 VCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚91014438897152 VCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚92206494795264 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚24127 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 144203776 C6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚53235818 YGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 1192038727680 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 251853312 [MDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚ 227080704 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚644 B6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚644€óX+²2² - _wandbruntime‚1324  _runtime‚1324¢ì®ÙU³ Ï /loss/FRt1/detection_loss‚2.1775083050841375 2loss/SRqqt1/assignment_loss‚1.9691703608592175 1loss/FBt1/assignment_loss‚0.41067718664205377 1loss/SRqqt4/detection_loss‚1.0779939099799756 trainer/global_step‚1599 /loss/FRt4/assignment_loss‚2.119378975552061 1loss/SRqqt3/assignment_loss‚2.048190070947875 -loss/FBt2/detection_loss‚2.35236714517369 epoch‚2 .loss/FRt2/detection_loss‚2.442648176150986 -loss/FRt4/detection_loss‚2.01582979299533 1loss/SRqqt1/detection_loss‚1.8751703345083766 1loss/SRqqt3/detection_loss‚1.8004466958667922 /loss/FRt3/assignment_loss‚3.760610498028037 2loss/SRqqt4/assignment_loss‚1.2795227529668762 /loss/FBt2/assignment_loss‚0.395786393303698 0loss/SRqqt2/detection_loss‚2.452580013058402 .loss/FBt1/detection_loss‚3.453968198461966 /loss/FBt4/detection_loss‚0.6955076596693487 % _timestamp‚1.7884157429800985e+09 /loss/FRt2/assignment_loss‚4.202464215363128 %loss/total_loss‚47.45155962230646 _step‚129 1loss/FBt3/assignment_loss‚0.10570488630938121 1loss/FBt4/assignment_loss‚0.09887553282913622 /loss/FRt3/detection_loss‚2.4323862272280232 /loss/FBt3/detection_loss‚1.9153268053418113 _runtime‚1324.629325675 /loss/FRt1/assignment_loss‚4.004943419070471 2loss/SRqqt2/assignment_loss‚2.3645020669156853£dùÏZ´ Ô /loss/FRt2/assignment_loss‚4.202464215363128 %loss/total_loss‚47.45155962230646 _step‚129 1loss/FBt3/assignment_loss‚0.10570488630938121 1loss/FBt4/assignment_loss‚0.09887553282913622 /loss/FRt3/detection_loss‚2.4323862272280232 /loss/FBt3/detection_loss‚1.9153268053418113 _runtime‚1324.629325675 /loss/FRt1/assignment_loss‚4.004943419070471 2loss/SRqqt2/assignment_loss‚2.3645020669156853 /loss/FRt1/detection_loss‚2.1775083050841375 2loss/SRqqt1/assignment_loss‚1.9691703608592175 1loss/FBt1/assignment_loss‚0.41067718664205377 1loss/SRqqt4/detection_loss‚1.0779939099799756 trainer/global_step‚1599 /loss/FRt4/assignment_loss‚2.119378975552061 1loss/SRqqt3/assignment_loss‚2.048190070947875 -loss/FBt2/detection_loss‚2.35236714517369 epoch‚2 .loss/FRt2/detection_loss‚2.442648176150986 -loss/FRt4/detection_loss‚2.01582979299533 1loss/SRqqt1/detection_loss‚1.8751703345083766 1loss/SRqqt3/detection_loss‚1.8004466958667922 /loss/FRt3/assignment_loss‚3.760610498028037 2loss/SRqqt4/assignment_loss‚1.2795227529668762 /loss/FBt2/assignment_loss‚0.395786393303698 0loss/SRqqt2/detection_loss‚2.452580013058402 .loss/FBt1/detection_loss‚3.453968198461966 /loss/FBt4/detection_loss‚0.6955076596693487 % _timestamp‚1.7884157429800985e+09á]í‘žµ ‚(:“ ‰žäÔØ®¨ gpu.0.gpu‚52.0 gpu.0.memory‚6, gpu.0.memoryAllocated‚12.165603637695312* gpu.0.memoryAllocatedBytes‚ 10450173952 gpu.0.temp‚36.0 gpu.0.powerWatts‚148.485' gpu.0.enforcedPowerLimitWatts‚500.0) gpu.0.powerPercent‚29.697000000000003 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚08èDoX¶ ‚(:Í ‰žäÔøž« disk.drbd1716.out‚0 disk.drbd2080.out‚0 disk.nvme0n1.in‚ 11.80859375 disk.sda2.in‚0$ disk./.usageGB‚250.7984619140625 disk.dm-1.in‚0 disk.dm-1.out‚ 51.8984375 disk.rbd0.in‚0 disk.sda2.out‚0$ memory_percent‚84.94889736175537) proc.memory.availableMB‚ 2465.97265625* proc.memory.percent‚17.096257209777832 network.recv‚534602 disk.drbd2265.out‚0 disk.rbd0.out‚0% disk.nvme0n1.out‚5156.16650390625 disk.drbd2266.out‚0 network.sent‚ 3.006568e+06 disk.drbd2080.in‚0 disk.drbd2266.in‚0# proc.memory.rssMB‚ 2801.05078125 cpu‚26.117903775354385* disk./.usagePercent‚16.831295212978606 disk.drbd2265.in‚0 disk.sda.out‚ 51.8984375 disk.drbd1716.in‚0 disk.sda1.in‚0 proc.cpu.threads‚17 disk.sda.in‚0 disk.sda1.out‚0 disk.rbd1.in‚0 disk.dm-0.in‚ 11.80859375 disk.rbd1.out‚ 2.08349609375" disk.dm-0.out‚5156.29150390625! disk.nvme0c0n1.in‚ 11.80859375' disk.nvme0c0n1.out‚5156.16650390625 disk.sda3.in‚0 disk.sda3.out‚ 51.8984375QnŠ“· ‚(:ˆ ˜žäÔë㊨ gpu.0.gpu‚23.0 gpu.0.memory‚3, gpu.0.memoryAllocated‚12.165603637695312* gpu.0.memoryAllocatedBytes‚ 10450173952 gpu.0.temp‚35.0 gpu.0.powerWatts‚119.619' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚23.9238 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0Šœ?*P¸ ‚(:Å ˜žäÔªúšª disk.sda3.in‚0 disk.dm-1.in‚0 disk.rbd1.out‚ 2.146484375 disk.dm-0.in‚ 11.80859375 disk.rbd0.in‚0$ memory_percent‚84.94746685028076 disk.sda1.out‚0 disk.rbd1.in‚0! disk.nvme0c0n1.in‚ 11.80859375 disk.drbd2080.in‚0 network.recv‚539367* disk./.usagePercent‚16.833593392613484 disk.drbd2266.out‚0 disk.dm-1.out‚ 52.7421875 cpu‚26.11338206812648 proc.cpu.threads‚17 disk.sda2.in‚0 disk.sda3.out‚ 52.7421875 disk.drbd1716.in‚0* proc.memory.percent‚17.096257209777832 disk.drbd1716.out‚0 disk.drbd2080.out‚0 network.sent‚ 3.093149e+06% disk./.usageGB‚250.83270645141602 disk.sda2.out‚0 disk.drbd2266.in‚0% disk.nvme0c0n1.out‚5218.689453125 disk.sda.out‚ 52.7421875 disk.drbd2265.in‚0 disk.sda.in‚0# disk.nvme0n1.out‚5218.689453125 disk.drbd2265.out‚0 disk.rbd0.out‚0 disk.nvme0n1.in‚ 11.80859375# proc.memory.rssMB‚ 2801.05078125 disk.sda1.in‚0 disk.dm-0.out‚5218.814453125) proc.memory.availableMB‚ 2466.20703125°àP2¹ - _wandbruntime‚1363  _runtime‚13638V~«Åº ¿ & lr-AdamW/pg1‚0.0017087934640895213 & lr-AdamW/pg2‚0.0017087934640895213 trainer/global_step‚1649 # _timestamp‚1.78841578163537e+09 _runtime‚1363.285827128 _step‚130ÄŠÐxÊ» Ä & lr-AdamW/pg1‚0.0017087934640895213 & lr-AdamW/pg2‚0.0017087934640895213 trainer/global_step‚1649 # _timestamp‚1.78841578163537e+09 _runtime‚1363.285827128 _step‚130‚Š)K2¼ - _wandbruntime‚1363  _runtime‚1363ÂÆŸ”Ä'½ ¾O JBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚213 _step‚131 PCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚23778608 _MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 1279885554688 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚24400 WEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 1227439620096 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 435025920 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚20037 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚3 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 239283712 SDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 9902751744 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚24127 dQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚94715939334144 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 248172032 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚213 _runtime‚1363.285935803 E8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚54830298 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 169267712 TADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚92349277110145 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 446693376 G?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚213 L?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚31051690 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚24127 [IDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 1227608887808 TADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚94954699301376 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚2605 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 @8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚644 O@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 9679899136 ZHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 1227439620096 M@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚54850337 cQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 1280100065792 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 435025920 YFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚95995800116224 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 186438144 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚24772608 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 O@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 9373790789 _LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚91128416180224 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 SGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚8159625 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 L=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 9679899136 ]JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚95996039399936 XJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 165393016 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚17170432 QDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚22444870 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚452 ZHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 1220877969281 N?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 9902751744 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 446693376 TGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚31071727 PCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚23778610 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 ^LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 1221043362297 \IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚93727276851712 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚20039 [HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚91128399140864 JBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚431 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚2 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚14284796 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚431 Q>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚94954699301376 _LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚93727276851712 VGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 9283702784 H?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚1003 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 $ _timestamp‚1.788415781636102e+09 [HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚93727259681280 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 L?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚23778608 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚17170432 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 144203776 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 XEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚93727259681280 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 XEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚92349459542521 TFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 446693376 E=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚644 M@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚22444418 IDeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚2605 SEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 313315328 QBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 9902751744 XIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 9456058368 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚8160074 NFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚985 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 D>DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 169267712 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚24400 ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 214511104 BDeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚24127 [IDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 1227608887808 TADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚94954699301376 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚2605 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 @8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚644 O@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 9679899136 ZHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 1227439620096 M@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚54850337 cQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 1280100065792 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 435025920 YFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚95995800116224 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 186438144 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚24772608 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 O@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 9373790789 _LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚91128416180224 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 SGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚8159625 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 L=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 9679899136 ]JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚95996039399936 XJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 165393016 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚17170432 QDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚22444870 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚452 ZHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 1220877969281 N?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 9902751744 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 446693376 TGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚31071727 PCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚23778610 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 ^LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 1221043362297 \IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚93727276851712 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚20039 [HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚91128399140864 JBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚431 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚2 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚14284796 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚431 Q>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚94954699301376 _LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚93727276851712 VGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 9283702784 H?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚1003 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 $ _timestamp‚1.788415781636102e+09 [HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚93727259681280 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 L?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚23778608 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚17170432 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 144203776 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 XEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚93727259681280 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 XEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚92349459542521 TFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 446693376 E=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚644 M@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚22444418 IDeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚2605 SEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 313315328 QBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 9902751744 XIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 9456058368 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚8160074 NFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚985 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 D>DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 169267712 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚24400 ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 214511104 B6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚644 R?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚92403863223989 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚431 FDeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 186451456 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17170432 NADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚23792610 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 182432584 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚644 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚213 PBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 435025920 ]JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚91182280847360 _step‚132 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚644 G:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚54882835 QBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 9283702784 REDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚31090225 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24772608 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 BDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 9679899136 XIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 9456058368 VCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚93782623370752 XFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 1228164974080 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1003 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 QBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 9902751744 trainer/global_step‚1649 [HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚96052736509952 bODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚94771873773568 VHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 165393224 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚2 YFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚93782623370752 TEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 9283702784 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚14293187 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚431 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚213 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚470 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚3 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚17039360 O@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 9902751744 L=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 9902751744 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 446693376 QCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 313315328 WIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 446693376 K>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚22457620 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚24400 ZGDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚93782640541184 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 169281024 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 TEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 8980994048 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚24400 S@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚95010974796288 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 NADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚31070162 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚431 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚213 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚644 OADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 186451456 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚20065 VCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚95010974796288 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚24127 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚467 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 243464704 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 431144269 R?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚95010788344832 G:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚54862770 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚2605 LDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚985 ^KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚94771849000960 K>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚54882835 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚20063 NADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚23792608 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 446693376 % _timestamp‚1.7884157823496048e+09 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚20063 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚23792608 J=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚31070162 ]JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚93782640541184 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚8164903 ]KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 1280644044288 M>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 9373790789 REDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚14293184 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚17170432 SEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 435025920 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 144203776 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 \JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 1228334255104 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 FDeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚22457620 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚24400 ZGDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚93782640541184 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 169281024 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 TEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 8980994048 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚24400 S@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚95010974796288 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 NADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚31070162 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚431 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚213 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚644 OADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 186451456 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚20065 VCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚95010974796288 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚24127 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚467 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 243464704 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 431144269 R?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚95010788344832 G:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚54862770 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚2605 LDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚985 ^KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚94771849000960 K>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚54882835 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚20063 NADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚23792608 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 446693376 % _timestamp‚1.7884157823496048e+09 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚20063 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚23792608 J=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚31070162 ]JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚93782640541184 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚8164903 ]KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 1280644044288 M>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 9373790789 REDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚14293184 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚17170432 SEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 435025920 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 144203776 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 \JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 1228334255104 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 F6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚644 R?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚92403863223989 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚431 FDeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 186451456 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17170432 NADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚23792610 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 182432584 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚644 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚213 PBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 435025920 ]JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚91182280847360 _step‚132 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚644 G:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚54882835 QBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 9283702784 REDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚31090225 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24772608 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 BDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 9679899136 XIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 9456058368 VCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚93782623370752 XFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 1228164974080 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1003 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 QBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 9902751744 trainer/global_step‚1649 [HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚96052736509952 bODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚94771873773568 VHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 165393224 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚2 YFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚93782623370752 TEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 9283702784 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚14293187 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚431 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚213 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚470 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚3 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚17039360 O@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 9902751744 L=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 9902751744 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 446693376 QCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 313315328„½€c¹2 - _wandbruntime‚1364  _runtime‚1364 ®^Xà Ò 0loss/FRt3/assignment_loss‚3.9514086261972206 0loss/FRt4/assignment_loss‚1.8668307662010193 % _timestamp‚1.7884157823517084e+09 _runtime‚1364.000934396 0loss/SRqqt2/detection_loss‚2.335447778562447 /loss/FBt2/detection_loss‚2.8551608681678773 .loss/FBt3/detection_loss‚2.344298242032528 epoch‚2 trainer/global_step‚1649 1loss/FBt1/assignment_loss‚0.17026112039628274 1loss/FBt4/assignment_loss‚0.06217984017852387 -loss/FRt3/detection_loss‚2.59858482147192 0loss/SRqqt1/detection_loss‚1.971467275931997 2loss/SRqqt4/assignment_loss‚1.2406571751205826 %loss/total_loss‚47.50320614046743 /loss/FRt1/assignment_loss‚3.952619125103128 0loss/FBt2/assignment_loss‚0.2110179477953352 1loss/FBt3/assignment_loss‚0.23405668109771796 1loss/SRqqt4/detection_loss‚0.9891594293520605 .loss/FBt1/detection_loss‚2.558445692062378 1loss/SRqqt3/assignment_loss‚1.984738345920425 /loss/FRt2/assignment_loss‚4.128850166614239 2loss/SRqqt1/assignment_loss‚2.0095933596196685 /loss/FBt4/detection_loss‚0.8949073838398737 2loss/SRqqt2/assignment_loss‚2.2147250701854757 /loss/FRt1/detection_loss‚2.3419056658943496 /loss/FRt2/detection_loss‚2.6111523440250983 /loss/FRt4/detection_loss‚1.9978790543489642 0loss/SRqqt3/detection_loss‚1.977859360348318 _step‚133Ÿv]Ä × /loss/FRt1/assignment_loss‚3.952619125103128 0loss/FBt2/assignment_loss‚0.2110179477953352 1loss/FBt3/assignment_loss‚0.23405668109771796 1loss/SRqqt4/detection_loss‚0.9891594293520605 .loss/FBt1/detection_loss‚2.558445692062378 1loss/SRqqt3/assignment_loss‚1.984738345920425 /loss/FRt2/assignment_loss‚4.128850166614239 2loss/SRqqt1/assignment_loss‚2.0095933596196685 /loss/FBt4/detection_loss‚0.8949073838398737 2loss/SRqqt2/assignment_loss‚2.2147250701854757 /loss/FRt1/detection_loss‚2.3419056658943496 /loss/FRt2/detection_loss‚2.6111523440250983 /loss/FRt4/detection_loss‚1.9978790543489642 0loss/SRqqt3/detection_loss‚1.977859360348318 _step‚133 0loss/FRt3/assignment_loss‚3.9514086261972206 0loss/FRt4/assignment_loss‚1.8668307662010193 % _timestamp‚1.7884157823517084e+09 _runtime‚1364.000934396 0loss/SRqqt2/detection_loss‚2.335447778562447 /loss/FBt2/detection_loss‚2.8551608681678773 .loss/FBt3/detection_loss‚2.344298242032528 epoch‚2 trainer/global_step‚1649 1loss/FBt1/assignment_loss‚0.17026112039628274 1loss/FBt4/assignment_loss‚0.06217984017852387 -loss/FRt3/detection_loss‚2.59858482147192 0loss/SRqqt1/detection_loss‚1.971467275931997 2loss/SRqqt4/assignment_loss‚1.2406571751205826 %loss/total_loss‚47.50320614046743…RÅ ‚(:… §žäÔ³¹¾¨ gpu.0.gpu‚1.0 gpu.0.memory‚0, gpu.0.memoryAllocated‚12.165603637695312* gpu.0.memoryAllocatedBytes‚ 10450173952 gpu.0.temp‚35.0 gpu.0.powerWatts‚119.48' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚23.896 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚02ÎIÆ ‚(:¾ §žäÔø€áª proc.cpu.threads‚17 disk.nvme0n1.in‚ 11.80859375 disk.sda.in‚0 disk.dm-1.out‚ 53.109375# disk.nvme0n1.out‚5277.498046875 disk.sda2.out‚0 disk.drbd2265.in‚0! disk.nvme0c0n1.in‚ 11.80859375 disk.drbd1716.in‚0 disk.drbd2080.out‚0* proc.memory.percent‚17.096257209777832 network.sent‚ 3.099682e+06 disk.sda1.in‚0 disk.drbd1716.out‚0 disk.sda1.out‚0 disk.drbd2266.in‚0% disk.nvme0c0n1.out‚5277.498046875 disk.dm-0.out‚5277.623046875# memory_percent‚84.9492073059082# proc.memory.rssMB‚ 2801.05078125 disk.dm-0.in‚ 11.80859375 disk.rbd0.in‚0 disk.rbd0.out‚0 cpu‚26.10907635210944) disk./.usagePercent‚16.83482990900386 disk.drbd2266.out‚0 disk.rbd1.out‚ 2.146484375' proc.memory.availableMB‚ 2465.921875 network.recv‚544507 disk.sda2.in‚0 disk.drbd2265.out‚0 disk.sda3.out‚ 53.109375 disk.dm-1.in‚0% disk./.usageGB‚250.85113143920898 disk.rbd1.in‚0 disk.sda.out‚ 53.109375 disk.sda3.in‚0 disk.drbd2080.in‚0€ê•“Ç ‚(:ˆ ¶žäÔª‰Ž¨ gpu.0.gpu‚40.0 gpu.0.memory‚4, gpu.0.memoryAllocated‚12.165603637695312* gpu.0.memoryAllocatedBytes‚ 10450173952 gpu.0.temp‚36.0 gpu.0.powerWatts‚165.209' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚33.0418 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0¢gÍOÈ ‚(:Ä ¶žäԢ¢ª disk.rbd0.out‚0% disk.nvme0n1.out‚5350.67041015625 disk.drbd2266.out‚0 disk.rbd1.out‚ 2.146484375 disk.sda3.in‚0 network.recv‚550815 disk.drbd2266.in‚0" disk.dm-0.out‚5350.79541015625 disk.drbd1716.in‚0$ memory_percent‚84.94770526885986 disk.sda1.out‚0* disk./.usagePercent‚16.835486312321652 disk.sda1.in‚0 disk.dm-1.in‚0# proc.memory.rssMB‚ 2801.05078125* proc.memory.percent‚17.096257209777832 disk.sda2.out‚0' disk.nvme0c0n1.out‚5350.67041015625 disk.dm-1.out‚53.59375 disk.sda2.in‚0 disk.drbd2080.out‚0! disk.nvme0c0n1.in‚ 11.80859375 proc.cpu.threads‚17% disk./.usageGB‚250.86091232299805 disk.drbd2080.in‚0 disk.dm-0.in‚ 11.80859375 disk.sda.in‚0 disk.sda.out‚53.59375) proc.memory.availableMB‚ 2466.16796875 disk.sda3.out‚53.59375 network.sent‚ 3.187246e+06 disk.rbd0.in‚0 disk.nvme0n1.in‚ 11.80859375 disk.drbd2265.in‚0 disk.rbd1.in‚0 disk.drbd2265.out‚0 disk.drbd1716.out‚0 cpu‚26.1049030647864±‘åšÉ ‚(:’ ÅžäÔ’ྨ gpu.0.gpu‚53.0 gpu.0.memory‚6, gpu.0.memoryAllocated‚12.165603637695312* gpu.0.memoryAllocatedBytes‚ 10450173952 gpu.0.temp‚35.0 gpu.0.powerWatts‚119.86' gpu.0.enforcedPowerLimitWatts‚500.0) gpu.0.powerPercent‚23.971999999999998 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0Ðæ¶UWÊ ‚(:Ì ÅžäÔ‚ÚΪ network.recv‚554664 disk.drbd2080.out‚0 disk.sda2.in‚0 cpu‚26.101314377479078 disk.sda1.out‚0 disk.drbd2265.in‚0 disk.sda3.in‚0 disk.rbd0.in‚0 disk.rbd0.out‚0 disk.drbd2266.in‚0! disk.nvme0c0n1.in‚ 11.80859375% disk.nvme0n1.out‚5432.88134765625 disk.sda2.out‚0 network.sent‚ 3.192654e+06 disk.drbd2265.out‚0 disk.sda1.in‚0# proc.memory.rssMB‚ 2801.05078125 disk.rbd1.out‚ 2.146484375 disk.dm-0.in‚ 11.80859375 disk.sda.in‚0 disk.drbd2080.in‚0) proc.memory.availableMB‚ 2466.16015625 disk.nvme0n1.in‚ 11.80859375* disk./.usagePercent‚16.837107352028465% disk./.usageGB‚250.88506698608398" disk.dm-0.out‚5433.00634765625 disk.drbd1716.in‚0$ memory_percent‚84.94775295257568 disk.sda.out‚ 54.1171875 disk.dm-1.in‚0 disk.drbd2266.out‚0* proc.memory.percent‚17.096257209777832 proc.cpu.threads‚17 disk.rbd1.in‚0' disk.nvme0c0n1.out‚5432.88134765625 disk.sda3.out‚ 54.1171875 disk.dm-1.out‚ 54.1171875 disk.drbd1716.out‚0Õpm:2Ë - _wandbruntime‚1402  _runtime‚1402²Ç:ÄÌ ¾ % lr-AdamW/pg2‚0.001693092867846318 trainer/global_step‚1699 % _timestamp‚1.7884158212139099e+09 _runtime‚ 1402.86327573 _step‚134 % lr-AdamW/pg1‚0.0016930928678463184–ÉÍ Ã % lr-AdamW/pg1‚0.001693092867846318 % lr-AdamW/pg2‚0.001693092867846318 trainer/global_step‚1699 % _timestamp‚1.7884158212139099e+09 _runtime‚ 1402.86327573 _step‚134†IéÆå2Î - _wandbruntime‚1402  _runtime‚1402MÄ ÎÏ ¾O JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 _LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚96495461325312 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚24400 @8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚644 L?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚24478608 XEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚97759365740544 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚20039 NFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚985 XJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 165393016 N?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 9902751744 [HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚96495444154880 TGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚31998679 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚482 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 QBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 9902751744 MFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚35 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚479 dQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚97512661305344 _step‚135 WEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 1263735147520 QDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚23105570 cQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 1318026782208 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 TADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚95078609974353 _runtime‚ 1402.86783033 JBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚431 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚14704346 UGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 431144269 L=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 9679899136 `MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚97512636532736 TFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 446693376 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚20037 SDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 9283702784 PCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚31978642 C;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚644 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚2 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 435025920 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 D>DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 ZHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 1256977476689 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚24400 XEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚95078792406729 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 IDeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚24127 [IDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 1263904415232 ^LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 1263904415232 H?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚1003 _LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚93821649537024 ^LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 1257142869705 VGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 9283702784 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚24127 M@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚23105088 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚24772608 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚431 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 169267712 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚20037 E=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚644 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚3 XEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚96495444154880 Q>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚97759179302400 TGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚24478610 UFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 9456058368 JBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚213 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚8401224 ]JDeviceStatsMonitor.on_train_batch_starZâtïöt/inactive_split_bytes.all.allocated‚98830688087552 BDeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚2605 YFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚98830436220928 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚20039 \IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚96495461325312 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 PCDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated‚31998679 TADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚97759179302400 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 435025920 PCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚24478608 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚2605 L?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚31978642 M@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚56477289IÖ*èÉ'Ð ÃO G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚2605 YFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚98830436220928 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚20039 \IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚96495461325312 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 PCDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated‚31998679 TADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚97759179302400 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 435025920 PCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚24478608 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚2605 L?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚31978642 M@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚56477289 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 _LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚96495461325312 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚24400 @8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚644 L?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚24478608 XEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚97759365740544 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚20039 NFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚985 XJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 165393016 N?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 9902751744 [HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚96495444154880 TGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚31998679 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚482 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 QBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 9902751744 MFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚35 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚479 dQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚97512661305344 _step‚135 WEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 1263735147520 QDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚23105570 cQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 1318026782208 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 TADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚95078609974353 _runtime‚ 1402.86783033 JBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚431 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚14704346 UGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 431144269 L=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 9679899136 `MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚97512636532736 TFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 446693376 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚20037 SDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 9283702784 PCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚31978642 C;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚644 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚2 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 435025920 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 D>DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 ZHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 1256977476689 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚24400 XEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚95078792406729 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 IDeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚24127 [IDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 1263904415232 ^LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 1263904415232 H?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚1003 _LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚93821649537024 ^LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 1257142869705 VGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 9283702784 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚24127 M@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚23105088 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚24772608 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚431 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 169267712 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚20037 E=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚644 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚3 XEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚96495444154880 Q>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚97759179302400 TGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚24478610 UFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 9456058368 JBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚213 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚8401224 ]JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚98830688087552 BDeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 J;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 9679899136 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚24492608 SEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 435025920 aODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 1318783386624 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 R?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚95133196088197 M>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 9373790789 XFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 1257698923397 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 YFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚93875497164800 J=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚31997114 LDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚985 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚213 trainer/global_step‚1699 % _timestamp‚1.7884158219028478e+09 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 SDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 9456058368 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚2 ]JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚96550825014784 ^KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚97568570972160 QBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 9902751744 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 446693376 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚644 _runtime‚1403.552715745 UCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 1264460501504 M>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 9679899136 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚20065 YFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚96550807844352 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 144203776 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 446693376 C6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚56489722 QBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 9283702784 XIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 9456058368 YGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 1264629782528 VCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚97815454797312 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 169281024 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚488 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 VHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 165393224 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚213 _step‚136 PBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 435025920 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 182432584 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚20063 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1003 REDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚14712734 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 O@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 9902751744 L=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 9902751744 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 ZGDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚96550825014784 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 431144269 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17170432 OADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 186451456 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚8405489 ]KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 1318556305920 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚644 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚24400 S@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚97815454797312 VCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚96550807844352 FDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 NADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚31997114 G:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚56509787 NADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚32017177 [MDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚ 227080704 \JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 1257864316621 L>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 186451456 \JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 1264629782528 K>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚56509787 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚14712737 WDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚98887127278080 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚644 G:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚56489722 XFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 1264460501504 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚644 K>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚23118223 VGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 9456058368 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚20063 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚17170432? Y¤Ô&Ó ÎM YGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 1264629782528 VCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚97815454797312 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 169281024 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚488 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 VHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 165393224 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚213 _step‚136 PBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 435025920 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 182432584 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚20063 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1003 REDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚14712734 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 O@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 9902751744 L=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 9902751744 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 ZGDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚96550825014784 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 431144269 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17170432 OADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 186451456 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚8405489 ]KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 1318556305920 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚644 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚24400 S@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚97815454797312 VCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚96550807844352 FDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 NADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚31997114 G:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚56509787 NADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚32017177 [MDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚ 227080704 \JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 1257864316621 L>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 186451456 \JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 1264629782528 K>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚56509787 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚14712737 WDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚98887127278080 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚644 G:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚56489722 XFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 1264460501504 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚644 K>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚23118223 VGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 9456058368 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚20063 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚17170432 TEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 9283702784 QCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 313315328 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 248172032 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 FDeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 J;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 9679899136 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚24492608 SEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 435025920 aODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 1318783386624 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 R?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚95133196088197 M>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 9373790789 XFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 1257698923397 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 YFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚93875497164800 J=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚31997114 LDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚985 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚213 trainer/global_step‚1699 % _timestamp‚1.7884158219028478e+09 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 SDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 9456058368 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚2 ]JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚96550825014784 ^KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚97568570972160 QBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 9902751744 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 446693376 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚644 _runtime‚1403.552715745 UCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 1264460501504 M>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 9679899136 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚20065 YFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚96550807844352 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 144203776 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 446693376 C6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚56489722 QBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 9283702784 XIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 9456058368ˆ ? 2Ô - _wandbruntime‚1403  _runtime‚1403ÅR Õ Ð 0loss/FBt1/assignment_loss‚0.1828458687564957 /loss/FRt2/detection_loss‚2.5346109122037888 .loss/FRt3/detection_loss‚2.434232880552131 /loss/FBt3/detection_loss‚2.1903385190587294 &loss/total_loss‚45.798265873157945 2loss/SRqqt3/assignment_loss‚1.9688877920595418 0loss/SRqqt2/detection_loss‚2.302517927189668 _step‚137 0loss/FBt4/assignment_loss‚0.0976734817992444 epoch‚2 0loss/FRt1/assignment_loss‚3.7817730669881784 2loss/SRqqt1/assignment_loss‚1.8649053287453HïóAM357 1loss/SRqqt1/detection_loss‚1.6752031419129498 1loss/SRqqt3/detection_loss‚1.6918547709958742 0loss/SRqqt4/detection_loss‚1.116243951022625 trainer/global_step‚1699 % _timestamp‚1.7884158219044766e+09 _runtime‚1403.553721854 /loss/FRt2/assignment_loss‚4.028887563943863 2loss/SRqqt2/assignment_loss‚2.6334082648158073 2loss/SRqqt4/assignment_loss‚1.5086700997100426 -loss/FBt2/assignment_loss‚0.1586698025702 0loss/FBt3/assignment_loss‚0.2629159893863528 .loss/FBt1/detection_loss‚2.178896106424786 0loss/FRt3/assignment_loss‚3.4011399903929376 /loss/FRt4/assignment_loss‚2.379781484603882 .loss/FRt1/detection_loss‚2.235595124828465 /loss/FBt2/detection_loss‚2.6747270609651292 /loss/FRt4/detection_loss‚1.7635395631151296 /loss/FBt4/detection_loss‚0.7309471811167896Si…[Ö Õ /loss/FBt4/detection_loss‚0.7309471811167896 0loss/FBt1/assignment_loss‚0.1828458687564957 /loss/FRt2/detection_loss‚2.5346109122037888 .loss/FRt3/detection_loss‚2.434232880552131 /loss/FBt3/detection_loss‚2.1903385190587294 &loss/total_loss‚45.798265873157945 2loss/SRqqt3/assignment_loss‚1.9688877920595418 0loss/SRqqt2/detection_loss‚2.302517927189668 _step‚137 0loss/FBt4/assignment_loss‚0.0976734817992444 epoch‚2 0loss/FRt1/assignment_loss‚3.7817730669881784 2loss/SRqqt1/assignment_loss‚1.8649053287453357 1loss/SRqqt1/detection_loss‚1.6752031419129498 1loss/SRqqt3/detection_loss‚1.6918547709958742 0loss/SRqqt4/detection_loss‚1.116243951022625 trainer/global_step‚1699 % _timestamp‚1.7884158219044766e+09 _runtime‚1403.553721854 /loss/FRt2/assignment_loss‚4.028887563943863 2loss/SRqqt2/assignment_loss‚2.6334082648158073 2loss/SRqqt4/assignment_loss‚1.5086700997100426 -loss/FBt2/assignment_loss‚0.1586698025702 0loss/FBt3/assignment_loss‚0.2629159893863528 .loss/FBt1/detection_loss‚2.178896106424786 0loss/FRt3/assignment_loss‚3.4011399903929376 /loss/FRt4/assignment_loss‚2.379781484603882 .loss/FRt1/detection_loss‚2.235595124828465 /loss/FBt2/detection_loss‚2.6747270609651292 /loss/FRt4/detection_loss‚1.7635395631151296‰îzÆž× ‚(:“ ÔžäÔ¥š©¨ gpu.0.gpu‚48.0 gpu.0.memory‚6, gpu.0.memoryAllocated‚12.165603637695312* gpu.0.memoryAllocatedBytes‚ 10450173952 gpu.0.temp‚36.0 gpu.0.powerWatts‚155.691' gpu.0.enforcedPowerLimitWatts‚500.0) gpu.0.powerPercent‚31.138199999999998 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0iÉQ’KØ ‚(:À ÔžäÔùµ¬«* disk./.usagePercent‚16.838689222582534 disk.drbd2266.in‚0 disk.drbd2266.out‚0 disk.rbd1.out‚ 2.146484375 disk.sda.in‚0 disk.sda3.out‚ 54.81640625 disk.drbd1716.in‚0$ memory_percent‚84.94815826416016! disk.nvme0n1.out‚ 5513.0859375 disk.sda1.in‚0 disk.sda1.out‚0% disk./.usageGB‚250.90863800048828 disk.sda2.out‚0 disk.sda2.in‚0# disk.nvme0c0n1.out‚ 5513.0859375 disk.dm-0.in‚ 11.80859375 disk.dm-1.out‚ 54.81640625 disk.rbd0.out‚0 disk.drbd2265.out‚0 disk.nvme0n1.in‚ 11.80859375 disk.drbd2265.in‚0 disk.drbd1716.out‚0 disk.rbd0.in‚0# proc.memory.rssMB‚ 2801.05078125 cpu‚26.097263649214376 disk.sda.out‚ 54.81640625 disk.drbd2080.in‚0 network.recv‚558129 disk.rbd1.in‚0! disk.nvme0c0n1.in‚ 11.80859375 disk.sda3.in‚0* proc.memory.percent‚17.096257209777832 proc.cpu.threads‚17 network.sent‚ 3.198005e+06 disk.dm-0.out‚ 5513.2109375 disk.dm-1.in‚0 disk.drbd2080.out‚0& proc.memory.availableMB‚ 2466.09375àÞÙ ‚(:’ ãžäÔ¹­¥¨ gpu.0.gpu‚23.0 gpu.0.memory‚3, gpu.0.memoryAllocated‚12.165603637695312* gpu.0.memoryAllocatedBytes‚ 10450173952 gpu.0.temp‚36.0 gpu.0.powerWatts‚138.28' gpu.0.enforcedPowerLimitWatts‚500.0) gpu.0.powerPercent‚27.656000000000002 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0]ÍÝÓAÚ ‚(:¶ ãžäÔÚ¿ª network.sent‚ 3.28572e+06! disk.nvme0c0n1.in‚ 11.80859375$ memory_percent‚84.94818210601807 disk.dm-1.out‚ 55.203125 disk.rbd0.out‚0 disk.sda2.in‚0 disk.sda2.out‚0* proc.memory.percent‚17.096257209777832 disk.drbd1716.out‚0 disk.drbd2080.in‚0 disk.sda3.out‚ 55.203125) proc.memory.availableMB‚ 2466.08984375% disk./.usageGB‚250.91573333740234 disk.drbd2080.out‚0 disk.sda.in‚0 cpu‚26.093355491508927 proc.cpu.threads‚17* disk./.usagePercent‚16.839165396596222 disk.dm-1.in‚0 disk.drbd2266.out‚0 disk.rbd1.out‚ 2.146484375! disk.nvme0c0n1.out‚ 5585.78125 disk.dm-0.out‚ 5585.90625 disk.sda.out‚ 55.203125 disk.drbd1716.in‚0 disk.rbd0.in‚0 disk.nvme0n1.in‚ 11.80859375 disk.sda1.in‚0 disk.dm-0.in‚ 11.80859375 network.recv‚564531 disk.drbd2265.in‚0 disk.sda3.in‚0 disk.nvme0n1.out‚ 5585.78125 disk.sda1.out‚0 disk.drbd2266.in‚0 disk.drbd2265.out‚0 disk.rbd1.in‚0# proc.memory.rssMB‚ 2801.05078125€HÖ“Û ‚(:ˆ òžäÔÔ¸£¨ gpu.0.gpu‚52.0 gpu.0.memory‚6, gpu.0.memoryAllocated‚12.165603637695312* gpu.0.memoryAllocatedBytes‚ 10450173952 gpu.0.temp‚36.0 gpu.0.powerWatts‚127.973' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚25.5946 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0åÏBõTÜ ‚(:É òžäÔ‹žñª network.recv‚568892 disk.rbd1.in‚0' disk.nvme0c0n1.out‚5643.65185546875 disk.sda3.in‚0 disk.dm-1.out‚ 55.79296875* proc.memory.percent‚17.096257209777832 proc.cpu.threads‚17 disk.sda3.out‚ 55.79296875 network.sent‚ 3.291971e+06) disk./.usagePercent‚16.84061491125509 disk.drbd2265.in‚0 disk.drbd2265.out‚0# proc.memory.rssMB‚ 2801.05078125 cpu‚26.08914018465392$ disk./.usageGB‚250.9373321533203 disk.rbd1.out‚ 2.146484375" disk.dm-0.out‚5643.77685546875 disk.rbd0.out‚0% disk.nvme0n1.out‚5643.65185546875 disk.sda2.in‚0 disk.drbd2266.in‚0 disk.dm-1.in‚0 disk.nvme0n1.in‚ 11.80859375 disk.sda1.in‚0 disk.sda1.out‚0 disk.drbd2080.in‚0! disk.nvme0c0n1.in‚ 11.80859375 disk.sda.in‚0 disk.sda.out‚ 55.79296875# memory_percent‚84.9482536315918 disk.drbd1716.in‚0 disk.drbd1716.out‚0 disk.sda2.out‚0 disk.drbd2080.out‚0 disk.rbd0.in‚0 disk.drbd2266.out‚0 disk.dm-0.in‚ 11.80859375' proc.memory.availableMB‚ 2466.078125ÿƒuA2Ý - _wandbruntime‚1442  _runtime‚1442—;£XÆÞ À $ _timestamp‚1.788415860847678e+09 _runtime‚1442.497060851 _step‚138 & lr-AdamW/pg1‚0.0016767612331306746 & lr-AdamW/pg2‚0.0016767612331306746 trainer/global_step‚1749h(cËß Å & lr-AdamW/pg2‚0.0016767612331306746 trainer/global_step‚1749 $ _timestamp‚1.788415860847678e+09 _runtime‚1442.497060851 _step‚138 & lr-AdamW/pg1‚0.0016767612331306746Š»ˆM2à - _wandbruntime‚1442  _runtime‚1442ŒaاÍ'á ÇO M@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚23766217 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 YEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚100563845741568 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 144203776 cQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 1355930394624 TADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚97807942838561 UADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚100563659303424 BDeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 QCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 182432376 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 [HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚99263628628480 PCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚25178608 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚15123893 L?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚25178608 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚20039 IDeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚24127 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 169267712 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚24127 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 TFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 446693376 M@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚58104241 QBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 9902751744 R>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚100563659303424 O@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 9679899136 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 SDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 9902751744 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 169267712 TGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚25178610 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚15123896 ZFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚101665068096000 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 XEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚97808125270937 XIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 9456058368 SDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 9283702784 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚8642804 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 _LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚96514882893824 [HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚96514865854464 VGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 8980994048 VGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 9283702784 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚24400 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚483 SEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 313315328 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 N@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 186438144 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚17170432 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚2605 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 446693376 G?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚431 eQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚100309383276544 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 UGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 431144269 L=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 9679899136 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 435025920 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 E8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚58084202 TGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚32925631 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 245575168 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚20039 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚17170432 H?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚1003 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 _LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚99263645798912 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚431 L?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚32905594 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 435025920 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚2605ÀmFÈÒ'â ÌO BDeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 QCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 182432376 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 [HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚99263628628480 PCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚25178608 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚15123893 L?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚25178608 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚20039 IDeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚24127 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 169267712 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚24127 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 TFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 446693376 M@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚58104241 QBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 9902751744 R>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚100563659303424 O@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 9679899136 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 SDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 9902751744 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 169267712 TGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚25178610 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚15123896 ZFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚101665068096000 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 XEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚97808125270937 XIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 9456058368 SDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 9283702784 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚8642804 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 _LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚96514882893824 [HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚96514865854464 VGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 8980994048 VGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 9283702784 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚24400 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚483 SEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 313315328 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 N@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 186438144 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚17170432 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚2605 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 446693376 G?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚431 eQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚100309383276544 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 UGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 431144269 L=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 9679899136 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 435025920 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 E8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚58084202 TGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚32925631 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 245575168 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚20039 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚17170432 H?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚1003 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 _LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚99263645798912 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚431 L?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚32905594 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 435025920 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚2605 M@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚23766217 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 YEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚100563845741568 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 144203776 cQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 1355930394624 TADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚97807942838561 UADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚100563659303424‹iq2ã - _wandbruntime‚1443  _runtime‚1443nþ9±tä ÑM NADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚25192610 VCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚99318992317952 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚8646992 L>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 186451456 VHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 165393224 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚20065 OADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 186451456 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚431 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚24127 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚15132287 UCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 1300756028928 F8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚644 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17170432 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1003 XDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚101721752912896 aODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 1356684953088 TEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 8980994048 VGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 9456058368 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚20063 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚8647478 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 J=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚32924066 M>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 9679899136 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚644 FDeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 cODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚100365317715968 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚644 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 169281024 OBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚23779765 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24772608 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚431 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚17170432 _step‚140 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 169281024 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 248172032 ZGDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚99319009488384 TEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 9283702784 G:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚58116674 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 \JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 1293963824029 XFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 1293798430805 XFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 1300756028928 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_byte;€Ðcs.large_pool.peak‚ 144203776 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 M>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 9373790789 QBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 9902751744 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚24400 EDeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚23779276 LDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚985 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 182432584 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚213 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚431 trainer/global_step‚1749 YGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 1300925309952 PBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 435025920 \JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 1300925309952 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚2605 BDeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚58136739 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 YFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚96568730521600 WIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 446693376 G:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚58136739 QBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 9283702784 ]KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 1356459969536 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 446693376 REDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚32944129 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚213 C6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚58116674 YFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚99318992317952 NADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚32924066 REDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚15132284 R?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚97862528952405WìÝzÜ&å ÖM UCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 1300756028928 F8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚644 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17170432 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1003 XDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚101721752912896 aODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 1356684953088 TEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 8980994048 VGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 9456058368 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚20063 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚8647478 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 J=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚32924066 M>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 9679899136 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚644 FDeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 cODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚100365317715968 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚644 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 169281024 OBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚23779765 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24772608 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚431 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚17170432 _step‚140 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 169281024 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 248172032 ZGDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚99319009488384 TEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 9283702784 G:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚58116674 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 \JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 1293963824029 XFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 1293798430805 XFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 1300756028928 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 144203776 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 M>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 9373790789 QBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 9902751744 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚24400 EDeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚23779276 LDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚985 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 182432584 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚213 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚431 trainer/global_step‚1749 YGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 1300925309952 PBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 435025920 \JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 1300925309952 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚2605 BDeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚58136739 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 YFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚96568730521600 WIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 446693376 G:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚58136739 QBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 9283702784 ]KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 1356459969536 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 446693376 REDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚32944129 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚213 C6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚58116674 YFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚99318992317952 NADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚32924066 REDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚15132284 R?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚97862528952405 NADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚25192610 VCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚99318992317952 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚8646992 L>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 186451456 VHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 165393224 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚20065 OADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 186451456 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚431 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚24127 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚15132287ŒƒðÚÊ2æ - _wandbruntime‚1443  _runtime‚1443íÌgZç Ô %loss/total_loss‚46.88850788423062 _runtime‚1443.201151513 /loss/FRt3/assignment_loss‚3.615671418607235 1loss/FBt3/assignment_loss‚0.09746392106229905 /loss/FRt4/detection_loss‚1.6565070668333455 1loss/SRqqt2/detection_loss‚2.4457981349670725 _step‚141 epoch‚2 .loss/FRt2/assignment_loss‚4.44789283825801 0loss/FRt4/assignment_loss‚2.2150653211694014 2loss/SRqqt4/assignment_loss‚1.4434532712500794 /loss/FRt2/detection_loss‚2.6598887516902043 1loss/SRqqt4/detection_loss‚1.0788195083912153 /loss/FBt1/detection_loss‚2.2845804402905125 /loss/FBt2/detection_loss‚2.6067854629622564 0loss/FRt1/assignment_loss‚3.8735776289745614 /loss/FRt3/detection_loss‚2.7893025232478976 trainer/global_step‚1749 1loss/SRqqt1/assignment_loss‚1.989763885313137 1loss/SRqqt3/assignment_loss‚1.896760509139107 0loss/SRqqt3/detection_loss‚1.891995297644728 $ _timestamp‚1.788415861551953e+09 1loss/SRqqt1/detection_loss‚2.0912160958078774 1loss/SRqqt2/assignment_loss‚2.142426123357799 0loss/FBt1/assignment_loss‚0.1256179924621979 1loss/FBt4/assignment_loss‚0.06532645972747203 /loss/FBt3/detection_loss‚2.2365137338638306 /loss/FBt4/detection_loss‚0.7229380925879783 1loss/FBt2/assignment_loss‚0.27384791103476247 /loss/FRt1/detection_loss‚2.2372954955876425$N!“_è Ù /loss/FBt1/detection_loss‚2.2845804402905125 /loss/FBt2/detection_loss‚2.6067854629622564 0loss/FRt1/assignment_loss‚3.8735776289745614 /loss/FRt3/detection_loss‚2.7893025232478976 trainer/global_step‚1749 1loss/SRqqt1/assignment_loss‚1.989763885313137 1loss/SRqqt3/assignment_loss‚1.896760509139107 0loss/SRqqt3/detection_loss‚1.891995297644728 $ _timestamp‚1.788415861551953e+09 1loss/SRqqt1/detection_loss‚2.0912160958078774 1loss/SRqqt2/assignment_loss‚2.142426123357799 0loss/FBt1/assignment_loss‚0.1256179924621979 1loss/FBt4/assignment_loss‚0.06532645972747203 /loss/FBt3/detection_loss‚2.2365137338638306 /loss/FBt4/detection_loss‚0.7229380925879783 1loss/FBt2/assignment_loss‚0.27384791103476247 /loss/FRt1/detection_loss‚2.2372954955876425 %loss/total_loss‚46.88850788423062 _runtime‚1443.201151513 /loss/FRt3/assignment_loss‚3.615671418607235 1loss/FBt3/assignment_loss‚0.09746392106229905 /loss/FRt4/detection_loss‚1.6565070668333455 1loss/SRqqt2/detection_loss‚2.4457981349670725 _step‚141 epoch‚2 .loss/FRt2/assignment_loss‚4.44789283825801 0loss/FRt4/assignment_loss‚2.2150653211694014 2loss/SRqqt4/assignment_loss‚1.4434532712500794 /loss/FRt2/detection_loss‚2.6598887516902043 1loss/SRqqt4/detection_loss‚1.0788195083912153Géžé ‚(:“ ŸäԜ溨 gpu.0.gpu‚39.0 gpu.0.memory‚3, gpu.0.memoryAllocated‚12.165603637695312* gpu.0.memoryAllocatedBytes‚ 10450173952 gpu.0.temp‚35.0 gpu.0.powerWatts‚151.374' gpu.0.enforcedPowerLimitWatts‚500.0) gpu.0.powerPercent‚30.274800000000003 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0P± iWê ‚(:Ì ŸäÔø‹É« disk.drbd1716.out‚0) proc.memory.availableMB‚ 2466.29296875* proc.memory.percent‚17.096257209777832 disk.dm-0.in‚ 11.80859375 disk.sda1.in‚0 disk.drbd2265.out‚0 disk.rbd1.in‚0 disk.rbd1.out‚ 2.146484375 disk.sda1.out‚0 disk.drbd2266.out‚0 disk.drbd2265.in‚0 cpu‚26.08567558276934% disk./.usageGB‚250.96107482910156" disk.dm-0.out‚5694.37060546875 disk.nvme0n1.in‚ 11.80859375 disk.rbd0.out‚0 disk.drbd2266.in‚0 disk.sda3.out‚ 56.26171875 disk.dm-1.in‚0 disk.dm-1.out‚ 56.26171875 disk.drbd2080.out‚0$ memory_percent‚84.94694232940674# proc.memory.rssMB‚ 2801.05078125( disk./.usagePercent‚16.8422083021482! disk.nvme0c0n1.in‚ 11.80859375 proc.cpu.threads‚17 network.recv‚572797' disk.nvme0c0n1.out‚5694.24560546875 disk.sda2.in‚0 disk.sda2.out‚0 disk.sda3.in‚0 disk.drbd1716.in‚0 disk.drbd2080.in‚0 disk.rbd0.in‚0% disk.nvme0n1.out‚5694.24560546875 network.sent‚ 3.297231e+06 disk.sda.in‚0 disk.sda.out‚ 56.26171875.D2žë ‚(:“ ŸäÔ¤¢ó¨ gpu.0.gpu‚42.0 gpu.0.memory‚4, gpu.0.memoryAllocated‚12.165603637695312* gpu.0.memoryAllocatedBytes‚ 10450173952 gpu.0.temp‚36.0 gpu.0.powerWatts‚164.339' gpu.0.enforcedPowerLimitWatts‚500.0) gpu.0.powerPercent‚32.867799999999995 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0ÙÒ;Xì ‚(:Í ŸäÔ©Þ€¬ disk.drbd2265.out‚0 network.sent‚ 3.384108e+06 disk.sda3.out‚ 56.98046875% disk.nvme0n1.out‚5754.44091796875% disk./.usageGB‚250.98183822631836 disk.sda1.in‚0 disk.drbd2265.in‚0 network.recv‚577801 disk.drbd1716.out‚0 disk.rbd0.in‚0 disk.nvme0n1.in‚ 11.80859375 disk.sda1.out‚0 cpu‚26.08195513527554" disk.dm-0.out‚5754.56591796875 disk.dm-1.in‚0 disk.drbd1716.in‚0 disk.drbd2080.out‚0# proc.memory.rssMB‚ 2801.05078125! disk.nvme0c0n1.in‚ 11.80859375' disk.nvme0c0n1.out‚5754.44091796875 disk.dm-0.in‚ 11.80859375 disk.sda.out‚ 56.98046875 disk.drbd2080.in‚0 disk.rbd0.out‚0 disk.sda2.in‚0 disk.sda3.in‚0 disk.dm-1.out‚ 56.98046875$ memory_percent‚84.35349464416504( proc.memory.availableMB‚ 2563.5234375 disk.sda.in‚0 disk.drbd2266.out‚0* proc.memory.percent‚17.096257209777832 proc.cpu.threads‚17* disk./.usagePercent‚16.843601751157074 disk.rbd1.in‚0 disk.rbd1.out‚ 2.146484375 disk.sda2.out‚0 disk.drbd2266.in‚0ˆ¿Úž2í - _wandbruntime‚1481  _runtime‚1481Oq60Æî À & lr-AdamW/pg1‚0.0016598121957323367 & lr-AdamW/pg2‚0.0016598121957323367 trainer/global_step‚1799 $ _timestamp‚1.788415900125097e+09 _runtime‚1481.774496929 _step‚142íÒ1Ëï Å _step‚142 & lr-AdamW/pg1‚0.0016598121957323367 & lr-AdamW/pg2‚0.0016598121957323367 trainer/global_step‚1799 $ _timestamp‚1.788415900125097e+09 _runtime‚1481.774496929Žw—)x2ð - _wandbruntime‚1481  _runtime‚1481àSXÓ'ñ ÍO BDeviceStatsMonitor.on_train_batch_start/allocation.all.current‚20039 cQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 1393841023488 @8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚644 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 186438144 ZHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 1336326202368 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 248172032 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚2605 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚431 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 VGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 8980994048 [IDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 1336495470080 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 435025920 ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 208219648 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 E8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚59711154 SGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚8883414 TFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 446693376 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 435025920 YEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚100537458135145 E=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚644 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚213 O@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 9679899136 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚2605 SEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 313315328 aMDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚103106080475136 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 UGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 431144269 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 169267712 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚20037 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚15543443 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 VBDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated‚103368325742592 L=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 9679899136 TGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚33852583 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 trainer/global_step‚1799 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚24127 NFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚985 D>DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚2 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 446693376 JBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚213 ZKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 9456058368 UADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚103368139304448 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 232992256 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 144203776 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 JBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚431 _step‚143 M@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚24426857 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 _LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚99208116250624 ZHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 1329176491505 % _timestamp‚1.7884159001262534e+09 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚24400 WEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 1336326202368 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚3 eQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚103106105247744 UADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚100537275702769 [HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚99208099211264 XJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 165393016 N?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 9902751744 ]IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚102031830272512 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚15543446 O@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 9373790789 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 _runtime‚1481.775882234 XIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 9456058368 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚17170432 ^JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚104499946271232 `LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚102031830272512 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚17170432 QDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚24427346 L?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚25878608 ^LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 1329341884521 SDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 9902751744 ^LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 1336495470080 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚24400 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 H?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚1003 QBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 9902751744 C;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚644 SDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 9283702784 N@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 186438144 YEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚103368325742592 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚489 MFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚35 G?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚213 TGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚25878610 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 R>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚103368139304448 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 169267712 M@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚59731193 IDeviceStatsMonitor.on_train_batch_start/allocation.all.current‚20039 cQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ ­Þk#1393841023488 @8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚644 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 186438144 ZHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 1336326202368 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 248172032 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚2605 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚431 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 VGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 8980994048 [IDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 1336495470080 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 435025920 ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 208219648 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 E8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚59711154 SGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚8883414 TFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 446693376 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 435025920 YEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚100537458135145 E=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚644 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚213 O@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 9679899136 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚2605 SEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 313315328 aMDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚103106080475136 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 UGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 431144269 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 169267712 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚20037 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚15543443 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 VBDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated‚103368325742592 L=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 9679899136 TGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚33852583 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 trainer/global_step‚1799 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚24127 NFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚985 D>DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚2 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 446693376 JBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚213 ZKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 9456058368 UADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚103368139304448 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 232992256 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 144203776 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 JBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚431 _step‚143 M@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚24426857 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 _LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚99208116250624 ZHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 1329176491505 % _timestamp‚1.7884159001262534e+09 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚24400 WEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 1336326202368 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚3 eQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚103106105247744 UADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚100537275702769 [HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚99208099211264 XJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 165393016 N?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 9902751744 ]IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚102031830272512 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚15543446 O@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 9373790789 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 _runtime‚1481.775882234 XIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 9456058368 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚17170432 ^JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚104499946271232 `LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚102031830272512 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚17170432 QDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚24427346 L?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚25878608 ^LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 1329341884521 SDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 9902751744 ^LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 1336495470080 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚24400 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 H?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚1003 QBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 9902751744 C;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚644 SDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 9283702784 N@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 186438144 YEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚103368325742592 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚489 MFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚35 G?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚213 TGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚25878610 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 R>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚103368139304448 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 169267712 M@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚59731193 I6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚644 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚213 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚213 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚213 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24772608 \JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 1330063331437 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17170432 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚479 S?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚100591861816613 QBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 9902751744 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚2 M>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 9679899136 aODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 1394607443968 ]KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 1394386654720 M>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 9373790789 WIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 446693376 J=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚33851018 WCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚102087176791552 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1003 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚15551837 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚644 EDeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚59763691 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚8888599 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 169281024 FDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 VGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 9456058368 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 F8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 G:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚59763691 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚17039360 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 446693376 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚431 trainer/global_step‚1799 PDeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚24439954 NADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚25892610 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚24400 XIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 9456058368 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚17170432 \JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 1337220837376 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 L=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 9902751744 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 C6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚59743626 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚20063 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚8888120 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 245561856 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 169281024 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 ]JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚99261980917760 TEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 8980994048 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚644 NADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚33851018 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚24127 O@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 9902751744 SDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 9456058368 REDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚33871081 VHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 165393224 NADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚25892608 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚482 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚25892608 L>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 186451456 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 431144269 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚431 cODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚103162039687168 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 446693376Ì5_â&õ ÜM J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚25892608 L>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 186451456 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 431144269 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚431 cODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚103162039687168 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 446693376 PBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 435025920 REDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚15551834 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚431 S?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚103424228347904 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 \HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚104556647131136 % _timestamp‚1.7884159008150554e+09 T@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚103424414799360 _step‚144 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 144203776 B6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚644 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚213 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚213 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚213 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24772608 \JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 1330063331437 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17170432 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚479 S?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚100591861816613 QBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 9902751744 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚2 M>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 9679899136 aODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 1394607443968 ]KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 1394386654720 M>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 9373790789 WIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 446693376 J=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚33851018 WCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚102087176791552 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1003 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚15551837 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚644 EDeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚59763691 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚8888599 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 169281024 FDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 VGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 9456058368 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 F8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 G:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚59763691 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚17039360 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 446693376 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚431 trainer/global_step‚1799 PDeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚24439954 NADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚25892610 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚24400 XIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 9456058368 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚17170432 \JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 1337220837376 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 L=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 9902751744 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 C6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚59743626 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚20063 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚8888120 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 245561856 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 169281024 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 ]JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚99261980917760 TEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 8980994048 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚644 NADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚33851018 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚24127 O@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 9902751744 SDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 9456058368 REDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚33871081 VHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 165393224 NADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚25892608 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚482ïe»Û2ö - _wandbruntime‚1482  _runtime‚1482×}™]÷ × .loss/FRt3/assignment_loss‚3.55643370483495 0loss/FRt2/assignment_loss‚4.1313695593012705 0loss/FRt4/assignment_loss‚2.0967484647969163 /loss/FBt1/detection_loss‚2.3314186075459356 _runtime‚1482.466412161 _step‚145 1loss/FBt3/assignment_loss‚0.15896680280360775 epoch‚2 2loss/SRqqt1/assignment_loss‚1.9339707687369778 2loss/SRqqt4/assignment_loss‚1.3602534511361186 .loss/FRt1/detection_loss‚2.151220268324802 .loss/FBt2/detection_loss‚2.840760111808777 .loss/FBt3/detection_loss‚2.419735634151627 0loss/FRt1/assignment_loss‚3.9807732920897636 /loss/FRt4/detection_loss‚1.9208951593343033 1loss/SRqqt1/detection_loss‚1.7984873628259723 1loss/SRqqt3/detection_loss‚1.8925650201316149 1loss/FBt1/assignment_loss‚0.23272210546786917 0loss/FBt2/assignment_loss‚0.2635161558631808 1loss/FBt4/assignment_loss‚0.05283760317443416 trainer/global_step‚1799 1loss/SRqqt2/assignment_loss‚2.403612008290504 2loss/SRqqt3/assignment_loss‚1.9032728688193943 1loss/SRqqt4/detection_loss‚1.0752369963372026 /loss/FBt4/detection_loss‚0.7627865849528461 % _timestamp‚1.7884159008172233e+09 &loss/total_loss‚46.883844049290076 /loss/FRt2/detection_loss‚2.5673826676275997 /loss/FRt3/detection_loss‚2.5716184170185765 1loss/SRqqt2/detection_loss‚2.4772604339158355†šÒ×bø Ü 1loss/SRqqt2/assignment_loss‚2.403612008290504 2loss/SRqqt3/assignment_loss‚1.9032728688193943 1loss/SRqqt4/detection_loss‚1.0752369963372026 /loss/FBt4/detection_loss‚0.7627865849528461 % _timestamp‚1.7884159008172233e+09 &loss/total_loss‚46.883844049290076 /loss/FRt2/detection_loss‚2.5673826676275997 /loss/FRt3/detection_loss‚2.5716184170185765 1loss/SRqqt2/detection_loss‚2.4772604339158355 .loss/FRt3/assignment_loss‚3.55643370483495 0loss/FRt2/assignment_loss‚4.1313695593012705 0loss/FRt4/assignment_loss‚2.0967484647969163 /loss/FBt1/detection_loss‚2.3314186075459356 _runtime‚1482.466412161 _step‚145 1loss/FBt3/assignment_loss‚0.15896680280360775 epoch‚2 2loss/SRqqt1/assignment_loss‚1.9339707687369778 2loss/SRqqt4/assignment_loss‚1.3602534511361186 .loss/FRt1/detection_loss‚2.151220268324802 .loss/FBt2/detection_loss‚2.840760111808777 .loss/FBt3/detection_loss‚2.419735634151627 0loss/FRt1/assignment_loss‚3.9807732920897636 /loss/FRt4/detection_loss‚1.9208951593343033 1loss/SRqqt1/detection_loss‚1.7984873628259723 1loss/SRqqt3/detection_loss‚1.8925650201316149 1loss/FBt1/assignment_loss‚0.23272210546786917 0loss/FBt2/assignment_loss‚0.2635161558631808 1loss/FBt4/assignment_loss‚0.05283760317443416 trainer/global_step‚1799‘i _C“ù ‚(:ˆ ŸŸäÔÊ‚¨¨ gpu.0.gpu‚44.0 gpu.0.memory‚5, gpu.0.memoryAllocated‚12.165603637695312* gpu.0.memoryAllocatedBytes‚ 10450173952 gpu.0.temp‚36.0 gpu.0.powerWatts‚167.828' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚33.5656 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0ËœòØ.ú ‚(:Î ŸŸäÔÊɶª disk.drbd1716.out‚0 disk.rbd0.out‚0' disk.nvme0c0n1.out‚5811.85498046875 disk.sda.in‚0 proc.cpu.threads‚17% disk./.usageGB‚251.01129531860352 disk.sda1.in‚0 disk.sda1.out‚0 disk.sda3.out‚ 57.41796875 disk.dm-1.out‚ 57.41796875 disk.drbd2080.out‚0 disk.sda.out‚ 57.41796875# proc.memory.rssMB‚ 2801.05078125 disk.rbd0.in‚0 disk.sda2.out‚0 disk.drbd2266.in‚0 disk.rbd1.in‚0$ memory_percent‚84.35521125793457 cpu‚26.078546153767064 network.sent‚ `°+ 3.390652e+06% disk.nvme0n1.out‚5811.85498046875 disk.drbd2266.out‚0* proc.memory.percent‚17.096257209777832* disk./.usagePercent‚16.845578641336477 disk.drbd2080.in‚0 disk.drbd2265.out‚0 disk.rbd1.out‚ 2.146484375" disk.dm-0.out‚5811.97998046875 disk.drbd1716.in‚0 disk.sda2.in‚0 disk.dm-1.in‚0( proc.memory.availableMB‚ 2563.2421875 network.recv‚582876 disk.nvme0n1.in‚ 11.80859375 disk.drbd2265.in‚0! disk.nvme0c0n1.in‚ 11.80859375 disk.dm-0.in‚ 11.80859375 disk.sda3.in‚0ÓçÚ‘û ‚(:† ®ŸäÔ²­“¨ gpu.0.gpu‚34.0 gpu.0.memory‚2, gpu.0.memoryAllocated‚12.165603637695312* gpu.0.memoryAllocatedBytes‚ 10450173952 gpu.0.temp‚34.0 gpu.0.powerWatts‚119.08' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚23.816 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0Vôv#Tü ‚(:É ®ŸäÔÅ«¬« disk.rbd0.in‚0 disk.sda1.in‚0 disk.drbd2080.out‚0 disk.drbd2266.in‚0 disk.sda3.out‚ 57.96484375 disk.dm-1.in‚0 disk.sda1.out‚0 disk.drbd2265.in‚0 disk.rbd1.in‚0$ memory_percent‚84.79411602020264 cpu‚26.11546934240087 network.recv‚588592 disk.dm-1.out‚ 57.96484375$ disk.nvme0n1.out‚5900.6279296875! disk.dm-0.out‚5900.7529296875 disk.sda.out‚ 57.96484375) proc.memory.availableMB‚ 2491.33203125$ disk./.usageGB‚251.0194854736328 disk.drbd1716.out‚0 disk.rbd1.out‚ 2.146484375 disk.sda2.in‚0 disk.sda2.out‚0 disk.drbd2265.out‚0& disk.nvme0c0n1.out‚5900.6279296875 disk.sda.in‚0 proc.cpu.threads‚17 disk.sda3.in‚0 disk.rbd0.out‚0 disk.nvme0n1.in‚ 11.80859375! disk.nvme0c0n1.in‚ 11.80859375# proc.memory.rssMB‚ 2873.27734375) proc.memory.percent‚17.53709316253662 network.sent‚ 3.478108e+06* disk./.usagePercent‚16.846128289512492 disk.drbd2080.in‚0 disk.drbd2266.out‚0 disk.dm-0.in‚ 11.80859375 disk.drbd1716.in‚0ºu‚ý ‚(:’ ½ŸäÔéÛ®¨ gpu.0.gpu‚2.0 gpu.0.memory‚0, gpu.0.memoryAllocated‚12.165603637695312* gpu.0.memoryAllocatedBytes‚ 10450173952 gpu.0.temp‚33.0 gpu.0.powerWatts‚118.653' gpu.0.enforcedPowerLimitWatts‚500.0) gpu.0.powerPercent‚23.730600000000003 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0\î“Qþ ‚(:Æ ½ŸäÔìÓÄ«$ memory_percent‚84.24406051635742* disk./.usagePercent‚16.849639176837062 disk.rbd1.out‚ 2.146484375% disk.nvme0n1.out‚5968.27197265625' proc.memory.availableMB‚ 2581.453125* proc.memory.percent‚16.982483863830566 network.recv‚591595 disk.sda1.in‚0 disk.sda1.out‚0 disk.drbd2266.in‚0 disk.drbd2265.in‚0 disk.drbd2265.out‚0' disk.nvme0c0n1.out‚5968.27197265625 disk.dm-0.in‚ 11.80859375 disk.dm-1.out‚ 59.515625 disk.drbd1716.in‚0 disk.drbd1716.out‚0 disk.drbd2080.in‚0 disk.rbd0.out‚0 disk.sda.out‚ 59.515625 disk.rbd1.in‚0 disk.drbd2080.out‚0 disk.drbd2266.out‚0# proc.memory.rssMB‚ 2782.41015625 proc.cpu.threads‚17" disk.dm-0.out‚5968.39697265625 disk.sda2.out‚0 disk.nvme0n1.in‚ 11.80859375 disk.sda.in‚0 network.sent‚ 3.482617e+06 disk.dm-1.in‚0 disk.rbd0.in‚0 cpu‚26.179150759274325$ disk./.usageGB‚251.0718002319336! disk.nvme0c0n1.in‚ 11.80859375 disk.sda3.in‚0 disk.sda3.out‚ 59.515625 disk.sda2.in‚0d‹—}2ÿ - _wandbruntime‚1529  _runtime‚1529Þ½Ÿq€ ͦ ?&Purity/1FRt0SRqqt1FBt/event_proportion‚0.049773186445236206 8 Purity/1FRt2SRqqt1FBt/FRt_purity‚0.22004608809947968 ,&Purity/4FRt2SRqqt4FBt/event_proportion‚0 ,&Purity/4FRt3SRqqt*FBt/event_proportion‚0 7validation_average_jet_accuracy‚0.33369380235671997 particle/accuracy_4_of_6‚1 particle/accuracy_5_of_7‚1 7 Purity/0FRt*SRqqt*FBt/FBt_purity‚0.9887565970420837 ;"Purity/1FRt*SRqqt0FBt/event_purity‚0.047739025205373764 A&Purity/1FRt1SRqqt3FBt/event_proportion‚0.00025201612152159214 ("Purity/4FRt1SRqqt0FBt/event_purity‚0 9"Purity/*FRt1SRqqt0FBt/event_purity‚0.1196458712220192 8 Purity/*FRt1SRqqt1FBt/FRt_purity‚0.29598844051361084 & Purity/*FRt3SRqqt1FBt/FBt_purity‚1 >&Purity/0FRt1SRqqt1FBt/event_proportion‚0.05185231938958168 ("Purity/3FRt0SRqqt*FBt/event_purity‚0 ,&Purity/3FRt3SRqqt2FBt/event_proportion‚0 ,&Purity/4FRt0SRqqt4FBt/event_proportion‚0 9"Purity/*FRt*SRqqt4FBt/event_purity‚0.1666666716337204 >&Purity/0FRt2SRqqt*FBt/event_proportion‚0.05361643061041832 ,&Purity/3FRt1SRqqt4FBt/event_proportion‚0 ("Purity/3FRt2SRqqt0FBt/event_purity‚0 ,&Purity/3FRt2SRqqt4FBt/event_proportion‚0 & Purity/1FRt0SRqqt4FBt/FRt_purity‚0 :"Purity/*FRt*SRqqt1FBt/event_purity‚0.39189282059669495 ?&Purity/*FRt0SRqqt3FBt/event_proportion‚0.003717237850651145 ,&Purity/1FRt3SRqqt3FBt/event_proportion‚0 ?&Purity/2FRt*SRqqt1FBt/event_proportion‚0.021169355139136314 ("Purity/2FRt0SRqqt0FBt/event_purity‚0 ("Purity/2FRt1SRqqt0FBt/event_purity‚0 ,&Purity/3FRt4SRqqt2FBt/event_proportion‚0 >&Purity/1FRt0SRqqt0FBt/event_proportion‚0.13142640888690948 ,&Purity/2FRt3SRqqt4FBt/event_proportion‚0 @&Purity/3FRt3SRqqt0FBt/event_proportion‚0.0005670362734235823 ,&Purity/4FRt4SRqqt*FBt/event_proportion‚0 :"Purity/2FRt4SRqqt*FBt/SRqqt_purity‚0.15000000596046448 & Purity/4FRt*SRqqt0FBt/FRt_purity‚0 =&Purity/*FRt0SRqqt2FBt/event_proportion‚0.0483240932226181 7 Purity/2FRt*SRqqt*FBt/FBt_purity‚0.9961161017417908 :"Purity/2FRt2SRqqt*FBt/SRqqt_purity‚0.22519007325172424 particle/accuracy_3_of_4‚1 ("Purity/0FRt4SRqqt1FBt/SRqqt_purity‚0 7 Purity/2FRt4SRqqt*FBt/FRt_purity‚0.4000000059604645 % _timestamp‚1.7884159477396815e+09 :"Purity/*FRt1SRqqt1FBt/event_purity‚0.08534605801105499 @&Purity/*FRt4SRqqt0FBt/event_proportion‚0.0012600806076079607 :"Purity/1FRt2SRqqt0FBt/SRqqt_purity‚0.21041135489940643 ,&Purity/1FRt3SRqqt4FBt/event_proportion‚0 9"Purity/2FRt1SRqqt2FBt/SRqqt_purity‚0.6228070259094238 @&Purity/3FRt*SRqqt2FBt/event_proportion‚0.0003150201519019902 @&Purity/3FRt0SRqqt2FBt/event_proportion‚6.300403038039804e-05 *jet/accuracy_1_of_2‚0.6041082143783569 :"Purity/0FRt*SRqqt2FBt/event_purity‚0.23743848502635956 ("Purity/2FRt0SRqqt2FBt/event_purity‚0 ,&Purity/4FRt*SRqqt3FBt/event_proportion‚0 particle/accuracy_1_of_5‚1 9"Purity/0FRt3SRqqt1FBt/SRqqt_purity‚0.3541666567325592 & Purity/*FRt3SRqqt2FBt/FRt_purity‚1 7 Purity/0FRt2SRqqt3FBt/FBt_purity‚0.6666666865348816 & Purity/4FRt2SRqqt*FBt/FRt_purity‚0 @&Purity/*FRt4SRqqt1FBt/event_proportion‚6.300403038039804e-05 :"Purity/1FRt3SRqqt0FBt/SRqqt_purity‚0.19502051174640656 @&Purity/1FRt4SRqqt*FBt/event_proportion‚0.0003780241822823882 7 Purity/2FRt*SRqqt2FBt/FBt_purity‚0.9911110997200012 jet/accuracy_4_of_7‚0 <"Purity/2FRt*SRqqt*FBt/event_purity‚0.0026922288816422224 8 Purity/2FRt*SRqqt0FBt/FRt_purity‚0.15440437197685242 ?&Purity/2FRt1SRqqt2FBt/event_proportion‚0.001890120911411941 ("Purity/4FRt*SRqqt*FBt/SRqqt_purity‚0 8 Purity/*FRt*SRqqt*FBt/FRt_purity‚0.20124734938144684 :"Purity/0FRt1SRqqt0FBt/SRqqt_purity‚0.22838279604911804 8 Purity/1FRt1SRqqt0FBt/FRt_purity‚0.20677201449871063 ("Purity/1FRt2SRqqt1FBt/event_purity‚0 8 Purity/3FRt2SRqqt*FBt/FRt_purity‚0.14814813435077667 >&Purity/1FRt*SRqqt1FBt/event_proportion‚0.09929435700178146 7 Purity/1FRt1SRqqt2FBt/FBt_purity‚0.9732142686843872 ("Purity/1FRt3SRqqt1FBt/event_purity‚0 ,&Purity/4FRt0SRqqt2FBt/event_proportion‚0 ,&Purity/4FRt3SRqqt3FBt/event_proportion‚0 jet/accuracy_6_of_6‚0 + Purity/1FRt*SRqqt3FBt/FBt_purity‚0.9375 @&Purity/*FRt1SRqqt4FBt/event_proportion‚6.300403038039804e-05 @&Purity/3FRt1SRqqt1FBt/event_proportion‚0.0009450604557059705 9"Purity/0FRt*SRqqt3FBt/SRqqt_purity‚0.6333333253860474 9"Purity/0FRt0SRqqt3FBt/event_purity‚0.1599999964237213 =&Purity/0FRt1SRqqt0FBt/event_proportion‚0.1694808453321457 ?&Purity/*FRt3SRqqt1FBt/event_proportion‚0.002142137149348855 :"Purity/*FRt*SRqqt3FBt/event_purity‚0.06609195470809937 >&Purity/0FRt*SRqqt1FBt/event_proportion‚0.14787046611309052 7 Purity/0FRt*SRqqt2FBt/FBt_purity‚0.9837964177131653 *loss/val_total_loss‚46.909293437171556 8 Purity/*FRt3SRqqt*FBt/FRt_purity‚0.23724336922168732 @&Purity/4FRt2SRqqt0FBt/event_proportion‚6.300403038039804e-05 9"Purity/0FRt4SRqqt0FBt/SRqqt_purity‚0.1785714328289032 >&Purity/*FRt2SRqqt0FBt/event_proportion‚0.10855595022439957 =&Purity/1FRt*SRqqt*FBt/event_proportion‚0.4366179406642914 8 Purity/1FRt3SRqqt1FBt/FRt_purity‚0.35555556416511536 @&Purity/2FRt3SRqqt1FBt/event_proportion‚0.0003780241822823882 ?&Purity/3FRt1SRqqt*FBt/event_proportion‚0.003717237850651145 9"Purity/*FRt2SRqqt2FBt/SRqqt_purity‚0.3114035129547119 & Purity/0FRt3SRqqt1FBt/FBt_purity‚1 7 Purity/1FRt0SRqqt2FBt/FBt_purity‚0.9828978180885315 @&Purity/2FRt2SRqqt2FBt/event_proportion‚6.300403038039804e-05 ("Purity/3FRt1SRqqt*FBt/event_purity‚0 7 Purity/*FRt0SRqqt4FBt/FBt_purity‚0.8500000238418579 >&Purity/0FRt0SRqqt0FBt/event_proportion‚0.04082661122083664 ;"Purity/0FRt2SRqqt0FBt/event_purity‚0.012148144654929638 8 Purity/1FRt0SRqqt0FBt/FRt_purity‚0.10697515308856964 ?&Purity/1FRt0SRqqt2FBt/event_proportion‚0.015120967291295528 ?&Purity/2FRt0SRqqt2FBt/event_proportion‚0.003465221729129553 particle/accuracy_6_of_6‚1 ("Purity/1FRt*SRqqt4FBt/event_purity‚0 >&Purity/2FRt*SRqqt*FBt/event_proportion‚0.10244455933570862 :"Purity/3FRt1SRqqt0FBt/SRqqt_purity‚0.34166666865348816 ("Purity/4FRt1SRqqt*FBt/SRqqt_purity‚0 8 Purity/*FRt*SRqqt2FBt/FRt_purity‚0.35563305020332336 >&Purity/*FRt0SRqqt1FBt/event_proportion‚0.14591734111309052 ,&Purity/1FRt4SRqqt2FBt/event_proportion‚0 ,&Purity/1FRt4SRqqt3FBt/event_proportion‚0 9"Purity/2FRt2SRqqt0FBt/SRqqt_purity‚0.1725439876317978 @&Purity/3FRt0SRqqt*FBt/event_proportion‚0.0027091733645647764 @&Purity/3FRt1SRqqt0FBt/event_proportion‚0.0025201612152159214 ,&Purity/3FRt3SRqqt3FBt/event_proportion‚0 @&Purity/*FRt4SRqqt*FBt/event_proportion‚0.0013230846961960196 @&Purity/0FRt2SRqqt2FBt/event_proportion‚0.0010080644860863686 A&Purity/3FRt1SRqqt2FBt/event_proportion‚0.00025201612152159214 ,&Purity/4FRt1SRqqt3FBt/event_proportion‚0 ,&Purity/4FRt4SRqqt0FBt/event_proportion‚0 particle/accuracy_2_of_3‚1 7 Purity/*FRt1SRqqt3FBt/FBt_purity‚0.8730158805847168 9"Purity/*FRt1SRqqt*FBt/SRqqt_purity‚0.2686524987220764 >&Purity/0FRt0SRqqt1FBt/event_proportion‚0.08606351166963577 :"Purity/1FRt0SRqqt*FBt/event_purity‚0.13411523401737213 @&Purity/4FRt0SRqqt0FBt/event_proportion‚6.300403038039804e-05 ("Purity/0FRt3SRqqt0FBt/event_purity‚0 & Purity/2FRt3SRqqt*FBt/FBt_purity‚1 ;"Purity/*FRt2SRqqt1FBt/event_purity‚0.013043108396232128 8 Purity/1FRt*SRqqt1FBt/FRt_purity‚0.28762194514274597 ,&Purity/2FRt1SRqqt3FBt/event_proportion‚0 jet/accuracy_1_of_7‚1 7 Purity/1FRt4SRqqt*FBt/FRt_purity‚0.1666666716337204 ("Purity/*FRt3SRqqt2FBt/event_purity‚0 ("Purity/1FRt1SRqqt3FBt/event_purity‚0 ,&Purity/4FRt3SRqqt2FBt/event_proportion‚0 9"Purity/1FRt2SRqqt2FBt/SRqqt_purity‚0.2569444477558136 *jet/accuracy_2_of_5‚0.3566853404045105 ("Purity/0FRt2SRqqt2FBt/event_purity‚0 :"Purity/2FRt3SRqqt1FBt/SRqqt_purity‚0.20000000298023224 ("Purity/1FRt*SRqqt3FBt/event_purity‚0 7 Purity/2FRt3SRqqt0FBt/FRt_purity‚0.2575216293334961 ,&Purity/2FRt4SRqqt2FBt/event_proportion‚0 ("Purity/*FRt4SRqqt1FBt/event_purity‚0 7 Purity/3FRt3SRqqt*FBt/FRt_purity‚0.1666666567325592 ,&Purity/0FRt4SRqqt3FBt/event_proportion‚0 8"Purity/1FRt*SRqqt1FBt/SRqqt_purity‚0.403650164604187 ,&Purity/1FRt4SRqqt4FBt/event_proportion‚0 ,&Purity/4FRt*SRqqt4FBt/event_proportion‚0 +"Purity/0FRt*SRqqt4FBt/event_purity‚0.25 & Purity/1FRt1SRqqt4FBt/FRt_purity‚0 & Purity/4FRt0SRqqt0FBt/FRt_purity‚0 :"Purity/*FRt0SRqqt0FBt/event_purity‚0.08908312022686005 :"Purity/2FRt1SRqqt1FBt/SRqqt_purity‚0.34615975618362427 @&Purity/2FRt4SRqqt0FBt/event_proportion‚0.0003150201519019902 & Purity/0FRt4SRqqt1FBt/FBt_purity‚1 7 Purity/3FRt*SRqqt1FBt/FRt_purity‚0.2901515066623688 7 Purity/*FRt*SRqqt3FBt/FRt_purity‚0.1764705926179886 9"Purity/2FRt*SRqqt2FBt/SRqqt_purity‚0.6052631735801697 ?&Purity/2FRt2SRqqt*FBt/event_proportion‚0.019468246027827263 ("Purity/4FRt2SRqqt*FBt/event_purity‚0 >&Purity/*FRt3SRqqt0FBt/event_proportion‚0.01827116869390011 7 Purity/*FRt*SRqqt1FBt/FBt_purity‚0.9988723993301392 7 Purity/1FRt0SRqqt*FBt/FBt_purity‚0.9916885495185852 ,&Purity/4FRt2SRqqt1FBt/event_proportion‚0 :"Purity/0FRt3SRqqt*FBt/SRqqt_purity‚0.23054611682891846 & Purity/3FRt2SRqqt1FBt/FBt_purity‚1 ,&Purity/*FRt2SRqqt4FBt/event_proportion‚0 ,&Purity/*FRt3SRqqt4FBt/event_proportion‚0 >&Purity/0FRt*SRqqt0FBt/event_proportion‚0.21931703388690948 & Purity/1FRt2SRqqt1FBt/FBt_purity‚1 7 Purity/2FRt1SRqqt*FBt/FBt_purity‚0.9964157938957214 @&Purity/3FRt2SRqqt1FBt/event_proportion‚0.0003780241822823882 ,&Purity/3FRt2SRqqt2FBt/event_proportion‚0 ("Purity/2FRt4SRqqt*FBt/event_purity‚0 & Purity/3FRt1SRqqt1FBt/FBt_purity‚1 7 Purity/0FRt0SRqqt2FBt/FBt_purity‚0.9858158230781555 >&Purity/2FRt1SRqqt*FBt/event_proportion‚0.04158266261219978 ("Purity/2FRt2SRqqt*FBt/event_purity‚0 @&Purity/2FRt4SRqqt*FBt/event_proportion‚0.0003150201519019902 ,&Purity/3FRt*SRqqt4FBt/event_proportion‚0 @&Purity/3FRt0SRqqt1FBt/event_proportion‚0.0011340725468471646 particle/accuracy_1_of_4‚1 7 Purity/*FRt1SRqqt1FBt/FBt_purity‚0.9980608224868774 7 Purity/*FRt2SRqqt*FBt/FBt_purity‚0.9953680634498596 8 Purity/*FRt3SRqqt1FBt/FRt_purity‚0.31111109256744385 ("Purity/2FRt*SRqqt0FBt/event_purity‚0 ("Purity/2FRt1SRqqt1FBt/event_purity‚0 7 Purity/*FRt4SRqqt*FBt/FRt_purity‚0.2083333283662796 ,"Purity/3FRt4SRqqt0FBt/SRqqt_purity‚0.125 & Purity/*FRt0SRqqt4FBt/FRt_purity‚0 :"Purity/1FRt*SRqqt1FBt/event_purity‚0.12268873304128647 7 Purity/1FRt3SRqqt*FBt/FRt_purity‚0.2519969344139099 ,&Purity/2FRt3SRqqt2FBt/event_proportion‚0 ,&Purity/3FRt4SRqqt4FBt/event_proportion‚0 *jet/accuracy_1_of_6‚0.8859648704528809 particle/accuracy_6_of_7‚0.5 & Purity/*FRt1SRqqt3FBt/FRt_purity‚0 ("Purity/4FRt2SRqqt0FBt/event_purity‚0 >&Purity/*FRt*SRqqt2FBt/event_proportion‚0.06747731566429138 =&Purity/*FRt0SRqqt*FBt/event_proportion‚0.3558467626571655 9"Purity/0FRt*SRqqt0FBt/event_purity‚0.1790110170841217 8 Purity/2FRt2SRqqt0FBt/FRt_purity‚0.20405493676662445 ("Purity/*FRt4SRqqt1FBt/SRqqt_purity‚0 ,&Purity/*FRt3SRqqt3FBt/event_proportion‚0 ,&Purity/3FRt0SRqqt4FBt/event_proportion‚0 ,&Purity/4FRt2SRqqt3FBt/event_proportion‚0 particle/accuracy_3_of_6‚1 ("Purity/2FRt3SRqqt1FBt/event_purity‚0 ("Purity/1FRt1SRqqt4FBt/event_purity‚0 & Purity/2FRt2SRqqt2FBt/FBt_purity‚1 ("Purity/0FRt2SRqqt3FBt/event_purity‚0 9"Purity/*FRt2SRqqt*FBt/SRqqt_purity‚0.2434745877981186 ,&Purity/*FRt4SRqqt3FBt/event_proportion‚0 ?&Purity/2FRt1SRqqt1FBt/event_proportion‚0.007875503972172737 particle/accuracy_4_of_5‚1 ("Purity/4FRt0SRqqt*FBt/event_purity‚0 :"Purity/*FRt2SRqqt1FBt/SRqqt_purity‚0.37011709809303284 6 Purity/0FRt1SRqqt2FBt/FBt_purity‚0.973981499671936 particle/accuracy_5_of_5‚1 ("Purity/*FRt2SRqqt2FBt/event_purity‚0 7 Purity/*FRt*SRqqt3FBt/FBt_purity‚0.9109196066856384 7 Purity/*FRt1SRqqt2FBt/FRt_purity‚0.2487362176179886 A&Purity/0FRt*SRqqt4FBt/event_proportion‚0.00025201612152159214 ?&Purity/1FRt3SRqqt*FBt/event_proportion‚0.009135584346950054 ("Purity/2FRt*SRqqt2FBt/event_purity‚0 & Purity/2FRt2SRqqt*FBt/FBt_purity‚1 @&Purity/4FRt2SRqqt*FBt/event_proportion‚6.300403038039804e-05 ,&Purity/3FRt3SRqqt1FBt/event_proportion‚0 +jet/accuracy_2_of_6‚0.32456138730049133 _runtime‚1529.391228984 7 Purity/0FRt1SRqqt*FBt/FBt_purity‚0.9888736605644226 8 Purity/1FRt*SRqqt*FBt/FRt_purity‚0.20560263097286224 @&Purity/4FRt0SRqqt*FBt/event_proportion‚6.300403038039804e-05 ("Purity/*FRt4SRqqt0FBt/event_purity‚0 7 Purity/1FRt2SRqqt2FBt/FRt_purity‚0.2222222238779068 ("Purity/*FRt1SRqqt3FBt/event_purity‚0 8"Purity/0FRt1SRqqt3FBt/SRqqt_purity‚0.699999988079071 8"Purity/1FRt0SRqqt1FBt/event_purity‚0.220099538564682 7 Purity/3FRt1SRqqt2FBt/FRt_purity‚0.0833333358168602 7 Purity/0FRt0SRqqt*FBt/FBt_purity‚0.9887112379074097 7 Purity/2FRt1SRqqt0FBt/FRt_purity‚0.1584039032459259 8 Purity/3FRt*SRqqt*FBt/FRt_purity‚0.16922254860401154 :"Purity/0FRt2SRqqt2FBt/SRqqt_purity‚0.32692307233810425 ) Purity/1FRt1SRqqt4FBt/FBt_purity‚0.75 :"Purity/*FRt0SRqqt*FBt/event_purity‚0.34444329142570496 9"Purity/2FRt*SRqqt*FBt/SRqqt_purity‚0.2299972027540207 7 Purity/2FRt1SRqqt2FBt/FBt_purity‚0.9868420958518982 ,&Purity/3FRt4SRqqt1FBt/event_proportion‚0 ,&Purity/4FRt3SRqqt1FBt/event_proportion‚0 particle/accuracy_4_of_4‚1 ("Purity/4FRt*SRqqt*FBt/event_purity‚0 8 Purity/*FRt2SRqqt0FBt/FRt_purity‚0.20199143886566162 + Purity/0FRt*SRqqt4FBt/FBt_purity‚0.8125 ("Purity/2FRt*SRqqt3FBt/event_purity‚0 9"Purity/*FRt3SRqqt2FBt/SRqqt_purity‚0.3333333432674408 @&Purity/*FRt3SRqqt2FBt/event_proportion‚6.300403038039804e-05 particle/accuracy_1_of_1‚1 7 Purity/*FRt2SRqqt2FBt/FBt_purity‚0.9868420958518982 ("Purity/1FRt2SRqqt2FBt/event_purity‚0 7 Purity/3FRt4SRqqt0FBt/FRt_purity‚0.1666666716337204 8 Purity/*FRt2SRqqt*FBt/FRt_purity‚0.21031200885772705 >&Purity/1FRt2SRqqt*FBt/event_proportion‚0.05727066472172737 @&Purity/1FRt3SRqqt1FBt/event_proportion‚0.0011340725468471646 ;"Purity/2FRt*SRqqt1FBt/event_purity‚0.014464925974607468 ("Purity/1FRt1SRqqt3FBt/SRqqt_purity‚1 9"Purity/0FRt*SRqqt1FBt/event_purity‚0.6332274675369263 & Purity/1FRt3SRqqt*FBt/FBt_purity‚1 ,&Purity/2FRt4SRqqt3FBt/event_proportion‚0 ,&Purity/3FRt0SRqqt3FBt/event_proportion‚0 7 Purity/3FRt4SRqqt*FBt/FRt_purity‚0.1666666716337204 & Purity/2FRt*SRqqt3FBt/FBt_purity‚1 9"Purity/0FRt0SRqqt2FBt/event_purity‚0.3085920810699463 ?&Purity/1FRt3SRqqt0FBt/event_proportion‚0.007938507944345474 8 Purity/2FRt1SRqqt*FBt/FRt_purity‚0.19843967258930206 8 Purity/3FRt*SRqqt0FBt/FRt_purity‚0.11486772447824478 7 Purity/*FRt*SRqqt4FBt/FBt_purity‚0.8333333134651184 =&Purity/*FRt*SRqqt*FBt/event_proportion‚0.9591733813285828 ,&Purity/1FRt4SRqqt1FBt/event_proportion‚0 @&Purity/4FRt*SRqqt*FBt/event_proportion‚0.0001890120911411941 & Purity/*FRt4SRqqt*FBt/FBt_purity‚1 7 Purity/0FRt2SRqqt2FBt/FBt_purity‚0.9615384340286255 & Purity/3FRt*SRqqt2FBt/FBt_purity‚1 ("Purity/1FRt*SRqqt3FBt/SRqqt_purity‚1 ("Purity/2FRt2SRqqt2FBt/event_purity‚0 7 Purity/0FRt*SRqqt3FBt/FBt_purity‚0.9012346863746643 :"Purity/0FRt2SRqqt1FBt/event_purity‚0.02700972929596901 8 Purity/1FRt*SRqqt0FBt/FRt_purity‚0.16713960468769073 ,jet/accuracy_4_of_5‚0.002481389557942748 /"Purity/*FRt4SRqqt0FBt/SRqqt_purity‚0.171875 7 Purity/3FRt0SRqqt2FBt/FRt_purity‚0.6666666865348816 7 Purity/*FRt0SRqqt2FBt/FRt_purity‚0.4051198661327362 7 Purity/1FRt*SRqqt2FBt/FBt_purity‚0.9803202152252197 9"Purity/1FRt1SRqqt*FBt/SRqqt_purity‚0.2579297721385956 ?&Purity/2FRt0SRqqt1FBt/event_proportion‚0.008946572430431843 ,&Purity/4FRt*SRqqt1FBt/event_proportion‚0 *jet/accuracy_1_of_4‚0.7875291109085083 & Purity/2FRt0SRqqt1FBt/FBt_purity‚1 -jet/accuracy_4_of_4‚0.0010080644860863686 7 Purity/1FRt1SRqqt3FBt/FBt_purity‚0.8333333730697632 ("Purity/3FRt*SRqqt*FBt/event_purity‚0 particle/accuracy_2_of_4‚1 *particle/sensitivity‚0.219760000705719 8 Purity/2FRt3SRqqt1FBt/FRt_purity‚0.10000000149011612 & Purity/2FRt0SRqqt3FBt/FRt_purity‚0 ;"Purity/0FRt1SRqqt2FBt/event_purity‚0.008095238357782364 ?&Purity/0FRt3SRqqt*FBt/event_proportion‚0.006741431541740894 7 Purity/1FRt*SRqqt*FBt/FBt_purity‚0.9915969371795654 =&Purity/1FRt*SRqqt0FBt/event_proportion‚0.3135710656642914 ?&Purity/1FRt*SRqqt2FBt/event_proportion‚0.022303426638245583 :"Purity/0FRt*SRqqt0FBt/SRqqt_purity‚0.22572781145572662 8 Purity/2FRt2SRqqt*FBt/FRt_purity‚0.21763528883457184 :"Purity/2FRt2SRqqt1FBt/SRqqt_purity‚0.37886905670166016 ,&Purity/3FRt*SRqqt3FBt/event_proportion‚0 ("Purity/3FRt4SRqqt*FBt/event_purity‚0 ?&Purity/0FRt1SRqqt2FBt/event_proportion‚0.008757560513913631 ,&Purity/0FRt4SRqqt2FBt/event_proportion‚0 ?&Purity/2FRt2SRqqt0FBt/event_proportion‚0.015435988083481789 ("Purity/1FRt*SRqqt4FBt/SRqqt_purity‚1 9"Purity/3FRt3SRqqt*FBt/SRqqt_purity‚0.2142857313156128 :"Purity/0FRt1SRqqt1FBt/event_purity‚0.14303721487522125 ?&Purity/0FRt3SRqqt0FBt/event_proportion‚0.006111391354352236 :"Purity/2FRt*SRqqt0FBt/SRqqt_purity‚0.19026660919189453 ;"Purity/2FRt0SRqqt1FBt/event_purity‚0.028888888657093048 8 Purity/2FRt0SRqqt2FBt/FRt_purity‚0.37738093733787537 & Purity/1FRt3SRqqt2FBt/FBt_purity‚1 ?&Purity/0FRt0SRqqt2FBt/event_proportion‚0.029674898833036423 7 Purity/1FRt3SRqqt0FBt/FRt_purity‚0.2798386812210083 9"Purity/3FRt1SRqqt1FBt/SRqqt_purity‚0.4615384638309479 9"Purity/3FRt2SRqqt*FBt/SRqqt_purity‚0.2934027910232544 9"Purity/1FRt4SRqqt*FBt/SRqqt_purity‚0.1666666716337204 ("Purity/4FRt2SRqqt0FBt/SRqqt_purity‚0 8 Purity/*FRt0SRqqt3FBt/FRt_purity‚0.20000000298023224 >&Purity/*FRt1SRqqt0FBt/event_proportion‚0.33039313554763794 ("Purity/0FRt0SRqqt0FBt/event_purity‚1 ;"Purity/0FRt2SRqqt*FBt/event_purity‚0.015407760627567768 7 Purity/0FRt2SRqqt*FBt/FBt_purity‚0.9892473816871643 @&Purity/0FRt2SRqqt3FBt/event_proportion‚6.300403038039804e-05 & Purity/3FRt*SRqqt*FBt/FBt_purity‚1 & Purity/3FRt0SRqqt1FBt/FBt_purity‚1 7 Purity/*FRt0SRqqt1FBt/FBt_purity‚0.9991505742073059 8 Purity/1FRt0SRqqt1FBt/FRt_purity‚0.30332961678504944 @&Purity/1FRt0SRqqt3FBt/event_proportion‚0.0010710685746744275 +jet/accuracy_2_of_4‚0.29374340176582336 7 Purity/*FRt4SRqqt0FBt/FRt_purity‚0.2083333283662796 ("Purity/3FRt4SRqqt0FBt/event_purity‚0 & Purity/*FRt1SRqqt4FBt/FRt_purity‚0 ("Purity/*FRt3SRqqt*FBt/event_purity‚0 ,&Purity/*FRt4SRqqt2FBt/event_proportion‚0 9"Purity/0FRt0SRqqt*FBt/event_purity‚0.8084040880203247 7 Purity/2FRt1SRqqt1FBt/FRt_purity‚0.3138824999332428 & Purity/1FRt1SRqqt3FBt/FRt_purity‚0 @&Purity/*FRt*SRqqt4FBt/event_proportion‚0.0003780241822823882 @&Purity/0FRt*SRqqt3FBt/event_proportion‚0.0030871976632624865 & Purity/1FRt1SRqqt1FBt/FBt_purity‚1 ,&Purity/4FRt4SRqqt1FBt/event_proportion‚0 ,&Purity/4FRt4SRqqt2FBt/event_proportion‚0 ,jet/accuracy_3_of_4‚0.043030958622694016 particle/accuracy_3_of_3‚1 ("Purity/1FRt0SRqqt4FBt/event_purity‚0 7 Purity/*FRt2SRqqt1FBt/FRt_purity‚0.2415153682231903 & Purity/*FRt3SRqqt*FBt/FBt_purity‚1 ("Purity/3FRt1SRqqt1FBt/event_purity‚0 jet/accuracy_6_of_7‚0 & Purity/*FRt*SRqqt4FBt/FRt_purity‚0 & Purity/2FRt0SRqqt3FBt/FBt_purity‚1 9"Purity/*FRt1SRqqt3FBt/SRqqt_purity‚0.8333333730697632 ("Purity/0FRt1SRqqt3FBt/event_purity‚0 7 Purity/0FRt*SRqqt1FBt/FBt_purity‚0.9983028173446655 9"Purity/0FRt1SRqqt1FBt/SRqqt_purity‚0.4184369742870331 :"Purity/1FRt2SRqqt*FBt/SRqqt_purity‚0.24118562042713165 ,&Purity/4FRt4SRqqt3FBt/event_proportion‚0 jet/accuracy_5_of_5‚0 & Purity/4FRt0SRqqt*FBt/FRt_purity‚0 :"Purity/0FRt1SRqqt*FBt/event_purity‚0.20061224699020386 9"Purity/1FRt3SRqqt*FBt/SRqqt_purity‚0.2253115028142929 @&Purity/3FRt0SRqqt0FBt/event_proportion‚0.0015120967291295528 ?&Purity/*FRt1SRqqt2FBt/event_proportion‚0.017011089250445366 ("Purity/2FRt2SRqqt1FBt/event_purity‚0 8 Purity/2FRt3SRqqt*FBt/FRt_purity‚0.23100000619888306 7 Purity/3FRt3SRqqt0FBt/FRt_purity‚0.1666666567325592 *"Purity/2FRt2SRqqt2FBt/SRqqt_purity‚0.5 & Purity/2FRt*SRqqt1FBt/FBt_purity‚1 >&Purity/2FRt1SRqqt0FBt/event_proportion‚0.03181703761219978 7 Purity/1FRt4SRqqt0FBt/FRt_purity‚0.1666666716337204 ,&Purity/1FRt2SRqqt3FBt/event_proportion‚0 ,&Purity/2FRt3SRqqt3FBt/event_proportion‚0 ("Purity/3FRt0SRqqt0FBt/event_purity‚0 & Purity/2FRt*SRqqt3FBt/FRt_purity‚0 7 Purity/*FRt0SRqqt3FBt/FBt_purity‚0.9236112236976624 8"Purity/0FRt*SRqqt*FBt/event_purity‚0.347979336977005 ,&Purity/0FRt3SRqqt3FBt/event_proportion‚0 7 Purity/2FRt2SRqqt1FBt/FRt_purity‚0.2633928656578064 ?&Purity/*FRt*SRqqt3FBt/event_proportion‚0.004473286215215921 =&Purity/1FRt1SRqqt0FBt/event_proportion‚0.1265120953321457 :"Purity/0FRt3SRqqt0FBt/SRqqt_purity‚0.22251984477043152 >&Purity/*FRt*SRqqt1FBt/event_proportion‚0.27079132199287415 >&Purity/0FRt*SRqqt2FBt/event_proportion‚0.03944052383303642 >&Purity/1FRt0SRqqt*FBt/event_proportion‚0.19745463132858276 @&Purity/1FRt4SRqqt0FBt/event_proportion‚0.0003780241822823882 ("Purity/3FRt1SRqqt0FBt/event_purity‚0 9"Purity/0FRt2SRqqt1FBt/SRqqt_purity‚0.3635176718235016 @&Purity/0FRt3SRqqt1FBt/event_proportion‚0.0006300403038039804 :"Purity/1FRt0SRqqt0FBt/event_purity‚0.10697515308856964 8 Purity/1FRt1SRqqt2FBt/FRt_purity‚0.32006803154945374 ("Purity/2FRt2SRqqt0FBt/event_purity‚0 7 Purity/2FRt*SRqqt2FBt/FRt_purity‚0.3288889229297638 :"Purity/2FRt3SRqqt*FBt/SRqqt_purity‚0.18352378904819489 ("Purity/0FRt4SRqqt*FBt/event_purity‚0 8 Purity/1FRt0SRqqt*FBt/FRt_purity‚0.18164362013339996 :"Purity/*FRt*SRqqt*FBt/event_purity‚0.17893172800540924 :"Purity/0FRt2SRqqt0FBt/SRqqt_purity‚0.22733958065509796 ?&Purity/2FRt2SRqqt1FBt/event_proportion‚0.003969253972172737 ,&Purity/3FRt2SRqqt3FBt/event_proportion‚0 A&Purity/3FRt4SRqqt0FBt/event_proportion‚0.00012600806076079607 ("Purity/*FRt4SRqqt*FBt/event_purity‚0 ("Purity/1FRt1SRqqt4FBt/SRqqt_purity‚1 6 Purity/*FRt3SRqqt0FBt/FRt_purity‚0.259640097618103 :"Purity/0FRt*SRqqt1FBt/SRqqt_purity‚0.39769095182418823 8"Purity/0FRt1SRqqt2FBt/SRqqt_purity‚0.408214271068573 @&Purity/2FRt0SRqqt3FBt/event_proportion‚6.300403038039804e-05 :"Purity/2FRt1SRqqt*FBt/SRqqt_purity‚0.24897189438343048 ("Purity/0FRt4SRqqt0FBt/event_purity‚0 *"Purity/0FRt2SRqqt3FBt/SRqqt_purity‚0.5 ?&Purity/0FRt2SRqqt1FBt/event_proportion‚0.009261592291295528 :"Purity/1FRt*SRqqt*FBt/event_purity‚0.06532695889472961 =&Purity/*FRt1SRqqt1FBt/event_proportion‚0.1001134067773819 9"Purity/2FRt*SRqqt1FBt/SRqqt_purity‚0.3536461889743805 ?&Purity/2FRt0SRqqt0FBt/event_proportion‚0.024571573361754417 ("Purity/2FRt0SRqqt3FBt/event_purity‚0 ) Purity/*FRt1SRqqt4FBt/FBt_purity‚0.75 & Purity/*FRt3SRqqt2FBt/FBt_purity‚1 ,&Purity/4FRt3SRqqt4FBt/event_proportion‚0 & Purity/0FRt3SRqqt*FBt/FBt_purity‚1 ("Purity/*FRt2SRqqt3FBt/event_purity‚0 8"Purity/1FRt1SRqqt2FBt/SRqqt_purity‚0.422789067029953 ,&Purity/1FRt2SRqqt4FBt/event_proportion‚0 ?&Purity/2FRt3SRqqt0FBt/event_proportion‚0.003654233878478408 9"Purity/3FRt*SRqqt*FBt/SRqqt_purity‚0.2526409327983856 8 Purity/*FRt*SRqqt0FBt/FRt_purity‚0.16120687127113342 >&Purity/*FRt0SRqqt0FBt/event_proportion‚0.15757308900356293 ("Purity/*FRt3SRqqt1FBt/event_purity‚0 :"Purity/0FRt1SRqqt0FBt/event_purity‚0.22838279604911804 8 Purity/1FRt1SRqqt*FBt/FRt_purity‚0.22650501132011414 ,&Purity/3FRt1SRqqt3FBt/event_proportion‚0 8 Purity/3FRt2SRqqt0FBt/FRt_purity‚0.15217390656471252 'particle/accuracy‚0.841109037399292 7 Purity/*FRt*SRqqt2FBt/FBt_purity‚0.9837027192115784 :"Purity/*FRt0SRqqt2FBt/event_purity‚0.22215358912944794 @&Purity/*FRt1SRqqt3FBt/event_proportion‚0.0006930443341843784 ;"Purity/1FRt1SRqqt*FBt/event_purity‚0.011829168535768986 ,&Purity/2FRt*SRqqt4FBt/event_proportion‚0 +jet/accuracy_3_of_6‚0.14912281930446625 jet/accuracy_2_of_7‚0 ("Purity/1FRt3SRqqt2FBt/event_purity‚0 8 Purity/1FRt0SRqqt2FBt/FRt_purity‚0.42576900124549866 ("Purity/1FRt3SRqqt0FBt/event_purity‚0 @&Purity/3FRt*SRqqt0FBt/event_proportion‚0.0071824598126113415 * Purity/1FRt*SRqqt4FBt/FBt_purity‚0.875 ("Purity/*FRt1SRqqt4FBt/SRqqt_purity‚1 7 Purity/*FRt2SRqqt3FBt/FBt_purity‚0.6666666865348816 8 Purity/2FRt*SRqqt*FBt/FRt_purity‚0.19616787135601044 =&Purity/0FRt1SRqqt*FBt/event_proportion‚0.2305317521095276 ,&Purity/0FRt1SRqqt4FBt/event_proportion‚0 ("Purity/1FRt2SRqqt0FBt/event_purity‚0 9"Purity/3FRt2SRqqt1FBt/SRqqt_purity‚0.5833333134651184 & Purity/2FRt3SRqqt1FBt/FBt_purity‚1 & Purity/4FRt2SRqqt0FBt/FRt_purity‚0 9"Purity/*FRt0SRqqt1FBt/event_purity‚0.6676110029220581 ,&Purity/0FRt4SRqqt4FBt/event_proportion‚0 7 Purity/2FRt0SRqqt1FBt/FRt_purity‚0.2747090458869934 9"Purity/3FRt*SRqqt1FBt/SRqqt_purity‚0.5722222924232483 @&Purity/4FRt1SRqqt0FBt/event_proportion‚6.300403038039804e-05 particle/accuracy_1_of_3‚1 :"Purity/0FRt2SRqqt*FBt/SRqqt_purity‚0.25006935000419617 ?&Purity/1FRt2SRqqt1FBt/event_proportion‚0.008946572430431843 & Purity/4FRt*SRqqt*FBt/FRt_purity‚0 =&Purity/*FRt2SRqqt*FBt/event_proportion‚0.1332535296678543 7 Purity/2FRt4SRqqt0FBt/FRt_purity‚0.4000000059604645 ?&Purity/*FRt3SRqqt*FBt/event_proportion‚0.020476309582591057 7 Purity/0FRt0SRqqt3FBt/FBt_purity‚0.9244444966316223 ,&Purity/0FRt2SRqqt4FBt/event_proportion‚0 :"Purity/3FRt2SRqqt0FBt/SRqqt_purity‚0.23369567096233368 ,&Purity/4FRt*SRqqt2FBt/event_proportion‚0 *jet/accuracy_1_of_3‚0.7273326516151428 9"Purity/*FRt0SRqqt3FBt/event_purity‚0.1071428582072258 ;"Purity/*FRt1SRqqt2FBt/event_purity‚0.005236698780208826 ?&Purity/0FRt2SRqqt0FBt/event_proportion‚0.043283771723508835 9"Purity/1FRt*SRqqt2FBt/SRqqt_purity‚0.3992063105106354 :"Purity/2FRt1SRqqt0FBt/SRqqt_purity‚0.19870418310165405 ("Purity/3FRt2SRqqt*FBt/event_purity‚0 ("Purity/1FRt4SRqqt*FBt/event_purity‚0 >&Purity/0FRt*SRqqt*FBt/event_proportion‚0.40996724367141724 9"Purity/1FRt*SRqqt0FBt/SRqqt_purity‚0.2026154100894928 & Purity/0FRt4SRqqt*FBt/FBt_purity‚1 jet/accuracy_5_of_6‚0 particle/accuracy_5_of_6‚1 ,"Purity/3FRt4SRqqt*FBt/SRqqt_purity‚0.125 7 Purity/0FRt1SRqqt3FBt/FBt_purity‚0.8333333730697632 ("Purity/4FRt1SRqqt0FBt/SRqqt_purity‚0 9"Purity/*FRt1SRqqt0FBt/SRqqt_purity‚0.2129640430212021 9"Purity/*FRt*SRqqt2FBt/SRqqt_purity‚0.4376591444015503 ?&Purity/1FRt1SRqqt2FBt/event_proportion‚0.006111391354352236 ("Purity/3FRt*SRqqt2FBt/event_purity‚0 ("Purity/3FRt3SRqqt*FBt/event_purity‚0 ("Purity/4FRt0SRqqt0FBt/event_purity‚0 >&Purity/*FRt2SRqqt1FBt/event_proportion‚0.02255544438958168 7 Purity/*FRt*SRqqt*FBt/FBt_purity‚0.9904725551605225 8 Purity/*FRt0SRqqt1FBt/FRt_purity‚0.30280929803848267 @&Purity/*FRt0SRqqt4FBt/event_proportion‚0.0003150201519019902 7 Purity/1FRt0SRqqt1FBt/FBt_purity‚0.9988052845001221 @&Purity/1FRt0SRqqt4FBt/event_proportion‚6.300403038039804e-05 ;"Purity/1FRt1SRqqt0FBt/event_purity‚0.006399922538548708 ;"Purity/1FRt1SRqqt1FBt/event_purity‚0.035152945667505264 7 Purity/1FRt0SRqqt3FBt/FRt_purity‚0.2142857164144516 =&Purity/1FRt1SRqqt*FBt/event_proportion‚0.1723790317773819 >&Purity/1FRt1SRqqt1FBt/event_proportion‚0.03944052383303642 @&Purity/2FRt*SRqqt3FBt/event_proportion‚6.300403038039804e-05 7 Purity/3FRt0SRqqt1FBt/FRt_purity‚0.3910256624221802 ,&Purity/3FRt4SRqqt3FBt/event_proportion‚0 :"Purity/*FRt0SRqqt4FBt/event_purity‚0.20000000298023224 & Purity/3FRt2SRqqt*FBt/FBt_purity‚1 ?&Purity/*FRt2SRqqt2FBt/event_proportion‚0.002079133177176118 >&Purity/0FRt0SRqqt*FBt/event_proportion‚0.11857359111309052 ,&Purity/0FRt3SRqqt2FBt/event_proportion‚0 7 Purity/1FRt1SRqqt1FBt/FRt_purity‚0.2837839722633362 & Purity/3FRt1SRqqt*FBt/FBt_purity‚1 A&Purity/3FRt4SRqqt*FBt/event_proportion‚0.00012600806076079607 ,&Purity/4FRt3SRqqt0FBt/event_proportion‚0 particle/accuracy_2_of_6‚1 :"Purity/1FRt1SRqqt1FBt/SRqqt_purity‚0.41902393102645874 & Purity/1FRt2SRqqt*FBt/FBt_purity‚1 & Purity/2FRt1SRqqt1FBt/FBt_purity‚1 particle/accuracy_3_of_5‚1 jet/accuracy_5_of_7‚0 particle/accuracy_3_of_7‚1 +"Purity/3FRt1SRqqt2FBt/SRqqt_purity‚0.25 9"Purity/3FRt3SRqqt0FBt/SRqqt_purity‚0.2142857313156128 9"Purity/0FRt1SRqqt*FBt/SRqqt_purity‚0.2792723774909973 ("Purity/1FRt3SRqqt*FBt/event_purity‚0 7 Purity/*FRt1SRqqt*FBt/FBt_purity‚0.9898663759231567 & Purity/*FRt2SRqqt1FBt/FBt_purity‚1 @&Purity/3FRt*SRqqt1FBt/event_proportion‚0.0024571572430431843 ("Purity/1FRt4SRqqt0FBt/event_purity‚0 ( Purity/2FRt2SRqqt2FBt/FRt_purity‚0.5 =&Purity/*FRt*SRqqt0FBt/event_proportion‚0.6160534024238586 ,&Purity/2FRt†È(åµ!2SRqqt4FBt/event_proportion‚0 :"Purity/2FRt3SRqqt0FBt/SRqqt_purity‚0.17875181138515472 @&Purity/4FRt1SRqqt*FBt/event_proportion‚6.300403038039804e-05 ,&Purity/4FRt2SRqqt2FBt/event_proportion‚0 particle/accuracy_2_of_5‚1 +jet/accuracy_4_of_6‚0.02631578966975212 & Purity/1FRt*SRqqt4FBt/FRt_purity‚0 A&Purity/1FRt*SRqqt4FBt/event_proportion‚0.00012600806076079607 ("Purity/1FRt2SRqqt*FBt/event_purity‚0 6 Purity/2FRt0SRqqt*FBt/FRt_purity‚0.171502947807312 *jet/accuracy_1_of_1‚0.3471873700618744 particle/accuracy_2_of_2‚1 9"Purity/1FRt3SRqqt2FBt/SRqqt_purity‚0.3333333432674408 ("Purity/4FRt*SRqqt0FBt/event_purity‚0 trainer/global_step‚1811 9"Purity/*FRt3SRqqt*FBt/SRqqt_purity‚0.2157500833272934 :"Purity/0FRt*SRqqt3FBt/event_purity‚0.08024691790342331 ,&Purity/2FRt1SRqqt4FBt/event_proportion‚0 ("Purity/3FRt*SRqqt0FBt/event_purity‚0 @&Purity/3FRt2SRqqt0FBt/event_proportion‚0.0024571572430431843 +jet/accuracy_2_of_3‚0.18661580979824066 *jet/accuracy_1_of_5‚0.8581917881965637 A&Purity/0FRt1SRqqt3FBt/event_proportion‚0.00044102821266278625 ("Purity/1FRt1SRqqt2FBt/event_purity‚0 @&Purity/1FRt1SRqqt4FBt/event_proportion‚6.300403038039804e-05 8 Purity/2FRt*SRqqt1FBt/FRt_purity‚0.30697375535964966 9"Purity/*FRt*SRqqt3FBt/SRqqt_purity‚0.7857142686843872 ("Purity/2FRt1SRqqt*FBt/event_purity‚0 ("Purity/3FRt*SRqqt1FBt/event_purity‚0 ,&Purity/3FRt3SRqqt4FBt/event_proportion‚0 particle/accuracy_1_of_2‚1 particle/accuracy_4_of_7‚1 ("Purity/3FRt1SRqqt2FBt/event_purity‚0 ("Purity/*FRt3SRqqt0FBt/event_purity‚0 :"Purity/1FRt*SRqqt*FBt/SRqqt_purity‚0.24936485290527344 7 Purity/1FRt0SRqqt3FBt/FBt_purity‚0.9523809552192688 & Purity/2FRt2SRqqt1FBt/FBt_purity‚1 ,&Purity/4FRt1SRqqt1FBt/event_proportion‚0 ("Purity/3FRt2SRqqt1FBt/event_purity‚0 8 Purity/*FRt2SRqqt2FBt/FRt_purity‚0.24358974397182465 :"Purity/*FRt2SRqqt0FBt/SRqqt_purity‚0.21457813680171967 ,&Purity/0FRt3SRqqt4FBt/event_proportion‚0 ("Purity/2FRt3SRqqt*FBt/event_purity‚0 ."Purity/0FRt4SRqqt*FBt/SRqqt_purity‚0.15625 ("Purity/2FRt4SRqqt0FBt/event_purity‚0 9"Purity/*FRt*SRqqt*FBt/SRqqt_purity‚0.2550710439682007 :"Purity/*FRt*SRqqt2FBt/event_purity‚0.16121737658977509 =&Purity/*FRt1SRqqt*FBt/event_proportion‚0.4482736885547638 6 Purity/0FRt1SRqqt1FBt/FBt_purity‚0.996178150177002 @&Purity/0FRt4SRqqt*FBt/event_proportion‚0.0005040322430431843 ,&Purity/2FRt0SRqqt4FBt/event_proportion‚0 7 Purity/2FRt1SRqqt2FBt/FRt_purity‚0.2850877344608307 0"Purity/*FRt4SRqqt*FBt/SRqqt_purity‚ 0.1640625 7 Purity/*FRt0SRqqt*FBt/FRt_purity‚0.1807754933834076 :"Purity/1FRt0SRqqt2FBt/event_purity‚0.08839864283800125 8 Purity/3FRt1SRqqt0FBt/FRt_purity‚0.10972223430871964 ,&Purity/4FRt1SRqqt2FBt/event_proportion‚0 +jet/accuracy_2_of_2‚0.08238111436367035 ,jet/accuracy_3_of_3‚0.016463248059153557 & Purity/1FRt2SRqqt2FBt/FBt_purity‚1 jet/accuracy_3_of_7‚0 9"Purity/*FRt1SRqqt2FBt/SRqqt_purity‚0.4573615491390228 9"Purity/*FRt3SRqqt1FBt/SRqqt_purity‚0.3373015820980072 9"Purity/0FRt0SRqqt1FBt/event_purity‚0.9993796944618225 7 Purity/0FRt0SRqqt1FBt/FBt_purity‚0.9993796944618225 :"Purity/1FRt*SRqqt2FBt/event_purity‚0.05831947550177574 8 Purity/1FRt2SRqqt0FBt/FRt_purity‚0.21108506619930267 ?&Purity/2FRt0SRqqt*FBt/event_proportion‚0.037046369165182114 8 Purity/3FRt0SRqqt*FBt/FRt_purity‚0.20959597826004028 :"Purity/0FRt*SRqqt*FBt/SRqqt_purity‚0.26566898822784424 7 Purity/*FRt1SRqqt2FBt/FBt_purity‚0.9764847755432129 @&Purity/3FRt2SRqqt*FBt/event_proportion‚0.0028351815417408943 +"Purity/3FRt*SRqqt2FBt/SRqqt_purity‚0.25 & Purity/3FRt1SRqqt2FBt/FBt_purity‚1 ("Purity/3FRt0SRqqt2FBt/event_purity‚0 ("Purity/*FRt*SRqqt4FBt/SRqqt_purity‚1 8 Purity/*FRt1SRqqt0FBt/FRt_purity‚0.18739721179008484 :"Purity/*FRt*SRqqt0FBt/SRqqt_purity‚0.21240432560443878 +particle/specificity‚0.9561266303062439 particle/accuracy_1_of_7‚1 & Purity/4FRt1SRqqt0FBt/FRt_purity‚0 & Purity/1FRt3SRqqt1FBt/FBt_purity‚1 9"Purity/*FRt*SRqqt0FBt/event_purity‚0.0880388393998146 :"Purity/*FRt1SRqqt*FBt/event_purity‚0.10754719376564026 8 Purity/*FRt1SRqqt*FBt/FRt_purity‚0.21578729152679443 A&Purity/0FRt0SRqqt4FBt/event_proportion‚0.00025201612152159214 & Purity/4FRt1SRqqt*FBt/FRt_purity‚0 8 Purity/*FRt0SRqqt0FBt/FRt_purity‚0.10613634437322617 :"Purity/1FRt3SRqqt1FBt/SRqqt_purity‚0.36666664481163025 ?&Purity/3FRt*SRqqt*FBt/event_proportion‚0.009954636916518211 7 Purity/3FRt1SRqqt*FBt/FRt_purity‚0.1769547164440155 & Purity/*FRt4SRqqt1FBt/FBt_purity‚1 & Purity/3FRt0SRqqt2FBt/FBt_purity‚1 ("Purity/4FRt1SRqqt*FBt/event_purity‚0 7 Purity/*FRt0SRqqt2FBt/FBt_purity‚0.9865975379943848 8 Purity/1FRt*SRqqt2FBt/FRt_purity‚0.39605605602264404 8 Purity/1FRt2SRqqt*FBt/FRt_purity‚0.21427854895591736 9"Purity/1FRt2SRqqt1FBt/SRqqt_purity‚0.3591013550758362 8 Purity/3FRt0SRqqt0FBt/FRt_purity‚0.07189542055130005 particle/accuracy_1_of_6‚1 & Purity/1FRt0SRqqt4FBt/FBt_purity‚1 ;"Purity/*FRt2SRqqt0FBt/event_purity‚0.005045590456575155 @&Purity/*FRt2SRqqt3FBt/event_proportion‚6.300403038039804e-05 ,&Purity/*FRt4SRqqt4FBt/event_proportion‚0 ,&Purity/2FRt4SRqqt4FBt/event_proportion‚0 :"Purity/3FRt1SRqqt*FBt/SRqqt_purity‚0.31790125370025635 +"Purity/0FRt0SRqqt4FBt/event_purity‚0.25 :"Purity/2FRt4SRqqt0FBt/SRqqt_purity‚0.15000000596046448 _step‚146 @&Purity/1FRt2SRqqt2FBt/event_proportion‚0.0010080644860863686 particle/accuracy_7_of_7‚0 8 Purity/3FRt*SRqqt2FBt/FRt_purity‚0.20000000298023224 epoch‚2 ("Purity/1FRt0SRqqt3FBt/event_purity‚0 ("Purity/0FRt3SRqqt1FBt/event_purity‚0 :"Purity/*FRt1SRqqt1FBt/SRqqt_purity‚0.41432592272758484 9"Purity/*FRt3SRqqt0FBt/SRqqt_purity‚0.2003585398197174 7 Purity/2FRt0SRqqt*FBt/FBt_purity‚0.9967741966247559 ?&Purity/0FRt0SRqqt3FBt/event_proportion‚0.002583165420219302 A&Purity/0FRt4SRqqt0FBt/event_proportion‚0.00044102821266278625 + Purity/1FRt*SRqqt3FBt/FRt_purity‚0.1875 ?&Purity/2FRt*SRqqt2FBt/event_proportion‚0.005418346729129553 9"Purity/3FRt*SRqqt0FBt/SRqqt_purity‚0.2171768844127655 ("Purity/3FRt0SRqqt1FBt/event_purity‚0 ,&Purity/4FRt4SRqqt4FBt/event_proportion‚0 ("Purity/0FRt4SRqqt1FBt/event_purity‚0 7 Purity/1FRt*SRqqt1FBt/FBt_purity‚0.9993913769721985 @&Purity/1FRt*SRqqt3FBt/event_proportion‚0.0013230846961960196 @&Purity/4FRt*SRqqt0FBt/event_proportion‚0.0001890120911411941 ,&Purity/4FRt0SRqqt1FBt/event_proportion‚0 ("Purity/0FRt3SRqqt*FBt/event_purity‚0 particle/accuracy_2_of_7‚1 *"Purity/*FRt2SRqqt3FBt/SRqqt_purity‚0.5 ("Purity/4FRt2SRqqt*FBt/SRqqt_purity‚0 :"Purity/*FRt*SRqqt1FBt/SRqqt_purity‚0.39736515283584595 @&Purity/1FRt3SRqqt2FBt/event_proportion‚6.300403038039804e-05 7 Purity/2FRt0SRqqt2FBt/FBt_purity‚0.9940475821495056 ("Purity/2FRt3SRqqt0FBt/event_purity‚0 ,&Purity/4FRt1SRqqt4FBt/event_proportion‚0 & Purity/1FRt3SRqqt2FBt/FRt_purity‚1 ("Purity/4FRt*SRqqt0FBt/SRqqt_purity‚0 >&Purity/1FRt2SRqqt0FBt/event_proportion‚0.04731602966785431 ;"Purity/2FRt0SRqqt*FBt/event_purity‚0.006313070189207792 8 Purity/2FRt0SRqqt0FBt/FRt_purity‚0.10620943456888199 ,&Purity/2FRt2SRqqt3FBt/event_proportion‚0 @&Purity/0FRt4SRqqt1FBt/event_proportion‚6.300403038039804e-05 jet/accuracy_7_of_7‚0 ("Purity/*FRt1SRqqt4FBt/event_purity‚0 ?&Purity/2FRt3SRqqt*FBt/event_proportion‚0.004032257944345474 @&Purity/3FRt3SRqqt*FBt/event_proportion‚0.0005670362734235823 9"Purity/1FRt4SRqqt0FBt/SRqqt_purity‚0.1666666716337204 7 Purity/*FRt*SRqqt1FBt/FRt_purity‚0.2937448024749756 7 Purity/*FRt0SRqqt*FBt/FBt_purity‚0.9903352856636047 ("Purity/2FRt1SRqqt2FBt/event_purity‚0 ,&Purity/4FRt0SRqqt3FBt/event_proportion‚0 +jet/accuracy_3_of_5‚0.07509229332208633 7 Purity/3FRt2SRqqt1FBt/FRt_purity‚0.1666666716337204 :"Purity/0FRt*SRqqt2FBt/SRqqt_purity‚0.41083329916000366 & Purity/0FRt2SRqqt1FBt/FBt_purity‚1 ,&Purity/2FRt4SRqqt1FBt/event_proportion‚0 & Purity/3FRt*SRqqt1FBt/FBt_purity‚1 & Purity/3FRt0SRqqt*FBt/FBt_purity‚1 'particle/f_score‚0.3016413450241089 ;"Purity/*FRt2SRqqt*FBt/event_purity‚0.006479581352323294 7 Purity/1FRt1SRqqt*FBt/FBt_purity‚0.9891883730888367 :"Purity/1FRt1SRqqt0FBt/SRqqt_purity‚0.19414813816547394 >&Purity/2FRt*SRqqt0FBt/event_proportion‚0.07579384744167328 7 Purity/3FRt1SRqqt1FBt/FRt_purity‚0.3076923191547394 + Purity/0FRt0SRqqt4FBt/FBt_purity‚0.8125 ("Purity/3FRt3SRqqt0FBt/event_purity‚01ö7§=^ Ò¦ 8 Purity/*FRt0SRqqt3FBt/FRt_purity‚0.20000000298023224 >&Purity/*FRt1SRqqt0FBt/event_proportion‚0.33039313554763794 ("Purity/0FRt0SRqqt0FBt/event_purity‚1 ;"Purity/0FRt2SRqqt*FBt/event_purity‚0.015407760627567768 7 Purity/0FRt2SRqqt*FBt/FBt_purity‚0.9892473816871643 @&Purity/0FRt2SRqqt3FBt/event_proportion‚6.300403038039804e-05 & Purity/3FRt*SRqqt*FBt/FBt_purity‚1 & Purity/3FRt0SRqqt1FBt/FBt_purity‚1 7 Purity/*FRt0SRqqt1FBt/FBt_purity‚0.9991505742073059 8 Purity/1FRt0SRqqt1FBt/FRt_purity‚0.30332961678504944 @&Purity/1FRt0SRqqt3FBt/event_proportion‚0.0010710685746744275 +jet/accuracy_2_of_4‚0.29374340176582336 7 Purity/*FRt4SRqqt0FBt/FRt_purity‚0.2083333283662796 ("Purity/3FRt4SRqqt0FBt/event_purity‚0 & Purity/*FRt1SRqqt4FBt/FRt_purity‚0 ("Purity/*FRt3SRqqt*FBt/event_purity‚0 ,&Purity/*FRt4SRqqt2FBt/event_proportion‚0 9"Purity/0FRt0SRqqt*FBt/event_purity‚0.8084040880203247 7 Purity/2FRt1SRqqt1FBt/FRt_purity‚0.3138824999332428 & Purity/1FRt1SRqqt3FBt/FRt_purity‚0 @&Purity/*FRt*SRqqt4FBt/event_proportion‚0.0003780241822823882 @&Purity/0FRt*SRqqt3FBt/event_proportion‚0.0030871976632624865 & Purity/1FRt1SRqqt1FBt/FBt_purity‚1 ,&Purity/4FRt4SRqqt1FBt/event_proportion‚0 ,&Purity/4FRt4SRqqt2FBt/event_proportion‚0 ,jet/accuracy_3_of_4‚0.043030958622694016 particle/accuracy_3_of_3‚1 ("Purity/1FRt0SRqqt4FBt/event_purity‚0 7 Purity/*FRt2SRqqt1FBt/FRt_purity‚0.2415153682231903 & Purity/*FRt3SRqqt*FBt/FBt_purity‚1 ("Purity/3FRt1SRqqt1FBt/event_purity‚0 jet/accuracy_6_of_7‚0 & Purity/*FRt*SRqqt4FBt/FRt_purity‚0 & Purity/2FRt0SRqqt3FBt/FBt_purity‚1 9"Purity/*FRt1SRqqt3FBt/SRqqt_purity‚0.8333333730697632 ("Purity/0FRt1SRqqt3FBt/event_purity‚0 7 Purity/0FRt*SRqqt1FBt/FBt_purity‚0.9983028173446655 9"Purity/0FRt1SRqqt1FBt/SRqqt_purity‚0.4184369742870331 :"Purity/1FRt2SRqqt*FBt/SRqqt_purity‚0.24118562042713165 ,&Purity/4FRt4SRqqt3FBt/event_proportion‚0 jet/accuracy_5_of_5‚0 & Purity/4FRt0SRqqt*FBt/FRt_purity‚0 :"Purity/0FRt1SRqqt*FBt/event_purity‚0.20061224699020386 9"Purity/1FRt3SRqqt*FBt/SRqqt_purity‚0.2253115028142929 @&Purity/3FRt0SRqqt0FBt/event_proportion‚0.0015120967291295528 ?&Purity/*FRt1SRqqt2FBt/event_proportion‚0.017011089250445366 ("Purity/2FRt2SRqqt1FBt/event_purity‚0 8 Purity/2FRt3SRqqt*FBt/FRt_purity‚0.23100000619888306 7 Purity/3FRt3SRqqt0FBt/FRt_purity‚0.1666666567325592 *"Purity/2FRt2SRqqt2FBt/SRqqt_purity‚0.5 & Purity/2FRt*SRqqt1FBt/FBt_purity‚1 >&Purity/2FRt1SRqqt0FBt/event_proportion‚0.03181703761219978 7 Purity/1FRt4SRqqt0FBt/FRt_purity‚0.1666666716337204 ,&Purity/1FRt2SRqqt3FBt/event_proportion‚0 ,&Purity/2FRt3SRqqt3FBt/event_proportion‚0 ("Purity/3FRt0SRqqt0FBt/event_purity‚0 & Purity/2FRt*SRqqt3FBt/FRt_purity‚0 7 Purity/*FRt0SRqqt3FBt/FBt_purity‚0.9236112236976624 8"Purity/0FRt*SRqqt*FBt/event_purity‚0.347979336977005 ,&Purity/0FRt3SRqqt3FBt/event_proportion‚0 7 Purity/2FRt2SRqqt1FBt/FRt_purity‚0.2633928656578064 ?&Purity/*FRt*SRqqt3FBt/event_proportion‚0.004473286215215921 =&Purity/1FRt1SRqqt0FBt/event_proportion‚0.1265120953321457 :"Purity/0FRt3SRqqt0FBt/SRqqt_purity‚0.22251984477043152 >&Purity/*FRt*SRqqt1FBt/event_proportion‚0.27079132199287415 >&Purity/0FRt*SRqqt2FBt/event_proportion‚0.03944052383303642 >&Purity/1FRt0SRqqt*FBt/event_proportion‚0.19745463132858276 @&Purity/1FRt4SRqqt0FBt/event_proportion‚0.0003780241822823882 ("Purity/3FRt1SRqqt0FBt/event_purity‚0 9"Purity/0FRt2SRqqt1FBt/SRqqt_purity‚0.3635176718235016 @&Purity/0FRt3SRqqt1FBt/event_proportion‚0.0006300403038039804 :"Purity/1FRt0SRqqt0FBt/event_purity‚0.10697515308856964 8 Purity/1FRt1SRqqt2FBt/FRt_purity‚0.32006803154945374 ("Purity/2FRt2SRqqt0FBt/event_purity‚0 7 Purity/2FRt*SRqqt2FBt/FRt_purity‚0.3288889229297638 :"Purity/2FRt3SRqqt*FBt/SRqqt_purity‚0.18352378904819489 ("Purity/0FRt4SRqqt*FBt/event_purity‚0 8 Purity/1FRt0SRqqt*FBt/FRt_purity‚0.18164362013339996 :"Purity/*FRt*SRqqt*FBt/event_purity‚0.17893172800540924 :"Purity/0FRt2SRqqt0FBt/SRqqt_purity‚0.22733958065509796 ?&Purity/2FRt2SRqqt1FBt/event_proportion‚0.003969253972172737 ,&Purity/3FRt2SRqqt3FBt/event_proportion‚0 A&Purity/3FRt4SRqqt0FBt/event_proportion‚0.00012600806076079607 ("Purity/*FRt4SRqqt*FBt/event_purity‚0 ("Purity/1FRt1SRqqt4FBt/SRqqt_purity‚1 6 Purity/*FRt3SRqqt0FBt/FRt_purity‚0.259640097618103 :"Purity/0FRt*SRqqt1FBt/SRqqt_purity‚0.39769095182418823 8"Purity/0FRt1SRqqt2FBt/SRqqt_purity‚0.408214271068573 @&Purity/2FRt0SRqqt3FBt/event_proportion‚6.300403038039804e-05 :"Purity/2FRt1SRqqt*FBt/SRqqt_purity‚0.24897189438343048 ("Purity/0FRt4SRqqt0FBt/event_purity‚0 *"Purity/0FRt2SRqqt3FBt/SRqqt_purity‚0.5 ?&Purity/0FRt2SRqqt1FBt/event_proportion‚0.009261592291295528 :"Purity/1FRt*SRqqt*FBt/event_purity‚0.06532695889472961 =&Purity/*FRt1SRqqt1FBt/event_proportion‚0.1001134067773819 9"Purity/2FRt*SRqqt1FBt/SRqqt_purity‚0.3536461889743805 ?&Purity/2FRt0SRqqt0FBt/event_proportion‚0.024571573361754417 ("Purity/2FRt0SRqqt3FBt/event_purity‚0 ) Purity/*FRt1SRqqt4FBt/FBt_purity‚0.75 & Purity/*FRt3SRqqt2FBt/FBt_purity‚1 ,&Purity/4FRt3SRqqt4FBt/event_proportion‚0 & Purity/0FRt3SRqqt*FBt/FBt_purity‚1 ("Purity/*FRt2SRqqt3FBt/event_purity‚0 8"Purity/1FRt1SRqqt2FBt/SRqqt_purity‚0.422789067029953 ,&Purity/1FRt2SRqqt4FBt/event_proportion‚0 ?&Purity/2FRt3SRqqt0FBt/event_proportion‚0.003654233878478408 9"Purity/3FRt*SRqqt*FBt/SRqqt_purity‚0.2526409327983856 8 Purity/*FRt*SRqqt0FBt/FRt_purity‚0.16120687127113342 >&Purity/*FRt0SRqqt0FBt/event_proportion‚0.15757308900356293 ("Purity/*FRt3SRqqt1FBt/event_purity‚0 :"Purity/0FRt1SRqqt0FBt/event_purity‚0.22838279604911804 8 Purity/1FRt1SRqqt*FBt/FRt_purity‚0.22650501132011414 ,&Purity/3FRt1SRqqt3FBt/event_proportion‚0 8 Purity/3FRt2SRqqt0FBt/FRt_purity‚0.15217390656471252 'particle/accuracy‚0.841109037399292 7 Purity/*FRt*SRqqt2FBt/FBt_purity‚0.9837027192115784 :"Purity/*FRt0SRqqt2FBt/event_purity‚0.22215358912944794 @&Purity/*FRt1SRqqt3FBt/event_proportion‚0.0006930443341843784 ;"Purity/1FRt1SRqqt*FBt/event_purity‚0.011829168535768986 ,&Purity/2FRt*SRqqt4FBt/event_proportion‚0 +jet/accuracy_3_of_6‚0.14912281930446625 jet/accuracy_2_of_7‚0 ("Purity/1FRt3SRqqt2FBt/event_purity‚0 8 Purity/1FRt0SRqqt2FBt/FRt_purity‚0.42576900124549866 ("Purity/1FRt3SRqqt0FBt/event_purity‚0 @&Purity/3FRt*SRqqt0FBt/event_proportion‚0.0071824598126113415 * Purity/1FRt*SRqqt4FBt/FBt_purity‚0.875 ("Purity/*FRt1SRqqt4FBt/SRqqt_purity‚1 7 Purity/*FRt2SRqqt3FBt/FBt_purity‚0.6666666865348816 8 Purity/2FRt*SRqqt*FBt/FRt_purity‚0.19616787135601044 =&Purity/0FRt1SRqqt*FBt/event_proportion‚0.2305317521095276 ,&Purity/0FRt1SRqqt4FBt/event_proportion‚0 ("Purity/1FRt2SRqqt0FBt/event_purity‚0 9"Purity/3FRt2SRqqt1FBt/SRqqt_purity‚0.5833333134651184 & Purity/2FRt3SRqqt1FBt/FBt_purity‚1 & Purity/4FRt2SRqqt0FBt/FRt_purity‚0 9"Purity/*FRt0SRqqt1FBt/event_purity‚0.6676110029220581 ,&Purity/0FRt4SRqqt4FBt/event_proportion‚0 7 Purity/2FRt0SRqqt1FBt/FRt_purity‚0.2747090458869934 9"Purity/3FRt*SRqqt1FBt/SRqqt_purity‚0.5722222924232483 @&Purity/4FRt1SRqqt0FBt/event_proportion‚6.300403038039804e-05 particle/accuracy_1_of_3‚1 :"Purity/0FRt2SRqqt*FBt/SRqqt_purity‚0.25006935000419617 ?&Purity/1FRt2SRqqt1FBt/event_proportion‚0.008946572430431843 & Purity/4FRt*SRqqt*FBt/FRt_purity‚0 =&Purity/*FRt2SRqqt*FBt/event_proportion‚0.1332535296678543 7 Purity/2FRt4SRqqt0FBt/FRt_purity‚0.4000000059604645 ?&Purity/*FRt3SRqqt*FBt/event_proportion‚0.020476309582591057 7 Purity/0FRt0SRqqt3FBt/FBt_purity‚0.9244444966316223 ,&Purity/0FRt2SRqqt4FBt/event_proportion‚0 :"Purity/3FRt2SRqqt0FBt/SRqqt_purity‚0.23369567096233368 ,&Purity/4FRt*SRqqt2FBt/event_proportion‚0 *jet/accuracy_1_of_3‚0.7273326516151428 9"Purity/*FRt0SRqqt3FBt/event_purity‚0.1071428582072258 ;"Purity/*FRt1SRqqt2FBt/event_purity‚0.005236698780208826 ?&Purity/0FRt2SRqqt0FBt/event_proportion‚0.043283771723508835 9"Purity/1FRt*SRqqt2FBt/SRqqt_purity‚0.3992063105106354 :"Purity/2FRt1SRqqt0FBt/SRqqt_purity‚0.19870418310165405 ("Purity/3FRt2SRqqt*FBt/event_purity‚0 ("Purity/1FRt4SRqqt*FBt/event_purity‚0 >&Purity/0FRt*SRqqt*FBt/event_proportion‚0.40996724367141724 9"Purity/1FRt*SRqqt0FBt/SRqqt_purity‚0.2026154100894928 & Purity/0FRt4SRqqt*FBt/FBt_purity‚1 jet/accuracy_5_of_6‚0 particle/accuracy_5_of_6‚1 ,"Purity/3FRt4SRqqt*FBt/SRqqt_purity‚0.125 7 Purity/0FRt1SRqqt3FBt/FBt_purity‚0.8333333730697632 ("Purity/4FRt1SRqqt0FBt/SRqqt_purity‚0 9"Purity/*FRt1SRqqt0FBt/SRqqt_purity‚0.2129640430212021 9"Purity/*FRt*SRqqt2FBt/SRqqt_purity‚0.4376591444015503 ?&Purity/1FRt1SRqqt2FBt/event_proportion‚0.006111391354352236 ("Purity/3FRt*SRqqt2FBt/event_purity‚0 ("Purity/3FRt3SRqqt*FBt/event_purity‚0 ("Purity/4FRt0SRqqt0FBt/event_purity‚0 >&Purity/*FRt2SRqqt1FBt/event_proportion‚0.02255544438958168 7 Purity/*FRt*SRqqt*FBt/FBt_purity‚0.9904725551605225 8 Purity/*FRt0SRqqt1FBt/FRt_purity‚0.30280929803848267 @&Purity/*FRt0SRqqt4FBt/event_proportion‚0.0003150201519019902 7 Purity/1FRt0SRqqt1FBt/FBt_purity‚0.9988052845001221 @&Purity/1FRt0SRqqt4FBt/event_proportion‚6.300403038039804e-05 ;"Purity/1FRt1SRqqt0FBt/event_purity‚0.006399922538548708 ;"Purity/1FRt1SRqqt1FBt/event_purity‚0.035152945667505264 7 Purity/1FRt0SRqqt3FBt/FRt_purity‚0.2142857164144516 =&Purity/1FRt1SRqqt*FBt/event_proportion‚0.1723790317773819 >&Purity/1FRt1SRqqt1FBt/event_proportion‚0.03944052383303642 @&Purity/2FRt*SRqqt3FBt/event_proportion‚6.300403038039804e-05 7 Purity/3FRt0SRqqt1FBt/FRt_purity‚0.3910256624221802 ,&Purity/3FRt4SRqqt3FBt/event_proportion‚0 :"Purity/*FRt0SRqqt4FBt/event_purity‚0.20000000298023224 & Purity/3FRt2SRqqt*FBt/FBt_purity‚1 ?&Purity/*FRt2SRqqt2FBt/event_proportion‚0.002079133177176118 >&Purity/0FRt0SRqqt*FBt/event_proportion‚0.11857359111309052 ,&Purity/0FRt3SRqqt2FBt/event_proportion‚0 7 Purity/1FRt1SRqqt1FBt/FRt_purity‚0.2837839722633362 & Purity/3FRt1SRqqt*FBt/FBt_purity‚1 A&Purity/3FRt4SRqqt*FBt/event_proportion‚0.00012600806076079607 ,&Purity/4FRt3SRqqt0FBt/event_proportion‚0 particle/accuracy_2_of_6‚1 :"Purity/1FRt1SRqqt1FBt/SRqqt_purity‚0.41902393102645874 & Purity/1FRt2SRqqt*FBt/FBt_purity‚1 & Purity/2FRt1SRqqt1FBt/FBt_purity‚1 particle/accuracy_3_of_5‚1 jet/accuracy_5_of_7‚0 particle/accuracy_3_of_7‚1 +"Purity/3FRt1SRqqt2FBt/SRqqt_purity‚0.25 9"Purity/3FRt3SRqqt0FBt/SRqqt_purity‚0.2142857313156128 9"Purity/0FRt1SRqqt*FBt/SRqqt_purity‚0.2792723774909973 ("Purity/1FRt3SRqqt*FBt/event_purity‚0 7 Purity/*FRt1SRqqt*FBt/FBt_purity‚0.9898663759231567 & Purity/*FRt2SRqqt1FBt/FBt_purity‚1 @&Purity/3FRt*SRqqt1FBt/event_proportion‚0.0024571572430431843 ("Purity/1FRt4SRqqt0FBt/event_purity‚0 ( Purity/2FRt2SRqqt2FBt/FRt_purity‚0.5 =&Purity/*FRt*SRqqt0FBt/event_proportion‚0.6160534024238586 ,&Purity/2FRt2SRqqt4FBt/event_proportion‚0 :"Purity/2FRt3SRqqt0FBt/SRqqt_purity‚0.17875181138515472 @&Purity/4FRt1SRqqt*FBt/event_proportion‚6.300403038039804e-05 ,&Purity/4FRt2SRqqt2FBt/event_proportion‚0 particle/accuracy_2_of_5‚1 +jet/accuracy_4_of_6‚0.02631578966975212 & Purity/1FRt*SRqqt4FBt/FRt_purity‚0 A&Purity/1FRt*SRqqt4FBt/event_proportion‚0.00012600806076079607 ("Purity/1FRt2SRqqt*FBt/event_purity‚0 6 Purity/2FRt0SRqqt*FBt/FRt_purity‚0.171502947807312 *jet/accuracy_1_of_1‚0.3471873700618744 particle/accuracy_2_of_2‚1 9"Purity/1FRt3SRqqt2FBt/SRqqt_purity‚0.3333333432674408 ("Purity/4FRt*SRqqt0FBt/event_purity‚0 trainer/global_step‚1811 9"Purity/*FRt3SRqqt*FBt/SRqqt_purity‚0.2157500833272934 :"Purity/0FRt*SRqqt3FBt/event_purity‚0.08024691790342331 ,&Purity/2FRt1SRqqt4FBt/event_proportion‚0 ("Purity/3FRt*SRqqt0FBt/event_purity‚0 @&Purity/3FRt2SRqqt0FBt/event_proportion‚0.0024571572430431843 +jet/accuracy_2_of_3‚0.18661580979824066 *jet/accuracy_1_of_5‚0.8581917881965637 A&Purity/0FRt1SRqqt3FBt/event_proportion‚0.00044102821266278625 ("Purity/1FRt1SRqqt2FBt/event_purity‚0 @&Purity/1FRt1SRqqt4FBt/event_proportion‚6.300403038039804e-05 8 Purity/2FRt*SRqqt1FBt/FRt_purity‚0.30697375535964966 9"Purity/*FRt*SRqqt3FBt/SRqqt_purity‚0.7857142686843872 ("Purity/2FRt1SRqqt*FBt/event_purity‚0 ("Purity/3FRt*SRqqt1FBt/event_purity‚0 ,&Purity/3FRt3SRqqt4FBt/event_proportion‚0 particle/accuracy_1_of_2‚1 particle/accuracy_4_of_7‚1 ("Purity/3FRt1SRqqt2FBt/event_purity‚0 ("Purity/*FRt3SRqqt0FBt/event_purity‚0 :"Purity/1FRt*SRqqt*FBt/SRqqt_purity‚0.24936485290527344 7 Purity/1FRt0SRqqt3FBt/FBt_purity‚0.9523809552192688 & Purity/2FRt2SRqqt1FBt/FBt_purity‚1 ,&Purity/4FRt1SRqqt1FBt/event_proportion‚0 ("Purity/3FRt2SRqqt1FBt/event_purity‚0 8 Purity/*FRt2SRqqt2FBt/FRt_purity‚0.24358974397182465 :"Purity/*FRt2SRqqt0FBt/SRqqt_purity‚0.21457813680171967 ,&Purity/0FRt3SRqqt4FBt/event_proportion‚0 ("Purity/2FRt3SRqqt*FBt/event_purity‚0 ."Purity/0FRt4SRqqt*FBt/SRqqt_purity‚0.15625 ("Purity/2FRt4SRqqt0FBt/event_purity‚0 9"Purity/*FRt*SRqqt*FBt/SRqqt_purity‚0.2550710439682007 :"Purity/*FRt*SRqqt2FBt/event_purity‚0.16121737658977509 =&Purity/*FRt1SRqqt*FBt/event_proportion‚0.4482736885547638 6 Purity/0FRt1SRqqt1FBt/FBt_purity‚0.996178150177002 @&Purity/0FRt4SRqqt*FBt/event_proportion‚0.0005040322430431843 ,&Purity/2FRt0SRqqt4FBt/event_proportion‚0 7 Purity/2FRt1SRqqt2FBt/FRt_purity‚0.2850877344608307 0"Purity/*FRt4SRqqt*FBt/SRqqt_purity‚ 0.1640625 7 Purity/*FRt0SRqqt*FBt/FRt_purity‚0.1807754933834076 :"Purity/1FRt0SRqqt2FBt/event_purity‚0.08839864283800125 8 Purity/3FRt1SRqqt0FBt/FRt_purity‚0.10972223430871964 ,&Purity/4FRt1SRqqt2FBt/event_proportion‚0 +jet/accuracy_2_of_2‚0.08238111436367035 ,jet/accuracy_3_of_3‚0.016463248059153557 & Purity/1FRt2SRqqt2FBt/FBt_purity‚1 jet/accuracy_3_of_7‚0 9"Purity/*FRt1SRqqt2FBt/SRqqt_purity‚0.4573615491390228 9"Purity/*FRt3SRqqt1FBt/SRqqt_purity‚0.3373015820980072 9"Purity/0FRt0SRqqt1FBt/event_purity‚0.9993796944618225 7 Purity/0FRt0SRqqt1FBt/FBt_purity‚0.9993796944618225 :"Purity/1FRt*SRqqt2FBt/event_purity‚0.05831947550177574 8 Purity/1FRt2SRqqt0FBt/FRt_purity‚0.21108506619930267 ?&Purity/2FRt0SRqqt*FBt/event_proportion‚0.037046369165182114 8 Purity/3FRt0SRqqt*FBt/FRt_purity‚0.20959597826004028 :"Purity/0FRt*SRqqt*FBt/SRqqt_purity‚0.26566898822784424 7 Purity/*FRt1SRqqt2FBt/FBt_purity‚0.9764847755432129 @&Purity/3FRt2SRqqt*FBt/event_proportion‚0.0028351815417408943 +"Purity/3FRt*SRqqt2FBt/SRqqt_purity‚0.25 & Purity/3FRt1SRqqt2FBt/FBt_purity‚1 ("Purity/3FRt0SRqqt2FBt/event_purity‚0 ("Purity/*FRt*SRqqt4FBt/SRqqt_purity‚1 8 Purity/*FRt1SRqqt0FBt/FRt_purity‚0.18739721179008484 :"Purity/*FRt*SRqqt0FBt/SRqqt_purity‚0.21240432560443878 +particle/specificity‚0.9561266303062439 particle/accuracy_1_of_7‚1 & Purity/4FRt1SRqqt0FBt/FRt_purity‚0 & Purity/1FRt3SRqqt1FBt/FBt_purity‚1 9"Purity/*FRt*SRqqt0FBt/event_purity‚0.0880388393998146 :"Purity/*FRt1SRqqt*FBt/event_purity‚0.10754719376564026 8 Purity/*FRt1SRqqt*FBt/FRt_purity‚0.21578729152679443 A&Purity/0FRt0SRqqt4FBt/event_proportion‚0.00025201612152159214 & Purity/4FRt1SRqqt*FBt/FRt_purity‚0 8 Purity/*FRt0SRqqt0FBt/FRt_purity‚0.10613634437322617 :"Purity/1FRt3SRqqt1FBt/SRqqt_purity‚0.36666664481163025 ?&Purity/3FRt*SRqqt*FBt/event_proportion‚0.009954636916518211 7 Purity/3FRt1SRqqt*FBt/FRt_purity‚0.1769547164440155 & Purity/*FRt4SRqqt1FBt/FBt_purity‚1 & Purity/3FRt0SRqqt2FBt/FBt_purity‚1 ("Purity/4FRt1SRqqt*FBt/event_purity‚0 7 Purity/*FRt0SRqqt2FBt/FBt_purity‚0.9865975379943848 8 Purity/1FRt*SRqqt2FBt/FRt_purity‚0.39605605602264404 8 Purity/1FRt2SRqqt*FBt/FRt_purity‚0.21427854895591736 9"Purity/1FRt2SRqqt1FBt/SRqqt_purity‚0.3591013550758362 8 Purity/3FRt0SRqqt0FBt/FRt_purity‚0.07189542055130005 particle/accuracy_1_of_6‚1 & Purity/1FRt0SRqqt4FBt/FBt_purity‚1 ;"Purity/*FRt2SRqqt0FBt/event_purity‚0.005045590456575155 @&Purity/*FRt2SRqqt3FBt/event_proportion‚6.300403038039804e-05 ,&Purity/*FRt4SRqqt4FBt/event_proportion‚0 ,&Purity/2FRt4SRqqt4FBt/event_proportion‚0 :"Purity/3FRt1SRqqt*FBt/SRqqt_purity‚0.31790125370025635 +"Purity/0FRt0SRqqt4FBt/event_purity‚0.25 :"Purity/2FRt4SRqqt0FBt/SRqqt_purity‚0.15000000596046448 _step‚146 @&Purity/1FRt2SRqqt2FBt/event_proportion‚0.0010080644860863686 particle/accuracy_7_of_7‚0 8 Purity/3FRt*SRqqt2FBt/FRt_purity‚0.20000000298023224 epoch‚2 ("Purity/1FRt0SRqqt3FBt/event_purity‚0 ("Purity/0FRt3SRqqt1FBt/event_purity‚0 :"Purity/*FRt1SRqqt1FBt/SRqqt_purity‚0.41432592272758484 9"Purity/*FRt3SRqqt0FBt/SRqqt_purity‚0.2003585398197174 7 Purity/2FRt0SRqqt*FBt/FBt_purity‚0.9967741966247559 ?&Purity/0FRt0SRqqt3FBt/event_proportion‚0.002583165420219302 A&Purity/0FRt4SRqqt0FBt/event_proportion‚0.00044102821266278625 + Purity/1FRt*SRqqt3FBt/FRt_purity‚0.1875 ?&Purity/2FRt*SRqqt2FBt/event_proportion‚0.005418346729129553 9"Purity/3FRt*SRqqt0FBt/SRqqt_purity‚0.2171768844127655 ("Purity/3FRt0SRqqt1FBt/event_purity‚0 ,&Purity/4FRt4SRqqt4FBt/event_proportion‚0 ("Purity/0FRt4SRqqt1FBt/event_purity‚0 7 Purity/1FRt*SRqqt1FBt/FBt_purity‚0.9993913769721985 @&Purity/1FRt*SRqqt3FBt/event_proportion‚0.0013230846961960196 @&Purity/4FRt*SRqqt0FBt/event_proportion‚0.0001890120911411941 ,&Purity/4FRt0SRqqt1FBt/event_proportion‚0 ("Purity/0FRt3SRqqt*FBt/event_purity‚0 particle/accuracy_2_of_7‚1 *"Purity/*FRt2SRqqt3FBt/SRqqt_purity‚0.5 ("Purity/4FRt2SRqqt*FBt/SRqqt_purity‚0 :"Purity/*FRt*SRqqt1FBt/SRqqt_purity‚0.39736515283584595 @&Purity/1FRt3SRqqt2FBt/event_proportion‚6.300403038039804e-05 7 Purity/2FRt0SRqqt2FBt/FBt_purity‚0.9940475821495056 ("Purity/2FRt3SRqqt0FBt/event_purity‚0 ,&Purity/4FRt1SRqqt4FBt/event_proportion‚0 & Purity/1FRt3SRqqt2FBt/FRt_purity‚1 ("Purity/4FRt*SRqqt0FBt/SRqqt_purity‚0 >&Purity/1FRt2SRqqt0FBt/event_proportion‚0.04731602966785431 ;"Purity/2FRt0SRqqt*FBt/event_purity‚0.006313070189207792 8 Purity/2FRt0SRqqt0FBt/FRt_purity‚0.10620943456888199 ,&Purity/2FRt2SRqqt3FBt/event_proportion‚0 @&Purity/0FRt4SRqqt1FBt/event_proportion‚6.300403038039804e-05 jet/accuracy_7_of_7‚0 ("Purity/*FRt1SRqqt4FBt/event_purity‚0 ?&Purity/2FRt3SRqqt*FBt/event_proportion‚0.004032257944345474 @&Purity/3FRt3SRqqt*FBt/event_proportion‚0.0005670362734235823 9"Purity/1FRt4SRqqt0FBt/SRqqt_purity‚0.1666666716337204 7 Purity/*FRt*SRqqt1FBt/FRt_purity‚0.2937448024749756 7 Purity/*FRt0SRqqt*FBt/FBt_purity‚0.9903352856636047 ("Purity/2FRt1SRqqt2FBt/event_purity‚0 ,&Purity/4FRt0SRqqt3FBt/event_proportion‚0 +jet/accuracy_3_of_5‚0.07509229332208633 7 Purity/3FRt2SRqqt1FBt/FRt_purity‚0.1666666716337204 :"Purity/0FRt*SRqqt2FBt/SRqqt_purity‚0.41083329916000366 & Purity/0FRt2SRqqt1FBt/FBt_purity‚1 ,&Purity/2FRt4SRqqt1FBt/event_proportion‚0 & Purity/3FRt*SRqqt1FBt/FBt_purity‚1 & Purity/3FRt0SRqqt*FBt/FBt_purity‚1 'particle/f_score‚0.3016413450241089 ;"Purity/*FRt2SRqqt*FBt/event_purity‚0.006479581352323294 7 Purity/1FRt1SRqqt*FBt/FBt_purity‚0.9891883730888367 :"Purity/1FRt1SRqqt0FBt/SRqqt_purity‚0.19414813816547394 >&Purity/2FRt*SRqqt0FBt/event_proportion‚0.07579384744167328 7 Purity/3FRt1SRqqt1FBt/FRt_purity‚0.3076923191547394 + Purity/0FRt0SRqqt4FBt/FBt_purity‚0.8125 ("Purity/3FRt3SRqqt0FBt/event_purity‚0 ?&Purity/1FRt0SRqqt1FBt/event_proportion‚0.049773186445236206 8 Purity/1FRt2SRqqt1FBt/FRt_purity‚0.22004608809947968 ,&Purity/4FRt2SRqqt4FBt/event_proportion‚0 ,&Purity/4FRt3SRqqt*FBt/event_proportion‚0 7validation_average_jet_accuracy‚0.33369380235671997 particle/accuracy_4_of_6‚1 particle/accuracy_5_of_7‚1 7 Purity/0FRt*SRqqt*FBt/FBt_purity‚0.9887565970420837 ;"Purity/1FRt*SRqqt0FBt/event_purity‚0.047739025205373764 A&Purity/1FRt1SRqqt3FBt/event_proportion‚0.00025201612152159214 ("Purity/4FRt1SRqqt0FBt/event_purity‚0 9"Purity/*FRt1SRqqt0FBt/event_purity‚0.1196458712220192 8 Purity/*FRt1SRqqt1FBt/FRt_purity‚0.29598844051361084 & Purity/*FRt3SRqqt1FBt/FBt_purity‚1 >&Purity/0FRt1SRqqt1FBt/event_proportion‚0.05185231938958168 ("Purity/3FRt0SRqqt*FBt/event_purity‚0 ,&Purity/3FRt3SRqqt2FBt/event_proportion‚0 ,&Purity/4FRt0SRqqt4FBt/event_proportion‚0 9"Purity/*FRt*SRqqt4FBt/event_purity‚0.1666666716337204 >&Purity/0FRt2SRqqt*FBt/event_proportion‚0.05361643061041832 ,&Purity/3FRt1SRqqt4FBt/event_proportion‚0 ("Purity/3FRt2SRqqt0FBt/event_purity‚0 ,&Purity/3FRt2SRqqt4FBt/event_proportion‚0 & Purity/1FRt0SRqqt4FBt/FRt_purity‚0 :"Purity/*FRt*SRqqt1FBt/event_purity‚0.39189282059669495 ?&Purity/*FRt0SRqqt3FBt/event_proportion‚0.003717237850651145 ,&Purity/1FRt3SRqqt3FBt/event_proportion‚0 ?&Purity/2FRt*SRqqt1FBt/event_proportion‚0.021169355139136314 ("Purity/2FRt0SRqqt0FBt/event_purity‚0 ("Purity/2FRt1SRqqt0FBt/event_purity‚0 ,&Purity/3FRt4SRqqt2FBt/event_proportion‚0 >&Purity/1FRt0SRqqt0FBt/event_proportion‚0.13142640888690948 ,&Purity/2FRt3SRqqt4FBt/event_proportion‚0 @&Purity/3FRt3SRqqt0FBt/event_proportion‚0.0005670362734235823 ,&Purity/4FRt4SRqqt*FBt/event_proportion‚0 :"Purity/2FRt4SRqqt*FBt/SRqqt_purity‚0.15000000596046448 & Purity/4FRt*SRqqt0FBt/FRt_purity‚0 =&Purity/*FRt0SRqqt2FBt/event_proportion‚0.0483240932226181 7 Purity/2FRt*SRqqt*FBt/FBt_purity‚0.9961161017417908 :"Purity/2FRt2SRqqt*FBt/SRqqt_purity‚0.22519007325172424 particle/accuracy_3_of_4‚1 ("Purity/0FRt4SRqqt1FBt/SRqqt_purity‚0 7 Purity/2FRt4SRqqt*FBt/FRt_purity‚0.4000000059604645 % _timestamp‚1.7884159477396815e+09 :"Purity/*FRt1SRqqt1FBt/event_purity‚0.08534605801105499 @&Purity/*FRt4SRqqt0FBt/event_proportion‚0.0012600806076079607 :"Purity/1FRt2SRqqt0FBt/SRqqt_purity‚0.21041135489940643 ,&Purity/1FRt3SRqqt4FBt/event_proportion‚0 9"Purity/2FRt1SRqqt2FBt/SRqqt_purity‚0.6228070259094238 @&Purity/3FRt*SRqqt2FBt/event_proportion‚0.0003150201519019902 @&Purity/3FRt0SRqqt2FBt/event_proportion‚6.300403038039804e-05 *jet/accuracy_1_of_2‚0.6041082143783569 :"Purity/0FRt*SRqqt2FBt/event_purity‚0.23743848502635956 ("Purity/2FRt0SRqqt2FBt/event_purity‚0 ,&Purity/4FRt*SRqqt3FBt/event_proportion‚0 particle/accuracy_1_of_5‚1 9"Purity/0FRt3SRqqt1FBt/SRqqt_purity‚0.3541666567325592 & Purity/*FRt3SRqqt2FBt/FRt_purity‚1 7 Purity/0FRt2SRqqt3FBt/FBt_purity‚0.6666666865348816 & Purity/4FRt2SRqqt*FBt/FRt_purity‚0 @&Purity/*FRt4SRqqt1FBt/event_proportion‚6.300403038039804e-05 :"Purity/1FRt3SRqqt0FBt/SRqqt_purity‚0.19502051174640656 @&Purity/1FRt4SRqqt*FBt/event_proportion‚0.0003780241822823882 7 Purity/2FRt*SRqqt2FBt/FBt_purity‚0.9911110997200012 jet/accuracy_4_of_7‚0 <"Purity/2FRt*SRqqt*FBt/event_purity‚0.0026922288816422224 8 Purity/2FRt*SRqqt0FBt/FRt_purity‚0.15440437197685242 ?&Purity/2FRt1SRqqt2FBt/event_proportion‚0.001890120911411941 ("Purity/4FRt*SRqqt*FBt/SRqqt_purity‚0 8 Purity/*FRt*SRqqt*FBt/FRt_purity‚0.20124734938144684 :"Purity/0FRt1SRqqt0FBt/SRqqt_purity‚0.22838279604911804 8 Purity/1FRt1SRqqt0FBt/FRt_purity‚0.20677201449871063 ("Purity/1FRt2SRqqt1FBt/event_purity‚0 8 Purity/3FRt2SRqqt*FBt/FRt_purity‚0.14814813435077667 >&Purity/1FRt*SRqqt1FBt/event_proportion‚0.09929435700178146 7 Purity/1FRt1SRqqt2FBt/FBt_purity‚0.9732142686843872 ("Purity/1FRt3SRqqt1FBt/event_purity‚0 ,&Purity/4FRt0SRqqt2FBt/event_proportion‚0 ,&Purity/4FRt3SRqqt3FBt/event_proportion‚0 jet/accuracy_6_of_6‚0 + Purity/1FRt*SRqqt3FBt/FBt_purity‚0.9375 @&Purity/*FRt1SRqqt4FBt/event_proportion‚6.300403038039804e-05 @&Purity/3FRŽ¢²-5t1SRqqt1FBt/event_proportion‚0.0009450604557059705 9"Purity/0FRt*SRqqt3FBt/SRqqt_purity‚0.6333333253860474 9"Purity/0FRt0SRqqt3FBt/event_purity‚0.1599999964237213 =&Purity/0FRt1SRqqt0FBt/event_proportion‚0.1694808453321457 ?&Purity/*FRt3SRqqt1FBt/event_proportion‚0.002142137149348855 :"Purity/*FRt*SRqqt3FBt/event_purity‚0.06609195470809937 >&Purity/0FRt*SRqqt1FBt/event_proportion‚0.14787046611309052 7 Purity/0FRt*SRqqt2FBt/FBt_purity‚0.9837964177131653 *loss/val_total_loss‚46.909293437171556 8 Purity/*FRt3SRqqt*FBt/FRt_purity‚0.23724336922168732 @&Purity/4FRt2SRqqt0FBt/event_proportion‚6.300403038039804e-05 9"Purity/0FRt4SRqqt0FBt/SRqqt_purity‚0.1785714328289032 >&Purity/*FRt2SRqqt0FBt/event_proportion‚0.10855595022439957 =&Purity/1FRt*SRqqt*FBt/event_proportion‚0.4366179406642914 8 Purity/1FRt3SRqqt1FBt/FRt_purity‚0.35555556416511536 @&Purity/2FRt3SRqqt1FBt/event_proportion‚0.0003780241822823882 ?&Purity/3FRt1SRqqt*FBt/event_proportion‚0.003717237850651145 9"Purity/*FRt2SRqqt2FBt/SRqqt_purity‚0.3114035129547119 & Purity/0FRt3SRqqt1FBt/FBt_purity‚1 7 Purity/1FRt0SRqqt2FBt/FBt_purity‚0.9828978180885315 @&Purity/2FRt2SRqqt2FBt/event_proportion‚6.300403038039804e-05 ("Purity/3FRt1SRqqt*FBt/event_purity‚0 7 Purity/*FRt0SRqqt4FBt/FBt_purity‚0.8500000238418579 >&Purity/0FRt0SRqqt0FBt/event_proportion‚0.04082661122083664 ;"Purity/0FRt2SRqqt0FBt/event_purity‚0.012148144654929638 8 Purity/1FRt0SRqqt0FBt/FRt_purity‚0.10697515308856964 ?&Purity/1FRt0SRqqt2FBt/event_proportion‚0.015120967291295528 ?&Purity/2FRt0SRqqt2FBt/event_proportion‚0.003465221729129553 particle/accuracy_6_of_6‚1 ("Purity/1FRt*SRqqt4FBt/event_purity‚0 >&Purity/2FRt*SRqqt*FBt/event_proportion‚0.10244455933570862 :"Purity/3FRt1SRqqt0FBt/SRqqt_purity‚0.34166666865348816 ("Purity/4FRt1SRqqt*FBt/SRqqt_purity‚0 8 Purity/*FRt*SRqqt2FBt/FRt_purity‚0.35563305020332336 >&Purity/*FRt0SRqqt1FBt/event_proportion‚0.14591734111309052 ,&Purity/1FRt4SRqqt2FBt/event_proportion‚0 ,&Purity/1FRt4SRqqt3FBt/event_proportion‚0 9"Purity/2FRt2SRqqt0FBt/SRqqt_purity‚0.1725439876317978 @&Purity/3FRt0SRqqt*FBt/event_proportion‚0.0027091733645647764 @&Purity/3FRt1SRqqt0FBt/event_proportion‚0.0025201612152159214 ,&Purity/3FRt3SRqqt3FBt/event_proportion‚0 @&Purity/*FRt4SRqqt*FBt/event_proportion‚0.0013230846961960196 @&Purity/0FRt2SRqqt2FBt/event_proportion‚0.0010080644860863686 A&Purity/3FRt1SRqqt2FBt/event_proportion‚0.00025201612152159214 ,&Purity/4FRt1SRqqt3FBt/event_proportion‚0 ,&Purity/4FRt4SRqqt0FBt/event_proportion‚0 particle/accuracy_2_of_3‚1 7 Purity/*FRt1SRqqt3FBt/FBt_purity‚0.8730158805847168 9"Purity/*FRt1SRqqt*FBt/SRqqt_purity‚0.2686524987220764 >&Purity/0FRt0SRqqt1FBt/event_proportion‚0.08606351166963577 :"Purity/1FRt0SRqqt*FBt/event_purity‚0.13411523401737213 @&Purity/4FRt0SRqqt0FBt/event_proportion‚6.300403038039804e-05 ("Purity/0FRt3SRqqt0FBt/event_purity‚0 & Purity/2FRt3SRqqt*FBt/FBt_purity‚1 ;"Purity/*FRt2SRqqt1FBt/event_purity‚0.013043108396232128 8 Purity/1FRt*SRqqt1FBt/FRt_purity‚0.28762194514274597 ,&Purity/2FRt1SRqqt3FBt/event_proportion‚0 jet/accuracy_1_of_7‚1 7 Purity/1FRt4SRqqt*FBt/FRt_purity‚0.1666666716337204 ("Purity/*FRt3SRqqt2FBt/event_purity‚0 ("Purity/1FRt1SRqqt3FBt/event_purity‚0 ,&Purity/4FRt3SRqqt2FBt/event_proportion‚0 9"Purity/1FRt2SRqqt2FBt/SRqqt_purity‚0.2569444477558136 *jet/accuracy_2_of_5‚0.3566853404045105 ("Purity/0FRt2SRqqt2FBt/event_purity‚0 :"Purity/2FRt3SRqqt1FBt/SRqqt_purity‚0.20000000298023224 ("Purity/1FRt*SRqqt3FBt/event_purity‚0 7 Purity/2FRt3SRqqt0FBt/FRt_purity‚0.2575216293334961 ,&Purity/2FRt4SRqqt2FBt/event_proportion‚0 ("Purity/*FRt4SRqqt1FBt/event_purity‚0 7 Purity/3FRt3SRqqt*FBt/FRt_purity‚0.1666666567325592 ,&Purity/0FRt4SRqqt3FBt/event_proportion‚0 8"Purity/1FRt*SRqqt1FBt/SRqqt_purity‚0.403650164604187 ,&Purity/1FRt4SRqqt4FBt/event_proportion‚0 ,&Purity/4FRt*SRqqt4FBt/event_proportion‚0 +"Purity/0FRt*SRqqt4FBt/event_purity‚0.25 & Purity/1FRt1SRqqt4FBt/FRt_purity‚0 & Purity/4FRt0SRqqt0FBt/FRt_purity‚0 :"Purity/*FRt0SRqqt0FBt/event_purity‚0.08908312022686005 :"Purity/2FRt1SRqqt1FBt/SRqqt_purity‚0.34615975618362427 @&Purity/2FRt4SRqqt0FBt/event_proportion‚0.0003150201519019902 & Purity/0FRt4SRqqt1FBt/FBt_purity‚1 7 Purity/3FRt*SRqqt1FBt/FRt_purity‚0.2901515066623688 7 Purity/*FRt*SRqqt3FBt/FRt_purity‚0.1764705926179886 9"Purity/2FRt*SRqqt2FBt/SRqqt_purity‚0.6052631735801697 ?&Purity/2FRt2SRqqt*FBt/event_proportion‚0.019468246027827263 ("Purity/4FRt2SRqqt*FBt/event_purity‚0 >&Purity/*FRt3SRqqt0FBt/event_proportion‚0.01827116869390011 7 Purity/*FRt*SRqqt1FBt/FBt_purity‚0.9988723993301392 7 Purity/1FRt0SRqqt*FBt/FBt_purity‚0.9916885495185852 ,&Purity/4FRt2SRqqt1FBt/event_proportion‚0 :"Purity/0FRt3SRqqt*FBt/SRqqt_purity‚0.23054611682891846 & Purity/3FRt2SRqqt1FBt/FBt_purity‚1 ,&Purity/*FRt2SRqqt4FBt/event_proportion‚0 ,&Purity/*FRt3SRqqt4FBt/event_proportion‚0 >&Purity/0FRt*SRqqt0FBt/event_proportion‚0.21931703388690948 & Purity/1FRt2SRqqt1FBt/FBt_purity‚1 7 Purity/2FRt1SRqqt*FBt/FBt_purity‚0.9964157938957214 @&Purity/3FRt2SRqqt1FBt/event_proportion‚0.0003780241822823882 ,&Purity/3FRt2SRqqt2FBt/event_proportion‚0 ("Purity/2FRt4SRqqt*FBt/event_purity‚0 & Purity/3FRt1SRqqt1FBt/FBt_purity‚1 7 Purity/0FRt0SRqqt2FBt/FBt_purity‚0.9858158230781555 >&Purity/2FRt1SRqqt*FBt/event_proportion‚0.04158266261219978 ("Purity/2FRt2SRqqt*FBt/event_purity‚0 @&Purity/2FRt4SRqqt*FBt/event_proportion‚0.0003150201519019902 ,&Purity/3FRt*SRqqt4FBt/event_proportion‚0 @&Purity/3FRt0SRqqt1FBt/event_proportion‚0.0011340725468471646 particle/accuracy_1_of_4‚1 7 Purity/*FRt1SRqqt1FBt/FBt_purity‚0.9980608224868774 7 Purity/*FRt2SRqqt*FBt/FBt_purity‚0.9953680634498596 8 Purity/*FRt3SRqqt1FBt/FRt_purity‚0.31111109256744385 ("Purity/2FRt*SRqqt0FBt/event_purity‚0 ("Purity/2FRt1SRqqt1FBt/event_purity‚0 7 Purity/*FRt4SRqqt*FBt/FRt_purity‚0.2083333283662796 ,"Purity/3FRt4SRqqt0FBt/SRqqt_purity‚0.125 & Purity/*FRt0SRqqt4FBt/FRt_purity‚0 :"Purity/1FRt*SRqqt1FBt/event_purity‚0.12268873304128647 7 Purity/1FRt3SRqqt*FBt/FRt_purity‚0.2519969344139099 ,&Purity/2FRt3SRqqt2FBt/event_proportion‚0 ,&Purity/3FRt4SRqqt4FBt/event_proportion‚0 *jet/accuracy_1_of_6‚0.8859648704528809 particle/accuracy_6_of_7‚0.5 & Purity/*FRt1SRqqt3FBt/FRt_purity‚0 ("Purity/4FRt2SRqqt0FBt/event_purity‚0 >&Purity/*FRt*SRqqt2FBt/event_proportion‚0.06747731566429138 =&Purity/*FRt0SRqqt*FBt/event_proportion‚0.3558467626571655 9"Purity/0FRt*SRqqt0FBt/event_purity‚0.1790110170841217 8 Purity/2FRt2SRqqt0FBt/FRt_purity‚0.20405493676662445 ("Purity/*FRt4SRqqt1FBt/SRqqt_purity‚0 ,&Purity/*FRt3SRqqt3FBt/event_proportion‚0 ,&Purity/3FRt0SRqqt4FBt/event_proportion‚0 ,&Purity/4FRt2SRqqt3FBt/event_proportion‚0 particle/accuracy_3_of_6‚1 ("Purity/2FRt3SRqqt1FBt/event_purity‚0 ("Purity/1FRt1SRqqt4FBt/event_purity‚0 & Purity/2FRt2SRqqt2FBt/FBt_purity‚1 ("Purity/0FRt2SRqqt3FBt/event_purity‚0 9"Purity/*FRt2SRqqt*FBt/SRqqt_purity‚0.2434745877981186 ,&Purity/*FRt4SRqqt3FBt/event_proportion‚0 ?&Purity/2FRt1SRqqt1FBt/event_proportion‚0.007875503972172737 particle/accuracy_4_of_5‚1 ("Purity/4FRt0SRqqt*FBt/event_purity‚0 :"Purity/*FRt2SRqqt1FBt/SRqqt_purity‚0.37011709809303284 6 Purity/0FRt1SRqqt2FBt/FBt_purity‚0.973981499671936 particle/accuracy_5_of_5‚1 ("Purity/*FRt2SRqqt2FBt/event_purity‚0 7 Purity/*FRt*SRqqt3FBt/FBt_purity‚0.9109196066856384 7 Purity/*FRt1SRqqt2FBt/FRt_purity‚0.2487362176179886 A&Purity/0FRt*SRqqt4FBt/event_proportion‚0.00025201612152159214 ?&Purity/1FRt3SRqqt*FBt/event_proportion‚0.009135584346950054 ("Purity/2FRt*SRqqt2FBt/event_purity‚0 & Purity/2FRt2SRqqt*FBt/FBt_purity‚1 @&Purity/4FRt2SRqqt*FBt/event_proportion‚6.300403038039804e-05 ,&Purity/3FRt3SRqqt1FBt/event_proportion‚0 +jet/accuracy_2_of_6‚0.32456138730049133 _runtime‚1529.391228984 7 Purity/0FRt1SRqqt*FBt/FBt_purity‚0.9888736605644226 8 Purity/1FRt*SRqqt*FBt/FRt_purity‚0.20560263097286224 @&Purity/4FRt0SRqqt*FBt/event_proportion‚6.300403038039804e-05 ("Purity/*FRt4SRqqt0FBt/event_purity‚0 7 Purity/1FRt2SRqqt2FBt/FRt_purity‚0.2222222238779068 ("Purity/*FRt1SRqqt3FBt/event_purity‚0 8"Purity/0FRt1SRqqt3FBt/SRqqt_purity‚0.699999988079071 8"Purity/1FRt0SRqqt1FBt/event_purity‚0.220099538564682 7 Purity/3FRt1SRqqt2FBt/FRt_purity‚0.0833333358168602 7 Purity/0FRt0SRqqt*FBt/FBt_purity‚0.9887112379074097 7 Purity/2FRt1SRqqt0FBt/FRt_purity‚0.1584039032459259 8 Purity/3FRt*SRqqt*FBt/FRt_purity‚0.16922254860401154 :"Purity/0FRt2SRqqt2FBt/SRqqt_purity‚0.32692307233810425 ) Purity/1FRt1SRqqt4FBt/FBt_purity‚0.75 :"Purity/*FRt0SRqqt*FBt/event_purity‚0.34444329142570496 9"Purity/2FRt*SRqqt*FBt/SRqqt_purity‚0.2299972027540207 7 Purity/2FRt1SRqqt2FBt/FBt_purity‚0.9868420958518982 ,&Purity/3FRt4SRqqt1FBt/event_proportion‚0 ,&Purity/4FRt3SRqqt1FBt/event_proportion‚0 particle/accuracy_4_of_4‚1 ("Purity/4FRt*SRqqt*FBt/event_purity‚0 8 Purity/*FRt2SRqqt0FBt/FRt_purity‚0.20199143886566162 + Purity/0FRt*SRqqt4FBt/FBt_purity‚0.8125 ("Purity/2FRt*SRqqt3FBt/event_purity‚0 9"Purity/*FRt3SRqqt2FBt/SRqqt_purity‚0.3333333432674408 @&Purity/*FRt3SRqqt2FBt/event_proportion‚6.300403038039804e-05 particle/accuracy_1_of_1‚1 7 Purity/*FRt2SRqqt2FBt/FBt_purity‚0.9868420958518982 ("Purity/1FRt2SRqqt2FBt/event_purity‚0 7 Purity/3FRt4SRqqt0FBt/FRt_purity‚0.1666666716337204 8 Purity/*FRt2SRqqt*FBt/FRt_purity‚0.21031200885772705 >&Purity/1FRt2SRqqt*FBt/event_proportion‚0.05727066472172737 @&Purity/1FRt3SRqqt1FBt/event_proportion‚0.0011340725468471646 ;"Purity/2FRt*SRqqt1FBt/event_purity‚0.014464925974607468 ("Purity/1FRt1SRqqt3FBt/SRqqt_purity‚1 9"Purity/0FRt*SRqqt1FBt/event_purity‚0.6332274675369263 & Purity/1FRt3SRqqt*FBt/FBt_purity‚1 ,&Purity/2FRt4SRqqt3FBt/event_proportion‚0 ,&Purity/3FRt0SRqqt3FBt/event_proportion‚0 7 Purity/3FRt4SRqqt*FBt/FRt_purity‚0.1666666716337204 & Purity/2FRt*SRqqt3FBt/FBt_purity‚1 9"Purity/0FRt0SRqqt2FBt/event_purity‚0.3085920810699463 ?&Purity/1FRt3SRqqt0FBt/event_proportion‚0.007938507944345474 8 Purity/2FRt1SRqqt*FBt/FRt_purity‚0.19843967258930206 8 Purity/3FRt*SRqqt0FBt/FRt_purity‚0.11486772447824478 7 Purity/*FRt*SRqqt4FBt/FBt_purity‚0.8333333134651184 =&Purity/*FRt*SRqqt*FBt/event_proportion‚0.9591733813285828 ,&Purity/1FRt4SRqqt1FBt/event_proportion‚0 @&Purity/4FRt*SRqqt*FBt/event_proportion‚0.0001890120911411941 & Purity/*FRt4SRqqt*FBt/FBt_purity‚1 7 Purity/0FRt2SRqqt2FBt/FBt_purity‚0.9615384340286255 & Purity/3FRt*SRqqt2FBt/FBt_purity‚1 ("Purity/1FRt*SRqqt3FBt/SRqqt_purity‚1 ("Purity/2FRt2SRqqt2FBt/event_purity‚0 7 Purity/0FRt*SRqqt3FBt/FBt_purity‚0.9012346863746643 :"Purity/0FRt2SRqqt1FBt/event_purity‚0.02700972929596901 8 Purity/1FRt*SRqqt0FBt/FRt_purity‚0.16713960468769073 ,jet/accuracy_4_of_5‚0.002481389557942748 /"Purity/*FRt4SRqqt0FBt/SRqqt_purity‚0.171875 7 Purity/3FRt0SRqqt2FBt/FRt_purity‚0.6666666865348816 7 Purity/*FRt0SRqqt2FBt/FRt_purity‚0.4051198661327362 7 Purity/1FRt*SRqqt2FBt/FBt_purity‚0.9803202152252197 9"Purity/1FRt1SRqqt*FBt/SRqqt_purity‚0.2579297721385956 ?&Purity/2FRt0SRqqt1FBt/event_proportion‚0.008946572430431843 ,&Purity/4FRt*SRqqt1FBt/event_proportion‚0 *jet/accuracy_1_of_4‚0.7875291109085083 & Purity/2FRt0SRqqt1FBt/FBt_purity‚1 -jet/accuracy_4_of_4‚0.0010080644860863686 7 Purity/1FRt1SRqqt3FBt/FBt_purity‚0.8333333730697632 ("Purity/3FRt*SRqqt*FBt/event_purity‚0 particle/accuracy_2_of_4‚1 *particle/sensitivity‚0.219760000705719 8 Purity/2FRt3SRqqt1FBt/FRt_purity‚0.10000000149011612 & Purity/2FRt0SRqqt3FBt/FRt_purity‚0 ;"Purity/0FRt1SRqqt2FBt/event_purity‚0.008095238357782364 ?&Purity/0FRt3SRqqt*FBt/event_proportion‚0.006741431541740894 7 Purity/1FRt*SRqqt*FBt/FBt_purity‚0.9915969371795654 =&Purity/1FRt*SRqqt0FBt/event_proportion‚0.3135710656642914 ?&Purity/1FRt*SRqqt2FBt/event_proportion‚0.022303426638245583 :"Purity/0FRt*SRqqt0FBt/SRqqt_purity‚0.22572781145572662 8 Purity/2FRt2SRqqt*FBt/FRt_purity‚0.21763528883457184 :"Purity/2FRt2SRqqt1FBt/SRqqt_purity‚0.37886905670166016 ,&Purity/3FRt*SRqqt3FBt/event_proportion‚0 ("Purity/3FRt4SRqqt*FBt/event_purity‚0 ?&Purity/0FRt1SRqqt2FBt/event_proportion‚0.008757560513913631 ,&Purity/0FRt4SRqqt2FBt/event_proportion‚0 ?&Purity/2FRt2SRqqt0FBt/event_proportion‚0.015435988083481789 ("Purity/1FRt*SRqqt4FBt/SRqqt_purity‚1 9"Purity/3FRt3SRqqt*FBt/SRqqt_purity‚0.2142857313156128 :"Purity/0FRt1SRqqt1FBt/event_purity‚0.14303721487522125 ?&Purity/0FRt3SRqqt0FBt/event_proportion‚0.006111391354352236 :"Purity/2FRt*SRqqt0FBt/SRqqt_purity‚0.19026660919189453 ;"Purity/2FRt0SRqqt1FBt/event_purity‚0.028888888657093048 8 Purity/2FRt0SRqqt2FBt/FRt_purity‚0.37738093733787537 & Purity/1FRt3SRqqt2FBt/FBt_purity‚1 ?&Purity/0FRt0SRqqt2FBt/event_proportion‚0.029674898833036423 7 Purity/1FRt3SRqqt0FBt/FRt_purity‚0.2798386812210083 9"Purity/3FRt1SRqqt1FBt/SRqqt_purity‚0.4615384638309479 9"Purity/3FRt2SRqqt*FBt/SRqqt_purity‚0.2934027910232544 9"Purity/1FRt4SRqqt*FBt/SRqqt_purity‚0.1666666716337204 ("Purity/4FRt2SRqqt0FBt/SRqqt_purity‚0’šÙñ>‚ ‚(:’ ÌŸäÔøǨ gpu.0.gpu‚0.0 gpu.0.memory‚0, gpu.0.memoryAllocated‚12.165603637695312* gpu.0.memoryAllocatedBytes‚ 10450173952 gpu.0.temp‚33.0 gpu.0.powerWatts‚118.638' gpu.0.enforcedPowerLimitWatts‚500.0) gpu.0.powerPercent‚23.727600000000002 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0!BNƒ ‚(:à ̟äÔô”Í« disk.sda2.out‚0$ memory_percent‚84.24801826477051( proc.memory.availableMB‚ 2580.8046875$ disk./.usageGB‚251.1307830810547 disk.dm-0.out‚6019.119140625 disk.sda.out‚ 60.0703125 network.sent‚ 3.487868e+06 network.recv‚594997 disk.drbd2265.out‚0 disk.rbd1.in‚0 disk.drbd1716.in‚0 disk.drbd2080.in‚0 disk.rbd1.out‚ 2.146484375 disk.sda3.out‚ 60.0703125 disk.nvme0n1.in‚ 11.80859375 disk.drbd2266.out‚0 cpu‚26.24224939309741! disk.nvme0c0n1.in‚ 11.80859375 disk.dm-0.in‚ 11.80859375 disk.sda3.in‚0 disk.dm-1.in‚0 disk.rbd0.out‚0 disk.sda2.in‚0 disk.drbd1716.out‚0 disk.sda1.out‚0 disk.drbd2266.in‚0# proc.memory.rssMB‚ 2783.11328125 proc.cpu.threads‚17 disk.sda.in‚0% disk.nvme0c0n1.out‚6018.994140625 disk.rbd0.in‚0* proc.memory.percent‚16.986775398254395 disk.drbd2265.in‚0 disk.sda1.in‚0# disk.nvme0n1.out‚6018.994140625* disk./.usagePercent‚16.853597565331484 disk.dm-1.out‚ 60.0703125 disk.drbd2080.out‚0“Yæ“„ ‚(:ˆ ÛŸäÔöÇš© gpu.0.gpu‚44.0 gpu.0.memory‚5, gpu.0.memoryAllocated‚12.165603637695312* gpu.0.memoryAllocatedBytes‚ 10450173952 gpu.0.temp‚34.0 gpu.0.powerWatts‚119.896' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚23.9792 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0`K†W… ‚(:Ì ÛŸäÔŽåÙ« disk.sda.in‚0 disk.sda3.in‚0 disk.dm-1.out‚ 60.55859375 disk.drbd2080.in‚0 proc.cpu.threads‚17 network.recv‚602023* disk./.usagePercent‚16.865619423131875 disk.drbd1716.out‚0" disk.dm-0.out‚6199.30224609375 disk.rbd0.in‚0 disk.nvme0n1.in‚ 11.80859375 disk.sda1.in‚0 disk.sda1.out‚0' proc.memory.availableMB‚ 2536.203125 disk.sda.out‚ 60.55859375 disk.drbd1716.in‚0 disk.rbd1.out‚ 2.146484375% disk./.usageGB‚251.30991744995117 disk.sda2.out‚0 disk.drbd2266.out‚0' disk.nvme0c0n1.out‚6199.17724609375 disk.dm-0.in‚ 11.80859375 disk.sda3.out‚ 60.55859375% disk.nvme0n1.out‚6199.17724609375 disk.rbd1.in‚0$ memory_percent‚84.52024459838867" proc.memory.rssMB‚ 2783.1171875* proc.memory.percent‚16.986799240112305 network.sent‚ 3.591479e+06 disk.rbd0.out‚0 disk.sda2.in‚0 disk.drbd2265.out‚0! disk.nvme0c0n1.in‚ 11.80859375 cpu‚26.235628061459696 disk.dm-1.in‚0 disk.drbd2080.out‚0 disk.drbd2266.in‚0 disk.drbd2265.in‚0W»ãÉ‘† ‚(:† êŸäÔ»Ûˆ¨ gpu.0.gpu‚48.0 gpu.0.memory‚4, gpu.0.memoryAllocated‚12.165603637695312* gpu.0.memoryAllocatedBytes‚ 10450173952 gpu.0.temp‚35.0 gpu.0.powerWatts‚146.23' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚29.246 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0µ`°Q‡ ‚(:Æ êŸäÔ„™«$ disk.nvme0n1.out‚6381.8994140625 disk.drbd2266.in‚0 disk.drbd1716.out‚0 proc.cpu.threads‚17* disk./.usagePercent‚16.869284170984525 disk.sda2.in‚0 disk.sda2.out‚0 disk.rbd1.in‚0 disk.sda.in‚0 disk.sda3.out‚ 61.1328125* proc.memory.percent‚16.986799240112305 disk.rbd0.out‚0 disk.rbd1.out‚ 2.294921875 disk.sda3.in‚0 network.recv‚606167 disk.nvme0n1.in‚ 11.80859375$ memory_percent‚85.27474403381348( proc.memory.availableMB‚ 2412.5859375" proc.memory.rssMB‚ 2783.1171875 disk.rbd0.in‚0 disk.sda1.in‚0 disk.sda1.out‚0 disk.drbd2265.in‚0& disk.nvme0c0n1.out‚6381.8994140625 disk.drbd2080.in‚0 disk.drbd2080.out‚0 disk.drbd2266.out‚0 disk.dm-1.in‚0 disk.drbd1716.in‚0 disk.drbd2265.out‚0! disk.nvme0c0n1.in‚ 11.80859375 disk.dm-0.in‚ 11.80859375! disk.dm-0.out‚6382.0244140625 disk.sda.out‚ 61.1328125 disk.dm-1.out‚ 61.1328125 network.sent‚ 3.597721e+06$ disk./.usageGB‚251.3645248413086 cpu‚26.230735214010476 Â2ˆ - _wandbruntime‚1561  _runtime‚1561Ì%BšÇ‰ Á trainer/global_step‚1849 % _timestamp‚1.7884159802216299e+09 _runtime‚1561.871029417 _step‚147 & lr-AdamW/pg1‚0.0016422599069297516 & lr-AdamW/pg2‚0.0016422599069297516=ßßeÌŠ Æ % _timestamp‚1.7884159802216299e+09 _runtime‚1561.871029417 _step‚147 & lr-AdamW/pg1‚0.0016422599069297516 & lr-AdamW/pg2‚0.0016422599069297516 trainer/global_step‚1849“ùWˆ2‹ - _wandbruntime‚1561  _runtime‚1561Ⱦ>‹Õ'Œ ÏO UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 435026432 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 D>DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚24128 MFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚35 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚9281121 ZFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚108550757073408 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚213 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚16179872 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 O@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 9373790793 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚431 L?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚34993131 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚462 QBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 9902751744 WEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 1387372476928 ZKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 9456058368 TFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 446693376 C;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚644 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 446693376 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 169268224 IDeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚24128 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚20038 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 446693376 G?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚213 UADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚107365751139328 `LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚105978395832832 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚17170432 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚2605 aMDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚107104262786048 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 XJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 165393020 YEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚104398252273202 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 trainer/global_step‚1849 N?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 9902751744 @8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚644 IDeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚2605 \HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚105978378662400 JBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚213 PCDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated‚35013169 SDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 9283702784 O@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 9679899648 NFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚985 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚465 `LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚103018106701824 SDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 9902751744 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 186438656 SGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚9280659 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 VBDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated‚107365937577984 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 248172032 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 _step‚148 VGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 9283702784 ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 216607744 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 N@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 186438656 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚20040 XIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 9456058368 UFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 9456058368 R>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚107365751139328;¸Æ ÔO SDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 9283702784 O@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 9679899648 NFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚985 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚465 `LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚103018106701824 SDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 9902751744 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 186438656 SGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚9280659 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 VBDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated‚107365937577984 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 248172032 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 _step‚148 VGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 9283702784 ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 216607744 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 N@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 186438656 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚20040 XIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 9456058368 UFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 9456058368 R>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚107365751139328 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 435026432 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 D>DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚24128 MFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚35 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚9281121 ZFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚108550757073408 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚213 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚16179872 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 O@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 9373790793 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚431 L?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚34993131 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚462 QBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 9902751744 WEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 1387372476928 ZKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 9456058368 TFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 446693376 C;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚644 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 446693376 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 169268224 IDeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚24128 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚20038 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 446693376 G?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚213 UADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚107365751139328 `LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚105978395832832 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚17170432 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚2605 aMDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚107104262786048 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 XJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 165393020 YEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚104398252273202 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 trainer/global_step‚1849 N?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 9902751744 @8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚644 IDeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚2605 \HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚105978378662400 JBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚213 PCDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated‚35013169”Ò–a2Ž - _wandbruntime‚1562  _runtime‚1562lJ†ß& ÙM TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 239269888 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚213 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 169281536 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 T@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚107422026634752 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚455 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 BDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 9373790793 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚431 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 182432588 VHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 165393228 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 446693376 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚644 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚20064 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚644 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚17039360 QBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 9902751744 WCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚107422026634752 OADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 186451968 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24772608 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 ZFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚103071954329600 TEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 8980994048 XFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 1380701625066 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚24401 J;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 9679899648 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚24128 K>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚25473751 ]KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 1447255934976 O@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 9902751744 XDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚108607453160448 SDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 9456058368 EDeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 WCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚106033742351872 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚9285491 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 144203776 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 WCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚104452838387254 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚644 G:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚61939693 NADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚35011603 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚213 PBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 435026432 NADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚26928090 LDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚985 [MDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚ 214497280 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 M>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 9679899648 \JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 1380867018294 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1003 cODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚107160221998080 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚24401 \HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚108607692430336 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚644 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚431 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚16188263 G:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚61959759 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚17170432 K>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚61959759 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 NADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚35031667 J=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚35011603 QBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 9283702784 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚452 VGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 9456058368 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 446693376 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚213 WIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 446693376 ZFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚106033742351872 XFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 1388097830912 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚9285943 S?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚104452655954666 XIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 9456058368 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 C6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚61939693 TEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 9283702784 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚35 QCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 313315328 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 _step‚149 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚26928090 FDeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 186451968 YGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 1388267112448 ^JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚106033759522304 REDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚16188260 PDeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 WCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚106033742351872 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚9285491 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 144203776 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 WCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚104452838387254 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚644 G:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚61939693 NADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚35011603 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚213 PBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 435026432 NADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚26928090 LDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚985 [MDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚ 214497280 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 M>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 9679899648 \JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 1380867018294 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1003 cODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚107160221998080 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚24401 \HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚108607692430336 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚644 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚431 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚16188263 G:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚61959759 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚17170432 K>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚61959759 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 NADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚35031667 J=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚35011603 QBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 9283702784 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚452 VGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 9456058368 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 446693376 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚213 WIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 446693376 ZFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚106033742351872 XFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 1388097830912 UIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚9285943 S?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚104452655954666 XIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 9456058368 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 C6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚61939693 TEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 9283702784 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚35 QCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 313315328 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 _step‚149 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚26928090 FDeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 186451968 YGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 1388267112448 ^JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚106033759522304 REDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚16188260 PDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 9373790793 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚431 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 182432588 VHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 165393228 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 446693376 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚644 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚20064 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚644 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚17039360 QBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 9902751744 WCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚107422026634752 OADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 186451968 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24772608 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 ZFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚103071954329600 TEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 8980994048 XFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 1380701625066 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚24401 J;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 9679899648 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚24128 K>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚25473751 ]KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 1447255934976 O@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 9902751744 XDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚108607453160448 SDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 9456058368 EDeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚2605 L=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 9679899648 ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 220802048 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 O@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 9373790793 JBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚213 \HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚108746563136000 ZHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 1423668004352 cQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 1484780914176 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 446693376 @8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚644 JBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚431 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 % _timestamp‚1.7884160192714274e+09 IDeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚24128 R>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚110170231140352 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 245574656 QBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 9902751744 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚20040 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚2 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 435026432 TGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚27614092 QCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 182432380 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 VGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 9283702784 D>DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 G?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚213 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚20040 H?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚1003 `LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚105711340058624 BDeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚24128 R>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚110170231140352 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 245574656 QBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 9902751744 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚20040 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚2 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 435026432 TGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚27614092 QCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 182432380 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 VGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 9283702784 D>DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 G?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚213 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚20040 H?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚1003 `LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚105711340058624 BDeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚2605 L=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 9679899648 ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 220802048 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 O@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 9373790793 JBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚213 \HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚108746563136000 ZHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 1423668004352 cQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 1484780914176 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 446693376 @8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚644 JBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚431 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 % _timestamp‚1.7884160192714274e+09 I8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 VHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 165393228 trainer/global_step‚1899 REDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚35958619 L=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 9902751744 _runtime‚1601.604398493 PBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 435026432 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1003 QCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 313315328 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚2 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚20064 K>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚26129523 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 ZFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚105765187686400 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚644 ^JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚108801943995904 REDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚27628092 G:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚63566645 O@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 9902751744 PDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 C6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚63566645 XFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 1416801132474 SDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 9456058368 [GDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚108801943995904 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17170432 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚9521713 \HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚111442478899200 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚27628090 QBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 9283702784 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚17170432 K>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚63586711 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚24401 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24772608 aODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 1485534929920 B6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚644 M>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 9373790793 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚644 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚213 J;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 9679899648 WCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚110226506635776 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚24128 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚17039360 VGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 9456058368 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚431 % _timestamp‚1.7884160199545038e+09 NADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚27628090 ]KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 1485316238336 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 169281536 \JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 1424562639872 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 G:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚63586711 REDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚16607810 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚644 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚2605 S?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚107181988818874 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 446693376 EDeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 186451968 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 446693376 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚431 QBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 9902751744 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 T@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚110226506635776 M>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 9679899648 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 TEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 8980994048 FDeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚26129523 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 ZFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚105765187686400 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚644 ^JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚108801943995904 REDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚27628092 G:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚63566645 O@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 9902751744 PDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 C6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚63566645 XFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 1416801132474 SDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 9456058368 [GDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚108801943995904 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17170432 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚9521713 \HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚111442478899200 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 F@DeviceSs›ÖG¶tatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚27628090 QBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 9283702784 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚17170432 K>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚63586711 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚24401 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24772608 aODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 1485534929920 B6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚644 M>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 9373790793 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚644 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚213 J;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 9679899648 WCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚110226506635776 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚24128 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚17039360 VGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 9456058368 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚431 % _timestamp‚1.7884160199545038e+09 NADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚27628090 ]KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 1485316238336 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 169281536 \JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 1424562639872 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 G:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚63586711 REDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚16607810 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚644 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚2605 S?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚107181988818874 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 446693376 EDeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 186451968 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 446693376 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚431 QBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 9902751744 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 T@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚110226506635776 M>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 9679899648 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 TEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 8980994048 F8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 VHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 165393228 trainer/global_step‚1899 REDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚35958619 L=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 9902751744 _runtime‚1601.604398493™bmì2¡ - _wandbruntime‚1601  _runtime‚1601¿äH]¢ × 0loss/FRt3/assignment_loss‚3.2711404622355595 0loss/FBt1/assignment_loss‚0.1528475899037651 .loss/FRt3/detection_loss‚2.510141097292115 1loss/SRqqt2/detection_loss‚2.7542457884433222 /loss/FRt1/detection_loss‚2.2874521722722996 2loss/SRqqt4/assignment_loss‚1.3521664405622296 /loss/FRt4/detection_loss‚1.7600612509615567 -loss/FBt2/detection_loss‚2.88569607266358 &loss/total_loss‚45.127206407614324 _runtime‚1601.605528085 _step‚154 /loss/FBt4/detection_loss‚0.7344807175050179 /loss/FRt2/detection_loss‚2.4607876274320812 /loss/FRt2/assignment_loss‚4.014932285120458 0loss/FRt4/assignment_loss‚1.9988064480071166 2loss/SRqqt1/assignment_loss‚1.5345730108650106 1loss/FBt2/assignment_loss‚0.28442226679596516 1loss/FBt3/assignment_loss‚0.21615113839148903 1loss/SRqqt3/detection_loss‚1.8081172019243241 epoch‚3 0loss/SRqqt2/assignment_loss‚2.50026388612448 2loss/FBt4/assignment_loss‚0.015360816399512771 1loss/SRqqt1/detection_loss‚1.9859580118405191 /loss/FBt3/detection_loss‚1.8846877286067376 /loss/FRt1/assignment_loss‚3.850654154702237 1loss/SRqqt4/detection_loss‚1.0306874637621162 trainer/global_step‚1899 % _timestamp‚1.7884160199563327e+09 2loss/SRqqt3/assignment_loss‚1.8315182104706764 /loss/FBt1/detection_loss‚2.0020545653321524rÎ!b£ Ü /loss/FBt4/detection_loss‚0.7344807175050179 /loss/FRt2/detection_loss‚2.4607876274320812 /loss/FRt2/assignment_loss‚4.014932285120458 0loss/FRt4/assignment_loss‚1.9988064480071166 2loss/SRqqt1/assignment_loss‚1.5345730108650106 1loss/FBt2/assignment_loss‚0.28442226679596516 1loss/FBt3/assignment_loss‚0.21615113839148903 1loss/SRqqt3/detection_loss‚1.8081172019243241 epoch‚3 0loss/SRqqt2/assignment_loss‚2.50026388612448 2loss/FBt4/assignment_loss‚0.015360816399512771 1loss/SRqqt1/detection_loss‚1.9859580118405191 /loss/FBt3/detection_loss‚1.8846877286067376 /loss/FRt1/assignment_loss‚3.850654154702237 1loss/SRqqt4/detection_loss‚1.0306874637621162 trainer/global_step‚1899 % _timestamp‚1.7884160199563327e+09 2loss/SRqqt3/assignment_loss‚1.8315182104706764 /loss/FBt1/detection_loss‚2.0020545653321524 0loss/FRt3/assignment_loss‚3.2711404622355595 0loss/FBt1/assignment_loss‚0.1528475899037651 .loss/FRt3/detection_loss‚2.510141097292115 1loss/SRqqt2/detection_loss‚2.7542457884433222 /loss/FRt1/detection_loss‚2.2874521722722996 2loss/SRqqt4/assignment_loss‚1.3521664405622296 /loss/FRt4/detection_loss‚1.7600612509615567 -loss/FBt2/detection_loss‚2.88569607266358 &loss/total_loss‚45.127206407614324 _runtime‚1601.605528085 _step‚154šÅÚÌž¤ ‚(:“ — äÔÕ£”¨ gpu.0.gpu‚43.0 gpu.0.memory‚4, gpu.0.memoryAllocated‚12.165603637695312* gpu.0.memoryAllocatedBytes‚ 10450173952 gpu.0.temp‚36.0 gpu.0.powerWatts‚137.452' gpu.0.enforcedPowerLimitWatts‚500.0) gpu.0.powerPercent‚27.490399999999998 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0+½HWU¥ ‚(:Ê — äÔƒ¸žª disk.sda1.out‚0 disk.sda.out‚ 62.8671875) proc.memory.availableMB‚ 2412.24609375% disk./.usageGB‚251.39878845214844 disk.sda2.in‚0 disk.drbd2265.in‚0" disk.dm-0.out‚6548.31298828125 disk.sda1.in‚0! disk.nvme0c0n1.in‚ 11.80859375 disk.dm-1.out‚ 62.8671875 network.sent‚ 3.696334e+06 disk.rbd0.out‚0 disk.drbd2265.out‚0 disk.drbd2080.in‚0" proc.memory.rssMB‚ 2783.1171875 disk.sda3.out‚ 62.8671875) disk./.usagePercent‚16.87158363065707 disk.sda2.out‚0 disk.drbd2266.out‚0 disk.dm-0.in‚ 11.80859375 disk.sda3.in‚0% disk.nvme0n1.out‚6548.18798828125 disk.drbd2266.in‚0$ memory_percent‚85.27681827545166* proc.memory.percent‚16.986799240112305 cpu‚26.21640974149913 disk.rbd1.out‚ 2.3271484375 disk.sda.in‚0 disk.drbd1716.out‚0 proc.cpu.threads‚17 disk.rbd0.in‚0 disk.nvme0n1.in‚ 11.80859375 disk.rbd1.in‚0' disk.nvme0c0n1.out‚6548.18798828125 disk.dm-1.in‚0 disk.drbd1716.in‚0 disk.drbd2080.out‚0 network.recv‚620547 ÈÂꓦ ‚(:ˆ ¦ äÔÄ›¦¨ gpu.0.gpu‚51.0 gpu.0.memory‚6, gpu.0.memoryAllocated‚12.165603637695312* gpu.0.memoryAllocatedBytes‚ 10450173952 gpu.0.temp‚36.0 gpu.0.powerWatts‚172.241' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚34.4482 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0 m<îX§ ‚(:Í ¦ äÔʤ« disk.sda1.out‚0 disk.dm-0.in‚ 11.80859375 disk.sda.in‚0 disk.sda.out‚ 63.51171875 disk.drbd1716.out‚0 disk.drbd2080.in‚0 network.recv‚626786 disk.nvme0n1.in‚ 11.80859375) proc.memory.availableMB‚ 2412.44140625" proc.memory.rssMB‚ 2783.1171875* proc.memory.percent‚16.986799240112305% disk.nvme0n1.out‚6611.08349609375 disk.sda3.in‚0 disk.drbd2266.in‚0 disk.drbd2266.out‚0' disk.nvme0c0n1.out‚6611.08349609375" disk.dm-0.out‚6611.20849609375 disk.drbd2080.out‚0$ disk./.usageGB‚251.4273338317871 disk.drbd2265.in‚0! disk.nvme0c0n1.in‚ 11.80859375 disk.dm-1.out‚ 63.51171875 disk.rbd0.out‚0 cpu‚26.211512867527027 disk.sda3.out‚ 63.51171875 disk.drbd1716.in‚0 network.sent‚ 3.78408e+06 disk.sda2.in‚0 disk.sda2.out‚0 disk.drbd2265.out‚0 disk.rbd1.in‚0 disk.rbd1.out‚ 2.3271484375 disk.dm-1.in‚0 disk.rbd0.in‚0$ memory_percent‚85.27562618255615 proc.cpu.threads‚17* disk./.usagePercent‚16.873499335035792 disk.sda1.in‚0…ºW¨ ‚(:Ì µ äÔñ›¡ª% disk./.usageGB‚251.43731689453125 disk.rbd1.in‚0' disk.nvme0c0n1.out‚6666.73583984375! disk.nvme0c0n1.in‚ 11.80859375 disk.sda3.in‚0 disk.sda3.out‚ 63.94140625$ memory_percent‚85.27412414550781% proc.memory.availableMB‚ 2412.6875 proc.cpu.threads‚17 network.sent‚ 3.789467e+06 disk.rbd1.out‚ 2.39013671875 disk.drbd2080.out‚0 disk.sda.out‚ 63.94140625 disk.dm-1.in‚0 disk.dm-1.out‚ 63.94140625 disk.drbd1716.in‚0 disk.rbd0.in‚0% disk.nvme0n1.out‚6666.73583984375 disk.drbd2266.in‚0 disk.sda1.in‚0 network.recv‚630617 disk.drbd2265.in‚0 disk.drbd1716.out‚0 disk.rbd0.out‚0 disk.sda2.in‚0* proc.memory.percent‚16.986799240112305 disk.drbd2080.in‚0 disk.drbd2266.out‚0" disk.dm-0.out‚6666.86083984375 disk.drbd2265.out‚0 disk.nvme0n1.in‚ 11.80859375" proc.memory.rssMB‚ 2783.1171875 cpu‚26.207070880501213 disk.sda1.out‚0* disk./.usagePercent‚16.874169306752897 disk.dm-0.in‚ 11.80859375 disk.sda.in‚0 disk.sda2.out‚0ùâ㥓© ‚(:ˆ µ äÔ–É€Ë gpu.0.gpu‚44.0 gpu.0.memory‚4, gpu.0.memoryAllocated‚12.165603637695312* gpu.0.memoryAllocatedBytes‚ 10450173952 gpu.0.temp‚36.0 gpu.0.powerWatts‚166.573' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚33.3146 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0¨ø¡µ2ª - _wandbruntime‚1639  _runtime‚1639|õÙÆ« À & lr-AdamW/pg1‚0.0016054046863565481 & lr-AdamW/pg2‚0.0016054046863565481 trainer/global_step‚1949 $ _timestamp‚1.788416057471382e+09 _runtime‚1639.121792336 _step‚155üDˬ Å & lr-AdamW/pg1‚0.0016054046863565481 & lr-AdamW/pg2‚0.0016054046863565481 trainer/global_step‚1949 $ _timestamp‚1.788416057471382e+09 _runtime‚1639.121792336 _step‚155›S9˯2­ - _wandbruntime‚1639  _runtime‚1639¬-u[Ô'® ÎO OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚3 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚17018969 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 `LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚108404573415424 C;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚644 PCDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated‚36867073 `LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚111514764780032 BDeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 UADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚112974711141376 \HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚111514747609600 ZKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 9456058368 E8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚65161125 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚17170432 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 \HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚108404556376064 N?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 9902751744 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚20038 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 169268224 UADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚109856735569238 QBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 9902751744 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 446693376 $ _timestamp‚1.788416057472111e+09 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚2 PCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚28314090 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚9755566 O@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 9679899648 TGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚28314092 H?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚1003 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 MFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚35 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚213 G?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚213 WEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 1459963531776 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚20040 SDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 9283702784 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚2605 XIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 9456058368 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚17170432 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 186438656 VBDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated‚112974897580032 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚20040 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 243477504 XJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 165393020 R>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚112974711141376 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 QCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 182432380 _step‚156 ]IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚111514764780032 SGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚9755085 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 VGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 8980994048 @8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚644 JBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚213 QDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚26774538 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚481 SEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 313315328 G?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚431 [IDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 1460132800000 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 M@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚65181165 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 VGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 9283702784 aMDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚112697706728448 UFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 9456058368 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 446693376 YEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚112974897580032 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 TGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚36867073 N@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 186438656 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚24128 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 L=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 9679899648 ^JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚114220574830592 eQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚112697731501056 ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 218704896 _MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 1522624624640 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚431 L?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚36847035 PCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚36847035 UGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 431144273 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 JBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚431 PCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚28314092 L?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚28314090 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚484 cQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 1522843329536 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 E=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚644 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚20038 O@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 9373790793 _runtime‚1639.121915889 M@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚26774054 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚24401 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚24401 NFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚985 @8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚644 ZHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 1452179193174 G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚2605 ZHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 1459963531776 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 435026432 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 435026432 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 169268224 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚24772608 YEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚109856918001618X°çíX¯ ÓO L=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 9679899648 ^JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚114220574830592 eQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚112697731501056 ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 218704896 _MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 1522624624640 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚431 L?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚36847035 PCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚36847035 UGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 431144273 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 JBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚431 PCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚28314092 L?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚28314090 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚484 cQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 1522843329536 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 E=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚644 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚20038 O@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 9373790793 _runtime‚1639.121915889 M@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚26774054 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚24401 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚24401 NFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚985 @8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚644 ZHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 1452179193174 G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚2605 ZHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 1459963531776 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 435026432 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 435026432 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 169268224 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚24772608 YEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚109856918001618 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚3 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚17018969 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 `LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚108404573415424 C;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚644 PCDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated‚36867073 `LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚111514764780032 BDeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 UADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚112974711141376 \HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚111514747609600 ZKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 9456058368 E8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚65161125 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚17170432 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 \HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚108404556376064 N?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 9902751744 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚20038 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 169268224 UADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚109856735569238 QBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 9902751744 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 446693376 $ _timestamp‚1.788416057472111e+09 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚2 PCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚28314090 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚9755566 O@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 9679899648 TGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚28314092 H?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚1003 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 MFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚35 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚213 G?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚213 WEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 1459963531776 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚20040 SDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 9283702784 KBDeviceStatsMonitor.on_train_batch_start/Áëö allocation.large_pool.peak‚2605 XIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 9456058368 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚17170432 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 186438656 VBDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated‚112974897580032 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚20040 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 243477504 XJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 165393020 R>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚112974711141376 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 QCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 182432380 _step‚156 ]IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚111514764780032 SGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚9755085 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 VGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 8980994048 @8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚644 JBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚213 QDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚26774538 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚481 SEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 313315328 G?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚431 [IDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 1460132800000 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 M@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚65181165 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 VGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 9283702784 aMDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚112697706728448 UFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 9456058368 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 446693376 YEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚112974897580032 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 TGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚36867073 N@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 186438656 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚24128 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0œ¬8I2° - _wandbruntime‚1639  _runtime‚1639ÔrìÝ&± ×M >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚644 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 169281536 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚17027363 cODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚112753665940480 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚20064 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 ^JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚111570128469504 \HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚114277264778752 ^JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚108458438082560 M>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 9679899648 TEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 9283702784 O@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 9902751744 trainer/global_step‚1949 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚24401 YGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 1460858167296 FDeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚65213663 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24772608 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 PDeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚26787321 XDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚114277023411712 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 248172032 M>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 9373790793 G:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚65213663 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚17170432 SEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 435026432 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚24128 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1003 REDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚17027360 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 144203776 _runtime‚1639.797680026 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 T@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚113030986636800 L>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 186451968 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 \JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 1453066033110 WCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚111570111299072 \JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 1460858167296 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚644 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚20064 WCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚113030986636800 XFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 1452900639882 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚3 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 # _timestamp‚1.78841605814788e+09 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 XIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 9456058368 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 446693376 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚431 [MDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚ 216594432 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 169281536 FDeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚213 _step‚157 _KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚112753641167872 WCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚109911504115670 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚17039360 ZFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚108458421043200 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 446693376 EDeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚213 J=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚36865507 UCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 1460688885760 XFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 1460688885760 LDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚985 ]KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 1523382243840 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 431144273 SDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 9456058368 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17170432 ZFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚111570111299072 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚481 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚9759961 S?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚109911321683082 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚644 C6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚65193597 [GDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚111570128469504 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚24401 VHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 165393228 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 PBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 435026432 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 241367040 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 B6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚644 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 169281536 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚17027363 cODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚112753665940480 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚20064 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 ^JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚111570128469504 \HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚114277264778752 ^JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚108458438082560 M>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 9679899648 TEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 9283702784 O@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 9902751744 trainer/global_step‚1949 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚24401 YGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 1460858167296 FDeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚65213663 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24772608 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 PDeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚26787321 XDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚114277023411712 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 248172032 M>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 9373790793 G:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚65213663 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚17170432 SEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 435026432 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚24128 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1003 REDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚17027360 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 144203776 _runtime‚1639.797680026 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 T@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚113030986636800 L>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 186451968 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 \JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 1453066033110 WCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚111570111299072 \JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 1460858167296 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚644 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚20064 WCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚113030986636800 XFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 1452900639882 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚3 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 # _timestamp‚1.78841605814788e+09 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 XIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 9456058368 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 446693376 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚431 [MDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚ 216594432 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 169281536 FDeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚213 _step‚157 _KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚112753641167872 WCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚109911504115670 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚17039360 ZFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚108458421043200 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 446693376 EDeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚213 J=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚36865507 UCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 1460688885760 XFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 1460688885760 LDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚985 ]KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 1523382243840 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 431144273 SDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 9456058368 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17170432 ZFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚111570111299072 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚481 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚9759961 S?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚109911321683082 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚644 C6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚65193597 [GDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚111570128469504 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚24401 VHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 165393228 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 PBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 435026432 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 241367040 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 BDeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚2605 @8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚644 SDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 9283702784 M@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚66808117 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 % _timestamp‚1.7884160958388352e+09 TGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚37794025 _MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 1560567717376 `LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚111097806772224 G?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚431 PCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚29014092 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚24401 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 446693376 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚2605 @8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚644 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚431 _runtime‚1677.488497733 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚24128 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚17170432 SEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 313315328 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 ZHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 1488278700582 ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 216607744 E8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚66788077 [IDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 1496428327424 \HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚114282932083200 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚9995556 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 169268224 XJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 165393020 QBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 9902751744 MFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚35 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 E=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚644 IDeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚115779191142400 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚24128 ZFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚117054996417024 D>DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 YEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚115779377581056 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 BDeviceStatsMonitor.on_train_batch_start/allocation.all.current‚20040 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 trainer/global_step‚1999 H?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚1003 YEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚114282932083200 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 VGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 8980994048aúÀ¼Ú'¿ ÔO IDeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚115779191142400 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚24128 ZFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚117054996417024 D>DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 YEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚115779377581056 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 BDeviceStatsMonitor.on_train_batch_start/allocation.all.current‚20040 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 trainer/global_step‚1999 H?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚1003 YEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚114282932083200 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 VGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 8980994048 L?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚37773987 O@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 9679899648 IDeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚2605 @8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚644 SDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 9283702784 M@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚66808117 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 % _timestamp‚1.7884160958388352e+09 TGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚37794025 _MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 1560567717376 `LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚111097806772224 G?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚431 PCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚29014092 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚24401 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 446693376 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚2605 @8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚644 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚431 _runtime‚1677.488497733 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚24128 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚17170432 SEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 313315328 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 ZHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 1488278700582 ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 216607744 E8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚66788077 [IDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 1496428327424 \HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚114282932083200 WKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚9995556 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 169268224 XJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 165393020 QBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 9902751744 MFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚35 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 E=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚644 2ÂP2À - _wandbruntime‚1678  _runtime‚1678€èwPà&Á ÚM NADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚29028090 K>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚27446810 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 ]KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 1561334788096 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 446693376 G:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚66840615 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 169281536 G:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚66820549 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚10000344 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 ^JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚111151671439360 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚213 QBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 9283702784 WCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚115835466637824 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚24401 TEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 9283702784 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚35 _KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚115550363139072 VGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 9456058368 YGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 1497153694720 ^JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚114338312943104 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚17446913 [MDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚ 206108672 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚431 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚213 XFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 1489000147290 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 431144273 QBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 9902751744 % _timestamp‚1.7884160965234911e+09 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 NADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚37812523 J=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚37792459 ZFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚114338295772672 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚9999900 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚644 S?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚112640654547290 O@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 9902751744 WIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 446693376 PDeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 EDeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚66840615 XDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚117111697927168 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 VHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 165393228 OADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 186451968 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24772608 M>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 9373790793 C6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚66820549 NADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚29028092 M>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 9679899648 cODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚115550387911680 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 [GDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚114338312943104 \JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 1489165540518 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚17039360 XIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 9456058368 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚2 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚17170432 REDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚37812523 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 248172032 T@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚115835466637824 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1003 REDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚17446910 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚644 _runtime‚1678.173132789 _step‚161 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 TEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 8980994048 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚644 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 OBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚27447257 J;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 9679899648 UCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 1496984413184 SEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 435026432 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 144203776 FDeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 186451968 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17170432 \HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚117111928808448 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚213 WCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚114338295772672 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚3 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚431 trainer/global_step‚1999 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚20064 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 446693376 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚444 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚29028090 PBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 435026432 S?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚115835280185856 \JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 1497153694720 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚24128 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 SDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 9456058368 NADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚37792459 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 L=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 9902751744 XFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 1496984413184 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚24401 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 ZFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚111151654400000ï–,êx ßM NADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚37812523 J=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚37792459 ZFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚114338295772672 QEDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚9999900 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚644 S?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚112640654547290 O@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 9902751744 WIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 446693376 PDeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 EDeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚66840615 XDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚117111697927168 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 VHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 165393228 OADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 186451968 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24772608 M>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 9373790793 C6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚66820549 NADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚29028092 M>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 9679899648 cODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚115550387911680 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 [GDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚114338312943104 \JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 1489165540518 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚17039360 XIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 9456058368 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚2 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚17170432 REDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚37812523 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 248172032 T@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚115835466637824 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1003 REDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚17446910 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚644 _runtime‚1678.173132789 _step‚161 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 TEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 8980994048 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚644 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 OBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚27447257 J;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 9679899648 UCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 1496984413184 SEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 435026432 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 144203776 FDeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 186451968 REDeviceStatsMonitor.on_train_batch_end/active€ª=m_bytes.large_pool.current‚17170432 \HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚117111928808448 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚213 WCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚114338295772672 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚3 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚431 trainer/global_step‚1999 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚20064 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 446693376 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚444 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚29028090 PBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 435026432 S?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚115835280185856 \JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 1497153694720 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚24128 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 SDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 9456058368 NADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚37792459 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 L=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 9902751744 XFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 1496984413184 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚24401 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 ZFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚111151654400000 NADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚29028090 K>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚27446810 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 ]KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 1561334788096 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 446693376 G:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚66840615 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 169281536 G:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚66820549 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚10000344 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 ^JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚111151671439360 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚213 QBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 9283702784 WCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚115835466637824 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚24401 TEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 9283702784 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚35 _KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚115550363139072 VGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 9456058368 YGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 1497153694720 ^JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚114338312943104 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚17446913 [MDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚ 206108672 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚431 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚213 XFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 1489000147290 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 431144273 QBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 9902751744 % _timestamp‚1.7884160965234911e+09 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0¡Fµ‰2à - _wandbruntime‚1678  _runtime‚1678MËÿ¦YÄ Ó 0loss/FRt3/assignment_loss‚3.6665644347667694 1loss/FBt4/assignment_loss‚0.06039758472172762 .loss/FRt4/detection_loss‚1.781926872186801 .loss/FBt2/detection_loss‚2.302249638657821 /loss/FBt4/detection_loss‚0.6475026940626483 &loss/total_loss‚44.594862407786046 1loss/FBt3/assignment_loss‚0.23248010594155252 1loss/FBt1/assignment_loss‚0.15289981050279625 /loss/FRt2/detection_loss‚2.5567981964581974 epoch‚3 2loss/SRqqt4/assignment_loss‚1.4907946136819434 1loss/SRqqt3/detection_loss‚2.0036317436134112 .loss/FBt1/detection_loss‚2.105530190680708 .loss/FBt3/detection_loss‚2.684042908747991 -loss/FRt2/assignment_loss‚3.9322427805368 1loss/SRqqt1/detection_loss‚1.7746864448894153 trainer/global_step‚1999 _runtime‚1678.174246009 0loss/FRt4/assignment_loss‚1.8163735845509696 1loss/SRqqt1/assignment_loss‚1.877975565712314 2loss/SRqqt2/assignment_loss‚2.0894726499533043 1loss/FBt2/assignment_loss‚0.10973650197408795 /loss/FRt1/detection_loss‚2.0886399702134506 /loss/FRt3/detection_loss‚2.5720408856868744 0loss/SRqqt2/detection_loss‚2.346269284303372 0loss/SRqqt4/detection_loss‚0.973423984373959 /loss/FRt1/assignment_loss‚3.594522769252459 2loss/SRqqt3/assignment_loss‚1.7346591923166723 % _timestamp‚1.7884160965250576e+09 _step‚162uöÍð^Å Ø 1loss/FBt1/assignment_loss‚0.15289981050279625 /loss/FRt2/detection_loss‚2.5567981964581974 epoch‚3 2loss/SRqqt4/assignment_loss‚1.4907946136819434 1loss/SRqqt3/detection_loss‚2.0036317436134112 .loss/FBt1/detection_loss‚2.105530190680708 .loss/FBt3/detection_loss‚2.684042908747991 -loss/FRt2/assignment_loss‚3.9322427805368 1loss/SRqqt1/detection_loss‚1.7746864448894153 trainer/global_step‚1999 _runtime‚1678.174246009 0loss/FRt4/assignment_loss‚1.8163735845509696 1loss/SRqqt1/assignment_loss‚1.877975565712314 2loss/SRqqt2/assignment_loss‚2.0894726499533043 1loss/FBt2/assignment_loss‚0.10973650197408795 /loss/FRt1/detection_loss‚2.0886399702134506 /loss/FRt3/detection_loss‚2.5720408856868744 0loss/SRqqt2/detection_loss‚2.346269284303372 0loss/SRqqt4/detection_loss‚0.973423984373959 /loss/FRt1/assignment_loss‚3.594522769252459 2loss/SRqqt3/assignment_loss‚1.7346591923166723 % _timestamp‚1.7884160965250576e+09 _step‚162 0loss/FRt3/assignment_loss‚3.6665644347667694 1loss/FBt4/assignment_loss‚0.06039758472172762 .loss/FRt4/detection_loss‚1.781926872186801 .loss/FBt2/detection_loss‚2.302249638657821 /loss/FBt4/detection_loss‚0.6475026940626483 &loss/total_loss‚44.594862407786046 1loss/FBt3/assignment_loss‚0.23248010594155252¢JpáÆ ‚(:… â äÔ𞨠gpu.0.gpu‚0.0 gpu.0.memory‚0, gpu.0.memoryAllocated‚12.165603637695312* gpu.0.memoryAllocatedBytes‚ 10450173952 gpu.0.temp‚35.0 gpu.0.powerWatts‚119.48' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚23.896 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0cUÇ ‚(:Ê â äÔÿ¦ª disk.rbd0.in‚0 disk.rbd0.out‚0 disk.nvme0n1.in‚ 11.80859375 disk.drbd2266.out‚0" disk.dm-0.out‚6857.34521484375 disk.drbd1716.out‚0% disk.nvme0n1.out‚6857.22021484375 disk.drbd2266.in‚0 disk.rbd1.in‚0" proc.memory.rssMB‚ 2783.1171875 disk.dm-1.out‚ 65.76953125 disk.sda3.out‚ 65.76953125! disk.nvme0c0n1.in‚ 11.80859375 disk.drbd2080.in‚0 disk.drbd2080.out‚0 disk.sda1.in‚0 disk.drbd2265.in‚0 disk.drbd2265.out‚0 cpu‚26.194265604142156 disk.dm-1.in‚0 network.sent‚ 3.888678e+06 network.recv‚645309' disk.nvme0c0n1.out‚6857.22021484375) proc.memory.availableMB‚ 2412.33203125 proc.cpu.threads‚17 disk.drbd1716.in‚0 disk.sda1.out‚0 disk.sda2.in‚0$ memory_percent‚85.27629375457764* proc.memory.percent‚16.986799240112305 disk.rbd1.out‚2.453125) disk./.usagePercent‚16.87656016111517% disk./.usageGB‚251.47294235229492 disk.sda.in‚0 disk.sda.out‚ 65.76953125 disk.sda3.in‚0 disk.sda2.out‚0 disk.dm-0.in‚ 11.80859375GÁåUžÈ ‚(:“ ñ äÔóö›¨ gpu.0.gpu‚42.0 gpu.0.memory‚4, gpu.0.memoryAllocated‚12.165603637695312* gpu.0.memoryAllocatedBytes‚ 10450173952 gpu.0.temp‚36.0 gpu.0.powerWatts‚170.842' gpu.0.enforcedPowerLimitWatts‚500.0) gpu.0.powerPercent‚34.168400000000005 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0‘ ôPÉ ‚(:Å ñ äÔôæ‘«$ memory_percent‚85.27629375457764) proc.memory.availableMB‚ 2412.33203125* disk./.usagePercent‚16.873902290894684$ disk./.usageGB‚251.4333381652832' disk.nvme0c0n1.out‚6909.81396484375" disk.dm-0.out‚6909.93896484375 disk.sda1.in‚0" proc.memory.rssMB‚ 2783.1171875 disk.dm-1.in‚0 disk.nvme0n1.in‚ 11.80859375% disk.nvme0n1.out‚6909.81396484375 disk.sda2.out‚0 disk.rbd1.in‚0 disk.dm-0.in‚ 11.80859375 disk.rbd0.in‚0 disk.sda1.out‚0 disk.drbd2266.in‚0! disk.nvme0c0n1.in‚ 11.80859375 disk.sda3.out‚ 66.5859375 disk.drbd1716.out‚0 disk.drbd2265.in‚0 proc.cpu.threads‚17 disk.sda.in‚0 network.sent‚ 3.97533e+06 network.recv‚650485 disk.drbd1716.in‚0 disk.drbd2265.out‚0 disk.sda.out‚ 66.5859375 disk.drbd2266.out‚0* proc.memory.percent‚16.986799240112305 disk.sda3.in‚0 disk.sda2.in‚0 cpu‚26.189926342997396 disk.rbd1.out‚2.84375 disk.dm-1.out‚ 66.5859375 disk.drbd2080.in‚0 disk.drbd2080.out‚0 disk.rbd0.out‚0,pÔ[’Ê ‚(:‡ €¡äÔÖº·¨ gpu.0.gpu‚44.0 gpu.0.memory‚4, gpu.0.memoryAllocated‚12.165603637695312* gpu.0.memoryAllocatedBytes‚ 10450173952 gpu.0.temp‚36.0 gpu.0.powerWatts‚166.325' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚33.265 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0"U_«PË ‚(:Å €¡äÔœÒÒª! disk.nvme0c0n1.in‚ 11.80859375 disk.sda2.out‚0 disk.drbd2266.in‚0 disk.drbd2265.out‚0 cpu‚26.185847881720182 proc.cpu.threads‚17 network.sent‚ 3.98078e+06 disk.drbd2080.out‚0 disk.rbd1.in‚0 disk.sda3.out‚ 67.31640625 disk.dm-1.out‚ 67.31640625* disk./.usagePercent‚16.876109843862437 disk.sda.out‚ 67.31640625 disk.drbd1716.in‚0 disk.drbd2080.in‚0 disk.rbd0.in‚0 disk.drbd2265.in‚0' proc.memory.availableMB‚ 2412.078125 disk.dm-1.in‚0 disk.nvme0n1.in‚ 11.80859375 disk.sda1.in‚0 disk.sda1.out‚0 disk.sda2.in‚0# memory_percent‚85.2778434753418' disk.nvme0c0n1.out‚6976.99755859375 disk.dm-0.in‚ 11.80859375" disk.dm-0.out‚6977.12255859375 disk.rbd0.out‚0% disk.nvme0n1.out‚6976.99755859375 disk.drbd2266.out‚0 network.recv‚653815 disk.sda.in‚0$ disk./.usageGB‚251.4662322998047 disk.rbd1.out‚2.84375 disk.sda3.in‚0 disk.drbd1716.out‚0" proc.memory.rssMB‚ 2783.1171875* proc.memory.percent‚16.986799240112305Ïiß2Ì - _wandbruntime‚1715  _runtime‚1715¡ˆ_NÆÍ À trainer/global_step‚2049 % _timestamp‚1.7884161341484702e+09 _runtime‚ 1715.79784385 _step‚163 & lr-AdamW/pg1‚0.0015663186319984108 & lr-AdamW/pg2‚0.0015663186319984108ÚÒ\ËÎ Å & lr-AdamW/pg2‚0.0015663186319984108 trainer/global_step‚2049 % _timestamp‚1.7884161341484702e+09 _runtime‚ 1715.79784385 _step‚163 & lr-AdamW/pg1‚0.0015663186319984108£p¸ð•2Ï - _wandbruntime‚1715  _runtime‚17154š™×'Ð ÑO YEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚117051116556800 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 446693376 % _timestamp‚1.7884161341494434e+09 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚24128 N?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 9902751744 E=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚644 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 ^LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 1524543601010 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 C;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚644 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 IDeviceStatsMonitor.on_train_batch_start/allocation.all.current‚20040 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 \HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚113791023089664 VGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 8980994048 TFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 446693376 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 186438656 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚474 @8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚644 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 N@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 186438656 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 169268224 PCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚29714090 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚471 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚10233757 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 ^JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚119889923109888 eQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚118291175443456 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚431 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚213 L=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 9679899648 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚17170432 D>DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 ]IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚117051133727232 QDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚28092300 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 144203776 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 248172032 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 ZKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 9456058368 WEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 1532554586624 YEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚118583857582080 ^LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 1532723854848 ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 220802048 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 E8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚68415029 R>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚118583671143424 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 435026432 PCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚38700939 G?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚213 trainer/global_step‚2049 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚17170432 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚24401 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚20038 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚24128 NFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚985 VBDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated‚118583857582080 ZHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 1532554586624 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚17858072 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 `LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚113791040129024 O@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 9373790793 SDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 9283702784 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚3 QBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 9902751744 UADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚118583671143424 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚24772608 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚24401 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 169268224 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 435026432 cQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 1598747666432 G?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚431 O@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 9679899648 aMDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚118291150670848 BDeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚2605 MFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚35 XJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 165393020 M@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚68435069 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚17858069 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚10234228 UGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 431144273 JBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚431 \HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚117051116556800 IDeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚2605 MFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚35 XJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 165393020 M@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚68435069 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚17858069 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚10234228 UGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 431144273 JBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚431 \HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚117051116556800 IDeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚24128 N?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 9902751744 E=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚644 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 ^LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 1524543601010 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 C;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚644 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 IDeviceStatsMonitor.on_train_batch_start/allocation.all.current‚20040 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 \HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚113791023089664 VGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 8980994048 TFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 446693376 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 186438656 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚474 @8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚644 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 N@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 186438656 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 169268224 PCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚29714090 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚471 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚10233757 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 ^JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚119889923109888 eQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚118291175443456 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚431 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚213 L=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 9679899648 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚17170432 D>DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 ]IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚117051133727232 QDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚28092300 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 144203776 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 248172032 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 ZKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 9456058368 WEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 1532554586624 YEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚118583857582080 ^LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 1532723854848 ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 220802048 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 E8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚68415029 R>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚118583671143424 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 435026432 PCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚38700939 G?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚213 trainer/global_step‚2049 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚17170432 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚24401 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚20038 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚24128 NFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚985 VBDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated‚118583857582080 ZHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 1532554586624 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚17858072 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 `LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚113791040129024 O@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 9373790793 SDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 9283702784 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚3 QBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 9902751744 UADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚118583671143424 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚24772608 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚24401 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 169268224 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 435026432 cQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 1598747666432 G?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚431 O@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 9679899648 aMDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚118291150670848 BDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 9679899648 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚24128 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚483 cODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚118347109882880 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 J=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚38719411 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 % _timestamp‚1.7884161348363392e+09 REDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚17866460 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 144203776 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚24401 NADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚38739475 S?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚118639760186880 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚644 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17170432 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 169281536 SEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 435026432 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 VGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 9456058368 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 446693376 LDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚985 M>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 9373790793 T@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚118639946638848 L=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 9902751744 QBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 9283702784 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24772608 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 \JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 1525265047926 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 169281536 XFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 1533279940608 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚35 WCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚115370169844086 ZFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚113844887756800 O@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 9902751744 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚29728090 K>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚68467567 G:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚68447501 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚20066 WCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚117106480246272 YGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 1533449222144 OADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 186451968 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚20064 K>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚28104746 REDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚10238286 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 FDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚644 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚431 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚17866463 [MDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚ 210302976 TEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 8980994048 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚2605 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 182432588 XIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 9456058368 SDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 9456058368 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚10238769 XDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚119946378430976 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚431 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚213 EDeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚213 \JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 1533449222144 PDeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚20064 L>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 186451968 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1003 B6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚644 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 \HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚119946613506560 _KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚118347085110272 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚3 ]KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 1599293320704 S?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚115369987411498 ^JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚113844904796160 QBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 9902751744 _runtime‚1716.486117913 [GDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚117106497416704 REDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚38739475 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚644 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 TEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 9283702784 QCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 313315328 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0:‰T„æ&Ô àM I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚2605 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 182432588 XIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 9456058368 SDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 9456058368 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚10238769 XDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚119946378430976 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚431 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚213 EDeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚213 \JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 1533449222144 PDeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚20064 L>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 186451968 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1003 B6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚644 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 \HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚119946613506560 _KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚118347085110272 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚3 ]KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 1599293320704 S?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚115369987411498 ^JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚113844904796160 QBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 9902751744 _runtime‚1716.486117913 [GDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚117106497416704 REDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚38739475 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚644 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 TEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 9283702784 QCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 313315328 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚24401 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚486 WIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 446693376 NADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚29728092 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚17039360 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚2 M>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 9679899648 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚24128 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚483 cODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚118347109882880 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 J=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚38719411 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 % _timestamp‚1.7884161348363392e+09 REDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚17866460 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 144203776 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚24401 NADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚38739475 S?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚118639760186880 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚644 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17170432 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 169281536 SEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 435026432 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 VGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 9456058368 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 446693376 LDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚985 M>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 9373790793 T@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚118639946638848 L=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 9902751744 QBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 9283702784 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24772608 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 \JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 1525265047926 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 169281536 XFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 1533279940608 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚35 WCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚115370169844086 ZFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚113844887756800 O@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 9902751744 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚29728090 K>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚68467567 G:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚68447501 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚20066 WCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚117106480246272 YGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 1533449222144 OADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 186451968 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚20064 K>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚28104746 REDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚10238286 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 FDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚644 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚431 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚17866463 [MDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚ 210302976 TEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 8980994048¥MÉ_2Õ - _wandbruntime‚1716  _runtime‚1716ÚJXÖ Ò 0loss/FRt3/assignment_loss‚3.8999368876982956 2loss/SRqqt3/assignment_loss‚2.0478100809626856 2loss/FBt4/assignment_loss‚0.050989654301969695 # _timestamp‚1.78841613483832e+09 1loss/SRqqt2/assignment_loss‚2.142839598442827 2loss/SRqqt4/assignment_loss‚1.2178976569823392 1loss/SRqqt1/detection_loss‚1.7973030382587063 .loss/FBt2/detection_loss‚2.363786378327538 .loss/FRt3/detection_loss‚2.664988787892537 /loss/FRt4/detection_loss‚1.7405404115636502 0loss/SRqqt2/detection_loss‚2.462172165938786 1loss/SRqqt4/detection_loss‚1.1489573992260798 /loss/FBt4/detection_loss‚0.7192715827841312 _runtime‚1716.487497517 0loss/FRt4/assignment_loss‚1.8812644603431865 2loss/SRqqt1/assignment_loss‚1.7202841462867875 .loss/FRt1/detection_loss‚2.136055813791851 /loss/FRt2/detection_loss‚2.5456971811921627 epoch‚3 /loss/FRt1/assignment_loss‚3.573202525575956 /loss/FRt2/assignment_loss‚4.208684981685796 1loss/SRqqt3/detection_loss‚1.9173108367965772 /loss/FBt1/detection_loss‚1.9962678448013638 /loss/FBt3/detection_loss‚1.7733206936606654 %loss/total_loss‚44.74321304095548 0loss/FBt1/assignment_loss‚0.2454417278362519 0loss/FBt2/assignment_loss‚0.2079470951205568 0loss/FBt3/assignment_loss‚0.2812420914847785 trainer/global_step‚2049 _step‚166Ýï,]× × /loss/FBt4/detection_loss‚0.7192715827841312 _runtime‚1716.487497517 0loss/FRt4/assignment_loss‚1.8812644603431865 2loss/SRqqt1/assignment_loss‚1.7202841462867875 .loss/FRt1/detection_loss‚2.136055813791851 /loss/FRt2/detection_loss‚2.5456971811921627 epoch‚3 /loss/FRt1/assignment_loss‚3.573202525575956 /loss/FRt2/assignment_loss‚4.208684981685796 1loss/SRqqt3/detection_loss‚1.9173108367965772 /loss/FBt1/detection_loss‚1.9962678448013638 /loss/FBt3/detection_loss‚1.7733206936606654 %loss/total_loss‚44.74321304095548 0loss/FBt1/assignment_loss‚0.2454417278362519 0loss/FBt2/assignment_loss‚0.2079470951205568 0loss/FBt3/assignment_loss‚0.2812420914847785 trainer/global_step‚2049 _step‚166 0loss/FRt3/assignment_loss‚3.8999368876982956 2loss/SRqqt3/assignment_loss‚2.0478100809626856 2loss/FBt4/assignment_loss‚0.050989654301969695 # _timestamp‚1.78841613483832e+09 1loss/SRqqt2/assignment_loss‚2.142839598442827 2loss/SRqqt4/assignment_loss‚1.2178976569823392 1loss/SRqqt1/detection_loss‚1.7973030382587063 .loss/FBt2/detection_loss‚2.363786378327538 .loss/FRt3/detection_loss‚2.664988787892537 /loss/FRt4/detection_loss‚1.7405404115636502 0loss/SRqqt2/detection_loss‚2.462172165938786 1loss/SRqqt4/detection_loss‚1.1489573992260798¦ª8 ’Ø ‚(:‡ ¡äÔ ®¨ gpu.0.gpu‚0.0 gpu.0.memory‚0, gpu.0.memoryAllocated‚12.165603637695312* gpu.0.memoryAllocatedBytes‚ 10450173952 gpu.0.temp‚35.0 gpu.0.powerWatts‚119.619' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚23.9238 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0nüå8QÙ ‚(:Æ ¡äÔýÿª disk.rbd1.in‚0! disk.nvme0c0n1.in‚ 11.80859375' disk.nvme0c0n1.out‚7037.62841796875 disk.sda.out‚ 67.74609375 disk.drbd2080.out‚0 disk.sda1.in‚0 disk.sda2.in‚0 disk.drbd2266.out‚0 network.sent‚ 3.987256e+06$ disk./.usageGB‚251.4798355102539 disk.sda.in‚0 disk.rbd0.out‚0 disk.nvme0n1.in‚ 11.80859375" proc.memory.rssMB‚ 2783.1171875 disk.drbd1716.out‚0 disk.drbd2080.in‚0 disk.sda1.out‚0 disk.dm-0.in‚ 11.80859375 disk.sda2.out‚0 network.recv‚658536 disk.drbd2265.in‚0 disk.drbd2265.out‚0 proc.cpu.threads‚17" disk.dm-0.out‚7037.75341796875 disk.dm-1.out‚ 67.74609375 disk.drbd1716.in‚0' proc.memory.availableMB‚ 2412.296875* proc.memory.percent‚16.986799240112305 cpu‚26.182098844015623) disk./.usagePercent‚16.87702276672954 disk.rbd1.out‚2.90625 disk.sda3.in‚0 disk.sda3.out‚ 67.74609375 disk.dm-1.in‚0 disk.drbd2266.in‚0$ memory_percent‚85.27650833129883 disk.rbd0.in‚0% disk.nvme0n1.out‚7037.62841796875Ob¬“Ú ‚(:ˆ ž¡äÔ¹û³¨ gpu.0.gpu‚42.0 gpu.0.memory‚4, gpu.0.memoryAllocated‚12.165603637695312* gpu.0.memoryAllocatedBytes‚ 10450173952 gpu.0.temp‚36.0 gpu.0.powerWatts‚170.643' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚34.1286 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0ÿ$‚QÛ ‚(:Æ ž¡äÔﵪ( proc.memory.availableMB‚ 2412.2890625 disk.rbd0.out‚0$ disk.nvme0n1.out‚7093.2392578125 disk.sda2.in‚0 disk.rbd1.out‚2.90625$ memory_percent‚85.27655601501465 disk.sda1.in‚0" proc.memory.rssMB‚ 2783.1171875* disk./.usagePercent‚16.877925193287737% disk./.usageGB‚251.49328231811523 disk.drbd1716.in‚0 disk.nvme0n1.in‚ 11.80859375 disk.sda2.out‚0! disk.dm-0.out‚7093.3642578125 disk.dm-1.in‚0 disk.dm-1.out‚ 68.45703125 network.sent‚ 4.074559e+06 disk.drbd1716.out‚0 disk.drbd2080.in‚0 disk.drbd2266.out‚0 disk.drbd2265.out‚0 disk.sda.out‚ 68.45703125 network.recv‚664541 disk.drbd2266.in‚0 disk.drbd2265.in‚0& disk.nvme0c0n1.out‚7093.2314453125 disk.sda3.out‚ 68.45703125 disk.rbd0.in‚0 disk.sda1.out‚0 disk.dm-0.in‚ 11.80859375* proc.memory.percent‚16.986799240112305 proc.cpu.threads‚17 disk.sda.in‚0 disk.rbd1.in‚0! disk.nvme0c0n1.in‚ 11.80859375 cpu‚26.177875874156854 disk.drbd2080.out‚0 disk.sda3.in‚0&ê\2Ü - _wandbruntime‚1753  _runtime‚1753'Õù·ÇÝ Á trainer/global_step‚2099 % _timestamp‚1.7884161721107497e+09 _runtime‚1753.760117308 _step‚167 & lr-AdamW/pg1‚0.0015459795471208296 & lr-AdamW/pg2‚0.0015459795471208296”@ŽÌÞ Æ % _timestamp‚1.7884161721107497e+09 _runtime‚1753.760117308 _step‚167 & lr-AdamW/pg1‚0.0015459795471208296 & lr-AdamW/pg2‚0.0015459795471208296 trainer/global_step‚2099§Rš2ß - _wandbruntime‚1753  _runtime‚17534|€Ïà ÑO D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚20040 PCDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated‚39647929 [IDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 1569019382272 cQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 1636703395840 ^LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 1560643108418 UGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 431144273 G?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚431 VBDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated‚121388337583104 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 YEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚118044916594242 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 446693376 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 446693376 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚2605 H?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚1003 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚477 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 PCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚30414092 `LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚119819318200832 VGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 9283702784 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚2605 JBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚213 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚24401 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 435026432 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 QCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 182432380 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚213 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 L=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 9679899648 YEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚119819301030400 WEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 1568850114048 YEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚121388337583104 IDeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚24128 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚17170432 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚24401 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 245574656 aMDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚121087872642048 ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 220802048 BDeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 JBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚431 ]IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚119819318200832 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚10471565 QBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 9902751744 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚20040 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚18277622 NFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚985 ^JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚122724600810496 SDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 9283702784 O@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 9679899648 _MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 1636482593792 @8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚644 R>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚121388151144448 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 trainer/global_step‚2099 N@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 186438656 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 169268224 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 435026432 `LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚116484273485824 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚20038 PCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚39627891 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚480 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 % _timestamp‚1.7884161721117833e+09 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚431 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚2 M@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚70062021 TGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚30414092 ZFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚122724355235840 SDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 9902751744 ZKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 9456058368 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 VGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 8980994048 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 186438656 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 248172032 TFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 446693376 _runtime‚1753.761394989 E8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚70041981 ^LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 1569019382272 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 C;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚644 _step‚168 IDeviceStatsMonitor.on_train_batch_start/allocation.all.current‚20040 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚18277622 NFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚985 ^JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚122724600810496 SDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 9283702784 O@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 9679899648 _MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 1636482593792 @8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚644 R>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚121388151144448 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 trainer/global_step‚2099 N@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 186438656 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 169268224 UGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.peak‚ 435026432 `LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚116484273485824 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚20038 PCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚39627891 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚480 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 % _timestamp‚1.7884161721117833e+09 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚431 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚2 M@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚70062021 TGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚30414092 ZFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚122724355235840 SDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 9902751744 ZKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 9456058368 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 VGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 8980994048 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 186438656 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 248172032 TFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 446693376 _runtime‚1753.761394989 E8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚70041981 ^LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 1569019382272 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 C;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚644 _step‚168 IDeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚2605 H?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚1003 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚477 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 PCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚30414092 `LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚119819318200832 VGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 9283702784 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚2605 JBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚213 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚24401 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 435026432 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 QCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 182432380 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚213 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 L=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 9679899648 YEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚119819301030400 WEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 1568850114048 YEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚121388337583104 IDeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚24128 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚17170432 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚24401 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 245574656 aMDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚121087872642048 ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 220802048 BDeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 JBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚431¨&8Ge2â - _wandbruntime‚1754  _runtime‚1754ª7$á&ã ÛM ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚24401 SEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 435026432 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 248172032 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 ^JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚119874681890304 TEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 9283702784 REDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚10475882 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 251852800 aODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 1637462978048 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 G:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚70074453 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚17039360 T@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚121444426639872 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚467 WIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 446693376 % _timestamp‚1.7884161727821786e+09 WCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚121444426639872 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚644 L=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 9902751744 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚20066 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚2 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17170432 S?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚121444240187904 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 169281536 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚35 ]KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 1637235897856 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚20064 M>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 9679899648 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚24128 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚18286013 [MDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚ 227080192 S?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚118099320275706 trainer/global_step‚2099 L>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 186451968 YGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 1569744749568 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 REDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚30428092 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚20064 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚644 FDeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚28761892 cODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚121143831854080 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚431 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 NADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚39666427 OADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 186451968 OBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚28762359 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚2605 \HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚122781294832128 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 144203776 TEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 8980994048 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 446693376 _runtime‚1754.431924656 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 M>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 9373790793 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 431144273 O@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 9902751744 _step‚169 _KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚121143807081472 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 NADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚39646363 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1003 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 J;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 9679899648 QBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 9283702784 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 169281536 K>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚70094519 FDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 9373790793 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 431144273 O@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 9902751744 _step‚169 _KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚121143807081472 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 NADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚39646363 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1003 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 J;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 9679899648 QBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 9283702784 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 169281536 K>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚70094519 F6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚644 L=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 9902751744 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚20066 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚2 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17170432 S?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚121444240187904 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 169281536 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚35 ]KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 1637235897856 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚20064 M>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 9679899648 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚24128 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚18286013 [MDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚ 227080192 S?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚118099320275706 trainer/global_step‚2099 L>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 186451968 YGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 1569744749568 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 REDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚30428092 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚20064 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚644 FDeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚28761892 cODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚121143831854080 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚431 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 NADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚39666427 OADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 186451968 OBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚28762359 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0©Ýù-2å - _wandbruntime‚1754  _runtime‚1754梎MXæ Ò 2loss/SRqqt1/assignment_loss‚1.9958070289248075 _step‚170 0loss/FRt4/assignment_loss‚1.8239445463575499 1loss/FBt4/assignment_loss‚0.02129738753529043 trainer/global_step‚2099 _runtime‚ 1754.43295982 1loss/FBt2/assignment_loss‚0.13895915714203425 .loss/FRt1/detection_loss‚2.193012422016549 .loss/FRt2/detection_loss‚2.469284358309276 1loss/SRqqt1/detection_loss‚1.8293876766394346 1loss/SRqqt4/detection_loss‚1.0136223123073578 /loss/FBt2/detection_loss‚2.1957879889579046 /loss/FRt4/detection_loss‚1.8193649440401731 /loss/FBt3/detection_loss‚2.1604712307453156 /loss/FBt4/detection_loss‚0.7399423209869344 &loss/total_loss‚44.611177727603646 epoch‚3 /loss/FRt1/assignment_loss‚3.635212956052838 /loss/FRt3/assignment_loss‚3.522805611292521 2loss/SRqqt4/assignment_loss‚1.0339699117019772 0loss/FBt3/assignment_loss‚0.3276336603975367 0loss/SRqqt2/detection_loss‚2.458680081016877 2loss/SRqqt3/assignment_loss‚1.7625411172810288 /loss/FRt2/assignment_loss‚3.956420231221327 1loss/FBt1/assignment_loss‚0.36105651623117463 /loss/FRt3/detection_loss‚2.6713191557813576 1loss/SRqqt3/detection_loss‚1.9037179203377557 -loss/FBt1/detection_loss‚2.47317297669018 0loss/SRqqt2/assignment_loss‚2.10376621563645 % _timestamp‚1.7884161727837489e+09>öÂÉ]ç × trainer/global_step‚2099 _runtime‚ 1754.43295982 1loss/FBt2/assignment_loss‚0.13895915714203425 .loss/FRt1/detection_loss‚2.193012422016549 .loss/FRt2/detection_loss‚2.469284358309276 1loss/SRqqt1/detection_loss‚1.8293876766394346 1loss/SRqqt4/detection_loss‚1.0136223123073578 /loss/FBt2/detection_loss‚2.1957879889579046 /loss/FRt4/detection_loss‚1.8193649440401731 /loss/FBt3/detection_loss‚2.1604712307453156 /loss/FBt4/detection_loss‚0.7399423209869344 &loss/total_loss‚44.611177727603646 epoch‚3 /loss/FRt1/assignment_loss‚3.635212956052838 /loss/FRt3/assignment_loss‚3.522805611292521 2loss/SRqqt4/assignment_loss‚1.0339699117019772 0loss/FBt3/assignment_loss‚0.3276336603975367 0loss/SRqqt2/detection_loss‚2.458680081016877 2loss/SRqqt3/assignment_loss‚1.7625411172810288 /loss/FRt2/assignment_loss‚3.956420231221327 1loss/FBt1/assignment_loss‚0.36105651623117463 /loss/FRt3/detection_loss‚2.6713191557813576 1loss/SRqqt3/detection_loss‚1.9037179203377557 -loss/FBt1/detection_loss‚2.47317297669018 0loss/SRqqt2/assignment_loss‚2.10376621563645 % _timestamp‚1.7884161727837489e+09 2loss/SRqqt1/assignment_loss‚1.9958070289248075 _step‚170 0loss/FRt4/assignment_loss‚1.8239445463575499 1loss/FBt4/assignment_loss‚0.02129738753529043ªH¾ÅBè ‚(:’ ­¡äÔè©´¨ gpu.0.gpu‚40.0 gpu.0.memory‚4, gpu.0.memoryAllocated‚12.165603637695312* gpu.0.memoryAllocatedBytes‚ 10450173952 gpu.0.temp‚36.0 gpu.0.powerWatts‚164.87' gpu.0.enforcedPowerLimitWatts‚500.0) gpu.0.powerPercent‚32.974000000000004 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0œný˜Vé ‚(:Ë ­¡äÔ «¨« disk.drbd2265.in‚0 disk.dm-0.in‚ 11.80859375 disk.sda.out‚ 68.85546875 cpu‚26.173959933591966 network.sent‚ 4.07993e+06 disk.rbd0.out‚0$ disk.nvme0n1.out‚7151.3603515625 disk.sda3.in‚0 disk.sda3.out‚ 68.85546875 disk.dm-1.out‚ 68.85546875* disk./.usagePercent‚16.879501175668523 disk.drbd2080.out‚0 disk.drbd2265.out‚0 disk.rbd1.out‚ 2.9384765625 disk.sda.in‚0 proc.cpu.threads‚17 disk.sda1.in‚0 disk.sda2.in‚0" proc.memory.rssMB‚ 2783.1171875 disk.drbd2266.out‚0 disk.rbd1.in‚0& disk.nvme0c0n1.out‚7151.3603515625 disk.drbd2266.in‚0! disk.nvme0c0n1.in‚ 11.80859375 disk.drbd1716.out‚0$ memory_percent‚85.27801036834717) proc.memory.availableMB‚ 2412.05078125 disk.drbd2080.in‚0 disk.nvme0n1.in‚ 11.80859375 disk.sda2.out‚0 disk.dm-1.in‚0 disk.drbd1716.in‚0* proc.memory.percent‚16.986799240112305 network.recv‚668154% disk./.usageGB‚251.51676559448242 disk.rbd0.in‚0 disk.sda1.out‚0! disk.dm-0.out‚7151.4853515625Æ×DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚2605 VGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 8980994048 XJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 165393020 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 PCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚31114090 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 169268224 \HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚122587485504000 IDeviceStatsMonitor.on_train_batch_start/allocation.all.current‚20040 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚455 ^JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚125559274271232 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 \HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚119177489803264 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚24128 WEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 1605145641472 `LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚122587502674432 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚431 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚20040 ^LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 1605314909696 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 446693376 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 446693376 [IDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 1605314909696 QCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 182432380 QBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 9902751744 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚20038 O@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 9373790793 JBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚431 _step‚172 ]IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚122587502674432 VGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 9283702784 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 144203776 _MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 1674446666240 N?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 9902751744 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚213 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚2605 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚3 ZFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚125559041279488 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 435026432 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚24128 UGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 431144273 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 JBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚213 PCDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated‚40574881 PCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚40554843 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚18697169 E=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚644 UADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚124192631145472 IDeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚124192631145472ãaÂÛ'ó ÕO ^LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 1596742615826 TFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 446693376 $ _timestamp‚1.788416210102712e+09 L?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚31114090 eQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚123884619385856 ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 208219136 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 L?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚40554843 TGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚31114092 SEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 313315328 _runtime‚1791.752260493 UADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚120774067026070 TGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚40574881 E8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚71668933 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚17170432 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 BDeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚124192631145472 aMDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚123884594613248 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 C;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚644 QDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚29406071 cQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 1674654885376 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 M@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚29405616 H?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚1003 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚452 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 248172032 D>DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚2605 VGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 8980994048 XJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 165393020 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 PCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚31114090 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 169268224 \HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚122587485504000 IDeviceStatsMonitor.on_train_batch_start/allocation.all.current‚20040 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚455 ^JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚125559274271232 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 \HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚119177489803264 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚24128 WEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 1605145641472 `LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚122587502674432 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚431 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚20040 ^LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 1605314909696 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 446693376 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 446693376 [IDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 1605314909696 QCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 182432380 QBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 9902751744 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚20038 O@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 9373790793 JBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚431 _step‚172 ]IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚122587502674432 VGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 9283702784 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 144203776 _MDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.freed‚ 1674446666240 N?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 9902751744 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚213 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚2605 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚3 ZFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚125559041279488 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 435026432 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚24128 UGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 431144273 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 JBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚213 PCDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated‚40574881 PCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚40554843 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚18697169 E=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚644 UADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚124192631145472 I6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚644 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚213 $ _timestamp‚1.788416210778579e+09 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚20064 TEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 8980994048 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚644 T@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚124248906640896 cODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚123940553825280 [MDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚ 216594432 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 248172032 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚24401 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 S?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚120828653139914 XFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 1597298669514 QBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 9902751744 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚31128090 J=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚40573315 OBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚29419103 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 182432588 trainer/global_step‚2149 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1003 aODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 1675418207232 WCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚120828835572502 M>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 9373790793 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 _step‚173 WIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 446693376 OADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 186451968 UCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 1605870995456 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚3 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚213 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚17170432 K>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚71721471 ^JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚119231371509760 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚20066 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 F@DeviceStatsMonitor.on_train_batc*ŸÓh_end/oversize_allocations.freed‚0 L=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 9902751744 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚431 G:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚71701405 REDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚10713066 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 144203776 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 431144273 C6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚71701405 L>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 186451968 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 169281536 XFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 1605870995456 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 O@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 9902751744 FDeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 241367040 _runtime‚1792.428286879 NADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚31128092 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24772608 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 446693376 S?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚124248720188928 [GDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚122642866363904 SEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 435026432 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚2605 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚644 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚213 NADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚40593379 J;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 9679899648 REDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚31128092 G:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚71721471 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚18705563 \JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 1597464062742 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚35 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 ZFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚119231354470400 VHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 165393228 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚477 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚24401 EDeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚29418626 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 SDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 9456058368 F6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚644 PBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 435026432 ^JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚122642866363904 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚10713540 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚474 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚17039360 WCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚122642849193472 M>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 9679899648 _KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚123940529052672 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 VGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 9456058368 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚431 ZFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚122642849193472 TEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 9283702784 \JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 1606040276992 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 446693376 REDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚18705560 \HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚125615972032512 REDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚40593379 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚20064 ]KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 1675201612800 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚24128 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0Ï‹å&ö ßM WIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 446693376 OADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 186451968 UCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 1605870995456 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚3 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚213 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚17170432 K>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚71721471 ^JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚119231371509760 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚20066 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 L=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 9902751744 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚431 G:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚71701405 REDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚10713066 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 144203776 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 431144273 C6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚71701405 L>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 186451968 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 169281536 XFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 1605870995456 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 O@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 9902751744 FDeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 241367040 _runtime‚1792.428286879 NADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚31128092 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24772608 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 446693376 S?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚124248720188928 [GDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚122642866363904 SEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 435026432 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚2605 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚644 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚213 NADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚40593379 J;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 9679899648 REDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚31128092 G:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚71721471 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚18705563 \JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 1597464062742 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚35 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 ZFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚119231354470400 VHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 165393228 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚477 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚24401 EDeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚29418626 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 SDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 9456058368 F6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚644 PBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 435026432 ^JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚122642866363904 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚10713540 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚474 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚17039360 WCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚122642849193472 M>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 9679899648 _KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚123940529052672 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 VGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 9456058368 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚431 ZFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚122642849193472 TEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 9283702784 \JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 1606040276992 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 446693376 REDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚18705560 \HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚125615972032512 REDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚40593379 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚20064 ]KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 1675201612800 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚24128 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚2 QCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 313315328 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚644 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚213 $ _timestamp‚1.788416210778579e+09 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚20064 TEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 8980994048 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚644 T@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚124248906640896 cODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚123940553825280 [MDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚ 216594432 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 248172032 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚24401 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 S?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚120828653139914 XFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 1597298669514 QBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 9902751744 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚31128090 J=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚40573315 OBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚29419103 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 182432588 trainer/global_step‚2149 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1003 aODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 1675418207232 WCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚120828835572502 M>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 9373790793 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 _step‚173­æ­Ô2÷ - _wandbruntime‚1792  _runtime‚1792¿•VXø Ò 2loss/SRqqt3/assignment_loss‚1.5911686995791063 2loss/SRqqt4/assignment_loss‚1.3896795746847248 /loss/FBt1/detection_loss‚2.3370279918114343 /loss/FBt4/detection_loss‚0.6305835326683933 %loss/total_loss‚44.94589912058833 epoch‚3 trainer/global_step‚2149 1loss/SRqqt3/detection_loss‚1.8062915226651564 1loss/SRqqt4/detection_loss‚1.0276019442672573 0loss/FBt3/assignment_loss‚0.1737420080336795 0loss/SRqqt1/detection_loss‚1.805934894056815 % _timestamp‚1.7884162107801878e+09 /loss/FRt1/assignment_loss‚3.882593149887888 /loss/FRt3/assignment_loss‚3.375769533609089 0loss/SRqqt1/assignment_loss‚2.05321829302131 0loss/FBt2/assignment_loss‚0.2212960608566694 /loss/FRt1/detection_loss‚2.2431050505881247 _runtime‚1792.429375153 0loss/FRt4/assignment_loss‚1.9709661124826787 1loss/FBt4/assignment_loss‚0.06590748534531482 -loss/FRt3/detection_loss‚2.57061667740345 /loss/FRt4/detection_loss‚1.7600023813320549 /loss/FBt2/detection_loss‚2.5897316296895343 /loss/FBt3/detection_loss‚2.4662600701505486 2loss/SRqqt2/assignment_loss‚2.2724230820313096 0loss/FBt1/assignment_loss‚0.1691620310472596 .loss/FRt2/detection_loss‚2.460025825782826 1loss/SRqqt2/detection_loss‚2.4452080205082893 0loss/FRt2/assignment_loss‚3.6375835490854165 _step‚174j·z/]ù × 2loss/SRqqt2/assignment_loss‚2.2724230820313096 0loss/FBt1/assignment_loss‚0.1691620310472596 .loss/FRt2/detection_loss‚2.460025825782826 1loss/SRqqt2/detection_loss‚2.4452080205082893 0loss/FRt2/assignment_loss‚3.6375835490854165 _step‚174 2loss/SRqqt3/assignment_loss‚1.5911686995791063 2loss/SRqqt4/assignment_loss‚1.3896795746847248 /loss/FBt1/detection_loss‚2.3370279918114343 /loss/FBt4/detection_loss‚0.6305835326683933 %loss/total_loss‚44.94589912058833 epoch‚3 trainer/global_step‚2149 1loss/SRqqt3/detection_loss‚1.8062915226651564 1loss/SRqqt4/detection_loss‚1.0276019442672573 0loss/FBt3/assignment_loss‚0.1737420080336795 0loss/SRqqt1/detection_loss‚1.805934894056815 % _timestamp‚1.7884162107801878e+09 /loss/FRt1/assignment_loss‚3.882593149887888 /loss/FRt3/assignment_loss‚3.375769533609089 0loss/SRqqt1/assignment_loss‚2.05321829302131 0loss/FBt2/assignment_loss‚0.2212960608566694 /loss/FRt1/detection_loss‚2.2431050505881247 _runtime‚1792.429375153 0loss/FRt4/assignment_loss‚1.9709661124826787 1loss/FBt4/assignment_loss‚0.06590748534531482 -loss/FRt3/detection_loss‚2.57061667740345 /loss/FRt4/detection_loss‚1.7600023813320549 /loss/FBt2/detection_loss‚2.5897316296895343 /loss/FBt3/detection_loss‚2.4662600701505486®ù”Ý“ú ‚(:ˆ Ú¡äÔÜË ¨ gpu.0.gpu‚41.0 gpu.0.memory‚4, gpu.0.memoryAllocated‚12.165603637695312* gpu.0.memoryAllocatedBytes‚ 10450173952 gpu.0.temp‚36.0 gpu.0.powerWatts‚161.084' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚32.2168 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0’Ö®/Qû ‚(:Æ Ú¡äÔûàŸª$ disk./.usageGB‚251.5519905090332 disk.drbd2265.out‚0! disk.nvme0c0n1.in‚ 11.82421875 disk.nvme0n1.in‚ 11.82421875 disk.dm-1.in‚0 disk.drbd1716.in‚0 disk.drbd2266.in‚0 disk.drbd2266.out‚0 disk.drbd2080.out‚0 disk.rbd0.out‚0$ disk.nvme0n1.out‚7361.4130859375 disk.sda.out‚ 70.5390625* disk./.usagePercent‚16.881865149239694 disk.rbd1.in‚0 disk.sda3.out‚ 70.5390625 disk.drbd2080.in‚0 network.sent‚ 4.179282e+06 disk.sda.in‚0 disk.drbd1716.out‚0 disk.sda2.in‚0" proc.memory.rssMB‚ 2783.1171875 disk.rbd1.out‚ 3.0009765625 disk.dm-1.out‚ 70.5390625 disk.sda2.out‚0 cpu‚26.16257283752873 disk.drbd2265.in‚0( proc.memory.availableMB‚ 2411.9296875 disk.sda1.in‚0 disk.sda1.out‚0! disk.dm-0.out‚7361.5380859375 network.recv‚683170& disk.nvme0c0n1.out‚7361.4130859375$ memory_percent‚85.27874946594238* proc.memory.percent‚16.986799240112305 disk.dm-0.in‚ 11.82421875 disk.sda3.in‚0 disk.rbd0.in‚0 proc.cpu.threads‚17Æ€¼Žžü ‚(:“ é¡äÔ‚Ϩ gpu.0.gpu‚52.0 gpu.0.memory‚6, gpu.0.memoryAllocated‚12.165603637695312* gpu.0.memoryAllocatedBytes‚ 10450173952 gpu.0.temp‚35.0 gpu.0.powerWatts‚147.947' gpu.0.enforcedPowerLimitWatts‚500.0) gpu.0.powerPercent‚29.589399999999998 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0ã½ÕSý ‚(:È é¡äÔ¬¼ª disk.sda1.out‚0 disk.rbd0.out‚0 disk.sda2.out‚0* disk./.usagePercent‚16.883691250981432 disk.dm-0.in‚ 11.82421875 disk.drbd2266.out‚0 disk.drbd2265.in‚0$ memory_percent‚85.27724742889404* proc.memory.percent‚16.986799240112305 disk.sda3.out‚ 71.3046875 disk.drbd2265.out‚0 cpu‚26.159075308953526 proc.cpu.threads‚17 disk.sda.out‚ 71.3046875 network.sent‚ 4.264974e+06$ disk./.usageGB‚251.5792007446289! disk.dm-0.out‚7415.1669921875 disk.sda2.in‚0 disk.drbd2266.in‚0& disk.nvme0c0n1.out‚7415.0419921875 disk.dm-1.out‚ 71.3046875 disk.sda.in‚0 disk.sda3.in‚0 network.recv‚687611 disk.dm-1.in‚0 disk.drbd2080.in‚0 disk.drbd2080.out‚0 disk.rbd1.in‚0 disk.drbd1716.out‚0 disk.nvme0n1.in‚ 11.82421875$ disk.nvme0n1.out‚7415.0419921875 disk.rbd1.out‚ 3.0166015625! disk.nvme0c0n1.in‚ 11.82421875 disk.rbd0.in‚0 disk.sda1.in‚0) proc.memory.availableMB‚ 2412.17578125" proc.memory.rssMB‚ 2783.1171875 disk.drbd1716.in‚0‚å“þ ‚(:ˆ ø¡äÔâÍ®¨ gpu.0.gpu‚42.0 gpu.0.memory‚4, gpu.0.memoryAllocated‚12.165603637695312* gpu.0.memoryAllocatedBytes‚ 10450173952 gpu.0.temp‚36.0 gpu.0.powerWatts‚167.828' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚33.5656 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0KÊ>…Wÿ ‚(:Ì ø¡äÔ¹±¤ª disk.sda3.in‚0 disk.dm-1.in‚0 network.sent‚ 4.270213e+06 disk.rbd1.in‚0 disk.sda.in‚0 disk.drbd2080.in‚0 disk.rbd0.in‚0 disk.sda1.in‚0 disk.sda3.out‚ 71.90234375 disk.dm-1.out‚ 71.90234375* disk./.usagePercent‚16.883833847178003% disk./.usageGB‚251.58132553100586 disk.drbd2265.in‚0 disk.rbd1.out‚ 3.0166015625! disk.nvme0c0n1.in‚ 11.82421875 disk.rbd0.out‚0$ disk.nvme0n1.out‚7504.8935546875 disk.sda1.out‚0 disk.sda2.out‚0 disk.drbd2266.out‚0! disk.dm-0.out‚7505.0185546875 disk.sda.out‚ 71.90234375 disk.drbd1716.out‚0) proc.memory.availableMB‚ 2412.17578125* proc.memory.percent‚16.986799240112305 cpu‚26.155491412664325 proc.cpu.threads‚17 network.recv‚691140 disk.sda2.in‚0 disk.drbd2265.out‚0& disk.nvme0c0n1.out‚7504.8935546875 disk.drbd1716.in‚0$ memory_percent‚85.27724742889404" proc.memory.rssMB‚ 2783.1171875 disk.nvme0n1.in‚ 11.82421875 disk.drbd2266.in‚0 disk.dm-0.in‚ 11.82421875 disk.drbd2080.out‚0ÔB=2€- _wandbruntime‚1830  _runtime‚1830àN!×Å¿ % lr-AdamW/pg1‚0.001503794156462663 % lr-AdamW/pg2‚0.001503794156462663 trainer/global_step‚2199 % _timestamp‚1.7884162485142334e+09 _runtime‚1830.163662154 _step‚175ò® Ê‚Ä % _timestamp‚1.7884162485142334e+09 _runtime‚1830.163662154 _step‚175 % lr-AdamW/pg1‚0.001503794156462663 % lr-AdamW/pg2‚0.001503794156462663 trainer/global_step‚2199¯ 5Žw2ƒ- _wandbruntime‚1830  _runtime‚1830HS¨Ç?„ÑO QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 186438656 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 UADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚123503399890278 N?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 9902751744 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 446693376 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 169268224 G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚2605 YEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚126997297585152 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚452 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 L=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 9679899648 O@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 9679899648 IDeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 TFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 446693376 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚24401 L?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚31814090 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 169268224 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 `LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚121870740199424 @8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚644 JBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚213 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚17170432 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚2605 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 \HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚121870723160064 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 IDeviceStatsMonitor.on_train_batch_start/allocation.all.current‚20040 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚19116722 BDeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚126997111146496 ]IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚125355687148032 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚17170432 \ODeviceStatsMonitor.on_train_batch_start/inactive7|˜_split_bytes.large_pool.current‚24772608 VGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 9283702784 ZFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚128393733741568 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚24128 UGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 431144273 ZKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 9456058368 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚20038 YEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚125355669977600 TGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚41501833 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚3 MFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚35 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚10946893 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚10946441 aMDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚126681316584448 WEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 1641441168896 PCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚41481795 NFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚985 cQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 1712633764864 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 248172032 _runtime‚1830.167910581 UADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚126997111146496 PCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚31814090 QDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚30063615 SEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 313315328 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0#t.VÜ'…ÖO L?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚31814090 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 169268224 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 `LDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.allocated‚121870740199424 @8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚644 JBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚213 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚17170432 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚2605 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 \HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚121870723160064 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 IDeviceStatsMonitor.on_train_batch_start/allocation.all.current‚20040 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚19116722 BDeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚126997111146496 ]IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚125355687148032 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚17170432 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚24772608 VGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 9283702784 ZFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚128393733741568 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚24128 UGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 431144273 ZKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 9456058368 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚20038 YEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚125355669977600 TGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚41501833 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚3 MFDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.peak‚35 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚10946893 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚10946441 aMDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚126681316584448 WEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 1641441168896 PCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚41481795 NFDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.peak‚985 cQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 1712633764864 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.peak‚ 248172032 _runtime‚1830.167910581 UADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚126997111146496 PCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚31814090 QDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚30063615 SEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 313315328 F@DeviceStatsMonitor.on_train_batch_start/segment.large_pool.freed‚0 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 186438656 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 UADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚123503399890278 N?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 9902751744 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 446693376 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 169268224 G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚2605 YEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚126997297585152 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚452 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 L=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 9679899648 O@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 9679899648 IDeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 TFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 446693376 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚24401°ì§Eº2†- _wandbruntime‚1831  _runtime‚1831>¸»á&‡ÛM NADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚31828090 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 M>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 9373790793 TEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 8980994048 NADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚41520331 F6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚644 WCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚123558168436710 ^JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚121924604866560 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚644 UCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 1642166522880 NADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚41500267 cODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚126737275796480 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 G:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚73328357 K>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚30076038 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚213 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚24128 PDeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚73348423 REDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚41520331 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚431 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚213 S?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚127053200189952 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚2 REDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚31828092 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚2605 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚20064 ZFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚125411033667072 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚10951390 \JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 1642335804416 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚431 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚431 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 182432588 \JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 1633563570150 YGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 1642335804416 WCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚125411033667072 J=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚41500267 L>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 186451968 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚20064 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 _step‚177 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 169281536 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚17039360 TEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 9283702784 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 G:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚73348423 J;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 9679899648 [MDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚ 220788736 ]KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 1713173340672 QBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 9902751744 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚644 \HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚128450669925888 VHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 165393228 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 446693376 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17170432 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚24401 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 245561344 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 248172032 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 431144273 VGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 9456058368 T@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚127053386641920 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 XFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 1633398176922 FDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 9679899648 XDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚128450424364544 WIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 446693376 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚19125113 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚35 BDeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1003 ZFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚121924587827200 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 446693376 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚213 QBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 9283702784 EDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 G:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚73328357 K>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚30076038 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚213 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚24128 PDeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚73348423 REDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚41520331 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚431 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚213 S?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚127053200189952 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚2 REDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚31828092 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚2605 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚20064 ZFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚125411033667072 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚10951390 \JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 1642335804416 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚431 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚431 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 182432588 \JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 1633563570150 YGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 1642335804416 WCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚125411033667072 J=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚41500267 L>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 186451968 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚20064 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 _step‚177 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 169281536 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚17039360 TEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 9283702784 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 G:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚73348423 J;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 9679899648 [MDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚ 220788736 ]KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 1713173340672 QBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 9902751744 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚644 \HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚128450669925888 VHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 165393228 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 446693376 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17170432 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚24401 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 245561344 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 248172032 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 431144273 VGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 9456058368 T@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚127053386641920 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 XFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 1633398176922 FDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 9679899648 XDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚128450424364544 WIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 446693376 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚19125113 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚35 BDeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1003 ZFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚121924587827200 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 446693376 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚213 QBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 9283702784 EDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 9373790793 TEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 8980994048 NADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚41520331 F6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚644 WCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚123558168436710 ^JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚121924604866560 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚644 UCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 1642166522880 NADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚41500267 cODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚126737275796480± È2‰- _wandbruntime‚1831  _runtime‚1831Š#0ŠØ 1loss/SRqqt4/detection_loss‚1.0706203418737608 /loss/FBt2/detection_loss‚2.3919029270901397 /loss/FBt3/detection_loss‚1.9155394339135714 /loss/FBt1/detection_loss‚2.5198380876989925 /loss/FRt2/assignment_loss‚4.437951799985525 1loss/FBt2/assignment_loss‚0.13660974130749204ߢHj. 1loss/FBt3/assignment_loss‚0.20822645301606307 2loss/FBt4/assignment_loss‚0.010358311616299421 1loss/SRqqt1/detection_loss‚1.7985462401554269 /loss/FBt4/detection_loss‚0.8561719155856478 $ _timestamp‚1.788416249430985e+09 2loss/SRqqt3/assignment_loss‚1.6769287931371708 _runtime‚1831.080191062 0loss/FRt3/assignment_loss‚3.0724834471606135 0loss/FBt1/assignment_loss‚0.2097632405039964 %loss/total_loss‚45.26173345178357 epoch‚3 _step‚178 /loss/FRt1/assignment_loss‚3.806628396414747 /loss/FRt4/assignment_loss‚2.378663459420204 2loss/SRqqt4/assignment_loss‚1.3598120734227677 /loss/FRt1/detection_loss‚2.2606735337487978 /loss/FRt3/detection_loss‚2.4017546849974085 /loss/FRt4/detection_loss‚1.9646078368648887 .loss/FRt2/detection_loss‚2.343233551528003 1loss/SRqqt2/detection_loss‚2.4472818935095373 trainer/global_step‚2199 2loss/SRqqt1/assignment_loss‚1.8480956868953862 2loss/SRqqt2/assignment_loss‚2.2690657897671658 1loss/SRqqt3/detection_loss‚1.8769758121699702›Pþc‹Ý _runtime‚1831.080191062 0loss/FRt3/assignment_loss‚3.0724834471606135 0loss/FBt1/assignment_loss‚0.2097632405039964 %loss/total_loss‚45.26173345178357 epoch‚3 _step‚178 /loss/FRt1/assignment_loss‚3.806628396414747 /loss/FRt4/assignment_loss‚2.378663459420204 2loss/SRqqt4/assignment_loss‚1.3598120734227677 /loss/FRt1/detection_loss‚2.2606735337487978 /loss/FRt3/detection_loss‚2.4017546849974085 /loss/FRt4/detection_loss‚1.9646078368648887 .loss/FRt2/detection_loss‚2.343233551528003 1loss/SRqqt2/detection_loss‚2.4472818935095373 trainer/global_step‚2199 2loss/SRqqt1/assignment_loss‚1.8480956868953862 2loss/SRqqt2/assignment_loss‚2.2690657897671658 1loss/SRqqt3/detection_loss‚1.8769758121699702 1loss/SRqqt4/detection_loss‚1.0706203418737608 /loss/FBt2/detection_loss‚2.3919029270901397 /loss/FBt3/detection_loss‚1.9155394339135714 /loss/FBt1/detection_loss‚2.5198380876989925 /loss/FRt2/assignment_loss‚4.437951799985525 1loss/FBt2/assignment_loss‚0.13660974130749204 1loss/FBt3/assignment_loss‚0.20822645301606307 2loss/FBt4/assignment_loss‚0.010358311616299421 1loss/SRqqt1/detection_loss‚1.7985462401554269 /loss/FBt4/detection_loss‚0.8561719155856478 $ _timestamp‚1.788416249430985e+09 2loss/SRqqt3/assignment_loss‚1.6769287931371708²…ŸNÏŒ‚(:’ ‡¢äÔèÙ±¨ gpu.0.gpu‚36.0 gpu.0.memory‚3, gpu.0.memoryAllocated‚12.165603637695312* gpu.0.memoryAllocatedBytes‚ 10450173952 gpu.0.temp‚36.0 gpu.0.powerWatts‚179.193' gpu.0.enforcedPowerLimitWatts‚500.0( gpu.0.powerPercent‚35.83860000000001 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0ÐI1íQ‚(:Æ ‡¢äÔÍÚœ« disk.rbd0.in‚0 disk.sda1.out‚0 disk.sda2.out‚0! disk.nvme0c0n1.in‚ 11.82421875 disk.dm-0.in‚ 11.82421875 disk.dm-1.out‚ 72.3828125 disk.sda1.in‚0 disk.drbd2265.out‚0 disk.sda.in‚0 network.recv‚696142 disk.sda3.in‚0 disk.dm-1.in‚0$ disk.nvme0n1.out‚7556.4912109375' proc.memory.availableMB‚ 2412.109375% disk./.usageGB‚251.60655212402344 disk.drbd2080.out‚0$ memory_percent‚85.27765274047852" proc.memory.rssMB‚ 2783.1171875* proc.memory.percent‚16.986799240112305* disk./.usagePercent‚16.885526825001936 disk.sda3.out‚ 72.3828125 disk.drbd2080.in‚0 disk.sda2.in‚0 disk.drbd2265.in‚0 proc.cpu.threads‚17 disk.rbd1.in‚0 disk.rbd1.out‚ 3.07958984375& disk.nvme0c0n1.out‚7556.4912109375 network.sent‚ 4.276803e+06 disk.drbd1716.in‚0 disk.drbd1716.out‚0 disk.rbd0.out‚0 disk.drbd2266.in‚0! disk.dm-0.out‚7556.6162109375 disk.sda.out‚ 72.3828125 cpu‚26.1520762754981 disk.nvme0n1.in‚ 11.82421875 disk.drbd2266.out‚0ßÅ!Ø“Ž‚(:ˆ –¢äÔïÆ—¨ gpu.0.gpu‚55.0 gpu.0.memory‚7, gpu.0.memoryAllocated‚12.165603637695312* gpu.0.memoryAllocatedBytes‚ 10450173952 gpu.0.temp‚35.0 gpu.0.powerWatts‚120.542' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚24.1084 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚00‹VT‚(:É –¢äÔõÒ¡ª disk.dm-0.in‚ 11.82421875" disk.dm-0.out‚7609.07275390625 disk.drbd2265.in‚0 disk.rbd1.in‚0* disk./.usagePercent‚16.884712465035516 disk.sda3.in‚0 disk.rbd0.in‚0 disk.rbd0.out‚0 disk.rbd1.out‚ 3.07958984375" proc.memory.rssMB‚ 2783.1171875* proc.memory.percent‚16.986799240112305' disk.nvme0c0n1.out‚7608.94775390625 disk.sda2.out‚0 disk.drbd2266.out‚0 network.sent‚ 4.364192e+06 disk.drbd1716.in‚0 disk.nvme0n1.in‚ 11.82421875% disk.nvme0n1.out‚7608.94775390625 disk.drbd2266.in‚0 disk.sda.out‚ 72.953125 disk.drbd2080.in‚0 proc.cpu.threads‚17 network.recv‚702179% disk./.usageGB‚251.59441757202148! disk.nvme0c0n1.in‚ 11.82421875 disk.drbd1716.out‚0 disk.sda1.in‚0 disk.sda1.out‚0$ memory_percent‚85.27991771697998) proc.memory.availableMB‚ 2411.73828125 disk.sda.in‚0 disk.drbd2080.out‚0 disk.sda2.in‚0 disk.drbd2265.out‚0 disk.dm-1.out‚ 72.953125 disk.sda3.out‚ 72.953125 disk.dm-1.in‚0 cpu‚26.14863730593171I³üÆ2- _wandbruntime‚1868  _runtime‚1868,~8!Å‘¿ & lr-AdamW/pg1‚0.0014819830725767244 & lr-AdamW/pg2‚0.0014819830725767244 trainer/global_step‚2249 $ _timestamp‚1.788416286613927e+09 _runtime‚ 1868.26356388 _step‚179üÄÍÊ’Ä $ _timestamp‚1.788416286613927e+09 _runtime‚ 1868.26356388 _step‚179 & lr-AdamW/pg1‚0.0014819830725767244 & lr-AdamW/pg2‚0.0014819830725767244 trainer/global_step‚2249³=ÄeŒ2“- _wandbruntime‚1868  _runtime‚1868ŽM±[×'”ÑO SDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 9283702784 VGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 8980994048 G?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚213 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 186438656 ^LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 1677905964544 ZHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 1677736696320 eQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚129478063328256 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚24772608 XIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 9456058368 ZFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚131228412892672 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚213 L=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 9679899648 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚17170432 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚24401 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚2605 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚3 UADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚126232732754486 ]IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚128123871621632 WEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 1677736696320 VGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 9283702784 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 UADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚129801591147520 O@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 9679899648 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 QCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 182432380 JBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚213 D>DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 YEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚126232915186866 ^LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 1668941630642 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚20038 \HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚128123854451200 N?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 9902751744 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚470 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚24401 L?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚42408747 QBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 9902751744 [IDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 1677905964544 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚11182883 ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 214510592 ZHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 1668776237622 TFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 446693376 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚467 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 ^JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚131228652175872 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 G?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚431 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚2 L?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚32514090 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 239283200 BDeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚24128 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚19536272 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 @8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚644 VBDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated‚129801777586176 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚17170432 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 169268224 IDeviceStatsMonitor.on_train_batch_start/allocation.all.current‚20040 E=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚644 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 % _timestamp‚1.7884162866155279e+09 G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚2605 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 C;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚644 E8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚74922837 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚24128 SEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 313315328 SDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 9902751744 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 TGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚32514092 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 PCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚42408747 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 QDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚30719155 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚19536269 ZKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 9456058368 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 169268224 M@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚74942877 PCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚32514090 cQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 1750588847616 @8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚644 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 446693376 _runtime‚1868.265213293 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚20040 R>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚129801591147520 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚20038 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 144203776 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 XJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 165393020 N@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 186438656 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚431 trainer/global_step‚2249 _step‚180 PCDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated‚42428785 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚11182416 aMDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚129478038555648ºFÚ‰Ü'•ÖO WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 446693376 IDeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚24128 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚19536272 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 @8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚644 VBDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated‚129801777586176 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚17170432 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 169268224 IDeviceStatsMonitor.on_train_batch_start/allocation.all.current‚20040 E=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚644 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 % _timestamp‚1.7884162866155279e+09 G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚2605 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 C;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚644 E8DeviceStatsMonitor.on_train_batch_start/active.all.freed‚74922837 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚24128 SEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 313315328 SDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 9902751744 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 TGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚32514092 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 PCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚42408747 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 QDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚30719155 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚19536269 ZKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 9456058368 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 169268224 M@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚74942877 PCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚32514090 cQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.allocated‚ 1750588847616 @8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚644 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 446693376 _runtime‚1868.265213293 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚20040 R>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚129801591147520 OEDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.current‚20038 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 144203776 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 XJDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.current‚ 165393020 N@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 186438656 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚431 trainer/global_step‚2249 _step‚180 PCDeviceStatsMonitor.on_train_batch_start/active.small_pool.allocated‚42428785 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚11182416 aMDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚129478038555648 SDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 9283702784 VGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 8980994048 G?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚213 HBDeviceStatsMonitor.on_train_batch_start/oversize_allocations.freed‚0 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 186438656 ^LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 1677905964544 ZHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 1677736696320 eQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚129478063328256 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚24772608 XIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 9456058368 ZFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚131228412892672 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚213 L=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 9679899648 TGDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.current‚17170432 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚24401 KBDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.peak‚2605 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚3 UADeviceStatsMonitor.on_train_batch_start/requested_bytes.all.freed‚126232732754486 ]IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚128123871621632 WEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 1677736696320 VGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 9283702784 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 UADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚129801591147520 O@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 9679899648 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 QCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 182432380 JBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚213 D>DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 YEDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.allocated‚126232915186866 ^LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 1668941630642 KADeviceStatsMonitor.on_train_batch_start/active.small_pool.current‚20038 \HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚128123854451200 N?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 9902751744 JBDeviceStatsMonitor.on_train_batch_start/inactive_split.all.current‚470 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚24401 L?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚42408747 QBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 9902751744 [IDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 1677905964544 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚11182883 ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 214510592 ZHDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.freed‚ 1668776237622 TFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 446693376 F@DeviceStatsMonitor.on_train_batch_start/segment.small_pool.freed‚0 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚467 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 ^JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚131228652175872 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 G?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚431 GADeviceStatsMonitor.on_train_batch_start/active.large_pool.current‚2 L?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚32514090 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 239283200 B6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚644 C6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚74955309 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚19544663 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚24401 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚17170432 L=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 9902751744 \HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚131285347444736 S?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚126287318868330 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚20064 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚35 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚11187535 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚17039360 G:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚74975375 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚644 ^JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚124617838223360 \JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 1669663077558 XIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 9456058368 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 169281536 NADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚32528090 K>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚30731726 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚469 QCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 313315328 WIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 446693376 ]KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 1751135179776 BDeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 186451968 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 169281536 _KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚129533972995072 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 144203776 G:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚74955309 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚213 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24772608 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 248172032 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚644 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚32528090 UCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 1678462050304 \JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 1678631331840 LDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚985 SDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 9456058368 [GDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚128179235311104 OADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 186451968 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 WCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚126287501300918 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 446693376 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚431 PDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 9373790793 XFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 1669497684330 _step‚181 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚24128 FDeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 K>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚74975375 NADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚42427219 TEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 8980994048 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚213 PBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 435026432 SEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 435026432 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚24401 REDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚11187066 VHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 165393228 YGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 1678631331840 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 239269888 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 182432588 WCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚128179218140672 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1003 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 TEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 9283702784 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚472 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚431 % _timestamp‚1.7884162872892156e+09 VGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 9456058368 EDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 9679899648 NADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚42447283 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 REDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚19544660 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚431 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚3n & æ&˜àM J;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 9679899648 ^JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚128179235311104 XDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚131285108174848 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 ZFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚124617821184000 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 446693376 [MDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚ 214497280 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚2605 REDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚42447283 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 K>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚74975375 NADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚42427219 TEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 8980994048 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚213 PBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 435026432 SEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 435026432 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚24401 REDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚11187066 VHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 165393228 YGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 1678631331840 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 239269888 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 182432588 WCDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.freed‚128179218140672 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1003 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 TEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 9283702784 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚472 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚431 % _timestamp‚1.7884162872892156e+09 VGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 9456058368 EDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 9679899648 NADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚42447283 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 REDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚19544660 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚431 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚3 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚2 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17170432 aODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 1751349677056 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 XFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 1678462050304 trainer/global_step‚2249 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 _runtime‚1868.938985314 WCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚129857866642944 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚20066 ZFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚128179218140672 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚644 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚644 C6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚74955309 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚19544663 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚24401 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚17170432 L=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 9902751744 \HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚131285347444736 S?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚126287318868330 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚20064 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚35 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚11187535 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚17039360 G:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚74975375 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚644 ^JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚124617838223360 \JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 1669663077558 XIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 9456058368 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 169281536 NADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚32528090 K>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚30731726 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚469 QCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 313315328 WIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 446693376 ]KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 1751135179776 BDeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 186451968 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 169281536 _KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚129533972995072 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 144203776 G:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚74955309 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚213 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24772608 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 248172032 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚644 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚32528090 UCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 1678462050304 \JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 1678631331840 LDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚985 SDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 9456058368 [GDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚128179235311104 OADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 186451968 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 WCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚126287501300918 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 446693376 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚431 PDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 9373790793 XFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 1669497684330 _step‚181 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚24128 FDeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚24128 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 435026432 SEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 313315328 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 VGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 9283702784 UGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 431144273 G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚2605 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚24772608 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 446693376 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 446693376 C;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚644 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚213 R>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚132606071148544 \HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚130892038924800 PCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚33214090 TGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚43355737 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚24128 aMDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚132274760526848 ^LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 1705041138050 H?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚1003 PCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚33214092 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 186438656 `LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚130892056095232 IDeviceStatsMonitor.on_train_batch_start/allocation.all.current‚20040 eQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚132274785299456 E=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚644 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚20040 ]IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚130892056095232 SDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 9283702784 M@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚31375056 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚11419701 @8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚644 UADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚132606071148544 PCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚43335699 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚19955822 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 @8DeviceStatsMonitor.on_train_batch_start/segment.all.peak‚644 A7DeviceStatsMonitor.on_train_batch_start/active.all.peak‚24401 VBDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated‚132606257587200 TGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚33214092 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 144203776 D>DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 QDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚31375523 IDeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 QDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚31375523 IDeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚24128 RDDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.peak‚ 435026432 SEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 313315328 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 VGDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.peak‚ 9283702784 UGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 431144273 G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚2605 \ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.current‚24772608 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 446693376 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 446693376 C;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚644 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚213 R>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚132606071148544 \HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚130892038924800 PCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚33214090 TGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚43355737 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚24128 aMDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.freed‚132274760526848 ^LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 1705041138050 H?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚1003 PCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚33214092 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 186438656 `LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.allocated‚130892056095232 IDeviceStatsMonitor.on_train_batch_start/allocation.all.current‚20040 eQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚132274785299456 E=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚644 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚20040 ]IDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.allocated‚130892056095232 SDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 9283702784 M@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚31375056¸Fvg]2¨- _wandbruntime‚1906  _runtime‚1906¯yfLà&©ÚM ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚24401 NADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚43354171 WIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 446693376 C6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚76582261 XFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 1714757577728 \HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚134120021361664 [MDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚ 224983040 T@DeviceStatsMonitor.on_train_batch_end/active_bytes.all.allocated‚132662346643968 M>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 9679899648 \JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 1705762584966 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 431144273 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 QCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 313315328 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚20066 NADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚33228092 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17170432 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚35 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 169281536 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚502 O@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 9902751744 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚431 trainer/global_step‚2299 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚17170432 G:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚76582261 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚499 _runtime‚ 1906.85115317 REDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚11423762 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 XFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 1705597191738 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚213 QBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 9283702784 _KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚132330694966272 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1003 VGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 9456058368 SDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 9456058368 ^JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚130947419784704 ]KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 1789076639744 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚644 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚431 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚431 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚24401 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚2 BDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 9373790793 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚644 OADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 186451968 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 169281536 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 182432588 XIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 9456058368 J;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 9679899648 PBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 435026432 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 249755648 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚20064 PDeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚31387972 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚17039360 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 446693376 K>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚76602327 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚3 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 446693376 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚644 QBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 9902751744 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24772608 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 144203776 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 S?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚129016651732538 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚33228090 S?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚132662160192000 \JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 1714926859264 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚19964213 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚213 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚24128 SEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 435026432 OBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚31388474 WCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚132662346643968 ZFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.freed‚130947402614272 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 248172032 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 WCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚129016834165126 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 _step‚185 REDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚19964210 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚11424261 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚2605 LDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚985 G:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚76602327 EDeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 186451968 UCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 1714757577728 L=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 9902751744 F6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚644 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚20064 cODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚132330719738880 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 % _timestamp‚1.7884163252014613e+09 NADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚43374235 J=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚43354171 XDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚134119771606016 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 F8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 WCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚129016834165126 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 _step‚185 REDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.freed‚19964210 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚11424261 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚2605 LDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚985 G:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚76602327 EDeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 186451968 UCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 1714757577728 L=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 9902751744 F6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚644 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚20064 cODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚132330719738880 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 % _timestamp‚1.7884163252014613e+09 NADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚43374235 J=DeviceStatsMonitor.on_train_batch_end/active.small_pool.freed‚43354171 XDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚134119771606016 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 FDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 9679899648 \JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 1705762584966 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 431144273 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 QCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 313315328 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚20066 NADeviceStatsMonitor.on_train_batch_end/active.large_pool.allocated‚33228092 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17170432 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚35 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 169281536 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚502 O@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 9902751744 JBDeviceStatsMonitor.on_train_batch_end/segment.large_pool.allocated‚431 trainer/global_step‚2299 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚17170432 G:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚76582261 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚499 _runtime‚ 1906.85115317 REDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚11423762 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 XFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 1705597191738 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚213 QBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 9283702784 _KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚132330694966272 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1003 VGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 9456058368 SDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 9456058368 ^JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚130947419784704 ]KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 1789076639744 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚644 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚431 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚431 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚24401 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚2 BDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 9373790793 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚644 OADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 186451968 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 169281536 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 182432588 XIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 9456058368 J;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 9679899648 PBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 435026432 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 249755648 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚20064 PDeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚31387972 UHDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.current‚17039360 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 446693376 K>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚76602327 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚3 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 446693376 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚644 QBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 9902751744 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24772608 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.peak‚ 144203776 GADeviceStatsMonitor.on_train_batch_end/oversize_segments.allocated‚0 S?DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.freed‚129016651732538 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚33228090 S?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚132662160192000 \JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.allocated‚ 1714926859264 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚19964213 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚213 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚24128¹2þ2«- _wandbruntime‚1906  _runtime‚19069J¤*]¬× /loss/FBt2/detection_loss‚2.5964011391003927 /loss/FBt4/detection_loss‚0.5794023542855837 _runtime‚1906.852234341 0loss/FBt3/assignment_loss‚0.5342722704626794 0loss/FRt3/assignment_loss‚3.5215970613490577 0loss/FRt4/assignment_loss‚1.9872842992461004 2loss/SRqqt4/assignment_loss‚1.3568508337416194 1loss/SRqqt1/detection_loss‚1.8153265942633152 /loss/FBt1/detection_loss‚1.9388181431726976 /loss/FRt3/detection_loss‚2.4497847927027734 1loss/SRqqt3/detection_loss‚1.9895979516453794 % _timestamp‚1.7884163252030177e+09 1loss/FBt1/assignment_loss‚0.21182389494408033 .loss/FRt1/detection_loss‚2.210673253688187 1loss/SRqqt4/detection_loss‚1.0048307029201704 _step‚186 1loss/SRqqt2/detection_loss‚2.3553512087091804 2loss/FBt4/assignment_loss‚0.027309281613880514 2loss/SRqqt3/assignment_loss‚1.8439722513104533 /loss/FRt4/detection_loss‚1.9748910261794579 /loss/FBt3/detection_loss‚2.1140504763884977 %loss/total_loss‚44.59274689900854 epoch‚3 .loss/FRt1/assignment_loss‚3.56243100260744 0loss/FRt2/assignment_loss‚3.4943528647224107 2loss/SRqqt1/assignment_loss‚1.9308379736542702 1loss/SRqqt2/assignment_loss‚2.279963099397719 trainer/global_step‚2299 0loss/FBt2/assignment_loss‚0.1434003745516141 .loss/FRt2/detection_loss‚2.669524048351579Sâ'»b­Ü 0loss/FRt4/assignment_loss‚1.9872842992461004 2loss/SRqqt4/assignment_loss‚1.3568508337416194 1loss/SRqqt1/detection_loss‚1.8153265942633152 /loss/FBt1/detection_loss‚1.9388181431726976 /loss/FRt3/detection_loss‚2.4497847927027734 1loss/SRqqt3/detection_loss‚1.9895979516453794 % _timestamp‚1.7884163252030177e+09 1loss/FBt1/assignment_loss‚0.21182389494408033 .loss/FRt1/detection_loss‚2.210673253688187 1loss/SRqqt4/detection_loss‚1.0048307029201704 _step‚186 1loss/SRqqt2/detection_loss‚2.3553512087091804 2loss/FBt4/assignment_loss‚0.027309281613880514 2loss/SRqqt3/assignment_loss‚1.8439722513104533 /loss/FRt4/detection_loss‚1.9748910261794579 /loss/FBt3/detection_loss‚2.1140504763884977 %loss/total_loss‚44.59274689900854 epoch‚3 .loss/FRt1/assignment_loss‚3.56243100260744 0loss/FRt2/assignment_loss‚3.4943528647224107 2loss/SRqqt1/assignment_loss‚1.9308379736542702 1loss/SRqqt2/assignment_loss‚2.279963099397719 trainer/global_step‚2299 0loss/FBt2/assignment_loss‚0.1434003745516141 .loss/FRt2/detection_loss‚2.669524048351579 /loss/FBt2/detection_loss‚2.5964011391003927 /loss/FBt4/detection_loss‚0.5794023542855837 _runtime‚1906.852234341 0loss/FBt3/assignment_loss‚0.5342722704626794 0loss/FRt3/assignment_loss‚3.5215970613490577ºV][£“®‚(:ˆ Ò¢äÔ˜¢°¨ gpu.0.gpu‚26.0 gpu.0.memory‚3, gpu.0.memoryAllocated‚12.165603637695312* gpu.0.memoryAllocatedBytes‚ 10450173952 gpu.0.temp‚36.0 gpu.0.powerWatts‚170.211' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚34.0422 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0{#$ ¯‚(:È Ò¢äÔ« disk.sda1.out‚0 disk.dm-0.in‚ 11.82421875! disk.dm-0.out‚7837.7275390625 disk.sda.out‚ 75.44921875 network.recv‚720321 disk.sda3.in‚0 disk.sda2.in‚0 disk.sda2.out‚0 disk.rbd1.out‚ 3.142578125 network.sent‚ 4.469039e+0¥” Õ36 disk.dm-1.in‚0 disk.rbd0.in‚0 disk.rbd0.out‚0 disk.sda1.in‚0 disk.drbd2265.in‚0& proc.memory.availableMB‚ 2411.90625* disk./.usagePercent‚16.888078452095705 disk.drbd1716.out‚0 disk.drbd2266.out‚0! disk.nvme0c0n1.in‚ 11.82421875 disk.sda.in‚0$ memory_percent‚85.27889251708984 disk.sda3.out‚ 75.44921875 disk.dm-1.out‚ 75.44921875 disk.drbd2080.in‚0 disk.drbd2265.out‚0" proc.memory.rssMB‚ 2783.1171875 cpu‚26.135274120235692 disk.nvme0n1.in‚ 11.82421875$ disk.nvme0n1.out‚7837.6025390625& disk.nvme0c0n1.out‚7837.6025390625% disk./.usageGB‚251.64457321166992* proc.memory.percent‚16.986799240112305 proc.cpu.threads‚17 disk.drbd1716.in‚0 disk.drbd2080.out‚0 disk.drbd2266.in‚0 disk.rbd1.in‚0ÄáV‡ž°‚(:“ á¢äÔ”‘Ú¨ gpu.0.gpu‚55.0 gpu.0.memory‚6, gpu.0.memoryAllocated‚12.165603637695312* gpu.0.memoryAllocatedBytes‚ 10450173952 gpu.0.temp‚35.0 gpu.0.powerWatts‚156.229' gpu.0.enforcedPowerLimitWatts‚500.0) gpu.0.powerPercent‚31.245800000000003 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0B\kÕP±‚(:Å á¢äÔõ– « disk.sda2.out‚0 disk.rbd0.out‚0 disk.sda1.in‚0 disk.drbd1716.in‚0 proc.cpu.threads‚17 disk.drbd2265.in‚0! disk.dm-0.out‚7899.3525390625 disk.sda3.out‚ 75.9765625 disk.drbd1716.out‚0 disk.drbd2266.in‚0 disk.sda1.out‚0 disk.sda.out‚ 75.9765625 disk.dm-1.out‚ 75.9765625" proc.memory.rssMB‚ 2783.1171875* disk./.usagePercent‚16.889483421443614! disk.nvme0c0n1.in‚ 11.82421875& disk.nvme0c0n1.out‚7899.2275390625 disk.sda.in‚0( proc.memory.availableMB‚ 2412.1015625 disk.sda3.in‚0 network.recv‚725642 disk.nvme0n1.in‚ 11.82421875 disk.sda2.in‚0 disk.rbd1.in‚0 disk.dm-0.in‚ 11.82421875$ memory_percent‚85.27770042419434 disk.rbd0.in‚0$ disk.nvme0n1.out‚7899.2275390625 disk.rbd1.out‚ 3.142578125 disk.drbd2080.out‚0 network.sent‚ 4.55571e+06% disk./.usageGB‚251.66550827026367 disk.dm-1.in‚0 cpu‚26.13232733874823 disk.drbd2266.out‚0 disk.drbd2265.out‚0 disk.drbd2080.in‚0* proc.memory.percent‚16.986799240112305]I«ó2²- _wandbruntime‚1944  _runtime‚1944V"yÛij¾ % lr-AdamW/pg1‚0.001437015166247771 % lr-AdamW/pg2‚0.001437015166247771 trainer/global_step‚2349 % _timestamp‚1.7884163626583798e+09 _runtime‚ 1944.30776082 _step‚187ÄþW3É´Ã % lr-AdamW/pg2‚0.001437015166247771 trainer/global_step‚2349 % _timestamp‚1.7884163626583798e+09 _runtime‚ 1944.30776082 _step‚187 % lr-AdamW/pg1‚0.001437015166247771»¦ˆÁé2µ- _wandbruntime‚1944  _runtime‚1944Ò4ª.×'¶ÑO D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚20040 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 186438656 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚464 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 VGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 8980994048 XIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 9456058368 JBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚431 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚213 ZHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 1750327751168 QDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚32033480 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 144203776 TFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 446693376 JBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚213 ^LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 1750497019392 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚20040 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 O@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 9373790793 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 R>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚135410551149568 L=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 9679899648 TGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚44282689 D>DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 C;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚644 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚24128 M@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚32033013 BDeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚2605 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 QBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 9902751744 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 446693376 SDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 9283702784 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚431 YEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚135410737588224 H?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚1003 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 M@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚78196781 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚20375372 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 UFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 9456058368 PCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚33914090 ZKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 9456058368 _runtime‚1944.312807553 L?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚44262651 WEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 1750327751168 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 169268224 TGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚33914092 @8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚644 N?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 9902751744 E=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚644 IDeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚24128 ^JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚136898017525248 QCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 182432380 trainer/global_step‚2349 YEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚133660223398400 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 G?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚431 _step‚188 UADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚135410551149568 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚11657644 SEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 313315328 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0 O@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 9679899648 ZFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚136897780339200 ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 212413440 SDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 9902751744 PCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚33914092 L?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚33914090 \HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚133660223398400 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚20375369 UGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 431144273 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 169268224 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚11658108Y•ž”Ü'·ÖO O@DeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.peak‚ 9679899648 ZFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚136897780339200 ]ODeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.small_pool.current‚ 212413440 SDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 9902751744 PCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚33914092 L?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚33914090 \HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚133660223398400 KEDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.current‚2 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.freed‚20375369 UGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 431144273 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 169268224 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.allocated‚11658108 D:DeviceStatsMonitor.on_train_batch_start/active.all.current‚20040 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 186438656 QIDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.current‚464 ICDeviceStatsMonitor.on_train_batch_start/oversize_segments.allocated‚0 VGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 8980994048 XIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.current‚ 9456058368 JBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚431 LDDeviceStatsMonitor.on_train_batch_start/segment.small_pool.allocated‚213 ZHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 1750327751168 QDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚32033480 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 144203776 TFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.peak‚ 446693376 JBDeviceStatsMonitor.on_train_batch_start/segment.small_pool.current‚213 ^LDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.allocated‚ 1750497019392 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚20040 60DeviceStatsMonitor.on_train_batch_start/num_ooms‚0 O@DeviceStatsMonitor.on_train_batch_start/requested_bytes.all.peak‚ 9373790793 F@DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.freed‚0 R>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚135410551149568 L=DeviceStatsMonitor.on_train_batch_start/active_bytes.all.peak‚ 9679899648 TGDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.allocated‚44282689 D>DeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 C;DeviceStatsMonitor.on_train_batch_start/segment.all.current‚644 LBDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.peak‚24128 M@DeviceStatsMonitor.on_train_batch_start/inactive_split.all.freed‚32033013 BDeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚2605 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 QBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 9902751744 WIDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.current‚ 446693376 SDDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.peak‚ 9283702784 @:DeviceStatsMonitor.on_train_batch_start/num_oom_rejections‚0 LDDeviceStatsMonitor.on_train_batch_start/segment.large_pool.allocated‚431 YEDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.allocated‚135410737588224 H?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚1003 ?9DeviceStatsMonitor.on_train_batch_start/segment.all.freed‚0 M@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚78196781 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚20375372 ?9DeviceStatsMonitor.on_train_batch_start/num_alloc_retries‚0 UFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 9456058368 PCDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.freed‚33914090 ZKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.allocated‚ 9456058368 _runtime‚1944.312807553 L?DeviceStatsMonitor.on_train_batch_start/active.small_pool.freed‚44262651 WEDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.freed‚ 1750327751168 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.freed‚0 XJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.current‚ 169268224 TGDeviceStatsMonitor.on_train_batch_start/allocation.large_pool.allocated‚33914092 @8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚644 N?DeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.peak‚ 9902751744 E=DeviceStatsMonitor.on_train_batch_start/segment.all.allocated‚644 IDeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚24128 ^JDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.allocated‚136898017525248 QCDeviceStatsMonitor.on_train_batch_start/requested_bytes.all.current‚ 182432380 trainer/global_step‚2349 YEDeviceStatsMonitor.on_train_batch_start/active_bytes.large_pool.freed‚133660223398400 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 G?DeviceStatsMonitor.on_train_batch_start/segment.large_pool.peak‚431 _step‚188 UADeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.freed‚135410551149568 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚11657644 SEDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.peak‚ 313315328 E?DeviceStatsMonitor.on_train_batch_start/oversize_segments.freed‚0 MGDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.freed‚0¼$|—2¸- _wandbruntime‚1944  _runtime‚1944°EÇò¼$¹ÚM L>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 186451968 REDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚44301187 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚3 QCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 313315328 NADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚44281123 ]KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 1827056723968 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚644 J;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 9679899648 OBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚32046528 K>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚32046047 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚35 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24772608 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 B6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚644 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 241367040 \JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 1741862092374 XIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 9456058368 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 446693376 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚644 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚20064 K>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚78229279 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 248172032 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 _KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚135127416937472 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 ^JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚133715604258304 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚478 SDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 9456058368 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚213 \HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚136954715028480 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 ^JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚130004304936960 WIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 446693376 F6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚644 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 trainer/global_step‚2349 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 446693376 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚2 YGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 1751222386688 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚481 XDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚136954473661440 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚213 PDeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚20066 QBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 9283702784 WCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚135466826644992 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚2605 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1003 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 M>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 9373790793 ZFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚130004287897600 TEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 8980994048 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 169281536 XFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 1751053105152 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚20383763 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 M>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 9679899648 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚20064 [MDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚ 216594432 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 431144273 VGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 9456058368 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚24401 REDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚33928092 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 [GDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚133715604258304 S?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚135466640193024 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 182432588 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚17170432 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 169281536 REDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚11662287 QBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 9902751744 UCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 1751053105152 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚11662765 LDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚985 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚213 NADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚44301187 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17170432 TEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 9283702784 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 O@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 9902751744 $ _timestamp‚1.788416363346747e+09 C6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚78209213 G:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚78209213 NADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚33928090 SEDeviceStatsMonitor.on_g9Më$train_batch_end/allocated_bytes.small_pool.peak‚ 435026432 L=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 9902751744 OADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 186451968 WCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚131746167029334 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚431 PBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 435026432 VHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 1653932286]P„å&ºßM TEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.peak‚ 9283702784 ;4DeviceStatsMonitor.on_train_batch_end/max_split_size‚-1 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 O@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 9902751744 $ _timestamp‚1.788416363346747e+09 C6DeviceStatsMonitor.on_train_batch_end/active.all.freed‚78209213 G:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚78209213 NADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚33928090 SEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 435026432 L=DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.peak‚ 9902751744 OADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 186451968 WCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚131746167029334 E=DeviceStatsMonitor.on_train_batch_end/segment.large_pool.peak‚431 PBDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.peak‚ 435026432 VHDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.current‚ 165393228 L>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 186451968 REDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚44301187 MGDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.current‚3 QCDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.peak‚ 313315328 NADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚44281123 ]KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.freed‚ 1827056723968 E?DeviceStatsMonitor.on_train_batch_end/oversize_segments.current‚0 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚644 J;DeviceStatsMonitor.on_train_batch_end/active_bytes.all.peak‚ 9679899648 OBDeviceStatsMonitor.on_train_batch_end/inactive_split.all.allocated‚32046528 K>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚32046047 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚35 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24772608 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 F@DeviceStatsMonitor.on_train_batch_end/oversize_allocations.freed‚0 B6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚644 TFDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.current‚ 241367040 \JDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.allocated‚ 1741862092374 XIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.allocated‚ 9456058368 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 446693376 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 A9DeviceStatsMonitor.on_train_batch_end/segment.all.current‚644 I?DeviceStatsMonitor.on_train_batch_end/active.small_pool.current‚20064 K>DeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚78229279 XJDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.peak‚ 248172032 E?DeviceStatsMonitor.on_train_batch_end/oversize_allocations.peak‚0 _KDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.freed‚135127416937472 HBDeviceStatsMonitor.on_train_batch_end/oversize_allocations.current‚0 ^JDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.allocated‚133715604258304 OGDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.current‚478 SDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.peak‚ 9456058368 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚213 \HDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.allocated‚136954715028480 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 ^JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚130004304936960 WIDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.allocated‚ 446693376 F6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚644 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 trainer/global_step‚2349 UGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.current‚ 446693376 E?DeviceStatsMonitor.on_train_batch_end/active.large_pool.current‚2 YGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 1751222386688 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚481 XDDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.all.freed‚136954473661440 E=DeviceStatsMonitor.on_train_batch_end/segment.small_pool.peak‚213 PDeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 B8DeviceStatsMonitor.on_train_batch_end/active.all.current‚20066 QBDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.peak‚ 9283702784 WCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚135466826644992 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚2605 F=DeviceStatsMonitor.on_train_batch_end/inactive_split.all.peak‚1003 @:DeviceStatsMonitor.on_train_batch_end/num_sync_all_streams‚2 M>DeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 9373790793 ZFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚130004287897600 TEDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.peak‚ 8980994048 SEDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.current‚ 169281536 XFDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.freed‚ 1751053105152 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚20383763 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 M>DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 9679899648 MCDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.current‚20064 [MDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.current‚ 216594432 SEDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.peak‚ 431144273 VGDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.current‚ 9456058368 ?5DeviceStatsMonitor.on_train_batch_end/active.all.peak‚24401 REDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚33928092 =7DeviceStatsMonitor.on_train_batch_end/segment.all.freed‚0 [GDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.allocated‚133715604258304 S?DeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.freed‚135466640193024 OADeviceStatsMonitor.on_train_batch_end/requested_bytes.all.current‚ 182432588 UHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.large_pool.current‚17170432 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 169281536 REDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚11662287 QBDeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.allocated‚ 9902751744 UCDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.freed‚ 1751053105152 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚11662765 LDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚985 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚213 NADeviceStatsMonitor.on_train_batch_end/active.small_pool.allocated‚44301187 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17170432½P Ý2»- _wandbruntime‚1944  _runtime‚1944Õ€šÑW¼Ñ 2loss/SRqqt1/assignment_loss‚1.8555734473271202 1loss/SRqqt4/assignment_loss‚1.280213130196135 1loss/SRqqt2/detection_loss‚2.4385287087896597 _runtime‚1944.997999824 /loss/FRt2/assignment_loss‚3.765119802307438 1loss/SRqqt3/assignment_loss‚1.675402424308241 0loss/FBt2/assignment_loss‚0.1183786087189912 .loss/FRt4/detection_loss‚1.640403456353354 /loss/FBt1/detection_loss‚1.9473903287540784 _step‚190 /loss/FBt3/detection_loss‚1.8381279933452606 /loss/FBt4/detection_loss‚0.6860169260698843 0loss/FRt1/assignment_loss‚3.5252019493352797 0loss/FBt3/assignment_loss‚0.2808308429684257 0loss/FBt4/assignment_loss‚0.0313983004676584 0loss/SRqqt1/detection_loss‚1.818942324982749 /loss/FRt3/assignment_loss‚3.546593678964151 2loss/SRqqt2/assignment_loss‚2.0705910731053008 %loss/total_loss‚44.06254121510308 0loss/FRt4/assignment_loss‚1.8455244825696044 1loss/FBt1/assignment_loss‚0.38166712135319936 /loss/FRt1/detection_loss‚2.4984988671550084 0loss/SRqqt3/detection_loss‚1.846964676854431 /loss/FBt2/detection_loss‚2.6193244059880576 .loss/FRt2/detection_loss‚2.571094467027767 /loss/FRt3/detection_loss‚2.7016342358814702 0loss/SRqqt4/detection_loss‚1.079119962279814 epoch‚3 trainer/global_step‚2349 % _timestamp‚1.7884163633488061e+09E3~l\½Ö 0loss/SRqqt1/detection_loss‚1.818942324982749 /loss/FRt3/assignment_loss‚3.546593678964151 2loss/SRqqt2/assignment_loss‚2.0705910731053008 %loss/total_loss‚44.06254121510308 0loss/FRt4/assignment_loss‚1.8455244825696044 1loss/FBt1/assignment_loss‚0.38166712135319936 /loss/FRt1/detection_loss‚2.4984988671550084 0loss/SRqqt3/detection_loss‚1.846964676854431 /loss/FBt2/detection_loss‚2.6193244059880576 .loss/FRt2/detection_loss‚2.571094467027767 /loss/FRt3/detection_loss‚2.7016342358814702 0loss/SRqqt4/detection_loss‚1.079119962279814 epoch‚3 trainer/global_step‚2349 % _timestamp‚1.7884163633488061e+09 2loss/SRqqt1/assignment_loss‚1.8555734473271202 1loss/SRqqt4/assignment_loss‚1.280213130196135 1loss/SRqqt2/detection_loss‚2.4385287087896597 _runtime‚1944.997999824 /loss/FRt2/assignment_loss‚3.765119802307438 1loss/SRqqt3/assignment_loss‚1.675402424308241 0loss/FBt2/assignment_loss‚0.1183786087189912 .loss/FRt4/detection_loss‚1.640403456353354 /loss/FBt1/detection_loss‚1.9473903287540784 _step‚190 /loss/FBt3/detection_loss‚1.8381279933452606 /loss/FBt4/detection_loss‚0.6860169260698843 0loss/FRt1/assignment_loss‚3.5252019493352797 0loss/FBt3/assignment_loss‚0.2808308429684257 0loss/FBt4/assignment_loss‚0.0313983004676584¾E¡4x¾‚(:’ ð¢äÔ·—ˆ¨ gpu.0.gpu‚58.0 gpu.0.memory‚5, gpu.0.memoryAllocated‚12.165603637695312* gpu.0.memoryAllocatedBytes‚ 10450173952 gpu.0.temp‚35.0 gpu.0.powerWatts‚167.392' gpu.0.enforcedPowerLimitWatts‚500.0( gpu.0.powerPercent‚33.47839999999999 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0{iú=1¿‚(:¦ ð¢äÔ§§ª disk.drbd2266.in‚0 disk.drbd2266.out‚0 disk.rbd1.in‚0( proc.memory.availableMB‚ 2411.8203125* proc.memory.percent‚16.986799240112305! disk.nvme0c0n1.in‚ 11.82421875* disk./.usagePercent‚16.890932424087417 disk.sda1.in‚0 disk.sda2.in‚0 disk.drbd2265.out‚0 disk.sda3.in‚0 disk.drbd1716.out‚0 disk.drbd2080.in‚0 disk.rbd1.out‚ 3.142578125 disk.nvme0c0n1.out‚7962.5 disk.sda.out‚76.84375 disk.dm-1.in‚0 disk.drbd2080.out‚0 disk.sda.in‚0 disk.sda3.out‚76.84375 disk.nvme0n1.out‚7962.5" proc.memory.rssMB‚ 2783.1171875 proc.cpu.threads‚17 disk.sda1.out‚0 network.recv‚729379 disk.drbd1716.in‚0 disk.rbd0.in‚0 disk.rbd0.out‚0 disk.sda2.out‚0$ disk./.usageGB‚251.6870994567871 disk.nvme0n1.in‚ 11.82421875 disk.drbd2265.in‚0 disk.dm-0.in‚ 11.82421875 disk.dm-1.out‚76.84375$ memory_percent‚85.27941703796387 network.sent‚ 4.561017e+06 disk.dm-0.out‚7962.625 cpu‚26.12894170765317}i0ê“À‚(:ˆ ÿ¢äÔ¡š®¨ gpu.0.gpu‚24.0 gpu.0.memory‚3, gpu.0.memoryAllocated‚12.165603637695312* gpu.0.memoryAllocatedBytes‚ 10450173952 gpu.0.temp‚36.0 gpu.0.powerWatts‚165.746' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚33.1492 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0l!…îXÁ‚(:Í ÿ¢äÔûÎÔª disk.nvme0n1.in‚ 11.82421875% disk.nvme0n1.out‚8004.58544921875 disk.sda1.in‚0 disk.drbd2080.out‚0$ memory_percent‚85.27941703796387 disk.drbd2265.in‚0 disk.sda1.out‚0* proc.memory.percent‚16.986799240112305 cpu‚26.125948637109786 proc.cpu.threads‚17 disk.dm-1.out‚ 77.33984375 disk.rbd0.in‚0 disk.rbd0.out‚0 disk.sda3.in‚0 disk.dm-1.in‚0 disk.drbd1716.out‚0 disk.drbd2080.in‚0 disk.drbd2266.out‚0 disk.sda.in‚0( proc.memory.availableMB‚ 2411.8203125 disk.drbd2265.out‚0" proc.memory.rssMB‚ 2783.1171875 disk.dm-0.in‚ 11.82421875 disk.sda3.out‚ 77.33984375 disk.drbd1716.in‚0 network.sent‚ 4.648973e+06 network.recv‚735978 disk.sda2.in‚0 disk.sda2.out‚0 disk.rbd1.out‚ 3.142578125! disk.nvme0c0n1.in‚ 11.82421875' disk.nvme0c0n1.out‚8004.58544921875" disk.dm-0.out‚8004.71044921875* disk./.usagePercent‚16.891472087969596% disk./.usageGB‚251.69514083862305 disk.drbd2266.in‚0 disk.rbd1.in‚0 disk.sda.out‚ 77.33984375ðAlƒ‘‚(:† Ž£äÔýª°¨ gpu.0.gpu‚42.0 gpu.0.memory‚4, gpu.0.memoryAllocated‚12.165603637695312* gpu.0.memoryAllocatedBytes‚ 10450173952 gpu.0.temp‚36.0 gpu.0.powerWatts‚168.46' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚33.692 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0Ÿœ ÅRÂ(:Ç Ž£äÔœ¹«% disk.nvme0c0n1.out‚8071.845703125 disk.sda.out‚ 78.20703125 disk.sda1.out‚0 network.recv‚740368 disk.drbd2265.in‚0 disk.rbd1.out‚ 3.142578125 disk.drbd1716.out‚0 disk.nvme0n1.in‚ 11.82421875# disk.nvme0n1.out‚8071.845703125 disk.sda2.in‚0 network.sent‚ 4.655081e+06 disk.drbd2266.out‚0 disk.drbd2265.out‚0 disk.dm-1.in‚0 disk.dm-1.out‚ 78.20703125 disk.drbd2080.out‚0 disk.rbd0.in‚0 disk.sda1.in‚0 disk.rbd0.out‚0) proc.memory.availableMB‚ 2411.81640625 proc.cpu.threads‚17 disk.drbd2266.in‚0 disk.rbd1.in‚0 disk.dm-0.in‚ 11.82421875 disk.sda.in‚0 disk.sda3.in‚0" proc.memory.rssMB‚ 2783.1171875* proc.memory.percent‚16.986799240112305* disk./.usagePercent‚16.892860928842847 disk.dm-0.out‚8071.970703125 disk.drbd1716.in‚0 disk.drbd2080.in‚0 disk.sda2.out‚0$ memory_percent‚85.27944087982178% disk./.usageGB‚251.71583557128906! disk.nvme0c0n1.in‚ 11.82421875 disk.sda3.out‚ 78.20703125 cpu‚26.12273937157593eĺí2Ä- _wandbruntime‚1982  _runtime‚1982^™êfÇÅÁ trainer/global_step‚2399 % _timestamp‚1.7884164009660413e+09 _runtime‚1982.615438014 _step‚191 & lr-AdamW/pg1‚0.0014138958889082743 & lr-AdamW/pg2‚0.0014138958889082743ÀÚ ÌÆÆ trainer/global_step‚2399 % _timestamp‚1.7884164009660413e+09 _runtime‚1982.615438014 _step‚191 & lr-AdamW/pg1‚0.0014138958889082743 & lr-AdamW/pg2‚0.0014138958889082743¿³#§2Ç- _wandbruntime‚1982  _runtime‚1982a¬ç×'ÈÑO N@DeviceStatsMonitor.on_train_batch_start/active_bytes.all.current‚ 186438656 H?DeviceStatsMonitor.on_train_batch_start/inactive_split.all.peak‚1003 OIDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.current‚3 QDDeviceStatsMonitor.on_train_batch_start/inactive_split.all.allocated‚32693075 GADeviceStatsMonitor.on_train_batch_start/oversize_segments.current‚0 G?DeviceStatsMonitor.on_train_batch_start/segment.small_pool.peak‚213 [IDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.allocated‚ 1786792546816 E;DeviceStatsMonitor.on_train_batch_start/allocation.all.peak‚24401 GADeviceStatsMonitor.on_train_batch_start/oversize_allocations.peak‚0 ^LDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.allocated‚ 1777240152866 VBDeviceStatsMonitor.on_train_batch_start/active_bytes.all.allocated‚138215217589248 \HDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.freed‚136428407872000 H>DeviceStatsMonitor.on_train_batch_start/allocation.all.current‚20040 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚20794922 trainer/global_step‚2399 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚17170432 IDeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 \HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚132643656587264 UGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 431144273 SDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 9902751744 % _timestamp‚1.7884164009670935e+09 PCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚34614092 G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚2605 PCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚45189603 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 446693376 _step‚192 R>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚138215031150592 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚11897681 ZFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚139732479526400 @8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚644 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 ZHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 1786623278592 M@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚79823733 eQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚137868229241856 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 144203776 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 VGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 8980994048 QBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 9902751744 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 169268224 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 186438656 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 243477504 UFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 9456058368 JBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚431 L?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚34614090 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚24128 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 IDeviceStatsMonitor.on_train_batch_start/allocation.all.current‚20040 XKDeviceStatsMonitor.on_train_batch_start/inactive_split.large_pool.allocated‚20794922 trainer/global_step‚2399 WJDeviceStatsMonitor.on_train_batch_start/allocated_bytes.large_pool.current‚17170432 IDeviceStatsMonitor.on_train_batch_start/oversize_segments.peak‚0 WJDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.current‚17039360 \HDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.freed‚132643656587264 UGDeviceStatsMonitor.on_train_batch_start/requested_bytes.small_pool.peak‚ 431144273 SDDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.allocated‚ 9902751744 % _timestamp‚1.7884164009670935e+09 PCDeviceStatsMonitor.on_train_batch_start/active.large_pool.allocated‚34614092 G>DeviceStatsMonitor.on_train_batch_start/active.large_pool.peak‚2605 PCDeviceStatsMonitor.on_train_batch_start/allocation.small_pool.freed‚45189603 =6DeviceStatsMonitor.on_train_batch_start/max_split_size‚-1 YKDeviceStatsMonitor.on_train_batch_start/reserved_bytes.small_pool.allocated‚ 446693376 _step‚192 R>DeviceStatsMonitor.on_train_batch_start/active_bytes.all.freed‚138215031150592 TGDeviceStatsMonitor.on_train_batch_start/inactive_split.small_pool.freed‚11897681 ZFDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.freed‚139732479526400 @8DeviceStatsMonitor.on_train_batch_start/num_device_alloc‚644 =7DeviceStatsMonitor.on_train_batch_start/num_device_free‚0 ZHDeviceStatsMonitor.on_train_batch_start/allocated_bytes.small_pool.freed‚ 1786623278592 M@DeviceStatsMonitor.on_train_batch_start/allocation.all.allocated‚79823733 eQDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.allocated‚137868229241856 ZLDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.large_pool.peak‚ 144203776 JDDeviceStatsMonitor.on_train_batch_start/oversize_allocations.current‚0 VGDeviceStatsMonitor.on_train_batch_start/requested_bytes.large_pool.peak‚ 8980994048 QBDeviceStatsMonitor.on_train_batch_start/reserved_bytes.all.current‚ 9902751744 UGDeviceStatsMonitor.on_train_batch_start/active_bytes.small_pool.current‚ 169268224 QCDeviceStatsMonitor.on_train_batch_start/allocated_bytes.all.current‚ 186438656 VHDeviceStatsMonitor.on_train_batch_start/inactive_split_bytes.all.current‚ 243477504 UFDeviceStatsMonitor.on_train_batch_start/reserved_bytes.large_pool.peak‚ 9456058368 JBDeviceStatsMonitor.on_train_batch_start/segment.large_pool.current‚431 L?DeviceStatsMonitor.on_train_batch_start/active.large_pool.freed‚34614090 H>DeviceStatsMonitor.on_train_batch_start/active.small_pool.peak‚24128 LFDeviceStatsMonitor.on_train_batch_start/oversize_allocations.allocated‚0 IDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 9679899648 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚644 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚213 SEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 435026432 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚24128 EDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 9373790793 ZFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚132697521254400 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 446693376 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚34628090 OADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 186451968 K>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚32705736 WCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚134475499893542 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17170432 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚2605 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚213 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚431 NADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚45208075 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚466 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24772608 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 FDeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚79856231 NADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚34628090 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚11902889 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚644 LDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚985 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 REDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚34628092 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚644 G:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚79836165 ^JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚132697538293760 O@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 9902751744 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 L>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 186451968 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 169281536 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚24401 REDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚45228139 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚20803313 _runtime‚1983.294666429 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 G:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚79856231 aODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 1865251126272 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚35 REDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚11902426 cODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚137924163681280 XFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 1777796206554 YGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 1787517914112 WCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚138271306646016 BDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.peak‚ 9679899648 >6DeviceStatsMonitor.on_train_batch_end/num_device_alloc‚644 JBDeviceStatsMonitor.on_train_batch_end/segment.small_pool.allocated‚213 SEDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.peak‚ 435026432 ICDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.current‚2 ;5DeviceStatsMonitor.on_train_batch_end/num_device_free‚0 C=DeviceStatsMonitor.on_train_batch_end/oversize_segments.freed‚0 J@DeviceStatsMonitor.on_train_batch_end/allocation.small_pool.peak‚24128 EDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.peak‚ 9373790793 ZFDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.freed‚132697521254400 D>DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.freed‚0 RDDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.peak‚ 446693376 J=DeviceStatsMonitor.on_train_batch_end/active.large_pool.freed‚34628090 OADeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.current‚ 186451968 K>DeviceStatsMonitor.on_train_batch_end/inactive_split.all.freed‚32705736 WCDeviceStatsMonitor.on_train_batch_end/requested_bytes.all.allocated‚134475499893542 REDeviceStatsMonitor.on_train_batch_end/active_bytes.large_pool.current‚17170432 I@DeviceStatsMonitor.on_train_batch_end/allocation.large_pool.peak‚2605 H@DeviceStatsMonitor.on_train_batch_end/segment.small_pool.current‚213 H@DeviceStatsMonitor.on_train_batch_end/segment.large_pool.current‚431 NADeviceStatsMonitor.on_train_batch_end/allocation.small_pool.freed‚45208075 H@DeviceStatsMonitor.on_train_batch_end/inactive_split.all.current‚466 ZMDeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.current‚24772608 >8DeviceStatsMonitor.on_train_batch_end/num_oom_rejections‚0 D>DeviceStatsMonitor.on_train_batch_end/segment.large_pool.freed‚0 FDeviceStatsMonitor.on_train_batch_end/allocation.all.allocated‚79856231 NADeviceStatsMonitor.on_train_batch_end/allocation.large_pool.freed‚34628090 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.allocated‚11902889 4.DeviceStatsMonitor.on_train_batch_end/num_ooms‚0 >6DeviceStatsMonitor.on_train_batch_end/segment.all.peak‚644 LDDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.peak‚985 D>DeviceStatsMonitor.on_train_batch_end/segment.small_pool.freed‚0 REDeviceStatsMonitor.on_train_batch_end/allocation.large_pool.allocated‚34628092 C;DeviceStatsMonitor.on_train_batch_end/segment.all.allocated‚644 G:DeviceStatsMonitor.on_train_batch_end/allocation.all.freed‚79836165 ^JDeviceStatsMonitor.on_train_batch_end/requested_bytes.large_pool.allocated‚132697538293760 O@DeviceStatsMonitor.on_train_batch_end/reserved_bytes.all.current‚ 9902751744 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.small_pool.freed‚0 L>DeviceStatsMonitor.on_train_batch_end/active_bytes.all.current‚ 186451968 VHDeviceStatsMonitor.on_train_batch_end/allocated_bytes.small_pool.current‚ 169281536 C9DeviceStatsMonitor.on_train_batch_end/allocation.all.peak‚24401 REDeviceStatsMonitor.on_train_batch_end/allocation.small_pool.allocated‚45228139 VIDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.allocated‚20803313 _runtime‚1983.294666429 KEDeviceStatsMonitor.on_train_batch_end/reserved_bytes.large_pool.freed‚0 G:DeviceStatsMonitor.on_train_batch_end/active.all.allocated‚79856231 aODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.small_pool.allocated‚ 1865251126272 =7DeviceStatsMonitor.on_train_batch_end/num_alloc_retries‚0 JDDeviceStatsMonitor.on_train_batch_end/oversize_allocations.allocated‚0 KDDeviceStatsMonitor.on_train_batch_end/inactive_split.large_pool.peak‚35 REDeviceStatsMonitor.on_train_batch_end/inactive_split.small_pool.freed‚11902426 cODeviceStatsMonitor.on_train_batch_end/inactive_split_bytes.large_pool.allocated‚137924163681280 XFDeviceStatsMonitor.on_train_batch_end/requested_bytes.small_pool.freed‚ 1777796206554 YGDeviceStatsMonitor.on_train_batch_end/active_bytes.small_pool.allocated‚ 1787517914112 WCDeviceStatsMonitor.on_train_batch_end/allocated_bytes.all.allocated‚138271306646016Á¸eÑ2Í- _wandbruntime‚1983  _runtime‚1983‰¸¨[ÎÕ epoch‚3 trainer/global_step‚2399 /loss/FRt2/detection_loss‚2.4126491600008153 0loss/FRt1/assignment_loss‚3.7254726755750047 0loss/FRt4/assignment_loss‚1.9349869520080334 2loss/SRqqt3/assignment_loss‚1.5491064687983858 1loss/FBt1/assignment_loss‚0.06556346034378056 _step‚194 1loss/FBt4/assignment_loss‚0.03339029310736827 1loss/SRqqt1/assignment_loss‚1.846545107025876 2loss/SRqqt2/assignment_loss‚2.0109721053693748 1loss/FBt3/assignment_loss‚0.16441820114850997 1loss/SRqqt4/detection_loss‚1.1819150959772449 .loss/FBt4/detection_loss‚0.756454613831269 0loss/FRt2/assignment_loss‚3.8934118463032283 /loss/FRt3/assignment_loss‚3.200471783056855 .loss/FRt4/detection_loss‚1.612609882934057 _runtime‚1983.295796261 2loss/SRqqt4/assignment_loss‚1.2378880408697672 0loss/SRqqt3/detection_loss‚2.002501609093613 /loss/FBt3/detection_loss‚2.3364771803220115 1loss/FBt2/assignment_loss‚0.07714241608354154 /loss/FRt3/detection_loss‚2.8385521634481847 %loss/total_loss‚44.09761579932727 -loss/FRt1/detection_loss‚2.39509813065876 1loss/SRqqt1/detection_loss‚1.6130652234085605 1loss/SRqqt2/detection_loss‚2.3219154513365514 /loss/FBt1/detection_loss‚2.0683974597765054 .loss/FBt2/detection_loss‚2.818610478849972 % _timestamp‚1.7884164016465843e+09E¨Ôs`ÏÚ /loss/FBt1/detection_loss‚2.0683974597765054 .loss/FBt2/detection_loss‚2.818610478849972 % _timestamp‚1.7884164016465843e+09 epoch‚3 trainer/global_step‚2399 /loss/FRt2/detection_loss‚2.4126491600008153 0loss/FRt1/assignment_loss‚3.7254726755750047 0loss/FRt4/assignment_loss‚1.9349869520080334 2loss/SRqqt3/assignment_loss‚1.5491064687983858 1loss/FBt1/assignment_loss‚0.06556346034378056 _step‚194 1loss/FBt4/assignment_loss‚0.03339029310736827 1loss/SRqqt1/assignment_loss‚1.846545107025876 2loss/SRqqt2/assignment_loss‚2.0109721053693748 1loss/FBt3/assignment_loss‚0.16441820114850997 1loss/SRqqt4/detection_loss‚1.1819150959772449 .loss/FBt4/detection_loss‚0.756454613831269 0loss/FRt2/assignment_loss‚3.8934118463032283 /loss/FRt3/assignment_loss‚3.200471783056855 .loss/FRt4/detection_loss‚1.612609882934057 _runtime‚1983.295796261 2loss/SRqqt4/assignment_loss‚1.2378880408697672 0loss/SRqqt3/detection_loss‚2.002501609093613 /loss/FBt3/detection_loss‚2.3364771803220115 1loss/FBt2/assignment_loss‚0.07714241608354154 /loss/FRt3/detection_loss‚2.8385521634481847 %loss/total_loss‚44.09761579932727 -loss/FRt1/detection_loss‚2.39509813065876 1loss/SRqqt1/detection_loss‚1.6130652234085605 1loss/SRqqt2/detection_loss‚2.3219154513365514Â4EuqЂ(:’ £äÔẗ© gpu.0.gpu‚0.0 gpu.0.memory‚0, gpu.0.memoryAllocated‚12.165603637695312* gpu.0.memoryAllocatedBytes‚ 10450173952 gpu.0.temp‚35.0 gpu.0.powerWatts‚119.341' gpu.0.enforcedPowerLimitWatts‚500.0) gpu.0.powerPercent‚23.868199999999998 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0 ÃFþFÑ‚(:» £äÔ¯™Âª disk.sda1.out‚0 disk.drbd2266.in‚0 disk.sda.out‚78.84375 disk.sda3.in‚0 network.recv‚744560) disk./.usagePercent‚16.89273010899285 disk.dm-0.out‚8123.982421875 disk.sda3.out‚78.84375 disk.rbd0.out‚0# disk.nvme0n1.out‚8123.857421875 disk.rbd1.out‚ 3.142578125& proc.memory.availableMB‚ 2412.03125 cpu‚26.119747216263356 disk.rbd0.in‚0 disk.sda2.in‚0 disk.sda.in‚0 disk.drbd1716.out‚0 disk.nvme0n1.in‚ 11.82421875 disk.dm-0.in‚ 11.82421875 disk.dm-1.in‚0 disk.dm-1.out‚78.84375* proc.memory.percent‚16.986799240112305 disk.sda1.in‚0 disk.sda2.out‚0 disk.drbd2265.out‚0 network.sent‚ 4.660561e+06% disk./.usageGB‚251.71388626098633 disk.drbd2266.out‚0! disk.nvme0c0n1.in‚ 11.82421875% disk.nvme0c0n1.out‚8123.857421875 disk.drbd1716.in‚0$ memory_percent‚85.27812957763672" proc.memory.rssMB‚ 2783.1171875 disk.drbd2080.in‚0 disk.drbd2080.out‚0 disk.drbd2265.in‚0 disk.rbd1.in‚0 proc.cpu.threads‚17ž\{Ò‚(:… ¬£äÔÒŠ¨ gpu.0.gpu‚30.0 gpu.0.memory‚2, gpu.0.memoryAllocated‚12.165603637695312* gpu.0.memoryAllocatedBytes‚ 10450173952 gpu.0.temp‚35.0 gpu.0.powerWatts‚139.35' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚27.87 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0’0%³SÓ‚(:È ¬£äÔ´›˜« disk.drbd2266.out‚0 disk.sda2.out‚0 network.recv‚749875 disk.rbd1.out‚ 3.142578125 disk.drbd2080.in‚0% disk./.usageGB‚251.73512268066406 disk.sda3.in‚0 disk.dm-1.out‚ 79.44921875 disk.drbd2080.out‚0" proc.memory.rssMB‚ 2783.1171875 disk.sda3.out‚ 79.44921875 disk.sda.in‚0 disk.dm-1.in‚0 disk.drbd1716.in‚0 disk.drbd2266.in‚0 disk.drbd2265.in‚0$ memory_percent‚85.27815341949463 cpu‚26.166289043544 network.sent‚ 4.747162e+06 disk.rbd0.out‚0 disk.nvme0n1.in‚ 11.82421875 disk.sda2.in‚0* disk./.usagePercent‚16.894155302935967! disk.nvme0c0n1.in‚ 11.82421875& disk.nvme0c0n1.out‚8191.0576171875 disk.dm-0.in‚ 11.82421875 disk.sda1.in‚0) proc.memory.availableMB‚ 2412.02734375* proc.memory.percent‚16.986799240112305 disk.sda.out‚ 79.44921875$ disk.nvme0n1.out‚8191.0576171875 disk.sda1.out‚0 proc.cpu.threads‚17! disk.dm-0.out‚8191.1826171875 disk.drbd1716.out‚0 disk.rbd0.in‚0 disk.drbd2265.out‚0 disk.rbd1.in‚0Ö¬“Ô‚(:ˆ »£äÔÏÜÀ¨ gpu.0.gpu‚53.0 gpu.0.memory‚3, gpu.0.memoryAllocated‚12.165603637695312* gpu.0.memoryAllocatedBytes‚ 10450173952 gpu.0.temp‚35.0 gpu.0.powerWatts‚182.151' gpu.0.enforcedPowerLimitWatts‚500.0 gpu.0.powerPercent‚36.4302 gpu.0.smClock‚1410 gpu.0.memoryClock‚1593! gpu.0.correctedMemoryErrors‚0# gpu.0.uncorrectedMemoryErrors‚0Y›»Õ‚(:Å »£äÔù羪& disk.nvme0c0n1.out‚8231.8974609375 disk.sda1.out‚0 disk.sda2.in‚0$ memory_percent‚85.27817726135254* proc.memory.percent‚16.986799240112305 disk.sda.in‚0 disk.dm-1.out‚ 80.2265625 disk.rbd0.out‚0 disk.sda.out‚ 80.2265625 disk.dm-1.in‚0 proc.cpu.threads‚17 disk.drbd2080.out‚0 disk.drbd2265.in‚0 disk.drbd2265.out‚0 network.sent‚ 4.753626e+06 ne´úç•twork.recv‚754775) disk./.usagePercent‚16.89339163246133 disk.rbd1.out‚ 3.142578125 disk.sda1.in‚0" proc.memory.rssMB‚ 2783.1171875$ disk.nvme0n1.out‚8231.8974609375 disk.rbd1.in‚0! disk.dm-0.out‚8232.0224609375 disk.sda3.out‚ 80.2265625 disk.drbd1716.out‚0 disk.nvme0n1.in‚ 11.82421875$ disk./.usageGB‚251.7237434387207! disk.nvme0c0n1.in‚ 11.82421875 disk.drbd2080.in‚0 disk.rbd0.in‚0 disk.sda2.out‚0 disk.drbd2266.in‚0( proc.memory.availableMB‚ 2412.0234375 disk.dm-0.in‚ 11.82421875 disk.sda3.in‚0 disk.drbd1716.in‚0 disk.drbd2266.out‚0 cpu‚26.215723597024848Íá–þ2Ö- _wandbruntime‚2031  _runtime‚2031Ÿðý$}×µ¦ jet/accuracy_5_of_7‚0 ("Purity/1FRt1SRqqt4FBt/SRqqt_purity‚1 & Purity/*FRt2SRqqt3FBt/FBt_purity‚1 ,&Purity/0FRt2SRqqt4FBt/event_proportion‚0 :"Purity/1FRt0SRqqt*FBt/event_purity‚0.15309707820415497 ?&Purity/2FRt3SRqqt0FBt/event_proportion‚0.003654233878478408 particle/accuracy_4_of_5‚1 & Purity/0FRt4SRqqt*FBt/FBt_purity‚1 ("Purity/3FRt2SRqqt1FBt/event_purity‚0 ("Purity/1FRt*SRqqt4FBt/event_purity‚0 :"Purity/*FRt0SRqqt2FBt/event_purity‚0.26003119349479675 ,&Purity/1FRt2SRqqt4FBt/event_proportion‚0 & Purity/2FRt0SRqqt1FBt/FBt_purity‚1 9"Purity/2FRt1SRqqt*FBt/SRqqt_purity‚0.3077162802219391 7 Purity/*FRt0SRqqt2FBt/FBt_purity‚0.9910675287246704 ?&Purity/0FRt1SRqqt2FBt/event_proportion‚0.008757560513913631 * Purity/1FRt*SRqqt3FBt/FRt_purity‚0.375 0"Purity/*FRt4SRqqt*FBt/SRqqt_purity‚ 0.2578125 ("Purity/0FRt4SRqqt*FBt/event_purity‚0 & Purity/3FRt*SRqqt2FBt/FBt_purity‚1 ("Purity/2FRt0SRqqt3FBt/event_purity‚0 & Purity/*FRt3SRqqt2FBt/FRt_purity‚1 6 Purity/*FRt*SRqqt2FBt/FBt_purity‚0.988559901714325 8 Purity/2FRt3SRqqt0FBt/FRt_purity‚0.23133116960525513 7 Purity/3FRt4SRqqt*FBt/FRt_purity‚0.1666666716337204 ?&Purity/*FRt0SRqqt3FBt/event_proportion‚0.003717237850651145 9"Purity/*FRt2SRqqt0FBt/SRqqt_purity‚0.2991968095302582 ;"Purity/*FRt2SRqqt1FBt/event_purity‚0.017076784744858742 ,&Purity/2FRt4SRqqt2FBt/event_proportion‚0 ("Purity/*FRt4SRqqt1FBt/SRqqt_purity‚0 ("Purity/2FRt*SRqqt3FBt/event_purity‚0 ("Purity/1FRt*SRqqt3FBt/SRqqt_purity‚1 ( Purity/1FRt1SRqqt3FBt/FRt_purity‚0.5 :"Purity/0FRt1SRqqt0FBt/SRqqt_purity‚0.34259894490242004 :"Purity/3FRt*SRqqt0FBt/SRqqt_purity‚0.27473074197769165 particle/accuracy_1_of_4‚1 *jet/accuracy_1_of_6‚0.9473684430122375 ) Purity/*FRt1SRqqt4FBt/FBt_purity‚0.75 ,&Purity/0FRt4SRqqt3FBt/event_proportion‚0 ,&Purity/0FRt4SRqqt4FBt/event_proportion‚0 7 Purity/1FRt*SRqqt*FBt/FBt_purity‚0.9929631352424622 7 Purity/2FRt*SRqqt*FBt/FRt_purity‚0.2111542522907257 ,&Purity/3FRt1SRqqt3FBt/event_proportion‚0 & Purity/1FRt3SRqqt2FBt/FBt_purity‚1 9"Purity/*FRt2SRqqt1FBt/SRqqt_purity‚0.3611535429954529 8"Purity/0FRt*SRqqt2FBt/event_purity‚0.265972763299942 & Purity/1FRt1SRqqt1FBt/FBt_purity‚1 ,&Purity/1FRt4SRqqt4FBt/event_proportion‚0 ("Purity/*FRt4SRqqt1FBt/event_purity‚0 9"Purity/*FRt1SRqqt*FBt/event_purity‚0.1541629135608673 8 Purity/*FRt1SRqqt*FBt/FRt_purity‚0.21831995248794556 9"Purity/0FRt2SRqqt*FBt/event_purity‚0.0284329392015934 7 Purity/1FRt1SRqqt*FBt/FBt_purity‚0.9906545877456665 ("Purity/2FRt2SRqqt0FBt/event_purity‚0 ("Purity/3FRt0SRqqt*FBt/event_purity‚0 ,&Purity/3FRt3SRqqt1FBt/event_proportion‚0 +jet/accuracy_2_of_3‚0.21289102733135223 @&Purity/0FRt2SRqqt3FBt/event_proportion‚6.300403038039804e-05 7 Purity/1FRt*SRqqt1FBt/FBt_purity‚0.9985867738723755 9"Purity/1FRt1SRqqt*FBt/SRqqt_purity‚0.3392636477947235 >&Purity/1FRt1SRqqt1FBt/event_proportion‚0.03944052383303642 ,&Purity/2FRt*SRqqt4FBt/event_proportion‚0 6validation_average_jet_accuracy‚0.3725806176662445 7 Purity/3FRt2SRqqt1FBt/FRt_purity‚0.2222222238779068 :"Purity/*FRt3SRqqt*FBt/SRqqt_purity‚0.29253751039505005 ,&Purity/0FRt3SRqqt2FBt/event_proportion‚0 ("Purity/2FRt1SRqqt2FBt/event_purity‚0 ,&Purity/4FRt3SRqqt1FBt/event_proportion‚0 particle/accuracy_1_of_5‚1 ( Purity/2FRt0SRqqt3FBt/FRt_purity‚0.5 ,&Purity/0FRt3SRqqt3FBt/event_proportion‚0 9"Purity/2FRt3SRqqt0FBt/SRqqt_purity‚0.2328643798828125 A&Purity/3FRt1SRqqt2FBt/event_proportion‚0.00025201612152159214 ,&Purity/4FRt0SRqqt2FBt/event_proportion‚0 ,&Purity/4FRt1SRqqt4FBt/event_proportion‚0 & Purity/*FRt4SRqqt1FBt/FBt_purity‚1 9"Purity/3FRt2SRqqt1FBt/SRqqt_purity‚0.3333333432674408 & Purity/1FRt0SRqqt4FBt/FRt_purity‚0 ,&Purity/0FRt4SRqqt2FBt/event_proportion‚0 ("Purity/1FRt3SRqqt0FBt/event_purity‚0 :"Purity/3FRt2SRqqt0FBt/SRqqt_purity‚0.32608696818351746 *jet/accuracy_1_of_2‚0.6576357483863831 & Purity/*FRt4SRqqt*FBt/FBt_purity‚1 8 Purity/*FRt2SRqqt2FBt/FRt_purity‚0.24358974397182465 ("Purity/*FRt*SRqqt4FBt/SRqqt_purity‚1 :"Purity/1FRt2SRqqt0FBt/SRqqt_purity‚0.29128792881965637 7 Purity/2FRt2SRqqt1FBt/FRt_purity‚0.2886905074119568 ,&Purity/4FRt3SRqqt2FBt/event_proportion‚0 *jet/accuracy_2_of_4‚0.3037717640399933 +"Purity/2FRt4SRqqt*FBt/SRqqt_purity‚0.25 +"Purity/3FRt*SRqqt2FBt/SRqqt_purity‚0.25 7 Purity/1FRt4SRqqt*FBt/FRt_purity‚0.1666666716337204 ("Purity/4FRt1SRqqt0FBt/SRqqt_purity‚0 9"Purity/*FRt1SRqqt*FBt/SRqqt_purity‚0.3494856059551239 7 Purity/0FRt1SRqqt2FBt/FBt_purity‚0.9798147678375244 9"Purity/0FRt2SRqqt*FBt/SRqqt_purity‚0.3223111629486084 +"Purity/2FRt4SRqqt0FBt/SRqqt_purity‚0.25 ("Purity/4FRt2SRqqt0FBt/SRqqt_purity‚0 7 Purity/*FRt3SRqqt1FBt/FRt_purity‚0.3444444239139557 ?&Purity/0FRt3SRqqt*FBt/event_proportion‚0.006741431541740894 ,&Purity/0FRt3SRqqt4FBt/event_proportion‚0 ;"Purity/1FRt1SRqqt*FBt/event_purity‚0.014884690754115582 ,&Purity/4FRt0SRqqt1FBt/event_proportion‚0 9"Purity/*FRt*SRqqt0FBt/event_purity‚0.1259285807609558 @&Purity/3FRt1SRqqt0FBt/event_proportion‚0.0025201612152159214 :"Purity/3FRt1SRqqt0FBt/SRqqt_purity‚0.36666664481163025 :"Purity/0FRt1SRqqt1FBt/event_purity‚0.15956179797649384 A&Purity/3FRt4SRqqt*FBt/event_proportion‚0.00012600806076079607 ,&Purity/4FRt2SRqqt1FBt/event_proportion‚0 * Purity/0FRt0SRqqt4FBt/FBt_purity‚0.875 @&Purity/*FRt1SRqqt3FBt/event_proportion‚0.0006930443341843784 7 Purity/1FRt2SRqqt*FBt/FBt_purity‚0.9890029430389404 ?&Purity/2FRt2SRqqt0FBt/event_proportion‚0.015435988083481789 :"Purity/0FRt2SRqqt2FBt/SRqqt_purity‚0.26923078298568726 ("Purity/1FRt0SRqqt4FBt/event_purity‚0 7 Purity/0FRt*SRqqt1FBt/FBt_purity‚0.9991740584373474 ?&Purity/0FRt3SRqqt0FBt/event_proportion‚0.006111391354352236 9"Purity/1FRt0SRqqt0FBt/event_purity‚0.1285238265991211 9"Purity/2FRt*SRqqt1FBt/SRqqt_purity‚0.3291139602661133 ?&Purity/2FRt0SRqqt1FBt/event_proportion‚0.008946572430431843 ,&Purity/2FRt1SRqqt4FBt/event_proportion‚0 @&Purity/3FRt*SRqqt1FBt/event_proportion‚0.0024571572430431843 ,&Purity/4FRt*SRqqt1FBt/event_proportion‚0 ("Purity/*FRt3SRqqt*FBt/event_purity‚0 7 Purity/0FRt*SRqqt*FBt/FBt_purity‚0.9921687841415405 :"Purity/0FRt0SRqqt3FBt/event_purity‚0.14000000059604645 ;"Purity/2FRt0SRqqt*FBt/event_purity‚0.018544865772128105 ,&Purity/4FRt0SRqqt3FBt/event_proportion‚0 particle/accuracy_4_of_4‚1 ("Purity/2FRt4SRqqt*FBt/event_purity‚0 particle/accuracy_4_of_6‚1 7 Purity/1FRt0SRqqt2FBt/FBt_purity‚0.9907549619674683 7 Purity/1FRt2SRqqt1FBt/FBt_purity‚0.9873272180557251 8 Purity/2FRt3SRqqt*FBt/FRt_purity‚0.21947617828845978 particle/accuracy_6_of_6‚1 7 Purity/3FRt4SRqqt0FBt/FRt_purity‚0.1666666716337204 8 Purity/*FRt1SRqqt0FBt/FRt_purity‚0.19284585118293762 7 Purity/2FRt1SRqqt*FBt/FBt_purity‚0.9964157938957214 ,&Purity/4FRt1SRqqt2FBt/event_proportion‚0 particle/accuracy_2_of_7‚1 % _timestamp‚1.7884164498830664e+09 =&Purity/*FRt0SRqqt2FBt/event_proportion‚0.0483240932226181 >&Purity/*FRt1SRqqt0FBt/event_proportion‚0.33039313554763794 >&Purity/*FRt3SRqqt0FBt/event_proportion‚0.01827116869390011 ("Purity/2FRt1SRqqt*FBt/event_purity‚0 9"Purity/2FRt1SRqqt2FBt/SRqqt_purity‚0.6359648704528809 ,&Purity/3FRt3SRqqt2FBt/event_proportion‚0 ("Purity/1FRt1SRqqt3FBt/SRqqt_purity‚1 @&Purity/*FRt0SRqqt4FBt/event_proportion‚0.0003150201519019902 8 Purity/2FRt*SRqqt0FBt/FRt_purity‚0.17074377834796906 ;"Purity/2FRt*SRqqt1FBt/event_purity‚0.026147358119487762 ,&Purity/2FRt4SRqqt1FBt/event_proportion‚0 ,&Purity/2FRt4SRqqt4FBt/event_proportion‚0 ("Purity/3FRt*SRqqt2FBt/event_purity‚0 & Purity/3FRt1SRqqt2FBt/FBt_purity‚1 ("Purity/1FRt4SRqqt0FBt/event_purity‚0 8 Purity/*FRt2SRqqt1FBt/FRt_purity‚0.23963364958763123 >&Purity/0FRt*SRqqt2FBt/event_proportion‚0.03944052383303642 :"Purity/0FRt*SRqqt3FBt/event_purity‚0.08641975373029709 A&Purity/0FRt*SRqqt4FBt/event_proportion‚0.00025201612152159214 & Purity/2FRt1SRqqt1FBt/FBt_purity‚1 @&Purity/4FRt2SRqqt*FBt/event_proportion‚6.300403038039804e-05 %particle/accuracy‚0.8438760638237 & Purity/3FRt0SRqqt2FBt/FBt_purity‚1 9"Purity/0FRt*SRqqt2FBt/SRqqt_purity‚0.4311111271381378 9"Purity/0FRt1SRqqt1FBt/SRqqt_purity‚0.4216614067554474 ?&Purity/1FRt0SRqqt1FBt/event_proportion‚0.049773186445236206 +jet/accuracy_3_of_6‚0.17543859779834747 & Purity/0FRt2SRqqt2FBt/FBt_purity‚1 ("Purity/0FRt1SRqqt3FBt/event_purity‚0 _runtime‚2031.534288604 particle/accuracy_2_of_2‚1 particle/accuracy_4_of_7‚1 7 Purity/1FRt4SRqqt0FBt/FRt_purity‚0.1666666716337204 ( Purity/2FRt2SRqqt2FBt/FRt_purity‚0.5 ("Purity/*FRt2SRqqt3FBt/SRqqt_purity‚0 9"Purity/1FRt*SRqqt0FBt/SRqqt_purity‚0.2929992079734802 ;"Purity/1FRt1SRqqt0FBt/event_purity‚0.010006660595536232 ("Purity/*FRt4SRqqt*FBt/event_purity‚0 ."Purity/0FRt4SRqqt*FBt/SRqqt_purity‚0.21875 & Purity/*FRt0SRqqt4FBt/FRt_purity‚0 ("Purity/3FRt0SRqqt1FBt/event_purity‚0 & Purity/*FRt*SRqqt4FBt/FRt_purity‚0 =&Purity/*FRt*SRqqt0FBt/event_proportion‚0.6160534024238586 7 Purity/*FRt0SRqqt2FBt/FRt_purity‚0.4407619833946228 9"Purity/0FRt*SRqqt0FBt/event_purity‚0.2699776589870453 :"Purity/1FRt2SRqqt1FBt/SRqqt_purity‚0.36363285779953003 ("Purity/0FRt4SRqqt0FBt/event_purity‚0 7 Purity/3FRt*SRqqt2FBt/FRt_purity‚0.3333333432674408 * Purity/1FRt*SRqqt4FBt/FBt_purity‚0.875 & Purity/*FRt3SRqqt2FBt/FBt_purity‚1 >&Purity/*FRt*SRqqt2FBt/event_proportion‚0.06747731566429138 7 Purity/*FRt*SRqqt2FBt/FRt_purity‚0.3882356286048889 @&Purity/0FRt*SRqqt3FBt/event_proportion‚0.0030871976632624865 ("Purity/0FRt3SRqqt1FBt/event_purity‚0 ( Purity/2FRt*SRqqt3FBt/FRt_purity‚0.5 9"Purity/0FRt*SRqqt3FBt/SRqqt_purity‚0.6666666269302368 ("Purity/2FRt2SRqqt2FBt/event_purity‚0 ("Purity/4FRt0SRqqt0FBt/event_purity‚0 *loss/val_total_loss‚45.136220755226944 :"Purity/0FRt3SRqqt0FBt/SRqqt_purity‚0.29206347465515137 ("Purity/1FRt2SRqqt2FBt/event_purity‚0 ) Purity/1FRt1SRqqt4FBt/FBt_purity‚0.75 =&Purity/*FRt1SRqqt*FBt/event_proportion‚0.4482736885547638 =&Purity/1FRt*SRqqt*FBt/event_proportion‚0.4366179406642914 8 Purity/1FRt2SRqqt0FBt/FRt_purity‚0.20098847150802612 @&Purity/1FRt4SRqqt*FBt/event_proportion‚0.0003780241822823882 @&Purity/3FRt2SRqqt0FBt/event_proportion‚0.0024571572430431843 +"Purity/3FRt4SRqqt0FBt/SRqqt_purity‚0.25 ( Purity/*FRt1SRqqt3FBt/FRt_purity‚0.5 ("Purity/1FRt1SRqqt3FBt/event_purity‚0 particle/accuracy_3_of_5‚1 particle/accuracy_7_of_7‚0 9"Purity/0FRt*SRqqt*FBt/event_purity‚0.4011649191379547 @&Purity/1FRt0SRqqt3FBt/event_proportion‚0.0010710685746744275 >&Purity/2FRt*SRqqt*FBt/event_proportion‚0.10244455933570862 8 Purity/2FRt0SRqqt*FBt/FRt_purity‚0.20549677312374115 ?&Purity/2FRt0SRqqt2FBt/event_proportion‚0.003465221729129553 ,&Purity/3FRt2SRqqt2FBt/event_proportion‚0 ,&Purity/4FRt1SRqqt1FBt/event_proportion‚0 +jet/accuracy_2_of_2‚0.09117655456066132 7 Purity/1FRt0SRqqt1FBt/FRt_purity‚0.2979013919830322 *jet/accuracy_1_of_4‚0.8387826681137085 8"Purity/0FRt1SRqqt3FBt/SRqqt_purity‚0.800000011920929 6 Purity/3FRt3SRqqt*FBt/FRt_purity‚0.190476194024086 ,&Purity/0FRt1SRqqt4FBt/event_proportion‚0 ("Purity/2FRt1SRqqt0FBt/event_purity‚0 ,&Purity/2FRt3SRqqt2FBt/event_proportion‚0 @&Purity/3FRt0SRqqt0FBt/event_proportion‚0.0015120967291295528 ("Purity/4FRt1SRqqt*FBt/event_purity‚0 ("Purity/*FRt2SRqqt3FBt/event_purity‚0 ,&Purity/*FRt4SRqqt4FBt/event_proportion‚0 @&Purity/0FRt2SRqqt2FBt/event_proportion‚0.0010080644860863686 :"Purity/1FRt3SRqqt*FBt/SRqqt_purity‚0.30266255140304565 ,&Purity/2FRt1SRqqt3FBt/event_proportion‚0 *jet/accuracy_1_of_5‚0.9055039286613464 particle/accuracy_2_of_3‚1 ) Purity/4FRt1SRqqt*FBt/FRt_purity‚0.25 9"Purity/0FRt1SRqqt2FBt/SRqqt_purity‚0.4734920561313629 ,&Purity/1FRt4SRqqt2FBt/event_proportion‚0 ("Purity/3FRt0SRqqt2FBt/event_purity‚0 >&Purity/1FRt2SRqqt0FBt/event_proportion‚0.04731602966785431 7 Purity/3FRt0SRqqt1FBt/FRt_purity‚0.3205128014087677 ,&Purity/4FRt3SRqqt4FBt/event_proportion‚0 jet/accuracy_6_of_6‚0 & Purity/*FRt2SRqqt2FBt/FBt_purity‚1 :"Purity/1FRt*SRqqt2FBt/event_purity‚0.09017863124608994 7 Purity/3FRt1SRqqt1FBt/FRt_purity‚0.3205128014087677 ,&Purity/4FRt1SRqqt3FBt/event_proportion‚0 ,&Purity/4FRt4SRqqt3FBt/event_proportion‚0 7 Purity/*FRt4SRqqt*FBt/FRt_purity‚0.1666666716337204 9"Purity/*FRt*SRqqt*FBt/event_purity‚0.2064225971698761 7 Purity/*FRt1SRqqt1FBt/FBt_purity‚0.9994532465934753 9"Purity/3FRt2SRqqt*FBt/SRqqt_purity‚0.3402777910232544 @&Purity/4FRt2SRqqt0FBt/event_proportion‚6.300403038039804e-05 9"Purity/*FRt3SRqqt2FBt/SRqqt_purity‚0.3333333432674408 & Purity/1FRt3SRqqt2FBt/FRt_purity‚1 trainer/global_step‚2415 8 Purity/1FRt*SRqqt*FBt/FRt_purity‚0.20915307104587555 7 Purity/2FRt*SRqqt*FBt/FBt_purity‚0.9961161017417908 ?&Purity/2FRt2SRqqt*FBt/event_proportion‚0.019468246027827263 ("Purity/3FRt3SRqqt0FBt/event_purity‚0 7 Purity/*FRt*SRqqt3FBt/FBt_purity‚0.9204980731010437 <"Purity/1FRt2SRqqt*FBt/event_purity‚0.0011123470030725002 & Purity/2FRt*SRqqt1FBt/FBt_purity‚1 9"Purity/0FRt3SRqqt1FBt/SRqqt_purity‚0.4583333134651184 8 Purity/2FRt1SRqqt0FBt/FRt_purity‚0.17747142910957336 6 Purity/*FRt0SRqqt3FBt/FBt_purity‚0.929563581943512 7 Purity/*FRt1SRqqt*FBt/FBt_purity‚0.9920945763587952 >&Purity/0FRt2SRqqt*FBt/event_proportion‚0.05361643061041832 7 Purity/1FRt1SRqqt*FBt/FRt_purity‚0.2220952957868576 @&Purity/1FRt1SRqqt4FBt/event_proportion‚6.300403038039804e-05 ;"Purity/1FRt2SRqqt1FBt/event_purity‚0.006451612804085016 8"Purity/0FRt3SRqqt*FBt/SRqqt_purity‚0.299263060092926 7 Purity/*FRt1SRqqt1FBt/FRt_purity‚0.2862626314163208 particle/accuracy_3_of_6‚1 ("Purity/4FRt*SRqqt0FBt/event_purity‚0 ("Purity/4FRt2SRqqt*FBt/SRqqt_purity‚0 & Purity/0FRt2SRqqt1FBt/FBt_purity‚1 :"Purity/2FRt3SRqqt*FBt/SRqqt_purity‚0.23025399446487427 8 Purity/3FRt2SRqqt*FBt/FRt_purity‚0.17129629850387573 _step‚195 :"Purity/*FRt1SRqqt0FBt/event_purity‚0.17966404557228088 9"Purity/*FRt2SRqqt*FBt/SRqqt_purity‚0.3111834228038788 & Purity/0FRt2SRqqt*FBt/FBt_purity‚1 :"Purity/1FRt*SRqqt*FBt/SRqqt_purity‚0.32543447613716125 6 Purity/2FRt*SRqqt1FBt/FRt_purity‚0.324860543012619 ,&Purity/3FRt*SRqqt3FBt/event_proportion‚0 @&Purity/4FRt1SRqqt0FBt/event_proportion‚6.300403038039804e-05 8 Purity/*FRt2SRqqt0FBt/FRt_purity‚0.19230587780475616 ("Purity/0FRt0SRqqt0FBt/event_purity‚1 ;"Purity/0FRt2SRqqt0FBt/event_purity‚0.027675585821270943 =&Purity/1FRt*SRqqt0FBt/event_proportion‚0.3135710656642914 :"Purity/2FRt0SRqqt2FBt/event_purity‚0.02500000037252903 ,jet/accuracy_4_of_4‚0.004436581861227751 particle/accuracy_1_of_1‚1 ("Purity/0FRt2SRqqt3FBt/SRqqt_purity‚0 @&Purity/*FRt3SRqqt2FBt/event_proportion‚6.300403038039804e-05 ?&Purity/0FRt0SRqqt2FBt/event_proportion‚0.029674898833036423 ?&Purity/3FRt1SRqqt*FBt/event_proportion‚0.003717237850651145 & Purity/3FRt1SRqqt*FBt/FBt_purity‚1 ,&Purity/3FRt4SRqqt4FBt/event_proportion‚0 & Purity/4FRt0SRqqt0FBt/FRt_purity‚0 7 Purity/1FRt0SRqqt3FBt/FRt_purity‚0.3928571343421936 8 Purity/2FRt2SRqqt*FBt/FRt_purity‚0.20954884588718414 & Purity/3FRt0SRqqt*FBt/FBt_purity‚1 ,&Purity/3FRt4SRqqt3FBt/event_proportion‚0 @&Purity/*FRt4SRqqt1FBt/event_proportion‚6.300403038039804e-05 7 Purity/2FRt1SRqqt2FBt/FBt_purity‚0.9868420958518982 & Purity/3FRt0SRqqt1FBt/FBt_purity‚1 ,&Purity/3FRt3SRqqt4FBt/event_proportion‚0 8 Purity/2FRt4SRqqt0FBt/FRt_purity‚0.30000001192092896 9"Purity/1FRt3SRqqt2FBt/SRqqt_purity‚0.3333333432674408 ?&Purity/1FRt3SRqqt*FBt/event_proportion‚0.009135584346950054 & Purity/3FRt1SRqqt1FBt/FBt_purity‚1 ,&Purity/3FRt2SRqqt3FBt/event_proportion‚0 0"Purity/*FRt4SRqqt0FBt/SRqqt_purity‚ 0.2734375 particle/accuracy_5_of_6‚1 7 Purity/0FRt*SRqqt2FBt/FBt_purity‚0.9887910485267639 @&Purity/1FRt3SRqqt2FBt/event_proportion‚6.300403038039804e-05 ,&Purity/1FRt3SRqqt4FBt/event_proportion‚0 ("Purity/*FRt2SRqqt2FBt/event_purity‚0 & Purity/4FRt2SRqqt*FBt/FRt_purity‚0 8 Purity/*FRt3SRqqt0FBt/FRt_purity‚0.21434710919857025 9"Purity/*FRt3SRqqt0FBt/SRqqt_purity‚0.2823820114135742 7 Purity/1FRt0SRqqt*FBt/FBt_purity‚0.9950029253959656 @&Purity/3FRt2SRqqt*FBt/event_proportion‚0.0028351815417408943 jet/accuracy_6_of_7‚0 ("Purity/1FRt4SRqqt*FBt/event_purity‚0 & Purity/1FRt0SRqqt1FBt/FBt_purity‚1 ;"Purity/2FRt*SRqqt0FBt/event_purity‚0.001638504909351468 @&Purity/2FRt4SRqqt0FBt/event_proportion‚0.0003150201519019902 & Purity/1FRt2SRqqt2FBt/FBt_purity‚1 @&Purity/1FRt4SRqqt0FBt/event_proportion‚0.0003780241822823882 ,&Purity/4FRt2SRqqt2FBt/event_proportion‚0 ("Purity/2FRt2SRqqt*FBt/event_purity‚0 particle/accuracy_1_of_6‚1 & Purity/1FRt*SRqqt4FBt/FRt_purity‚0 9"Purity/*FRt0SRqqt1FBt/event_purity‚0.6694325804710388 7 Purity/0FRt1SRqqt1FBt/FBt_purity‚0.9989919066429138 ?&Purity/0FRt2SRqqt0FBt/event_proportion‚0.043283771723508835 ;"Purity/2FRt0SRqqt0FBt/event_purity‚0.007540846709161997 @&Purity/3FRt1SRqqt1FBt/event_proportion‚0.0009450604557059705 ("Purity/4FRt0SRqqt*FBt/event_purity‚0 ?&Purity/*FRt*SRqqt3FBt/event_proportion‚0.004473286215215921 =&Purity/*FRt2SRqqt*FBt/event_proportion‚0.1332535296678543 7 Purity/0FRt0SRqqt3FBt/FBt_purity‚0.9311111569404602 7 Purity/0FRt1SRqqt*FBt/FBt_purity‚0.9924134612083435 ,&Purity/4FRt2SRqqt3FBt/event_proportion‚0 ,&Purity/4FRt4SRqqt*FBt/event_proportion‚0 particle/accuracy_2_of_4‚1 & Purity/1FRt1SRqqt4FBt/FRt_purity‚0 @&Purity/*FRt2SRqqt3FBt/event_proportion‚6.300403038039804e-05 9"Purity/1FRt0SRqqt1FBt/event_purity‚0.2217732071876526 ,&Purity/1FRt4SRqqt3FBt/event_proportion‚0 @&Purity/3FRt0SRqqt1FBt/event_proportion‚0.0011340725468471646 7"Purity/*FRt*SRqqt3FBt/SRqqt_purity‚0.80952388048172 9"Purity/3FRt3SRqqt*FBt/SRqqt_purity‚0.4047618806362152 epoch‚3 ,&Purity/*FRt2SRqqt4FBt/event_proportion‚0 :"Purity/0FRt1SRqqt0FBt/event_purity‚0.34259894490242004 8 Purity/1FRt1SRqqt2FBt/FRt_purity‚0.33494895696640015 >&Purity/2FRt*SRqqt0FBt/event_proportion‚0.07579384744167328 ("Purity/3FRt*SRqqt0FBt/event_purity‚0 7 Purity/*FRt1SRqqt2FBt/FBt_purity‚0.9808046221733093 ;"Purity/*FRt2SRqqt0FBt/event_purity‚0.011335928924381733 9"Purity/2FRt*SRqqt2FBt/SRqqt_purity‚0.6184210777282715 ,&Purity/2FRt0SRqqt4FBt/event_proportion‚0 ,&Purity/4FRt3SRqqt*FBt/event_proportion‚0 ,&Purity/4FRt4SRqqt4FBt/event_proportion‚0 ("Purity/*FRt0SRqqt4FBt/event_purity‚0 :"Purity/*FRt1SRqqt1FBt/event_purity‚0.09356243908405304 :"Purity/*FRt1SRqqt2FBt/event_purity‚0.02013685181736946 >&Purity/0FRt*SRqqt0FBt/event_proportion‚0.21931703388690948 8 Purity/1FRt*SRqqt2FBt/FRt_purity‚0.39816588163375854 8 Purity/3FRt0SRqqt0FBt/FRt_purity‚0.05882352963089943 7 Purity/*FRt0SRqqt4FBt/FBt_purity‚0.8999999761581421 :"Purity/*FRt*SRqqt2FBt/SRqqt_purity‚0.45156362652778625 :"Purity/*FRt0SRqqt*FBt/event_purity‚0.35795295238494873 :"Purity/0FRt*SRqqt1FBt/SRqqt_purity‚0.40581440925598145 8 Purity/1FRt3SRqqt*FBt/FRt_purity‚0.19779826700687408 9"Purity/2FRt*SRqqt*FBt/SRqqt_purity‚0.2912462055683136 7 Purity/2FRt*SRqqt2FBt/FBt_purity‚0.9911110997200012 8 Purity/3FRt1SRqqt0FBt/FRt_purity‚0.17083334922790527 * Purity/*FRt*SRqqt4FBt/FBt_purity‚0.875 7 Purity/*FRt2SRqqt*FBt/FBt_purity‚0.9954545497894287 =&Purity/1FRt1SRqqt0FBt/event_proportion‚0.1265120953321457 ?&Purity/2FRt0SRqqt0FBt/event_proportion‚0.024571573361754417 ("Purity/2FRt1SRqqt1FBt/event_purity‚0 *jet/accuracy_1_of_1‚0.4023129642009735 particle/accuracy_5_of_5‚1 particle/accuracy_2_of_6‚1 ("Purity/4FRt*SRqqt*FBt/SRqqt_purity‚0 :"Purity/*FRt*SRqqt3FBt/event_purity‚0.06034482643008232 @&Purity/*FRt4SRqqt0FBt/event_proportion‚0.0012600806076079607 @&Purity/1FRt3SRqqt1FBt/event_proportion‚0.0011340725468471646 & Purity/2FRt0SRqqt3FBt/FBt_purity‚1 :"Purity/0FRt1SRqqt*FBt/SRqqt_purity‚0.36538684368133545 ("Purity/3FRt*SRqqt*FBt/event_purity‚0 ("Purity/0FRt4SRqqt1FBt/SRqqt_purity‚0 ("Purity/3FRt1SRqqt2FBt/event_purity‚0 7 Purity/*FRt2SRqqt1FBt/FBt_purity‚0.9950512647628784 A&Purity/1FRt*SRqqt4FBt/event_proportion‚0.00012600806076079607 >&Purity/1FRt0SRqqt0FBt/event_proportion‚0.13142640888690948 8 Purity/2FRt1SRqqt1FBt/FRt_purity‚0.31724271178245544 :"Purity/3FRt1SRqqt*FBt/SRqqt_purity‚0.35185185074806213 ?&Purity/1FRt1SRqqt2FBt/event_proportion‚0.006111391354352236 ("Purity/1FRt2SRqqt0FBt/event_purity‚0 ?&Purity/1FRt2SRqqt1FBt/event_proportion‚0.008946572430431843 ("Purity/4FRt1SRqqt0FBt/event_purity‚0 ("Purity/4FRt2SRqqt0FBt/event_purity‚0 9"Purity/1FRt3SRqqt0FBt/SRqqt_purity‚0.2896057367324829 ("Purity/3FRt2SRqqt0FBt/event_purity‚0 +jet/accuracy_3_of_3‚0.02158878929913044 ("Purity/0FRt4SRqqt1FBt/event_purity‚0 ("Purity/0FRt2SRqqt3FBt/event_purity‚0 ,&Purity/*FRt3SRqqt4FBt/event_proportion‚0 A&Purity/1FRt1SRqqt3FBt/event_proportion‚0.00025201612152159214 7 Purity/2FRt0SRqqt2FBt/FRt_purity‚0.3958333432674408 ("Purity/2FRt3SRqqt0FBt/event_purity‚0 7 Purity/1FRt2SRqqt1FBt/FRt_purity‚0.2079876959323883 9"Purity/2FRt1SRqqt1FBt/SRqqt_purity‚0.3324885070323944 jet/accuracy_5_of_5‚0 particle/accuracy_5_of_7‚1 ?&Purity/*FRt3SRqqt*FBt/event_proportion‚0.020476309582591057 9"Purity/0FRt*SRqqt1FBt/event_purity‚0.6393033862113953 @&Purity/0FRt4SRqqt*FBt/event_proportion‚0.0005040322430431843 @&Purity/2FRt0SRqqt3FBt/event_proportion‚6.300403038039804e-05 @&Purity/3FRt3SRqqt*FBt/event_proportion‚0.0005670362734235823 particle/accuracy_3_of_3‚1 +"Purity/3FRt4SRqqt*FBt/SRqqt_purity‚0.25 :"Purity/*FRt1SRqqt1FBt/SRqqt_purity‚0.43543773889541626 & Purity/*FRt3SRqqt1FBt/FBt_purity‚1 >&Purity/0FRt*SRqqt*FBt/event_proportion‚0.40996724367141724 A&Purity/0FRt0SRqqt4FBt/event_proportion‚0.00025201612152159214 + Purity/1FRt*SRqqt3FBt/FBt_purity‚0.9375 9"Purity/2FRt2SRqqt0FBt/SRqqt_purity‚0.2854035198688507 ?&Purity/3FRt*SRqqt*FBt/event_proportion‚0.009954636916518211 (particle/f_score‚0.34501731395721436 8 Purity/*FRt*SRqqt*FBt/FRt_purity‚0.20939284563064575 & Purity/*FRt3SRqqt*FBt/FBt_purity‚1 7 Purity/1FRt0SRqqt0FBt/FRt_purity‚0.1285238265991211 ?&Purity/1FRt3SRqqt0FBt/event_proportion‚0.007938507944345474 * Purity/4FRt*SRqqt0FBt/FRt_purity‚0.125 8 Purity/*FRt0SRqqt*FBt/FRt_purity‚0.20200170576572418 <"Purity/1FRt1SRqqt2FBt/event_purity‚0.0071428571827709675 7 Purity/3FRt*SRqqt*FBt/FRt_purity‚0.1956605464220047 ,&Purity/3FRt0SRqqt3FBt/event_proportion‚0 particle/accuracy_1_of_2‚1 particle/accuracy_6_of_7‚1 7 Purity/1FRt1SRqqt3FBt/FBt_purity‚0.8333333730697632 @&Purity/*FRt*SRqqt4FBt/event_proportion‚0.0003780241822823882 & Purity/2FRt2SRqqt*FBt/FBt_purity‚1 8 Purity/3FRt0SRqqt*FBt/FRt_purity‚0.18434342741966248 @&Purity/3FRt0SRqqt2FBt/event_proportion‚6.300403038039804e-05 +jet/accuracy_2_of_6‚0.34210526943206787 =&Purity/0FRt1SRqqt*FBt/event_proportion‚0.2305317521095276 =&Purity/0FRt1SRqqt0FBt/event_proportion‚0.1694808453321457 >&Purity/1FRt2SRqqt*FBt/event_proportion‚0.05727066472172737 8 Purity/1FRt3SRqqt0FBt/FRt_purity‚0.20076805353164673 ,&Purity/2FRt4SRqqt3FBt/event_proportion‚0 7 Purity/3FRt1SRqqt*FBt/FRt_purity‚0.2294238954782486 ,&Purity/4FRt0SRqqt4FBt/event_proportion‚0 ,&Purity/4FRt3SRqqt0FBt/event_proportion‚0 >&Purity/*FRt2SRqqt1FBt/event_proportion‚0.02255544438958168 >&Purity/0FRt0SRqqt*FBt/event_proportion‚0.11857359111309052 8 Purity/2FRt1SRqqt2FBt/FRt_purity‚0.22587721049785614 @&Purity/3FRt*SRqqt2FBt/event_proportion‚0.0003150201519019902 ,&Purity/4FRt3SRqqt3FBt/event_proportion‚0 *jet/accuracy_4_of_5‚0.0115591399371624 jet/accuracy_5_of_6‚0 & Purity/1FRt0SRqqt4FBt/FBt_purity‚1 ?&Purity/*FRt1SRqqt2FBt/event_proportion‚0.017011089250445366 ,&Purity/*FRt3SRqqt3FBt/event_proportion‚0 7 Purity/0FRt*SRqqt3FBt/FBt_purity‚0.9115226864814758 & Purity/2FRt2SRqqt1FBt/FBt_purity‚1 :"Purity/3FRt1SRqqt1FBt/SRqqt_purity‚0.42307692766189575 ,jet/accuracy_3_of_4‚0.047573719173669815 +"Purity/3FRt1SRqqt2FBt/SRqqt_purity‚0.25 >&Purity/1FRt*SRqqt1FBt/event_proportion‚0.09929435700178146 9"Purity/1FRt1SRqqt1FBt/SRqqt_purity‚0.4684332609176636 ;"Purity/2FRt*SRqqt2FBt/event_purity‚0.017777778208255768 & Purity/3FRt2SRqqt*FBt/FBt_purity‚1 ("Purity/*FRt1SRqqt3FBt/event_purity‚0 :"Purity/*FRt*SRqqt0FBt/SRqqt_purity‚0.30667024850845337 7 Purity/0FRt0SRqqt2FBt/FBt_purity‚0.9903548955917358 A&Purity/0FRt4SRqqt0FBt/event_proportion‚0.00044102821266278625 7 Purity/1FRt1SRqqt1FBt/FRt_purity‚0.2726256847381592 :"Purity/2FRt0SRqqt1FBt/event_purity‚0.05203704163432121 ("Purity/3FRt1SRqqt0FBt/event_purity‚0 9"Purity/*FRt*SRqqt2FBt/event_purity‚0.1902415156364441 9"Purity/1FRt*SRqqt1FBt/event_purity‚0.1229730099439621 ,&Purity/3FRt4SRqqt2FBt/event_proportion‚0 @&Purity/4FRt*SRqqt*FBt/event_proportion‚0.0001890120911411941 ,&Purity/4FRt4SRqqt0FBt/event_proportion‚0 ("Purity/3FRt4SRqqt*FBt/event_purity‚0 particle/accuracy_1_of_7‚1 :"Purity/*FRt1SRqqt2FBt/SRqqt_purity‚0.48483288288116455 7 Purity/1FRt*SRqqt1FBt/FRt_purity‚0.2784653306007385 ("Purity/2FRt4SRqqt0FBt/event_purity‚0 & Purity/3FRt2SRqqt1FBt/FBt_purity‚1 @&Purity/1FRt2SRqqt2FBt/event_proportion‚0.0010080644860863686 @&Purity/3FRt0SRqqt*FBt/event_proportion‚0.0027091733645647764 jet/accuracy_3_of_7‚0 ("Purity/*FRt1SRqqt4FBt/SRqqt_purity‚1 ;"Purity/1FRt*SRqqt0FBt/event_purity‚0.057892706245183945 :"Purity/1FRt*SRqqt1FBt/SRqqt_purity‚0.43653905391693115 9"Purity/1FRt1SRqqt0FBt/SRqqt_purity‚0.2917046844959259 ;"Purity/2FRt*SRqqt*FBt/event_purity‚0.007124627474695444 particle/accuracy_2_of_5‚1 ("Purity/4FRt*SRqqt*FBt/event_purity‚0 :"Purity/1FRt1SRqqt2FBt/SRqqt_purity‚0.42338433861732483 @&Purity/2FRt*SRqqt3FBt/event_proportion‚6.300403038039804e-05 >&Purity/*FRt0SRqqt1FBt/event_proportion‚0.14591734111309052 =&Purity/*FRt1SRqqt1FBt/event_proportion‚0.1001134067773819 ;"Purity/1FRt1SRqqt1FBt/event_purity‚0.031224550679326057 & Purity/*FRt1SRqqt4FBt/FRt_purity‚0 8 Purity/*FRt*SRqqt0FBt/FRt_purity‚0.16979724168777466 9"Purity/0FRt1SRqqt*FBt/event_purity‚0.2886705994606018 8 Purity/1FRt0SRqqt*FBt/FRt_purity‚0.19959375262260437 ,&Purity/2FRt3SRqqt4FBt/event_proportion‚0 :"Purity/3FRt*SRqqt*FBt/SRqqt_purity‚0.28627532720565796 ("Purity/3FRt1SRqqt1FBt/event_purity‚0 8 Purity/*FRt0SRqqt0FBt/FRt_purity‚0.13054756820201874 @&Purity/2FRt4SRqqt*FBt/event_proportion‚0.0003150201519019902 8 Purity/3FRt*SRqqt0FBt/FRt_purity‚0.15589947998523712 ("Purity/3FRt1SRqqt*FBt/event_purity‚0 @&Purity/3FRt3SRqqt0FBt/event_proportion‚0.0005670362734235823 *particle/sensitivity‚0.263375461101532 9"Purity/1FRt4SRqqt0FBt/SRqqt_purity‚0.2916666567325592 ?&Purity/*FRt3SRqqt1FBt/event_proportion‚0.002142137149348855 @&Purity/4FRt0SRqqt0FBt/event_proportion‚6.300403038039804e-05 7 Purity/1FRt2SRqqt2FBt/FRt_purity‚0.2222222238779068 :"Purity/*FRt0SRqqt0FBt/event_purity‚0.10782642662525177 7 Purity/2FRt0SRqqt1FBt/FRt_purity‚0.3146825432777405 ,&Purity/2FRt2SRqqt3FBt/event_proportion‚0 ("Purity/3FRt0SRqqt0FBt/event_purity‚0 * Purity/4FRt*SRqqt*FBt/FRt_purity‚0.125 9"Purity/*FRt3SRqqt1FBt/SRqqt_purity‚0.3915344476699829 9"Purity/1FRt0SRqqt3FBt/event_purity‚0.0357142873108387 9"Purity/1FRt3SRqqt1FBt/SRqqt_purity‚0.4037036895751953 9"Purity/2FRt1SRqqt0FBt/SRqqt_purity‚0.2697879672050476 8 Purity/*FRt2SRqqt*FBt/FRt_purity‚0.20290347933769226 :"Purity/1FRt0SRqqt2FBt/event_purity‚0.13428610563278198 ("Purity/0FRt0SRqqt4FBt/event_purity‚0 ("Purity/*FRt1SRqqt4FBt/event_purity‚0 >&Purity/*FRt2SRqqt0FBt/event_proportion‚0.10855595022439957 ,&Purity/2FRt2SRqqt4FBt/event_proportion‚0 ,&Purity/4FRt*SRqqt2FBt/event_proportion‚0 9"Purity/0FRt0SRqqt1FBt/event_purity‚0.9990224242210388 ?&Purity/2FRt*SRqqt2FBt/event_proportion‚0.005418346729129553 7 Purity/3FRt2SRqqt0FBt/FRt_purity‚0.1751207709312439 9"Purity/2FRt3SRqqt1FBt/SRqqt_purity‚0.1666666716337204 9"Purity/1FRt4SRqqt*FBt/SRqqt_purity‚0.2916666567325592 ("Purity/1FRt3SRqqt2FBt/event_purity‚0 >&Purity/0FRt*SRqqt1FBt/event_proportion‚0.14787046611309052 A&Purity/0FRt1SRqqt3FBt/event_proportion‚0.00044102821266278625 7 Purity/1FRt1SRqqt2FBt/FBt_purity‚0.9761905074119568 ("Purity/3FRt*SRqqt1FBt/event_purity‚0 *particle/specificity‚0.951317548751831 8 Purity/2FRt4SRqqt*FBt/FRt_purity‚0.30000001192092896 ("Purity/3FRt4SRqqt0FBt/event_purity‚0 ("Purity/3FRt3SRqqt*FBt/event_purity‚0 8 Purity/*FRt1SRqqt2FBt/FRt_purity‚0.28158310055732727 & Purity/1FRt3SRqqt1FBt/FBt_purity‚1 @&Purity/4FRt*SRqqt0FBt/event_proportion‚0.0001890120911411941 ("Purity/0FRt*SRqqt4FBt/event_purity‚0 ("Purity/0FRt3SRqqt0FBt/event_purity‚0 >&Purity/*FRt0SRqqt0FBt/event_proportion‚0.15757308900356293 7 Purity/*FRt0SRqqt3FBt/FRt_purity‚0.4000000059604645 >&Purity/0FRt0SRqqt1FBt/event_proportion‚0.08606351166963577 7 Purity/1FRt*SRqqt2FBt/FBt_purity‚0.9863749742507935 @&Purity/1FRt*SRqqt3FBt/event_proportion‚0.0013230846961960196 & Purity/1FRt3SRqqt*FBt/FBt_purity‚1 9"Purity/2FRt2SRqqt*FBt/SRqqt_purity‚0.2968691289424896 :"Purity/2FRt2SRqqt1FBt/SRqqt_purity‚0.32053571939468384 8 Purity/*FRt*SRqqt3FBt/FRt_purity‚0.38235294818878174 :"Purity/0FRt*SRqqt0FBt/SRqqt_purity‚0.32810071110725403 7 Purity/2FRt*SRqqt2FBt/FRt_purity‚0.3338888883590698 7 Purity/2FRt2SRqqt0FBt/FRt_purity‚0.1883169263601303 +jet/accuracy_4_of_6‚0.05263157933950424 +"Purity/0FRt4SRqqt0FBt/SRqqt_purity‚0.25 & Purity/4FRt0SRqqt*FBt/FRt_purity‚0 7 Purity/*FRt*SRqqt1FBt/FBt_purity‚0.9990310072898865 ;"Purity/*FRt2SRqqt*FBt/event_purity‚0.012276850640773773 :"Purity/1FRt*SRqqt*FBt/event_purity‚0.07499208301305771 >&Purity/2FRt1SRqqt0FBt/event_proportion‚0.03181703761219978 ,&Purity/3FRt3SRqqt3FBt/event_proportion‚0 *jet/accuracy_1_of_3‚0.7759329676628113 particle/accuracy_3_of_7‚1 & Purity/2FRt*SRqqt3FBt/FBt_purity‚1 7 Purity/*FRt0SRqqt1FBt/FBt_purity‚0.9994959831237793 ("Purity/2FRt2SRqqt1FBt/event_purity‚0 @&Purity/2FRt2SRqqt2FBt/event_proportion‚6.300403038039804e-05 & Purity/3FRt*SRqqt1FBt/FBt_purity‚1 ("Purity/4FRt*SRqqt0FBt/SRqqt_purity‚0 7 Purity/*FRt0SRqqt*FBt/FBt_purity‚0.9930060505867004 ?&Purity/0FRt0SRqqt3FBt/event_proportion‚0.002583165420219302 & Purity/3FRt*SRqqt*FBt/FBt_purity‚1 7 Purity/*FRt1SRqqt3FBt/FBt_purity‚0.8730158805847168 9"Purity/3FRt3SRqqt0FBt/SRqqt_purity‚0.4047618806362152 ("Purity/4FRt2SRqqt*FBt/event_purity‚0 & Purity/4FRt2SRqqt0FBt/FRt_purity‚0 7 Purity/*FRt*SRqqt*FBt/FBt_purity‚0.9928374290466309 8 Purity/*FRt3SRqqt*FBt/FRt_purity‚0.20967674255371094 ("Purity/2FRt3SRqqt*FBt/event_purity‚0 ?&Purity/2FRt3SRqqt*FBt/event_proportion‚0.004032257944345474 8"Purity/3FRt*SRqqt1FBt/SRqqt_purity‚0.472222238779068 ,&Purity/1FRt4SRqqt1FBt/event_proportion‚0 ?&Purity/2FRt1SRqqt2FBt/event_proportion‚0.001890120911411941 ("Purity/2FRt3SRqqt1FBt/event_purity‚0 ,&Purity/4FRt*SRqqt3FBt/event_proportion‚0 :"Purity/*FRt2SRqqt2FBt/SRqqt_purity‚0.30701756477355957 8 Purity/2FRt3SRqqt1FBt/FRt_purity‚0.10000000149011612 jet/accuracy_4_of_7‚0 =&Purity/*FRt*SRqqt*FBt/event_proportion‚0.9591733813285828 @&Purity/*FRt1SRqqt4FBt/event_proportion‚6.300403038039åŒc804e-05 ?&Purity/*FRt2SRqqt2FBt/event_proportion‚0.002079133177176118 7 Purity/2FRt0SRqqt2FBt/FBt_purity‚0.9940475821495056 jet/accuracy_7_of_7‚0 9"Purity/*FRt1SRqqt3FBt/SRqqt_purity‚0.9047619104385376 ("Purity/1FRt3SRqqt1FBt/event_purity‚0 ("Purity/*FRt3SRqqt1FBt/event_purity‚0 @&Purity/1FRt0SRqqt4FBt/event_proportion‚6.300403038039804e-05 ,&Purity/3FRt*SRqqt4FBt/event_proportion‚0 ("Purity/*FRt4SRqqt0FBt/event_purity‚0 jet/accuracy_2_of_7‚0 & Purity/0FRt3SRqqt1FBt/FBt_purity‚1 @&Purity/*FRt4SRqqt*FBt/event_proportion‚0.0013230846961960196 8 Purity/1FRt2SRqqt*FBt/FRt_purity‚0.20392940938472748 @&Purity/4FRt1SRqqt*FBt/event_proportion‚6.300403038039804e-05 & Purity/2FRt3SRqqt1FBt/FBt_purity‚1 >&Purity/*FRt*SRqqt1FBt/event_proportion‚0.27079132199287415 ?&Purity/0FRt2SRqqt1FBt/event_proportion‚0.009261592291295528 ,&Purity/1FRt2SRqqt3FBt/event_proportion‚0 ("Purity/1FRt3SRqqt*FBt/event_purity‚0 & Purity/2FRt3SRqqt*FBt/FBt_purity‚1 ,&Purity/*FRt4SRqqt2FBt/event_proportion‚0 7 Purity/1FRt0SRqqt2FBt/FRt_purity‚0.4592787027359009 ,&Purity/3FRt2SRqqt4FBt/event_proportion‚0 A&Purity/3FRt4SRqqt0FBt/event_proportion‚0.00012600806076079607 particle/accuracy_1_of_3‚1 & Purity/0FRt3SRqqt*FBt/FBt_purity‚1 8 Purity/*FRt*SRqqt1FBt/FRt_purity‚0.29269590973854065 =&Purity/*FRt0SRqqt*FBt/event_proportion‚0.3558467626571655 9"Purity/0FRt2SRqqt0FBt/SRqqt_purity‚0.3154376447200775 ;"Purity/0FRt2SRqqt1FBt/event_purity‚0.028033794835209846 @&Purity/0FRt3SRqqt1FBt/event_proportion‚0.0006300403038039804 :"Purity/1FRt2SRqqt*FBt/SRqqt_purity‚0.30945125222206116 ?&Purity/2FRt2SRqqt1FBt/event_proportion‚0.003969253972172737 @&Purity/3FRt*SRqqt0FBt/event_proportion‚0.0071824598126113415 :"Purity/*FRt*SRqqt1FBt/SRqqt_purity‚0.41080600023269653 6 Purity/*FRt0SRqqt1FBt/FRt_purity‚0.306494802236557 9"Purity/*FRt1SRqqt0FBt/SRqqt_purity‚0.3153991401195526 6 Purity/3FRt*SRqqt1FBt/FRt_purity‚0.269696980714798 ("Purity/*FRt*SRqqt4FBt/event_purity‚0 ("Purity/0FRt3SRqqt*FBt/event_purity‚0 *"Purity/2FRt2SRqqt2FBt/SRqqt_purity‚0.5 :"Purity/*FRt*SRqqt*FBt/SRqqt_purity‚0.33113816380500793 ,&Purity/2FRt3SRqqt3FBt/event_proportion‚0 ("Purity/3FRt2SRqqt*FBt/event_purity‚0 ,&Purity/*FRt4SRqqt3FBt/event_proportion‚0 7 Purity/2FRt0SRqqt0FBt/FRt_purity‚0.1411932110786438 ,&Purity/4FRt2SRqqt4FBt/event_proportion‚0 +jet/accuracy_2_of_5‚0.38139086961746216 9"Purity/1FRt2SRqqt2FBt/SRqqt_purity‚0.2986111342906952 6 Purity/3FRt3SRqqt0FBt/FRt_purity‚0.190476194024086 & Purity/0FRt2SRqqt3FBt/FBt_purity‚1 7 Purity/0FRt0SRqqt1FBt/FBt_purity‚0.9990224242210388 >&Purity/0FRt1SRqqt1FBt/event_proportion‚0.05185231938958168 ,&Purity/1FRt3SRqqt3FBt/event_proportion‚0 ("Purity/1FRt1SRqqt4FBt/event_purity‚0 7 Purity/1FRt3SRqqt1FBt/FRt_purity‚0.3888888955116272 particle/accuracy_3_of_4‚1 7 Purity/*FRt4SRqqt0FBt/FRt_purity‚0.1666666716337204 ("Purity/*FRt3SRqqt2FBt/event_purity‚0 9"Purity/*FRt*SRqqt1FBt/event_purity‚0.3961000144481659 >&Purity/1FRt0SRqqt*FBt/event_proportion‚0.19745463132858276 ?&Purity/1FRt0SRqqt2FBt/event_proportion‚0.015120967291295528 ) Purity/4FRt1SRqqt0FBt/FRt_purity‚0.25 6 Purity/0FRt0SRqqt*FBt/FBt_purity‚0.991509735584259 >&Purity/0FRt0SRqqt0FBt/event_proportion‚0.04082661122083664 8"Purity/0FRt2SRqqt1FBt/SRqqt_purity‚0.362519234418869 ?&Purity/1FRt*SRqqt2FBt/event_proportion‚0.022303426638245583 ,&Purity/3FRt0SRqqt4FBt/event_proportion‚0 7 Purity/0FRt1SRqqt3FBt/FBt_purity‚0.8333333730697632 ("Purity/1FRt*SRqqt4FBt/SRqqt_purity‚1 ;"Purity/0FRt1SRqqt2FBt/event_purity‚0.033888887614011765 ."Purity/1FRt*SRqqt3FBt/event_purity‚0.03125 7 Purity/1FRt0SRqqt3FBt/FBt_purity‚0.9523809552192688 ,&Purity/3FRt1SRqqt4FBt/event_proportion‚0 =&Purity/1FRt1SRqqt*FBt/event_proportion‚0.1723790317773819 8 Purity/1FRt1SRqqt0FBt/FRt_purity‚0.20165486633777618 9"Purity/2FRt*SRqqt0FBt/SRqqt_purity‚0.2713819146156311 7 Purity/2FRt1SRqqt*FBt/FRt_purity‚0.2078690081834793 ?&Purity/2FRt1SRqqt1FBt/event_proportion‚0.007875503972172737 @&Purity/3FRt2SRqqt1FBt/event_proportion‚0.0003780241822823882 @&Purity/4FRt0SRqqt*FBt/event_proportion‚6.300403038039804e-05 ("Purity/4FRt1SRqqt*FBt/SRqqt_purity‚0 9"Purity/*FRt0SRqqt3FBt/event_purity‚0.0773809552192688 ("Purity/*FRt3SRqqt0FBt/event_purity‚0 :"Purity/0FRt0SRqqt2FBt/event_purity‚0.34798091650009155 :"Purity/1FRt*SRqqt2FBt/SRqqt_purity‚0.39567330479621887 >&Purity/2FRt1SRqqt*FBt/event_proportion‚0.04158266261219978 * Purity/0FRt*SRqqt4FBt/FBt_purity‚0.875 ) Purity/3FRt1SRqqt2FBt/FRt_purity‚0.25 & Purity/2FRt2SRqqt2FBt/FBt_purity‚1 :"Purity/0FRt*SRqqt*FBt/SRqqt_purity‚0.34678059816360474 8 Purity/1FRt*SRqqt0FBt/FRt_purity‚0.17052078247070312 ?&Purity/2FRt*SRqqt1FBt/event_proportion‚0.021169355139136314 ,&Purity/4FRt*SRqqt4FBt/event_proportion‚0 ,&Purity/4FRt4SRqqt2FBt/event_proportion‚0 7 Purity/3FRt0SRqqt2FBt/FRt_purity‚0.6666666865348816 9"Purity/0FRt0SRqqt*FBt/event_purity‚0.8158611059188843 7 Purity/2FRt0SRqqt*FBt/FBt_purity‚0.9967741966247559 @&Purity/2FRt3SRqqt1FBt/event_proportion‚0.0003780241822823882 ,&Purity/3FRt4SRqqt1FBt/event_proportion‚0 ,&Purity/4FRt4SRqqt1FBt/event_proportion‚0 +jet/accuracy_3_of_5‚0.08306130021810532 jet/accuracy_1_of_7‚1 @&Purity/0FRt4SRqqt1FBt/event_proportion‚6.300403038039804e-05 ?&Purity/2FRt0SRqqt*FBt/event_proportion‚0.037046369165182114 & Purity/0FRt4SRqqt1FBt/FBt_purity‚1 ("Purity/0FRt2SRqqt2FBt/event_purity‚08œ‚Úiغ¦ ("Purity/0FRt0SRqqt4FBt/event_purity‚0 ("Purity/*FRt1SRqqt4FBt/event_purity‚0 >&Purity/*FRt2SRqqt0FBt/event_proportion‚0.10855595022439957 ,&Purity/2FRt2SRqqt4FBt/event_proportion‚0 ,&Purity/4FRt*SRqqt2FBt/event_proportion‚0 9"Purity/0FRt0SRqqt1FBt/event_purity‚0.9990224242210388 ?&Purity/2FRt*SRqqt2FBt/event_proportion‚0.005418346729129553 7 Purity/3FRt2SRqqt0FBt/FRt_purity‚0.1751207709312439 9"Purity/2FRt3SRqqt1FBt/SRqqt_purity‚0.1666666716337204 9"Purity/1FRt4SRqqt*FBt/SRqqt_purity‚0.2916666567325592 ("Purity/1FRt3SRqqt2FBt/event_purity‚0 >&Purity/0FRt*SRqqt1FBt/event_proportion‚0.14787046611309052 A&Purity/0FRt1SRqqt3FBt/event_proportion‚0.00044102821266278625 7 Purity/1FRt1SRqqt2FBt/FBt_purity‚0.9761905074119568 ("Purity/3FRt*SRqqt1FBt/event_purity‚0 *particle/specificity‚0.951317548751831 8 Purity/2FRt4SRqqt*FBt/FRt_purity‚0.30000001192092896 ("Purity/3FRt4SRqqt0FBt/event_purity‚0 ("Purity/3FRt3SRqqt*FBt/event_purity‚0 8 Purity/*FRt1SRqqt2FBt/FRt_purity‚0.28158310055732727 & Purity/1FRt3SRqqt1FBt/FBt_purity‚1 @&Purity/4FRt*SRqqt0FBt/event_proportion‚0.0001890120911411941 ("Purity/0FRt*SRqqt4FBt/event_purity‚0 ("Purity/0FRt3SRqqt0FBt/event_purity‚0 >&Purity/*FRt0SRqqt0FBt/event_proportion‚0.15757308900356293 7 Purity/*FRt0SRqqt3FBt/FRt_purity‚0.4000000059604645 >&Purity/0FRt0SRqqt1FBt/event_proportion‚0.08606351166963577 7 Purity/1FRt*SRqqt2FBt/FBt_purity‚0.9863749742507935 @&Purity/1FRt*SRqqt3FBt/event_proportion‚0.0013230846961960196 & Purity/1FRt3SRqqt*FBt/FBt_purity‚1 9"Purity/2FRt2SRqqt*FBt/SRqqt_purity‚0.2968691289424896 :"Purity/2FRt2SRqqt1FBt/SRqqt_purity‚0.32053571939468384 8 Purity/*FRt*SRqqt3FBt/FRt_purity‚0.38235294818878174 :"Purity/0FRt*SRqqt0FBt/SRqqt_purity‚0.32810071110725403 7 Purity/2FRt*SRqqt2FBt/FRt_purity‚0.3338888883590698 7 Purity/2FRt2SRqqt0FBt/FRt_purity‚0.1883169263601303 +jet/accuracy_4_of_6‚0.05263157933950424 +"Purity/0FRt4SRqqt0FBt/SRqqt_purity‚0.25 & Purity/4FRt0SRqqt*FBt/FRt_purity‚0 7 Purity/*FRt*SRqqt1FBt/FBt_purity‚0.9990310072898865 ;"Purity/*FRt2SRqqt*FBt/event_purity‚0.012276850640773773 :"Purity/1FRt*SRqqt*FBt/event_purity‚0.07499208301305771 >&Purity/2FRt1SRqqt0FBt/event_proportion‚0.03181703761219978 ,&Purity/3FRt3SRqqt3FBt/event_proportion‚0 *jet/accuracy_1_of_3‚0.7759329676628113 particle/accuracy_3_of_7‚1 & Purity/2FRt*SRqqt3FBt/FBt_purity‚1 7 Purity/*FRt0SRqqt1FBt/FBt_purity‚0.9994959831237793 ("Purity/2FRt2SRqqt1FBt/event_purity‚0 @&Purity/2FRt2SRqqt2FBt/event_proportion‚6.300403038039804e-05 & Purity/3FRt*SRqqt1FBt/FBt_purity‚1 ("Purity/4FRt*SRqqt0FBt/SRqqt_purity‚0 7 Purity/*FRt0SRqqt*FBt/FBt_purity‚0.9930060505867004 ?&Purity/0FRt0SRqqt3FBt/event_proportion‚0.002583165420219302 & Purity/3FRt*SRqqt*FBt/FBt_purity‚1 7 Purity/*FRt1SRqqt3FBt/FBt_purity‚0.8730158805847168 9"Purity/3FRt3SRqqt0FBt/SRqqt_purity‚0.4047618806362152 ("Purity/4FRt2SRqqt*FBt/event_purity‚0 & Purity/4FRt2SRqqt0FBt/FRt_purity‚0 7 Purity/*FRt*SRqqt*FBt/FBt_purity‚0.9928374290466309 8 Purity/*FRt3SRqqt*FBt/FRt_purity‚0.20967674255371094 ("Purity/2FRt3SRqqt*FBt/event_purity‚0 ?&Purity/2FRt3SRqqt*FBt/event_proportion‚0.004032257944345474 8"Purity/3FRt*SRqqt1FBt/SRqqt_purity‚0.472222238779068 ,&Purity/1FRt4SRqqt1FBt/event_proportion‚0 ?&Purity/2FRt1SRqqt2FBt/event_proportion‚0.001890120911411941 ("Purity/2FRt3SRqqt1FBt/event_purity‚0 ,&Purity/4FRt*SRqqt3FBt/event_proportion‚0 :"Purity/*FRt2SRqqt2FBt/SRqqt_purity‚0.30701756477355957 8 Purity/2FRt3SRqqt1FBt/FRt_purity‚0.10000000149011612 jet/accuracy_4_of_7‚0 =&Purity/*FRt*SRqqt*FBt/event_proportion‚0.9591733813285828 @&Purity/*FRt1SRqqt4FBt/event_proportion‚6.300403038039804e-05 ?&Purity/*FRt2SRqqt2FBt/event_proportion‚0.002079133177176118 7 Purity/2FRt0SRqqt2FBt/FBt_purity‚0.9940475821495056 jet/accuracy_7_of_7‚0 9"Purity/*FRt1SRqqt3FBt/SRqqt_purity‚0.9047619104385376 ("Purity/1FRt3SRqqt1FBt/event_purity‚0 ("Purity/*FRt3SRqqt1FBt/event_purity‚0 @&Purity/1FRt0SRqqt4FBt/event_proportion‚6.300403038039804e-05 ,&Purity/3FRt*SRqqt4FBt/event_proportion‚0 ("Purity/*FRt4SRqqt0FBt/event_purity‚0 jet/accuracy_2_of_7‚0 & Purity/0FRt3SRqqt1FBt/FBt_purity‚1 @&Purity/*FRt4SRqqt*FBt/event_proportion‚0.0013230846961960196 8 Purity/1FRt2SRqqt*FBt/FRt_purity‚0.20392940938472748 @&Purity/4FRt1SRqqt*FBt/event_proportion‚6.300403038039804e-05 & Purity/2FRt3SRqqt1FBt/FBt_purity‚1 >&Purity/*FRt*SRqqt1FBt/event_proportion‚0.27079132199287415 ?&Purity/0FRt2SRqqt1FBt/event_proportion‚0.009261592291295528 ,&Purity/1FRt2SRqqt3FBt/event_proportion‚0 ("Purity/1FRt3SRqqt*FBt/event_purity‚0 & Purity/2FRt3SRqqt*FBt/FBt_purity‚1 ,&Purity/*FRt4SRqqt2FBt/event_proportion‚0 7 Purity/1FRt0SRqqt2FBt/FRt_purity‚0.4592787027359009 ,&Purity/3FRt2SRqqt4FBt/event_proportion‚0 A&Purity/3FRt4SRqqt0FBt/event_proportion‚0.00012600806076079607 particle/accuracy_1_of_3‚1 & Purity/0FRt3SRqqt*FBt/FBt_purity‚1 8 Purity/*FRt*SRqqt1FBt/FRt_purity‚0.29269590973854065 =&Purity/*FRt0SRqqt*FBt/event_proportion‚0.3558467626571655 9"Purity/0FRt2SRqqt0FBt/SRqqt_purity‚0.3154376447200775 ;"Purity/0FRt2SRqqt1FBt/event_purity‚0.028033794835209846 @&Purity/0FRt3SRqqt1FBt/event_proportion‚0.0006300403038039804 :"Purity/1FRt2SRqqt*FBt/SRqqt_purity‚0.30945125222206116 ?&Purity/2FRt2SRqqt1FBt/event_proportion‚0.003969253972172737 @&Purity/3FRt*SRqqt0FBt/event_proportion‚0.0071824598126113415 :"Purity/*FRt*SRqqt1FBt/SRqqt_purity‚0.41080600023269653 6 Purity/*FRt0SRqqt1FBt/FRt_purity‚0.306494802236557 9"Purity/*FRt1SRqqt0FBt/SRqqt_purity‚0.3153991401195526 6 Purity/3FRt*SRqqt1FBt/FRt_purity‚0.269696980714798 ("Purity/*FRt*SRqqt4FBt/event_purity‚0 ("Purity/0FRt3SRqqt*FBt/event_purity‚0 *"Purity/2FRt2SRqqt2FBt/SRqqt_purity‚0.5 :"Purity/*FRt*SRqqt*FBt/SRqqt_purity‚0.33113816380500793 ,&Purity/2FRt3SRqqt3FBt/event_proportion‚0 ("Purity/3FRt2SRqqt*FBt/event_purity‚0 ,&Purity/*FRt4SRqqt3FBt/event_proportion‚0 7 Purity/2FRt0SRqqt0FBt/FRt_purity‚0.1411932110786438 ,&Purity/4FRt2SRqqt4FBt/event_proportion‚0 +jet/accuracy_2_of_5‚0.38139086961746216 9"Purity/1FRt2SRqqt2FBt/SRqqt_purity‚0.2986111342906952 6 Purity/3FRt3SRqqt0FBt/FRt_purity‚0.190476194024086 & Purity/0FRt2SRqqt3FBt/FBt_purity‚1 7 Purity/0FRt0SRqqt1FBt/FBt_purity‚0.9990224242210388 >&Purity/0FRt1SRqqt1FBt/event_proportion‚0.05185231938958168 ,&Purity/1FRt3SRqqt3FBt/event_proportion‚0 ("Purity/1FRt1SRqqt4FBt/event_purity‚0 7 Purity/1FRt3SRqqt1FBt/FRt_purity‚0.3888888955116272 particle/accuracy_3_of_4‚1 7 Purity/*FRt4SRqqt0FBt/FRt_purity‚0.1666666716337204 ("Purity/*FRt3SRqqt2FBt/event_purity‚0 9"Purity/*FRt*SRqqt1FBt/event_purity‚0.3961000144481659 >&Purity/1FRt0SRqqt*FBt/event_proportion‚0.19745463132858276 ?&Purity/1FRt0SRqqt2FBt/event_proportion‚0.015120967291295528 ) Purity/4FRt1SRqqt0FBt/FRt_purity‚0.25 6 Purity/0FRt0SRqqt*FBt/FBt_purity‚0.991509735584259 >&Purity/0FRt0SRqqt0FBt/event_proportion‚0.04082661122083664 8"Purity/0FRt2SRqqt1FBt/SRqqt_purity‚0.362519234418869 ?&Purity/1FRt*SRqqt2FBt/event_proportion‚0.022303426638245583 ,&Purity/3FRt0SRqqt4FBt/event_proportion‚0 7 Purity/0FRt1SRqqt3FBt/FBt_purity‚0.8333333730697632 ("Purity/1FRt*SRqqt4FBt/SRqqt_purity‚1 ;"Purity/0FRt1SRqqt2FBt/event_purity‚0.033888887614011765 ."Purity/1FRt*SRqqt3FBt/event_purity‚0.03125 7 Purity/1FRt0SRqqt3FBt/FBt_purity‚0.9523809552192688 ,&Purity/3FRt1SRqqt4FBt/event_proportion‚0 =&Purity/1FRt1SRqqt*FBt/event_proportion‚0.1723790317773819 8 Purity/1FRt1SRqqt0FBt/FRt_purity‚0.20165486633777618 9"Purity/2FRt*SRqqt0FBt/SRqqt_purity‚0.2713819146156311 7 Purity/2FRt1SRqqt*FBt/FRt_purity‚0.2078690081834793 ?&Purity/2FRt1SRqqt1FBt/event_proportion‚0.007875503972172737 @&Purity/3FRt2SRqqt1FBt/event_proportion‚0.0003780241822823882 @&Purity/4FRt0SRqqt*FBt/event_proportion‚6.300403038039804e-05 ("Purity/4FRt1SRqqt*FBt/SRqqt_purity‚0 9"Purity/*FRt0SRqqt3FBt/event_purity‚0.0773809552192688 ("Purity/*FRt3SRqqt0FBt/event_purity‚0 :"Purity/0FRt0SRqqt2FBt/event_purity‚0.34798091650009155 :"Purity/1FRt*SRqqt2FBt/SRqqt_purity‚0.39567330479621887 >&Purity/2FRt1SRqqt*FBt/event_proportion‚0.04158266261219978 * Purity/0FRt*SRqqt4FBt/FBt_purity‚0.875 ) Purity/3FRt1SRqqt2FBt/FRt_purity‚0.25 & Purity/2FRt2SRqqt2FBt/FBt_purity‚1 :"Purity/0FRt*SRqqt*FBt/SRqqt_purity‚0.34678059816360474 8 Purity/1FRt*SRqqt0FBt/FRt_purity‚0.17052078247070312 ?&Purity/2FRt*SRqqt1FBt/event_proportion‚0.021169355139136314 ,&Purity/4FRt*SRqqt4FBt/event_proportion‚0 ,&Purity/4FRt4SRqqt2FBt/event_proportion‚0 7 Purity/3FRt0SRqqt2FBt/FRt_purity‚0.6666666865348816 9"Purity/0FRt0SRqqt*FBt/event_purity‚0.8158611059188843 7 Purity/2FRt0SRqqt*FBt/FBt_purity‚0.9967741966247559 @&Purity/2FRt3SRqqt1FBt/event_proportion‚0.0003780241822823882 ,&Purity/3FRt4SRqqt1FBt/event_proportion‚0 ,&Purity/4FRt4SRqqt1FBt/event_proportion‚0 +jet/accuracy_3_of_5‚0.08306130021810532 jet/accuracy_1_of_7‚1 @&Purity/0FRt4SRqqt1FBt/event_proportion‚6.300403038039804e-05 ?&Purity/2FRt0SRqqt*FBt/event_proportion‚0.037046369165182114 & Purity/0FRt4SRqqt1FBt/FBt_purity‚1 ("Purity/0FRt2SRqqt2FBt/event_purity‚0 jet/accuracy_5_of_7‚0 ("Purity/1FRt1SRqqt4FBt/SRqqt_purity‚1 & Purity/*FRt2SRqqt3FBt/FBt_purity‚1 ,&Purity/0FRt2SRqqt4FBt/event_proportion‚0 :"Purity/1FRt0SRqqt*FBt/event_purity‚0.15309707820415497 ?&Purity/2FRt3SRqqt0FBt/event_proportion‚0.003654233878478408 particle/accuracy_4_of_5‚1 & Purity/0FRt4SRqqt*FBt/FBt_purity‚1 ("Purity/3FRt2SRqqt1FBt/event_purity‚0 ("Purity/1FRt*SRqqt4FBt/event_purity‚0 :"Purity/*FRt0SRqqt2FBt/event_purity‚0.26003119349479675 ,&Purity/1FRt2SRqqt4FBt/event_proportion‚0 & Purity/2FRt0SRqqt1FBt/FBt_purity‚1 9"Purity/2FRt1SRqqt*FBt/SRqqt_purity‚0.3077162802219391 7 Purity/*FRt0SRqqt2FBt/FBt_purity‚0.9910675287246704 ?&Purity/0FRt1SRqqt2FBt/event_proportion‚0.008757560513913631 * Purity/1FRt*SRqqt3FBt/FRt_purity‚0.375 0"Purity/*FRt4SRqqt*FBt/SRqqt_purity‚ 0.2578125 ("Purity/0FRt4SRqqt*FBt/event_purity‚0 & Purity/3FRt*SRqqt2FBt/FBt_purity‚1 ("Purity/2FRt0SRqqt3FBt/event_purity‚0 & Purity/*FRt3SRqqt2FBt/FRt_purity‚1 6 Purity/*FRt*SRqqt2FBt/FBt_purity‚0.988559901714325 8 Purity/2FRt3SRqqt0FBt/FRt_purity‚0.23133116960525513 7 Purity/3FRt4SRqqt*FBt/FRt_purity‚0.1666666716337204 ?&Purity/*FRt0SRqqt3FBt/event_proportion‚0.003717237850651145 9"Purity/*FRt2SRqqt0FBt/SRqqt_purity‚0.2991968095302582 ;"Purity/*FRt2SRqqt1FBt/event_purity‚0.017076784744858742 ,&Purity/2FRt4SRqqt2FBt/event_proportion‚0 ("Purity/*FRt4SRqqt1FBt/SRqqt_purity‚0 ("Purity/2FRt*SRqqt3FBt/event_purity‚0 ("Purity/1FRt*SRqqt3FBt/SRqqt_purity‚1 ( Purity/1FRt1SRqqt3FBt/FRt_purity‚0.5 :"Purity/0FRt1SRqqt0FBt/SRqqt_purity‚0.34259894490242004 :"Purity/3FRt*SRqqt0FBt/SRqqt_purity‚0.27473074197769165 particle/accuracy_1_of_4‚1 *jet/accuracy_1_of_6‚0.9473684430122375 ) Purity/*FRt1SRqqt4FBt/FBt_purity‚0.75 ,&Purity/0FRt4SRqqt3FBt/event_proportion‚0 ,&Purity/0FRt4SRqqt4FBt/event_proportion‚0 7 Purity/1FRt*SRqqt*FBt/FBt_purity‚0.9929631352424622 7 Purity/2FRt*SRqqt*FBt/FRt_purity‚0.2111542522907257 ,&Purity/3FRt1SRqqt3FBt/event_proportion‚0 & Purity/1FRt3SRqqt2FBt/FBt_purity‚1 9"Purity/*FRt2SRqqt1FBt/SRqqt_purity‚0.3611535429954529 8"Purity/0FRt*SRqqt2FBt/event_purity‚0.265972763299942 & Purity/1FRt1SRqqt1FBt/FBt_purity‚1 ,&Purity/1FRt4SRqqt4FBt/event_proportion‚0 ("Purity/*FRt4SRqqt1FBt/event_purity‚0 9"Purity/*FRt1SRqqt*FBt/event_purity‚0.1541629135608673 8 Purity/*FRt1SRqqt*FBt/FRt_purity‚0.21831995248794556 9"Purity/0FRt2SRqqt*FBt/event_purity‚0.0284329392015934 7 Purity/1FRt1SRqqt*FBt/FBt_purity‚0.9906545877456665 ("Purity/2FRt2SRqqt0FBt/event_purity‚0 ("Purity/3FRt0SRqqt*FBt/event_purity‚0 ,&Purity/3FRt3SRqqt1FBt/event_proportion‚0 +jet/accuracy_2_of_3‚0.21289102733135223 @&Purity/0FRt2SRqqt3FBt/event_proportion‚6.300403038039804e-05 7 Purity/1FRt*SRqqt1FBt/FBt_purity‚0.9985867738723755 9"Purity/1FRt1SRqqt*FBt/SRqqt_purity‚0.3392636477947235 >&Purity/1FRt1SRqqt1FBt/event_proportion‚0.03944052383303642 ,&Purity/2FRt*SRqqt4FBt/event_proportion‚0 6validation_average_jet_accuracy‚0.3725806176662445 7 Purity/3FRt2SRqqt1FBt/FRt_purity‚0.2222222238779068 :"Purity/*FRt3SRqqt*FBt/SRqqt_purity‚0.29253751039505005 ,&Purity/0FRt3SRqqt2FBt/event_proportion‚0 ("Purity/2FRt1SRqqt2FBt/event_purity‚0 ,&Purity/4FRt3SRqqt1FBt/event_proportion‚0 particle/accuracy_1_of_5‚1 ( Purity/2FRt0SRqqt3FBt/FRt_purity‚0.5 ,&Purity/0FRt3SRqqt3FBt/event_proportion‚0 9"Purity/2FRt3SRqqt0FBt/SRqqt_purity‚0.2328643798828125 A&Purity/3FRt1SRqqt2FBt/event_proportion‚0.00025201612152159214 ,&Purity/4FRt0SRqqt2FBt/event_proportion‚0 ,&Purity/4FRt1SRqqt4FBt/event_proportion‚0 & Purity/*FRt4SRqqt1FBt/FBt_purity‚1 9"Purity/3FRt2SRqqt1FBt/SRqqt_purity‚0.3333333432674408 & Purity/1FRt0SRqqt4FBt/FRt_purity‚0 ,&Purity/0FRt4SRqqt2FBt/event_proportion‚0 ("Purity/1FRt3SRqqt0FBt/event_purity‚0 :"Purity/3FRt2SRqqt0FBt/SRqqt_purity‚0.32608696818351746 *jet/accuracy_1_of_2‚0.6576357483863831 & Purity/*FRt4SRqqt*FBt/FBt_purity‚1 8 Purity/*FRt2SRqqt2FBt/FRt_purity‚0.24358974397182465 ("Purity/*FRt*SRqqt4FBt/SRqqt_purity‚1 :"Purity/1FRt2SRqqt0FBt/SRqqt_purity‚0.29128792881965637 7 Purity/2FRt2SRqqt1FBt/FRt_purity‚0.2886905074119568 ,&Purity/4FRt3SRqqt2FBt/event_proportion‚0 *jet/accuracy_2_of_4‚0.3037717640399933 +"Purity/2FRt4SRqqt*FBt/SRqqt_purity‚0.25 +"Purity/3FRt*SRqqt2FBt/SRqqt_purity‚0.25 7 Purity/1FRt4SRqqt*FBt/FRt_purity‚0.1666666716337204 ("Purity/4FRt1SRqqt0FBt/SRqqt_purity‚0 9"Purity/*FRt1SRqqt*FBt/SRqqt_purity‚0.3494856059551239 7 Purity/0FRt1SRqqt2FBt/FBt_purity‚0.9798147678375244 9"Purity/0FRt2SRqqt*FBt/SRqqt_purity‚0.3223111629486084 +"Purity/2FRt4SRqqt0FBt/SRqqt_purity‚0.25 ("Purity/4FRt2SRqqt0FBt/SRqqt_purity‚0 7 Purity/*FRt3SRqqt1FBt/FRt_purity‚0.3444444239139557 ?&Purity/0FRt3SRqqt*FBt/event_proportion‚0.006741431541740894 ,&Purity/0FRt3SRqqt4FBt/event_proportion‚0 ;"Purity/1FRt1SRqqt*FBt/event_purity‚0.014884690754115582 ,&Purity/4FRt0SRqqt1FBt/event_proportion‚0 9"Purity/*FRt*SRqqt0FBt/event_purity‚0.1259285807609558 @&Purity/3FRt1SRqqt0FBt/event_proportion‚0.0025201612152159214 :"Purity/3FRt1SRqqt0FBt/SRqqt_purity‚0.36666664481163025 :"Purity/0FRt1SRqqt1FBt/event_purity‚0.15956179797649384 A&Purity/3FRt4SRqqt*FBt/event_proportion‚0.00012600806076079607 ,&Purity/4FRt2SRqqt1FBt/event_proportion‚0 * Purity/0FRt0SRqqt4FBt/FBt_purity‚0.875 @&Purity/*FRt1SRqqt3FBt/event_proportion‚0.0006930443341843784 7 Purity/1FRt2SRqqt*FBt/FBt_purity‚0.9890029430389404 ?&Purity/2FRt2SRqqt0FBt/event_proportion‚0.015435988083481789 :"Purity/0FRt2SRqqt2FBt/SRqqt_purity‚0.26923078298568726 ("Purity/1FRt0SRqqt4FBt/event_purity‚0 7 Purity/0FRt*SRqqt1FBt/FBt_purity‚0.9991740584373474 ?&Purity/0FRt3SRqqt0FBt/event_proportion‚0.006111391354352236 9"Purity/1FRt0SRqqt0FBt/event_purity‚0.1285238265991211 9"Purity/2FRt*SRqqt1FBt/SRqqt_purity‚0.3291139602661133 ?&Purity/2FRt0SRqqt1FBt/event_proportion‚0.008946572430431843 ,&Purity/2FRt1SRqqt4FBt/event_proportion‚0 @&Purity/3FRt*SRqqt1FBt/event_proportion‚0.0024571572430431843 ,&Purity/4FRt*SRqqt1FBt/event_proportion‚0 ("Purity/*FRt3SRqqt*FBt/event_purity‚0 7 Purity/0FRt*SRqqt*FBt/FBt_purity‚0.9921687841415405 :"Purity/0FRt0SRqqt3FBt/event_purity‚0.14000000059604645 ;"Purity/2FRt0SRqqt*FBt/event_purity‚0.018544865772128105 ,&Purity/4FRt0SRqqt3FBt/event_proportion‚0 particle/accuracy_4_of_4‚1 ("Purity/2FRt4SRqqt*FBt/event_purity‚0 particle/accuracy_4_of_6‚1 7 Purity/1FRt0SRqqt2FBt/FBt_purity‚0.9907549619674683 7 Purity/1FRt2SRqqt1FBt/FBt_purity‚0.9873272180557251 8 Purity/2FRt3SRqqt*FBt/FRt_purity‚0.21947617828845978 particle/accuracy_6_of_6‚1 7 Purity/3FRt4SRqqt0FBt/FRt_purity‚0.1666666716337204 8 Purity/*FRt1SRqqt0FBt/FRt_purity‚0.19284585118293762 7 Purity/2FRt1SRqqt*FBt/FBt_purity‚0.9964157938957214 ,&Purity/4FRt1SRqqt2FBt/event_proportion‚0 particle/accuracy_2_of_7‚1 % _timestamp‚1.7884164498830664e+09 =&Purity/*FRt0SRqqt2FBt/event_proportion‚0.0483240932226181 >&Purity/*FRt1SRqqt0FBt/event_proportion‚0.33039313554763794 >&Purity/*FRt3SRqqt0FBt/event_proportion‚0.01827116869390011 ("Purity/2FRt1SRqqt*FBt/event_purity‚0 9"Purity/2FRt1SRqqt2FBt/SRqqt_purity‚0.6359648704528809 ,&Purity/3FRt3SRqqt2FBt/event_proportion‚0 ("Purity/1FRt1SRqqt3FBt/SRqqt_purity‚1 @&Purity/*FRt0SRqqt4FBt/event_proportion‚0.0003150201519019902 8 Purity/2FRt*SRqqt0FBt/FRt_purity‚0.17074377834796906 ;"Purity/2FRt*SRqqt1FBt/event_purity‚0.026147358119487762 ,&Purity/2FRt4SRqqt1FBt/event_proportion‚0 ,&Purity/2FRt4SRqqt4FBt/event_proportion‚0 ("Purity/3FRt*SRqqt2FBt/event_purity‚0 & Purity/3FRt1SRqqt2FBt/FBt_purity‚1 ("Purity/1FRt4SRqqt0FBt/event_purity‚0 8 Purity/*FRt2SRqqt1FBt/FRt_purity‚0.23963364958763123 >&Purity/0FRt*SRqqt2FBt/event_proportion‚0.03944052383303642 :"Purity/0FRt*SRqqt3FBt/event_purity‚0.08641975373029709 A&Purity/0FRt*SRqqt4FBt/event_proportion‚0.00025201612152159214 & Purity/2FRt1SRqqt1FBt/FBt_purity‚1 @&Purity/4FRt2SRqqt*FBt/event_proportion‚6.300403038039804e-05 %particle/accuracy‚0.8438760638237 & Purity/3FRt0SRqqt2FBt/FBt_purity‚1 9"Purity/0FRt*SRqqt2FBt/SRqqt_purity‚0.4311111271381378 9"Purity/0FRt1SRqqt1FBt/SRqqt_purity‚0.4216614067554474 ?&Purity/1FRt0SRqqt1FBt/event_proportion‚0.049773186445236206 +jet/accuracy_3_of_6‚0.17543859779834747 & Purity/0FRt2SRqqt2FBt/FBt_purity‚1 ("Purity/0FRt1SRqqt3FBt/event_purity‚0 _runtime‚2031.534288604 particle/accuracy_2_of_2‚1 particle/accuracy_4_of_7‚1 7 Purity/1FRt4SRqqt0FBt/FRt_purity‚0.1666666716337204 ( Purity/2FRt2SRqqt2FBt/FRt_purity‚0.5 ("Purity/*FRt2SRqqt3FBt/SRqqt_purity‚0 9"Purity/1FRt*SRqqt0FBt/SRqqt_purity‚0.2929992079734802 ;"Purity/1FRt1SRqqt0FBt/event_purity‚0.010006660595536232 ("Purity/*FRt4SRqqt*FBt/event_purity‚0 ."Purity/0FRt4SRqqt*FBt/SRqqt_purity‚0.21875 & Purity/*FRt0SRqqt4FBt/FRt_purity‚0 ("Purity/3FRt0SRqqt1FBt/event_purity‚0 & Purity/*FRt*SRqqt4FBt/FRt_purity‚0 =&Purity/*FRt*SRqqt0FBt/event_proportion‚0.6160534024238586 7 Purity/*FRt0SRqqt2FBt/FRt_purity‚0.4407619833946228 9"Purity/0FRt*SRqqt0FBt/event_purity‚0.2699776589870453 :"Purity/1FRt2SRqqt1FBt/SRqqt_purity‚0.36363285779953003 ("Purity/0FRt4SRqqt0FBt/event_purity‚0 7 Purity/3FRt*SRqqt2FBt/FRt_purity‚0.3333333432674408 * Purity/1FRt*SRqqt4FBt/FBt_purity‚0.875 & Purity/*FRt3SRqqt2FBt/FBt_purity‚1 >&Purity/*FRt*SRqqt2FBt/event_proportion‚0.06747731566429138 7 Purity/*FRt*SRqqt2FBt/FRt_purity‚0.3882356286048889 @&Purity/0FRt*SRqqt3FBt/event_proportion‚0.0030871976632624865 ("Purity/0FRt3SRqqt1FBt/event_purity‚0 ( Purity/2FRt*SRqqt3FBt/FRt_purity‚0.5 9"Purity/0FRt*SRqqt3FBt/SRqqt_purity‚0.6666666269302368 ("Purity/2FRt2SRqqt2FBt/event_purity‚0 ("Purity/4FRt0SRqqt0FBt/event_purity‚0 *loss/val_total_loss‚45.136220755226944 :"Purity/0FRt3SRqqt0FBt/SRqqt_purity‚0.29206347465515137 ("Purity/1FRt2SRqqt2FBt/event_purity‚0 ) Purity/1FRt1SRqqt4FBt/FBt_purity‚0.75 =&Purity/*FRt1SRqqt*FBt/event_proportion‚0.4482736885547638 =&Purity/1FRt*SRqqt*FBt/event_proportion‚0.4366179406642914 8 Purity/1FRt2SRqqt0FBt/FRt_purity‚0.20098847150802612 @&Purity/1FRt4SRqqt*FBt/event_proportion‚0.0003780241822823882 @&Purity/3FRt2SRqqt0FBt/event_proportion‚0.0024571572430431843 +"Purity/3FRt4SRqqt0FBt/SRqqt_purity‚0.25 ( Purity/*FRt1SRqqt3FBt/FRt_purity‚0.5 ("Purity/1FRt1SRqqt3FBt/event_purity‚0 particle/accuracy_3_of_5‚1 particle/accuracy_7_of_7‚0 9"Purity/0FRt*SRqqt*FBt/event_purity‚0.4011649191379547 @&Purity/1FRt0SRqqt3FBt/event_proportion‚0.0010710685746744275 >&Purity/2FRt*SRqqt*FBt/event_proportion‚0.10244455933570862 8 Purity/2FRt0SRqqt*FBt/FRt_purity‚0.20549677312374115 ?&Purity/2FRt0SRqqt2FBt/event_proportion‚0.003465221729129553 ,&Purity/3FRt2SRqqt2FBt/event_proportion‚0 ,&Purity/4FRt1SRqqt1FBt/event_proportion‚0 +jet/accuracy_2_of_2‚0.09117655456066132 7 Purity/1FRt0SRqqt1FBt/FRt_purity‚0.2979013919830322 *jet/accuracy_1_of_4‚0.8387826681137085 8"Purity/0FRt1SRqqt3FBt/SRqqt_purity‚0.800000011920929 6 Purity/3FRt3SRqqt*FBt/FRt_purity‚0.190476194024086 ,&Purity/0FRt1SRqqt4FBt/event_proportion‚0 ("Purity/2FRt1SRqqt0FBt/event_purity‚0 ,&Purity/2FRt3SRqqt2FBt/event_proportion‚0 @&Purity/3FRt0SRqqt0FBt/event_proportion‚0.0015120967291295528 ("Purity/4FRt1SRqqt*FBt/event_purity‚0 ("Purity/*FRt2SRqqt3FBt/event_purity‚0 ,&Purity/*FRt4SRqqt4FBt/event_proportion‚0 @&Purity/0FRt2SRqqt2FBt/event_proportion‚0.0010080644860863686 :"Purity/1FRt3SRqqt*FBt/SRqqt_purity‚0.30266255140304565 ,&Purity/2FRt1SRqqt3FBt/event_proportion‚0 *jet/accuracy_1_of_5‚0.9055039286613464 particle/accuracy_2_of_3‚1 ) Purity/4FRt1SRqqt*FBt/FRt_purity‚0.25 9"Purity/0FRt1SRqqt2FBt/SRqqt_purity‚0.4734920561313629 ,&Purity/1FRt4SRqqt2FBt/event_proportion‚0 ("Purity/3FRt0SRqqt2FBt/event_purity‚0 >&Purity/1FRt2SRqqt0FBt/event_proportion‚0.04731602966785431 7 Purity/3FRt0SRqqt1FBt/FRt_purity‚0.3205128014087677 ,&Purity/4FRt3SRqqt4FBt/event_proportion‚0 jet/accuracy_6_of_6‚0 & Purity/*FRt2SRqqt2FBt/FBt_purity‚1 :"Purity/1FRt*SRqqt2FBt/event_purity‚0.09017863124608994 7 Purity/3FRt1SRqqt1FBt/FRt_purity‚0.3205128014087677 ,&Purity/4FRt1SRqqt3FBt/event_proportion‚0 ,&Purity/4FRt4SRqqt3FBt/event_proportion‚0 7 Purity/*FRt4SRqqt*FBt/FRt_purity‚0.1666666716337204 9"Purity/*FRt*SRqqt*FBt/event_purity‚0.2064225971698761 7 Purity/*FRt1SRqqt1FBt/FBt_purity‚0.9994532465934753 9"Purity/3FRt2SRqqt*FBt/SRqqt_purity‚0.3402777910232544 @&Purity/4FRt2SRqqt0FBt/event_proportion‚6.300403038039804e-05 9"Purity/*FRt3SRqqt2FBt/SRqqt_purity‚0.3333333432674408 & Purity/1FRt3SRqqt2FBt/FRt_purity‚1 trainer/global_step‚2415 8 Purity/1FRt*SRqqt*FBt/FRt_purity‚0.20915307104587555 7 Purity/2FRt*SRqqt*FBt/FBt_purity‚0.9961161017417908 ?&Purity/2FRt2SRqqt*FBt/event_proportion‚0.019468246027827263 ("Purity/3FRt3SRqqt0FBt/event_purity‚0 7 Purity/*FRt*SRqqt3FBt/FBt_purity‚0.9204980731010437 <"Purity/1FRt2SRqqt*FBt/event_purity‚0.0011123470030725002 & Purity/2FRt*SRqqt1FBt/FBt_purity‚1 9"Purity/0FRt3SRqqt1FBt/SRqqt_purity‚0.4583333134651184 8 Purity/2FRt1SRqqt0FBt/FRt_purity‚0.17747142910957336 6 Purity/*FRt0SRqqt3FBt/FBt_purity‚0.929563581943512 7 Purity/*FRt1SRqqt*FBt/FBt_purity‚0.9920945763587952 >&Purity/0FRt2SRqqt*FBt/event_proportion‚0.05361643061041832 7 Purity/1FRt1SRqqt*FBt/FRt_purity‚0.2220952957868576 @&Purity/1FRt1SRqqt4FBt/event_proportion‚6.300403038039804e-05 ;"Purity/1FRt2SRqqt1FBt/event_purity‚0.006451612804085016 8"Purity/0FRt3SRqqt*FBt/SRqqt_purity‚0.299263060092926 7 Purity/*FRt1SRqqt1FBt/FRt_purity‚0.2862626314163208 particle/accuracy_3_of_6‚1 ("Purity/4FRt*SRqqt0FBt/event_purity‚0 ("Purity/4FRt2SRqqt*FBt/SRqqt_purity‚0 & Purity/0FRt2SRqqt1FBt/FBt_purity‚1 :"Purity/2FRt3SRqqt*FBt/SRqqt_purity‚0.23025399446487427 8 Purity/3FRt2SRqqt*FBt/FRt_purity‚0.17129629850387573 _step‚195 :"Purity/*FRt1SRqqt0FBt/event_purity‚0.17966404557228088 9"Purity/*FRt2SRqqt*FBt/SRqqt_purity‚0.3111834228038788 & Purity/0FRt2SRqqt*FBt/FBt_purity‚1 :"Purity/1FRt*SRqqt*FBt/SRqqt_purity‚0.32543447613716125 6 Purity/2FRt*SRqqt1FBt/FRt_purity‚0.324860543012619 ,&Purity/3FRt*SRqqt3FBt/event_proportion‚0 @&Purity/4FRt1SRqqt0FBt/event_proportion‚6.300403038039804e-05 8 Purity/*FRt2SRqqt0FBt/FRt_purity‚0.19230587780475616 ("Purity/0FRt0SRqqt0FBt/event_purity‚1 ;"Purity/0FRt2SRqqt0FBt/event_purity‚0.027675585821270943 =&Purity/1FRt*SRqqt0FBt/event_proportion‚0.3135710656642914 :"Purity/2FRt0SRqqt2FBt/event_purity‚0.02500000037252903 ,jet/accuracy_4_of_4‚0.004436581861227751 particle/accuracy_1_of_1‚1 ("Purity/0FRt2SRqqt3FBt/SRqqt_purity‚0 @&Purity/*FRt3SRqqt2FBt/event_proportion‚6.300403038039804e-05 ?&Purity/0FRt0SRqqt2FBt/event_proportion‚0.029674898833036423 ?&Purity/3FRt1SRqqt*FBt/event_proportion‚0.003717237850651145 & Purity/3FRt1SRqqt*FBt/FBt_purity‚1 ,&Purity/3FRt4SRqqt4FBt/event_proportion‚0 & Purity/4FRt0SRqqt0FBt/FRt_purity‚0 7 Purity/1FRt0SRqqt3FBt/FRt_purity‚0.3928571343421936 8 Purity/2FRt2SRqqt*FBt/FRt_purity‚0.20954884588718414 & Purity/3FRt0SRqqt*FBt/FBt_purity‚1 ,&Purity/3FRt4SRqqt3FBt/event_proportion‚0 @&Purity/*FRt4SRqqt1FBt/event_proportion‚6.300403038039804e-05 7 Purity/2FRt1SRqqt2FBt/FBt_purity‚0.9868420958518982 & Purity/3FRt0SRqqt1FBt/FBt_purity‚1 ,&Purity/3FRt3SRqqt4FBt/event_proportion‚0 8 Purity/2FRt4SRqqt0FBt/FRt_purity‚0.30000001192092896 9"Purity/1FRt3SRqqt2FBt/SRqqt_purity‚0.3333333432674408 ?&Purity/1FRt3SRqqt*FBt/event_proportion‚0.009135584346950054 & Purity/3FRt1SRqqt1FBt/FBt_purity‚1 ,&Purity/3FRt2SRqqt3FBt/event_proportion‚0 0"Purity/*FRt4SRqqt0FBt/SRqqt_purity‚ 0.2734375 particle/accuracy_5_of_6‚1 7 Purity/0FRt*SRqqt2FBt/FBt_purity‚0.9887910485267639 @&Purity/1FRt3SRqqt2FBt/event_proportion‚6.300403038039804e-05 ,&Purity/1FRt3SRqqt4FBt/event_proportion‚0 ("Purity/*FRt2SRqqt2FBt/event_purity‚0 & Purity/4FRt2SRqqt*FBt/FRt_purity‚0 8 Purity/*FRt3SRqqt0FBt/FRt_purity‚0.21434710919857025 9"Purity/*FRt3SRqqt0FBt/SRqqt_purity‚0.2823820114135742 7 Purity/1FRt0SRqqt*FBt/FBt_purity‚0.9950029253959656 @&Purity/3FRt2SRqqt*FBt/event_proportion‚0.0028351815417408943 jet/accuracy_6_of_7‚0 ("Purity/1FRt4SRqqt*FBt/event_purity‚0 & Purity/1FRt0SRqqt1FBt/FBt_purity‚1 ;"Purity/2FRt*SRqqt0FBt/event_purity‚0.001638504909351468 @&Purity/2FRt4SRqqt0FBt/event_proportion‚0.0003150201519019902 & Purity/1FRt2SRqqt2FBt/FBt_purity‚1 @&Purity/1FRt4SRqqt0FBt/event_proportion‚0.0003780241822823882 ,&Purity/4FRt2SRqqt2FBt/event_proportion‚0 ("Purity/2FRt2SRqqt*FBt/event_purity‚0 particle/accuracy_1_of_6‚1 & Purity/1FRt*SRqqt4FBt/FRt_purity‚0 9"Purity/*FRt0SRqqt1FBt/event_purity‚0.6694325804710388 7 Purity/0FRt1SRqqt1FBt/FBt_purity‚0.9989919066429138 ?&Purity/0FRt2SRqqt0FBt/event_proportion‚0.043283771723508835 ;"Purity/2FRt0SRqqt0FBt/event_purity‚0.007540846709161997 @&Purity/3FRt1SRqqt1FBt/event_proportion‚0.0009450604557059705 ("Purity/4FRt0SRqqt*FBt/event_purity‚0 ?&Purity/*FRt*SRqqt3FBt/event_proportion‚0.004473286215215