Abnormal training progress

[yunchu]

Describe the bug

While running a regression test for instance-segmentation, I've met the training progress exceed 100%.

Steps to Reproduce

1. prepare dataset for running regression test
2. run a test
   CI_DATA_ROOT=<regression-dataset-root-path> tests-iseg-py310-pt1 -- tests/regression/instance_segmentation/test_instance_segmentation.py::TestRegressionInstanceSegmentation::test_otx_train[Custom_Counting_Instance_Segmentation_MaskRCNN_SwinT_FP16]
3. monitor displayed log

Environment:

• OS: Ubuntu 22.04 WSL
• Framework version: torch 1.13
• Python version: 3.10
• OpenVINO version: 2022.3
• CUDA/cuDNN version: 11.7.1
• GPU model and memory: 3090 / 24G

[goodsong81]

Can we close this issue? @sungmanc might have fixed this issue but need to check if it's applied to all release branches.

[Fxisxl]

Can we close this issue? @sungmanc might have fixed this issue but need to check if it's applied to all release branches.

Is this issue solved ????

[sungmanc]

Can we close this issue? @sungmanc might have fixed this issue but need to check if it's applied to all release branches.

Is this issue solved ????

In our tests , it seems that this was resolved. Found anything?