Low GPU Usage during inference

[aakashshankar]

I've used the 7_yolov3 to train a model for real time hand tracking. To infer, I'm using NVIDIA 1060 Max Q GPU (6GB VRAM). Even after installing the CUDA toolkit and CUDA Apex, there is only 20% usage of the GPU whereas 80% of the CPU is being used.
I'm getting <10 FPS while loading the webcam using the yolov3-tiny model.
Is there a way to maximize GPU usage to get higher frame rates?

[abhi-kumar]

For inference, the model may not use the entire GPU, and the CPU usage maybe for data loading and transfer to GPU.

Thanks for pointing out the issue that the FPS is staying quite low, our team will check it here and get back to you as soon as possible.