My CPU loaded on 89% but GPU not used

[koljanich]

Hello! I try to use yolov3.pt tiny-yolov3.pt and retinanet_resnet50_fpn_coco-eeacb38b.pth. But result is same. CPU Usage 89%, GPU on minimal 18%. I use 1660 Super GPU from NVIDIA. Script:
`# -- coding: utf-8 --
import cv2
import os
from imageai.Detection import ObjectDetection

camera = cv2.VideoCapture(1)
execution_path = os.getcwd()

detector = ObjectDetection()
detector.setModelTypeAsTinyYOLOv3()
detector.setModelPath(os.path.join(execution_path, "tiny-yolov3.pt"))
detector.loadModel()

while True:

ret, frame = camera.read()


detections = detector.detectObjectsFromImage(input_image=frame)


for detection in detections:
    print(detection["name"], " : ", detection["percentage_probability"])
    print(detection["box_points"])
    cv2.rectangle(frame, detection["box_points"][0:2], detection["box_points"][2:4], (0, 255, 0), 2)
    cv2.putText(frame, detection["name"], (detection["box_points"][0], detection["box_points"][1] - 10), cv2.FONT_HERSHEY_SIMPLEX, 0.5, (0, 255, 0), 2)

cv2.imshow("Frame", frame)
key = cv2.waitKey(1) & 0xFF


if key == ord("q"):
    break

camera.release()
cv2.destroyAllWindows()`

[koljanich]

One more: os Windows 10. CUDA and tensorflow installed by instruction https://www.tensorflow.org/install/pip?hl=en#windows-native. Verification python3 -c "import tensorflow as tf; print(tf.config.list_physical_devices('GPU'))" returns [PhysicalDevice(name='/physical_device:GPU:0', device_type='GPU')]

[Haicaji]

Do you use an mirror image to install the library? If you use a Tsinghua image to install, the PyTorch you installed is a CPU version.

[Haicaji]

Please go to the official website to download the corresponding version.

[mdowst]

I ran into a similar problem using Windows 11 with a NVIDIA GeForce RTX 3080 Ti. Even though the checks would show my GPU, it would only use CPU when processing a video. I ran the nvidia-smi.exe in a command prompt and it showed I was using cuda version 12.2. At this time Pytorch only supports up to version 12.1. To resolve this, I downgraded my driver to 12.1 and it works using the GPU.

These steps assume you have Anaconda installed.

1. Uninstall all Nvidia cuda versions and display drivers. (This required a couple of reboots.)
2. Downloaded CUDA Toolkit 12.1
3. Installed CUDA Toolkit 12.1, this will also include the display drivers.
4. Open PowerShell or command prompt, run nvidia-smi.exe and confirm you CUDA version. (I am using CUDA Version 12.1. My drive version is 531.14.)
5. Installed the CUDA 12.1 toolkit in conda
   conda install cuda --channel nvidia/label/cuda-12.1.0
6. Create new conda environment with Python 3.9
   conda create --name imageai python=3.9
7. Active the new environment
   conda activate imageai
8. Install PyTorch
   pip3 install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu121
9. Navigate to the folder where you downloaded the ImageAI repository
10. Open requirements_gpu.txt and change the URL on the torch lines so the end in cu121. (This may not be needed because torch should already be installed from the previous steps, but it made me feel safer)
11. Install the GPU requirements
    pip install -r requirements_gpu.txt
12. Install ImageAI
    pip install imageai --upgrade

Thanks to Nilotpal Sinha PhD. His article on Medium helped me figure this out. - How to setup PyTorch with CUDA in Windows 11