Added a script to automatically compile all the cuda functions.

[odysseus0]

Here is how it works.

>  python compile_cuda_func.py -h
usage: compile_cuda_func.py [-h] -a ARCH

Compile CUDA functions for the current machine's GPU

optional arguments:
  -h, --help            show this help message and exit
  -a ARCH, --arch ARCH  The architecture code of this machine's GPU

It is a simple command-line Python script that automatically compiles all the CUDA functions given an arch code.

>  python compile_cuda_func.py --arch sm_75

Including CUDA code.
/home/odysseus/medicaldetectiontoolkit/cuda_functions/nms_2D
generating /tmp/tmppevdo6dr/_nms.c
setting the current directory to '/tmp/tmppevdo6dr'
running build_ext
building '_nms' extension
creating home
creating home/odysseus
creating home/odysseus/medicaldetectiontoolkit
creating home/odysseus/medicaldetectiontoolkit/cuda_functions
creating home/odysseus/medicaldetectiontoolkit/cuda_functions/nms_2D
creating home/odysseus/medicaldetectiontoolkit/cuda_functions/nms_2D/src
gcc -pthread -B /home/odysseus/miniconda3/envs/mdt/compiler_compat -Wl,--sysroot=/ -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -DWITH_CUDA -I/home/odysseus/miniconda3/envs/mdt/lib/python3.7/site-packages/torch/utils/ffi/../../lib/include -I/home/odysseus/miniconda3/envs/mdt/lib/python3.7/site-packages/torch/utils/ffi/../../lib/include/TH -I/home/odysseus/miniconda3/envs/mdt/lib/python3.7/site-packages/torch/utils/ffi/../../lib/include/THC -I/usr/local/cuda/include -I/home/odysseus/miniconda3/envs/mdt/include/python3.7m -c _nms.c -o ./_nms.o -std=c99
gcc -pthread -B /home/odysseus/miniconda3/envs/mdt/compiler_compat -Wl,--sysroot=/ -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -DWITH_CUDA -I/home/odysseus/miniconda3/envs/mdt/lib/python3.7/site-packages/torch/utils/ffi/../../lib/include -I/home/odysseus/miniconda3/envs/mdt/lib/python3.7/site-packages/torch/utils/ffi/../../lib/include/TH -I/home/odysseus/miniconda3/envs/mdt/lib/python3.7/site-packages/torch/utils/ffi/../../lib/include/THC -I/usr/local/cuda/include -I/home/odysseus/miniconda3/envs/mdt/include/python3.7m -c /home/odysseus/medicaldetectiontoolkit/cuda_functions/nms_2D/src/nms.c -o ./home/odysseus/medicaldetectiontoolkit/cuda_functions/nms_2D/src/nms.o -std=c99
gcc -pthread -B /home/odysseus/miniconda3/envs/mdt/compiler_compat -Wl,--sysroot=/ -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -DWITH_CUDA -I/home/odysseus/miniconda3/envs/mdt/lib/python3.7/site-packages/torch/utils/ffi/../../lib/include -I/home/odysseus/miniconda3/envs/mdt/lib/python3.7/site-packages/torch/utils/ffi/../../lib/include/TH -I/home/odysseus/miniconda3/envs/mdt/lib/python3.7/site-packages/torch/utils/ffi/../../lib/include/THC -I/usr/local/cuda/include -I/home/odysseus/miniconda3/envs/mdt/include/python3.7m -c /home/odysseus/medicaldetectiontoolkit/cuda_functions/nms_2D/src/nms_cuda.c -o ./home/odysseus/medicaldetectiontoolkit/cuda_functions/nms_2D/src/nms_cuda.o -std=c99
/home/odysseus/medicaldetectiontoolkit/cuda_functions/nms_2D/src/nms_cuda.c: In function ‘gpu_nms’:
/home/odysseus/medicaldetectiontoolkit/cuda_functions/nms_2D/src/nms_cuda.c:29:35: warning: initialization from incompatible pointer type [-Wincompatible-pointer-types]
   unsigned long long* mask_flat = THCudaLongTensor_data(state, mask);
                                   ^~~~~~~~~~~~~~~~~~~~~
/home/odysseus/medicaldetectiontoolkit/cuda_functions/nms_2D/src/nms_cuda.c:37:40: warning: initialization from incompatible pointer type [-Wincompatible-pointer-types]
   unsigned long long * mask_cpu_flat = THLongTensor_data(mask_cpu);
                                        ^~~~~~~~~~~~~~~~~
/home/odysseus/medicaldetectiontoolkit/cuda_functions/nms_2D/src/nms_cuda.c:40:39: warning: initialization from incompatible pointer type [-Wincompatible-pointer-types]
   unsigned long long* remv_cpu_flat = THLongTensor_data(remv_cpu);
                                       ^~~~~~~~~~~~~~~~~
/home/odysseus/medicaldetectiontoolkit/cuda_functions/nms_2D/src/nms_cuda.c:23:7: warning: unused variable ‘boxes_dim’ [-Wunused-variable]
   int boxes_dim = THCudaTensor_size(state, boxes, 1);
       ^~~~~~~~~
gcc -pthread -shared -B /home/odysseus/miniconda3/envs/mdt/compiler_compat -L/home/odysseus/miniconda3/envs/mdt/lib -Wl,-rpath=/home/odysseus/miniconda3/envs/mdt/lib -Wl,--no-as-needed -Wl,--sysroot=/ ./_nms.o ./home/odysseus/medicaldetectiontoolkit/cuda_functions/nms_2D/src/nms.o ./home/odysseus/medicaldetectiontoolkit/cuda_functions/nms_2D/src/nms_cuda.o /home/odysseus/medicaldetectiontoolkit/cuda_functions/nms_2D/src/cuda/nms_kernel.cu.o -o ./_nms.so
Including CUDA code.
/home/odysseus/medicaldetectiontoolkit/cuda_functions/nms_3D
generating /tmp/tmpb30tanu9/_nms.c
setting the current directory to '/tmp/tmpb30tanu9'
running build_ext
building '_nms' extension
creating home
creating home/odysseus
creating home/odysseus/medicaldetectiontoolkit
creating home/odysseus/medicaldetectiontoolkit/cuda_functions
creating home/odysseus/medicaldetectiontoolkit/cuda_functions/nms_3D
creating home/odysseus/medicaldetectiontoolkit/cuda_functions/nms_3D/src
gcc -pthread -B /home/odysseus/miniconda3/envs/mdt/compiler_compat -Wl,--sysroot=/ -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -DWITH_CUDA -I/home/odysseus/miniconda3/envs/mdt/lib/python3.7/site-packages/torch/utils/ffi/../../lib/include -I/home/odysseus/miniconda3/envs/mdt/lib/python3.7/site-packages/torch/utils/ffi/../../lib/include/TH -I/home/odysseus/miniconda3/envs/mdt/lib/python3.7/site-packages/torch/utils/ffi/../../lib/include/THC -I/usr/local/cuda/include -I/home/odysseus/miniconda3/envs/mdt/include/python3.7m -c _nms.c -o ./_nms.o -std=c99
gcc -pthread -B /home/odysseus/miniconda3/envs/mdt/compiler_compat -Wl,--sysroot=/ -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -DWITH_CUDA -I/home/odysseus/miniconda3/envs/mdt/lib/python3.7/site-packages/torch/utils/ffi/../../lib/include -I/home/odysseus/miniconda3/envs/mdt/lib/python3.7/site-packages/torch/utils/ffi/../../lib/include/TH -I/home/odysseus/miniconda3/envs/mdt/lib/python3.7/site-packages/torch/utils/ffi/../../lib/include/THC -I/usr/local/cuda/include -I/home/odysseus/miniconda3/envs/mdt/include/python3.7m -c /home/odysseus/medicaldetectiontoolkit/cuda_functions/nms_3D/src/nms.c -o ./home/odysseus/medicaldetectiontoolkit/cuda_functions/nms_3D/src/nms.o -std=c99
gcc -pthread -B /home/odysseus/miniconda3/envs/mdt/compiler_compat -Wl,--sysroot=/ -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -DWITH_CUDA -I/home/odysseus/miniconda3/envs/mdt/lib/python3.7/site-packages/torch/utils/ffi/../../lib/include -I/home/odysseus/miniconda3/envs/mdt/lib/python3.7/site-packages/torch/utils/ffi/../../lib/include/TH -I/home/odysseus/miniconda3/envs/mdt/lib/python3.7/site-packages/torch/utils/ffi/../../lib/include/THC -I/usr/local/cuda/include -I/home/odysseus/miniconda3/envs/mdt/include/python3.7m -c /home/odysseus/medicaldetectiontoolkit/cuda_functions/nms_3D/src/nms_cuda.c -o ./home/odysseus/medicaldetectiontoolkit/cuda_functions/nms_3D/src/nms_cuda.o -std=c99
/home/odysseus/medicaldetectiontoolkit/cuda_functions/nms_3D/src/nms_cuda.c: In function ‘gpu_nms’:
/home/odysseus/medicaldetectiontoolkit/cuda_functions/nms_3D/src/nms_cuda.c:29:35: warning: initialization from incompatible pointer type [-Wincompatible-pointer-types]
   unsigned long long* mask_flat = THCudaLongTensor_data(state, mask);
                                   ^~~~~~~~~~~~~~~~~~~~~
/home/odysseus/medicaldetectiontoolkit/cuda_functions/nms_3D/src/nms_cuda.c:37:40: warning: initialization from incompatible pointer type [-Wincompatible-pointer-types]
   unsigned long long * mask_cpu_flat = THLongTensor_data(mask_cpu);
                                        ^~~~~~~~~~~~~~~~~
/home/odysseus/medicaldetectiontoolkit/cuda_functions/nms_3D/src/nms_cuda.c:40:39: warning: initialization from incompatible pointer type [-Wincompatible-pointer-types]
   unsigned long long* remv_cpu_flat = THLongTensor_data(remv_cpu);
                                       ^~~~~~~~~~~~~~~~~
/home/odysseus/medicaldetectiontoolkit/cuda_functions/nms_3D/src/nms_cuda.c:23:7: warning: unused variable ‘boxes_dim’ [-Wunused-variable]
   int boxes_dim = THCudaTensor_size(state, boxes, 1);
       ^~~~~~~~~
gcc -pthread -shared -B /home/odysseus/miniconda3/envs/mdt/compiler_compat -L/home/odysseus/miniconda3/envs/mdt/lib -Wl,-rpath=/home/odysseus/miniconda3/envs/mdt/lib -Wl,--no-as-needed -Wl,--sysroot=/ ./_nms.o ./home/odysseus/medicaldetectiontoolkit/cuda_functions/nms_3D/src/nms.o ./home/odysseus/medicaldetectiontoolkit/cuda_functions/nms_3D/src/nms_cuda.o /home/odysseus/medicaldetectiontoolkit/cuda_functions/nms_3D/src/cuda/nms_kernel.cu.o -o ./_nms.so

[odysseus0]

@pfjaeger Let me know if you find it helpful.

[ivanwilliammd]

@pfjaeger Let me know if you find it helpful.

can nms and roi align run on RTX GPU?
since after compiling it using sm_75 it won't load on pytorch 0.41

[odysseus0]

Damn. That is something to know. My machine uses Titan RTX.

I still haven't finished preprocessing the data. So not sure it is going to give me hell as well.

Regardless, you could always compile it using another architecture code. It is an argument that you can adjust.

[ivanwilliammd]

Damn. That is something to know. My machine uses Titan RTX.

I still haven't finished preprocessing the data. So not sure it is going to give me hell as well.

Regardless, you could always compile it using another architecture code. It is an argument that you can adjust.

please tell me if you have some quirk to run it using sm75. Thx

[odysseus0]

@ivanwilliammd Have you finished data preprocessing? Did you use this repo?

[ivanwilliammd]

@ivanwilliammd Have you finished data preprocessing? Did you use this repo?

I use the master branch one, since v1.01 outdated

[odysseus0]

Hey @ivanwilliammd, can I give you a call and ask some questions about the way you deal with the pre-processing of the LIDC dataset, specifically the master branch of this repo? I had a lot of troubles trying to get them to work.

[ivanwilliammd]

Hey @ivanwilliammd, can I give you a call and ask some questions about the way you deal with the pre-processing of the LIDC dataset, specifically the master branch of this repo? I had a lot of troubles trying to get them to work.

Hello @odysseus0
For preprocessing dataset using LIDC repo, I face some problem when processing it on Ubuntu, and it was solved when I preprocess it using Windows10 with python 3.6 (+ other pip library specified without installing conda),

The python library needed is SimpleITK and NumPy
pip install SimpleITK
pip install numpy

3rd party apps used are in this link : http://mitk.org/wiki/Phenotyping --> download it, and then find the bin folder and reroute the directory in code with your directory

For the LIDC dataset you need to download it using TCIA with CLASSIC DIRECTORY (not descriptive one), since it has different directory naming

[odysseus0]

I see. I tried with Ubuntu today and it really did not work out. I guess I will move to Windows 10.

I will keep you updated.