This is the repository for pointcloud grad-CAM (p-grad-CAM)[8], which was originally inspired by the Gradient Class Activation Mapping (Grad-CAM)[1] approach. It was implemented utlizing the Pointnet[2] network and Tensorflow. For testing the validity of our approach it was compared to a similar algorithm, called Saliency Maps[7]. This algorithm was slightly rewritten to fit our testing environent and was renamed ASM, it is also included in this repository.
The goal of this application is to yield a similar grade of transparency for pointcloud neuronal networks as grad-CAM does for 2D image convolutional neuronal networks (CNN). Usually the user is granted no deeper insight as of why a current input yields a particular results delivered by the network. This is particularly an issue if the output does not yield the expected result. In order to increase the confidence and trust in the performance of neuronal networks it is of virtue to be able to understand the reasoning behind the networks decisions.
We seek to achieve this transparency by finding the salient regions of the input pointcloud in order to be able to identify the areas of interest for the underlying neuronal network. This should yield a visual representation of the important areas that were responsible for the predicion result of the network. The pointcloud data is then visualized using the Open3D[4] library.
To run the the code Python 3.5[5] or better and Anaconda[6] is required. To use this algorithm follow the next steps:
-
Open the Anaconda prompt and
cdinto the cloned repo directory. -
Create the environment with all necessary dependencies with the following command:
conda env create -f "..\Pointcloud-grad-CAM\Anaconda\p-grad-CAM.yml"
- Activate the virtual environment by typing the following command:
conda activate p-grad-CAM
- Train the network:
python train.py
- Run either p-grad-CAM or ASM on it:
python p_grad_CAM.py or python ASM.py
Identifying salient regions using point cloud gradient Class Activation Mapping
Dennis Struhs
Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localization
Ramprasaath R. Selvaraju, Abhishek Das, Ramakrishna Vedantam, Michael Cogswell, Devi Parikh, Dhruv Batra
https://arxiv.org/abs/1610.02391
PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation
Charles R. Qi, Hao Su, Kaichun Mo, Leonidas J. Guibas
https://arxiv.org/abs/1610.02391
Saliency Maps: Learning deep features for discriminative localization
Tianhang Zheng, Changyou Chen, Junsong Yuan, Kui Ren
http://arxiv.org/abs/1812.01687