Live interaction with the 3D Score-CAM result.
Intepretability is one of the concerns regarding the application of AI, or, to be exact, deep learning, in the medical field, especially medical image recognition. In a venture seeking to explain what is going on or what the network perceives computationally, one can leverage the class activation map (CAM) of the model. Score-CAM, one of the CAM variants, breakthroughs the preceding CAM methods by dropping the reliance on gradients. Instead, it benefits the full potential of the forward propagation of the model, running inference by normalized-weighting via element-wise product with the input. Then, the output logit of the target category is combined with the CAM acquired before to get the final outcome. In this project, the 3D version of ResNet is employed. To evaluate the aforementioned methods, the OrganMNIST3D dataset of MedMNIST is used. The deletion area under the curve (DAUC) and the insertion area under the curve (IAUC) are adopted to measure the performance of Score-CAM.
Catch up on this link to comprehend the training, testing, and visualization pertaining to this project.
The table below exhibits the outcome, quantitatively.
| Test Metric | Score |
|---|---|
| Loss | 0.570 |
| Accuracy | 89.79% |
Accuracy curves of 3D ResNet on the train and validation sets.
Loss curves of 3D ResNet on the train and validation sets.
Overall DAUC and IAUC scores of the Score-CAM on the 4th layer of 3D ResNet:
| Test Metric | Score |
|---|---|
| DAUC | 0.2576 ± 0.1710 |
| IAUC | 0.6750 ± 0.1944 |
The following are snapshots of the individual DAUC and IAUC scores and their Score-CAM outcomes.
The result of the bladder.
The result of the heart.
The result of the left kidney.
The result of the right kidney.
The result of the left lung.
The result of the right lung.
The result of the spleen.
To cite this repository:
@misc{3dviz-scorecam,
title = {Visualizing 3D ResNet for Medical Image Classification With Score-CAM},
url = {https://github.com/reshalfahsi/3d-viz-score-cam},
author = {Resha Dwika Hefni Al-Fahsi},
}
- Learning Spatio-Temporal Features with 3D Residual Networks for Action Recognition
- Score-CAM: Score-Weighted Visual Explanations for Convolutional Neural Networks
- RISE: Randomized Input Sampling for Explanation of Black-box Models
- RISE official code
- K3D Jupyter
- Interpretable Explanations of Black Boxes by Meaningful Perturbation
- Score-CAM official code
- MedMNIST v2-A large-scale lightweight benchmark for 2D and 3D biomedical image classification
- MedMNIST official web page
- MedMNIST official code
- PyTorch Lightning