Common Test 1.Multi-Class Classification

The task is to build a model that can identify different types of lensing images, which are images of distant galaxies that are distorted by the gravity of a massive object in front of them.

The model should be able to distinguish between images that have no substructure, images that have subhalo substructure, which are small clumps of dark matter, and images that have vortex substructure.

Dataset

The Dataset Consists of 30k train images and 3k test images, all of size 1x150x150

Note: The Original Directories names are train for the 30k one and val for the 3k one, but since we used it for testing we will call it test for the rest of the document.

Data Preprocessing

The data is resized and the channels are duplicaetd to make it 3 channels

Evaluation Criteria

The model should be evaluated by how well it can separate the classes using a ROC curve and an AUC score, which measure the trade-off between true positive and false positive rates.

Approach

The task of classifying lensing images can benefit from trying different architectures that trade-off between accuracy and latency.

MobileNetV3 small is a lightweight model that uses depth-wise convolutions, squeeze and excitation modules, and h-swish activation to achieve high accuracy with low latency.
Resnet18 is a deeper model that uses residual connections, batch normalization, and ReLU activation to learn complex features with moderate latency.
EfficientNetb5 is a scalable model that uses compound scaling, depth-wise separable convolutions, and swish activation to achieve state-of-the-art accuracy with high latency. Comparing these architectures can help to find the optimal balance between performance and resource consumption for the task.

Results

Summary

The Following table shows the results of the models on the test set (3k images)

Model	Validation Loss	Validation Accuracy	Validation ROC AUC
EfficientNet b5	0.1710	95.88%	0.9931
Resnet18	0.2997	93.63%	0.9871
MobileNetV3_s	0.3270	88.67%	0.9741

ROC Curves

EfficientNet b5
Resnet18
MobileNetV3_s|

Confusion Matrices

EfficientNet b5
Resnet18
MobileNetV3_s

Checkpoints

you can find the checkpoints of the 3 models here

Conclusion

EfficientNet b5 b5 model is the best model for this task, it has the highest accuracy and the highest ROC AUC score, but it is also the most computationally expensive model.
Resnet18 model is the second best model, it has a lower accuracy and ROC AUC score, but it is also less computationally expensive.
MobileNetV3_s model is the worst model for this task, it has the lowest accuracy and ROC AUC score, but it is also the least computationally expensive model.

Specific Test III. Learning Mass of Dark Matter Halo

The task is to use machine learning to estimate the mass of dark matter halos around galaxies from images of gravitational lensing. Gravitational lensing is a phenomenon where the light from distant sources is bent by the gravity of massive objects in the foreground.

By analyzing the shape and distortion of the lensed images, one can infer the properties of the dark matter halos that cause the lensing.

Dataset

The Dataset consists of 20k images of size 1x150x150, and it has been splitted into 81:9:10 for train, validation, and test respectively.

Data Preprocessing

The data is resized to 3x224x224 for EfficientNet b5 only, however, horizontal and vertical flips are applied to the trained data randomly for all used models

Approach

The same Approach in Task1 is used here, but the model is trained on the log of the mass of the halo, and the loss function is the mean squared error (MSE)