A summarized version of the research is available in the PDF file.
This project presents a novel approach for optimizing the architecture of symmetric, undercomplete autoencoders for dimensionality reduction using a genetic algorithm. The approach aims to find the optimal combination of the number of hidden layers, neurons per layer, activations, and optimizers for the autoencoder, with the goal of achieving good reconstruction quality as measured by the reconstruction loss. The proposed method is evaluated on a 16x16 MNIST dataset, and results show that the genetic algorithm-based approach is effective in finding suitable architectures for the autoencoder.
Autoencoders are a type of neural network that is commonly used for dimensionality reduction, feature extraction, anomaly detection, and data reconstruction. They consist of an encoder that maps the input to a lower-dimensional code, and a decoder that reconstructs the input from the code (Fig 1). In this research, I propose a genetic algorithm-based approach for optimizing the architecture of symmetric, undercomplete autoencoders for dimensionality reduction, with desired code size. Genetic algorithms are search algorithms inspired by natural selection processes and are commonly used for optimization problems. They work by iteratively generating and selecting a population of potential solutions, known as individuals or chromosomes. In the context of neural network design, genetic algorithms can be used to automatically search for the optimal architecture and hyperparameters of a model.
In my proposed approach, I have used a microbial genetic algorithm to search for the optimal architecture of symmetric autoencoders. The genotype, or genetic representation, of each individual in the population consists of the number of hidden layers, the number of neurons per layer, and the choice of activation and optimization functions. The fitness of each individual is determined by the reconstruction loss, which measures how well the input data is reconstructed 1 from the code. The genetic algorithm mutates, crosses over, and applies other biologically-inspired operators to find an optimal combination of these parameters for the autoencoder. The resulting autoencoder is expected to achieve good reconstruction quality, as measured by the reconstruction loss. The training and evaluation of the autoencoder are done on a 16x16 MNIST dataset, for increasing the training speed. In the end, I was able to find an optimal architecture, which can compress the MNIST data set to an array with a length of 4 and reconstruct the original data with a reasonable loss.
Additionally, the proposed approach also allows for the specification of the desired code size, enabling the user to control the level of dimensionality reduction. This is particularly useful in applications where the amount of data reduction required is known in advance. Furthermore, the use of a symmetric architecture in the autoencoder ensures that the encoder and decoder have the same structure, making the model more robust and easier to train.
In conclusion, the proposed approach of using a microbial genetic algorithm to optimize the architecture of symmetric, undercomplete autoencoders for dimensionality reduction, with desired code size, has been shown to be effective in finding optimal architectures that can achieve good reconstruction quality while reducing the dimensionality of the input data. This approach can be applied to other datasets and applications where dimensionality reduction is needed, making it a valuable tool for data preprocessing and analysis.