MONE2022-experiments

Copyright © 2018-2019, 2022-2023 Vincent A. Cicirello

This repository contains code to reproduce the experiments, and analysis of experimental data, from the following paper:

Vincent A. Cicirello. 2023. On Fitness Landscape Analysis of Permutation Problems: From Distance Metrics to Mutation Operator Selection. Mobile Networks and Applications 28(2): 507-517, April 2023. doi:10.1007/s11036-022-02060-z

The above article is an extended version the following earlier conference paper:

Vincent A. Cicirello. 2019. Classification of Permutation Distance Metrics for Fitness Landscape Analysis. In Proceedings of the 11th International Conference on Bio-inspired Information and Communication Technologies, pages 81-97. Springer Nature, March 2019. doi:10.1007/978-3-030-24202-2_7

Related Publications
License
Packages and Releases

Requirements to Build and Run the Experiments

To build and run the experiments on your own machine, you will need the following:

• JDK 11: I used OpenJDK 11, but you should be fine with Oracle's JDK as well.
• Apache Maven: In the root of the repository, there is a pom.xml for building the Java programs for the experiments. Using this pom.xml, Maven will take care of downloading the exact versions of Chips-n-Salsa (release 4.2.1), JavaPermutationTools (release 3.0.0), and ρμ (release 1.1.0) that were used in the experiments.
• Python 3: The repository contains Python programs that were used to compute summary statistics, and to generate graphs for the figures of the paper. If you want to run the Python programs, you will need Python 3. I specifically used Python 3.9.6. You also need
  matplotlib installed.
• Make: The repository contains a Makefile to simplify running the build, running the experiment's Java programs, and running the Python program to analyze the data. If you are familiar with using the Maven build tool, and running Python programs, then you can just run these directly, although the Makefile may be useful to see the specific commands needed.

Building the Java Programs (Option 1)

The source code of the Java programs, implementing the experiments is in the src/main directory. You can build the experiment programs in one of the following ways.

Using Maven: Execute the following from the root of the repository.

mvn clean package

Using Make: Or, you can execute the following from the root of the repository.

make build

This produces a jar file containing 6 Java programs for running different parts of the experiments and analysis. The jar also contains all dependencies, including Chips-n-Salsa, JavaPermutationTools, and ρμ. If you are unfamiliar with the usual structure of the directories of a Java project built with Maven, the .class files, the .jar file, etc will be found in a target directory that is created by the build process.

Downloading a prebuilt jar (Option 2)

As an alternative to building the jar (see above), you can choose to instead download a prebuilt jar of the experiments from the Maven Central repository. The Makefile contains a target that will do this for you, provided that you have curl installed on your system. To download the jar of the precompiled code of the experiments, run the following from the root of the repository:

make download

The jar that it downloads contains the compiled code of the experiments as well as all dependencies within a single jar file.

Running the Experiments

If you just want to inspect the data from my runs, then you can find that output in the /data directory. If you instead want to run the experiments yourself, you must first either follow the build instructions or download a prebuilt jar (see above sections). Once the jar of the experiments is either built or downloaded, you can then run the experiments with the following executed at the root of the repository:

make experiments

If you don't want to overwrite my original data files, then first change the variable pathToDataFiles in the Makefile before running the above command.

This will run each of the experiment programs in sequence, with the results piped to text files in the /data directory. Note that this directory already contains the output from my runs, so if you execute this command, you will overwrite the data that was used in the paper. Some parts of this will not change, but certain parts, due to randomization may not be exactly the same, although should be statistically consistent.

There are also several other targets in the Makefile if you wish to run only some of the experiments from the paper. See the Makefile for details.

Analyzing the Experimental Data

To run the Python program that I used to generate summary statistics,
and generate the graphs for the figures from the paper, you need Python 3 installed. The source code of the Python programs is found in the src/analysis directory. To run the analysis execute the following at the root of the repository:

make analysis

If you don't want to overwrite my original data files, and figures, then change the variable pathToDataFiles in the Makefile before running the above command.

This will analyze the data from the /data directory. It will also generate the figures in that directory, as well as output a few txt files with summary statistics into that directory. This make command will also take care of installing any required Python packages if you don't already have them installed, such as matplotlib.

To convert the eps versions of the figures to pdf, then after running the above analysis, run the following (this assumes that you have epstopdf installed):

make epstopdf

Running the Principle Component Analysis and Fitness Distance Correlation Examples

The above Makefile targets only runs and analyzes the experiments that originated with the paper for MONE. That paper is an extended version of a conference paper from the BICT 2019 conference. A couple sections of the MONE paper originated in that BICT 2019 paper, including sections that classified the various distance metrics for permutations using principal component analysis, along with a few examples of fitness distance correlation.

To run the code to recreate that, use the following Makefile target:

make bict2019

Other Files in the Repository

There are a few other files, potentially of interest, in the repository, which include:

• system-stats.txt: This file contains details of the system I used to run the experiments, such as operating system, processor specs, Java JDK and VM. It is in the /data directory.

License

The code to replicate the experiments from the paper, as well as the Chips-n-Salsa, JavaPermutationTools, and ρμ libraries are licensed under the GNU General Public License 3.0.