mlkit-bench

This repository provides tools and benchmark programs for the purpose of benchmarking MLKit executables across versions (and configurations) and against MLton-generated executables.

Here is an overview of the tools in this repository:

• mlkit-bench. This tool is what actually coordinates compilation and execution of benchmark programs. The output of the command is a json-file, which can then be processed by mlkit-bench-press or mlkit-charting.

• mlkit-bench-press. This tool allows for inspection and querying of the json-file generated by mlkit-bench.

• mlkit-charting. This tool is useful for charting the MLKit performance on benchmarks over time. The tool makes use of smltojs to create a web-site that allows for investing the performance characteristics of various versions of MLKit. The web-site is hosted at Github.

Notice: A useful setup is to have the MLKit sources checked-out at a sister-repository:

~/gits/mlkit-bench/
~/gits/mlkit/

Benchmark programs

Currently, there are two sets of benchmarks:

1. Sequential benchmarks. These benchmarks feature a variety of Standard ML programs, ranging over small micro-benchmarks to more involved benchmarks.

2. Parallel benchmarks. These benchmarks make use of the parallel threading possibilities of MLKit, which features a fork-join model for parallism combined with region-based memory management.