Macau is one of the tools used in [1].
Macau consists of two parts, the data generation and the statitical hypothesis testing.
To generate data, use data_generator.py script. Use -h to see usage.
macau.py uses resampling to compute test statistics and p-values.
Usage:
python3 macau.py <data_path> <permutations> <hypotheses_path>The data_path is a data file output by data_generator.
The permutations is number of permutations.
The hypotheses_path is a file containing hypotheses that are formatted using the encoding scheme intoduced in [1].
[1] Zgraggen, Zhao, Zeleznik and Kraska, Investigating the Effect of the Multiple Comparison Problem in Visual Analysis, CHI 2018.