Specifying and executing multiverse analysis

A Python library to determine what exactly the user ran at the command line, along with default values, and any transformations that happened after parsing arguments.

Implementation of metaheuristic optimization methods in Python for scientific, industrial, and educational scenarios. Experiments can be executed in parallel or in a distributed fashion. Experimental results can be evaluated in various ways, including diagrams, tables, and export to Excel.

Keep track of your results

TriScale software

Resources on the Artifact Evaluation (AE) Process

A Python Interface to Reproducibility Measures of System-Oriented IR Experiments

A framework-agnostic deep learning package and experiment manager

Special Issue: The Potential of Notebooks for Scientific Publication: Reproducibility, and Dissemination

zcurve R package for assessing the reliability and trustworthiness of published literature with the z-curve method

Position Paper for the J-STARS Special Issue on Reproducible Research in Remote Sensing

Code repository accompanying "Assessing data availability and research reproducibility in hydrology and water resources"

Git copy of our Overleaf project for the preprint of Nine Best Practices for Research Software Registries and Repositories: A Concise Guide

Materials for the MIT PML workshop on Good coding practices for social scientists.

Wide Residual Networks in Keras and PyTorch

[ECIR 2022] Automation of Citation Screening for Systematic Literature Reviews Using Neural Networks: A Replicability Study

A curated list of publicly available datasets for replication studies.

Repository for R package and website of replication diagnosis tools

radjust: Replicability Adjusted p-values for Two Independent Studies with Multiple Endpoints

conducting the decision tree for capabilities of frameworks, that assist in reproducing results in Machine Learning: wandb, guild.ai and mlflow