How to review research software in archaeology?
The aim of this poster is to stimulate discussion about the content and objectives of research software reviews in archaeology and ancient studies.
The assessment of a text publication is a traditional part of academic discourse. Hence, there is an unspoken understanding of what a review should include. However, in regard to software, tools and small helpers (Minions) there is no such common ground and instead one is confronted by a variety of challenges, like for example: What constitutes a good review of software intended for use in archaeological research? Do aspects of usability, sustainability and interoperability need to be considered? What role do technical and legal aspects play in the discussion of source code documentation or licensing? This leads to the overall questions: Does the software itself perhaps represent a scientific contribution? Which achievements of the software developers should be considered in a review?
The poster focusses on a list of questions to help with the critical evaluation of software. These questions are bundled in three sections, which align different areas of competence and knowledge. The first two sections deal with the scientific field of application, as well as the utilisation and usability from the user's point of view. The third section focuses on questions that are particularly relevant for developers and IT administrators. The result is a catalogue of questions, that entails a ranking of questions we suggest as „very important“ or mandatory, as „important“ and as „inspiring“. The ranking is the result of a collaborative and open writing exercise on Github and a final poll among the authors, who present together a broad range of experts in the field of computational archaeology, software-engineering, IT-administration and archaeological research (Homburg et al. 2021).
In modern research, it is unthinkable to work without digital tools. This also applies to historical, classical and ancient studies. With the rise of the digital humanities, software is increasingly becoming an important part of the research process and substantially affects it, both implicitly and explicitly (for archaeology, see e.g. Schmidt and Marwick 2020).
Despite the vital role that research software plays in many projects, the assessment of technical quality, usability, sustainability, fairness and ethical principles does not form a regular part of research design, nor are the achievements of those developing and programming sufficiently recognised academically (Hettrick 2016). Our poster is a contribution to raise awareness and overcome this critical shortcoming in the methodological development of archaeology.
Hettrick, Simon, Mario Antonioletti, Les Carr, Neil Chue Hong, Stephen Crouch, David De Roure, Iain Emsley, et al. 2014. ‘UK Research Software Survey 2014’. Zenodo. https://doi.org/10.5281/zenodo.14809.
Homburg, Timo, Anne Klammt, Hubert Mara, Clemens Schmid, Sophie C. Schmidt, Florian Thiery, and Martina Trognitz. 2021. ‘Recommendations for the Review of Archaeological Research Software’, Archäologische Informationen, 43: 15.
Schmidt, Sophie C., and Ben Marwick. 2020. ‘Tool-Driven Revolutions in Archaeological Science’. Journal of Computer Applications in Archaeology 3 (1): 18–32. https://doi.org/10.5334/jcaa.29.