A minimal list of resources for students, collaborators, and myself

A list of useful resources for a modern, transparent, and reproducible scientific workflow. These resources cover many stages of the scientific process from designing an experimental procedure to writing a report and collaborating with others. This list will be continuously updated. However, this list is not intended to be exhaustive. Instead, it aims to provide only a minimal set of essential resources to get acquainted with the topic.

Open science, a transparent, and reproducible workflow

The way we do research in Psychology quickly evolved during the last decade. To tackle issues related to the researcher degrees of freedom (amongst other things), novel practises emerged. For instance, it is now common practise to register in advance (i.e., before data collection) the hypotheses to be examined, the way data will be collected and analysed, etc. The following resources will guide you through these new practises and their practical implementation.

Psychology's renaissance/crisis/revolution, an overview

What happened during the last decade in Psychology? For those who missed the party, the following resources cover the recent events and discuss their implications.

• Psychology's Renaissance (Nelson, Simmons, & Simonsohn, 2018)

• Reproducibility of Scientific Results (Fidler, 2018)

Preregistration and (replication) registered reports

• The preregistration revolution (Nosek et al., 2018)

• What are registered reports?

Version control and automation, a reproducible workflow

Using Git and Github, eventually from RStudio.

• Curating Research Assets: A Tutorial on the Git Version Control System (Vuorre & Curley, 2018)

• RStudio, Git and Github (Wickham, 2015)

Data collection (sensitivity and sequential analyses)

How should we determine the sample sitze of our study? Do we have to do it a priori or can we stop recruitment when we have sufficient evidence to conclude?

• When power analyses based on pilot data are biased: Inaccurate effect size estimators and follow-up bias (https://psyarxiv.com/b7z4q/)

• At what sample size do correlations stabilize? (Schönbrodt & Perugini, 2013)

• Sequential hypothesis testing with Bayes factors: Efficiently testing mean differences (Schönbrodt, Wagenmakers, Zehetleitner, & Perugini, 2017).

• BFDA Bayes factor design analysis: https://github.com/nicebread/BFDA

• A fully automatised, transparent, reproducible and blind protocol for sequential analyses (Beffara Bret, Beffara Bret, & Nalborczyk, 2021).

Analysing data

Introduction to Frequentist and Bayesian statistical modelling in R.

• Free and complete course in R programming at Coursera

• MOOC - Improving your statistical inferences (Lakens, 2017)

• Course - Statistical Rethinking: A Bayesian Course Using R and Stan (McElreath, 2019)

• Introduction to Bayesian statistical modelling (20h doctoral course, in French) (Nalborczyk, 2021)

Writing empirical papers

Writing style

• "Writing in the Sciences" MOOC (Sainani Kristin, 2017): https://www.coursera.org/learn/sciwrite/

• Writing and Revising, writing guide (Simmons, 2019)

• Writing Empirical Articles: Transparency, Reproducibility, Clarity, and Memorability (Gernsbacher, 2018)

• Improving Scholarly Communication: An Online Course (Gernsbacher, 2013)

• Teaching Graduate Students How to Write Clearly (Wagenmakers, 2009)

• Writing the Empirical Journal Article (Bem, 2002)

• Strunk, William Jr. The Elements of Style. Pearson Education Limited (England, 2014).

Writing transparent and reproducible empirical papers

• LaTex + Overleaf v2 permits to collaborate online on LaTex documents: https://www.overleaf.com/

• Writing paper from RStudio in RMarkdown with the papaja package (APA6-formatted manuscripts)

Large-scale collaborations

To counter the pervasive problem of low-powered experimental designs, researchers started to mutualise efforts to provide high-powered responses to debated questions in psychological researchers as well as to assess the generalisability of the findings. The idea is pretty simple: instead of running a single study in our lab, what if we could run the study in (let's say) 20 different labs all across the world?

• Many-labs initiatives, such many lab (1 to 5), many primates, many babies, and so on...

• Study-swap: a research community for sharing resources and collaborations (https://osf.io/meetings/studyswap/)

• The Psychological Science Accelerator: Advancing Psychology through a Distributed Collaborative Network (Moshontz, H., Campbell, L., Ebersole, C. R., IJzerman, H., Urry, H. L., Forscher, P. S., ... Chartier, C. R., 2018)