Past, Present and Future of Open Science (Emergent session): Developing collective action campaigns to change norms and drive progress in academia

[jsheunis]

Developing collective action campaigns to change norms and drive progress in academia

By Cooper Smout, Institute for Globally Distributed Open Research and Education

• Theme: Past, Present and Future of Open Science
• Format: Emergent session

Abstract

Academia functions like a ‘tragedy of the commons’ dilemma: Open Science practices have the potential to benefit the collective research community (and beyond), but their adoption is limited by incentive structures that reward sloppy science and high-impact publications at the individual level. ‘Crowd-acting’ platforms (e.g., Kickstarter, Collaction) overcome such conflicting incentives by organising a critical mass of support for the intended action, prior to its adoption. Similarly, Free Our Knowledge is a new collective action problem for the research community. Researchers can pledge to support a new behaviour, but only act on that pledge if and when there is a sufficient level of community support to protect their interests. Free Our Knowledge launched last year with a number of open access campaigns, but is designed to accommodate a much wider range of behavioural change campaigns (e.g., publish open access, post data to a repository). In this session, we will develop new campaigns for the Free Our Knowledge platform (https://www.freeourknowledge.org). Please bring any ideas you have for campaigns, or just come along and help others develop theirs. Much of the work will involve determining parameters of the campaigns, so no coding skills are required (but Python skills would be a massive help!) -- just bring a healthy desire for progress.

Useful Links

https://www.freeourknowledge.org/
https://github.com/FreeOurKnowledge/

Tagging @CooperSmout

[CooperSmout]

Updated title and abstract:

Developing collective action campaigns to drive progress in neuroscience
Neuroscientists are faced with a ‘tragedy of the commons’ dilemma: Open Science practices have the potential to benefit everyone in the neuroscience community (and beyond), but their adoption is limited by incentive structures that reward ‘fast’ and unreliable science at the individual level. ‘Crowd-acting’ platforms (e.g., Kickstarter, Collaction) overcome such collective action problems by organising ‘pledges’ for a particular behaviour, which are only acted on if and when a pre-determined critical mass of support is met. By protecting individuals’ interests until such time that they have the support of their community, crowd-acting platforms can thus resolve the collective action problem and increase uptake of the behaviour in question. Recently, we recently launched a novel crowd-acting platform for the global research community (Free Our Knowledge), which can accommodate a wide array of collective action campaigns with variable features. In this session, we will develop campaigns to support open science behaviours in the neuroscience community (e.g., pre-registering studies, making data available, or publishing in community-run journals like ‘Neurons, Behaviour, Data Analysis and Theory’). Bring any ideas you might have for campaigns, or just come along and help others develop theirs. Much of the work will involve discussing campaign features, so no requirements aside from a healthy desire for progress. But if you do happen to also have Python skills, we could definitely use your help on the project!

[CooperSmout]

Anyone interested please indicate your availability here and/or in the doodle form: https://doodle.com/poll/2zm3cdurtssbhddy

[r03ert0]

it'd be good to have someone from https://mattermost.brainhack.org/brainhack/channels/brainahack-proceedings! We hope to edit a free open access venue for publishing project reports (like https://joss.theoj.org but not only for code) (tagging @pbellec @agahkarakuzu @katjaq @complexbrains @anibalsolon)

[tiborauer]

Is it the same proposal as this one?

[CooperSmout]

Hi @r03ert0, yes it would! If what you're proposing needs some kind of critical mass to achieve (e.g. reviewers, editors, authors?) then yes we could host a campaign on the FOK platform and try to achieve that critical mass. Let me know what you're thinking!

[CooperSmout]

Hi @tiborauer it's the same project, but different proposals. Friday's session was more of an introduction to the project, and you can see the recording here. In the next session we will get together and actually plan some new collective action campaigns for the platform. These can be as wide reaching as supporting a new journal (e.g. NBDT), encouraging pre-registration, collecting datasets, demanding reviewers get paid for their services... you name it! Do you have something in mind?

[InquisitiveVi]

Thanks @CooperSmout ! This is an awesome idea to crowdsource campaign slogans.
I am adding in decoupling peer review of pre-registered hypothesis, data and reports. Integrating human language agnostic tools to communicate and participate (https://elifesciences.org/labs/9fdd6f3c/innovator-story-wrestling-the-octopus) rather than letting one language (English) be a barrier for participation.
Happy to help debating further ideas on paid service for peer review or better recognition in the incentive structure for a rigorous and healthy research culture.
My Python skills - only to the level of "hello world !" - learning in progress
UX editing skills- good enough to edit webinar slogans (high efficiency between 06:00 to 12:00 UTC)
Structured problem solving/ facilitation related to scholarly communication- expert level.
Looking forward !

[tiborauer]

Thanks, @CooperSmout. It is a very inspiring proposal. A few years ago, I started an INCF SIG on Open Publication. My idea was to upgrade reviewing process considering already existing techniques (semantic linking, document annotation, collaborative editing) to move away from the traditional pen-and-pencil-inspired unformatted copy-paste reviewing. My longer-term vision was to include both human- and machine-readable provenance, manuscripts and reviews so that scientists can focus more on science and less on how to 'sell' them; and also to bring authors and reviewers closer and to make the reviewing process collaboration a collaborative process rather than a fight.

[agahkarakuzu]

Thank you for pinging me @r03ert0! I will do my best to attend, this is a good opportunity to attract more attention towards community driven publishing venues.

@tiborauer sounds amazing!

[CooperSmout]

@InquisitiveVi great suggestions! Will be great to have you join us. Paid service for peer review is something that was just suggested on twitter. I think this could be a good idea for a campaign, and possibly get a lot of traction. Pre-registration will also definitely be a campaign in some capacity, but we need to figure out the parameters. We've discussed this one a bit before, so will be great to get your input. Also totally agree we need to be supporting non-English speakers better -- look forward to discussing ideas for this :) very excited!

I'm not quite sure what you mean by 'preregistered data and reports', if you can clarify?

[CooperSmout]

Thanks @tiborauer, means a lot to hear you say that! Also thanks for filling out your availability on the doodle poll (@agahkarakuzu please also fill out, if possible!). I fully support your push to move beyond paper-and-pencil publishing, the possibilities are truly endless if we can break this paper/journal trap and go fully digital.

And @agahkarakuzu agreed, the original purpose of the platform was to increase support for community-owned fee-free OA journals (but now it's more reaching, for any collective action problem). We'll be developing a campaign/s to support NBDT, but also open to other campaign ideas to support community venues!

[CooperSmout]

^^ on that note, this session will work best if we have some idea about the campaigns we want to work on ahead of time, so I'll post some ideas below and please thumbs up any that you like and/or add other campaigns that you'd like to work on. There's three elements to think about when developing a collective action campaign:

1. The action. What specific action/s are you asking people to take? I think it helps to be very simple/specific, e.g. posting a preprint
2. The threshold. At what level of community support does this pledge take place? Again, simple is best here (e.g., when 1000 people have signed the pledge)?
3. The duration. How long does the pledge last for? e.g., is it over once they complete a single action, after 2 years, or does it last forever?

NOTE: I've booked the session for this Thursday 09:00-10:00 UTC and will post the details in the Mattermost channel #hbmhack-fok

[CooperSmout]

Campaign idea: Supporting Neurons, Behavior, Data analysis, and Theory

1. Action: Publish one manuscript in NBDT? Send all manuscripts to NBDT so they can have first pick? (please comment with any other suggestions!!!)
2. Threshold: 100 cognitive neuroscience researchers? (who have previously published in a peer-reviewed journal)
3. Duration: 2 years? (this would help NBDT develop a relatively high impact factor, which are calculated over a 2 year period)

[CooperSmout]

Campaign idea: Reducing environmental costs of neuroimaging research

1. Action: Fly to only one conference per year (after covid restrictions are lifted)? Agree to re-use one dataset for every new dataset that you acquire? (please comment with any other suggestions!!!)
2. Threshold: 1000 cognitive neuroscience ECRs?
3. Duration: 5 years?

[CooperSmout]

Campaign idea: Increasing pre-registration in neuroimaging research

Action: Pre-register one study?
Threshold: 100 cognitive neuroscience researchers?
Duration: 1 year? (i.e. pledgers would be asked to submit one pre-registration within a year after the pledge activates)

[CooperSmout]

Campaign idea: Demanding pay for peer review services from commercial journals?

Action: Withhold peer-review services from all commercially-owned journals, unless remunerated at $X/hour?
Threshold: X cognitive neuroscience researchers? or % of journal author pool?
Duration: 2 years?

[tiborauer]

Campaign: Reducing environmental costs of neuroimaging research

1. Action: Fly to only one conference per year (after covid restrictions are lifted)? Agree to re-use one dataset for every new dataset that you acquire? (please comment with any other suggestions!!!)
2. Threshold: 1000 cognitive neuroscience ECRs?
3. Duration: 5 years?

Why fly at all? Wynes et al., 2019

[CooperSmout]

Ooohhh that is a great resource! You make a good point. Personally, I'd be happy to sign a campaign saying that I won't travel to conferences by air. Just depends on who the target audience is for such a campaign. In designing these campaigns there will inevitably be a balance between going too extreme and scaring people off, and designing something that gets heaps of support but doesn't make a big enough impact.

[CooperSmout]

Note: this session has been booked for Thursday 2nd July 09:00-10:00 UTC

Anyone wishing to attend will need to be registered for the OHBM Open Science Room ahead of time. Please see the Brainhack Mattermost channel #hbmhack-fok or email me directly for details on how to enter the Zoom meeting. Look forward to seeing you there! #therevolutionwillbetelevised

[InquisitiveVi]

Campaign idea: Increasing pre-registration in neuroimaging research

Duration: 1 year? (i.e. pledgers would be asked to submit one pre-registration within a year after the pledge activates)

a) We should specifically ask independent principle investigators (PIs) to pledge this as a goal within the next 12 months. We are in a system that has structural inequalities on who participates in producing the hypothesis vs data and who decides / gives approval to register them as reports.
b) Early career researchers without independent decision making responsibility can advocate with their PIs and collaborators within the next 18 months and share the registered reports for wider feedback.

Do we need a generic and cool hashtag for social media ?

[InquisitiveVi]

Campaign idea: Demanding pay for peer review services from commercial journals?

Action: Withhold peer-review services from all commercially-owned journals, unless remunerated at $X/hour?
Threshold: X cognitive neuroscience researchers? or % of journal author pool?
Duration: 2 years?

Love this one ! Specifically target journals from select publisher(s) not aligning to unlock research outputs. Need to curate a Wiki-like white list to demand financial compensation through verified ORCiD credentials.

[InquisitiveVi]

I'm not quite sure what you mean by 'preregistered data and reports', if you can clarify?

I meant, data, codes and other interim research outputs produced during the process needs to become available and the peer review process should be decoupled at every stage if necessary to reflect on the diversity. More as a live article/paper than a static document with fixed sections.

[CooperSmout]

We should specifically ask independent principle investigators (PIs) to pledge this as a goal within the next 12 months

Great point to consider: who is the target audience for a campaign? Do you think it better to target PIs or ECRs? To me, it's the ECRs who are doing the work and so makes sense to target ECRs for a prereg campaign. PIs are relatively free to do what they want, so there's little obstacle to doing prereg. But ECRs have to consider their advisors/future/productivity more, and so are more reliant on getting support from their community before they take action.

Early career researchers without independent decision making responsibility can advocate with their PIs and collaborators within the next 18 months and share the registered reports for wider feedback.

Are you suggesting different options within the same campaign? We can certainly do this, but I would suggest simple is best -- the campaigns we currently have up haven't got much support, and I suspect this is because they're too complex. Ideally should be able to summarise the campaign in a single slogan, like they do on Collaction.org

[CooperSmout]

Need to curate a Wiki-like white list to demand financial compensation through verified ORCiD credentials.

this raises another point: actions need to be clearly spelled out -- so in this case you're right, we would need to develop a new list of journals that we demand payment from. Sherpa Romeo could be useful here, as it shows which publisher owns each journal. I can't think of a resource that classifies publishers by their for-profit status, but it should be easy enough to compile a list of non-open access journals if that is the goal.

[InquisitiveVi]

Campaign: Reducing environmental costs of neuroimaging research

Why fly at all? Wynes et al., 2019

Thanks @tiborauer ! Unfortunately a pay-walled article from a for-profit publisher that many of us working from home cannot access.

Here is a Preprint by Sarabipour et. al., that analyzed how much environmental impact is caused by scientists traveling to SfN and highlights other inequalities of in-person conferences.
For a discussion on this please take a look at this webinar on reforming academic conferences (estimated watch time ~60 minutes).

[CooperSmout]

More as a live article/paper than a static document with fixed sections.

amen to that. there's a few people working on these ideas with Octopus, Libscie and Flashpub, but the challenge is building the critical mass to support such a radically different system. To me, Registered Reports seems the most tangible step toward these models.

[CooperSmout]

Unfortunately a pay-walled article from a for-profit publisher that many of us working from home cannot access.

Have you tried Sci Hub? https://sci-hub.tw/

[InquisitiveVi]

amen to that. there's a few people working on these ideas with Octopus, Libscie and Flashpub, but the challenge is building the critical mass to support such a radically different system. To me, Registered Reports seems the most tangible step toward these models.

Thanks @CooperSmout ! Agreed the most tangible and time sensitive approach is registered reports. I wanted to talk about Libscie, Octopus and Flashpub in today's discussion. If you will be covering that, I will not focus.

[InquisitiveVi]

Have you tried Sci Hub? https://sci-hub.tw/

Yes, I always do !
Full disclosure: I am part of a group that models responsible behaviors in scholarly communication among early-career researchers. Hence I do not publicly recommend certain ways of accessing scholarly content while I use the same unspoken method in practice.
I jump on chances to promote preprints and work from other inspiring ECRs that needs more attention.

[CooperSmout]

Awesome! I certainly would never recommend using certain ways of accessing scholarly content that contravene established legal protocols wrt copyright law etc. But feel it's good for people to know what tools are available and make up their own mind about whether to use them or not ;)

It sounds like you'll have a lot of input on the topic today, looking fwd to it! I'll kick off with a brief intro (and touch on Octopus etc) then pass it over to everyone else to introduce themselves and raise any thoughts they have on the topic of collective action, before we get on with the campaign ideas.

[tiborauer]

More as a live article/paper than a static document with fixed sections.

I would also add using data modeling techniques and semantic web technologies to enable easier links between papers, methods, live results. See NIDM

[tiborauer]

Campaign idea: Demanding pay for peer review services from commercial journals?

I have a bit related idea/campaign:

Campaign idea: Demanding modern peer review services from commercial journals?
Action: Withhold peer-review services from all commercially-owned journals, unless they updated the unformatted copy-paste system motivated by paper-based era. Here I have several ideas, which may need to be formulated as separate campaigns:

• The system should allow formatting options (e.g. markdown), annotations, tags, live documents (e.g. jupyter).
• The system should allow publishing reviews along with the paper (if desired) potentially as citable documents.
• The system should encourage collaboration between authors and reviewers potentially leading to reviewers' coauthorship properly annotated with CRediT (Contributor Roles Taxonomy).
• The system should allow an unlimited number of reviewers potentially even after the publication process. A contribution above a certain level should update the CRediT ~ live publication.

Threshold: X cognitive neuroscience researchers? or % of journal author pool?
Duration: 2 years?

[AllyGwl]

Just an idea (too late sorry): To improve diversity in academic departments, advertise all PhD and Postdoc posts in a publicly- and widely-available forum (something like FindaPHD/FindaPostDoc perhaps?). Many vacancies are publicised only to the recruiter’s own network (via email, JISCMail, Twitter etc.), which excludes applicants whose supervisors do not belong to those networks. To facilitate this, engage in outreach programs with universities outside your own network to ensure all students know how to access the forum.

[CooperSmout]

Definitely not too late... the revolutions is only just beginning ;) I like that idea. These sorts of community standards are great examples of collective action problems -- everyone would be better of if there were some agreement on where to post/where to look, and achieving that agreement could be achieved ahead of time, rather than the hodge-podge way that has brought us to the current 'standard' (not that there really is one). In this case I guess you would have to target PIs who are doing the advertising? And would help if the message board were already set up, so that people had some idea what they were committing to.

[robertoostenveld]

At our university, and I think in general in the Netherlands, it is required that all academic job postings appear on a public platform. This is implemented by the HR department, upon each new position being initiated. Although as a PI it is annoying in a way (since rather expensive and it takes extra time, especially if you already have the perfect candidate in mind), it being part of the system - rather than the individual PI's initiative - has the advantage that it is done properly. Also, if the platform were to change, we (i.e., the Dutch universities together) would not have to instruct 1000s of PIs and hope that they comply, but only update the (much more central) protocol for the 100s of HR employees. So perhaps it is worth to consider whom to target with this: the PIs or the HR department of the institutes (the formal employers).

[CooperSmout]

Great points, thanks @robertoostenveld. And encouraging to hear that this is happening on a national level, if not yet a global one. In the early days, I think it makes sense for our platform to target individual actions, taken by individual researchers, as this is the simplest type of collective action to get off the ground. But I hope that the platform will eventually get buyin from institutions, and be used to solve collective action problems at that level too. I can imagine campaigns in the future that asks institutions to take action X (e.g. advertise on the same forum, change hiring policies, etc) when Y other institutions have agreed