having a --template flag for empirica create to initiate project without demo game code

[malsobay]

Is there an existing issue for this?

• I have searched the existing issues

Is your feature request related to a problem?

No response

Describe the solution you'd like

The current default behavior for empirica create is to create a project with the "jellybean and minesweeper" demo code included. It would be nice to be able to generate an empty template with the relevant files and empty function definitions (e.g. for callbacks) -- more or less the structure that someone following the IC2S2 '23 tutorial would have at the end of part 4.1. Demos are great for getting a newbie up and running quickly, but the issue is that a newcomer might also struggle with figuring out what to take out.

Also, I suppose that if this functionality is built in a sufficiently generic way, then it can probably be used to create projects of other "curated" Empirica demos, e.g. empirica create --demo-pgg [project name].

Describe alternatives you've considered

No response

Teachability, Documentation, Adoption, Migration Strategy

No response

Code of Conduct

• I agree to follow this project's Code of Conduct

[npaton]

Agreed. The default should not be cluttered. I personally think we'd like to have a few default templates (we've talked about implementing a few minimal experiments of different genres) and when you run empirica create app, it presents a menu like this:

[x] Blank template
[ ] Prisoner's dilemma
[ ] Experiment genre C
[ ] Experiment genre D

And we can also have a flag on create that chooses the template --template blank and you can pass it a repo --template https://github.com/malsobay/demo-pgg, and it will bare clone that and clean it (e.g. regen .empirica/id).

That's generally what I've been thinking about. I don't remember where we're tracking the few default templates, maybe it needs a new ticket.

[malsobay]

@npaton That kind of interaction sounds great -- I especially like the idea of cloning & cleaning a repo. What kind of support do you need from me on this?

[npaton]

Right now I am focusing on deployment and general fixes as they come along. We're getting more and more stable, so once we have cloud and reasonable stability, I will refocus on this. With @amaatouq we made a quick list of potential templates. We had these 2 selected as the first demo template candidates:

1. Economic game: Public Good Game or Prisoner’s Dilemma, to study cooperation and decision-making within groups.
   • all or nothing and progressive
   • less factors
2. Psychophysics: Visual random dot motion stimuli, to investigate perception and decision-making in uncertain environments.
   • continuous guesses visible
   • look up game in v1

And of course the nearly blank template. Since we'll be removing almost all code from the blank template, I think we'll want to have at least 1 of the demo templates listed above. PGG seems to make sense.

I'm also thinking there should be a minimal template along with the blank template. Blank makes 0 assumptions, it just sets up the necessary folders and files. And minimal makes a few assumptions, such as setting up Tailwind and prettier, maybe some other quality of life things.

Amongst the quality of life things, I'm looking into:

• Tailwind
• Prettier
• shadcn/ui / Radix components
• MDX This is immediate nice for instructions in intro steps, but I wonder if it could even go further for things like forms, etc.
• Treatment/factor validation. Really not sure how to do this one. Might need to be in core since I think it will need code generation

I'm also wondering if people would be into a Typescript template. I would… I used it in the latest experiment I made and for me at least, it makes things easier, less typos, more autocomplete.

I just went on a rant, and didn't really answer your question. I'd certainly like your opinion about all of this. And for the 2 demo templates mentioned above, if you have ideas on what the minimal factors and features we should support. And if you don't like the selection, I'm also interested in other ideas. I think we really want to strike that balance of enough to show how you might build an experiment of a certain type, and show some good coding practices, but not making it too large and hard to jump into and modify. All ideas are welcome.

Quick recap for the templating:

• blank: just the folders
• minimal: blank + quality of life
• pgg-ish: minimal + small example pgg type experiment
• psycho: minimal + small example perception type experiment
• and a generic bring your own repo mechanism

It might seem like a lot, but the blank to minimal to pgg kinda build upon each other, so it makes sense to do them at the same time. And for the bring your own, I'm not sure, but I might build all of them as separate repos, so they might all be built on that clone+clean model, just you'll have these particular ones listed in the command line menu for convenience. Or maybe I'll embed them into the command line, I haven't decided yet.

[malsobay]

Quality of life: Those all sound kinda useful, but even as a somewhat-frontend-literate researcher, having to think about all those things sounds exhausting. To the extent that any of them are included in the Empirica experience, I would urge us to:

1. Have sensible defaults for the inclusion/configuration of any of these tools.
2. Add examples to the documentation illustrating how the typical Empirica user can make use of them. Sure, people can look up docs, but a guide to the features that are most valuable to the Empirica context would really reduce the mental load.

Demo experiments: I'm not too familiar with the psychophysics one, but for PGG, I'd say the factors that are most illustrative of Empirica's capabilities (and useful to someone out of the box) are:

• Number of players and rounds
• Multiplier
• Punishment/Reward toggle and cost/magnitude
• Chat
• All or nothing
• Stage duration (just use one parameter for all stages, instead of how each stage is separately parameterized atm)

I think that goes a long way... information toggling (e.g. punisher IDs, # of rounds visible, etc.) is nice but might be overwhelming. We can point to the full PGG repo somewhere for those interested in more elaborate designs.

Another demo that I think would be useful would be a data annotation task; with the ease of fine-tuning LLMs and image generation models, more attention is being driven to curating a custom dataset. I've been asked more than once if Empirica could be used for something like this, so the interest is there, and it's somewhat easy to design something sufficiently generic that it's useful out of the box. Relatedly, I have an Empirica app I wrote for a pairwise comparison task where I load pairs of images and ask people to choose the one most similar to a reference image; something like this could be cleaned up as a template.

Templating: The templates you suggested sound great as a first step -- having the blank/minimal would be a great QoL upgrade.