Group Experiment Failed

[Kirak91]

We are currently conducting an online group study, with 15 participants in each group. Unfortunately, some participants decide to not take the study although they already signed up for it. If they leave the tab open but they don't continue, our experiment won't fill up and fails.

Is there any way to reject participants automatically who spend longer than 7 minutes on the instructions or who just went offline?

• Link to your project:
  https://hgvl-2.meteorapp.com

[amaatouq]

My general recommendation is to always recruit more people than you need. That is, if you need 15 people for your experiment to start, then you should recruit 25 people. They will all be able to go through the instructions (and they will be assigned to different conditions randomly) but only the first 15 to finish the intro steps (including passing the quiz, if it was included) will join the actual experimental trial. Otherwise, people who go idle will be redirected to the exit steps and you can handle them differently.

Here is a thread on how to run multi-participants experiments on MTurk: https://forum.turkerview.com/threads/launching-multiplayer-games-poor-click-through.2419/#post-1280331

Here is an example of how to handle participants differently based on their exitStatus:
in client main.js:

Empirica.exitSteps((game, player) => {
  return player.exitStatus !== "finished"
    ? [Sorry, Thanks]
    : [ExitSurvey, Thanks];
});

Here is client/exit/Sorry.jsx: https://github.com/amaatouq/guess-the-correlation/blob/master/client/exit/Sorry.jsx

[amaatouq]

Also, you can push participants to the exit steps using player.exit("some good reason here") at any point.

[JamesPHoughton]

Here's what I do to launch a 40 player game (and has scaled well to 80 players). Assume I want the game to launch at 2 pm:

1. Launch a 'signup' HIT at 1 pm explaining the HIT, and having individuals say "I commit to showing up at 2 pm" as the completion code. This runs for 45 mins (till 1:45) and gives players $0.05 just to sign up. It takes <30 seconds, and I allow them 5 mins once they accept the HIT. I ask ~100 players to sign up, so about 250% oversubscribed at this point.
2. At 2 pm I launch the game HIT that is only visible to the folks who signed up in advance. I release 60 HITs. (so about 50% oversubscribed at this point). The HIT description says that the game is oversubscribed, and says what I'll pay for completing each stage (showing up, training, play, and performance bonuses).
3. As folks arrive (usually within 2-3 mins) they enter training, but only the first 40 to complete the training progress to the game. Those who train but don't get to play get a completion code that says they trained, but the game was full when they arrived. Those who show up after the game has launched won't get to train, so they get the $0.10 'showing up' payment.