Custom crowdsourcing task

[Sea94]

Hello!

Thanks for developing this great platform.

I would like to run some crowdsourcing experiments on Mturk and was wondering if the following use cases are supported at the moment.

• Can I have a human chat with (several) model agent(s) and another human within one task, given that conversations should happen sequentially (i.e., not as a simultaneous group chat). In other words, can I combine “Chat demo” and “Model chat” example tasks under one task?
• Can the two workers who are paired to chat with each other under one task have different instructions displayed to them/different rewards offered/different follow-up questions asked?

Thank you in advance

[mojtaba-komeili]

I recommend you using our crowdsourcing tool, Mephisto. You have full control over the agents in the world and their interactions.
Also, for your second question, see this task where two humans involved have different roles with different instructions and onboarding.

[Sea94]

Thanks a lot for the references, @mojtaba-komeili ! I will have a look.
If I launch a task with the help of Mephisto, is it possible to include some custom UI elements (multiple choice questions, drag'n'drop controls, etc.)?

[mojtaba-komeili]

Mephisto is is very flexible when it comes to designing customized UI for your task. I recommend you check their customized task for ParlAI chat here.