Skip to content
This repository has been archived by the owner on May 24, 2019. It is now read-only.

Support for Batches and ReviewPolicies

Compare
Choose a tag to compare
@leeper leeper released this 07 Oct 09:40
· 218 commits to master since this release

MTurkR v0.5.5

This release includes several smaller bug fixes and a few larger enhancements, described below.

  • MTurkR finally supports MTurk ReviewPolicies, which allows the MTurk system to automatically approve, reject, and extend assignments for HITs based on either "known answers" in workers' responses or "plurality" agreement across multiple workers' responses. To achieve this, GenerateReviewPolicy is removed in lieu of two new functions: GenerateHITReviewPolicy and GenerateAssignmentReviewPolicy, both of which have a simplified function API and include extensive examples in the documentation. The use of CreateHIT remains unchanged.
  • Additional support for working with "batches" created via the online Requester User Interface (RUI) has been added. Specifically, HITStatus, ApproveAllAssignments, ChangeHITType, DisposeHIT, DisableHIT, ExpireHIT, ExtendHIT, GetAssignment, GetBonuses, SetHITAsReviewing now include an optional annotation argument that can be used to apply the function to all HITs of a given batch. For example, to add two assignments to every HIT in a batch, simply find the batch number from the RUI and pass it as follows: ExtendHIT(annotation = "BatchId:78382;", add.assignments = 2).
  • A substantial effort has been made to improve and expand documentation for the package. As part of this, examples in package documentation for all functions have been substantially expanded and checked for accuracy. And, the MTurkR wiki now includes numerous updated code examples, including full documentation for both the text-based wizard.simple and graphical mturkr.wizard interactive interfaces for MTurkR.
  • All functions should now return type-consistent responses. This replaces the previous behavior of returning NULL on certain failures where dataframes have always been returned on successes.

See NEWS and GitHub commit logs for full details.