Skip to content
This repository has been archived by the owner on May 24, 2019. It is now read-only.

HIT Status

Jump to bottom
leeper edited this page Jan 4, 2015 · 3 revisions

There are many ways to check the status of your HITs. For requesters familiar with the Requester User Interface (RUI), the Manage Batches screen will be familiar. This allows you to see batches of HITs - which can only be created through the RUI - and various summary statistics about them in one place. The Batches page describes how to recreate some of those statistics through MTurkR. In addition, the RUI allows you to view the status of individual HITs, from which you can approve and reject assignments, extend the HIT (with time or additional assignments), as well as a few other operations.

MTurkR allows you to check on the status of HITs in several ways. One of them was already mentioned (using the Batches tutorial). A second way involves the use of the HITStatus function, which supplies information about the status of a single HIT or HITs of a given HITType. This is the function underlying the Batches tutorial.

A third way involves the use of "notifications," which are specified events that trigger a message from MTurk. By default, requesters receive no such notifications. When assignments are submitted or returned, or when HITs are completed and become reviewable, MTurk defaults to requiring the requester to log in (through the RUI or through MTurkR) and check the status manually. Notifications change this paradigm by triggering a notification event at specified points. The easiest way to use this service is to configure email notifications, which involve MTurk sending an email message to a specified address whenever a particular event occurs. Notifications can also be setup using the Amazon Simple Queue Service. Interested users can see the documentation for details on SQS.

A simple way to test out notifications is with the SendTestEventNotification function, which pretends that a specified event has occurred in order to test a notification. To use it, we first create a notification using GenerateNotification using our intended destination email address and an event.type (see the documentation for GenerateNotification for the list of available events. We can then use SendTestEventNotification to see what occurs when different types of events are called. In the example below, we set up a "HITExpired" notification and test what happens when both a "HITReviewable" and "HITExpired" events occur. The latter triggers an email, while the former does not.

a <- GenerateNotification("requester@example.com",event.type="HITExpired")

# no notification email will be sent
SendTestEventNotification(a,test.event.type="HITReviewable")

# notification email will be sent
SendTestEventNotification(a,test.event.type="HITExpired")

The latter function should cause MTurk to send an email similar to the following to the specified email address:

Greetings from Amazon Mechanical Turk,

You are receiving this email because you subscribed to be notified when
certain events related to your HITs or Qualifications occurred.

Specific event information is shown below:

    Event Type: HITExpired
    Event Time: 2014-02-23T21:37:58Z
    HIT Type ID: 09876543210987654321
    HIT ID: 12345678901234567890
    Assignment ID: 1234567890123456789012345678901234567890

    Event Type: HITExpired
    Event Time: 2014-02-23T21:37:58Z
    HIT Type ID: 09876543210987654321
    HIT ID: 12345678901234567890
    Assignment ID: 1234567890123456789012345678900987654321


Sincerely,
Amazon Mechanical Turk
https://requester.mturk.com
410 Terry Avenue North
SEATTLE, WA 98109-5210 USA

Now that we've seen what a notification looks like, we can put them into use on our HITs. Notifications can be attached to HITTypes using SetHITTypeNotification. This function takes a HITTypeId and a notification object (created above). A third (optional) active parameter can be used to turn a notification on and off (e.g., to deactive it before a project is over).

SetHITTypeNotification(hit.type="2FFNCWYB49F9BBJWA4SJUNST5OFSOW", notification=a, active=TRUE)

Now, whenever a HIT with this HITType expires an email like the one above will be sent.

.

Clone this wiki locally