Aggregate Publishers Implementation Details

Introduction

ODK Aggregate is intended to act as a locus for aggregation of data from devices running ODK Collect.

Once collected, data can be published to 3rd party servers. There are 2 publishers available:

1. Google Spreadsheet
2. JSON publisher

Details of the Simple JSON Publisher

The simple JSON publisher constructs a JSON representation of the form, preserving all groups and nesting, including nested repeat groups. A

The data is published as POST with a content type of "application/json". The POST has the following string-valued elements:

• "token" : the auth token specified when publisher is created
• "content" : "record"
• "formId" : the unique form id
• "formVersion" : the form's version
• "data" : submission-list

The "data" is a normal JSON serialization of an array of submissions following the general conventions specified on [json.org]. Multiple-choice values are represented as an array of values (or null if the array is empty). The Media attachments are written strings using one of two options:

Option A) String URLs that can be used to fetch the binary contents of the attachment.

_"Image": {

• "url": "https://opendatakit.appspot.com/view/binaryData?blobKey=gme-geo-tagger-b1%5B%40version%3Dnull+and+%40uiVersion%3Dnull%5D%2Fdata%5B%40key%3Duuid%3A57a5c38d-e903-429e-857d-985bb415b057%5D%2FImage",
• "filename": "1377188872644.jpg",
• "type": "image/jpeg" }_

Option B) Binary strings.

_"Image": {

• "bytes": Base64 encoded binary
• "filename": "1377188872644.jpg",
• "type": "image/jpeg" }_

Notes from a Django integration

• The Django server must be visible to the Aggregate server, so it might not be able to receive the POST on localhost.
• If the Aggregate server and receiving server are on different IPs, the receiver may think that it is a cross site request forgery and block it automatically. To get around this either use the token option in the publish feature, or allow cross site requests for this one function. In Django, this can be done by adding the @csrf_exempt decorator before the view.
• Because the JSON POST is not formatted in the same way as a form, a Django server might not recognize it as a POST request, and the request.POST field will be empty. Try looking in request.body, which contains the raw request received. This is particularly difficult to find because print(request) does not include request.body, and instead appears that no form data is in the request at all.
• After receiving the POST request, send a 200 status code back to the Aggregate server, or else it will think there was an error and not send any further POSTs.
• For a working example.

   def odk_submit(request):
      if request.method=='POST' and len(request.body) > 0:
          try:
              json_data = json.loads(request.body.decode('utf-8'))
              survey_data = json_data['data']
          except Exception as e:
              pass # DO any action
          else:
              # Process the survey data. its dict format 
              # data access using key, eg: survey_data['form field']
              # if the form have images, it pprotected with username
              # and password so we need
              image_url = survey_data['image_key']['url']
              byteImgIO = io.BytesIO()
              # get_content_image is added below
              image_content = get_content_image(image_url)
              byteImg = Image.open(BytesIO(image_content))
              byteImg.save(byteImgIO, "PNG")
              byteImgIO.seek(0)
              byteImg = byteImgIO.read()
              # store the 'byteImg' byte code in to db field
  
  def get_content_image(url):
      import httplib2
      from django.http import Http404
      username,password = "odku","odkp"
      ht = httplib2.Http(".cache")
      ht.add_credentials(username, password) 
      resp, content = ht.request(url, "GET", body="foobar")
      if resp.status:
          return content
      raise Http404