Skip to content
This repository has been archived by the owner on Nov 7, 2018. It is now read-only.

Latest commit

 

History

History
90 lines (51 loc) · 5.22 KB

usability-testing-synthesis.md

File metadata and controls

90 lines (51 loc) · 5.22 KB
Raw

July 2017 Usability Testing

The 18F FOIA National Site team completed a round of remote prototype testing during the week of June 19th. Participants in the study were largely first-time requesters. The session lasted 45 minutes and each participated completed one of the planned activities.

Issues and recommendations

The following are observations coming out of the usability testing and corresponding recommendations. Each observation has a severity rating associated with it to aid in the prioritization in correcting the issue. The severity ratings (from Nielsen Normal Group) are as follows:

  • 1 - Cosmetic issue: no need to fix unless extra time is available on project
  • 2 - Minor usability problem: fixing this should be given low priority
  • 3 - Major usability problem: important to fix, so should be given high priority
  • 4 - Usability catastrophe: imperative to fix before product can be released
  • ? — Needs further discussion

Submission

Requesters did not receive an email confirmation.

Rating: 3

Requesters consistently stated that they expected to receive an email confirmation.

Recommendation: Send confirmation, include (MVP) summary of submission and a link to status tracker if possible.

Requesters were unable to estimate response time before submission.

Rating:3

Requesters were often surprised by estimated response time after submission or arbitrarily guessed before submitting request.

Recommendation: Median processing time should be clearly presented in the request view. Consider explaining the difference between complex and simple requests. Consider providing more information on how agencies fill requests.

Transition to FOIAOnline is disorienting

Rating: 3

For many users, the transition to FOIAOnline for non-participating agencies was jarring. One user assumed they had clicked the wrong link all together. Many were confused because they didn’t understand the distinction between FOIAOnline and the national portal.

Recommendation: Direct user straight to form (already addressed in latest prototype). Better copy for the transition.

Requesters expressed confusion over fee rules.

Rating: 2

While scanning the form, some requesters were surprised when they came across the fee field. Additionally, confused about whether or not they could qualify for a waiver. Some requesters would respond with a “sure why not” and select the option for fee waiver, leave blank, or enter non-numeric responses into the field.

Recommendation: Provide more guidance on fees. Only ask qualifying users whether they’d like to request a waiver or expedited processing. (i.e. “I am a journalist” vs “I’d like to request a fee waiver”)

Requesters given no guidance on distinction between centralized and decentralized agencies.

Appeared to be some confusion when user entered a decentralized agency name (i.e. USDA) and so many results were returned.

Rating: 2

Recommendation: Consider a different task flow for for decentralized agencies (Currently all treated the same).

Status page

FOIA office contact information not available on status page.

Rating: 2

Some requesters expressed a desire to be able to contact FOIA office particularly in case of long turn-around time requests. In order to do so, requesters had to go back to request page.

Recommendation: Add FOIA contact to page. Research indicated that requesters particularly liked having the officer’s name on the page.

Requesters confused by milestones

Rating: 3

Users were unable to confidently define some of the status milestones. Especially “record collection” and “closed”. The word “processing” was also unclear. Additionally, should consider adding more milestones for higher fidelity.

Requesters believe annually reported median processing times are based on real-time data (backlog) and their specific request.

Rating: 3

Recommendation: Clarify what information each agency uses to create estimate. Clarify distinction between estimated response time and median response time.

Other observations

  • All users started the document scenario with a Google search, typically including some combination of the name of the federal building they were asked to research and the city it is located in.
  • Many requesters attempted to find the relevant documents outside of FOIA first. However, the exhaustiveness of the search was varying. Few searched through FOIAOnline or similar FOIA response repositories.
  • People seemed to have preference for .gov sites for document exercise.
  • People often ignored Google feature box in search results.
  • Many users attempted to find a contact phone number both before and after filing request.
    • Finding contact info before filing request was a means of trying to find the scenario documents outside of FOIA.
    • Finding contact info after filing a request was a means of getting more detailed status on request or attempt to nudge it along.