Final User Testing & Report

Deadline: Tuesday, December 16 (before classtime)
Point Total: This assignment is worth approximately 11% of your project grade (4% overall).

Assignment Overview
This assignment should be read in conjunction with TA07 Final Interactive Prototype and TA08 Final Video. Together, TA07, TA08, and this assignment (TA09) represent the culmination of work on your team project over the course of the semester. They also serve as a "final exam" of sorts as we will not be having a formal exam during our final exam slot (instead, we will be viewing your TA10 final videos, see TA10).

While TA08 and TA10 focus on the specifications and expectations for your final interactive prototype and final video respectively, this assignment describes the usability test for your prototype and the required sections for your final report.

What To Do

  1. Begin by reading over TA07 Final Interactive Prototype and TA08 Final Video. Operationally, I suggest you assign "leaders" within your team to manage different components of TA07, TA08, and TA09 to help collaborate and divide labor more effectively. While I expect that each individual member should contribute to all three assignments, I think having a specific leader will help organize the remaining tasks.

  2. Once you implement the three primary tasks for TA08, you must perform a usability test with three users. Ordinarily, this would be done with "target" users pulled from your expected user population; however, given the constraints of this class, I am loosening this restriction. Thus, you can use members of this class who are not on your team for usability tests. Follow a study procedure similar to the one you used for TA05 Paper Prototypes with a think-aloud strategy. Remember to collect informed consent. Scan in and include this with your report (in the appendix).

  3. Analyze the collected data and flesh out implications for design. Focus your analysis on the best/worst performing design elements and think about how these findings can be used to improve future design revisions. You will combine these findings with findings from your final in-class design critique (on December 4)

The Report

Submit a report of no more than 5 pages of text (approximately 2,500-3,500 words in a font similar to 11 pt. Times). Images, tables, figures, etc. are strongly encouraged, do not count against the page limit, and are thus effectively free. Please use the following full section headers (style Headings 1) in your report; it makes it easier for us to grade. So, for example, you should literally have headings with Section 1: Abstract and Section 2: Task Descriptions in your report.

Title Page

On your title page, please include a title (centered and bold), a catchphrase for your project that highlights its primary benefit, and a list of each team member. For each person listed, describe their primary role/accomplishments for TA07, TA08, and TA109(for each TA, there should be 2-3 sentences per person). If there's room, you may want to include some iconography or a screenshot of your app on the title page. Note: each team needs to assign an official proofreader that looks over the entire report, makes sure all of the requisite content is included, and eliminates typos/grammatical errors. Please include in the report who was assigned this role.

Section 1: Abstract

The abstract should provide a 5-7 sentence overview of your report including: (i) a description of your application, (ii) a description of your three primary tasks, (iii) a description of your evaluation method, (iv) a summary of your primary findings, and (v) design implications for future revisions.

Section 2: Prototype Implementation Overview
Describe your prototype implementation: what language did you use? Why? How did you build it? What libraries, if any, did you use? How were your design decisions informed? Did you rapidly iterate and test as you built the system? What interactions/features did you leave out but hoped to build? Justify your design decisions based on course content (e.g., lectures, readings, discussions).

Section 3: Task Descriptions
Describe your three primary tasks (iterate from previous incarnations) This should be a four paragraph section. The intro paragraph summarizes the three tasks. Each subsequent paragraph should start with a style Heading 2 with the name of the task and a more lengthy description. The descriptions should include screenshots of your actual interactive prototype in the style of either: a sequential storyboard, a state transition diagram, or a branching storyboard (see this Greenberg reading).

If the tasks changed from the previous assignment, please describe these changes and why they were made (e.g., was it based on your own design reflections, was it based on the experiences of TA05 and TA06, etc.). You can also reference your TA08 Final Video here (this is called a video figure rather than a pictorial figure and is becoming increasingly common in HCI reports). So, for example, if you have a skit in your video that goes over each of the tasks, you can refer to the video timeline in your prose (e.g., See 01:20-01:44 for a video describing Task #2). This should not replace your prose but should augment it.

Section 4: Results from Design Critique and User Testing

  • Describe the Dec 4th design critique and main takeaways
  • Provide an overview of participants and user testing study method
  • Describe positive and negative results from user testing.

Section 5: Project Conclusions and Design Recommendations

  • Discuss the quality of your overall interface design. Identify limitations in the design (some of these may be due to the initial scope of the project). Identify design elements that work particularly well.
  • Provide recommendations for the next logical steps. Do you feel your design has been validated? Are minor adjustments needed, but the overall approach is validated? Is the concept still worth investigating but serious problems have been identified? Based on this conclusion, what are the logical next steps that you would recommend be done on this project?

Section 6: Design Process Reflections

Reflect upon the design process you followed in this project. What aspects of the human-centered, iterative design process worked well? What aspects did not work so well? It may be effective to do this as a brainstorm session with your group. For example, you could start with questions like:
  • What were the most significant ways in which the design concept and the actual interface design changed under the influence of user involvement? What were the biggest surprises for you—the things you learned from or about users that you would not have predicted based on your own experience and intuition?
  • Did the methods you chose for your evaluation and prototyping get at what you were looking for? In hindsight, would a different approach (process, not specifics of your interface) have been better?
  • What were the most and least valuable among the methods you used, either generally or specifically for your project?

The appendix should include the following. Each of the following bullets should start on their own page and should be titled Appendix A: <title>, Appendix B: <title>, etc. in Heading 1 style.
  • Raw notes from the two user testing sessions (notes should be clearly marked with a timestamp and session number)
  • Scanned signed IRB informed consent forms