CS 5714- Usability Engineering - Dr. Hartson - Fall 2004

Homework

See course calendar for due dates.

Homework assignments

Homework #1:

Type up and hand in (via a Word file email attachment sent to the GTA or a typed list handed in at class) your list of critical incidents that you observed during the in-class exercise we did on Thursday, September 30, when we had the NY Ticket Kiosk data collection demo in the HCI lab. You should probably have at least a dozen critical incidents in your list and could have over twenty. You won't be graded so much on the total number of critical incidents as on the quality of your observations and statements.

FOR YOUR REFERENCE: use the link from the calendar (or below) to explore the NY Ticket Kiosk prototype, as desired. Use the links from the calendar (or below) to see the benchmark tasks that we used in the demo. We will provide a link to the Morae file containing the video and audio recording of the demo session.

The NY Kiosk Web site

Benchmark Task #1

Benchmark Task #2

The '.avi' file captured by Morae during the NY Ticket Kiosk data collection demo

Homework #2:

This is a team homework assignment (the only team homework). The teams are the teams you ended-up with at the end of the in-class kiosk exercise, which should be your regular project team, plus the final person you got as a participant from another team, minus the person you traded off to another team as a participant.

Hand-in a spreadsheet file the contains a complete cost-importance analysis of the full set of usability problems you got from the critical incidents you observed during data collection during the class exercise. As I said in class, after you do all the analysis and sort the table, select a hypothetical value for resources available and draw a line somewhere on your spreadsheet, representing the cutoff point. Then fill in the final column, resolution.

Be sure to put all your names (of the people who worked together on this homework) on this document somewhere (e.g., at the top of the spreadsheet); otherwise we will not know exactly who was on the team for this homework.

Homework #3

This homework is to be done as an individual assignment only!

General instructions:

We have selected 10 critical incidents (or usability problems) from the NY Ticket Kiosk that Dr. Hartson encountered during the in-class usability testing demo in the HCI lab in McBryde 102. We used Morae in the lab to extract 10 video clips from the one large Morae video/audio we made during the in-class demo.

For your class assignment, you are to diagnose each of these usability problems by classifying it within the UAF (identifying the problem type and subtypes and cause(s) within the interaction design), using the "UAF Viewer: Fall 2004 Usability Engineering". I will send the URL for this Viewer in my listserv message to the class. No username or password will be required to access the viewer; just click on the Login button to get in. To review the critical incident occurrence, task, and context, you can also use the NY Kiosk site, the benchmark tasks, and the entire Morae video/audio recording of the evaluation session, as posted under Homework #1 above.

Deliverable:

A sheet, or sheets (stapled together), of paper with your name and "Homework #3" at the top, printed from a Word (or similar document). For each problem:

Give the problem number from the list (no need to copy the problem over) and copy your chosen diagnosis path from the UAF. This is called "Diagnosis formatted for copying" and is located in the upper right-hand side, just under the "Current Diagnosis Path".
Write a brief rationale statement about why you picked the category you did and not any other likely one.
Declare whether you used the Wizard for help with this problem.

At the end, please indicate any feedback about the experience, including the UAF Viewer and the Wizard.

Hints:

Treat each problem as a single, separate problem (not having to extract multiple problems and not combining related problems this time).
UP4 actually identifies two problems, but both have essentially the same diagnosis, so treat them together as one.
UP10 is not about the problem of which object to click on to select the Three Tenors and the fact that the text looks like a link but isn't clickable. This problem is about something that is true in this clip, but the participant didn't comment on it explicitly. This is a critical incident noticed by the evaluator independent of the participant's actions and comments. This is about the fact that the little blue box you have to click on is very small and, therefore, it is more difficult to click on it accurately.
While many diagnoses are obvious, there is not always necessarily one "correct" answer. With this kind of diagnosis, "correctness" is more a question of reliability, how similar are your results to those of experienced usability engineering practitioners.
- If you think it is a toss-up between two classification categories and you definitely cannot decide between the two, give both as your answer (will be considered your first and second choices in order), showing them as separate answers. (Don't give more than two possibilities, please.)
- Grading will be based first on how well your classification matches the expected norm (by experienced practitioners).
- If your classification doesn't match, then we look at your rationale and any second choices. If one of your choices is a good match, you will get at least partial credit. If your choice is not a match to the "norm" but your rationale is good, you can get up to full credit.
Since the clips are from a real evaluation session, expect realistic distribution of diagnoses across the Interaction Cycle.
IMPORTANT: If you have trouble making your diagnosis at the top level, or you feel you're having to guess, or don't feel confident about your top-level diagnosis choice (a common occurrence for beginners, as you saw in class), try using the Wizard, found in the box in the upper left-hand corner.

The Morae Video/Audio Clips Defining the Usability Problems

Please download the video clips to view, rather than viewing them on this server. Thanks for your cooperation on this matter.

UP1
UP2
UP3
UP4
UP5
UP6
UP7
UP8
UP9
UP10