CS 3724: thinking aloud and model-based Evaluation                 In-class Activity 9: April 3

Names and Student ID number of people participating in your activity:
 
 
 
 
 

0. Bring at least two digital watches per group to class today. You will analyze one of them and use the other as a timer.

1.Choose two people in your group who have never used the watch before. Excuse one of them, then listen while the other provides a "think aloud" commentary as s/he tries to (a) set the current date and time to be Monday, May 11, 2003, 8:32am; and (b) set a daily alarm for 10:30pm. (If the watch doesn’t have these functions, come up with two similar tasks it does support). Write down interesting comments or problems from the episode (on scratch paper). Do the same for the second person.

2. As a group, discuss what you learned about the watch’s user interface. What did your test users expect? Were they confused or surprised by anything? What did you learn about the two users’ mental models of how digital watches work? Do these observations/comments suggest anything to you about how to redesign the user interface? Summarize the highlights of what you learned below:
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

3. Build a simple GOMS model for each task, similar to what you did in Homework 9 -- remember this is essentially a hierarchical decomposition of goals into sub-goals. Each goal is satisfied by a method. Each method is implemented by a set of operators, or "primitive actions." In cases where there are multiple methods for a goal, a selection rule is used to choose between them.

                        Task 1                                                             Task 2

4. Have one test user execute one of your two tasks 5-6 times as quickly as possible (making no errors!); keep track of how long it takes to perform the task, and try to estimate times for component operators as best you can. Working from the average overall time, the distribution of operators in the tasks, and the estimates you made for particular operators, generate estimates for each individual operators and/or selection rule (i.e., make a best guess as to how to divide up the time). Annotate your two models with these estimates, and use them to predict long it should take to do the second task. Write down your estimate below, then have this same user execute the second task 5-6 times. How close were you? Can you think of ways to improve the predictive accuracy of your model?