Homework Assignments for CS 3724, Spring 2007

Homework 3: Pilot study for evaluation

For this homework, you will perform a pilot test of the features of your evaluation design that you decided on class. You may use the Informed Consent Form, questionnaires and written instructions if you already have them.

  1. Pilot test your evaluation with at least one volunteer. Treat this like an actual evaluation session (perform all steps of your procedure, gather all data, etc.). It might be helpful to have one group member doing a “meta-evaluation” (i.e. evaluating the evaluation) to look for issues with your evaluation design.
  2. Write down a list of problems encountered in your evaluation design. These might contain metrics that you did not choose well, limitations of your prototype that you found out make it impossible to acquire the data you want, issues with your procedure, etc.
  3. Redesign your evaluation based on the results of your pilot tests.
  4. Create a short PowerPoint presentation covering the following three topics:
    1. Your initial evaluation design, including the critical aspects you evaluated, the metrics you used to gather data, and the usability spec you used.
    2. Your pilot evaluation. Focus not on the results of the evaluation, but on strengths and weaknesses on the evaluation design itself.
    3. Your redesigned evaluation. Talk about what needed to be changed and why, and justify why you came up with the new design.
  5. Send your PowerPoint file to the TA (kopper@vt.edu) no later than 12:00 noon on Wednesday, April 11.

Homework 2: Reflecting on Usability Problems

Some find it hard to see why we need to study HCI or use a process like usability engineering. But with a little reflection, we can see that all of us – even the most expert computer users – experience usability problems almost daily. Your task in this homework is to keep a journal for one week. In this journal you should record negative experiences you had while interacting with software or websites. Each journal entry should contain:

·       Detailed information on your goal or task (e.g. What information were you trying to find? What result were you trying to produce?) (1 paragraph)

·       If you were using a software application, the name of the application and a URL to its website

·       If you were using the web, the site(s) you used, including URLs

·       A detailed description of the problem you encountered (1-3 paragraphs)

·       How you were eventually able to accomplish your original goal or task (or that you eventually gave up!) (1-3 paragraphs)

Your journal should contain at least five detailed entries, and should be in Word or HTML format. Make sure that URLs are linked to the actual sites so we can click on them and see for ourselves.

Turn in your journal by email to the TA. The deadline is 2:30 PM (class time) on January 29. Remember that no late homeworks will be accepted.

Homework 1:

Welcome to the class! In order to assign project teams for this semester, we need you to fill out a short online survey. To help "encourage" everyone to do this, this will count as your first homework grade. Complete the survey, earn a grade of 100. This is due by Monday, January 22, at 9:00 AM.

The survey is here: https://survey.vt.edu/survey/entry.jsp?id=1168897306299


(c) Copyright 2007 Virginia Tech, (based on design by Mary Beth Rosson and John M. Carroll)
Last Updated: January 22, 2007