CS3724, Fall 2000

Due as per class calendar

Activities

1. Develop two benchmark task descriptions to support each of the three usability goals you specified in Project 2 (this makes six benchmark task descriptions). In addition, develop four more benchmark tasks that represent the most common tasks that users do. Make sure you have at least three benchmark tasks for each of the three (or more) user groups you defined in Project 2. If the 10 benchmark tasks you have so far do not meet this requirement, add enough additional benchmark tasks to do so.
2. Build your low- or high-fidelity prototype, to be used in formative evaluation in the next project assignment. For your final prototype medium, you have a choice. You may build a full paper and plastic low-fidelity prototype or you may implement a higher-fidelity computer based prototype. You will not get more credit if the system backend contains a lot of functionality, so fake or stub as much functionality as appropriate if that helps you with your prototyping schedule. Here "appropriate" means what you need to support usability testing. Regardless of your choice, the submission procedure will "freeze" your prototype development at the time of submission. I expect implementation of a computer-based prototype on a PC or as a web application. Please get my approval for any different platform. Regardless of the prototyping tool you use, it is your responsibility to learn what you need about it outside of class.

Warning: To repeat the warning we gave you in an earlier project assignment, do not turn your prototype into an overwhelming programming project. Probably the most important thing in considering how to handle the design for your project is getting the right size. Students often tend to err in the direction of making it too big and too complex. Students sometimes try to maximize functionality because they think this will earn a higher grade. Some students spend long hours building in lots of functionality in the hope that it will somehow make the design look better. Remember, the goal is the user interaction design. You should studiously avoid large amounts of work to build in functionality. You need to "stub" functionality, giving only the appearance of functionality as much as possible, while retaining the ability to support the benchmark tasks in formative evaluation. This means, for example, not entering large numbers of database records. Teams that have tried to pack in the functionality in the past have regretted it! If one of your team wants to put lots of functionality in the prototype, they are not getting the point. They are trying to do what they already know best instead of learning more about the usability process. Try to talk them out of it. If that doesn't work, let me know and I will convince them. I will not have much tolerance for this kind of approach. If you are unsure about scope, ask me as your design begins to develop. 

Final things to do for this part of your project are to pilot test, pilot test, and pilot test. Make sure to do enough pilot testing to shake-down the benchmark tasks you have chosen for evaluation.

Deliverables

Inside the binder, create a "tabbed" section labeled "Project 4". Add this section to the front of your team binder. This way, your binder becomes a cumulative record of your whole project, with the most recent parts first. This section should start with its own separate cover page with (mostly the same as on the front of the binder):

• "Project 4: Usability specifications"
• Team number
• Project name
• Name of client organization
• One-line description of project
• Team member names
• "CS3724 – Fall 2000"

Number and label your items per this list:

1. To make this report a stand-alone document, repeat your synopsis or a revised version thereof.
2. Repeat the three usability goals from Project 3.
3. For each usability goal, give the two associated benchmark task descriptions. Each of your benchmark task descriptions for this report should appear just as you will give them to your usability evaluation participants. Thus, they should be expressed in terms of what to do with no hints about how to do it. In particular, make no references to any object, label, etc. in the interaction design. Make sure each benchmark task has a decisive starting and ending point, so you can know exactly what is included in timing measurements. (E.g., if the task is a search to find certain information, have the last step be, for example, clicking on a print button.)
4. Describe how each benchmark task in item 3 above supports the associated usability goal.
5. Indicate the relevant user class(es) for each task of item 3 above.
6. Describe your starting point and ending point for each of the tasks above.
7. Write the additional four or more benchmark task descriptions, indicate the relevant user class(es), and describe the starting and ending points for each.
8. Include a copy of any general written instructions you will give each participant to explain the overall process.
9. Include a copy of the informed consent form you plan to use. (A sample participant informed consent form is given in the class notes.)
10. Describe how you approached the problem of limiting functionality (where you draw the line) in your prototype.
11. The final deliverable of  this project part is perhaps the most important of your project - a prototype of the interface of the system that users can try out for evaluation. Your prototype needs to be a robust reliable tool to be ready for evaluation; in short, it needs to run like a top.

This stage focuses on developing an evaluatable low-fidelity prototype. This includes a design concept, navigation, visual design, layout, interaction objects, look and feel. The prototype must fully support (be able to "run") all of your benchmark tasks.

Start with simplest possible background for each screen in pencil or pen on full size paper, as base for all moving parts. Include only parts that never change (e.g., for calendar, use monthly "grid", but no month name). Everything else is drawn in pencil or pen on paper (or printed out from a computer), cut out and taped (in aligned position, relative to other objects) on separate plastic sheet; in general, paper cutouts of interaction objects are taped on to plastic. (Hint: Blank plastic sheets are available at Mish Mish or 14 cents each, last time we checked.)

Use an "easel" (we will show you a sample one in class) to register each sheet of plastic with other sheets; your prototype will be "executed" on this easel. During "execution" of the prototype during usability evaluation in Project 5, most dynamics (changes in screens) will be created by adding and removing various registered plastic sheets to/from the easel, not by changing the whole screen.

Make prototypes as modular as possible. The less you put on each layer, the more modular it is.  You generally shouldn't have a sheet with a lot on it; this probably will mean repeating something on another sheet. Whatever changes when a user gives input should go on a separate paper-on-plastic sheet. If a user will type in values (e.g., an appointment) use clear sheet on top and marking pen to simulate typing. Make a highlight for major selectable objects (e.g., day in month view, appointment slot). For example, use a plastic square or rectangle with a "handle" and color with a marking pen. Fasten some objects (e.g., pull-down menus) to top or side of easel with tape "hinges", so they "flap down" to overlay the screen. Use any creative techniques you wish to demonstrate motion, dynamics, feedback, changes in screen. For example, scrolling can be done with paper through slits cut in paper or plastic with all taped to plastic sheet.

Full paper screens vs. small moving parts
Some students are tempted to print out complete paper screens and use them to portray the sequencing in the prototype, with each different sheet showing a new screen state (in its entirety). You should definitely not do this! Fixed screens don't have anywhere near the flexibility that separate pieces of paper do and cannot cover anywhere near the number of different combinations of things that might come up in user task performance using your prototype.

Your prototype (and later your system) cannot be designed with just one way to do anything. If you do pilot testing on your prototype with someone outside the development team, you'll likely already see the variation in task performance that is possible. You cannot count on the GTA or an user to do anything in a given way using your prototype. Your prototype has to be flexible enough to accommodate much of this variation.

This doesn't mean you have to design for every possibility and every error condition, but the more variation in user task performance you do cover the better your prototype is and the more effective it is in doing its intended job (supporting formative evaluation of the interactive design).

Submission and Grading

Paper Prototype Submission:
If you choose to submit a paper and plastic low-fidelity prototype as your final prototype as your final prototype, submit it in a large envelope with cover sheet information on it and hand it in at the beginning of class on the due date.

Electronic Prototype Submission:
If you choose to implement your final prototype on a computer, it should still be low-fidelity (most functionality faked). You have two choices for submitting a computer-based prototype at the beginning of class on the due date:

1. Hand it in on a PC floppy disk, or
2. Put it on an Internet site and hand in a sheet of paper with complete and clear instructions on how the GTA can download it. Test out your own instructions. The GTA will make a reasonable attempt to download your prototype. Failure in this download could be expensive to your project score. In short, your submission must be downloadable and able to run on the GTA's machine. Do not just give a URL to run it on the web.

1. Verify that your prototype functions properly on more than one host machine. One important note here regarding HTML is to make sure that you use relative links. Be warned that if your prototype does not run on the GTA's system, there will be substantial negative consequences to your grade.
2. You may submit either a .tar.gz file or a .zip file. Please make sure that you build your archive with the directory structure intact so that it does not decompress 10,000 files into the GTA's directory.
3. Test the decompression of your archive file to make sure that it renders a workable directory structure. You should verify that the product runs properly in the decompressed directory structure.
4. Email the .zip or .tar.gz file as an attachment to the GTA. Make the subject line read: Project 4 Submission for Team #?? where the ?? is replaced by your team number. The body of your message must contain instructions for decompressing and running the prototype. The GTA will strictly check the time stamps. The GTA will send a brief confirmation message to each team that submitted electronically as soon as possible.

How Many Changes to Make From Pilot Testing?
On the job, usability practitioners make use of every bit of usability problem data they get, regardless of the source, to constantly improve the interaction design. Therefore, they would always try to fix any usability problems that show up in pilot testing, giving formal usability testing the best starting point possible. However, since this course is process-oriented, you should use the pilot test to make sure the prototype works flawlessly. This means to fix all the show-stoppers, but I prefer you do not make many regular usability-related changes (yet) to the prototype based on usability problems you discover in pilot testing. Write them down as part of your usability data and include them in your next evaluation deliverable (for Project 5, Formative Usability Evaluation).

The reason for this approach is that the Formative Evaluation assignment works better as a learning experience if you have lots of usability problems in the design. So after the pilot test, it's best to fix only those problems that might have a negative impact on usability testing (e.g., something important that is missing, places where the prototype breaks down, or a benchmark task description that need clarification but not added explanation about how to do it).