Home

Syllabus

Calendar&
Class Notes

Projects

Homework

Grades

Participation

CS/ISE 5714 - Usability Engineering - Spring 2008
Dr. Pardha S. Pyla

Syllabus

COURSE DESCRIPTION

Usability has become an essential requirement of interactive software. In response, human-computer interaction (HCI) has grown into a serious field of research, development, and application. To meet the specific needs of practitioners for methods and tools to develop user interaction designs with provably high levels of usability, Usability Engineering (UE) has emerged as a sub-discipline within HCI. Despite this attention to usability on some levels, much interactive software is still too difficult to use - as most computer users know. It is turning out to be more difficult to design for high usability than many first imagined. A major cause underlying poor usability is a lack of understanding of a usability engineering process for developing usable interaction designs. Although many software development teams now have roles for a usability engineer or usability specialist, software developers often still have primary responsibility for developing interactive systems. Most software engineers are not trained in usability methods and, therefore, do not have the knowledge, skills, or mindset to include usability methods in their life cycle activities. Many software developers wrongly believe that usability engineering is merely usability testing, done near the end of the development process. This course, which has been designed to focus on usability methods and a usability engineering development process, uses an integrative and cross-disciplinary approach to bring together a broad variety of topics together in relation to the problem of developing quality user interaction designs.

The topics include principles and guidelines for designing the product: effective interaction design content and interaction style. Additionally, much emphasis is given to the development process for user interaction designs as integral with, but different from, interactive software development. The development process includes management of a life cycle that must be both iterative and cost effective. User interaction development activities include needs and requirements analysis, task analysis, user class definitions, usability specifications, design, prototyping, and formative usability evaluation. It is a goal of this course to help students realize that usability engineering is an ongoing process throughout the full product life cycle, and developing the human-computer interface is not something to be done at the last minute, when the "rest of the system" is finished.

This is an active learning course, meaning that you, the student, have more responsibility than usual in the learning process. A major student responsibility is to read and study the class notes applicable to each class before that class meets. It is the student's job to manage the pace of this reading so that it fits the class schedule. No fixed reading assignments will be announced. Class time will be split between content-based lectures and in-class activities to demonstrate techniques and principles and to practice the skills being presented. The part of class time used for lectures will be devoted to highlighting course materials, questions, and discussion. Outside of the classroom, students will acquire more in-depth hands-on experience in applying these skills and techniques in a semester-long team project. In this project, students will develop a usable interaction design for their own application system in a real development project for a real client selected from the local community.

SUBJECT MATTER OBJECTIVES

The outcome-oriented objectives for the course are that each student, upon successful completion, will be able to:

• participate as an HCI- or user interface-oriented development team member to represent usability issues within an interactive software system development project,
• perform individual guidelines-based critical analysis and evaluation of a user interaction design, and
• understand the dynamics of interactive-software development in an interdisciplinary setting while working with software engineers.

These translate into the following specific instructional objectives, that each student will be able to:

• apply usability design principles and interaction design guidelines in the design and critical evaluation of interaction designs,
• relate some basic cognitive characteristics of users to interaction design,
• apply an iterative evaluation-centered life cycle for user interaction development within a broader traditional software/ system development process,
• formulate quantitative, measurable usability specifications for user interaction designs,
• construct low-fidelity paper user interface rapid prototypes and high-fidelity computer-based prototypes,
• conduct a formative usability evaluation process using appropriate users as subjects/participants,
• analyze quantitative and qualitative usability evaluation data,
• perform cost/importance and other analyses for management of the iterative life cycle,
• understand the role of software engineers and usability engineers in the development space of interactive systems,
• learn about the constraints and dependencies within and between the various activities in the software engineering and usability engineering life cycles, and
• discern the importance of communication in an interdisciplinary setting while working with software engineers.

MY PERSONAL GOALS FOR YOU IN THE COURSE

In addition to the content-specific objectives listed above, I have a few personal goals for each student. Although the primary objective of HCI as a field is to study methods and techniques to develop and evaluate computer related interfaces, many of these principles also apply to the design and evaluation of "every day" things. I would like each student:

• to think deeply and carefully about design in general,
• to become perceptive to all the design manifestations in the world around us,
• to be able to critique and analyze all the bad and good designs that surround our lives based on scientific theory and facts , and
• to become champions of user oriented design and thought.

PREREQUISITES

Graduate standing in any major (despite what you might read elsewhere, e.g., the university catalog) is the nominal prerequisite for CS5714. Some qualified undergraduate seniors may also be admitted by permission of the instructor. No official coursework is required as a prerequisite, but students should have substantial experience with computers, especially their interactive use, and an intense interest in making them easier to use. Some knowledge of software engineering fundamentals, gained either from coursework or from practice, would be very useful. We expect most students in this course to be from Computer Science and Human Factors Engineering (in ISE). We do welcome the diversity of others (e.g., art and design, psychology, communications studies, technical writing, music, etc.), but they must be interested and willing to devote some extra effort in learning the technical aspects of development processes. If concepts such as interaction design evaluation, usability, prototyping, life cycles, and development methodology mean nothing to you, this may not be the class for you.

WARNING: At the other end of the spectrum, occasionally we get students with considerable experience in HCI and usability. You are still welcome to participate in this course, but be warned that this is not an advanced course in usability engineering. Although this course gives thorough treatment to the usability engineering process, this is an introductory course.

TEXTBOOK

D. Hix and H. R. Hartson, Developing User Interfaces: Ensuring Usability Through Product and Process, John Wiley, 1993. This book is optional. It is getting a bit old, but much of its content is still current. Quite a bit of the material for this course still comes from the book and it makes a good reference book in usability engineering. The book is not required to be successful in this course. The class notes are very complete and are definitely up to date. You decide. If you decide to buy the book, you can probably find good prices for new and used copies on Amazon.com (it is not in local bookstores).

ADDITIONAL STUDY MATERIALS

Class notes will be posted as pdf files on the calendar page at least on day before the corresponding class. They contain handouts for the same PowerPoint presentations I will use in class.This is the major source of content and discussion for the course. You are required to print these slides and bring them to every class in a notebook, so we can refer to them and you can use them for note-taking in class. Most of what we do in class and in your team projects will be based on these notes.

GRADING

*Individual project grades will be weighted by amount of participation in team effort (see Team Member Evaluations below).

HOMEWORK

Occasional homework will be assigned as appropriate.

PARTICIPATION ACTIVITIES

There are several kinds of participation activities, including in-class activities, research project participation, and contribution to the class collection of usability problem descriptions and analyses. To achieve a perfect grade in the Participation category (see Gradingabove), you are expected to complete both in-class and research-oriented participation credit (PC) requirements. Contribution to the class collection of usability problem samples is a backup possibility to research-oriented participation, in case you are not able to find a research project that needs a participant.

In-class activities

Getting full in-class participation credit is easy: Just be willing to participate in each in-class activity and do a good job of it. As a key part of active learning in the classroom, individuals and teams will be asked to perform part of an ongoing analysis or design exercise in class, to illustrate the application of concepts covered in the class notes and class discussion. In assessing the "do a good job" part of this activity, we'll be looking for:

• Presence or absence of the individual
• Preparedness, knowledge of material
• Care and correctness in applying it
• Intangibles (getting into role, etc.)

Everyone begins the course with full in-class participation credit and most will retain it to the end. However, deductions can be made from an individual's in-class participation credit for various shortcomings (e.g., being absent when your team needs you for an in-class exercise).

Research project participation

To expose you to the experience of being part of an HCI research project/study, this component of the participation grade can be fulfilled by serving as a participant/subject (or other needed role) in one or more active HCI research projects on campus (or other location), for a total of four hours of research project participation. See the Research Participation page for more details. Your participation does not have to be as a study subject; it can be in almost any role that exposes you to how such projects work. Other research projects are also an option, subject to our approval. After participating in one of these projects, fill out this participation certification form, have the person in charge of the project sign it, and hand it in before the last day of class.

Alternative participation option

As a class, we will be documenting and accumulating a collection of usability problem cases. If you cannot locate a research project needing a participant, you may fulfill your research participation component of the participation credit requirement by constructing and submitting several usability problem cases (problem descriptions and analyses), using our Problem Reporting Tool. This alternative participation option, however, is intended only as a fallback; we prefer that you participate in an HCI research project. See the instructor or GTA for further information about this option.

TEAM PROJECT

The major work (and major credit) component for the course is the semester team-oriented development project. It involves defining, analyzing, specifying, designing, prototyping, evaluating, and iterating the interaction design for the user interface of an interactive system. The purpose of the project is to give you real-world exposure to all steps involved in developing a significant user interaction design.

This is a team project. I will assign students to teams, trying to balance knowledge, skills, and backgrounds, based on a demographic survey given the first day/week of class. All development activities, including writing the deliverables, are team activities. All team members are to participate in all development activities. Do not divide the overall process among the team members. Even though this might seem like a more efficient way to proceed, this leads to a kind of specialization that poses a barrier to each person learning the overall process. The is especially true for a person who gets the job of programming, at the price of not learning usability engineering skills.

THE GRADING PROCESS

The process. All assignments (in-class exercises, homework, project, and final exam) are designed by the instructor. It is the instructor's responsibility to establish grading standards and work with the GTA in grading. The GTA has the primary responsibility for grading homework. The GTA and the instructor work together in grading project reports, with the bulk of the grading being done by the GTA. The process begins with the GTA carefully reading each report and writing comments on it. The GTA and instructor meet and have a detailed discussion of each report. On the basis on these discussions, the GTA sorts the reports in order of overall quality and assigns numeric grades.

Teams will be operating under somewhat varying conditions, reflecting various real-world development situations. Therefore, expectations for different teams will vary, as will the bases for grading project deliverables, so this is not about comparing the final products or deliverables across teams. The emphasis in this class is on learning the process and your project deliverables will be graded with that perspective.

The objective part. The first thing we assess objectively is whether all requirements are met. Mechanical aspects such as formatting, labeling, grammar, spelling, following instructions, etc. are easy to grade because they are objective. Since these mechanical aspects are just expected, we don't give positive points for those, but we do deduct points if they are wrong or missing.

The subjective part. The hard part in grading is the subjective part, which is about quality of content. We lay the reports out on a table in approximate order of overall quality. During a discussion of their relative merits, a given report may be moved up or down in the sorting as we calibrate our judgments and make adjustments. There are two components to this subjective evaluation: how well requirements are met (how well you did the job) and how well you reported it. Our evaluation of these components is based on our own knowledge and experience and is necessarily somewhat relative among the project teams of the class. The "how well you met requirements" part is based on our perception of how much you put into it, how completely you pursued the assignment, and how well you understood, interpreted, and applied the material covered in class to your project.

We try to write comments on your reports about these qualitative parts, so you know what aspects of your work and writing are possible issues. Comments of this type are along the lines of: why didn't you also do such-and-such, this isn't really what we are asking for here, wording not tight, not quite as complete as it could be, a little bit rambling, grammar and spelling not the best, etc.

An example of what we mean. Here is an example from our perspective, so you can relate to what we are faced with in this part of the grading. Team A reports their findings of a client interview in a crisp, easy to digest form using summary tables and short clear descriptive paragraphs. You report your findings in a somewhat lengthy and verbose, narrative form. You cover all the important points but it is not as easy for the reader to pick them out as it is in Team A's report.

So we write on Team A's report: Nice job of clearly presenting the important points. On yours, we write: Slightly verbose; need to clarify/highlight important points. Then you come to see us and ask: Where is it verbose? We say that verbosity is a distributed thing but, as one example, here are two sentences that you could have tightened into one short one. It's your job to extrapolate that comment to the rest of the report.

The point here is that we cannot always say exactly where a specific number of points was deducted to arrive at your grade. In fact, it is usually the case that points are not deducted but your score is assigned to reflect your relative standing among all the reports, as explained in the next paragraph.

Grades are based on overall quality. Please remember that, beyond any comments we write on your report, the grades are based on *overall* perceived and relative quality. So, let's say your team got an 85 and Team X got 90 and the only apparent (noted in comments) difference between you and Team X was that you got a comment about completeness. That does NOT mean that the completeness issue was necessarily worth 5 points. The completeness comment contributes to where your report finally falls on the table, but its relative position in that sorting is what determines the numeric grade. So *that* is the answer when you come back to us and ask why you got the grade you got.

You may also ask what you can do to get a better grade. Often there is not a simple quick answer, like 'do such and such'. We are willing, however, to sit down with you and go through your report in detail and point out some aspects of the work and the writing that could be improved. And maybe we might be able to make a general conclusion about a particular aspect of your work or writing to focus on.

Bottom line. If you get a grade of, for example, 80 or 85 and another team gets, say, 90 or 95, it does not mean that you did a poor job. We have to assign some number as a grade and we use it to distinguish among the project reports from all the teams. We evaluate your assignments in detail. You can be sure this is done in the fairest way possible.

Keep this in mind, especially if you get discouraged about the project workload: Our experience in teaching this course has been that the project work separates the class into two groups: those who simply cannot keep up and who drop the course before the end and those who hang in there and do a good job on the project. Students who stay in the class, stand up under the heavy workload, and carry out the team project with dedication and enthusiasm will do well in the course in the end.

Grading appeals. If you have any questions about grading or feel that something has been graded incorrectly or unfairly, you should contact the GTA first. If you do not get satisfaction from the GTA, you can appeal to the instructor by requesting a regrading. Regrading requests must be submitted to the instructor within three class meetings after the graded work was returned to the class. A regrade will entail a COMPLETE RE-EVALUATION OF THE ENTIRE ASSIGNMENT. This may increase or decrease your original score. The regraded score is final.

This risk of a grade reduction is not to discourage you from discussing your project grade with the GTA or instructor. This is to address the problem of students who mainly want to squeeze out more points in their scores. You need not worry about this if your desire to discuss project grading is motivated toward learning and not all about getting points.

Keep all graded work until after you get your final course grade at the end of the semester.  You should check the posting of your grades periodically to be sure our records reflect your correct grades . In case your grade is incorrectly recorded, you will need to bring in the graded original in order for the recorded grade to be changed.

Team member evaluations

The parts of the project assignments are described separately in the course Web site, under "Projects". Each member of the team is expected to contribute equally to each part of the project. It is possible that the most difficult part of the project assignments is working well together in a group. Be aware of possible group problems and be ready to solve them. Don't make the mistake of taking this aspect for granted or waiting for it to fix itself; you have too much at stake.

Sometimes, despite our best efforts, some team members end up not pulling their fair share of the weight. To ensure that each team member is given a project grade reflecting individual contributions, the final project assignment is a Team Member Evaluation. Each team must INDIVIDUALLY turn in a paper copy of the confidential Team Member Evaluation Form (print from Web) as a required deliverable to report the relative effort/contribution of each person on your project team over the whole project (including yourself).

This form is not optional. Be professional and give a careful rating. The ratings on these forms will be used as weightings, as explained at the beginning of the semester, to convert team project grades into individual student project grades. The team is given a grade for each part of the project. Each individual team member's grade for each project assignment is a weighting of the team grade, where the weighting is based on an evaluation of individual contributions, collected from each team member at the end of the semester (and moderated as necessary by the instructor).

CLASS POLICIES

Reading assignments

You are responsible for keeping up with the reading of class notes per the schedule given in the course calendar. You are also responsible for knowing where we are in our class discussions, with respect to finding your place in the class notes, and setting your own reading pace to keep ahead enough to be prepared for class discussions and activities. There may also be a few academic papers assigned as reading associated with various class topics.

Skipping through material

Sometimes students say "Please don't flash through the slides in class." I need to explain that it is sometimes necessary for me to skim over some less important material in order to highlight the most important things. You should be able to keep up with this kind of occasional skimming because:

1. You are supposed to have just read the notes for what we're covering at any point in class. If you have questions about parts I don't cover in class, you are welcome to ask.
2. Class time is for review, highlights, examples, and explanations. It's not the traditional lecture format and the examples, etc. not in the class notes are a more important way to use class time than total coverage of what you can read for yourself.
3. You should have the Class Notes with you in class, so you can follow along easily without getting lost, even though we might skim over some material. You can even mark what we skip in class, to go over it yourself at home.

Due dates and times for handing in homework and project assignments

All homework and project assignments must be turned in at the beginning of class on the due date.  You should think of all due dates for assignments as firm.  The tight schedule of deliverables throughout the whole semester makes it nearly impossible to slip or extend due dates. Any assignment that you do not hand in on time will be penalized in grading. If you are not able to complete an assignment by the due date, it would be best for you to hand in as much of it as you have done.  You must prepare your assignments using a word processor and hand in a hardcopy by the due date/time.  Assignments may not be submitted via email to either the professor or a GTA (some exceptions may be negotiated in advance).

Attendance

You will not be graded directly on attendance. You are adults in a graduate-level course and are expected to attend every class.  Beyond the occasional need to be absent from class for a good reason, please consider that much of the learning for the course occurs in class. You cannot participate in this learning if you are not present.

If you have to miss class for an extended period due to a protracted illness or similar reason, we will treat your needs as a special case and I will do everything I can to help you survive.

No extra credit work

Students sometimes ask for some extra credit work near the end of the semester in an attempt to bring up sagging grades. However, university policy does not allow extra credit work to be given to any student on an individual basis.

Weather contingencies

If the university is closed (for any reason) on an assignment due date, the assignment will be due by 2 p.m. on the first day the university reopens. In such cases you should hand the assignment in to Ms. Carol Roop, the CS secretary in 2050 Torgersen Hall.

Responding to e-mail

All email concerning the class should be addressed to the GTA with a copy to the instructor. We will sort out which of us should act on the message and will make every effort to answer your email in a timely fashion.  However, you should not necessarily expect to get a reply in less than 24 hours or over a weekend.  Many times you may get a reply in less than 24 hours, but you should not count on it (e.g., to answer questions about a homework or project assignment within the last few hours before that assignment is due).  Please put "CS5714" as the subject line of your email; that will help us identify your emails more quickly.

Grades via e-mail

Neither the professor nor the GTA will be able to reply to individual email requests for final exam and/or class grades at the end of the semester, but grades will be posted on the Blackboard system by the end of the day of the final exam.

HONOR CODE

The Virginia Tech honor code is in effect for all work, whether performed individually or in teams. All assignments submitted shall be considered graded work unless otherwise noted. All aspects of your coursework are covered by the honor system. Any suspected violations of the honor code will be promptly reported to the honor system. Honesty in your academic work will develop into professional integrity. The faculty and students of Virginia Tech will not tolerate any form of academic dishonesty.

ACCOMMODATING DISABILITIES

We wish to make any accommodations needed by any student because of a disability. Please contact the instructor during the first week of class.