CS 3724 Project Overview

In this class we have examined various principles and guidelines for the development of user interfaces.  All too often these rules are overlooked or ignored, resulting in an interface that is difficult to understand and use. An especially challenging area of interface design involves notification systems.  Notification systems attempt to deliver current, important information to the computer user in an efficient and effective manner while they are busy with other tasks.  Notifications are involved in alerts, presence and awareness updates, location and context awareness, communication initiation, instant messaging, and information-on-demand (information that is "pushed" to the device according to a standing request from the user).  Although these systems certainly include off-desktop and ubiquitous devices (like wearable and vehicular systems), you should limit your project to a platform that you have access to for development and testing (which I suspect will mainly be desktop interfaces or large screen displays).

Description

This is a group project, with groups consisting of 3-5 people. Each project group will be graded as a team; that is, each person receives the same grade. However, I will poll team members to make sure that all members are contributing. Lack of participation may precipitate an individual reduction of grade. Within the team, you must negotiate on how much and what each person will contribute. Think carefully about your team members: Where do people live and what hours do they work? Where will you meet? What skills do the different individuals bring to the group (computing, programming, design, evaluation, statistics, etc.)? I would strongly encourage you to form a heterogeneous team full of individuals with varying skills.

There are many possible domains that notification systems support, but you will select one of the following (or email the professor if you think one not listed):

• collaborative work support (especially document editing, meeting support systems, and coordination tools)
• general information-of-interest windows (news, weather, sports, stocks, traffic, etc)
• status update screens (such as those supporting network management or describing complex systems)
• reminder systems (linked to calendars, email, tasks list, etc)
• recommender systems/software agents
• entertainment applications

Remember, notification systems are unique in that they provide these services while users are engaged in another task!

The phases, due dates, grade breakdown, and requirements of the class project for CS 3724 are:

• Phase 0 (due Sep 5), 5%:  select groups, system domain, and find at least one example system
• Phase 1 (due Sep 26), 15%:  identify design models and potential usability problems for three existing notification system interfaces within your domain
• Phase 2 (due Nov 7), 15%:  address these problems by developing an original system or redesigning two of the existing systems
• Phase 3 (due Dec 10), 10%:  conduct a usability test of the solution and analyze the result
• Final Presentation (in class, Dec 5 & 10), 5%

All submissions must be emailed to Christa Chewar at cchewar@cs.vt.edu (Phase 0 only) or posted to your group webpage prior to class time on the due date.

Requirement Details

Phase 0 of the project is straightforward: come up with your group, notification system domain (see above), and an example interface of a notification system from that domain. This example system may be an interface that you personally use or have available, or something you find on the Internet, in product reports, in the library, or in a store--the more details available about the system, the easier your efforts will be.  Therefore, systems detailed in scholarly papers (conference or journal) may be best.  Feel free to borrow or reuse ideas from previous or ongoing projects from other classes or from jobs. If you are having trouble finding a system or enough information about a system, you may want to reconsider your domain selection, since you will have to find two more systems for Phase 1.  You will establish an online project report book that you will update throughout the semester. By the Phase 0 due date, the report book should contain 1) a list of team members and their anticipated roles, 2) the notification systems domain with a brief (one-sentence) description of the domain, and 3) a short description (a few paragraphs) of the example system you found.  You may want to provide a link to a demo, screenshots or photos, and a discussion of its functions and features, as well as the information that it presents. Email the URL for your team to the TA.

Phase 1 of this project asks you to find two additional examples of notification systems and discuss their effectiveness.  This requires you to:

• describe two additional example systems in a similar manner as the first one (include that here as well).  It may also be helpful to provide a short summary or table of all three that compares and contrasts supported functions, features, and information types. 
• discuss what you think the designers were targeting as a design model (according to Norman's definition of the term).  Focus on what they thought users would want in terms of interruption from their primary task, immediate reaction to the notifications presented, and long-term comprehension or memory of information presented over a period of time. You may be able to contact the designers of the system to best understand this.  Which did they think were most important and least important?  Justify your opinions based on information they provide about their system and the design decisions they made.  It may be helpful to reconstruct scenarios (that address tasks, situations, and users) for each interface that focus on these three ideas (interruption, reaction, and comprehension).
• identify usability problems that are apparent with in each system.  Try to relate the problems that you identify to the design models you inferred, and again, focus on interruption, reaction, and comprehension.  Be specific, and mention particular information and interaction design problems (i.e., "the use of tickering animation is non-interruptive, but important information that should be reacted-to is difficult to pick out quickly").  Whenever possible, discuss the problems in terms of the usability guidelines that we have been studying.

This should provide for you an understanding of the goals and problems in existing systems. The deliverable for phase 1 is a 1500-word writeup in HTML linked to your project web page by the due date.

Phase 2 requires you to address the problems within the notification systems domain that you identified in Phase 1 with either a new system design or two design modifications (of the systems you analyzed).  Your system(s) do not have to be completely operational, but the interface should be so that you can conduct user testing with it.  For this submission, you will demonstrate your system(s) and provide a written justification (approx. 1500-words) of your design decisions, which includes:

• a self-contained summary of the problems that you are targeting from Phase 1 and the solutions you propose.  Stress why these problems are important to solve.  Ensure that you articulate your design model, and discuss these anticipated user goals in terms of interruption, reaction, and comprehension.  Using scenarios may be helpful.
• an explanation of how your interface(s) is designed to look and act, and how your interface(s) avoids the problems of previous interfaces without adding new problems.  Discuss the design guidelines that you found most important in designing the new system. Include screen shots of the important states in your system.
• a description of your planned evaluation methods. What combination of questionnaires, interviews, ethnographic observation, usability tests, software logging, etc will you use? Explain why you think your choices are appropriate for your interface(s). Also, be sure to think about how to instill your design models goals in your test participants.  This should help define the final phase of the project.

It is not necessary to email or send your writeup to the TA, but please make sure it is linked to your Web page by the due date!

Phase 3 of this project asks you to evaluate your new interface. You will need to conduct an experiment, analyze the results, and draw conclusions.  You will use participants from two other groups (assigned on the project Web site ) to conduct a heuristic evaluation and obtain survey data (use the forms provided).  As discussed in class, when running an experiment you will have to prepare a consent form for your participants. Here is a sample form (in MS Word format)--submit all consent forms and evaluation forms to Christa. The deliverable for this phase (due December 10 by midnight) is a 4000-word writeup in HTML or a .doc file in publication format (use  the ACM conference style template). Place the writeup in your group directory and link it in to your main project page. The writeup should include:

• a self-contained summary of the problems that you found in example notification system interfaces within your domain, why they are important, and the solutions you implemented (phase 1 and 2).
• a detailed description of the evaluation you conducted. Explain why you think each evaluation method as well as each question/task/etc is appropriate for your interface. If you used questionnaires, list the questions and discuss why you chose them (discuss the type and contents of the question). If you performed usability tests, discuss the subject pool, the dependent and independent variables, the hypothesis, your experimental design, and your evaluation methods.
• a summary of the results of your experiment, including both numerical data from usability tests and user opinions from interviews and questionnaires. Use methods that are appropriate for the type of data: charts, graphs, tables, written summaries, and so forth.
• conclusions that can be drawn from the experiment and future work that should be done. Design is a continual process, and your experiment should expose good and bad aspects of the interface(s)--be sure and discuss both. Assess how well the design model and the system image matched, and speculate on effective redesign strategies.  Comment on whether you think your design ideas could be influential in your notification systems domain--have you uncovered a novel design discovery?

Ensure this writeup is a self-contained document such that a person with no knowledge of your efforts can read the document and understand the nature of your project.  

Final Presentation.  Your project presentations will consist of an 8-10 minute presentation of your work plus a 2 minute question period. Because of the large number of class groups, the time period will be rigidly enforced. Your presentation should address an overview of the three systems you found, the interface that you developed, the experiment that you conducted, and your results and conclusions. The most important part of the presentation is to convince your classmates (many of whom will have not yet seen your work) that you have addressed an interesting problem with an innovative solution. One common error is to jump too quickly to your results without sufficiently motivating the problem!

I encourage you to prepare a Powerpoint presentation. If you choose to do so, please email me (mccricks@cs.vt.edu) the Powerpoint by midnight on the day of your presentation. I will put the presentations on my laptop. Also, bring a copy of your presentation on floppy (just in case). Let me know of other special needs that you have.

Contact Information: