home purpose join itg ITG Publication job bank itg listserv hf/Web conferences library

ARTICLE Creating Web Site Designs Based on User Expectations and Feedback Jeanette Fuccella, jfuccell@us.ibm.com Jack Pizzolato, jackp@us.ibm.com IBM Corporation Introduction With the exponential growth in Web sites and Web traffic during the last five years, companies have felt compelled to create a "Web presence" in order to remain competitive. Unfortunately, while the average company willingly invests tens of thousands of dollars in building and advertising a Web site, remarkably few companies make any investment at all in determining how easy their Web site is to use. The typical result is a Web site that fails to meet company expectations over time because it has failed to meet user expectations and needs. This article outlines a powerful, but relatively straightforward, methodology for designing Web sites based on user expectations and feedback. This methodology has been used within the IBM Corporation to create a wide variety of intranet and Internet Web sites, including the company's Java site (www.ibm.com/java) and its eNetwork product family site (www.software.ibm.com/enetwork). The success of this methodology has been demonstrated through increased satisfaction ratings, increased "visit" rates, positive write-in feedback, and most importantly, the longevity of the design. The structural framework for IBM's Java site, for example, has been in place since August 1997, and based on continued positive feedback from users and the adaptability of the design, there are no current plans to change this structure. Usability and the Web In recent years, software designers have retooled their processes to involve actual users in the early stages of software design. This "user-centered" approach to design has led to a general improvement in software interfaces and to an improved correlation between product features and functions and user wants and expectations. Today's software user expects a product's interface to be easy to use. So why aren't businesses demanding that their Web sites be easy to use? In part because they don't know to expect it, and in part, because most Web site design firms don't offer that service. Web site design is still a fairly new and emerging discipline. As a result, there are few publications that discuss effective methodologies for integrating user-centered design into the overall Web design process. This is true despite the fact that Web users have even less incentive than software users for learning to navigate a difficult design. If a user can't quickly find the information they're looking for on a Web site, they will leave the site and may not return. In this case, not only has a lot of money been wasted, but the experience could also negatively affect the user's perception of the company itself. The intent of this article is to contribute to the small, but emerging body of literature on Web site usability by offering a practical and tested process for creating the high-level structure of a Web site based on user feedback. This process is geared toward simple, relatively hierarchical sites, and can be completed within two to three weeks. On our Web site development team, the usability engineer is primarily responsible for completing this process. The end result is a user-validated "wire frame" or skeletal model of the site that the rest of the design team can then flesh out. Because the wire frame provides a low-fidelity version of the projected site that can be quickly changed and refined, and because we rely on user feedback to resolve design disputes, we've found this process has significantly shortened the design phase of our Web site development cycle. Before moving on, we should note that the following steps are intended to provide Web site designers with a "quick and dirty" approach for letting user feedback drive the design of their sites. Web site design, like software design, is a complicated process with many variables. This article will not be able to cover every aspect of user-centered design and its role in Web site design. Step 1: Audience Definition As with software interface design, the ability to create usable and useful Web site designs is highly dependent upon the availability of a crisp audience definition. In many organizations, the marketing team defines the target audience for the site. However, this audience definition may not be detailed enough to create a highly usable and competitive Web site. In other cases, the Web site audience is different from the non-Web audience that marketing has previously been targeting. Conducting User Surveys. The easiest and most cost-effective means for collecting audience definition data is to conduct a survey. There are essentially two different methods for conducting a survey: active and passive. Active survey collection assumes that you or someone on your team is going to actively recruit people to complete your survey. One common method is to e-mail members of your target audience and either send them the actual survey or point them to a Web page where they can fill out the survey. If there is an existing site, an e-mail distribution can be created from users who have sent in feedback or have registered with the site (to download a product, for example, or to obtain services or information). If you do not have e-mail addresses for your target audience, you can purchase an e-mail list, either from a publishing/advertising company or from a company that specializes in selling targeted e-mail distribution lists. Another method is to seek out places where your audience is likely "hang out" - either virtually or physically. Newsgroups and mailing lists are good virtual hangouts. User group meetings are good physical hangouts. Passive survey collection is the easiest method of data collection, but it requires an appropriate existing site to host your survey. For a passive survey, the Web site owner merely places a pointer to the survey from somewhere on the site (preferably the home page). Another passive technique that has been tried is to use banner advertisements to interest users in completing a survey. We do not recommend this technique, however, because it may draw users that would not typically visit your site, especially if an incentive is offered for completing the survey. With any type of survey collection, the number of respondents can easily be increased by offering an incentive. The problem with incentives, however, is that they can attract a non-representative sampling of the total population visiting the site. While it is not the intent of this article to review methods for analyzing survey data, for most surveys all that is needed is an experienced CGI programmer. We have also used database software that provides an interface to Web to collect and process survey data. For extremely large and complex surveys, many companies now offer specialized software that streamlines the survey distribution and data collection process. Survey Questions Survey questions typically fall into one of three categories: professional profile (what is the user's job title and what do they do for a living) surfing profile (how, when, and why the user uses the Web for job-related information) site usage (likes and dislikes; tasks users would like to perform) If the target audience for the survey does not currently use the Web site or if the site has not yet been designed, then the site usage questions should focus on the primary competitor to the site. An effective survey will also ask what tasks a user would like to accomplish using the Web that they cannot currently do. Such a question will help to identify the primary user tasks for your site. Before developing survey questions, it is critical that the survey author clearly understand of how each piece of survey data gathered will assist in the design and creation of the Web site. This will eliminate extraneous questions or questions of marginal value. Unnecessarily long surveys result in incomplete surveys (particularly if there is no incentive to answer questions) and might even harm the user's perception of the Web site. However, the reverse is also true: if you leave out important questions, you may not get another chance to ask those questions before you must complete your design. Adequate time should be set aside for all team members to review the survey and to provide input. See Appendix A for sample survey questions. Step 2: Requirements and Task Gathering After the designer has gathered data about the target audience, the next step in the process is to gain a better understanding of the Web site content. This is accomplished primarily by identifying and prioritizing current and future tasks and requirements for the site. In this process, content requirements for the site are referred to as "objects." A Web site object is any piece of information that can be found on a particular Web site. Web site objects are defined by the site designer and can be as specific or general as necessary in order to obtain meaningful results. Examples of objects on a software marketing Web site include: white papers, FAQs, downloadable code, brochures, product support phone numbers, success stories, and so forth. The process for identifying these objects depends upon the goal of the exercise and whether there is already a Web site in existence. If there is an existing site, and the goal is to better organize existing information, the designer can identify the Web site objects by simply going through the pages of the existing site and creating a list of objects. If the designer is creating a new site or wishes to enhance the information on an existing Web site, a requirements-gathering activity should be conducted to determine user expectations based on the goals and missions of the proposed site. There are many different methods for collecting requirements for a Web site, and the method chosen will depend on the amount of time and money available to the designer. See Table 1 for descriptions and comparisons of these different methods. Table 1: Comparison of Requirements and Task Gathering Methods Focus Group Focus group sessions are conducted using one of two methods: traditional or electronic. In traditional focus group sessions, a moderator leads a verbal discussion with a small group of individuals (usually less than 10). Because data capture is difficult during the session, these types of sessions are often videotaped and then transcribed. Electronic focus group sessions utilize groupware software to capture electronic "discussions" among participants. Electronic focus groups are much more structured than traditional focus groups and allow for less verbal discussion (although there still is a fair amount of verbal discussion). Because the discussions are captured electronically, report data is available immediately after the session is concluded. Additionally, because electronic focus group sessions encompass a variety of different activities, participants will tolerate longer sessions than are practical using the traditional method. Pro's Can collect large amounts of data in short period of time Fairly quick and easy to run Electronic sessions can provide fairly detailed reports immediately after session Con's Costly; usually requires a professional facilitator and/or moderator Group is limited to less than 20 participants Traditional sessions require a significant amount of analysis and interpretation Iterative Surveys Using traditional survey techniques, requirements can be gathered and prioritized through a series of surveys. During the first survey, requirements are gathered through a series of open-ended questions. Participants can be prompted with a list of known requirements or asked to start from scratch. After collecting the data from the first survey, the researcher is responsible for removing duplicate entries and clarifying vague responses. The second survey includes the compiled list of requirements for participants to rate or rank based on importance (or other factors, if relevant). If desired, follow-up surveys can be conducted to gather further detail on specific requirements. Note: The results from this approach will closely resemble the output of an electronic focus group. Pro's Remote participation, including non-U.S. participation, adds nothing to the overall cost Facilitates non-U.S. participation because no travel is required Large sample sizes can be used without significant increases to cost or overall data analysis time Con's Time consuming. Because the activity consists of series of surveys, each built upon the responses from the previous survey, the entire process takes between two and four weeks to complete Exploratory Surveys If a large survey is already planned, then the simplest and least costly method of gathering requirements data is to simply ask users to list the specific content items they would like to have on the site. See Site Questions 3 and 4 in Appendix A. Pro's Inexpensive and simple Survey a large sample size in a relatively short period of time Con's Data will likely be difficult to analyze and compile; Requires a follow-up survey to prioritize requirements Scenario Building Exercises Scenario building exercises can be done in conjunction with other requirements and task identification activities with little or no additional cost. See Appendix B for sample questionnaire. Pro's Inexpensive and simple Complements other data on requirements and tasks Gives users a context in which to specify requirements and tasks Con's Works best in a one-on-one situation where respondents receive detailed instructions and example scenarios. Competitive Review This can be done with or without users. After identifying competing Web sites, the researcher conducts a thorough and systematic review of these sites. The review should focus on content and functions that are missing from your existing site or site plan. Pro's Inexpensive and simple If conducted with users, you can also gain insight into the value of specific content and features Con's Time consuming, especially if done with users Without users, it is difficult to gauge the value of the content and features found When compiling the content requirements for a Web site, include all objects in your list, regardless of whether the objects can be incorporated into your immediate design or not. Avoid the tendency to focus on the content you have on hand and to ignore suggestions for future content. Taking the longer view will help you create a flexible design that can accommodate expansion over time, including new categories of Web objects. Step 3: Information Organization During this step, users are called upon to organize and structure the Web site objects identified during the previous step. The output of this step is a wire frame model of the site. Card Sorting The purpose of a card sorting activity is to better understand the user's concept of how the information on the Web site should be organized. During this activity, each user is given a stack of "cards." Each card contains the name of one Web site object, and the entire stack represents all of the Web site objects identified during Step Two. Various tools, including labels, word processors, index cards, and Post-It (TM) notes can be used to create the cards. The method we prefer consists of printing the objects on labels and then attaching the labels to index cards. This method allows us to easily create multiple sets of cards. In addition, the index cards provide plenty of room to write. During the card sorting activity, users are given the stack of cards (arranged randomly) and are instructed to organize the cards in any way that is meaningful to them. Users can create any number of groups and each group can have any number of cards in it. Users are also given blank index cards in case they come up with new Web site objects (content requirements) during the activity. They can then write their new requirement on the blank card and organize it with the other cards. Also, if users want to put the same piece of content into two different categories, they can create a second card and place each card where they want. When users have completed organizing the cards, they are asked to provide a description for each group. The description given by the user does not need to be short and concise (as a Web site label might be), but rather should distinguish why the objects in that particular category were grouped together. After the user has described a particular category, we staple the cards in that category together and write the description on the top card. Evaluation of the results from this activity can be quantitative or qualitative, as desired. We prefer a more qualitative approach due to the low number of individuals participating in this activity (usually 5-10). Our method for evaluating the cards is very simple and resembles the game Concentration. We start with user #1's set of cards and lay each stapled category on a table. Then, we sort user #2's cards. If user #2 has a category that matches one of user #1's categories, we place the cards directly on top of each other. If there is no match, we create a new category. This process continues for each user's stack of cards. After completing this process, the overriding structure of the site becomes fairly apparent. For example, there will be some Web site objects that are always grouped together and others that are not. Also, certain category labels/descriptions are used consistently by different users to describe the same type of content. Don't worry if users put information into different places; that will be resolved soon enough. Category Identification Before beginning the category identification activity, the designer will need to make an initial decision about the category labels for the site. The results of the card sorting task should be the foundation for this decision. During the category identification activity, the user is given the entire set of Web site objects along with the category labels for the site. For each Web site object, the participant is asked to indicate the category to "click on" in order to find that specific object. (See Appendix C for a sample category identification questionnaire.) When the data is collected from all the participants, a measure of consensus should be obtained. That is, the designer should calculate, for each Web site object, the percentage of users that agreed on each category label. Measurements gathered in previous research suggest that designers should set a goal of a minimum of 70 percent of the Web site objects reaching at least 80 percent consensus. For Web site objects that are consistently split across two categories, usually the decision is made to link to that content from both pages. Information organization is comprised of four activities: card sorting, category identification, category description, and category labeling. Because the results of the card sorting task feed directly into the remaining three activities, it must be performed first. The remaining three activities are typically performed simultaneously and repeated as necessary. See Appendix D: Sample Category Identification Results Category Description For the category description activity, the user is presented with the proposed category labels for the site (the same labels that were used in category identification) and is asked to describe the information they would expect to find if they clicked on that particular label. The description should consist of a brief two-to-three-sentence definition of the category along with examples of items the user would expect to find under this label. After users complete their description of the label, they are then asked to rate the level of confidence they have in the description they just provided. (See Appendix E for a sample category description questionnaire.) In evaluating the results from the category description activity, the designer is looking for: 1) high confidence levels from the participants, and 2) correct definitions and examples for each category label. An additional side benefit of this task is that, in their descriptions of the category labels, users will provide additional content ideas that the designer had not previously thought of. See Appendix F: Sample Category Description Results Note: If the same set of users are completing both the category identification and category description activities, the order of these activities should be balanced to control for order effects. (That is, while one half of the participants perform the activities in a certain order, the remaining participants perform the activities in reverse order to minimize order bias.) Category Labeling Category labeling is an activity that is most beneficial when you are having difficulty identifying the best label for a particular category. During this activity, users are presented with sample contents for a particular category along with three to five labels that are descriptive of that category (this can be done for all categories or just the ones you are having difficulty with). Users are asked to scan the information and choose the best label for the information or to enter a new label in a write-in field. When analyzing the data, you are looking for high consensus for the labels chosen. If you are having difficulty achieving consensus, it might be the result of the poor category groupings, in which case further iterations with the category identification and description tasks are necessary. Note: If this activity is performed with the same set of users as the category identification and description tasks, it should be performed last. You also will not want to perform this task until you are fairly confident of your site organization scheme. Wire Frame Validation During this last step in the process, the designer begins to use the previously collected data to prototype concepts for the structure of the Web site. As in software design, it is important to conduct iterative testing with low-fidelity prototypes before investing a significant amount of time and energy in designing the final product. Wire frame validation allows us to perform this type of iterative testing, with the added benefit of being able to hand over a Web site structure to the site author to use as the basis for their final design. The "wire frame" is a simple model of the site used to identify the main navigation and the content of secondary pages. The model, which contains no artwork and does not reflect the actual design of the pages, identifies the structure of the site and the location of content. Because of its simplicity, the wire frame is an excellent tool for low-fidelity usability tests that evaluate the overall structure of the site. In particular, we found that the absence of graphics allowed users to focus on specific content issues rather than the "look" of the site. The themes during this stage are simplicity and speed. The process of involving users should not add significantly to design and development time but rather enhance and expedite the team's ability to make design decisions. For example, if there is a dispute over two different designs for the same information, the usability engineer can quickly create two wire frames containing the same content and gather comparative feedback on each. Overall, usability evaluations at this stage should not require more than half an hour and should be scheduled to allow for rapid iterations. Our team, for example, typically schedules usability evaluations for Monday, Wednesday, and Friday to allow for iterations on Tuesday and Thursday. Conclusion Having a presence on the Web is not the same as having a successful Web site. To reap the full benefits of their Internet investment, companies must ensure that their Web sites are usable, easy to navigate, and meet user expectations. The methodology presented in this article gives site designers the tools they need to collect and apply user feedback throughout the design process without significantly impacting budgets or schedules. Appendices: Appendix A: Example Survey Questions Appendix B: Scenario Building Questionnaire Appendix C: Category Identification Questionnaire Appendix D: Sample Category Identification Results Appendix E: Category Description Questionnaire Appendix F: Sample Category Description Results Appendix G: Sample Wire Frame Home Page © Internet Technical Group Last update: June 1, 1998 URL: http://www.sandia.gov/itg/newsletter/web_design.html hosted by Sandia National Labs Disclaimer: Neither Sandia Corporation, the United States Government, nor any agency thereof, nor any of their employees makes any warranty, express or implied, or assumes any legal liability or responsibility for the accuracy, completeness, or usefulness of any information, apparatus, product, or process disclosed, or represents that its use would not infringe privately-owned rights. Reference herein to any specific commercial product, process, or service by trade name, trademark, manufacturer, or otherwise does not necessarily constitute or imply its endorsement, recommendation, or favoring by Sandia Corporation, the United States Government, or any agency thereof. The views and opinions expressed herein do not necessarily state or reflect those of Sandia Corporation, the United States Government or any agency thereof.