Investigating Web Usability
Rakhi Rajani, rakhi@dircon.co.uk
Abstract
Considering how users conceptualise a web site upon use is important in understanding their needs and the manner in which they structure their use of a site. A study carried out at the Centre for Information Environments, Brunel University provided some interesting findings about how users conceptualise a web site after a short period of use.
The Approach
The focus of the study was to consider how a user conceptualises a web site after a short period of use. In order to answer this question, a sub-set of questions was identified (these will be addressed further on in this article). These questions were to be the guide by which we observed users at their tasks.
Considerable time was spent researching aspects of human perception from a psychology perspective and this was accompanied by an intensive look at what current usability design guidelines and methodologies were proposing with relevance to web site design.
The notion behind this was to also consider how applicable 'conventional' interface design principles were to a web based interface. Principles were considered and analysed with respect to the implications of the WWW and their worth and applicability was documented. Also considered were web development guidelines such as those proposed by IBM and Apple etc.
The details gathered here influenced the development of the test web site with the site containing some supposedly 'usable' and 'unusable' pages. The content of the site pertained to the findings of the study into the Psychology of Perception and Usability Design Considerations.
The 'Test' Situation
A total of 17 people participated in the tests with 8 people being observed locally and the other 9 testing the site 'remotely'. Of these users, 10 had significant computing experience and had used the Internet before, and
all 10 had expressed an interest in the proposed content of the site. Of the remaining 7 users all had some computing experience, 3 had little experience of using the Internet and 4 possessed a moderate amount of Internet knowledge.
The users were exposed to a brief orientation detailing the purpose of the tests and their role in the process, after which they were given approximately 5 minutes to browse the web site. A task was then generated such as that which they might ordinarily have set out to achieve - for example, the user wanted to develop a web site and thus had a desire to find out about how the differing use of colour on colour can effect usability.
The task had to be accomplished in approximately 10-15 minutes as this was estimated (through research into other usability tests) as the absolute maximum amount of time an average 'information hunting' user would spend searching a site before exploring other options. Following this task, during which local participants were videotaped, audiotaped and observed, users were asked to draw a 'conceptual model' of the site highlighting their overall concept of the site. A questionnaire was then answered and the users were debriefed.
For those users who participated in the local tests, the room layout was devised (as shown in Figure 1) from literature on the organisation of test rooms. Standard room lighting was used and there was no deliberate attempt to simulate any particular contrived environment.
Figure 1.
Room Layout
Results
The results of the tests on the developed web site illustrated the following: (Please note, in order to gain insight into the question of how users conceptualise a web site, smaller sub-questions were identified. It is the results to these questions that are detailed here, followed by consideration of the more broad question of conceptualisation).
i) "When they use the site, do users find themselves ‘in the site’? – That is, do they feel they are in the world of the site or is it just a browsing experience?"
Users had a task, and they set out to achieve it. Sometimes interesting pages distracted them but they did not engross themselves 'within' the site. Users demonstrated that in the case of an 'information' site (versus a
'virtual world') they used it as a reference book, taking notes, 'flicking' through pages, consulting the index etc.
The participants often became 'engrossed' in reading about a particular topic if it interested them, but unlike the imaginary world of a work of fiction, in an information resource, even if it was interactive and created a sense of participation, the task and the accomplishment of the task dictated behaviour and thus the aim was to find what they were looking for!
Although this behaviour was predicted and the main consensus was to find what they were looking for, it was still interesting to see users revisit pages of interest if they were unable to find what they wanted or if they found what they wanted and had time to spare (even though they were given the option of just stating they had finished even if they had not used the allocated time!).
ii) "Is it “walk up and use”, that is, Are the operations and navigation’s of the interface intuitive?" The navigation scheme (when found) made exploring the site very easy. However, the method by which this could be achieved was not made obvious aside from an instruction at the top of the site to scroll down in the top frame to find the navigational aid.
Only one person noted the existence of the instruction at the outset and scrolled down to find the aid by which they were able to complete the task within a few minutes. Of the other users, about 1/2 noticed the instruction and scroll bars and started to scroll to find the aids. However, even when they did this, the majority did not scroll down as far the frame would let them. They stopped half way when they saw something that might help them - although the aid was not of any help unless they scrolled down some more to find the 'page-fetching' buttons! - Indicating a similarity with "Banner Blindness" (see Benway, J. P. 1998 - http://www.ruf.rice.edu/~jan/BannerBlindness.pdf ) (see also article in this issue of Internetworking by Benway and Lane on Banner Blindness)
The questionnaire results showed that most users thought the navigation was intuitive and in fact when asked to carry out an instruction on a main page, they were able to follow these with ease and navigated around the site using more 'conventional' means such as the 'back and forward' buttons and links at the bottom of pages.
iii) "Does the user need any prerequisite information before using the site?"
The majority of users stated that they preferred sites which had a clear site map that was instantly visible. They claimed they would use this to structure their tasks.
As web sites do not generally come with 'user manuals' they must be instinctively usable and the results showed that pages and parts of the site that were designed taking into consideration points raised by the psychology of perception and some design guidelines, were more intuitive.
Thus some conventional design principles can be successfully applied to web design despite the differences in web based interfaces when compared to more conventional 'manipulation' interfaces. (These web design considerations are detailed in Table 1)
Table 1.
Issues Affecting the Design of WWW Sites
| Consideration |
Description |
| Maintenance |
Once posted, a page has to be frequently maintained to
incorporate the ever-changing web technology. Also, if the developer
does not maintain the links, then these too have to be checked to ensure
that the owner has not withdrawn the page. |
| Speed Of Access |
This is perhaps one of the most contentious issues in
WWW design and often involves conflict. To aid the speed of retrieval it
is essential that only necessary information be displayed. Graphic
images that aid usability may cause sites to download slower. It may
thus be necessary to reduce the image content of a page and substitute
it with text. An apt balance between speed and design for usability
should be established. |
| WYSIWYG | With the WWW, it IS true to say that what YOU
see is what YOU get. However, what YOU see is not always
what everyone else gets. Every WWW user will have different browsers,
different screen settings, different graphics cards and different
computers. Designing for the average majority is the key, but still
difficult? |
| Navigational Aid |
Navigational aids such as navigational bars at the top of a page, a content index, or site map on every page of the site will aid usability. As many WWW sites are non-linear, the use of tools to aid navigation around a complex site that spans over many pages is essential in helping a user find their way. Link colours should be consistent. The destination of links should be obvious. |
| Anonymity |
It is possible to predict a 'possible' user base, but
web designers are essentially designing for an 'unknown' audience. |
| Design Traits |
These include Clarity, Accessibility, Consistency, Simplicity, Navigability, Integratability, Feedback, Informativeness of Displays and Speed. |
| Limitations Of HTML |
HTML structures a web page, but it is not a layout tool. The use of tables, frames and tags etc allows for some layout procedures, but these are not ideal. However, as technology is constantly changing, new versions of HTML (v4.0) and DHTML (that some browsers support) are evolving, as are other techniques such as the use of Java and XML etc. Also, certain typefaces are not supported, and
designers use graphics instead, but these slow down the access speed.
Thus, certain conventional design traits are unattainable. |
iv) "Does Appearance Surpass Content?"
Users seemed to associate the content of pages with the images they had seen on them. We also observed that the overall layout, images and pages' background colour were immediately commented upon - Images and interactive components attracted users and they were drawn to pages that contained graphics and
animation.
If users were looking for textual information and clues to completing their task, they often did not scroll down in a page to find the information and did not even use the browser's own 'find on this page' facility. What most users did was to look at the first few paragraphs, and maybe quickly scroll down the page looking for large 'clues'. However, they did not really absorb the content.
Nevertheless, when specifically asked if appearance surpassed content, the vast majority of users said that content in fact surpassed experience!? From this we concluded that graphics were used as 'landmarks' and although appearance was perhaps more important in the initial stages of site navigation, once the user had oriented himself in the site, the content then became more important and many users would struggle to identify content on a page that quite frankly 'looked awful'!
v) "How Does the User Navigate Around the Site?"
The majority of users looked for links at the bottom of the page as their main source of navigation. They assumed that they were there and intuitively scrolled to the bottom of the page.
On some pages, links had deliberately been excluded and in such instances, we observed that users hesitated for a while before deciding upon how to navigate out of that page - in which case the browsers BACK button was usually used.
It was clear that users brought with them, existing notions of how to move around a site and so intuitively used this information.. This was common regardless of the level of expertise.
The site was essentially split into four main sections or topic areas. We observed that when moving around the site, users explored one of the four main topics before moving onto another and so explored the site in four stages. Thus, our design and structure of the site influenced the users navigation of the site. The design however, was not intentionally aimed at structuring the users ability to explore the site.
vi) "How does the user structure his/her tasks when using the site?"
Approximately half of the users launched straight into accomplishing their task and starting delving into the site to find the information. However, the other half consulted their previous experience of browsing the site and
thought about what they had to do before doing it.
Many users looked for specific page titles when trying to accomplish their task and approximately half of the participants looked for icons and images and structured their tasks around such 'landmark' positions.
On some occasions, if the user saw a page that interested them, they were distracted by it and often lost their stream of navigation leading them to think again about what they were doing.
However, in many instances tasks were not necessarily structured and observations showed that users went 'fishing' for information, throwing in a line and hoping they were in the right place before actually considering where in the site they were. Neverthless, the audio recordings from approximately half of the users demonstrated that many of them did spend some time thinking about how they should proceed one from section of the site to another in an attempt to structure their path.
vii) "How does the user establish relationships between the pages? "
Users 'bunched' pages together according to topic areas that were defined by the design decisions we had made - they used this schema to establish relationships between ‘like’ pages.
Perhaps if we had not structured the site in this manner there would have been more opportunity to see how users created relationships that were not so greatly influenced by design decisions be they deliberate or otherwise!
viii) "Does the user live in the moment?"
They absolutely did live in the moment! Their concentration spanned to that moment in time. Users knew where they were and what they were doing at a certain point in time. Most participants would remember where in the site they had been previously with reference to not more than 2-3 pages.
Nevertheless, many recalled some detail about where they had been 5-6 pages before. They were not always concerned with what had gone on before unless it had direct relevance to their task.
What was most interesting to note was how if the standard background/foreground colours of pages suddenly changed, then although this did distract the user for a brief moment, they soon adapted to the new colour scheme and continued with their task.
Users did not always plan ahead but decided how to proceed by using information they had either gained on that particular page, or they just dived in and took a guess!. They really did seem to live in the moment!
ix) "How ‘successful’ is their experience of using the site?"
The interactive elements of the site helped to maintain a users attention. The fact that users could experiment with different colour schemes when considering the use of colour on colour meant they could take a 'hands on'
approach to their information gathering. This assisted in allowing them to process the information with ease and when asked, it was features such as these that made their experience of using the site successful.
Our observations echoed these comments while the video and audio evidence also showed changes in facial expression and intonation that portrayed a sense of enjoyment/excitement when such features were encountered!
Conclusions
The results of the above questions assisted in forming ideas about how users conceptualise a web site after a short period of use. What we concluded from the questionnaire results, observations and users drawings of their conceptual models -- all of which represented either spider or tree diagrams and hierarchical organisation drawings (see Figure 2) -- was that users did not form an overall 'pictorial' representation of a site even if they did use graphics as landmarks.
Figure 2.
User's Representations of the Site Organization
The users seemed to conceptualise navigation schemes and build hierarchies of how links are structured. This is in line with their aims. In the test situation, the scenarios and tasks described were ones that sought the retrieval of information and so upon browsing, in their quest for efficient information retrieval they used 'survival' techniques by building up an idea of how information was structured by grouping that information. So, if you want a user/visitor to your site to go away with a pictorial representation of your site, draw the picture for them!
We observed that users really did 'browse' when looking for information. In fact we can compare our observations here to that of observing someone browsing through a magazine where the person may pick up a magazine, flick through some pages, explicitly read certain parts and ignore others.
"Essentially, the users needed to be in control. Links were clicked on because they represented something and users looked for familiarity in their surroundings. The majority of users craved consistency in layout as long as content was interesting and they did not appreciate unexpected surprises such as sound that loaded without warning. This made users jump and they often moved away from the page immediately. With sound, when it loaded and played loudly, users did not realise that they could just turn the volume down on the speakers. Instead, they looked for volume control buttons on the screen and moved on without reading content if they could not find any controls." (Rajani, 1998)
It is clear then that interfaces, be they web based or otherwise, should be intuitive and flexible with an emphasis on "making things visible" (Norman, D. 1988). The use of multimedia techniques enhanced the users experience of using the site, but only if they were given an option to use the techniques. No user appreciated being forced to use a particular modality of communication. "A flexible interface is approached with less apprehension and the user will spend more time seeking out information and browsing ‘just for the sake of it’." (Rajani, 1998)
So, users did not really form concrete ideas and concepts about the site in terms of an overall picture. Instead, they formed concepts about navigational elements and used these to fulfill their tasks. Perhaps a further development would be to retest the participants of our study to see if any of those concepts have remained after a period of time. We can assume that a user who uses a particular site often, will build and maintain conceptual/navigational models of that site, but for those sites that are used rarely, or only once, it seems unrealistic to expect users to conceptualise notions that designers may have (unless they are explicitly stated!).
The study produced some interesting findings about how first time users to a web site will 'experience' the site. Of course, site experience is dependent on context, but some of the findings raised here may be used to further question the usability of web sites.
References
- Rajani, R. (1998). An Investigation Into the Usability of Web Sites. Project Report Submitted in Partial Fulfillment of the Requirements for the degree of Bachelor of Science. Department of Information Systems and Computing, Brunel University. London.
- Benway, J. P. (1998). Banner blindness: The irony of attention grabbing on the World Wide Web. Proceedings of the Human Factors and Ergonomics Society 42nd Annual Meeting, 1, 463-467.
Additional Information
This article has provided a very selective description of our research and findings. Further information about the research, tests and findings is available on my web site at http://www.rakhi.dircon.co.uk/webindex.htm
Acknowledgments
This study was undertaken in Partial Fulfillment of the Requirements for the degree of Bachelor of Science in the Department of Information Systems and Computing, Brunel University, London. It would not have been feasible without
the guidance of my supervisors Dr. Duska Rosenberg and Dr. Simon Taylor, both of whom provided incredible support and encouragement.
© Internet Technical Group
Last update: December 5, 1998
URL: http://www.sandia.gov/itg/newsletter/dec98/???.html
hosted by Sandia National Labs
Disclaimer: Neither Sandia Corporation, the United States Government, nor any agency thereof, nor any of their employees makes any warranty, express or implied, or assumes any legal liability or responsibility for the accuracy, completeness, or usefulness of any information, apparatus, product, or process disclosed, or represents that its use would not infringe privately-owned rights. Reference herein to any specific commercial product, process, or service by trade name, trademark, manufacturer, or otherwise does not necessarily constitute or imply its endorsement, recommendation, or favoring by Sandia Corporation, the United States Government, or any agency thereof. The views and opinions expressed herein do not necessarily state or reflect those of Sandia Corporation, the United States Government or any agency thereof.