This paper, co-authored with Sarah Allen Miller, was originally presented at the Society for Technical Communication Conference, Chicago, Illinois, 2001
Why set usability requirements?
In traditional systems development, users were locked into systems and had little choice about whether to use them. If they were involved at all, it was often only at the very end of the process at a point where their input could not be acted on.
With the advent of the web, many organisations are trying to adopt a more user-centred view of systems development where the aim is to create sites that meet users’ needs because users can go where they wish.
But users are often external to the organisation and we don’t generally know too much about them. Can we do without a definition of users in this case? If not, how can we define them if they could be anyone?
Websites with forms are even more difficult. Organisations often do not know exactly why they are asking their users for information or how the information will be used. Often, they fail to think at all about the effort required from the user to locate the answers to their questions.
In this paper, we describe our experience in setting usability requirements. It is not always easy but has benefits:
- For the project team, the exercise raises awareness of usability factors and allows all the website stakeholders to air their views.
- For the development of the website, it is simply easier to design something if you know what it has got to do and who it is for.
- When you evaluate the website, you need to set usability requirements, so that you know whether or not the site is a success.
Finally, we suggest that the evaluation may help to encourage your organisation to investigate and refine your usability requirements.
What are usability requirements?
Usability requirements are a statement describing who will use the website and for what purpose? When the website contains a form, there are two purposes: the organisation’s in asking for the information and the user’s in delivering it.
Background to the definition
We have chosen to define usability requirements by considering four aspects – user, environment, domain of knowledge and task. The definition is drawn from the Open University definition given in their new course M873 – User Interface Design and Evaluation (1). We find that it works well for us in practice.
1. User
Who are the users and what are their characteristics? Deborah Mayhew (2) defines user characteristics as:
- “Psychological characteristics”, e.g. motivation – why should a user research and reveal information in filling out the form?
- “Knowledge and experience”, e.g. what do they know about the organisation? Do they trust the organisation sufficiently to give true answers to the questions?
- “Physical characteristics”, e.g. age. Will they need a larger font size due to declining eye sight?
Mayhew also mentions job and task characteristics but when defining users, we find it more helpful to refer to the characteristics of the user with respect to the task, rather than the task itself, e.g. is the user familiar with the task, how often do they do the task?
There may be more than one type of user, differing significantly from one another. If so, we find that it helps if we define each type of user separately, contrasting the differences between their respective environments, domains of knowledge and tasks.
2. Environment
What are the circumstances of use? Environment can be defined in terms of physical location, the equipment used and supporting information.
Physical location: Is the user at home, at the office or perhaps in a public place, e.g. the library or an internet café? If users have to leave the website to find the answer to the form in a different location, will they lose their work so far?
Equipment used: Will the user have all the necessary equipment to achieve their goals, e.g. the need for a printer attached to the PC to print the completed form. Does the equipment used impose constraints on the user, e.g. modem speed, screen size and resolution, browser capabilities?
Supporting information: Where will the user find the answers to the questions in the form? Is all the necessary information to hand? Is help available if the user gets stuck?
3. Domain of knowledge
What do the users know? More importantly, what don’t they know? The domain of knowledge is the skills and experience that the users bring to the task in hand. In our experience, this is a factor commonly misunderstood. On the one hand, it is assumed that the user will know and understand the organisation’s jargon. On the other hand, there is an unspoken assumption that the users will approach the website with a blank mind – no preconceptions about the website, and no other knowledge or experience. Both assumptions are usually incorrect.
- Experience of the organisation: What is the user’s background knowledge of the organisation? What experience does the user have of the organisation’s product? In filling out a form to order a brochure, for example, the user is likely to know what a brochure is and why they might need it. However, they probably won’t know the brochure order code or necessarily understand the title of the brochure given by the organisation.
- Skills: What practical experience of the web does the user already have? Has the user experienced other websites performing similar functions, e.g. a competitor’s website? It is also important to consider other skills that might be relevant to the user’s task. A specialist site for doctors asking about medical practice can use a different style of language to a site that is aimed at untrained users who are worrying about whether they may have a particular disease.
4. Task
- Goals and tasks: In approaching the website, what overall goals will the user want to achieve? Which tasks will then be performed by the user in order to move towards those goals? For example, a user’s overall goal may be to get some information about the organisation’s product. A related task will be providing personal details to the organisation so that a brochure can be sent out.
- Success in the task: What is the result of the task? For example, the result to the user of ordering a brochure is a confirmation the order and understanding when the brochure will be delivered. The result to the organisation is the existence of an order on the computer system. The user’s overall goal is only achieved when the brochure appears in the mail. This may be challenging for an organisation where different departments are responsible for separate tasks that contribute to the goal.
- Task duration: How long should the task take? On the web, stakeholders often resist setting time scales because response times can vary enormously. However, even broad time scales, e.g. about half an hour, are useful in order to know what we are aiming for. Also, factors such as expense (in Europe the cost of using the phone line) and frustration levels may mean that the user sets their own arbitrary time limit.
- Frequency and choice: How frequently will the task be performed? You might buy a car once every two or three years, whereas you might buy books every week. Does the user have the choice of an alternative? Instead of using the web form to order a car, the user could use the telephone or go to a showroom.
How to set usability requirements
It is easier to design something if you know what it has to do and who it is for. We have suggested some questions that define your users, and their environment, domain of knowledge and tasks. The next step is to set requirements by answering those questions.
Who should set them?
Usually, the project stakeholders have to set or approve the usability requirements. Stakeholders are those with an interest in the product: business sponsor, content owner, marketing department, customer service representative, development or analysis department, creative or visual designer. Often users or user representatives are involved as well.
When identifying the stakeholders, beware of user representatives whose views may be limited to one segment of the user population. For example, a customer service representative’s experience will be skewed by the fact that they spend most of their time helping users with problems. These users may actually only constitute 5% of the total population and while it is important that they are represented, the other 95% who have no reason to ring must not be forgotten.
When to involve users
Where real users are available, make use of them as early as possible. However, bear in mind that the real users may not feel empowered to comment if brought into an unfamiliar forum such as a project meeting. The alternative is to go out and investigate users in their own environment, employing a technique such as user and task analysis. This is usually more time-consuming but gives a more realistic view. Our favourite book on this topic is User and Task Analysis for Interface Design by Hackos and Redish (3).
How to get started
When looking at a website, the stakeholders may be reluctant to set usability requirements if they perceive that the users could be anybody. We find that it helps to ask the stakeholders who the users are likely to be.
For example, a website set up for Small Businesses could in theory be used by novice users. In practice however, the users are likely to have acquired some web knowledge before choosing to visit this kind of site. When the website contains a form, the exchange of information is two-way. The usability requirements must therefore consider both flows of information. Having obtained information from the website, what will the user use it for? And where will they get the answers that the form is expecting?
Resolving issues
Setting usability requirements allows the website stakeholders to air their views. The resulting set of requirements should be a consensus of opinion. However,we often find that getting to consensus isn’t easy and that it helps the process if we create a list of open issues. Each issue should state the percentage of users that it applies to.
If the percentage is small, it may be easier to solve the problem by directing those users to other methods of communication such as the telephone. This is especially relevant in web development where there can be pressure to deliver something quickly.
When should you set usability requirements?
We recommend that usability requirements are set as early on in the project as possible. Writing them early on will help the specification and design of the website because instead of trying to meet the needs of any user, you are designing with a specific goal in mind.
Effect on the usability evaluation
Defining users to recruit them
Although we recommend setting usability requirements as early on as possible, we often find that the first opportunity we have is when we are asked to undertake an evaluation. In order to recruit users to participate in the evaluation, we need to know who they are and what tasks we are going to set for them. Because evaluation forces the stakeholder to consider users and their tasks, we find we are able to get them to think about environment and domain of knowledge at the same time.
Deciding on the environment
To what extent do we need to recreate the user’s own environment for the evaluation? For example, should the user test the website at home (where they would normally use the internet) or is it acceptable to bring them into the office (of the owning organisation)? The office is likely to have a high speed network for accessing the internet meaning that problems related to using the website with a modem connection won’t be uncovered.
In practice, we have found that testing in the office is an acceptable compromise. Recently, the websites we have
seen either have many problems or are at an early stage of development. In both cases, response time is not yet an important factor. Testing in the office is also less expensive than visiting users in their homes.
However, if users are not in their usual home environment how will they look up the answers when filling in the form? We have found various ways of dealing with this:
- by providing the user with a scenario with the information
- by warning the users that they will need to bring certain information with them to the evaluation
- by relying on the users having the necessary information in their heads, which is acceptable for a simple form where the users are likely to know the answers.
Tasks for the evaluation
In planning the evaluation, check that the version of the website to be tested supports the tasks defined in the
usability requirements. For example, if a prototype or very early design is to be tested, it won’t have all the intended functionality. However, we have found that the value of getting early feedback is worth the problem of steering the user around the missing tasks. 
Users views of requirements
Having set the usability requirements, we have often found that the users participating in the evaluation have different views. For example, we were asked to test an administrative policy website where the usability requirements centred on navigation. The evaluation did identify navigation issues but the users were actually more interested in the usefulness of the content and prioritised this over the other problems. This resulted in revisiting the usability requirements after the evaluation as part of the design improvement process.
Investigating the users
Having defined the requirements, recruited a set of users and evaluated the website, we will know how well the website worked for those users. The users we recruit can also help us to know whether the requirements worked for them.
But how do we know that the defined users are the same as the actual users? Figure 2 shows a mismatch between defined and actual users. This can happen when the organisation defines the usability requirements without investigating the users and their needs.
In a recent example, the organisation decided that the website had two sets of users: registered users with passwords, and new users. When we investigated, we found a third group of users: infrequent visitors who need help to remember their passwords.
In the same way that undertaking an evaluation creates motivation for writing usability requirements, dealing with the results of the evaluation can help promote enthusiasm for investigative techniques such as user and task analysis, sampling, interviewing and observation.
Conclusion
We find that setting usability requirements helps us to develop websites that meet user needs. Although we prefer to define requirements early in development, being asked to evaluate a website often gives us an opportunity to persuade the organisation to set requirements. Similarly, the results from an evaluation may help to persuade the organisation to investigate its users and their needs.
References
(1) The Open University (in press) M873 User Interface Design and Evaluation The Open University, Milton Keynes
(2) Mayhew, D.J. (1999) The Usability Engineering Lifecycle. Morgan Kaufmann Publishers, Inc.
(3) Hackos, J.T.and Redish, J.C. (1998) User and Task Analysis for Interface Design John Wiley & Sons, Inc
#usability