Knowledge Bridge

//Tomas Ulej /March 11 / 2014

Developing a website? A usability test is the best thing you can do

Lots of tools are available when developing a website – from mining data from user management systems, through analyses and heatmaps, to drowning in numbers from Google Analytics. And there are of course many others.

As is usually the case though, the most effective and helpful tool – or rather a technique from those mentioned above – is the one that also costs the least. Have you done usability testing yet? No? In that case, if you’re going to do one single thing, throw away all the other tools and techniques and start doing usability tests – they will take your website the farthest.

In this article, we’ll take a look at what you need to do and what you should know before you start testing.

What we’ll need

A usability test is nothing more than tracking a user during the time he or she is using your site. Really, that’s it. No measuring of eye movement, level of stress or anything else that needs hi-tech tracking – just a simple recorded (shot of the computer screen they’re sitting behind and their voice) dialogue between you and them. And of course the tasks you request them to do on your website.

So, no measurable indicators – CTR, time spent on site, etc.? Precisely! Questionnaires or Google Analytics will provide you with lots of data, but it doesn’t get much better than seeing the person in action with the possibility of asking what they think. I guarantee that after a usability test, you’ll know right away what needs to be done even without knowing exact percentages or having data to feed into a graph.

Like with any cooking recipe, we too will start with the ingredients we’ll need. There isn’t much of it, yet it’s still a good thing to specify them.

• A quiet room – it may sound strange but if there is something that can ruin the result of a usability test, it’s having someone disturb you. Are you in an open space office? If so, forget right away that you would do a usability test at your work station. An ideal place for a usability test is a small room, two chairs and a desk – no disturbing elements. Let people know not to disturb you during the time of the test. Turn off your phone.

• A standard computer and browser – although the object of testing during the usability test will be your website, people you’ll be working with will have a feeling you are testing them. Despite being told otherwise, they will not get rid of this feeling. Don’t add to their stress by making them learn to use your gaming mouse or keyboard with different characters. Use the most standard computer most people are used to. In the author’s country, this would be a laptop with Windows, a plain mouse and Chrome browser. Have more browsers prepared and let the participant choose which to use. The more they feel at ease and as if working with their own computer, the better for them and you too.

• Microphone and a program for recording your computer screen – During the test, you will record what the participant is saying as well as the browser window they’ll be surfing in. This is important because you won’t have time to take notes and will not remember all the issues that will come up. Laptops, as well as desktops, often have built-in microphones; however, it has proven useful to the author of this text to have an external microphone. A simple cheap mic for Skyping will do very well.

Selection of a recording program may be something of an obstacle as there aren’t many good quality solutions. For us, a good choice proved to be Camtasia Studio. It can record the screen as well as sound and later allows the recording to be cut, highlight parts of the video such as a mouse click, rolling over an element, etc. The program is not exactly cheap but has a 30-day trial version, so you can start with tests for free and later decide whether investing in the software is worth it.

What will we test?

Question “What will we test?” may sound ridiculous to you. You probably already know about the weak spots on your website – perhaps you were notified by email from users, or you’ve seen something in the statistics. Maybe you’ve asked yourself already why no one clicks on the Editorial Team link at the bottom of the page, while at the same time you get asked by users whether you’ve considered having one.

If you haven’t done usability tests before, I would guess the first ones you do will be of a very general character. You will want to look at your website as it is and its key functions, and find out whether users know how to use them or if they have a problem using them, and last but not least, how you could improve and simplify the way your website is used.

Later, you will probably concentrate more on in-depth issues and usability tests will focus on only one of the sections or just an individual function of your website. Either way, your task now is to transform all intentions into assignments for the participants in your test. How do you go about it?

Close your eyes and imagine all the key functions that make up your website. You can forget about the details for now – the most important elements will do for now. Imagine an average person and what he or she actually does on your website. Usually arrives, looks at the homepage, clicks on an article. Then what? Perhaps he wants to enter the discussion? Or maybe he wants to sign into your system email and read messages? Obviously, it’s different for every website but you should always keep in mind the most frequent and concrete scenarios of your website’s visitors.

Once you have them, you can start turning these scenarios into a story with assignments, ideally with some update theme. Set up concrete and fun assignments.

For example: “You’ve seen on TV that the government wants to increase taxes for people with higher income. You missed some parts of this information, therefore you decided to visit our website and confirm the missing details. Where would you look for this information on our website? Find out the percentage of this planned tax increase and the date the tax law should become effective”.

Notice that I haven’t used in this assignment the name of the section (go to section Economy, in the left-side menu click item xxx). This is exactly the method you should use too, as normally a user doesn’t know where things are. Since nobody tells users on the homepage where to find the information they are looking for, they have to find it on their own. Effectively, the user will now test your site for you and whether you have developed it well. Following the user’s path to finding the right link will show you how to make your website better.

Prepare about ten assignments in a similar manner. For each of them, try to create a specific situation and make it interesting or fun so as not to bore the participant. Be sure to make it clear exactly when the given assignment can be considered finished.

Are you testing one specific part of your website? It’s good to have the first two assignments of a different kind – “to trick” a little, but also for loosening up the user. When testing a specific thing, I personally try to hide it by breaking it into a number of different assignments so I don’t give away my intention. Ideally, at the end of the test, the participant shouldn’t really know what the main purpose of the whole test was.

Ready for some inspiration? A complete usability test can be found here: http://www.youtube.com/watch?v=QckIzHC99Xc

Selecting participants

How many people need to take the test in order to reach the optimal results? It may come as a shock to you, but 6-7 people will be enough.

Are you wondering why so few are needed when for any A/B testing questionnaire you need to have at least a few hundred participants to consider the data reliable? The answer is that when it comes to usability issues, we are all very much alike. You will find this out as soon as you start testing: the second participant will have very similar problems with your website as the first one, and this will repeat until you get to the last one. Using, say, 20 people instead of ten of course means finding slightly more issues on your website, but I guarantee that testing 6-7 people is absolutely sufficient for finding the most significant ones.

Which people to choose? You may be tempted to strictly adhere to demographic distribution, so here is my tip – don’t take it too seriously. (In the end, it will be hard to stick to a demographic distribution of users in a sample of 6-7 people.) Include men and women, younger and older, but the best thing you can do is to call on users that could potentially have a problem with your site.  If you call on a programmer to take part, he’ll probably be able to manage almost anything, but how about a user who isn’t so good with computers in the first place? You are better off choosing participants that will show you as many deficiencies as possible.

An important rule: never invite someone who is a “heavy user” of your site – a member of your editorial team for example, if you have a news website. Of course, there are exceptions when this rule does not apply (e.g. intranet testing), but for the majority of websites, inviting outside people – strangers not familiar with your website – is crucial. Ideally, of course, you would call on people that do not know your site at all. This, however, cannot be done in all cases – if your website targets brokers, an ordinary person won’t be able to help you much.

Another thing to consider is the question of payment for taking the test. You’ll find out in US literature that you should. My personal experience is to invite people without the promise of remuneration but rewarding them at the end anyhow (usually not with money though, but rather in the form of a small gift such as a book).

You may now have a good idea of who to invite. If not, here are a few tips from practice:

• Create a site with a simple questionnaire for people who would like to help with website development and link to it from your site. In a few hours, you are sure to have perhaps hundreds of contacts for willing people.

• If there is a place where users come, e.g. the reception area of a newspaper office where they renew subscriptions, position yourself there and ask directly whether they have time to take the usability test. Another way to do this is for the person on the desk in reception to ask and take the interested users directly to your office.

• Place an ad in the classifieds or post it on your Facebook page.

If you don’t want to do any of the above? We all have friends and acquaintances who don’t even know where we work or have never visited our websites – use them. Just one rule to keep in mind: never, never invite the same person for tests twice, especially if it’s the same test!

Carrying out a usability test

You have a quiet room, you have a computer ready (you’ve tested the recording and mic functionality), you have people coming and you have the assignments ready for them. What now? It’s time to carry out the usability test.

At the beginning of the test, you have to tell people what awaits them. In order not to influence the results, it’s ideal to tell everyone the same thing – that is, read the same text to all of them. You don’t have to come up with the wording on your own; usability expert Steve Krug has done it for you already, so don’t worry. You can download the English version of it here.

The text describes what will happen during the usability test and also includes a printable form of agreement for making sound and video recording. So now you can smoothly begin your assignments.

Now comes the most important part. You could have done everything perfectly up to now, but this is the point when everything can go wrong – your communication during the test. The following are the three most fatal mistakes. Avoiding them and sticking to the following method can make usability testing a brilliant tool.

• Do not help. Do you remember how you deliberately didn’t include specific instructions about what to do? This is exactly what you shouldn’t do either while the test is in process. Let’s say a participant has a problem with completing your assignment (can’t find a link to a page or has forgotten her password). Do not help her under any circumstances. You can try a formulation such as, “Try thinking about it some more”, “What would you do if I were not here?”, or “Unfortunately I can’t help you but as soon as the test is over, I’ll answer any of your questions”. If the participant cannot complete the assignment for a long time, end it. It is difficult not to help a person who is obviously in trouble. I personally managed to “fail” a few times when I administered my first usability test. While it is difficult, it is very important because if you help the participant, you may as well throw the results away because you can’t be sure that you didn’t bring the user somewhere she wouldn’t otherwise have got to on her own. There of course are situations when you can help – for example if a strange window, not relevant to your website, pops up, or the mouse is disconnected, etc. – but don’t ever say, “And why don’t you try clicking here?”

• Ask what they’re thinking about – keep on prompting the participant to say what he’s thinking about, to verbalize what he’s trying to do as much as possible. The instruction to do so was already included in the introductory text you read out at the beginning, however, some participants will tend to forget – you should do all you can to make sure there isn’t silence in the room and the participant speaks as much as possible. That is the key to improving your website – every one of the participant’s thoughts will move you forward.

• Praise and eat humble pie – many people you invite will say, “I’m not smart enough for this” and when they can’t do something, they will repeat the phrase. Participants will constantly feel as if you are testing their abilities. Use every opportunity to disagree with them: “I’m not sure where to click?” “Thank you – you have just found a huge shortcoming in our website. Actually, you’re doing it right, and it’s us who developed the website who got it wrong because you can’t find what you’re looking for.” After the test is over, tell them again how much they have helped you. You can even show them the right solution to their task.

After the test

The first thing that shows you’ve done the usability test right is that the results are absolutely horrible. Users had problems finishing your tasks, you’ve noticed mistakes you made in planning the site, users do things in a very different way than you thought when building the site. These are the results you’re looking for. If you, the website developer, feel miserable, be assured that this is exactly the kind of feeling you needed to reach.

In order to transform the usability test results into a better website, do the following:

• Share the results with everyone – Do not keep, under any circumstance, the test results just to yourself. Let as many colleagues as possible see the video recorded during the test. Definitely show it to designers, project managers and programmers. Don’t be shy to show it to other people who worked on the development of the site. Practically anyone from the team can learn something from a usability test. Before you show them the video, give them a pen and paper and ask them to take notes with ideas on what to change on the web in order to prevent some of the problems the next time around. I’ll bet everyone will have a paper full of various thoughts. Are you all too busy to watch a 5- or 6-hour long recording? Cut the usability test video – leave out the parts where participants succeed in their tasks and pick out the “cherries on the top”.

• Have a discussion and make a list of changes. After watching the video, compare your lists and write the problems on a big board. A huge discussion is surely to arise. A long list will be created. Set a priority for each item – mark the issues according to how big they are.

You will notice as you go along how the view of the web changes in the team. At the beginning of the exercise, someone may say something like, “Are these users completely stupid?” And then, when they see that every test participant had a problem with the same thing, they will go quiet. You will notice the effect of a usability test on the work of the team in the future. A programmer will try to simplify everything as much as possible right in the beginning. A project manager may want to run a usability test during the design stage. The owner will approve a budget for usability tests.

The possibility of seeing users utilizing your site in real time is something that will completely change the way you work.

How to solve issues discovered

One of the things that happens to you after a usability test is that you suddenly get a feeling that you completely need to change your website. You’ll probably feel that it’s badly designed; programmers may arrive with an idea to re-program the whole site. One of the worst things you could do, though, would be succumbing to these temptations.

Any usability issue found can be resolved in two ways: the complex way (total redesign) or the dilettante/superficial way (do the absolute minimum in order to remove the problem). Based on a number of years of experience, I recommend choosing the second option.

If you don’t believe me, take a look at the following list:

Dilettante solution vs. complex solution

• Much less people will be affected by the issue / No one or almost no one will be affected by the error
• Easily implemented / Requires much effort to implement
• Finished in a few minutes, a day the most / Can take a month or two, and may actually never be finished

Complex solutions seem to be clean and thorough. In reality though, it means you are starting from zero. In the end, the fact that you have done it over again doesn’t mean that you have not made the mistakes again or haven’t created new ones. On the contrary, if you redo something completely, you will end up facing a dilemma at the end what to do with the new issues.

Usability expert Steve Krug introduced a rule he calls “Do as little as possible”. Always look for the fastest solution to fix an issue. Only when that doesn’t work, use something more complex.

Here is an overview of Krug’s points:

Do not redesign, tweak (Krug’s No. 1 rule of successful mistake correction)

• Everybody hates redesigns, so if you suggest a wholly new solution, they will complain anyway.
• You may solve your problem with a redesign, however, it’s almost certain you will make at least five other, perhaps more serious, mistakes.
• Do the smallest changes possible – tweak until the problem is solved. If that doesn’t help, revert to a more serious solution.
• It often means that you want to increase the font, change the background color of a message or something along those lines.

Do not add, removing something is much better (Krug’s No. 2 rule of successful mistake correction)

• People usually feel that adding something will work better, let’s say a new notice or phrase. Often, it’s much better to remove something. Most mistakes are actually caused by organization on the site being so complicated that it becomes chaotic. This is a very frequent reason for mistakes.
• Always consider the list of issues in relation to effectiveness. If there are small issues which affect only a few users and would take too long to fix, don’t bother with them. There are probably ten larger issues that can be addressed quickly. Focus on these first and then on the others.

How do you find out whether you have improved the product? Simple, run the same usability test again … with different participants, of course.

Now you know everything important about usability tests. If you’d like to know more, I recommend two of Steve Krug’s books: Don’t make me think and Rocket surgery made easy. They are short, funny and will tell you everything you need to know.

Article by Tomas Ulej