Module 4 – Usability Evaluation Techniques
Topic 1: What is usability evaluation?
This module talks about 4 techniques to improve your interface design. The purpose behind usability evaluation is to ensure the design is fit for purpose.
When using usability evaluation?
- assessing usability of an existing website
- refining the design of an existing website
- embarking on a new design
Conversely, usability evaluation is not a good technique if:
- you already know you have flaws in your design
- you do not actually know who your users are
Usability evaluation is an iterative process, so you can continue and do it through the design process.
Important points to consider before starting a usability evaluation:
- be clear about the purpose: are you assessing the usability? are you refining an existing design? designing a new website?
- think about the scope: are you covering the entire website? or certain features?
- be aware of known issues, not to waste your usability evaluation on the things that you already know have problems
- be aware of your users, make sure you recruit accordingly, and also the numbers of users is important to ensure you cover a wide range of usability issues
- think about the context the user is in: in the office, at home?
- think about the budget: recruiting users and running evaluation is costly
- think about the deliverables: who is the target audience? stakeholders, developers, project team?
Topic 2: Usability walkthrough
Usability walkthrough is a very informal way to perform usability evaluation. It often involves using a very low fidelity or rough mock up.
First step: introducing the design to a user, giving an idea of the context and what is the design for. Start by example with your website homepage, and discuss every element, every feature of the design with the user.
Cheap, informal, easy to set-up, you get criticism early and it helps you refine the design early in the design process.
Topic 3: Usability testing
Another usability evaluation technique is usability testing. It is often done in a controlled environment using real users of the website. It is a one on one session, the idea being to reveal usability issues early and help recommend fixes to the design. Ultimately, the goal of usability testing is to help improve the interface.
Here are the different levels of formality for usability testing:
- an informal usability test, such as usability walkthrough, early in the design phase;
- a formal usability test at the end of the design phase: when the design has been solidified, before you hand over to the developers.
Usability testing can be run ‘moderated’ or ‘unmoderated’:
- ‘moderated’ when a facilitator is in the room with the user. The facilitator is able to support the user, run them through the task they need to perform, takes attention to the body language, mood and having discussion with the user if s/he has issues.
- ‘unmoderated’ is when there is no facilitator. It is often done remotely, the process is a lot more automated, so often involves using a system that tracks what the user is doing and capturing their mouse clicks.
Be aware of the difference between UT and UAT acronyms:
UT = Usability or User Testing
UAT = User Acceptance Testing = done by the business, this is a way of ensuring every feature that was specced or documented exists in the design. However, it is not about usability testing of the site itself.
Topic 4: Before you test
To conduct a usability test or walkthrough, planning is critical. Make sure you have the following:
- your designs: either rough sketches or a working prototype
- a room to conduct the test in
- the user (!)
- some scenarios or realistic tasks
- an observer, someone to help you and support in terms of taking notes
- a stakeholder watching, in a different room: it makes a lot easier to actually talk through some of the findings and agree on the rationale in terms of improving the design.
The facilitator has a key role: making the user comfortable enough to give feedback. And especially sincere and constructive feedback with criticism: the user has to say your “baby is ugly”, and why.
Things you need in a scenario:
- user goals
- user motivations
- writing a story
- tasks to perform by the user
- not talking about the system or the features
- keep the language simple
Data is very important: the more you capture data (with additional notes takers by example), the best you get users testing.
Topic 5: Conducting a usability test
There are 5 different phases during a usability test session:
- introduction and formalities: talk the user about what you are trying to cover, make them really comfortable, tell them ‘my baby’s ugly, why?’
- pre-test session: give the user a questionnaire to fill out, often used as a screener to make sure that you understand who they are, that they are comfortable being there, as well ensure that they are fit for the test
- the usability test itself: give the user some scenarios, get them to complete the tasks, capture as much data as possible. Observe, take notes, do not lead, this is all about silence
- post-test phase: discuss with the user about what they did, what they liked, disliked, clarify any questions that you had about what they did, and get them to give you feedback about anything that they would change with the design
- last phase: debrief. You tell the user what happens after the test, where they are within the entire process of the project
Topic 6: Conducting an expert review
Conducting an expert review is a great technique to evaluate the design when you do not have a privilege of recruiting users. The session involve getting some usability experts, who understand the context of use and understand the users, to come in and evaluates your design, based on web best practices.
In the user experience industry, an expert review is sometimes known as heuristic evaluation. A heuristic basically involves best practice guidelines for the web. And the practitioner would use these as a way of informing their recommendations.
To run a heuristic evaluation, or an expert review, you need 3 things:
- the design of the website
- the expert
- a subject matter expert to give you an understanding and a first pass of the design, to explain the features and functionality
4 steps to conduct an expert review:
- familiarization: you become familiar with the design, getting understanding of the users and the tasks they perform
- evaluation: thinking of the heuristics, the best practice guidelines and reviewing the design
- with colleagues and subject matter expert: discussing the findings and agree if there are usability issues
- writing up your recommendations
An expert review is a cost saving exercise: the goal is about thinking about your user goals and business goals and making sure first and foremost you focus on those.
Do not focus on every feature of the website. Think of your users, how they would enter the site, and try to follow that journey of the user rather than systematically going through every single feature that the website has.
- fast to perform: from half a day to 3 days according to the complexity of the product
- users are not involved
- this is not done in context
Topic 7: Competitor analysis
Competitor analysis is not so much a usability evaluation technique, but it still a type of evaluation. So it is a useful technique to use to support your organisation or your client in terms of understanding how they could improve their website.
The whole idea behind doing a competitive analysis is not necessarily just to see what your competitors are doing, but also seeing what similar industries are doing, or what other retail groups are doing, what features and functionalities they have. In other words, picking the best from each website to help build a strategy to improve your website.
Topic 8: Analyzing and reporting
How to analyse results and report on your findings, based on the 4 evaluation techniques we’ve covered before?
Consider these points:
1) The analysis process:
- look at the root cause of the problem: sometimes it might be a bit hidden
- always think about the user
- bring your client along the journey, talk to them about the findings and the recommendations, agree and collaborate; it helps making sure your findings will be accepted
2) The reporting process:
- the format for the report is really important (from A3 to slides with bullet points)
- understand who your audience (for your findings) is: developers, project team, designers, stakeholders?
- keep language simple to be sure your findings are understood
- quantitative versus qualitative: do not provide percentage when you did a usability test with only 6 users…
- always confirm your objectives: make sure you are going to deliver the right document for your stakeholder
Topic 9: Summary
In the first module: we made an overview about user experience by covering themes like user-centered design, usability and the evolution of user experience.
In the second module: we reviewed the elements that make up user experience and interface design. We covered off information design, interaction design and visual design.
The third module was all about users: the importance of knowing our users, understanding their needs, and some techniques to help you gather some insights.
In the fourth module we talked about usability evaluation by covering four useful techniques to redesign a website, or improve an existing website, or design a new website from scratch, using feedback from users or experts in the field.
Feel free also to have a look at these other posts:
- Online course ‘User Experience for the Web’ – module 1 – Overview of User Experience
- Online course ‘User Experience for the Web’ – module 2 – The Elements of User Experience
- Online course ‘User Experience for the Web’ – module 3 – Knowing your users
Photo credits: http://salzertechnologies.com/heuristic-evaluation/