Category: UX advice / Research articles
Over the decade or so I’ve been a UX designer, I’ve run a lot of user tests. They’re a great way to find out how people really behave when confronted with a website or app.
Whilst it’s important to run them, if you don’t record the results then they may as well not have happened. If only one or two people try to keep the findings in their heads then they’re liable to misremember and by not sharing the results with others you’re missing out on a chance to convince others of the changes that need to happen.
Of course creating a big report can mean you end up with a document that no-one reads, so it’s important to keep it lightweight and follow lean principles so it doesn’t slow down your process. After all, the tests are just a means to an end of improving your site experience.
I always aim to keep my reports under 10 pages or slides, with several clearly defined sections and plenty of bullet points, so it’s all easy to digest. Here’s my guide to making a comprehensive but light user testing report, with a rough indication of how big each section should be.
Keeping the documents online is a great idea, as people can refer to them wherever they are, so I tend to use Google Drive for my testing reports. I initially did them in a Doc (like Word), but this looked quite text-heavy so I have now switched to a Presentation (like PowerPoint). Here the sections are more clearly marked by slides so it’s easier to consume.
Even though we’re aiming to keep our report short, we can still make room for an executive summary. This is where we put the most important two or three findings from the round of testing, so even if a client or stakeholder reads nothing else, they learn this.
What were the pressing issues that arose from the tests? If you wanted nothing else to change after this, what is the thing that must happen to improve life for your users? This should help you focus your findings, and you can wrap up a few little issues into a theme, like ‘navigation’, ‘ordering of information’, or ‘the mobile experience’.
Here you should summarise the demographics of the users you tested with and how many there were. This generally means age, gender, devices, and possibly income and education level. It’s good to show that the people who took the tests match your actual users, so people will take the findings seriously.
If you used any screener questions to filter the people to take the tests, then here’s where you should mention them too.
At this point you should show the tasks that you ran the tests with. If you’re doing this for a client you should have already cleared these with them but for future reference is always useful to be able to quickly see what the test covered. I aim to keep the tasks to no more than five or six so this won’t take up too much space.
1 slide (hopefully…)
There are lots of things you can discover when running user tests, and one of them is bugs or broken parts of the experience. For example if a menu doesn’t work on a certain device, or nothing happens when a user clicks a link, then you know this isn’t working and should be fixed.
Clearly state the problem that the user has found and embed or link to a video clip showing the issue so it can be recreated. It’s also worth noting the device and browser being used too.
Ideally bugs should be found during QA and not during user testing but it’s hard to catch everything: things can break over time and users can do things you just didn’t plan for.
I like to keep this separate from the rest of the results as they are things that can immediately be sent to developers for fixing. If you’re not sure whether something is a bug then record it as a ‘maybe’ or with a question.
Approx 3-4 slides
Usability issues are the core of your testing report and the main purpose of running these tests. Thus this should be the main focus.
I use a bullet point for each thing discovered, with a clear statement explaining the problem and then ideally a link to a video clip showing the problem happening (this is easy to do if you’re using a remote testing tool like Validately). These videos are really powerful for backing up what you say you saw with proof. If you have a repeated issue you only need to show a single example of it to illustrate your point.
Try to avoid suggesting any solutions here. This report is a place to clearly state what you observed in the user tests and so you have a record of how the website performed at that time. Even if they seem obvious, solutions should be separate from this.
To help make this more digestible I try to order the issues by severity, with the issues that I witnessed the most at the top. If more than one user is finding a problem then it probably needs fixing. If only one is then it can be treated as less important.
I also aim to break this content up in other ways to make it easier to scan. I do this either by splitting up the content by the page of the site it was found on (e.g. homepage, about page, product page, etc) or by device. If there are a lot of usability problems then I might do both.
If you discovered any odd bits other than bugs or usability problems during the tests, then it’s worth keeping a record of them too. I like to make sure any positive feedback that users expressed is recorded—it’s easy to get lost in the problems and negatives!
You can also record any little observations that don’t fit in the other sections. Perhaps the way users generally behave can inspire some changes, such as ‘people scroll fast on mobile’, or ‘users rely on the browser back button on desktop’, etc.
Finally, a lot of remote user testing platforms offer the opportunity to survey participants after the test. These questions can be of varying usefulness but any chance to gather a bit of extra detail from real folk should be taken. You could use it as research into other sites they use or find out a bit more about them.
State any questions you asked and record the answers—you can potentially summarise the general themes of the answers rather than put them all in, if there are a lot of empty words.
To make your process even quicker, I’ve put together a Keynote, Powerpoint, and Google Slides template of my user testing report, which you can get hold of here and customise as much as you want.
If you want to know how you can combine user testing with a few other cheap methods of research to successfully redesign your website, take a look at my video course, The Evidence-Based Redesign.
Main photo by Hannah Olinger on Unsplash
Sign up here to get a guide to my favourite (mostly free) tools for evidence-based designing. Plus a massive, advice-filled reading list.You'll also get UX articles & content emailed to you once a week. Your email is never shared. Unsubscribe at any time.