As UX or product designers we all know that user testing is really important for finding issues with your design and for checking if it meets user needs.
The process of running tests can put designers off though, as it seems like a lot of extra work that will eat into deadlines (this is misguided thinking, by not checking with users you’re likely to be adding more work in the long term).
Sometimes called hallway tests or cafe tests, guerrilla user tests are basically when the person being tested doesn’t know about it in advance and I find they are perfect for prototypes. They’re quick to run and are fairly informal so you can contextualise and explain anything that is unfinished or doesn’t work in your mock-up.
You can do guerrilla testing internally at big companies where suitable testers can be easy to come across, or you can out in public and find people. It works best for testing on mobile devices as it doesn’t really matter where you are.
Ideally when running a user test you’d have two people to carry it out: one to facilitate and one to take notes but that isn’t always possible. Here’s my method for when you need some user feedback on your product and you’re a one-man band or the only person prepared to do it.
Get the following together:
- Two smartphones—one to give to the user for the test, one for you to record the test on.
- Your prototype loaded onto the test phone—there are now plenty of apps like Marvel or Invision, which are great for making these. All you need to do is upload exported images of your designs and link them together.
- The scenario written down for you to refer to. This is what you tell the user at the beginning and sets the scene for the test.
- The tasks written down. This is meant to be a short test so no more than three or four tasks should be required. Ideally there should be a single main task that tests the key flow of the app (e.g. “show how you would sign up”) with sub-tasks for any specific functionality you’d like to check, in case it doesn’t come up naturally.
- Identify somewhere to go where your target demographic will be hanging out. For testing a home-selling website I tested internally at a company filtering only people who owned homes, for testing a learning app for students, I went to a library full of people studying.
These tests are meant to be quick, you won’t need much more than about 10 minutes per person, so if you test five people (which is all you really need) you should be able to do it in an hour or so.
For each test:
- Approach a person who looks like they fit your target audience and explain that you’re researching a new product and would love just 10 minutes of their time to get their feedback.
- Start with the standard introduction of a user test: make sure they realise it’s a test of the app not them, tell them to think aloud their thought process, and encourage them to be honest.
- Hand them the mobile with the prototype on it and introduce the scenario for how they would come to be using the app so they have the context.
- Now set them the main task, whilst you use your second phone to film them using the app. Don’t worry about taking any notes. As there’s only one of you just pay attention to what they’re doing and engage them where necessary (asking open questions about what they would expect to happen if they get stuck, reminding them to think aloud etc).
It’s tempting to stop once the tests are over and rush off into making design changes, but analysis is an important step to take so you record what happened accurately.
As you were conducting the test you can think you already have the insight you need but it is common to misremember what actually happened: exaggerating certain things and forgetting others. Watching it back helps you check exactly how serious each issue was and helps you spot things you might have missed.
It also means you’ve got a record of what you did that can be accessed later and shared with others, as proof the test actually happened.
- Get the videos off your recording phone and onto a laptop, or even better a shared drive.
- Watch the first video and note down on a post-it note any time the user struggles, is unsure, finds a bug, or anything else of note. For each user put an identifier on each post-it note or use different coloured post-its, this way you’ll know which test to refer back to for each issue.
- Repeat for the other four tests. Each one should take about 15 minutes.
- Now you should have a load of post-it notes, hopefully with some repeating themes. Do a bit of affinity mapping and cluster together the similar ones to help you see where the biggest issues were. Give yourself half an hour to play with the groupings and possibly discard irrelevant ones.
- Finally, crack open a text doc of some form and write up the issues you’ve seen in a list, ordered by the most recurring. Link off to where the videos are stored and this doc can act as your index and a lightweight report of the user test findings.
Pat yourself on the back, you’ve got out there as a solo designer and put your work in front of real people! It’s not easy to approach people who aren’t expecting it but you should be feeling the reward of now knowing what works and what doesn’t. Your work is more resilient as a result and you can defend it in front of clients and stakeholders.
Now go solve the issues you’ve found!
Last updated on 15 October 2019