Often you need quick and regular feedback on what you’re designing to help you adjust designs and prototypes early. Here’s a breakdown of how I have carried out guerrilla user tests, including prep time and analysis of the results in under four hours. This applies for doing it internally at big companies where suitable testers are easy to come across, as well as going out in public and finding people. Sometimes called hallway tests or cafe tests, guerrilla user tests are basically any time you don't have a formal test lined up.
Ideally you’d have two people to carry out a test, one to facilitate and one to take notes but that isn’t always possible. So this method works for when you need some user feedback on your product and you’re a one-man band or the only person prepared to do it. It works best for testing on mobiles as you can do it anywhere.
You will need:
- Two smartphones—one to test with, one to record the test. Handily most of us now carry around decent video cameras in our pockets.
- Your prototype loaded onto the test phone—there are now plenty of apps like Marvel, Flinto, POP or Invision, which are great for making these.
- The scenario written down for you to refer to. This is where you set the scene for the user.
- The tasks written down. This is meant to be a short test so no more than three or four tasks should be required. The main task should test the key flow of the app (e.g. “show how you would send a message”) with other tasks for any specific functionality you’d like to check.
- Five users lined up—research shows you only need five to discover most of the issues with your app. If you’re doing it internally send a meeting invite to the calendars of those you have targeted as a suitable audience for the product. If you’re testing on the fly have somewhere identified that your demographic will be hanging out.
- These tests are meant to be quick, you won’t need much more than about 10 minutes per person.
- For each one, go through the standard introduction of a user test: making sure they realise it’s a test of the app not them, that they should think aloud their thought process, and that there are no right or wrong answers.
- Then introduce the scenario for how they would come to be using the app so they have the context.
- Now run your test, going through the tasks, whilst you use your second phone to film them using the app. Don’t worry about taking any notes at this stage, just pay attention to what they’re doing and engage them where necessary (asking open questions if they get stuck, reminding them to think aloud etc).
2 hours 15 mins
Here’s how to tackle the analysis stage that some designers are tempted to ignore. This is an important step to take so you record your data accurately. As you were conducting the test you can think you already have the insight you need but it is common to misremember what actually happened: watching it back helps you check exactly how serious each issue was and helps you spot things you might have missed. Also it means you’ve got the data in a useful format that can be accessed later and shared with others, as proof the test actually happened.
- Get the videos off your recording phone and onto a laptop, or even better a shared drive.
- Watch the first video and note down on a post-it note any time the user struggles, is unsure, finds a bug, or anything else of note. For each user put an identifier on each post-it note or use different coloured post-its, this way you’ll know which test to refer back to for each issue.
- Repeat for the other four tests. Each one should take about 15 minutes.
- Now you should have a load of post-it notes with some hopefully repeating themes. Do a bit of affinity mapping and cluster together the common themes and issues. This should take about 45 minutes to give yourself time to play with the groupings and possibly discard irrelevant ones.
- Finally, crack open a Google Doc or note of some form and write up the issues you’ve seen in a list ordered by the most recurring. Or prioritised however you wish. Link off to where the videos are stored and this doc can act as your index and report that the user testing took place.