Advice: Evidence methods

Guerrilla User Testing

What you can learn

User testing involves watching people use your site or app to see what difficulties they have. Guerrilla user tests can be done in a number of ways (covered below) but what they have in common is that they are relatively unplanned and quick to conduct. They're also usually done in-person rather than remotely. Generally this is good for testing a work-in-progress in the form of flat designs or a prototype. Of course you could test finished websites or apps but there are other methods for this, which can better reflect the user’s actual experience.

The main purpose of these tests is for sense-checking your work as you go along and for testing out ideas with people who are coming to the project afresh. You'll discover usability issues with your work and whether people understand what they're interacting with. It's probably the most useful type of evidence for ‘works-in-progress’ as it doesn't require a completed product to gather evidence on. You can even test with sketches or paper prototypes.

It's missing some of the scientific rigour of other user testing methods but it certainly beats sitting around and theorising or guessing at what you think users are going to do.

How to do it

First off, a bit of prep: think about what you're testing and decide the tasks you want people to carry out on your design (two or three tasks is about right for this). If you're showing a single screen then decide what your question to them will be (and make sure it isn't a leading one). Either way, note down what you want to test so you're consistent with each person. Also make sure you have a way of recording the feedback: write it down immediately after or—even better—make a video recording.

It's then a matter of finding people and getting them to try out your design or prototype. As with most types of user testing, five users is a good number to aim for but if it's quick to get a couple more then do so. Here are some options for finding people:

After testing it's a very good idea to spend a bit of time going over your results. You might think you can remember what the issues are but it's surprisingly easy to forget one or two a few days later. Make sure you get all your findings written down and then sort them into some kind of order of severity. Don't forget to include things people liked as well.

Watch out for

Testing with other people who work on your project or in your team is fairly useless as they are going to have such a clear idea of what you're doing that they won't ask the 'dumb questions', which is the kind of information you want to find out.

Don't develop long tests with multiple tasks to go over. People's attention will wander pretty quickly. You should be aiming to take about 5-10 minutes of their time. If you've got a big prototype then focus on the things you're most unsure about or run multiple guerrilla tests.

It can be hard to do it all on your own (especially in public) so you might need someone else to help. They can note-take or record while you engage with the user and build a bit of rapport, otherwise it can look rude if you're not paying the user attention as they speak.

A test that isn't recorded may as well have not happened. By getting it written down and having video clips you can share your findings with others. Definitely don't write a full report though or you're defeating the objective of speed.

Example tools (and cost)

Something to test on (your laptop, tablet, or phone). Something to record with (another phone, QuickTime screen recording). Obviously these cost money but I'm assuming you have access to them to design so it shouldn’t cost any extra.

How long does it take?

It can be as quick as an hour to run five tests but I recommend spending half a day to properly record and analyse the results.

How often should you use it?





Last updated on 25 March 2017

Learn how to make evidence-based improvements to your website

Sample video images from The Evidence-Based Redesign course

Learn how you can redesign a website based on evidence of how users actually behave, without spending a fortune. A new video course coming in January 2018 but register today and get a massive pre-launch discount.

Here's another method

Net Promoter Score

This is a classic method for understanding user sentiment, here's what you need to know to incoporate it in your design research. Learn more

Last updated March 2017

View all the methods in the guide

Note: the examples in this guide are for website design, but most of the content is also applicable for native apps and software.