Category: UX advice / Evidence-based design methods

Surveys

What you can learn

Surveys promise you the opportunity to gather the thoughts of lots of users without too much effort: you can potentially reach thousands of people with just a single form. You can ask them almost anything (though some types of question are better than others, as explained below).

They also offer the opportunity to quantitatively assess behaviours, by asking how many people do certain things, which often appeals to the analytical folk (managers). This is something you should be very careful with, as a good survey is better as a qualitative tool and a starting point for further investigation than a quant one that gives you absolute truth. Unless you deeply understand the subject it's best for asking open-ended questions and finding out issues and thoughts that users are having.

There are lots of types of survey out there, including short website feedback pop-ups. For the purposes of this I will cover the one-off type that you might run to research a subject or potential design project.

One of the things surveys are often used for is Net Promoter Score, which I cover separately.

How to do it

There are plenty of tools that make creating online surveys easy and give you all the different field types you might need (I cover some below). Distribution shouldn't be a problem either: you can send it out to an email list, or share on social media, or put a link on a site/forum. Of course the users you choose to distribute to will affect your results but the actual practicalities for running a survey are fairly straight forward.

Preparation

Most of the work is actually in the planning and preparation. You need to know what you want to ask and to determine if a survey is the right approach to take. If you're after detailed behavioural understandings then perhaps an interview or user testing is a better bet. A survey can be a good starting point for research, which gives you ideas of the issues to investigate further with interviews.

When writing the questions, it will depend on the type of question as to whether you either keep them specific (for multiple choice answers) or open-ended (for answers where you want the user's free text). Open-ended questions are more useful for getting to the heart of real issues as the user isn't limited by what they can say.

If all of your questions are multiple choice, then it's a lot harder to find out what you don't know. However if you want to survey a large number of people then using multiple choice questions will make life easier when it comes to analysing the results.

Unless you're paying your partipants well, keep the survey on the shorter side (fewer than 10 questions) to maximise your response rate. Just because you have the opportunity to ask people anything doesn't mean that you should—try and keep questions on your particular subject of interest. This will help users stay focussed and go into more depth.

Finally, giving users the opportunity to answer anonymously can help them feel confident about opening up so they might tell you things they wouldn’t if the answers were attributable to them.

Analysis

When it comes to how to analyse your results, it's going to depend on the size of your survey. If it's a small one (fewer than 50 respondents), you can read all of the answers and potentially act on them too.

If you have open-ended questions, it’s helpful to group responses by sentiment: go through each answer and try to categorise them. For example if you're asking people about their problems with a site then you should be able to group them into things like 'navigation', 'search', 'payment' etc. This should help you order the key things that need to be solved with any new design, and you can delve back into the written answers to get quotes and more detailed requirements.

If it’s a big survey (100+ respondents) then you’ll have to focus on doing quantitative analysis of the results. As well as seeing which answers performed better than others, you can dig deeper and segment your results to see which types of user were more likely to answer which way, and find patterns in the data.

Watch out for

Don’t quantify the qualitative

As tempting as it may be, try not to turn naturally qualitative questions into quantitative ones. For example, avoid questions that ask how much people 'like' something (often on a scale of 'strongly like' to 'strongly dislike'), as it's all so subjective it can be pretty meaningless. You could get back a survey with people saying they love your site but they still may not be buying your product and you wouldn't know why. Erika Hall writes well about this here.

You can of course quantify quantitative data with a survey, so asking how old people are, how much they earn, or whether they like x over y is good material for charts and graphs.

Don’t ask for predictions

Just like when interviewing, try not to ask users to predict future behaviour. Don't ask 'how many times will you go to the gym in the next month?' because you'll just get back their ideal answer or one to impress you, whereas the reality is likely to be different. For more solid results ask about actual past behaviour instead ('How many times did you go to the gym last month?').

The who matters

When declaring any survey results, make sure you explain who you surveyed, especially if they're not representative of your actual user base. Be careful of taking your results out of context and declaring that 'all users think this'.

Incentivise

These days people are pretty over-surveyed and have inboxes packed with requests for feedback. To stand out you should offer some kind of benefit to people for completing the survey, otherwise they’re just not going to do it.

Do be careful about the level of reward however. If you offer too good a prize then you're likely to get people rushing through to complete it and not caring about what they write.

Example tools (and cost)

There are lots of tools for making forms and surveys out there. I've used several including Google Forms (free), which is fairly basic but collects your results in a spreadsheet for analysis. Others include Typeform (free and from £28/mo), which is arguably the best-looking form website out there, and Wufoo (free and from £12/mo), which I've used to create fairly complex forms with their conditional rules. Finally SurveyMonkey (free and from £34/mo) has some great analysis tools.

When it comes to surveying tool is generally less important than the content—as long as it's usuable, people don’t really care how the survey looks.

How long does it take?

Writing a good survey isn't the quickest of tasks—it will depend on length but expect to spend half a day at least. Getting results can take up to a week.

How often should you use it?

Often

Sometimes

Rarely

Resources

Last updated on 4 December 2018
14 tools to gather evidence – guide

Here's another method

Friends & Family Opinions

Not something you may think of as a piece of evidence but certainly something that will influence you. Learn how to incorporate this feedback. Learn more

Last updated April 2018

View all the methods in the guide