Category: UX advice / Research guides

How to write effective remote user testing tasks (that don’t bias your results)



As someone who has run a lot of remote user tests over the years, I’ve spent a lot of time crafting tasks. This is the only direct interface you have with the user during the test so it’s important to make sure they are clear and bring you meaningful and unbiased results.

I like to have about five tasks in order to guide users through the steps in the flow I’m testing. The aim when doing this is to make sure you let them travel through this flow as naturally as possible.

If you’re not careful it’s easy to tell the user exactly what you want them to do in the tasks, which stops it being a test of how they might actually interact with your site. Instead you’ll end up with a test of how well they can read instructions, and your results won’t be much use.

Here I’ve broken down some of the common tasks I use to get users through a website journey in the fewest words possible. These examples are wordings I’ve arrived at after lots of tests and tweaking and have given me good results. They work for an ecommerce site but most of them could also work for a content or service website.

Landing page

First of all, please take a look at this page. What do you think the website is for? What are your first impressions?

This is a classic task that I start almost all of my user tests with. You only get one shot at raw first impressions from people so it’s good to actively gather them. Asking them to consider the landing page also prevents them shooting off around the site too early.

Getting users to tell you what they think the site is for can be very instructive in terms of understanding if your branding and copy is clear enough. Asking about first impressions is good for getting them to express what they think of the design, photography, and layout.

Searching

Please look for a [product] that would be most suitable for you/your family (…in September, although the exact dates are up to you)

When starting users out on a search task, I don’t want to add in too many constraints, which can make the task feel artificial to the user and not engage them. I’ll always ask the user to bring to the task their own requirements by getting them to think about what they would actually want. If it’s relevant to the product, getting people to think about their family is often good to bring a dose of reality and restrictions, e.g. ‘my partner doesn’t like green’, ‘my kid won’t eat this’, ‘it needs to work for five of us’ etc.

I also try to avoid using a verb which might directly match how things are labelled on the website (such as ‘search’). By getting them to do something generic like ‘look for’ I see if they gravitate towards browsing or a search field.

The second part in brackets is a wording I often use if the task requires the user to search for something in a time period, like a travel booking. I guide them for how far in future to search but again want them to choose suitable dates as they would in real life.

Filtering searches

How would you choose to see only items in red?

This task is to see how easy search filters are to find, and whether the location, labelling, icons, etc are obvious enough. I don’t want to expressly use terms such as ‘filter’, which is tech jargon and not how people naturally think. This would bias them in looking for that particular button. It’s better to phrase it as an end goal that they want to achieve—seeing only items of a certain colour or size.

Getting to products/content

How you would find out more about the [product/content] you like the most?

This task is primarily to get users onto the next page in the flow, and also to see how they go about doing this, e.g. do they look for specific link text to click? There are only a limited number of ways they can do this so it’s often most useful as a transitional task to get users to a page type that you’re interested in them seeing.

I avoid telling them to ‘click’ or ‘tap’ anything as they might expect to find more information on the current page, which is useful to know. I also want to get them to choose a product (or content) item as if they were really browsing so I can see what decision-making criteria they use.

Page content

Looking at this page, does it have all the information that you need? Is anything unclear?

Once I’ve got a user to a destination page, like a product page on an ecommerce site, it’s good to understand if they’re happy with what is presented. Rather than specify anything like photos or reviews, which could guide them to only focus only on this, I keep it generic and ask them if it contains the ‘information’ they require. This way I see what in particular draws their attention.

Another good thing to ask here is whether anything is missing or unclear. This usually encourages them to interrogate the page a bit closer and with more of a critical eye, and not to gloss over the task by assuming everything is there that they expect.

Stay back

Of course you can never truly have unbiased results when running a user test—just by doing it you’re creating something that isn’t as realistic as watching users behave ‘in the wild’. However you can attempt to influence users as little as possible, and remove yourself from the process. Try and keep your tasks short, with open-ended wording, and let the users bring some of their real life to the process.

There are several other things to consider when aiming to run effective remote user tests, but getting the task wording right is probably the most important.

Photo by Jerry Kiesewetter on Unsplash.

Last updated on 9 April 2018

user testing / user research / users / evidence / qualitative data / ecommerce /

14 tools to gather evidence – guide

Articles on similar topics

A user testing report template

The 6 step process for running free remote desktop user tests

How to write a user testing report (that people will actually read)