Category: UX advice / Research articles

Why you should be evidence-based in your design (and less data-driven)



A lot of business owners like to think they’re evidence-based but they’re usually paying this lip service, rather than following a process.

Most of their web design projects start off clearly enough: a problem is identified (usually low conversion), and a goal to correct this is defined. The danger comes when solutions arrive too soon after this, with vague justifications.

Without intending to, assumptions appear in decision-making (always with the best of intentions). “I’ve heard users don’t scroll, we should make this page short”, “I’m pretty sure red converts better than green so we should make our buttons red”, “We should make the search field much bigger so more users find it”, etc.

If you were a detective trying to solve a crime, you wouldn’t just guess at how it happened, based on things you’d heard before. Well you could but you can be sure your case wouldn’t stand up in a court of law.

What should come before solutions is an understanding of what is happening and why on the current version of your website. A full picture of user behaviour.

The opposite approach to assumptions would be to find metrics for everything on your site, that can tell you whether something is working or not. Otherwise known as being data-driven.

Going the other way: the dangers of being data-driven

I used the term data-driven to describe my design process in the past. This is something that I’ve moved away from, for the following reasons:

The reality for the startups I work with is that gathering bullet-proof quant data is really hard and time-consuming and in the end may not actually give us the answers we need. The design process needs to be based on facts but needs to be much quicker and more pragmatic than large-scale data-gathering.

What we're really looking for is proof that we have a problem and proof that we've got a good solution. Enough to get something out of the door that improves what was there before and keeps up growth.

What we want is evidence-based UX design.

Draw a picture with evidence

There are lots of ways you can gather evidence in a design process. Just as in a court case there is strong evidence and weak evidence, but the more sources you can get the stronger your case tends to be.

Quant data may often be one of the most powerful forms of evidence you have but if you're a small company without much traffic then that data will be poor. With an evidence-based approach you can make up for this by gathering qualitative data and even some things you wouldn't think of as data (interviewing clients, customer feedback).

My design process can touch on 9 or 10 evidence-gathering methods. I'll start by talking to the client to get their knowledge; I'll use my experience to assess their site in an expert audit; I'll look at audience data and conversion data; I'll study heatmaps and visitor recordings to understand pain points; I'll probably run a user test to see why there are problems; I'll assess competitor sites to get inspiration; as I design I'll get feedback from potential users in the form of guerrilla user tests on prototypes; and I might run quick preference tests or surveys.

Using this process with clients is how we know as we go along that we're designing something strong. The different methods build on each other to create a richer picture of what is happening and provides lots of ideas for solutions.

Evolve what you have

There are still plenty of designers who sit there in a vacuum with just a brief and Sketch open hoping to come up with some magic solution. Or startup founders who think they have a problem and code up the first solution that comes to mind.

You’re usually not designing completely from scratch. Most design work is redesign work of some kind and when you’re trying to improve something you have a lot of ways you can get hold of evidence of user behaviour.

The big challenge in redesigning is knowing the order to gather your evidence in and which method is best for finding out different things. This is a central theme that I tackle in my video course, The Evidence-Based Redesign.

With a solid framework for redesigning you can avoid obvious mistakes. Learn more about it here and by following this link you get 75% off.

Further reading

Some further interesting articles that has helped my thinking on the challenges of gathering evidence and being data-driven:

Photo by Jo Szczepanska on Unsplash.

Last updated on 26 June 2018

ux / evidence / data-driven / data / quantitive / qualitative / google analytics / design process /

14 tools to gather evidence – guide