Advice: Evidence methods

User Feedback

What you can learn

This method refers to the ad-hoc user feedback a company might receive, potentially coming from enquiry forms, emails, social media, over the phone, or even in-person. This is feedback that doesn't just exist on live chat or feedback you have sought out in surveys, it can come in at any time.

With ad-hoc user or customer feedback you often learn the things that truly bother people, as they have taken the time to seek you out and complain or suggest something. They haven't just waited until asked. It is important to collect this as it is the kind of thing that can be easily forgotten or lost if there is no system.

How to do it

Being ad-hoc this kind of feedback can be hard to track, so you'll need to set up a system to get it recorded, which will usually involve a bit of manual work. The majority of your feedback will come into customer service teams, either via email or phone. You will need them to log every time a customer complains about something or makes a suggestion for improvement.

This can be done via CRM software if you have the budget or it can simply be a shared spreadsheet where they can copy/paste it or enter a summary of the issue and who the user was that raised it. These customer service staff need to be aware when something is just plain broken however—as this means a bug ticket needs raising with the development team. The feedback this doc gathers should be more long-term and focussed around improving features.

Once you've got a solid system for logging ad-hoc feedback someone (probably a designer or product manager) will need to check it regularly and classify the different issues to make spotting similar ones easier. Either tag in a CRM or add an extra column of data in a spreadsheet where you specify what category issue each thing is, for example 'password reset emails slow', 'wants wishlist functionality', 'bigger upload capacity' etc.

When you group issues like this you can keep a running total of the most requested or complained about features. This leaderboard will then be another element in you evidence for deciding the priority order for redesigns and the biggest problems you need to tackle with those new designs. By checking it every week or so you can see if there are sudden spikes in certain issues or a leading problem that needs fixing more urgently.

Watch out for

When dealing with this kind of feedback there will always be a bias towards the negative. As humans the frustrations or the bad things that happen stand out more than the smooth easy experiences, and as online user experiences improve people get more accustomed to everything working really well first time. Just be aware that you're mostly going to see problems highlighted in this feedback so be careful not to throw out everything that is good when redesigning to solve the issues.

Generally, people don't like change. A common time for a raft of complaints is if you release a big redesign of a major feature or part of your website. If it's something lots of people use every day then don't be surprised to see a load of confused and sometimes angry feedback when you first launch. This is to be expected as they adjust. However if they're still making the same complaints a week later then you might have a real problem.

Not all complaints are created equal. Someone quickly firing off a moaning Tweet is not as meaningful as someone taking the time to phone up and explain their problem. You might want to weight your feedback to reflect the source it came from.

Be careful not to misunderstand the feedback. It can be hard to truly understand what someone means if they've just put a few sentences down. People can have all sorts of odd ways of phrasing online behaviour and may not know the correct technical terms. Flag feedback you're not sure about and try and clear it up with the person that gathered it.

Customers and users don't always know what they need so focus more on their problems than suggestions for new features. They may think they need a big all-singing, all-dancing feature but a simple tweak may be just as good. It's your job to define the solution, don't just implement what they ask for.

Example tools (and cost)

There are lots of different CRMs out there, and they’re generally not that cheap to implement. Some focus more around sales but usually incorporate customer service too and the most well-known are: Salesforce, Zoho, and Sugar. The most basic free shared log you can use would be a Google Spreadsheet (a tool I've mentioned a few times in this guide and something many startups are built on!).

How long does it take?

To check on and classify 30-50 pieces of feedback a week should take an hour or two.

How often should you use it?





Last updated on 9 March 2017

Learn how to make evidence-based improvements to your website

Sample video images from The Evidence-Based Redesign course

Learn how you can redesign a website based on evidence of how users actually behave, without spending a fortune. A new video course coming in January 2018 but register today and get a massive pre-launch discount.

Here's another method

Customer Interviews

Speaking to customers is a great way to properly understand their behaviours and motives. Learn more

Last updated March 2017

View all the methods in the guide

Note: the examples in this guide are for website design, but most of the content is also applicable for native apps and software.