Four ways to win at user research
At the heart of any product design project is user research. It’s important to validate assumptions and learn about your users before actually designing and developing anything.
Getting started with research can feel daunting, especially when there are many different forms it can take. The good news is that whilst conducting UX research is one of the most powerful sources of information, it’s also a skill that can be learned.
Learning to interview people is a skill that people can develop with training, and something most organisations can, with the right training, advice and guidance, undertake in-house.
We’ll discuss the pros and cons of four main methods here, as well as our feelings on each of them. It goes without saying that as the requirements of every product and business vary, so does the type of user interview that will serve them best.
“Everyone likes to get their opinions heard. Don’t underestimate the loyalty you’ll also build with customers by including them in research exercises; it’s a bonus benefit to all of these techniques.”
#1 Ethnographic field study
Ethnographic research involves observing people in their natural environment. It is immersive and contextual, and an in-depth exercise in which you follow someone around for period of time to observe exactly what they do while asking questions along the way. In many ways, following a user around and completely immersing yourself in their experience seems like the gold standard when it comes to finding out about them. Even the UK government talks about how they use ethnographic research for creating digital health products.
Benefits
You’ll see the minutiae of a user’s life, and the moments that people don’t even realise are important. In user interviews, most people will talk about major, memorable things that happen, not the little bits of non-verbal gold that you might observe doing this.
Directly seeing someone’s pains is going to give you an unparalleled insight into them, and ultimately no business gets worse for knowing a lot about their user.
Downsides
It’s time intensive considering the time onsite and subsequent analysis, and therefore can be costly.
Also however skilled a researcher you are, there will always be a strong risk that users won’t act naturally with you in their environment. Even if they try not to, people behave differently when they know they’re being observed.
“It’s a fine balance between observing, prompting and being a bit of a nuisance. Getting good at this requires practice and an innate instinct for what’s suitable for the person you’re currently shadowing.”
Do we use it?
Full disclosure, this is actually something we don’t do very often. It takes time to organise this type of research, and for many of our clients the resource & time required doesn’t justify the value it provides their product. It can be quite a task to convince stakeholders of the benefit of lower effort research methods, so this method is often reserved for those further along their research journey.
That being said, when used in the right context, it is incredibly insightful. For example:
One of our team has conducted this research with two different supermarkets - one where they were working on an internal product and observed the staff member on a typical working day; another where we followed customers around the supermarket observing them use our beta product whilst they did their weekly shop. For both cases it was eye opening seeing them juggle their device whilst physically trying to carry out another task.
Our team shadowed delivery drivers for a transport company on a chilly November morning, and witnessed them fiddling around with their gloves whilst trying to click the CTA on the device.
For organisations such as charities, direct understanding and really learning about the service users whose lives they hope to improve is crucial. They often have dedicated research teams to make this level of activity more affordable.
Our verdict
If it’s valuable knowing the use of product in the physical context, and there is appetite to invest in solid product research, then this research method is absolute gold. We love it and definitely recommend it. 👍
#2 Focus groups
Getting a bunch of people in a room is one of the most traditional ways of conducting market research. It would probably be the one mentioned most often if you asked a person on the street to name a research method. (Which in itself would be a form of user research 😁 - guerrilla research to be specific, but more on that another time!)
Benefit
Focus groups are a relatively low cost way to get access to a decent sized group of people at once - essentially you’re killing 5 birds with one stone (or to use a nicer proverb, eating 5 grapes with one mouth 🍇). You gain the non-verbal observation benefits of immersive research and get insights in a structured way all in one go. Being able to playback a video recording of the session is also a great way to pick up on missed comments that were overlooked in the group discussion.
Having a group of people all in one place is a good way to start a conversation around the topic you’re discussing, something that’s almost impossible using the other techniques. You have total control over who you invite so you can make the audience as diverse as you want to see how opinions differ.
“It’s also a great way to show visual assets for discussion and feedback. Focus groups are very popular with companies bringing new physical products to market.”
Downsides
The biggest issue with focus groups is the nature of groups themselves. Managing the loud, the overbearing, the shy and the awkward in such a way that everyone can a) get their opinion heard and b) doesn’t go along with ‘group think.’ The person who shouts the loudest often leads the way. If someone was particularly introverted or shy, they could keep their head down and contribute little, and it can be challenging to encourage them to engage.
Do we use it?
Not really. For us, the group bias we’ve just talked about can be really hard to overcome, and we find other research methods are our preference.
We rarely conduct focus groups, but if push comes to shove and it’s the only reasonable option - due to budget or time constraints - then we make it work. In fact a couple of months ago we facilitated a focus group with a Women in STEM group, as there wasn’t enough time to incorporate 1-1 interviews into the plan. To overcome the downside of focus group research, we put extra measures in place to reduce potential group bias, facilitated the session as more of a workshop than a discussion, and created activities that elicited equal contributions from the individuals.
Our verdict
We avoid this method if we can help it. If it’s a choice between no research and focus groups, then we’ll definitely opt for focus groups and put measures in place to mitigate the downsides.
#3 Remote interviews
Interviews over a video call allows you to speak to your users without having to ever meet them in person. These typically are 45-60 minute interviews with only two people present: the researcher and the participant. These can take various forms, most often it’ll consist of open questions followed by providing feedback to a visual stimulus.
Benefits
A one-to-one chat with someone is a great way to have a detailed conversation around a topic. You can follow a discussion guide, but at the same time you’re able to explore avenues and react directly to replies and cues from them.
Video calls are not particularly intrusive, and easier to fit around people’s schedules than meeting in person, so users are more likely to commit to taking part.
Findings can easily be recorded and several people can sit in without crowding into their front room. With WhatsApp, Zoom, Google, et al. you can reach users all over the world with minimal cost, you just need to give them an incentive to turn up to the session - usually £50 does the trick.
Downsides
If you’re just seeing someone from the shoulders upwards on a screen, there’s a chance you’re missing out on small details of their body language and non-verbal reactions.
You likely only have an hour max for these calls so fitting it all in can be tricky, especially if there’s lots to talk about. It’s quite easy for people to ramble so if you’re not in control of the conversation you might find you don’t get the insight you need before your time’s up.
Conducting interviews over video call at large scale can be time consuming. Remember that the more you do, the more you have to analyse. Formatting results can be a challenge – you really need to be on top of what you’re going to do with all this information in order to get value from it.
Last but not least, bear in mind that people can ignore your attempts to contact them this way more readily – missing a video call isn’t a huge social faux pas in the same way not turning up to a face–to-face session would be.
Do we use it?
Absolutely we do, it’s our preferred method. We usually run video calls as the very first part of any product design project, to speak to existing or potential users directly.
We’ll usually book in a small set of people (5-10) to talk to first, discuss findings and then evaluate whether we need to do more. Often you’ll get enough from a number that small to see trends in conversations. Having a good hypothesis and structured conversation here will also help the process.
Our verdict
Our favourite research method by far. It’s a great balance between the effort it takes to conduct, with the invaluable insights you glean. 👍
#4 Online quantitative surveys
You’ve no doubt filled in loads of surveys online. Survey Monkey, Typeform, Google Forms – these are some of the biggest names in the game for running online surveys. Quantitative surveys have closed questions, meaning you can analyse the results numerically and objectively.
Benefits
If you need statistically significant findings, and to test your assumptions at scale, this is the way to go. A quantitative online survey will produce data that takes a lot less effort to analyse than any other form.
Add to this the ability to target demographics, whilst allowing users to remain anonymous, and you can see why many people are tempted to do lots of their research this way.
From a cost perspective it’s a huge win. The amount you’ll pay per response will be drastically lower than all the other options here.
Downsides
We can’t stress this enough – survey design is SO much harder than you’d think. We see a lot of surveys on a regular basis that won’t actually deliver any insight.
This can be due to length – it’s a fact of life that human attention spans aren’t long and a set of questions that takes 20 minutes to answer is rarely going to do well.
It can also be due to badly written or formatted questions. We often see survey questions where the answers available don’t allow for a level of nuance that is needed, making the question impossible to answer correctly. Allowing for shades of grey is something that takes a lot of thought and requires a good level of understanding of the end user.
Expressing personality through an online survey is tricky too, so striking up a rapport with users is practically impossible. You lose the human element using this technique, which can be a very important way to get unexpected insight from your users.
It’s harder to target exactly the right audience you want. If you use an online platform’s audience,
Lastly though, and potentially most frustrating of all, are fake responses that impact the validity of the results. Sometimes people just answer ‘N/A’ or skip through questions to get through quickly without much thought. Although annoying, it’s easy to spot these and therefore you can eliminate those responses from your analysis. The real stinkers are the ones that choose real answers; you have to embody your inner Poirot to identify inconsistencies in answers to be confident enough to eliminate these fake results from your analysis. Looks like SurveyMonkey has integrated a method to overcome this common challenge!
Do we use it?
We do, but usually as an extension or follow up to video interviews. It’s a great way to back up your initial findings at scale and explore the detail that you’ve uncovered to see whether what you’ve found out is shared by a wider group. In these instances our surveys are quantitative and consist of closed questions, in order to quantify our insights.
A very effective use of surveys we’ve found is a super targeted one, for one specific aspect of the research done already. This is a great way to further test a topic that still has unknowns. Shorter surveys can often lead to better response rates and can also be easier to design.
It’s still hard to overcome those fake responses, we try different ways to overcome it - if you have any ideas, we’d love to hear them!
Our verdict
Despite the list of downsides, we do really value this research method. It comes with its challenges, but it is a relatively low effort method compared to others, and the results are highly valued; senior stakeholders in particular are fans of quantifying insights to give confidence in a way forward for your product. 👍
Conclusion
The beauty of all of these options is that they all work in the right scenario. Doing some research is always better than doing none, but it’s down to you to choose which technique suits your user and what you’re trying to learn about them.
Above all, have fun with it. Research is one of the most exciting parts of a digital design project for us as we get to learn loads about groups of interesting people. You’ll hear some surprising things, you’ll make mistakes and you’ll have your plans shattered but it will all be of benefit to your project.
Need some support as to which method to use? We’d be happy to have a chat and give some guidance!
