“Write code. Talk to users”
- YC Combinator essential advice
“Talk to your users” is one of the most common mantras in the tech world, and with good reason. It’s really hard to build good products if you don’t understand your customers and what their needs are, and the fastest way to understand their needs is to speak to them.
Starting to have regular conversations with users will get you a long way, regardless of what you talk about. But just talking only gets you so far. People being interviewed can be unreliable witnesses and poor judges of their own character, even when they have the best intentions. And similarly, even experienced researchers can run unfocused conversations or inadvertently bias towards confirming their existing beliefs.
If you want to get the most out of your customer conversations then you need to talk to people the right way.
This article is about how to have really high quality conversations with your users. We’ll touch on how to get in front of users only briefly.
If you want more guidance on recruiting, screening and scheduling user tests then check out our article on Continuous Discovery Habits.
Defining our terms
Product discovery is fundamentally the practice of working out what to build, and stands in contrast to product delivery, or the actual act of building.
User testing and user research are general terms for gathering information on your users in discovery. Whilst this includes quantitative methods like AB testing, it’s often used as a shorthand for qualitative methods - i.e. speaking to users - which is what we’ll focus on in this article.
Product discovery itself can be further broken down into two phases, each served by its own type of user research.
This is where you work out the most important customer problems to solve (e.g. is a hirer more concerned by diversity of hires, quality of hires, or time to hire?).
This phase is informed by user interviews, which are open, exploratory conversations with users that attempt to understand their needs and what they think of high level concepts.
This is also known as generative research.
This is where you test if the proposed solution is effective at solving the problem (e.g. do anonymised interviews increase hiring diversity?).
Here you use usability testing to test out a specific concept, whether that’s with a prototype, MVP or live product.. Here you ask users to complete specific tasks (e.g. “buy some hiking boots”), and watch what happens.
This is also known as evaluative research.
That said, it’s not uncommon to run sessions that are a mix between a generative user interview and a evaluative usability test - e.g. ask some open ended questions before testing a prototype on a user. As always, don’t get too hung up on the terms, just do what you need to to build great products.
Variations
There are a couple of other variations of user research worth covering in case you come across the terms.
Moderated vs. unmoderated
Moderated vs. unmoderated refers to whether there is someone watching and facilitating the user test live (moderated), or whether the participant is following instructions and the results (often a video recording) can be watched later (unmoderated).
Generative research is usually moderated, so that you can follow up on interesting things that the participant says. Evaluative research can be moderated or unmoderated. Unmoderated testing allows you to run usability testing at scale quickly, often using a dedicated platform (e.g. usertesting.com)
Remote vs. in person
If you’re doing moderated tests, you can do these remotely or in-person. Post-pandemic people often default to remote conversations as they are easier to schedule and you don’t need a physical space to run the test. However, as with other human interactions, in-person testing leads to a richer experience, where you can read the body language of the people you’re talking to. Unmoderated tests are always remote.
User research process
Before you can speak to users you’ll need to find some to speak to. We cover this in more detail in continuous discovery habits, but here’s a quick recap.
Design