How do you know whether the app you’re building is the one your users need? This is a common challenge for organizations doing digital ministry. It’s easy to get caught up in your own excitement, what best serves internal stakeholders, or what users think they want. And those don’t always align with building what users truly need.
Let’s take a look at how to conduct user research that actually answers this question and helps you build the right app with the right features.
Step 1: Form a Hypothesis and Set a Goal
Before you begin your user research, start by forming a hypothesis and setting a goal for your analysis. This provides a framework for all of the data and insights you collect along the way.
A hypothesis gives you something to evaluate. Once you have a clear yes or no to that hypothesis, you can form a new one. On the other hand, performing research without a hypothesis will leave you constantly chasing the next bit of data.
Along the way, you’ll probably come across other ideas to question and test. Those will pave the way for future hypotheses and goals. User research isn’t the goal; the goal is delivering user outcomes.
Think of it this way: figuring out what your users need is a bit like putting a puzzle together. For most people, the first step in completing a puzzle is to hunt through the box to find all of the edge pieces. You use those pieces to assemble the frame. Then you continue working on the puzzle, one section at a time.
Along the way, you set aside those that you don’t need for the area of the puzzle you’re currently working on. But as you complete one section, you’re able to turn your attention to other pieces you saw along the way to continue assembling a full picture.
Let’s stick with this analogy for a bit and see how it applies to user research.
How do you choose a hypothesis?
First, you may be wondering how you’re supposed to choose a hypothesis in the first place. How do you know if your hypothesis is the right one?
Don’t worry, we’ve all been there!
The most important thing to remember is you just need to get started. Don’t let being afraid of choosing the wrong hypothesis hold you back from progress. Your first hypothesis might not be the best one, but if it helps you gather data and answer questions about your users needs, you’re on the right track. And that’s true even if your hypothesis is proven false.
A typical UX hypothesis follows this layout:
We believe (the change we want to introduce)… for (the users that this change is for)… will result in (the outcome we expect to have)…
And here are a couple of examples:
- We believe making the donation process feel more collaborativefor first-time donors will help new donors feel like they belong to something bigger.
- We believe adding a social element to the donation process for recurring donorswill help them feel like they are a part of a team.
- We believe moving the opt-in button higher on the pagefor website visitorswill make them more likely to sign up for the newsletter.
As you can see, these don’t have to be super sophisticated or complicated. They’re simply educated guesses about things you can change to achieve a certain outcome so you can test to see if your gut feelings are correct.
Step 2: Gather Data
Not everything that can be counted counts, and not everything that counts can be counted.
William Bruce Cameron
It’s easy to get caught up in adding features to your product that you think your users need or want, without any data to back up those assumptions. Without research, we often overestimate the importance of some features while missing other user needs altogether.
To figure out what your users truly need, there’s no way around the need for data. And there are two primary types of data we use in this process: qualitative data and quantitative data.
Qualitative data via ethnographic research
Qualitative data provides insights we can use to form our hypothesis for testing. Qualitative research is done through observation and interviews. Looking back at our puzzle analogy, this process is about hunting for the pieces we’ll use to form the frame—our hypothesis—before we can start assembling the interior of the puzzle. This type of research is focused on observing signals that offer some clue about the future.
You might be wondering why we’re starting with this qualitative research rather than things like Google Analytics and surveys. This type of subjective research allows us to step back and listen to users so we can gain insights into their behavior. Those insights often lead us to a new or better hypothesis. And from there, we can test the hypothesis using objective tools such as a survey or tracking data.
In other words, talking with real users—whether through individual interviews or larger focus groups—allows us to create hypotheses that can be validated across a larger sample.
This is especially important for early research because it gives us a broad understanding of the motivations, desires, and pain points of our users. We can then narrow those down through additional quantitative research.
Quantitative data via survey and tracking data
On the other hand, quantitative data allows us to take the hypothesis we’ve built (i.e. the frame) and begin filling in a more complete picture. This type of research relies on surveys and tracking data—measurable metrics—so we can learn from observable events in the past.
To start this process, begin with the tools and mechanisms you have in place already. This could include things like Google Analytics, a heat map on the site, survey tools, etc.
Next, narrow down your research to what truly matters for evaluating your hypothesis. There are plenty of data points to choose from, and trying to include them all will make it harder to get a clear answer to your hypothesis. Look at the data for very specific behaviors and narrow down your questions to the ones that directly connect back to your hypothesis.
Step 3: Evaluate the Data
Here’s the tricky part when it comes to this type of research: It’s easy for us to layer our own assumptions, desires, or goals on top of the data to get the results we’re hoping for. To conduct rigorous research that truly helps you build the right thing, you have to start by assuming all data is suspect until it’s validated.
It can be helpful to have a neutral third party walking alongside you during this process to bring objectivity to the analysis. But if you’re conducting the research on your own, here are some questions you should be asking along the way:
What is the data telling us?
Data itself doesn’t carry meaning; we have to interpret the meaning ourselves. And we do that by looking for the story behind the data. This is an opportunity to move from findings—the objective results that we can quantify—to insights that help the organization move forward.
It’s important to note that findings might not have any implication for the organization at all. Just because we can observe it, doesn’t mean it’s important. Instead, we have to decide whether the finding merits our further attention. One way to determine the value of the objective data is to ask yourself two questions:
- Does the customer need it?
- Can we do it?
If the answer to either of those is no, it’s not sustainable and should be set aside rather than pursuing it further.
Does the data look like the goal?
The next step, of course, is to consider whether the data and insights align with the hypothesis you’re testing. Do they confirm that you should continue on the current path, or is it time to pivot?
But that’s not the only alignment you want to look for. You also need to consider whether the hypothesis is still aligned with your business objectives considering the data and insights you’ve gathered. If it doesn’t, it could mean that your hypothesis needs to change.
Is the data pointing to a new path?
If the insights do not support the current hypothesis, then what are they telling you? The key to good user research isn’t making your findings fit your hypothesis. You shouldn’t have to squint to make them align!
Instead, be open to surprises. Being blindsided by something you didn’t see coming is really the beauty of this type of research. Surprises are good!
Sure, they can be disruptive to the way that we think about the value of what we provide to those we’re serving. But truly, this is an opportunity to expose those things and discover gold nuggets along the way. And the sooner this is done, the sooner you can pivot—saving you time, money, and effort.
Is there a conflict between qualitative and quantitative insights?
This question is a bit misleading; there will almost always be conflict between these insights. The question is whether that conflict matters. Sometimes this happens because of human nature: people often express different values verbally than their behavior shows. Other times, the mechanism we use to gather the data (for example, leading questions) might be the source of the conflict.
A conflict between qualitative and quantitative insights doesn’t mean anything is wrong. But it should prompt curiosity and a desire to identify the source of the conflict. If there’s not a clear opportunity to explore further, it’s okay to put a pin in it until a later date.
Can I prove it?
We start with a hypothesis: “Here’s what we think will happen. Here’s what we expect to learn.” From there, we gather data and findings. And then we create insights that can lead to decisions.
Once we decide that a change or feature will move the needle, we can begin to set performance goals. (That’s where the HEART framework comes in.) And from there, we can look for ways to prove our theory in the real world.
You won’t be able to prove all of your insights. And that doesn’t mean they’re not worth pursuing. But when you can prove them, it helps mitigate risk for your organization.
There is no easy way to know what your users need. It’s not something that can be decided simply by brainstorming as a team or building a good app. Instead, it involves rigorous, incremental research to get to know your users and identify where their needs intersect with your organization’s capabilities. But this process is worth it because building what users truly need will enhance the longevity and fruitfulness of what you’ve built!