Formulating Hypotheses from Data and Heuristics

Hello! Welcome back to our course.

In our last few lessons, we've built a powerful toolkit for auditing user flows. We've conducted heuristic evaluations, used quantitative metrics like drop-off rates in funnel analysis, and mapped out actual user journeys with click path analysis. You've become a detective, gathering clues from different sources.

Today, we'll focus on the crucial next step: turning those clues into a compelling case. Our learning outcome is to synthesize quantitative data and heuristic findings to form testable hypotheses about user behavior. This is where your analysis transforms into a clear, actionable plan for improving the user experience.

1. From Data Points to a Full Picture: The Mixed-Methods Approach

So far, we've looked at two broad categories of data:

• Quantitative data: The what, how much, and how many. This includes metrics like drop-off rates, time-on-task, and the percentage of users who follow a certain path.
• Qualitative data: The why. This includes your heuristic findings, observations from session replays, and user comments from surveys.

Relying on only one type leaves you with blind spots. The real power comes from weaving them together. This formal approach is known as mixed-methods research.

To start, let's get a crisp definition of the two research types.

Now that we've refreshed those definitions, let's explore how intentionally combining them creates a much richer understanding.

As the article states, the goal is to gain a "layered understanding that connects what’s happening with why it’s happening." This is the essence of synthesis.

For example, in our previous lessons, we've been practicing an Explanatory Sequential design (a concept from the article, though you don't need to memorize the name). We start with quantitative data to find a problem, then use qualitative methods to explain it:

1. Quantitative Finding: "Analytics show a 40% drop-off on the checkout page." (What)
2. Qualitative Follow-up: "Session replays show users hovering over the 'Promo Code' field for a long time before leaving." (Why)

This synthesis leads to a much stronger insight than either data point alone.

2. Crafting a Testable Hypothesis

Once you've synthesized your findings into an insight, the next step is to frame it as a testable hypothesis. A hypothesis isn't just an observation; it's a proposed explanation for a phenomenon that can be tested.

But what makes a hypothesis truly testable and useful in a product design context?

To summarize, a hypothesis is worth testing if:

1. It's not based on strong, existing data: Your synthesis provides evidence, but it's still an inference, not a proven fact.
2. The opposite could be validly held: Could a reasonable person on your team argue the opposite? If so, it's a real question to be answered. For example, "Users need a simpler checkout" is testable because someone could argue, "Users actually need more payment options, not a simpler flow."
3. Being wrong would lead to a different design: If the outcome of the test doesn't change your design decision, it's not worth the effort. The test must have real consequences for the product.

3. A Framework for Writing Hypotheses

A well-formed hypothesis connects your evidence, your proposed solution, and your expected outcome into a single, clear statement. Here is a common and effective framework:

We believe that [PROPOSED CHANGE]
for [AFFECTED USERS]
will result in [EXPECTED OUTCOME / MEASURABLE CHANGE].

We will know this is true when we see [SPECIFIC METRIC(S)].

This structure forces you to be specific and connects your design change directly to a business or user goal. The "why" is implicitly baked into your choice of change and expected outcome, based on the synthesis you've already done.

Let's apply this to the exercise from our last lesson:

• Evidence: Path analysis showed users looping from the course page to the instructor bio page. The heatmap showed clicks on the non-interactive instructor photo.
• Synthesis: Users want to know more about the instructor before enrolling, and they expect to access this information from the course details page.

Using the framework, we can formulate a strong hypothesis:

We believe that making the instructor's photo on the Course Details Page a link that opens their bio in a modal
for prospective students evaluating a course
will result in a more streamlined evaluation process.

We will know this is true when we see a reduction in navigations from the Course Details Page to the full Instructor Bio Page, and an increase in the course enrollment rate for users who view the bio.

This statement is specific, measurable, and directly tied to the evidence we gathered.

4. Application Exercise

You are a UX designer for a project management SaaS tool. You've been tasked with improving user engagement on the main dashboard. After some research, you have the following data points:

• Quantitative Data (Analytics): Only 15% of daily active users click the "Create New Project" button. The average time spent on the dashboard is only 25 seconds before users navigate elsewhere.
• Heuristic Finding (Expert Review): The "Create New Project" button is a small, low-contrast icon located in the top-right corner, potentially violating the Visibility and Recognition heuristics. It gets lost among other navigation items.
• Qualitative Data (User Survey): In a recent in-app survey, a common theme in the free-text feedback was, "I wish it was easier to just start a new project right away."

Now, it's your turn to be the detective and the strategist.

1. Synthesize the Findings: In 1-2 sentences, describe the core user problem by connecting the three data points above. Think of it as "following a thread" between the pieces of evidence, as described in the Dovetail article "Mixed Methods Research Guide With Examples".
2. Formulate a Hypothesis: Using the framework provided above, write a single, testable hypothesis for a design change you would propose.

Conclusion

Today we bridged the gap between collecting data and taking action. By synthesizing different types of evidence and framing your insights as testable hypotheses, you create a clear, evidence-based rationale for your design decisions.

Key Takeaways:

• Synthesis is key: Combining quantitative data (the what) with qualitative findings (the why) provides a complete and compelling story of the user experience. This is the core principle of mixed-methods research.
• A hypothesis must be testable: It should be a specific, falsifiable statement where the outcome will directly influence your design.
• Use a framework: Structuring your hypothesis (e.g., "We believe that...") ensures clarity, connects your change to a measurable outcome, and communicates your reasoning effectively.

Next Up

You now have a list of well-formulated, testable hypotheses. But in any real-world project, you'll have more ideas than you have time to implement. In our next and final lesson for this module, we will address this by learning how to prioritize identified usability issues using an impact/effort matrix.