Hello! Welcome to the first lesson in our module, "Incrementality and True Marketing Impact."
In our last module, we focused on experimentation, culminating in how to build a strategic roadmap for A/B testing. Those tests are fantastic for optimizing elements within your marketing funnel—improving a landing page, testing ad creative, or refining a user flow.
Today, we take a significant step up in our measurement thinking. We will address one of the most fundamental and challenging questions for any marketing leader: how do you know if your marketing spend is actually causing business growth, or just getting credit for sales that would have happened anyway?
Our learning outcome is to explain the core concept of incrementality and why it is critical for measuring marketing's true value. This concept is the foundation for making sound, multi-million dollar budget allocation decisions and for truly understanding the financial impact of your team's efforts.
1. What Is Incrementality? Moving Beyond Correlation to Causation
In your 12 years of experience with platforms like Google and Meta Ads, you've seen countless reports showing conversions and Return on Ad Spend (ROAS). These platforms are very good at showing a correlation: people who saw or clicked your ads also converted.
However, as a strategic leader, you need to answer a tougher question: did your marketing effort cause those conversions? This is the central idea of incrementality.
Marketing Incrementality: The Ultimate Guide to ...
To start, let's get a formal definition. This article from Improvado, 'Marketing Incrementality: The Ultimate Guide to Measurement & Testing', provides a clear and concise introduction to the concept.
Please read the introduction and the section titled 'What Is Marketing Incrementality?'. Pay close attention to the core question it poses: 'What would have happened anyway?' and the distinction it draws between correlation and causation.
As the article highlights, incrementality measurement is about isolating the outcomes that happened only because of your marketing. The most basic way to do this is through a controlled experiment. You split your audience into two groups:
- Test Group (or Treatment Group): This group is exposed to your marketing campaign (e.g., they see your ads).
- Control Group: This group is deliberately held back and is not exposed to the campaign.
By comparing the behavior of these two groups, you can isolate the true impact.
Incremental Lift = (Conversions from Test Group) - (Conversions from Control Group)
The conversions from the control group represent the baseline—what would have happened organically or through other channels. The difference is the incremental value your campaign created.
Now, let's see a practical example of how this calculation works.
Incrementality Testing in Digital Marketing & testing architecture
This video from the 'Senator We Run Ads' channel walks through a simple case study. It's a great example of calculating incremental leads and a more accurate CPA.
Please watch the section from 01:19 to 03:32. As you watch, note one important point: the presenter reverses the standard terminology, calling the group that sees the ads the 'control group'. In standard practice, the group that sees the ad is the 'test' or 'treatment' group. However, the logic and the calculation he demonstrates are perfectly correct and valuable for understanding the concept.
This simple calculation reveals a more truthful Cost Per Acquisition (CPA) or ROAS, which we'll call incremental CPA (iCPA) or incremental ROAS (iROAS). This is your first step toward measuring the true efficiency of your marketing spend.
2. Why Is Incrementality Critical for Business Strategy?
Relying solely on platform-reported metrics can lead to significant strategic errors. The metrics are designed to show the platform's value, but not necessarily your business's incremental growth.
A famous example comes from Uber. They discovered they were spending over $100 million on digital ads that were providing virtually no incremental value. The ads were simply capturing conversions from users who were already intending to use Uber.
What is incrementality testing?
The article 'What is incrementality testing?' from Funnel.io does an excellent job explaining why this measurement approach is so vital for modern business growth, especially in a world of complex customer journeys and increasing privacy constraints.
Please read the sections 'How does it help marketers?' and 'Why incrementality testing matters for business growth'. Focus on how incrementality helps you: Separate real growth from vanity metrics. Enable data-backed budget decisions. Prove marketing's value to finance and other executives.
As the article points out, adopting an incrementality mindset allows you to:
- Justify your budget with causal data: When you go to the CFO to ask for more budget, you can move from "Our ROAS on this channel is 4:1" to "For every $1 we spend on this channel, we cause $4 in new sales that would not have happened otherwise." This is a far more powerful and defensible position.
- Avoid cannibalization: You might find that your branded search campaigns have a fantastic ROAS. But an incrementality test could reveal that 80% of those users would have clicked your organic link anyway. In that case, you're paying to acquire customers you already earned.
- Navigate a privacy-first world: As tracking becomes more difficult due to changes like iOS14 and the phase-out of third-party cookies, attribution models are becoming less reliable. Incrementality testing, which is based on controlled groups, is a robust alternative that doesn't rely on tracking every single touchpoint.
Test your understanding!
Your team is running a retargeting campaign on Meta for users who abandoned their shopping cart. The Meta Ads dashboard reports a 10x ROAS for this campaign. As a strategic leader, how would you use the concept of incrementality to question this result and frame a hypothesis for testing?
Show answer
A great response would be grounded in the core question: "What would have happened anyway?"
-
Question the Result: "A 10x ROAS is impressive, but it's based on correlation. These are already high-intent users who added items to their cart. How many of them would have returned to purchase on their own, even without seeing the ad?"
-
Frame the Hypothesis: "My hypothesis is that the true, incremental ROAS of this retargeting campaign is significantly lower than 10x because we are primarily showing ads to users who were already likely to convert. The platform is taking credit for conversions that were not caused by the ads."
-
Propose a Test: "To test this, we should run a holdout test. We can split our cart abandoner audience into two groups: a test group that sees the retargeting ads, and a control (or holdout) group that sees no ads. By comparing the conversion rates and revenue from both groups, we can calculate the true incremental lift and iROAS."
This approach demonstrates strategic thinking that goes beyond surface-level platform metrics.
3. Incrementality vs. Attribution: Different Tools for Different Jobs
Your experience has likely been steeped in the world of marketing attribution. Attribution models (last-click, first-click, linear, data-driven) are focused on distributing credit for a conversion across various touchpoints. Incrementality answers a different question.
Understanding this distinction is crucial for building a sophisticated measurement strategy.
Marketing Incrementality: The Ultimate Guide to ...
Let's return to the Improvado article, which has an excellent section and table comparing Incrementality, Attribution, and Marketing Mix Modeling (MMM).
Please read the section 'Incrementality vs. Attribution vs. Marketing Mix Modeling (MMM)'. Focus on the table and the core question each framework answers. This is a critical distinction for a marketing leader.
Here's the summary for your role as a leader:
- Use Attribution for... Tactical, Intra-Channel Optimization. It helps your team understand which campaigns, ad sets, or keywords within a single platform (like Google Ads) are contributing most to the customer journey. It helps answer, "Within my search budget, where should I put the next dollar?"
- Use Incrementality for... Strategic, Cross-Channel Budget Allocation. It helps you decide the total budget for a channel. It answers, "Should I move $1 million from Meta Ads to TikTok, or to offline channels?"
They are not enemies; they are partners. A mature marketing organization uses both to make smart decisions at different levels.
Conclusion
In this lesson, we've introduced the concept of incrementality as a cornerstone of strategic marketing measurement. Moving from correlation to causation is what separates good operational management from great strategic leadership.
Key Takeaways:
- Incrementality's Core Question: "What would have happened if we didn't run this marketing activity?" It measures the causal lift of your marketing.
- Methodology: It relies on creating a test group (exposed to marketing) and a control group (held back) to isolate the true impact.
- Why It's Critical: It helps you calculate true ROI (iROAS), eliminate wasted spend on non-impactful campaigns, and make defensible, data-driven budget allocation decisions.
- Incrementality vs. Attribution: They answer different questions. Attribution assigns credit along a path, while incrementality proves causation. Use attribution for tactical optimization and incrementality for strategic budget planning.
Preview of the Next Lesson:
Now that you understand what incrementality is and why it matters, our next lesson will focus on the "how." We will explore the common methods for running these experiments, such as geo-lift studies, conversion lift tests, and holdout groups. You will learn how to critique the design of these tests to ensure the insights you receive are valid and actionable.