Incrementality is the most important measurement question most teams aren't answering. "Did this channel actually cause the conversion, or would it have happened anyway?" Platform attribution says yes — it always does. Every ad platform is incentivized to claim credit. Incrementality testing is the only method that gives you a defensible causal answer: hold out a group of users from seeing the ad, then measure the difference. The gap between exposed and unexposed is your true lift.
There are three main ways to run a holdout test. User-based holdouts split your audience randomly into exposed and control groups — highest precision, but requires platform support and often underestimates true lift due to spillover. Geo-based holdouts use geographic markets as the unit of control — harder to game, works across channels, and is increasingly the preferred method for large cross-channel budgets. Time-based tests use pre/post analysis with a control period — simplest to run, but weakest on causal validity. Each method answers the same question with different confidence levels and different data requirements.
Below $1M in annual ad spend, incrementality testing is usually overkill — you don't have the statistical power to get clean results, and the design time costs more than the insight is worth. Between $1M–$5M: one or two geo holdout tests per year, run through GeoLift or Meta's native Conversion Lift, will tell you more than months of MTA analysis. Above $5M: systematic testing across channels is table stakes. Above $20M: a dedicated platform like Measured or Haus pays for itself in the first quarter by surfacing channels where platform-reported ROAS is materially overstated.
User-based holdouts have more statistical power per dollar spent but can suffer from spillover — users in your holdout group still see your brand elsewhere. Geo-based holdouts are harder to design but measure total cross-channel incrementality, which is often the truer number. Most mature measurement programs run both.
A properly powered geo holdout test needs 4–8 weeks of runtime and enough markets to reach significance. Teams that rush tests or run them in too few markets get noisy results — often confirming what they already believed rather than learning anything new. The test design is where most programs fail, not the analysis.
Meta and Google both offer free conversion lift tests — and both are run on their own infrastructure, with their own methodology, measuring their own channels. That's not independent. For budget decisions above $500K, you need a third-party platform running the test. The platform-native tools are a starting point, not an answer.
Real incrementality measurements consistently find that platform-reported attribution is wrong in both directions — over- and under-counting depending on channel and methodology. The gap between what platforms claim and what holdout tests find is the most important number in your measurement stack.
Sources: Measured customer case studies. Results are specific to these brands and not guaranteed.
Not sure which measurement approach fits your spend?
Answer 5 questions and get a personalized stack recommendation — matched to your ad spend level, channels, and team capacity.
Get a personalized stack recommendation →Clearpath Analytics specializes in incrementality test design, geo holdout methodology, and translating results into budget decisions. Run by the founder of SaaSMatchup.