Geo Holdout Test Planner
A geo holdout turns off a channel in some regions and compares them with the rest. Whether it can prove anything depends on how noisy your regional revenue is and how big the channel's effect really is. Plan it before you switch anything off.
Run 9 weeks to detect 8.0%, or reduce noise with better-matched regions or a longer baseline. Read the geo holdout guide for market selection and readout.
Talk to us about your measurementA brand with $200,000 of weekly revenue holds out regions worth 20% of it. Their weekly revenue swings about 10% and tracks the control regions with a correlation of 0.8. With 12 weeks of baseline, a 6-week test can detect a lift of about 8.9%. If the channel drives 8% of revenue in those regions, the test needs 9 weeks, and the holdout forgoes about $19,200 of revenue over 6 weeks.
What the planner calculates
The planner treats the test as a difference-in-differences on weekly revenue: how the gap between holdout and control regions changes once the channel is switched off, compared with the same gap in the baseline weeks. The noise in that weekly gap sets everything else.
Week-to-week swing and correlation set the noise. Regions that rise and fall together cancel most of their shared ups and downs, so a correlation of 0.9 instead of 0.7 cuts the noise by about 40%. Better-matched regions shorten the test more than anything else you control.
Choosing holdout regions
Match holdout and control regions on revenue trend, seasonality and customer mix, not just size. Use whole media markets so ads cannot leak across the border, and avoid regions with store openings, local promotions or pricing tests during the window.
A holdout worth 10% to 25% of revenue is a common balance between statistical power and the cost of going dark. Smaller holdouts are cheaper but noisier; the swing you enter should reflect the regions you actually pick.
When the answer is "not detectable"
If the planner says the test cannot detect the lift you expect, a longer test is rarely the best fix. Lengthen the baseline, pick better-matched regions, increase the holdout share, or test a bigger change such as switching off two related channels together.
Frequently asked questions
What is a geo holdout test?
An experiment where a marketing channel is paused in selected regions while it keeps running elsewhere. The difference in revenue between the two groups, against their pre-test baseline, estimates the channel's incremental effect.
How long should a geo holdout test run?
Long enough to detect the lift you expect, usually 4 to 8 weeks, plus a clean baseline of 8 to 12 weeks. The planner gives the exact number for your volatility and expected effect.
Is geo holdout the same as geo-lift testing?
They are the same design seen from two sides. A holdout turns spend off in test regions; a geo-lift test turns spend on or up. The planning math is the same.
Why not just trust platform attribution?
Platform attribution counts conversions the ads touched, including many that would have happened anyway. A holdout measures what disappears when the ads stop, which is the number your budget decisions need.
More free tools
Ready to prove your marketing ROI?
Book a free 30-minute consultation. Bring your ad account numbers; leave knowing which channels earn their budget.
Free. No commitment.