Problem
Marketing teams need to know how many conversions a campaign adds beyond organic ones. User-level A/B tests do not work for TV, radio and billboards, and privacy rules complicate them on third-party platforms. The usual fallback is last-click attribution, which gives all credit to the final touchpoint. It ignores earlier exposures and over-credits paid media for conversions that would have come organically. Channel ROI and budget allocation then rest on a wrong picture.
Example from the article: LinkedIn needed to know how much of the app installs reported by a Google Universal App campaign on Android were truly incremental.
Solution in brief
Run the campaign in selected regions only and use the other regions as controls. A Bayesian structural time series (BSTS) model, trained on the pre-campaign period, predicts what the test regions would have done without the campaign. The gap between observed and predicted values is the incremental effect. Before launch, an A/A test and a simulation-based power analysis check that the design can work.
Questions this case answers
- How to estimate the causal effect of an intervention when units cannot be randomized individually and only aggregated time series per region are available
- How to design and power a region-level experiment when no closed-form power formula exists
- How to measure the incremental conversions of a marketing campaign when user-level A/B tests are impossible?
- How to choose test and control regions for a geo experiment?
Tools and methods
- Geo experiment — regions, not users, are the experimental units.
- BSTS counterfactual (CausalImpact R package) — predicts test-region performance without the campaign.
- MarketMatching / stratified sampling — pick comparable control regions or build representative test samples.
- A/A test and simulated power analysis — validate the model and size the test before spending budget.
Google or a one-time email code. No password.