Problem
Retail and marketing teams often need to know whether a promotion or marketing event changed sales, but cannot always run an A/B test. A plain before/after comparison or a correlation mixes the effect of the intervention with confounders. A wrong estimate scales campaigns that did nothing.
Example from the article: to measure the impact of a marketing event on sales, compare actual sales with the counterfactual, meaning sales if the event had not been introduced.
Solution in brief
Pick the estimand first (ATE, ATT, ATC or CATE). Draw a causal graph to find confounders, optionally learning it from data with PC, FCI, GES or LiNGAM. Then estimate the effect with the method that fits the data: difference-in-differences for treated and control groups with pre and post periods, DoWhy/EconML or a Bayesian network otherwise, CausalImpact for an effect over time in a time series.
Questions this case answers
- How to estimate the causal effect of an intervention from observational data when randomization is not available and confounders may bias the comparison
- How to choose a causal estimation method given whether pre-intervention history, a control group or a single time series is available
- How to measure the effect of a promotion or marketing event on sales when an A/B test is not possible?
- How to choose between difference-in-differences, DoWhy/EconML, a Bayesian network and CausalImpact?
Tools and methods
- PC, FCI, GES, LiNGAM — learn a causal graph (DAG) from data
- Difference-in-differences with matching — effect from pre/post change of treated vs control
- DoWhy and EconML — model, identify, estimate and refute an effect
- Bayesian network with do-calculus — effect via backdoor adjustment
- CausalImpact — effect over time via Bayesian structural time series
Google or a one-time email code. No password.