Problem
An analyst wants to know what happens to an outcome if the team changes a lever , but has only observational data and no experiment. The usual fix is multiple regression with controls, where the model is picked by a chain of tests: functional form, set of controls, transformations. After such selection the estimates have a distribution whose closeness to the textbook limit is unknown, so confidence intervals and tests are generally invalid.
Example from the article: a naive regression of demand on price for some products usually gives a positive slope. Store management raises prices ahead of holidays, when many customers are expected, so the data mix the effect of price with the effect of the season.
Solution in brief
Treat the dependence of and on the controls as unknown and estimate it with machine learning. Then estimate the effect of from orthogonalized residuals (double machine learning, DML). The estimate keeps the same limit across a wide class of dependences on the controls, which repairs inference, at the price of a harder procedure and higher variance.
Questions this case answers
- How to estimate the causal effect of a lever on an outcome from observational data when the functional form of the controls is unknown and model selection by tests invalidates confidence intervals
- How to decide between a regression with selected controls and an orthogonalized estimator when the goal is a causal effect rather than a prediction
- How to estimate the causal effect of price on demand from observational data with many controls
- How to get valid confidence intervals for a treatment effect when the functional form of the controls is unknown
Tools and methods
- Double machine learning (DML) — nonparametric nuisance estimates plus orthogonalized moments
- Double selection (e.g. LASSO) — select controls for both equations, merge, run a long regression
- Multiple regression with test-based selection (LSE approach) — the baseline being criticized
- DoubleML library — R and Python implementation of DML
Google or a one-time email code. No password.