Problem
LinkedIn cannot randomize everything. Brand campaigns on TV or billboards, bugs and downtime, and economic shocks all affect members, yet nobody can assign them at the user level. In such cases a raw comparison of treated and untreated users is confounded: for example, more active users take the treatment and also have more sessions anyway. Before the platform, one study took a few data scientists (an expert plus a domain expert) up to six weeks, and only 10-20 studies existed.
Example from the article: a table of sessions where treated users look better only because highly active users selected the treatment (confounding).
Solution in brief
Ocelot is an internal web app plus data pipelines that offers five causal methods behind a guided form. The pipeline prepares the data with correct date windows and always runs the checks. A weekly review committee decides whether a study may be called causal.
Questions this case answers
- How to estimate the causal effect of a change at scale when randomization is impossible and few analysts know the methods
- How to decide whether an observational effect estimate can be trusted and presented as causal
- How to estimate the effect of a product change when an A/B test is infeasible or too expensive?
- How to let ordinary data scientists run observational causal studies without a causal-inference expert on every project?
Tools and methods
- Coarsened Exact Matching (CEM) — matching on coarsened covariates, cross-sectional data.
- Doubly Robust (DR) estimator — augmented inverse propensity weighting, cross-sectional data.
- Instrumental variables (IV) — when a valid instrument exists.
- Fixed effects models (FEM) — panel data.
- Bayesian Structured Time Series (BSTS) — time series data.
- A/A-style checks — settings where the effect must be zero.
- Apache Spark — data preparation at the scale of 875+ million members.
Google or a one-time email code. No password.