Correlation Lied to Us: Rethinking Product Impact with Causal Inference

Enderson Santos4 Aug 2026

Problem

Professional sellers on OLX pay for higher-tier packages that include more boosts (1, 2, 3 or 4 boosts over 30 days), so more visibility should mean better results. Yet in some cases premium-package ads performed worse than cheaper ones. Sellers pick their own package, so premium ads differ in inventory, price and seller behavior, and a package-level comparison mixes ad quality with exposure. The team needed the incremental value of extra exposure, but randomized tests of free boosts are costly.

Example from the article: a new, low-mileage SUV with automatic transmission and a premium brand is likely to sit in a high package, while an old high-mileage manual car sits in a low one.

Solution in brief

Estimate each ad's probability of being in the higher package from its observable attributes, then match ads with similar probabilities across tiers and compare their outcomes. Check the matching with overlap, covariate balance and the time pattern of the gap. Calibrate the result against a randomized A/B test.

Questions this case answers
  • How to estimate the incremental value of a paid visibility product when buyers of different tiers differ systematically from each other
  • How to verify that matching on observed covariates removed selection bias before trusting the effect estimate
  • How to evaluate many treatment levels offline when randomized tests are too costly to run for each one
  • How to measure what extra exposure (boosts) really adds to an ad when sellers choose their own package?

Tools and methods

  • Propensity score matching — pairs ads from different tiers that had a similar chance of getting the higher package
  • XGBoost — regularized, complexity-constrained classifier that produces the propensity scores
  • Standardized mean difference (SMD) — covariate balance check before and after matching
  • A/B test — randomized benchmark for the matching estimates
Continue reading — it's free

Google or a one-time email code. No password.