Estimating Incremental Lift in Customer Value (Delta CV) using Synthetic Control

Mahshid Moha14 Feb 2025

Problem

PayPal needs to know what an action is worth, such as adopting a product or adding a credit card to the wallet. The direct profit of such an action can be $0, yet the user may start generating more profit elsewhere in the ecosystem. Product teams need one comparable number for 40+ products to size campaigns, rank products in the app and decide on ramp-ups. Users adopt on their own, so there is no randomized control, and customer lifetime value (CLV) does not show what changes after adoption.

Example from the article: if the average Delta CV for adopting Crypto is $20, customers who adopt Crypto for the first time bring an additional $20 of margin on average over the next 12 months.

Solution in brief

Define Delta CV as a customer's incremental profit margin in the first year after adoption. Per product and quarter, match each first-time adopter to up to 10 never-adopters on 12 months of pre-adoption transactional features, always including pre-period CV. The margin gap over the following 12 months is the lift.

Questions this case answers
  • How to estimate the incremental long-term value of a user action when it cannot be randomized and its direct value is zero or indirect
  • How to construct a counterfactual group from never-treated units by matching on pre-period outcome history
  • How to measure what a product adoption or user action is worth when its direct profit is zero?
  • How to estimate the incremental one-year profit lift from users adopting a product when an A/B test is not possible?

Tools and methods

  • Synthetic control — counterfactual group built from never-adopters
  • K-nearest-neighbors matching — picks up to 10 controls per adopter by Euclidean distance, with a distance cut-off
  • Bias analysis — checks that the matched control resembles the treatment group
Continue reading — it's free

Google or a one-time email code. No password.