Problem
PayPal needs to know what an action is worth, such as adopting a product or adding a credit card to the wallet. The direct profit of such an action can be $0, yet the user may start generating more profit elsewhere in the ecosystem. Product teams need one comparable number for 40+ products to size campaigns, rank products in the app and decide on ramp-ups. Users adopt on their own, so there is no randomized control, and customer lifetime value (CLV) does not show what changes after adoption.
Example from the article: if the average Delta CV for adopting Crypto is $20, customers who adopt Crypto for the first time bring an additional $20 of margin on average over the next 12 months.
Solution in brief
Define Delta CV as a customer's incremental profit margin in the first year after adoption. Per product and quarter, match each first-time adopter to up to 10 never-adopters on 12 months of pre-adoption transactional features, always including pre-period CV. The margin gap over the following 12 months is the lift.
Questions this case answers
- How to estimate the incremental long-term value of a user action when it cannot be randomized and its direct value is zero or indirect
- How to construct a counterfactual group from never-treated units by matching on pre-period outcome history
- How to measure what a product adoption or user action is worth when its direct profit is zero?
- How to estimate the incremental one-year profit lift from users adopting a product when an A/B test is not possible?
Tools and methods
- Synthetic control — counterfactual group built from never-adopters
- K-nearest-neighbors matching — picks up to 10 controls per adopter by Euclidean distance, with a distance cut-off
- Bias analysis — checks that the matched control resembles the treatment group
Google or a one-time email code. No password.