Problem
The New York Times lets non-subscribers read a fixed number of free articles per month (the meter limit). Originally the limit was the same for everyone. A tighter limit gives more subscriptions but cuts readership, and a looser one does the opposite, so the limit for each registered user has to balance the two. The catch: nobody sees what the same user would have done under another limit.
Example from the article: In a randomized control trial (R.C.T.) with different meter limits for registered users, a higher limit raised average page views and lowered the subscription conversion rate.
Solution in brief
The Dynamic Meter, a causal model, trains two S-learner-style models on R.C.T. data, one for subscription propensity and one for engagement. A weight blends them into one score, and each user gets the meter limit with the best score. Sweeping gives a Pareto front, which is evaluated offline by inverse probability weighting on R.C.T. data.
Questions this case answers
- How to assign each user one of a few discrete treatments so that two conflicting outcomes are traded off explicitly, when the only unbiased training data come from a randomized experiment
- How to backtest a treatment-assignment policy on historical randomized data when only users whose random assignment matches the policy's choice are observable under that policy
- How to choose a personalized free-article limit for each registered user instead of one limit for everyone?
- How to balance subscription conversion against reader engagement when tighter limits raise one and cut the other?
Tools and methods
- S-learner base models — predict subscription propensity and normalized engagement from user features plus the meter limit as a categorical treatment
- Pareto front over weight — blends the two predictions into one objective; sweeping from 0 to 1 gives the set of solutions
- Inverse probability weighting with Hájek estimation — backtests the policy on past R.C.T. data
Google or a one-time email code. No password.