Problem
Yandex Plus, a subscription with more than 40 million users, shows a chain of screens when a user taps "Cancel". Some screens only describe what the subscription includes. Others cost the company money, and the Winback screen offers one to three months of a very cheap or free subscription. The business kept adding screens until clicking through them became hard, and the cancellation flow itself was too complicated. The team had to cut the flow to four screens, choose them per user, and not lose retention.
Example from the article: a recommender-style model scored each screen by its own save probability and put the high-converting Winback screen first. In the experiment against the existing business logic, significantly more people canceled.
Solution in brief
Score the whole sequence, not each screen. A classifier predicts the probability of staying on a given screen at a given position, and these probabilities combine into the chance that the user cancels after all four screens. A greedy pass picks four candidate screens, and the best order among them is found by enumeration.
Questions this case answers
- How to order a short fixed-length sequence of offers so that the probability that the user takes none of them is minimized when each offer's conversion depends on its position
- How to rank options when the target is the conversion of the whole sequence rather than of each option, and the best sequence must be found without scoring every permutation
- How to order retention screens in a subscription cancellation flow to maximize saves within a four-screen limit
- How to personalize the sequence of retention offers shown to a user who taps Cancel
Tools and methods
- Binary classifier on (user, screen, position) — predicts the probability that the user stays on this screen at this position
- Sequence-level cancellation probability — product of per-position "not retained" probabilities
- Greedy position-by-position selection — shortlists four screens
- Permutation enumeration — picks the best order among the four
- Local search — extension for long sequences
Google or a one-time email code. No password.
