Problem
Netflix sends hundreds of millions of personalized push messages, emails and in-app alerts. Its previous system used a causal model of one message's effect over a short horizon, so it missed fatigue and opt-outs that appear weeks later. Frequency was a by-product of daily decisions: a relevance threshold on the model score, calibrated to a target aggregate send rate. Moving that threshold to change frequency also changed which messages were chosen.
Example from the article: a message that drives a click today may also make the member less responsive in the following weeks, and a single-message model cannot see that cost.
Solution in brief
Split the system into two policies. A slow policy picks a personal weekly pacing plan (push and email frequency) by maximizing a utility that trades engagement against messaging cost. A fast policy picks which message to send at each send opportunity, inside that plan. The plan travels between them through a feature store.
Questions this case answers
- How to control the long-run contact frequency of each user explicitly when a per-event send model only sees short-horizon rewards
- How to split a coupled send-or-not and what-to-send decision into two layers that can be tuned and A/B tested independently
- How to set a personal weekly notification frequency for each member instead of getting frequency as a by-product of per-message decisions?
- How to stop a notification model that scores single messages from over-sending when explicit negative feedback is rare?
Tools and methods
- Slow policy (planner) — weekly, picks each member's pacing plan from about O(100) push and email frequency combinations.
- Fast policy (executor) — reads the plan at each send opportunity and picks the message.
- Utility with a universal message cost — scores plan actions; the cost term stops "always send".
- Feature store — carries the plan from slow to fast policy.
Google or a one-time email code. No password.