Next Best Action platform: democratizing personalization with contextual bandits

Zillow9 Dec 2024

Problem

Product and marketing teams at Zillow choose messages and calls to action (CTAs) by A/B testing and then ship the winner to everyone. If a message wins with 60% of customers, the other 40% get a message that does not fit them. Sequential tests also take weeks or months, and every new variant risks hurting engagement until it is tested. Hand-tuned rules and static models go stale as customers and business priorities change.

Example from the article: the four "Reason to Believe" tiles on the Zillow homepage (Home Loan Prequalification, Buy with Agent, Rent, Sell) were shown to every user in the same order, whatever their history.

Solution in brief

Zillow treats each tile as an action in a contextual bandit. A gradient-boosting regressor predicts the reward for each user and action. An ε-greedy rule mixes in random exploration, and the model is refit daily on recent feedback. The same loop covers cold start, new actions and changes of reward weights.

Questions this case answers
  • How to choose per user which of a few candidate actions to show first when a single global winner leaves part of the audience underserved
  • How to introduce new candidate actions into a live ranking and learn their value without a separate exploration phase or a loss in total reward
  • How to let non-technical teams change the objective weights of a ranking policy without retraining from scratch or rerunning experiments
  • How to personalize the order of calls to action and messages instead of shipping one A/B-test winner to everyone?

Tools and methods

  • Contextual bandit — chooses an action per user from context and keeps exploring.
  • Gradient-boosting tree regression — base model mapping user and action features to expected reward.
  • ε-greedy exploration — random action with probability ε, best predicted action otherwise.
  • Daily batch retraining on a sliding window — keeps the model current with fresh feedback.
  • Offline batch pipeline plus online decision store — decisions are precomputed and served on request.
Continue reading — it's free

Google or a one-time email code. No password.