Problem
Whatnot is a live shopping platform. Shows are scheduled ahead and last from 20 minutes to 2 hours, sometimes several days. Users want entertainment, deals, a specific product or a favorite host. The team wants to get each user to a host they will enjoy with little trial and error. Live hosts and schedules change constantly, so the usual offline ranking metrics were not robust over consecutive daily runs.
Example from the article: a user who started with sneakers made 4 purchases across 3 sellers and ended up buying vintage clothing. Their embedding moved toward the vintage cluster, and the recommendations followed.
Solution in brief
Learn user and host embeddings from implicit engagement with Bayesian Personalized Ranking (BPR) or its variant WARP, and rank hosts by dot product. Validate offline with ROC AUC rather than top-K metrics. To control drift, inspect purchase trajectories and re-rank with nearest-neighbor voting, a fitted line or Thompson sampling.
Questions this case answers
- How to train a first personalized ranker from implicit feedback when the set of rankable items differs between training and serving
- How to choose an offline ranking metric that stays stable and predicts online behavior when the catalog turns over daily
- How to keep embedding-based user preferences from drifting after a few atypical interactions while still allowing exploration
- How to build a first personalized recommender for live shows from implicit user engagement with hosts?
Tools and methods
- Matrix factorization (BPR, WARP) — user and host embeddings from implicit engagement
- ROC AUC — offline metric chosen as the proxy for online performance
- implicit, LightFM — libraries with ready BPR and WARP implementations named by the author
- UMAP — 2D projection of host embeddings to inspect a user's trajectory
- Nearest-neighbor voting, trajectory fit, Thompson sampling — guard rails against preference drift
Google or a one-time email code. No password.