The New User Dilemma: Why We Killed Our Heuristics (& What We Built Instead)

Miguel Fernandez-Montes Cuberta, David Kawashima, Ford Bohrmann3 Sep 2025

Problem

Whatnot is a live-shopping marketplace with a personalized For You Feed. Regular users get a feed ranked by models that rely on past interactions. New users have none, so a separate lookalike heuristic ranked their feed. It gave acceptable results, but it was hard to understand, needed its own backend path, jobs and datasets, and could not learn from data. New signups are a critical segment, so the team wanted their ranking on par with the ranking for mature users.

Example from the article: a user signs up, says in onboarding that they like Pokemon cards and sneakers, and the first For You Feed must already fit those interests.

Solution in brief

Drop the separate new-user path. Train one global GBDT ranking model on all users and add the onboarding answers as features that the client sends with each request. Build the training data so those request features carry the values that were true at the time of each training example, using database change data capture (CDC) timestamps.

Questions this case answers
  • How to produce personalized rankings for entities with no behavioral history while keeping one ranking path for all entities
  • How to guarantee point-in-time consistency between features supplied in the request at serving time and the same features in offline training data
  • How to rank a feed for new users who have no interaction history without keeping a separate heuristic pipeline?
  • How to decide between a dedicated new-user model and one global ranking model?

Tools and methods

  • Gradient boosted decision trees (GBDT) — one ranking model for all tenures
  • Request-time features — onboarding choices sent by the client
  • Feature store (Chalk) — aggregated features, point-in-time correct for training
  • Change data capture (CDC) to the warehouse — timestamped history of request features for training
  • A/B test — compares the new ranking against the old heuristic
Continue reading — it's free

Google or a one-time email code. No password.