Distilling Long-Tail User Behavior into Scalable Embeddings for Job Search

Marsan Ma, Nikhil Lopes, Raj Amrit, Hong Lu, Dipankar Biswas, Trent Kyono3 Jun 2026

Problem

Production ranking and recommendation models at a large job marketplace are compact, supervised and must score huge candidate sets within millisecond budgets. They cannot read a full user history, so teams keep the last K actions, aggregate the rest into statistics, or use sparse one-hot features that lose similarity between titles, skills and industries. These shortcuts distort intent. Each team also rebuilds its own behavior features from scratch.

Example from the article: a job seeker moved from civil engineering to medical software trainer roles, then to senior account management. Aggregating title tokens over-weights "software" and pushes software engineering jobs, although recent behavior points elsewhere.

Solution in brief

Indeed trains a heavy sequence model offline, the user behavior model (UBM), that turns a long history into one fixed-length user embedding. Daily batch inference writes the embeddings to a feature store. Existing online models take the embedding as one extra dense feature, so they are retrained but not rewritten.

Questions this case answers
  • How to exploit long, noisy interaction histories in latency-bound ranking models that cannot run sequence models online
  • How to share one learned user representation across many heterogeneous downstream models with minimal serving changes
  • How to use months of long-term user behavior in ranking models that must answer within milliseconds?
  • How to give many existing production models the same view of user intent without rewriting them as sequence models?

Tools and methods

  • Job encoder (embeddings + Deep & Cross Network) — one vector per job
  • Transformer encoder with self-attention — down-weights off-topic events
  • Unified event timeline with event-type embeddings — one sequence for all actions
  • Feature store — serves daily user embeddings to downstream models
  • Cached GPU lookups of job attributes — avoids row-level joins in training
Continue reading — it's free

Google or a one-time email code. No password.