Building a Deep Learning Based Retrieval System for Personalized Recommendations

Yuri M. Brovman1 Mar 2022

Problem

A marketplace with about 152 million users and 1.5 billion live listings needs an "items based on your recent views" module. Showing items similar to recently viewed ones is simple, but it builds no user representation and cannot learn that "red dress" and "scarlet dress" are close. A deep-learning model fixes that, but it needs an index, an embedding pipeline and low-latency serving, a multi-year, multi-team project if built at once.

Example from the article: eBay's first system ran offline, so a click reached the module maybe one or two days later.

Solution in brief

Train a two-tower model (one tower for items, one for users) and ship the serving system in three phases. Phase 1 computes everything offline and caches results per user. Phase 2 moves nearest-neighbor search to a real-time service. Phase 3 computes the user embedding from a click stream, so the delay drops to seconds.

Questions this case answers
  • How to turn a sequence of recent user events into candidate items under a strict latency budget when item and user embeddings must be refreshed continuously
  • How to stage the build of an embedding-based retrieval stack so that each phase ships value before the real-time infrastructure exists
  • How to generate personalized item recommendations from a user's click history when there is no seed item?
  • How to move a two-tower recommender from daily batch scoring to near real-time without building everything at once?

Tools and methods

  • Two-tower model (PyTorch, PyTorch Lightning) — item and user towers map both to one embedding space
  • FAISS, HNSW, ScaNN — exact search offline, approximate nearest-neighbor service in real time
  • Apache Spark — daily ETL of item metadata and user history
  • Kafka, Flink, Triton, TorchScript, ONNX — click stream, event aggregation and GPU inference for real-time user embeddings
  • Couchbase — key-value cache with results (phase 1) or user embeddings (phases 2 and 3)
Continue reading — it's free

Google or a one-time email code. No password.