eBay’s Blazingly Fast Billion-Scale Vector Similarity Engine

Santosh Shahane, Kui Yang, Leonard Dahlmann, Amarendra Belineni, Yan Sun, Shrivatsan Navalpakkam Rajagopalan, Pavel Petrushkov1 May 2023

Problem

A marketplace needs to show buyers listings similar to those they viewed and to personalize recommendations from clicks, views and add-to-cart events. Exact k-nearest-neighbor search is too slow when answers are needed fast. Keyword retrieval struggles with homonyms, long-tail queries and users who do not know the category jargon. eBay has about 134 million users and 1.7 billion live listings (December 2022), and listings change constantly. Query-time filters (category, location, item attributes) are needed for relevance but are hard to combine with vector search.

Example from the article: given an input listing, find the most similar listings by title, aspects and image, or generate recommendations from a user's past browsing.

Solution in brief

eBay built a shared similarity engine: embeddings are split into shards, each shard gets an HNSW or ScaNN index, and a gateway queries all shards and merges the results. Near-real-time updates, a control plane for shard placement and filter-aware graph search keep it fresh and relevant at scale.

Questions this case answers
  • How to serve low-latency nearest-neighbor retrieval over a billion-scale, constantly changing embedding corpus with query-time attribute filters
  • How to choose an approximate nearest-neighbor index type when build time, memory, recall and update behavior all trade off
  • How to serve nearest-neighbor search over 1.7 billion item embeddings with low latency?
  • How to keep a vector index fresh when listings are added, changed and removed all the time?

Tools and methods

  • HNSW (hnswlib, eBay's modified version) — graph-based ANN backend.
  • ScaNN — Google's ANN backend using product quantization and partitioning.
  • Sharded index with replicas — scales memory and traffic, gives fault tolerance.
  • Inline attribute-based filtering — applies filters during graph search, not after.
  • Brute-force fallback — handles very restrictive filters.
Continue reading — it's free

Google or a one-time email code. No password.