Modernizing Nextdoor Search Stack — Part 2

Bojan Babic20 May 2022

Problem

A neighborhood platform built more than 10 years ago, with a large store of data, needs search over several content types (verticals) at high volume. The first-stage engine returns candidates in an order that is not good enough for users. Adding a machine-learned ranker raises the cost of every query, because features must be fetched and a model run before results come back.

Example from the article: Query understanding output (device, location, time, the query and its expansions, a query embedding, intent, predicted vertical and topic) goes into the Elasticsearch query, and the ranker then reorders the returned candidates.

Solution in brief

Retrieval and ranking are separate stages. Elasticsearch, split into several indices across clusters, returns only entity ids per vertical. A ranker then scores those ids with features pulled from the real-time feature store. The ranker is LightGBM's LGBMRanker, chosen after offline and online evaluation for its accuracy and latency tradeoff. The authors cite LambdaRank and LambdaMART as the way to make training follow the ranking metric.

Questions this case answers
  • How to rank candidates from a fast first-stage retriever with a learned model while keeping query latency bounded
  • How to supply fresh per-user and per-item features to a ranking model at request time
  • How to add a machine-learned ranking stage on top of Elasticsearch retrieval without breaking latency?
  • How to serve user, document and interaction features to a ranker at query time?

Tools and methods

  • Elasticsearch — candidate recall; one query per vertical, fanned out to the clusters that hold that vertical's index
  • Real-time feature store — raw, statistical and embedding features at query time
  • Learning to rank (LTR) — learns a scoring function over feature vectors from labeled lists
  • LambdaRank / LambdaMART — adjusts the training loss using the ranking metric
  • LightGBM LGBMRanker — the production ranker
Continue reading — it's free

Google or a one-time email code. No password.