Candidate Generation Using a Two Tower Approach With Expedia Group Traveler Data

Eric Rincon12 Dec 2023

Problem

A recommender over millions of items cannot re-rank the whole catalog per request, so a retrieval stage must first cut it to a short candidate list. A two-tower model fits this stage but is trained on positive interactions only. If other positives in the batch serve as negatives, popular items appear as negatives far more often than rare ones. The model learns a popularity bias and retrieves poorly.

Example from the article: with 10 million items, re-ranking all of them per request is intractable. The authors train on an Expedia open dataset of lodging search impressions from 2021-06-01 to 2021-07-31.

Solution in brief

Train a query tower and an item tower with a softmax over the items in the batch, using the clicked pairs as positives and the other clicked items as negatives. Subtract the log of each item's frequency from its logit (logQ correction) and mask duplicate ids. Index item vectors in an approximate nearest neighbor (ANN) index and measure recall@k.

Questions this case answers
  • How to retrieve a few hundred relevant items from a catalog too large to rank in full when the only training signal is positive interactions
  • How to correct the popularity bias that appears when negatives are sampled from other positives in the same training batch
  • How to build a candidate retrieval stage for a catalog of millions of items from click logs
  • How to train a two-tower model with in-batch negatives without biasing it toward popular items

Tools and methods

  • Two-tower model — query encoder and item encoder whose same-size outputs are compared by dot product
  • In-batch sampled softmax with logQ correction — cheap negatives with popularity bias removed
  • TensorFlow Recommenders (tfrs) — Retrieval task computes the loss and applies the correction
  • PySpark with TFRecords — click filtering, item probabilities, export
  • ScaNN — ANN index over item embeddings (FAISS is an alternative)
Continue reading — it's free

Google or a one-time email code. No password.