Generating Diverse Travel Recommendations

Lucy (Jingyu) Zou, Eric Song25 May 2023

Problem

Recommenders are usually judged by relevance and ranking metrics such as NDCG and precision, which do not reward variety. A price-sensitive traveler who sees five properties at nearly the same price has nothing to compare and may stop exploring. Expedia's recommendations team wanted more variety in its lists without retraining every model and without a large loss in relevance. That needs a metric for variety and a way to reorder existing model output.

Example from the article: five hotels priced at $150, $140, $155, $135 and $147 shown to a price-sensitive traveler, who may become unhappy and unwilling to explore further.

Solution in brief

Measure diversity as intra-list dissimilarity: the average pairwise distance between items in the top-k list. Raise it by reranking model output with Maximal Marginal Relevance (MMR, called the Neighborhood Cosine Distance approach in the article), where one weight balances relevance against distance. Use explicit item features rather than trained embeddings, and pick the weight from the offline diversity versus NDCG curve.

Questions this case answers
  • How to increase the variety of a ranked list as a post-processing step while keeping the loss in ranking quality under control
  • How to quantify the diversity of a ranked list of items so that different models can be compared on it
  • How to measure how diverse a recommended list is?
  • How to make a recommended list more diverse without retraining the ranking model?

Tools and methods

  • Intra-list dissimilarity — diversity metric, usable with any model that has item vectors
  • MMR reranking (Neighborhood Cosine Distance) — post-processing with a diversity weight w; the approach Expedia chose
  • K-means clustering on explicit features — alternative reranking: take items from different clusters
  • Hotel2Vec embeddings — trained property vectors, rejected as the diversity space
  • SciPy cdist, NumPy — pairwise distances for the metric
Continue reading — it's free

Google or a one-time email code. No password.