Generating Tailored Travel Recommendations

Lucy (Jingyu) Zou, Eric Song22 May 2023

Problem

A recommender tuned only for NDCG and precision can return a list of near-identical items. Expedia's recommendations team found that a price-sensitive traveler who sees five properties at almost the same price gets unhappy and stops exploring the site. Ranking metrics do not capture this variety, and users prefer to see it.

Example from the article: five recommended properties priced $150, $140, $155, $135 and $147 are too similar for a traveler who wants the best value.

Solution in brief

Measure diversity as intra-list dissimilarity: the average pairwise distance between the items in a list. Raise it in post-processing, not in training, with a greedy reranker (maximal marginal relevance, called Neighborhood Cosine Distance in the post) that adds items by a weighted mix of relevance and distance to the previous item. One weight w sets the trade-off, and the team chose this method over clustering.

Questions this case answers
  • How to increase the diversity of a ranked recommendation list at serving time without retraining the ranking model or losing too much relevance
  • How to measure the diversity of a ranked list for any model that has item feature vectors
  • How to measure how diverse a list of recommended items is?
  • How to add diversity to a recommender without retraining the model?

Tools and methods

  • Intra-list dissimilarity — diversity metric for any list with item vectors.
  • Maximal marginal relevance (MMR) — greedy reranking with a diversity weight w.
  • k-means — alternative reranking: take items from different clusters.
  • Hotel2Vec embeddings — learned item vectors, tested and rejected for diversity.
  • Popularity baseline — demo model ranking items by clicks.
Continue reading — it's free

Google or a one-time email code. No password.