Problem
A ranker that scores each listing by its own booking probability fills every position with the choice most guests made in the past. In Airbnb search this means a page of cheap, similar listings. A minority of guests prefers quality: in Rome, for two guests and two nights, about 20% of bookings carry 50% of booking value, and for them cheaper is not better. These guests are served worse, and the page offers little choice to anyone.
Example from the article: in the Rome two-night, two-guest segment, booked prices follow a log-normal distribution, yet the ranker follows the majority preference at every position, so economical listings crowd out quality ones.
Solution in brief
Train a second neural network that estimates how similar a listing is to the listings already placed above it. Subtract this similarity from the booking-probability score and build the result page greedily, one position at a time. Similar listings drop down, so quality listings enter the top positions.
Questions this case answers
- How to rebuild a ranked result list so it covers the varied preferences of a population when a pointwise relevance score favors the majority at every position
- How to learn item-to-item redundancy from interaction logs without similarity labels
- How to stop a search ranking from filling every position with near-identical listings
- How to account for similarity between results when the ranker scores each item on its own
Tools and methods
- Pairwise ranking network — existing model that predicts which of two listings is more likely to be booked.
- Similarity network — companion model that scores similarity between a candidate and an already placed listing.
- Greedy sequential re-ranking — fills each position with the best similarity-discounted listing.
- Online A/B test — measures bookings, booking value and 5-star ratings.
Google or a one-time email code. No password.