Problem
Expedia Group runs a "similar properties" carousel on the hotel details page. A traveler who dislikes the viewed hotel can find alternatives without going back to the search results page. The first model was trained and evaluated on next-click prediction, and online the team tracked CTR and CVR for the placement. Then the online correlation between CTR and conversion rate (CVR) turned out slightly negative. So a model that wins on clicks can leave travelers exploring longer without booking.
Example from the article: traveler A clicks a hotel, wants similar available properties, and sees them in the carousel on the same page instead of restarting the search.
Solution in brief
Move the training target from clicks toward bookings in steps. The team moved from click co-occurrence to a booking-oriented ranker with search context, then added features from the traveler's historical clicks, and finally trained a multi-task model (pAction) on clicks, long clicks and bookings.
Questions this case answers
- How to pick the optimization target of a ranking model when the cheap engagement signal (clicks) stops predicting the business outcome (purchases)
- How to add a mid-funnel engagement label to a ranker so it learns from sparse final conversions
- How to choose the training target of a recommender so that it drives bookings and not only clicks?
- How to check whether CTR is a good proxy for conversion on a recommendation placement?
Tools and methods
- Item-to-item interaction matrix — baseline from co-clicks of hotel pairs in sessions
- Matrix factorization — tested as an alternative to the matrix, scored lower offline
- XGBoost and LightGBM — tree models compared for the booking-oriented ranker
- LightGBM ranker with pairwise loss — contextualized model trained on bookings
- Multi-task model pAction — shared model for clicks, long clicks and bookings
Google or a one-time email code. No password.