Problem
eBay’s “Similar Sponsored Items” strip on an item page recommends items similar to the page’s main item, called the seed. The ranker behind it was a gradient boosted tree trained as learning-to-rank. Its most important features were popularity features, so popular items outranked relevant items without sales history, such as new listings. So the recommended items sometimes did not match the seed.
Example from the article: for a $14,500 luxury wristwatch, the top three recommended watches differed from the seed in title, appearance, brand and price range.
Solution in brief
Put a relevance model in front of the existing ranker. It is trained without popularity features and with a loss that gives more weight to purchases of items similar to the seed. Only its top-k candidates go to the original ranker, and k is tuned per marketplace and per high-traffic category with online tests.
Questions this case answers
- How to keep a popularity-driven ranker from surfacing candidates that do not match the anchor item, without replacing the ranker
- How to choose a per-segment candidate cutoff for a relevance filter placed before a ranker when segments differ in catalog size
- How to stop a conversion-trained ranker from showing popular but dissimilar items in a similar-items strip?
- How to make recommendations follow the seed item without losing purchase conversion?
Tools and methods
- Relevance cascade (two-stage recommender) — a relevance filter keeps the top-k candidates, then the original ranker orders them.
- Gradient boosted trees — the family used by the original learning-to-rank model.
- Custom objective function — cross entropy loss with the similarity score built into the purchase term.
- Online A/B test — compares the model without a filter against several k values, and the final model against the old one.
Google or a one-time email code. No password.