From RankNet to LambdaMART: Leveraging XGBoost for Enhanced Ranking Models

Enderson Santos10 Feb 2025

Problem

A marketplace search shows a list of ads for a query, and someone has to decide their order. A relevance label on an ad means little alone: an iPhone ad and a car ad can both be "relevant" in training data, yet only one fits a given query. Relevance also depends on neighbors, because an ad that looks weak next to a better one may look fine alone. A plain classifier on ad features ignores both the query and the comparison, and does not care where the relevant ads land in the list.

Example from the article: two ads, an iPhone and a car, both labeled 1, but each is relevant only to the person searching for it.

Solution in brief

Group the training data by session, train a pairwise ranker in XGBoost with "objective": "rank:pairwise" (RankNet), and score the result with NDCG. To weight the top of the list, switch to "objective": "rank:ndcg" (LambdaRank-style), which scales each pair's gradient by the NDCG change that swapping the pair would cause.

Questions this case answers
  • How to train a ranker for a list of items shown together when relevance only makes sense relative to the other items in the same list
  • How to optimize a ranking model for a top-weighted quality metric when the metric itself is not differentiable
  • How to train a model that orders ads inside one search listing instead of classifying each ad as relevant or not?
  • How to build a training table for a ranker from sessions and user clicks?

Tools and methods

  • XGBoost (XGBRanker) — gradient-boosted trees with ranking objectives
  • RankNet (rank:pairwise) — minimizes the number of wrongly ordered ad pairs per session
  • LambdaRank (rank:ndcg) — pairwise gradients weighted by the NDCG change
  • NDCG@k — ranking quality metric, averaged over listings
  • scikit-learn ndcg_score — computes NDCG per listing
Continue reading — it's free

Google or a one-time email code. No password.