Problem
A marketplace search team ran a gradient-boosted decision tree (GBDT) ranker. Trees depend on manually engineered features and accept few input types, so relevance gains plateaued even with stronger features. A neural ranker can take embeddings directly, but migrating a production tree model means changing the training pipeline and the serving system. The baseline keeps improving meanwhile, so the gap to close grows.
Example from the article: Etsy spent one year shipping its first unified neural ranker for the second-pass layer, after dropping a tree plus neural ensemble.
Solution in brief
Skip the intermediate tree plus neural ensemble and train a single neural ranking model. Port the tree's engineered features, normalize them with log1p, add embeddings trained on your own query and listing text, and lengthen the training window. The first online test served mainly as a serving check; a later test with all features available reached parity with the tree on web and beat it on mobile.
Questions this case answers
- How to migrate a production ranking model from hand-engineered features and trees to a neural network when the existing serving stack and the moving baseline make the migration costly
- How to decide between an intermediate two-model ensemble and a direct switch to a single model when the ensemble doubles feature work and serving cost
- How to replace a gradient-boosted tree search ranker with a neural ranking model without losing online performance?
- How to decide whether to ship a tree plus neural ensemble or go straight to a single neural ranker?
Tools and methods
- TF Ranking — TensorFlow's learning-to-rank library with ready losses and metrics
- Gradient-boosted decision trees — the production baseline and the source of engineered features
- Custom text embeddings — trained on Etsy queries and listings, used directly as model inputs
- Kubeflow Pipelines — the redesigned ML pipeline, with Google Cloud services
- GPU training — cut training time to one eighth with 8 GPUs
Google or a one-time email code. No password.