Method
Interleaving
Online comparison of rankers by mixing their results in one list.
9 cases from 9 companies apply Interleaving. 6 were checked in an online A/B test; 2 report results in production.
Typical tasks: Get experiment results faster, Evaluate models, Rank search results, Retrieve and rank from millions of items and 3 more. Companies: Expedia Group, DoorDash, Algolia, Thumbtack, Canva, Airbnb and 3 more. Often combined with: A/B test, t-test, Bootstrap, A/A test, Query autocomplete and 14 more.
| How Vinted Serves Personalised Search Autocomplete | marketplace | 2026-04 | |
| Interleaving for Accelerated Testing | travel | 2026-02 | |
| How DoorDash is pushing experimentation boundaries with interleaving designs | food-delivery | 2024-08 | |
| Conference Recap: ECIR23 Take-aways | media-streaming | 2023-08 | |
| Accelerating Ranking Experimentation at Thumbtack with Interleaving | marketplace | 2023-07 | |
| Reliable ML: Model Robustness After Production Rollout | general | 2023-05 | |
| Search Pipeline: Part II | general | 2022-11 | |
| Beyond A/B test : Speeding up Airbnb Search Ranking Experimentation through Interleaving | travel | 2022-10 | |
| Faster ML Experimentation at Etsy with Interleaving | e-commerce | 2022-08 |