Method

Interleaving

Online comparison of rankers by mixing their results in one list.

9 cases from 9 companies apply Interleaving. 6 were checked in an online A/B test; 2 report results in production.

Typical tasks: Get experiment results faster, Evaluate models, Rank search results, Retrieve and rank from millions of items and 3 more. Companies: Expedia Group, DoorDash, Algolia, Thumbtack, Canva, Airbnb and 3 more. Often combined with: A/B test, t-test, Bootstrap, A/A test, Query autocomplete and 14 more.