Method

Stratified sampling / assignment

Stratified splitting or sampling of units, users or items.

18 cases from 14 companies apply Stratified sampling / assignment. 3 were checked in an online A/B test; 9 report results in production.

Typical tasks: Evaluate models, Get experiment results faster, Catch fraud and abuse, Optimize decisions and 10 more. Companies: Grab, LinkedIn, Airbnb, Avito, Pinterest, Yandex and 8 more. Often combined with: Gradient boosting, Power analysis / MDE, LLM-as-a-judge, Holdout / global control group, LoRA fine-tuning and 47 more.

Automated merchant review summary system with integrated feedbackfood-delivery2026-09
From weeks to a day: how we made LLM evaluation fast enough to iterate ontravel2026-07
The 52x Multiplier: How Zepto Mastered AI Agent Evaluation at Scalee-commerce2026-06
ML Scoring of 3DS Authentications in a Payment Systemfintech2026-05
Solving the Task: Life Hacks from the Task Author (AvitoTech ML Cup 2026)classifieds2026-05
LLM-Powered Relevance Assessment for Pinterest Searchsocial2025-12
A Data-Driven Approach to Subscriber Scoring in Telecomtelecom2025-05
ML-Readability: How Yandex Browser Extracts Web Page Content for Summarizationsearch-engine2025-05
How Duolingo sped up data analysis by 68X with samplingedtech2025-05
Contextual Offer Recommendations Engine at Targetretail2024-08
Deep Learning Clinical Decision Support Systems Reduce Time and Improve Radiologists' Accuracy in Routine CT Interpretationhealthtech2024-05
How we evaluated the business impact of marketing campaignsmobility2024-05
Grab Experiment Decision Engine - a Unified Toolkit for Experimentationmarketplace2024-04
Beyond global sentiment: object, aspect sentiments—general2023-05
Measuring marketing incremental impacts beyond last click attributionsocial2022-07
Representative Sampling: A Statistical Method to Derive Data Insights from Noisy Data Distributionsretail2022-03
Balancing Network Effects, Learning Effects, and Power in Experimentsfood-delivery2022-02
How our content abuse defense systems work to keep members safesocial2022-01