Method

Data drift monitoring

PSI, KS and distribution checks on features and predictions.

32 cases from 24 companies apply Data drift monitoring. 4 were checked in an online A/B test; 21 report results in production. Includes variants: Population Stability Index.

Typical tasks: Detect anomalies, Evaluate models, Catch fraud and abuse, Optimize decisions and 11 more. Companies: Pinterest, Lyft, Alfa-Bank, Airbnb, Atlassian, DoorDash and 18 more. Often combined with: Population Stability Index, Gradient boosting, Logistic regression, Threshold selection, A/B test and 74 more.

How Hilton Grand Vacations Optimizes Inventory with AI on Databrickstravel2026-06
ML Scoring of 3DS Authentications in a Payment Systemfintech2026-05
When Metrics Lie: Countering Domain Shift in MRIhealthtech2026-05
What COVID did to our forecasting models (and what we built to handle the next shock)travel2026-03
How Pinterest Built a Real‑Time Radar for Violative Content using AIsocial2025-12
Comment ranker - An ML-based classifier to improve LLM code review quality using Atlassian's proprietary datasaas-b2b2025-08
A Data-Driven Approach to Subscriber Scoring in Telecomtelecom2025-05
How to investigate the online vs offline performance for DNN modelsfood-delivery2024-12
ETA (Estimated Time of Arrival) Reliability at Lyftmobility2024-06
Real-Time Pre-Authorization for Taxi Orders: From Rules to Machine Learningmobility2024-05
Scoring on Non-Bank Data—fintech2024-05
Monitoring Machine Learning Modelstelecom2024-05
How We Launched a New Promotion Model: Matching and Regional Testsclassifieds2024-05
Incidents in ML Services and How to Monitor Themhealthtech2024-05
Handling Online-Offline Discrepancy in Pinterest Ads Ranking Systemadtech2024-01
Product Features: The Key to Increasing Model Efficiency for Account Propensitysaas-b2b2023-10
Data Generation for Information Security Tasksgeneral2023-10
Let AI Entertain You: Increasing User Engagement with Generative AI and Rejection Samplingsocial2023-10
Training Foundation Improvements for Closeup Recommendation Rankersocial2023-09
NLP at Alfa-Bank, Part 4: The Development History of a Chatbotbanking2023-05
How to Build Real-Time ML at a Crypto Exchangefintech2023-05
Reliable ML: Model Robustness After Production Rolloutgeneral2023-05
Warden: Real Time Anomaly Detection at Pinterestsocial2023-05
Building a large scale unsupervised model anomaly detection system — Part 2mobility2023-04
Automated profile picture moderation at BlaBlaCar using Deep Learningmobility2023-03
D3: An Automated System to Detect Data Driftsmobility2023-02
Recommend API: one shared pipeline for many recommendation features at Slacksaas-b2b2022-09
Feature Investigation: Automatically Detect Drift in your Data Over Timefintech2022-08
DL in finance: key results, production and new challengesbanking2022-06
What is new in ML model interpretabilitygeneral2022-06
Full-Spectrum ML Model Monitoring at Lyftmobility2022-06
Machine Learning for Snapchat Ad Rankingadtech2022-02