Problem
A marketplace that wants to stop scammers has to run a machine learning model in production. Features computed for training differ from those computed at serving. Labels come from several sources and get aggregated differently in dashboards and in training sets. Scammers also adapt quickly, so old features describe old fraud patterns, and a slow or fragile scoring service can damage the trip publication and booking funnels.
Example from the article: BlaBlaCar scores every trip publication and booking event in near real time, so a suspicious driver is scored and handled before the page after the publish click is shown.
Solution in brief
Score events from a Kafka stream outside the critical path of the product. Write every serving feature into the fraud score event and train only on that log, so training and serving cannot diverge. Build one consolidated label table per member, used by both reporting and dataset building.
Questions this case answers
- How to guarantee that training features match serving features when scores must be computed in near real time and the adversary changes behavior quickly
- How to build one consistent ground-truth label per entity when labels come from several sources of different quality and purpose
- How to make sure the features a fraud model trains on are the same as those it sees in production?
- How to score every trip publication and booking for fraud without slowing the product down?
Tools and methods
- Apache Kafka — carries publication and booking events and the resulting fraud score events
- Bigtable-based feature store — key-value store with historized member and ride data, a projection of Kafka events
- XGBoost / scikit-learn — models that take the encoded feature array
- Serving-feature log — the encoded input array stored in the score event and used as the training set
- Consolidated label table — one value per member from all label sources, refreshed daily
Google or a one-time email code. No password.
