Problem
Online payment fraud scoring must be accurate, fast and cheap per transaction. It also has to avoid blocking good payments, which hurts buyers and merchants. Fraud is rare, about 1 in 1,000 payments, and attack patterns keep changing. Until mid-2022 Stripe's Radar used an ensemble of XGBoost and a deep neural network. XGBoost parallelizes poorly, so retraining was slow and the team could not try newer techniques such as transfer learning or embeddings.
Example from the article: simply removing the XGBoost part of the ensemble would have cost 1.5% of recall.
Solution in brief
Replace the ensemble with one deep network that has several parallel branches (a ResNeXt-style design), which kept the memorization the XGBoost part provided. Training became over 85% faster at no stated loss of quality, so the team could test more features, and 10x more training data raised recall further. In parallel, merchants see the features behind each decision.
Questions this case answers
- How to move a production scoring model from a tree-ensemble hybrid to a single neural network without a drop in recall on a rare positive class
- How to keep a scoring model improving when the adversary changes behavior, using attack reviews to generate new features
- How to give users reasons for decisions of a black-box scoring model
- How to replace a gradient-boosted fraud model with a neural network without losing recall?
Tools and methods
- Wide & Deep ensemble (XGBoost + DNN) — the old architecture: trees memorize, the network generalizes.
- Multi-branch DNN (ResNeXt-style) — the new architecture: parallel small networks inside each block, outputs summed.
- Attack reviews and feature prototyping — the process that produces new features.
- Risk insights — a merchant-facing list of the features that drove a decision.
- Elasticsearch — fast lookup of related transactions shown next to a decline.
Google or a one-time email code. No password.