Problem
A ride-hailing and delivery app sometimes cannot collect payment for a completed ride: the card is empty, stolen or counterfeit. The company can reserve part of the fare before the ride (pre-authorization), but doing it for everyone hurts the experience. Some customers read it as distrust, and a hold can last one to three weeks. So each order needs a decision: allow, deny or pre-authorize. Hand-written rules are hard to maintain, give only yes/no answers and are easy for fraudsters to reverse-engineer.
Example from the article: a rule blocked orders when a feature was below a threshold . After a couple of weeks fraud rose sharply because fraudsters had found the threshold and started acting at .
Solution in brief
Replace most rules with a gradient boosting model that scores every order in real time and feeds the score into the existing rule platform as a feature. Thresholds per country and city pick the action. Validate on sliding time windows, run in shadow mode first, and monitor feature distributions.
Questions this case answers
- How to replace hand-written allow, deny or hold rules with a real-time probability score, so that one threshold sets the trade-off for a fixed recall target
- How to roll out a real-time risk model safely when adversaries adapt to the decision threshold and feature distributions shift after an attack
- How to replace rule-based pre-authorization fraud checks with a real-time ML model
- How to choose a decision threshold for a fraud model when the business demands a fixed recall
Tools and methods
- CatBoost — about 20 hand-built features
- Sliding time validation — windows move forward; also used for backtesting
- Shadow mode and gradual rollout — log first, then ramp 5% to 50% to 100%
- Rule platform — takes the model score as a feature; thresholds and actions by country and city
- Distribution monitoring — in-house anomaly detector on features and predictions
Google or a one-time email code. No password.