ML Scoring of 3DS Authentications in a Payment System

Pavel Tsvetov23 May 2026

Problem

A payment system (the MIR card network) scores every card-not-present 3DS authentication online. It recommends to the issuing bank one of three outcomes: block, send an SMS or push (challenge), or pass without confirmation (frictionless flow). Scoring was done by a rule-based decision system (SPR), and fraudsters adapt, so rules go stale. Each SMS costs the bank money, but a wider frictionless zone is worthless if fraud inside it grows. Volume is 5–7 million authentications per day, with very little fraud.

Example from the article: in the year after launch, the frictionless zone grew by several tens of percent at an unchanged fraud rate in basis points.

Solution in brief

Train a CatBoost classifier whose positive class is a legitimate transaction ("non-fraud scoring") on a stratified sample that keeps all fraud. Convert the score to the rule system's point scale and add it to the rule score. Choose the transformation so that fraud in the frictionless zone stays acceptable.

Questions this case answers
  • How to widen the share of traffic that skips an extra verification step while keeping the bad-outcome rate inside that zone fixed
  • How to add a learned score to an existing rule-based decision score so the rules keep control and training matches inference
  • How to expand the frictionless (no-SMS) zone of 3DS authentication without raising the fraud rate
  • How to train a classifier for a very rare, late-labeled fraud target on a billion-row dataset

Tools and methods

  • Non-fraud scoring (CatBoost) — the model predicts the probability that an authentication is legitimate
  • Stratified sampling on Polars — all fraud, legitimate sampled by week, MCC, country, amount
  • YAML-serialized Polars pipelines — one pipeline for training and inference
  • Score transformation — curve into rule-score points, added to the rule score
  • Feature funnel — screening, importance and SHAP, 700+ to about 100 features
  • Drift monitoring — Kafka, ClickHouse, Evidently AI, Grafana
Continue reading — it's free

Google or a one-time email code. No password.