How to Build Real-Time ML at a Crypto Exchange

Andrey Denisenko27 May 2023

Problem

A large crypto exchange, fully remote and growing about 4x a year at the time, had no ML practice and scattered data. Fraud is direct revenue loss, and some of it must be refunded. Scam victims pushed to buy crypto and send it out make up about 80% of losses; fake accounts and "drop" accounts of real people working for others cover the rest. A rule set caught only about 30% of fraud, and the data platform got production data a day late, so a model there could not decide in real time.

Example from the article: a fake account opened on someone else's documents deposits money from stolen cards and withdraws it, and the exchange compensates the loss.

Solution in brief

Split the customer lifecycle into events (onboarding, funding, trading, withdrawal) and compute a score per fraud type at each event, inside the production loop. A Rust service fetches data, caches it and computes features. Each release passes an out-of-time backtest and a 3-4 week shadow test checked by the business.

Questions this case answers
  • How to score each event in a live decision loop with a per-type risk model when the analytics environment lags production by a day
  • How to release a risk classifier safely when only out-of-time backtests and a shadow run are available before it can block real events
  • How to score users in real time at onboarding, deposit, trading and withdrawal to separate fraudsters from legitimate customers
  • How to put a fraud model into the production loop when the data platform lags production by a day

Tools and methods

  • Event-driven scoring — one score per fraud type and event
  • Rust and C++ serving library, LightGBM, Redis — data access, cache, features, model
  • Kafka, S3, Airflow — events, offline data, scheduled retraining
  • Out-of-time backtest and shadow test — two-stage release
  • Population stability index — drift monitoring
Continue reading — it's free

Google or a one-time email code. No password.