Problem
A card fraud model has to decide in real time, while the cardholder waits at checkout, whether a purchase is fraud or routine spending. The same $1,400 electronics charge is normal for an IT manager and strange for someone who only buys lunch, rideshares, software and subscriptions. Counting past events per card cannot separate these cases: an unfamiliar hotel and an unfamiliar gift card look equally new, and counts do not say what similar cardholders usually do next. Fraud labels are rare and arrive weeks later, so the history model has to learn without them.
Example from the article: a lunch-and-rideshare cardholder suddenly pays $1,400 at an electronics store.
Solution in brief
Pre-train a small transformer on two years of transactions to predict the next one, so it learns normal spending without fraud labels. Feed its embedding, per-field surprisal scores and swap scores into the existing XGBoost model. Train XGBoost with a dollar-weighted loss and ship it all as one CPU package.
Questions this case answers
- How to judge whether an event is anomalous for one entity when counting past events cannot say how likely a never-seen value is
- How to learn a behavior model from an entity's event history when labels are rare and arrive weeks late
- How to embed a very large, changing categorical vocabulary with a small model that can be retrained often
- How to tell whether a card purchase is fraud or the cardholder's normal spending, using the cardholder's own recent history?
Tools and methods
- Small transformer — reads the card's recent transactions and predicts the next one.
- Merchant tokenization — own vectors, hashed shared slots and category fallbacks.
- Surprisal and swap scores — per-field measures of how unusual the purchase is for this person.
- XGBoost — final fraud scorer over old features, counts and transformer outputs.
- Feature Platform — one history definition for training and streaming serving.
Google or a one-time email code. No password.