Learning Richer Credit Representations with Transformers at Affirm

Kelli-Jean Chun17 Sep 2026

Problem

Affirm makes a separate underwriting decision at every checkout. For users with no repayment history at Affirm, the credit report from other lenders is the main signal. For years Affirm turned it into hand-built features for XGBoost models. As those models matured, each new gain required a new feature, domain expertise and slow backfilling of that feature for training.

Example from the article: two consumers each have two credit accounts and two late account-months over six months. One was late early on one account, the other recently on two accounts. A count of late payments cannot tell them apart.

Solution in brief

A transformer reads each credit report as record tokens (tradelines, inquiries, payment histories), plus selected Affirm context, and is trained on repayment outcomes. Its embeddings, taken before the prediction heads, go as extra inputs into an accelerated failure time (AFT) XGBoost model alongside the existing features.

Questions this case answers
  • How to learn useful representations from variable-length, structured record histories when hand-engineered summary features have stopped improving a tree-based model
  • How to add a learned neural representation to an established gradient-boosted model under a real-time latency limit and a per-decision explanation requirement
  • How to get more predictive signal out of raw credit-report histories than hand-built summary features give?
  • How to add a neural sequence model to a gradient-boosted underwriting score without rebuilding the existing feature pipeline?

Tools and methods

  • Transformer encoder over credit-report records — learns embeddings from tradelines, inquiries and payment-history sequences.
  • Multi-head supervised training — trains on early and late delinquency labels at the same time.
  • XGBoost (AFT form) — second-stage risk model that combines embeddings with established credit, merchant and user features.
  • Hybrid explainability algorithm — in-house attribution for the stacked model that produces decline reasons.
Continue reading — it's free

Google or a one-time email code. No password.