Problem
A bank trains credit scoring models on loans that an earlier model and the credit policy approved. Rejected applicants never get a default flag, so the sample holds mostly "good" clients and the model ranks unfamiliar applicants worse. The bias grows with each retraining. A second problem is the target: default means 90 or more days past due, so the label is known only a year after issuance and test data lags production by a year.
Example from the article: Alfa-Bank's card-transaction model was trained on internal approvals only but scores the whole incoming flow in production.
Solution in brief
Add two samples to the biased internal approvals: a small "challenger" stream where loans go to everyone regardless of credit policy, and loans that rejected applicants took at other banks (from credit bureau data). Train on the union with tuned sample weights, initialized from a logistic regression over three scores. Watch stability with a short-delinquency proxy while the true label matures.
Questions this case answers
- How to train a risk model when labels exist only for objects the previous model accepted, so the training sample is biased toward easy cases
- How to combine samples with different distributions and sizes in one training run without the small, most representative sample being ignored
- How to monitor model stability when the true outcome is observed only after a long delay
- How to correct selection bias in a credit scoring model trained only on approved loans?
Tools and methods
- Reject inference — learn about rejected applicants from a challenger sample and external loans.
- Challenger sample — a permanent small flow with no credit policy, used for training, final blending and testing.
- Sample-type feature plus logistic regression — gives initial sample weights.
- Blending with logistic regression — combines the scores of four models.
- Short-delinquency proxy target — 30 days past due within 4 or 6 months, for early stability checks.
Google or a one-time email code. No password.