DL in finance: key results, production and new challenges

Dmitry Koptelov4 Jun 2022

Problem

A bank trains new default models on its loan history. That history contains only clients approved by the previous scoring model, so the new model never sees the "bad" clients it will meet when it scores the full flow of applicants. Issuing loans to everyone to get an unbiased sample is too expensive. The unbiased sample the bank does have is tiny, and the true default label arrives only 4–6 months after scoring, so quality drift is noticed late.

Example from the article: Alfa-Bank has several million internal originations but only a few thousand "challenger" originations (credit cards issued monthly to everyone, regardless of rejection policy).

Solution in brief

Train on both samples at once and give the challenger sample a higher weight than the internal one. Check Gini on each sample separately. Blend the neural scores with a boosting score using logistic regression. Run the whole chain monthly in Airflow and watch score and feature distributions, because real quality is known only months later.

Questions this case answers
  • How to train a risk model when the labeled history was filtered by the previous model and only a small unbiased sample exists
  • How to monitor a prediction model in production when ground truth arrives with a delay of months
  • How to train a default-risk model when the historical sample contains only clients approved by the previous model?
  • How to combine a small unbiased sample with millions of biased records in one training run?

Tools and methods

  • Weighted combined training — one network on the challenger and internal samples, with a larger weight per challenger record.
  • Logistic regression blending — fits weights for the four scores (card transactions, settlement account, credit bureau, boosting).
  • Distribution monitoring — month-over-month comparison of score and feature distributions.
  • Apache Airflow, Apache Spark, Apache Hadoop — monthly pipeline from raw data to scores.
Continue reading — it's free

Google or a one-time email code. No password.