Problem
A bank scores loan applicants with several neural networks, each trained on its own sequence source: card transactions, credit bureau histories, later receipts. Their scores were blended with logistic regression, so each model's view is squeezed into one number and cross-source interactions are lost. One end-to-end model on all raw sources would capture them, but it trains for a long time and makes it hard to test improvements to a single source. The target is default: an overdue payment of 90 or more days, or restructuring, within 12 months of loan issue.
Example from the article: At Alfa-Bank, replacing the logistic-regression blend of neural scores with a model on client embeddings raised Gini by 1.8 points with no new data.
Solution in brief
Take the embedding of each already trained per-source model (the vector from one of its last fully connected layers) and feed all embeddings into one fully connected network. Missing sources get the source's average embedding, and tabular features join as entity embeddings. Base models keep being developed separately.
Questions this case answers
- How to combine predictions from several per-source sequence models into one model that captures cross-source interactions, when end-to-end training is too slow and hard to maintain
- How to handle clients who are missing from some data sources when feeding source embeddings into a joint model
- How to combine several sequential data sources (transactions, credit bureau histories, receipts) in one default-prediction model?
- How to improve a credit scoring model without collecting new data?
Tools and methods
- Credit scoring — default probability, evaluated by Gini
- Per-source sequence encoders — give a client embedding for each source
- Unified fully connected network — learns interactions between sources
- Mean-embedding imputation — keeps clients with missing sources
- Entity embeddings — bring tabular features into the network
- Logistic-regression blending — mixes the unified score with the boosting score
Google or a one-time email code. No password.