Neural Network Models on Card Transactions

Alexey Firstov4 Jun 2022

Problem

Bank credit scoring is a binary classification task, solved in practice with gradient boosting. Its quality depends on aggregate features that analysts build by hand and then prune for stability, which takes a lot of time. The bank wants a model that reads raw card transactions and adds quality on top of the boosting scorecard.

Example from the article: at Alfa-Bank the training set is about 5 million credit applications with about 2 billion card transactions (about half a terabyte raw), and each month the model scores about 10 million clients.

Solution in brief

A neural network encodes a client's card transactions for the last year with embeddings and a bidirectional LSTM and outputs a default probability. Three training techniques make it practical: cyclical learning rate, length-bucketed batches and sample weights by application date. Quality is the Gini gain of a blend with the boosting score.

Questions this case answers
  • How to add a learned sequence model over raw event histories to a boosting model and measure what it adds beyond the existing score
  • How to train a recurrent network efficiently on event sequences of very different lengths without padding waste
  • How to add a neural network over raw card transactions to a gradient-boosting credit scorecard and measure the gain?
  • How to train a sequence model on clients with 10 to 1000 transactions without wasting compute on padding?

Tools and methods

  • Embeddings + BiLSTM with min/max pooling — encode a transaction sequence into a fixed-size vector.
  • Gini of the blend — the gain over the boosting scorecard, not stand-alone Gini.
  • Length-bucketed batches — similar lengths share a batch, so padding is minimal.
  • Time-based sample weights — recent applications count more in the loss.
  • Apache Hadoop, Apache Hive, Apache Spark — storage, access and preprocessing of the transaction data.
Continue reading — it's free

Google or a one-time email code. No password.