Churn and Propensity Tasks on Neural Network Transaction Models

Alexander Senin4 Jun 2022

Problem

A bank loses customers to competitors and wants to find them while they are still active enough to retain. It also recommends products by email, push and SMS and wants to know who will open a product. Both tasks already have a boosting model built on aggregated customer features, which sees transactions only through aggregates and misses the order and timing of events. The team needs a measurable gain and a way to prove it in production.

Example from the article: Alfa-Bank trained neural networks on card and settlement-account transactions to predict pre-churn two months ahead, and on card transactions to predict product opening within a month after a communication.

Solution in brief

Train a neural network on raw transaction sequences and blend its score with the boosting score through logistic regression. Bucket sequences by length and keep transactions in a columnar format on the cluster. Test the churn model in production with a two-by-two design that separates the model effect from the retention offer effect.

Questions this case answers
  • How to add event-sequence models on top of an aggregate-feature boosting model to improve scoring of user outcomes
  • How to train a sequence model when the number of event histories reaches billions of records
  • How to separate the effect of a scoring model from the effect of the treatment it triggers when the treatment is only offered to high-score users
  • How to improve a churn or product-propensity model that already runs on gradient boosting by adding raw transaction sequences?

Tools and methods

  • Gradient boosting — existing baseline on aggregated customer features.
  • Bidirectional LSTM over transaction embeddings — sequence model: categorized features, per-value embeddings, pooling over time, fully connected layers.
  • Logistic regression blend — combines boosting and neural scores.
  • Length buckets and Parquet storage — keep memory and disk use bounded.
  • Score-bin A/B design — separates offer effect from model effect.
Continue reading — it's free

Google or a one-time email code. No password.