Problem
A bank's collection team calls clients with overdue loan payments. A baseline model built on client, loan and delinquency data predicts payment well, but it ignores what happened in the call. Without that, the team keeps calling clients who will pay anyway and cannot tell early which clients will not pay. Operators also get no feedback on which phrases helped.
Example from the article: success is defined as the client paying off the overdue amount within 7 days after the call; the goal is to raise the quality of the baseline model with call data.
Solution in brief
Build two separate models on the call: one on the audio track and one on the speech-to-text transcript. Each predicts the probability of success. Blend both predictions with the baseline model's probability into the final payment probability. The two modalities add more together than each alone.
Questions this case answers
- How to improve an existing tabular scoring model with signals from recorded conversations when labeled conversations are few
- How to combine predictions from several modalities of the same interaction into one probability of a customer outcome
- How to predict whether a client will pay an overdue loan after a collection call
- How to use call audio and call transcripts as features on top of a baseline scoring model
Tools and methods
- LSTM over MFCC sequences — audio model core, better than convolutional networks in the authors' tests
- Gradient boosting — one emotion model per emotion, trained on audio features
- Pretrained BERT — embeds each utterance (average pooling) in the text model
- GRU with self-attention — runs over the sequence of utterance embeddings
- Hugging Face pretrained audio models — penultimate-layer outputs used as embeddings
- Data augmentation — cutting dialogues at reply endings (audio) and random cuts or utterance removal (text)
- Blending — merges audio, text and baseline probabilities
Google or a one-time email code. No password.