Applying NLP Models to Automate Call Center Work

Albert Akopyan25 May 2024

Problem

Alfa-Bank chat operators handle the questions that chatbots could not answer, so these questions are non-trivial. Operators lose time in three places: finding the right one of 15 interfaces, searching for a reply template in personal Word files, and navigating a knowledge base of several thousand articles that newcomers struggle to navigate. All three slow down handling and onboarding. Only text chats are in scope.

Example from the article: the knowledge base holds current facts such as the interest rate on a credit card, and a newcomer has to find the right article among thousands by search that the speaker called "not the best".

Solution in brief

Three models read the dialogue context and suggest what the operator needs next: interfaces (multi-label classifier), templates and ready answers (contrastive retrieval), and knowledge-base suggestions (RAG, still a demo). The template model trains on a small labeled set plus 200M unlabeled messages in alternating batches. An A/B test on 200 vs 200 operators showed 5% lower average handling time (AHT).

Questions this case answers
  • How to recommend the next reply template, interface or knowledge article to a human agent in real time from the dialogue context, when labeled pairs are few and unlabeled logs are large
  • How to speed up a BERT-style recommender enough to run on CPU for thousands of concurrent agents
  • How to recommend interfaces, reply templates and knowledge-base articles to chat operators in real time
  • How to train a recommender when only a small labeled set of dialogue-to-template pairs exists

Tools and methods

  • fastText + BiLSTM — multi-label interface recommendation
  • BERT encoders with contrastive loss — template and ready-answer retrieval, pre-trained with masked language modeling on operator-client chats
  • NER plus regular expressions — masking personal data in recommended answers
  • RAG on OpenSearch with an internal Llama 3-based LLM — knowledge-base suggestions
  • ONNX and ONNX Runtime — 4x faster CPU inference
Continue reading — it's free

Google or a one-time email code. No password.