NLP at Alfa-Bank, Part 3: Modeling

Vladislav Galichanskiy, Eduard Yanakov, Demid Gaibov27 May 2023

Problem

An ML team that ships NLP models must pick an approach before it knows how much accuracy it needs, how much data it has and what production allows. A simple model misses the accuracy target. A complex one burns weeks on debugging or is too slow to serve. After the first model the team is often a few percent short of the business target with no ordered list of things to try.

Illustrative example: a support-ticket classifier built with plain BERT fine-tuning scores a few points under the target, and the team does not know whether to change pooling, augment rare classes or fight overfitting.

Solution in brief

Pick the simplest model that fits five constraints: development time, required quality, data, developer expertise and production limits. For BERT-class tasks start with plain fine-tuning, then work through a fixed list of quality tricks, regularization fixes and speed-ups only if the target is still missed.

Questions this case answers
  • How to choose between rules, classical models and a pretrained transformer for text classification under fixed development time, data and production limits
  • How to get the last few points of accuracy from a fine-tuned transformer text classifier using an ordered list of fixes
  • How to choose an NLP model between rules, classical ML and BERT under time, data and production limits
  • How to fine-tune BERT for text classification and squeeze out the last few percent of accuracy

Tools and methods

  • Rules, regular expressions, classical ML — fast baselines with little or no data
  • FastText, Word2Vec, ELMo, BERT — embeddings, from context-free to contextual
  • BERT fine-tuning — pooling, layer-wise learning rates, MLM pretraining on own data
  • BIO-tagging and CRF — named entity recognition as token classification with valid tag sequences
  • Augmentation, LR schedulers, dropout, AdamW, LogitNorm — quality and overfitting fixes
  • Mixed precision, ONNX, TensorRT, distillation, xFormers — training and inference speed
Continue reading — it's free

Google or a one-time email code. No password.