Problem
A voice assistant (Salute at SberDevices) answers free conversation with a retrieval model. It picks the best reply among editor-written ones, using the last six phrases of the dialogue as context. One core encoder serves about 10 tasks at once (paraphrasing, dialogue modeling, NER, intents, sentiment). With 2 million training samples and a 32 GB V100, only 24–32 samples per task fit into a batch, and BERT-like models depend strongly on batch size. The embeddings are also fragile to typos and synonyms, and the index is large.
Example from the article: the dialogue model embeds a six-phrase context and ranks editor-written replies by recall@top-k.
Solution in brief
Train the encoder with triplet loss on context and reply vectors, and add four tricks: weight averaging across GPUs, distillation of LaBSE embeddings, adversarial text augmentations as extra positives, and "full batching" with all other triples as negatives. A critic score from labeled data is added to the triplet margin.
Questions this case answers
- How to train a text embedding model for ranking candidate replies when many simultaneous tasks leave only tiny per-task batches
- How to shrink a text embedding index while keeping ranking relevance, using a larger multilingual model as the teacher
- How to make text embeddings stable under typos and synonym substitutions in user input
- How to train a retrieval-based reply ranker for a voice assistant when only 24–32 samples per task fit into a batch?
Tools and methods
- Triplet loss — pulls the context toward the relevant reply and away from irrelevant ones.
- Weight averaging — twin networks trained on different shards, weights averaged.
- Knowledge distillation — MSE between compressed student vectors and LaBSE vectors.
- Data augmentation — a "broken" query becomes a positive for the original.
- In-batch negatives — positives of other triples serve as negatives.
- LaBSE, RoBERTa, Horovod — teacher, student, multi-GPU training.
Google or a one-time email code. No password.
