Method
Masked language modeling
Predict masked tokens (BERT-style pretraining).
10 cases from 8 companies apply Masked language modeling. 1 was checked in an online A/B test; 3 report results in production.
Typical tasks: Learn user and item embeddings, Recommend from what the user just did, Detect anomalies, Retrieve and rank from millions of items and 5 more. Companies: Alfa-Bank, Ramp, Grab, Raiffeisenbank, Instacart, CFT and 2 more. Often combined with: Self-supervised learning, Event sequence encoder, Contrastive learning, SHAP, Multi-label classification and 28 more.
| Putting a Transformer Inside Every Card Authorization | fintech | 2026-10 | |
| Learning from Unstructured Sequences in 2026 | fintech | 2026-05 | |
| User foundation models for Grab | food-delivery | 2025-09 | |
| Chasing All the Hares: Managing Different Content Types in One Recommender System | banking | 2025-05 | |
| Sequence models for Contextual Recommendations at Instacart | e-commerce | 2024-10 | |
| Applying NLP Models to Automate Call Center Work | banking | 2024-05 | |
| Learning to Understand Dialogs Based on Real DataCF | CF | fintech | 2023-10 |
| NLP at Alfa-Bank, Part 8: The Development History of an Operator Assistant | banking | 2023-05 | |
| NLP at Alfa-Bank, Part 3: Modeling | banking | 2023-05 | |
| A small BERT towards Large Medical Models | healthtech | 2022-06 |