Method

Masked language modeling

Predict masked tokens (BERT-style pretraining).

10 cases from 8 companies apply Masked language modeling. 1 was checked in an online A/B test; 3 report results in production.

Typical tasks: Learn user and item embeddings, Recommend from what the user just did, Detect anomalies, Retrieve and rank from millions of items and 5 more. Companies: Alfa-Bank, Ramp, Grab, Raiffeisenbank, Instacart, CFT and 2 more. Often combined with: Self-supervised learning, Event sequence encoder, Contrastive learning, SHAP, Multi-label classification and 28 more.