Method
Language-model pretraining
MLM, continual or domain-adaptive pretraining of language models.
18 cases from 14 companies apply Language-model pretraining. 4 were checked in an online A/B test; 7 report results in production. Includes variants: Masked language modeling.
Typical tasks: Recommend from what the user just did, Learn user and item embeddings, Retrieve and rank from millions of items, Recommend to new users and new items and 9 more. Companies: Alfa-Bank, Netflix, Yandex, Ramp, X5 Tech, Grab and 8 more. Often combined with: Masked language modeling, Self-supervised learning, Sequential recommendation, Event sequence encoder, Gradient boosting and 49 more.

