Method

Self-supervised learning

Pretraining on unlabeled data with pretext tasks.

18 cases from 17 companies apply Self-supervised learning. 3 were checked in an online A/B test; 1 report results in production.

Typical tasks: Learn user and item embeddings, Catch fraud and abuse, Predict churn and lifetime value, Retrieve and rank from millions of items and 8 more. Companies: Skoltech, Ramp, Affirm, Meta, Grab, Avito and 11 more. Often combined with: Contrastive learning, Gradient boosting, Masked language modeling, Multimodal embeddings, Event sequence encoder and 37 more.