Method
Multimodal fusion
Combining modalities by early, late or cross-attention fusion.
13 cases from 11 companies apply Multimodal fusion. 5 were checked in an online A/B test; 2 report results in production.
Typical tasks: Recommend from what the user just did, Retrieve and rank from millions of items, Match entities, Learn user and item embeddings and 8 more. Companies: Yandex, VK, Avito, Netflix, Grab, DoorDash and 5 more. Often combined with: Time-based validation, Gradient boosting, Two-tower model, Hard negative mining, Deep & Cross Network and 46 more.
| Neural Ranking for Recommendations | classifieds | 2026-05 | |
X-split ARGUS: An Autoregressive Encoder-Decoder for Ad Ranking![]() | ![]() | adtech | 2026-05 |
Digital Twins of the Heart and Speeding Them Up![]() | ![]() | healthtech | 2026-05 |
| Synchronizing the Senses: Powering Multimodal Intelligence for Video Search | media-streaming | 2026-04 | |
Building Multimodal Product Embeddings to Improve Matching at Yandex Market![]() | ![]() | marketplace | 2025-11 |
| User foundation models for Grab | food-delivery | 2025-09 | |
| Developing Multimodal Content Models for Recommendations | social | 2025-05 | |
| Using LLM to transcribe restaurant menu photos | food-delivery | 2025-03 | |
| HRBert2.0: Improving Vectorization of Vacancies and Resumes | classifieds | 2024-11 | |
| Machine Learning in Content Moderation at Etsy | marketplace | 2024-08 | |
| Transforming Recommendations | social | 2024-05 | |
| Beyond Words: How Multimodal Embeddings Elevate eBay's Product Recommendations | marketplace | 2023-09 | |
| How Pinterest Leverages Realtime User Actions in Recommendation to Boost Homefeed Engagement Volume | social | 2022-11 |

