Method

LLM inference optimization

KV cache, batching, speculative decoding, parallelism for serving speed.

21 cases from 13 companies apply LLM inference optimization. 3 were checked in an online A/B test; 11 report results in production. Includes variants: KV cache.

Typical tasks: Reduce ML cost, Retrieve and rank from millions of items, Recommend from what the user just did, Catch fraud and abuse and 10 more. Companies: Pinterest, Netflix, DoorDash, Yandex, Avito, LinkedIn and 7 more. Often combined with: KV cache, Knowledge distillation, LoRA fine-tuning, LLM-as-a-judge, Quantization and 58 more.

From Activity to Intent: Generating User Journeys with LLMssocial2026-10
Advancing Generative Recommendation Inference: our collaboration with NVIDIA on TensorRT LLMmedia-streaming2026-10
Developing a model to detect harmful content from OpenChat titles and descriptionssocial2026-08
GenRec: Towards LLM-Native Recommendation at Netflixmedia-streaming2026-07
Building Food Metadata with LLM Juries, Context Optimization & Multimodal AIfood-delivery2026-07
AgentOps in Production: Infrastructure for LLM Agents and Support Automation for Yandex Personal Servicesmedia-streaming2026-05
Deploying LLMs in Moderation: From PoC to the Target Solutionclassifieds2026-05
Scaling Recommendation Systems with Request-Level Deduplicationsocial2026-04
Mind the Gap: Using LLMs to bridge behavioral silos in multi-vertical recommendationsfood-delivery2025-12
Life Without Servers: Comparing Libraries for Embedding Neural Networks into C++ Applicationsgeneral2025-11
Building the next generation of job search at LinkedInmarketplace2025-05
Graphs and Agents in Action: Building a CoPilot for All Sber Branchesbanking2025-05
Moderation Platformizationclassifieds2025-05
5 Stops on the Way to LLM in Your Productiongeneral2025-05
JUDE: LLM-based representation learning for LinkedIn job recommendationsmarketplace2025-05
Foundation Model for Personalized Recommendationmedia-streaming2025-03
Sequence learning: A paradigm shift for personalized ads recommendationsadtech2024-11
LLM-based Approach to Large-scale Item Category Classificationmarketplace2024-05
Marketplace Product Matching in Detail: Models, Approaches and Tips for Building an Effective Matcher for Tens of Millions of Productsmarketplace2024-05
Accelerating Large Generative Models in NLP—general2023-05
Speeding up transformers—general2022-10