Method
KV cache
Cache attention keys and values across decoding steps.
8 cases from 5 companies apply KV cache. 2 were checked in an online A/B test; 3 report results in production.
Typical tasks: Retrieve and rank from millions of items, Reduce ML cost, Recommend from what the user just did, Analyze user behavior and 8 more. Companies: Pinterest, Yandex, LY Corporation, LinkedIn, Netflix. Often combined with: LLM-as-a-judge, RAG, Knowledge distillation, LoRA fine-tuning, Synthetic data generation (LLM) and 36 more.
| From Activity to Intent: Generating User Journeys with LLMs | social | 2026-10 | |
Developing a model to detect harmful content from OpenChat titles and descriptions![]() | ![]() | social | 2026-08 |
AgentOps in Production: Infrastructure for LLM Agents and Support Automation for Yandex Personal Services![]() | ![]() | media-streaming | 2026-05 |
| Scaling Recommendation Systems with Request-Level Deduplication | social | 2026-04 | |
| Building the next generation of job search at LinkedIn | marketplace | 2025-05 | |
5 Stops on the Way to LLM in Your Production![]() | ![]() | general | 2025-05 |
| Foundation Model for Personalized Recommendation | media-streaming | 2025-03 | |
| Speeding up transformers | — | general | 2022-10 |

