Problem
Yandex Images ranks pictures from a stable document base and a fresh base with a high update rate. For a news-driven query such as "Dune" in the week of a film release, the final ranking must learn when a query is fresh. Fresh queries are a few percent of traffic, their intent changes over time, and user signals suffer from cold start, so the team must mine and label them by date. Crowd labeling is expensive, because an annotator has to open news sources and judge each query. A plain LLM has a static context and knows nothing about new events.
Example from the article: "World Cup" is fresh on 20 November 2022 (a football World Cup was running) and not fresh on 10 October 2025 (no significant news).
Solution in brief
Split labeling into two steps. A search-grounded assistant (Search with Alice) collects and summarizes news for the query and date. A separate large static-context LLM judges significance against explicit criteria and outputs the freshness label. Both prompts are tuned to each other, with explicit rules and few-shot examples.
Questions this case answers
- How to label whether a query is time-sensitive on a given date with an LLM pipeline that first gathers current sources and then judges significance against explicit criteria, instead of crowd labeling
- How to validate and monitor an LLM labeling pipeline against a crowd benchmark after launch
- How to label whether a search query is fresh (news-related) on a given date without crowd annotators
- How to mine rare fresh queries to train a ranking model that mixes fresh and non-fresh documents
Tools and methods
- Search with Alice (RAG) — collects and summarizes news for a query and date
- Static-context LLM — GPT-4o, DeepSeek or YandexGPT; gives the final 0/1 label
- Few-shot prompting — examples of step-1 output with correct answers
- Crowd benchmark — small daily sample for precision/recall charts and alerts
Google or a one-time email code. No password.
