AI Blender in Search: How We Stopped Worshipping Clicks and Moved Block Placement to LLMs

Alexey Zabelin23 May 2026

Problem

A search page mixes web documents with dozens of thematic blocks (images, video, weather, calculator, object card). The team must decide which blocks to show per query and where. Click metrics misjudge blocks: a useless block still gets clicks, a new block is ignored until users learn it, and a block that answers on the page gets no click. A personalization model trained on such logs then hides the block for good.

Example from the article: a user searches "weather Stavropol", sees in the weather block that rain is coming, and clicks through to the precipitation map. The log shows the block skipped, so it counts as useless and personalization stops showing it.

Solution in brief

An LLM grades every block for every query by relevance (TOP, REL+, REL−, IRREL), using a per-block "passport" that says what the block is for. The LLM is distilled into BERT for production. The grade-based score is added to the online click model with a weight chosen on an A/B grid, then checked by SbS against production.

Questions this case answers
  • How to decide which heterogeneous content blocks to show next to search results and at which position when click logs misjudge the usefulness of blocks
  • How to combine an LLM-graded relevance signal with a click-based model when clicks are biased against blocks that answer on the page
  • How to decide which thematic blocks (images, video, weather, object card) to show on a search page and at which position
  • How to fix click-metric bias when ranking blocks that users rarely click or have not yet got used to

Tools and methods

  • LLM with two prompts — a query primer, then a grade per block.
  • Block (TB) passport — text on what the block solves and how it looks.
  • Offline blending — a fixed position per grade.
  • Distillation into BERT — query and block embedders plus a shared layer.
  • A/B experiment and SbS — pick the weight, verify page quality.
Continue reading — it's free

Google or a one-time email code. No password.