Dense Retrieval

18 Nov 2025

Problem

Keyword search on a visual, multilingual marketplace has low recall. Queries that return no results, even though relevant items exist, are a missed business opportunity. Adding dense retrieval to the live engine breaks things users notice. Vector matches flood the results, a filter raises the number of hits, and approximate search times out into zero-result sessions. Vinted ran about 50 A/B tests before enabling it for all searches.

Example from the article: the query zx750 with a brand filter had about 300 lexical matches. The tuned distance threshold added about 7000 mostly random nearest-neighbor items. Without the filter the same query returned about 450 matches (about 300 lexical and about 150 nearest neighbor).

Solution in brief

A two-tower model embeds queries and items into one 256-dimension space, and an HNSW index in Vespa serves the neighbors. The ranking profile detects vector-only matches and cuts them to a fixed count. A split latency budget with an exact-search retry removes most timeouts.

Questions this case answers
  • How to add vector retrieval to a lexical search engine so that the number of vector-only matches stays bounded under filters and re-sorting
  • How to keep approximate nearest neighbor search within a hard latency budget when timeouts produce empty results
  • How to add embedding-based retrieval to a keyword search without flooding results with poor matches?
  • How to cap the number of nearest-neighbor matches when filters and sort orders change the search strategy?

Tools and methods

  • Two-tower model, contrastive training — query and item embeddings, 256 dimensions.
  • CLIP — frozen multilingual query encoder, precomputed image embeddings.
  • Vespa with HNSW — filtered nearest neighbor search, hosts both models.
  • ONNX — item tower export with preprocessing inside the graph.
  • Reciprocal rank fusion (RRF) score — turns similarity rank into a position threshold.
  • Global-phase rank-score-drop-limit — drops excess vector-only matches.
Continue reading — it's free

Google or a one-time email code. No password.