LLM-assisted vector similarity search

Md Riyadh23 Oct 2024

Problem

Embedding search ranks texts by closeness of meaning, and a statement and its opposite are often close. "I like fishing" and "I do not like fishing" get similar vectors. As a result, queries with negations, extra constraints or a loosely worded concept return irrelevant items at the top. Teams that search over entity descriptions in a relational database hit this on queries with negations, extra constraints or conceptual requirements.

Example from the article: for the query "food with no fish or shrimp" over a list of 100 dishes, plain vector search returned tempura, ceviche and sushi, which all contain seafood.

Solution in brief

Run the usual vector search, but take a larger shortlist (10–50 items) instead of the final top 1–3. Pass the shortlist and the original query to an LLM, and ask it to pick the best matches. The vector index keeps search fast, and the LLM applies the negations and context in the query.

Questions this case answers
  • How to make embedding-based retrieval respect negations and constraints in a free-text query when the embeddings place opposite statements close together
  • How to make vector search handle queries with negations and extra constraints?
  • How to improve the relevance of semantic search over descriptions of entities in a relational database?
  • How to add an LLM to an existing embedding search without replacing it?

Tools and methods

  • Vector similarity search (FAISS) — builds the shortlist from embeddings of entity descriptions.
  • OpenAI text-embedding-ada-002 — produces the embeddings.
  • GPT-4o — reads the query and the shortlist and returns the top matches.
  • LLM reranking — second stage that reorders the shortlist.
Continue reading — it's free

Google or a one-time email code. No password.