Semantic Search for AI Agents at Scale: Retrieval and Ranking for LinkedIn’s Hiring Assistant

Nikita Zhiltsov, Achyuthan Jootoo Ramesh Bapu, Nikhil Thakur, Dan Liu, Evan L.11 Jun 2026

Problem

Recruiters on LinkedIn describe the person they need in a paragraph of free text, and the system must find matches among more than 1.3 billion member profiles. Faceted search answers almost every query but returns many unqualified profiles. Boolean keyword search is more precise but returns zero results for nearly half of queries. Many qualifications, such as cross-functional leadership, have no structured field.

Example from the article: the query "Licensed nurse practitioner, 3+ years in pediatric care, bilingual Spanish preferred" must find profiles that describe this experience in other words.

Solution in brief

An LLM judge, written down as a product definition of "qualified", labels millions of query-profile pairs. A dual-tower embedding model (MUSE) is fine-tuned on those labels only, so that cosine similarity reflects qualification fit. One query embedding, truncated to 2048 dimensions, drives billion-scale ANN retrieval. The same embedding at 4096 dimensions is a feature in the engagement-trained ranker.

Questions this case answers
  • How to retrieve items that satisfy a long free-text requirement from a corpus of a billion documents with sub-second latency, when keyword matching and attribute filters both fail
  • How to train a retrieval model on relevance labels that follow a product-specific definition of a match when only a small set of expert labels exists
  • How to retrieve people who match a free-text list of qualifications when keyword search misses synonyms and filters cannot express the requirement?
  • How to get millions of relevance labels for training an embedding model when human or expert-LLM labeling is too slow and costly?

Tools and methods

  • LLM judge (Expert Judge) and open-weight teacher — label qualification match; the teacher replicates the judge at scale
  • Dual-tower shared-weight LLM encoder — encodes queries and profiles into one space
  • Matryoshka InfoNCE loss — contrastive training at several truncation sizes
  • IVFPQ ANN index with attribute post-filters — retrieval over 1B+ profiles
  • DCNv2 ranker — consumes the embeddings as features
  • vLLM, Apache Spark, Apache Iceberg — query serving, indexing and batch inference, embedding storage
Continue reading — it's free

Google or a one-time email code. No password.