JUDE: LLM-based representation learning for LinkedIn job recommendations

Nikita Zhiltsov, Benjamin Le, Rajat Arora, Yu Jiang, Xiao Shi, Timothy J. Hazen, Jianqiang Jerry Shen22 May 2025

Problem

A job marketplace has to match free-text job postings with member profiles and resumes. Hand-built standardized features rely on small ML models, hard-to-maintain taxonomies and rigid upstream pipelines. General LLM embeddings miss marketplace terms and hiring signals in proprietary data. Fine-tuning and serving a 7B+ model is expensive at over 1 billion members and tens of millions of active postings. Embeddings must also reflect a profile edit or new posting within seconds.

Example from the article: LinkedIn replaced overlapping standardized features in the job recommendation and search L2 ranking models with LLM embeddings from JUDE (Job Understanding Data Expert).

Solution in brief

Fine-tune a shared LLM as a two-tower model on relevance labels first, then engagement labels, with three combined losses. Distill a cross-encoder into it to recover part of the quality gap. Serve embeddings by nearline inference with change detection and a one-time bulk bootstrap, and feed them to ranking models.

Questions this case answers
  • How to turn free-text entities on both sides of a matching problem into embeddings from a fine-tuned LLM that a ranking model can use as features
  • How to keep LLM-derived embeddings fresh within seconds when source texts change constantly and GPU inference is expensive
  • How to replace hand-built text features in a job recommender with fine-tuned LLM embeddings?
  • How to fine-tune a 7B LLM into a two-tower embedding model without paying for a cross-encoder at serving time?

Tools and methods

  • Two-tower LLM fine-tuning — shared base LLM and tokenizer, separate prompt templates for jobs and members
  • LoRA, FlashAttention 2, Liger kernels, DeepSpeed ZeRO stage 1 — make 7B+ training fit in memory and run fast
  • Cross-encoder distillation — moves part of cross-attention quality into the cheap two-tower model
  • Nearline inference (Kappa architecture) — Kafka/Brooklin streams, Samza, GPU serving, Venice store
  • Bulk bootstrap (Flyte/Spark/K8S/Ray) — initial embeddings and backfills
Continue reading — it's free

Google or a one-time email code. No password.