Problem
A job marketplace has to match free-text job postings with member profiles and resumes. Hand-built standardized features rely on small ML models, hard-to-maintain taxonomies and rigid upstream pipelines. General LLM embeddings miss marketplace terms and hiring signals in proprietary data. Fine-tuning and serving a 7B+ model is expensive at over 1 billion members and tens of millions of active postings. Embeddings must also reflect a profile edit or new posting within seconds.
Example from the article: LinkedIn replaced overlapping standardized features in the job recommendation and search L2 ranking models with LLM embeddings from JUDE (Job Understanding Data Expert).
Solution in brief
Fine-tune a shared LLM as a two-tower model on relevance labels first, then engagement labels, with three combined losses. Distill a cross-encoder into it to recover part of the quality gap. Serve embeddings by nearline inference with change detection and a one-time bulk bootstrap, and feed them to ranking models.
Questions this case answers
- How to turn free-text entities on both sides of a matching problem into embeddings from a fine-tuned LLM that a ranking model can use as features
- How to keep LLM-derived embeddings fresh within seconds when source texts change constantly and GPU inference is expensive
- How to replace hand-built text features in a job recommender with fine-tuned LLM embeddings?
- How to fine-tune a 7B LLM into a two-tower embedding model without paying for a cross-encoder at serving time?
Tools and methods
- Two-tower LLM fine-tuning — shared base LLM and tokenizer, separate prompt templates for jobs and members
- LoRA, FlashAttention 2, Liger kernels, DeepSpeed ZeRO stage 1 — make 7B+ training fit in memory and run fast
- Cross-encoder distillation — moves part of cross-attention quality into the cheap two-tower model
- Nearline inference (Kappa architecture) — Kafka/Brooklin streams, Samza, GPU serving, Venice store
- Bulk bootstrap (Flyte/Spark/K8S/Ray) — initial embeddings and backfills
Google or a one-time email code. No password.