Building a Playlist You Will Like

Rinat Mullakhmetov, Vladimir Kravtsov25 May 2024

Problem

The Zvuk streaming service wanted its assistant to answer "generate a playlist for ..." with tracks from its own catalog. The first version asked an LLM API for track names and then looked those names up in the catalog. It was hard to personalize and knew nothing about new releases or rare songs. It also produced names that did not exist and depended on a paid third-party model.

Example from the article: a query such as "hookah rap" should return fitting catalog tracks, and writing playlist titles for every track by hand was too expensive.

Solution in brief

Turn the catalog into a vector index and search it with a 7B LLM used as a text embedder. The training data is fully synthetic: an LLM writes playlist titles that a track fits and does not fit, built from a JSON description of each track. The model is fine-tuned with LoRA and a contrastive loss, and checked on about 500 queries written by music experts.

Questions this case answers
  • How to build free-text search over a catalog of items described by metadata when no labelled query-item pairs exist, using synthetic queries generated by an LLM
  • How to fine-tune a large embedding model for a niche domain with a contrastive loss on synthetic data and check it against an expert-written test set
  • How to build a text-to-playlist search that selects tracks from a catalog by a free-form query
  • How to train an embedding retriever without labeled query-document pairs, using LLM-generated synthetic data

Tools and methods

  • e5-mistral-7b-instruct, GritLM-7B — base embedding models, fine-tuned with LoRA
  • Synthetic triplets (query, positive, hard negative) — LLM-generated training data, following the e5-mistral paper
  • Triplet contrastive loss, CoSENT loss — training objectives for short and long queries
  • R-precision, Precision@k — offline metrics on an expert test set
  • Qdrant, FastAPI, Streamlit, Transformers — demo stack
Continue reading — it's free

Google or a one-time email code. No password.