Problem
A team builds recommendations for a service with many new or rarely active users. Collaborative models such as matrix factorization have little to learn from, so the only signal left is content: a few genres or tags per user. This is "sparse profile cold start", where the history is minimal but not empty. The study asks whether a neural reranker on top of retrieval helps here. Judged only by HitRate and NDCG, a recommender hides low serendipity and exposure concentration on a few popular items.
Example from the article: on the Serendipity 2018 movie dataset, an MS MARCO cross-encoder cut HitRate by 34% against the candidate list without reranking, and collapsed catalog coverage from 3% to 1%.
Solution in brief
Treat the neural reranker as a hypothesis to test, not an upgrade. Compare it with simple baselines (popularity, random, content-based) and with a hybrid retriever (semantic search, BM25 and popularity merged by Reciprocal Rank Fusion), and report accuracy, serendipity and exposure metrics together. The authors found the reranker harmful and the retriever the real bottleneck.
Questions this case answers
- How to improve recommendations for users with only a few tags of history, where a neural reranker on top of retrieval may not help
- How to find out whether the retriever or the reranker limits quality of a sparse-profile recommender, measuring serendipity and exposure concentration alongside accuracy
- How to improve cold-start recommendations when users have only a few tags of history
- How to check that a neural reranker actually helps in a sparse-profile recommender
Tools and methods
- Hybrid candidate retrieval — semantic search, BM25 and popularity merged by Reciprocal Rank Fusion (RRF)
- FAISS — semantic search over item vectors
- Cross-encoder (MS MARCO) — reranker of the candidates
- Two-head ranker (pilot) — mixes relevance and serendipity scores
- Bootstrap — 1000-iteration confidence intervals
Google or a one-time email code. No password.
