MegaFon RecSys Workshop, part 1

Artem Glazunov4 Jun 2022

Problem

A team building a recommender needs a pipeline that works for new users and new items, not only for those seen in training. Plain matrix factorization on interactions ranks new items like a random model, so it cannot handle novelties. Quality must also be reported per user and per slice (warm, cold users, cold items), otherwise cold-start failures hide inside an average.

Example from the article: a music service dataset with 7 million listening records, where the target is whether a user replayed a song within a month.

Solution in brief

Hold out 5% of songs and 10% of users as cold entities before any training. Evaluate every model through one interface on validation, test, cold users and cold items, against random and popularity baselines. A LightFM model trained on user IDs plus item features keeps interaction quality and is the only one that handles cold items well.

Questions this case answers
  • How to build and evaluate a ranking model that must also serve entities absent from the training history
  • How to compare a latent-factor recommender with popularity and random baselines separately on warm and cold users and items
  • How to build a recommender that also works for new users and new items?
  • How to split data and pick baselines so that cold-start quality is measured separately?

Tools and methods

  • LightFM — hybrid factorization model: embeddings and biases per categorical feature, score is a dot product plus biases
  • Popularity and random rankers — baselines every model must beat
  • Binning of numeric features — LightFM accepts only categorical features
  • Bayesian optimization — 20 trials over loss, embedding size and column set (the optimizer is presumably Optuna)
  • t-SNE and UMAP — projection of item embeddings to check genre structure
Continue reading — it's free

Google or a one-time email code. No password.