Attempt to improve item recommendation accuracy using Item2vec

UchidaNaotaka4 Oct 2022

Problem

Mercari's Recommendation Team wanted home-page recommendations for books and manga that match user interests. The app already logs every item a user views. It also assigns series ("title") metadata, so "Kingdom Vol. 1", "Kingdom Vol. 25-50" and "Complete set of Kingdom" all carry the title "Kingdom". The team combined the two sources.

Example from the article: for the novel "Downtown Rocket" by Jun Ikeido, the model returned mostly other works by Jun Ikeido as similar titles.

Solution in brief

Treat each title as a word and each browsing session as a sentence, train Item2Vec (word2vec on titles) with Gensim, and store title-to-title cosine similarity. Hourly jobs map each user's recent views to similar titles and store the list for the home page. The A/B test was positive and the logic was adopted.

Questions this case answers
  • How to recommend catalog items from browsing logs when many listings are variants of the same underlying work and a heavy model cannot be served at scale
  • How to refresh personalized recommendations within hours while keeping the expensive similarity computation on a monthly schedule
  • How to recommend books and manga that match a user's interests when the catalog holds many volumes, sets and editions of the same series?
  • How to build item-to-item similarity from browsing logs without training a heavy ranking model?

Tools and methods

  • Item2Vec (word2vec on view sessions) — learns title vectors; cosine similarity between vectors gives related titles
  • Gensim — trains the model and gives most_similar lists for sanity checks
  • BigQuery — processes the view logs and runs the hourly recommendation step
  • Apache Airflow (Cloud Composer) — schedules the monthly training and hourly serving jobs
  • Bigtable — production store of the per-user list of recommended titles
Continue reading — it's free

Google or a one-time email code. No password.