Problem
A recommender is retrained, say, once a day, but users keep clicking between two trainings. A new user, or a known user with fresh clicks, has no up-to-date embedding, so the system uses a random vector or waits for the next training. Transformers avoid this: they take the new click sequence as input. Matrix factorization (MF) and graph neural networks (GNN) tie each user to an ID-specific embedding, and retraining a GNN is far too slow for this. Hybrid models are as slow as their graph part.
Example from the article: on MovieLens, full retraining of the graph part for test users takes about 4 orders of magnitude longer than a fold-in update in the hybrid model.
Solution in brief
Fold-in: freeze all item and other user embeddings and re-solve the optimization for one user's embedding. For graph models the problem is of fourth order with no closed form, and it is solved with ADMM or quasi-Newton methods, or analytically through UltraGCN. Quality stays close to full retraining at a small overhead over doing nothing.
Questions this case answers
- How to update a graph-based recommender with a new user's interactions between full retrainings without retraining the model
- How to keep the graph component from slowing down a hybrid recommender at serving time
- How to update a graph-based recommender for a user's new interactions between retrainings without retraining the model
- How to build recommendations for a new or returning user in a matrix factorization or GNN model in real time
Tools and methods
- Fold-in — update one user's embedding against frozen item embeddings
- UltraGCN — graph model with an analytical fold-in
- LightGCN, NGCF — standard graph collaborative-filtering models
- ADMM, quasi-Newton methods — explicit solvers
- SASRec plus a graph part — transformer and graph hybrid
Google or a one-time email code. No password.
