Query Rewards: Building a Recommendation Feedback Loop During Query Selection

Bella Huang, Raymond Hsu, Dylan Wang26 Oct 2022

Problem

Pinterest Homefeed gets about 30% of its recommended pins from pin-to-pin retrieval. Retrieval sends query pins, usually the user's earlier engaged pins, and gets similar pins back. A user may have hundreds or thousands of such pins, so the system must pick the right ones. The old picker sampled clusters of engaged pins by raw action counts. The choice stays static until many new actions arrive. It ignores what the user did with the last recommendations, and it weights same-type actions equally unless their timestamps differ.

Example from the article: a user has a large Recipes cluster and a small Furniture cluster. Many recipe pins are shown without engagement, so the Recipes reward falls and Furniture gets a greater chance.

Solution in brief

Add a Query Reward step before cluster sampling. An offline workflow computes the engagement rate of each query cluster from the engagement that followed its recommendations and stores it. At the next request the sampler uses this rate instead of raw counts, so ignored clusters lose weight and engaged ones gain it.

Questions this case answers
  • How to select seed items for similarity-based retrieval from a long interaction history when the selection must adapt to downstream engagement
  • How to choose which of a user's past items to use as queries for candidate retrieval?
  • How to make query selection react to what the user did with the previous recommendations?
  • How to avoid showing the same dominant interest over and over when the user ignores it?

Tools and methods

  • PinnerSAGE clusters — clusters of a user's engaged pins, built from embeddings; the source of queries
  • Query Reward workflow — offline job that stores each cluster's engagement rate
  • Reward-based cluster sampling — the reward raises or lowers a cluster's chance of selection; clusters without data get an average weight
Continue reading — it's free

Google or a one-time email code. No password.