Real-Time Personalization Using Microservices

5 Jun 2023

Problem

More than half of the visitors to Target.com are new or not logged in. A batch job has no history for them before they arrive, so it cannot precompute their recommendations. By the time the next batch runs, many have left without finding anything relevant, which hurts engagement and retention. Target's Personalization team (PRZ) had to recommend from what the visitor does right now, and still answer fast enough for a web page.

Example from the article: a visitor who looked at a keyboard and a monitor is probably building a work-from-home setup, so the page shows HDMI cables or monitor mounts instead of generic items.

Solution in brief

Replace daily batch predictions with online prediction. Each model runs as its own gRPC microservice with trained model files in memory, reads fresh clickstream and catalog features from a feature store, and scores and sorts items per request. Data scientists write the services in Python; Java serves very high traffic.

Questions this case answers
  • How to serve personalized recommendations to users with no history when batch-computed predictions cannot exist for them
  • How to deploy ML scoring as independent services that meet a strict per-request latency budget at a known traffic level
  • How to recommend relevant items to new and logged-out visitors who have no precomputed recommendations?
  • How to serve ML recommendations online within a fixed latency budget?

Tools and methods

  • Online prediction — score items per request from the guest's latest activity
  • gRPC microservices — one service per model, called by the personalization engine
  • Feature store — clickstream and catalog features for fast lookup at request time
  • Graph neural network — scores items from recent activity
  • Bandits — adjust recommendations from feedback
  • Hadoop cluster and GCP storage — offline training; model files go to GCP and are loaded into services
Continue reading — it's free

Google or a one-time email code. No password.