How We Personalized Recommendations for Professionals on Thumbtack

Aishwarya V Srinivasan, Shishir Dash17 Jun 2024

Problem

Thumbtack is a home-services marketplace with hundreds of thousands of professionals (pros) and over 500 service categories. In the app, a carousel suggests actions that can bring a pro more leads: widening job preferences inside a category, adding categories or areas, or raising bids. Every pro saw the same order, set by product intuition, even though pros differ in what they need and which action suits them. The team wanted an order that follows each pro's expected gain.

Example from the article: some pros optimize for more leads, others for higher-quality leads or a higher job conversion, yet all of them got the same ordering of cards.

Solution in brief

Rank each recommendation by an expected lead gain: the probability that the pro enables it after viewing it, multiplied by the expected lead volume if enabled. The first factor comes from a boosted-tree classifier, the second from a fixed effects regression. Training data came from past launch experiments, because new recommendations had always been tested at carousel position 1.

Questions this case answers
  • How to rank a set of suggested actions per user by expected gain when only a static, intuition-based order has been used so far
  • How to build a first training set for a ranker when position and impression logging did not exist and the displayed order was never varied
  • How to rank a carousel of recommended actions separately for each seller instead of one fixed order for everyone
  • How to train a ranking model when no position or impression logs exist yet

Tools and methods

  • Boosted-tree binary classifier — predicts P(enable | view) per pro and recommendation type
  • Fixed effects regression (value given enablement, VGE) — estimates the lead volume gained when a recommendation is enabled
  • Expected-value ranking — multiplies the two outputs to order the carousel for each pro
  • Airflow, BigQuery, Google Cloud Storage — weekly offline inference and storage of predictions
Continue reading — it's free

Google or a one-time email code. No password.