How Zillow Data Science Measures Business Outcomes with Bayesian Statistics

Zillow14 Feb 2023

Problem

Zillow's Transaction & Engagement Analytics team tracks the transaction rate: the share of connections (a buyer or seller meets an agent through Zillow) that end in a bought or sold home. The nationwide rate rests on a large sample and is precise. Cut by zip code, neighborhood or a new agent partner, the same rate rests on a few observations and varies in precision from segment to segment. Raw rates for such segments are right-skewed and include exact 0% values.

Example from the article: the transaction rate of a rural zip code or a new Zillow partner with only a few connections.

Solution in brief

Treat each segment's rate as a draw from a Beta distribution and combine it with the segment's own counts. The segment estimate then starts from the higher-level rate and moves toward the segment's data as observations accumulate. Fit the Beta parameters from the data (empirical Bayes), or learn them layer by layer in a hierarchical model.

Questions this case answers
  • How to estimate a rate for many small segments when each segment has too few observations and raw rates are noisy or exactly zero
  • How to borrow strength from the overall population or a parent group to stabilize estimates for sparse segments
  • How to estimate a conversion rate for a zip code, neighborhood or new partner that has only a handful of observations?
  • How to compare conversion rates across segments that have very different amounts of data?

Tools and methods

  • Beta-binomial conjugate prior — closed-form posterior for a segment's rate
  • Empirical Bayes — set the Beta prior from the observed distribution of segment rates
  • Hierarchical Bayesian model — share information between zip codes within an MSA and across MSAs
  • PyMC3 (MCMC) — samples the posterior of the hierarchical model
Continue reading — it's free

Google or a one-time email code. No password.