Problem
Zillow's Transaction & Engagement Analytics team tracks the transaction rate: the share of connections (a buyer or seller meets an agent through Zillow) that end in a bought or sold home. The nationwide rate rests on a large sample and is precise. Cut by zip code, neighborhood or a new agent partner, the same rate rests on a few observations and varies in precision from segment to segment. Raw rates for such segments are right-skewed and include exact 0% values.
Example from the article: the transaction rate of a rural zip code or a new Zillow partner with only a few connections.
Solution in brief
Treat each segment's rate as a draw from a Beta distribution and combine it with the segment's own counts. The segment estimate then starts from the higher-level rate and moves toward the segment's data as observations accumulate. Fit the Beta parameters from the data (empirical Bayes), or learn them layer by layer in a hierarchical model.
Questions this case answers
- How to estimate a rate for many small segments when each segment has too few observations and raw rates are noisy or exactly zero
- How to borrow strength from the overall population or a parent group to stabilize estimates for sparse segments
- How to estimate a conversion rate for a zip code, neighborhood or new partner that has only a handful of observations?
- How to compare conversion rates across segments that have very different amounts of data?
Tools and methods
- Beta-binomial conjugate prior — closed-form posterior for a segment's rate
- Empirical Bayes — set the Beta prior from the observed distribution of segment rates
- Hierarchical Bayesian model — share information between zip codes within an MSA and across MSAs
- PyMC3 (MCMC) — samples the posterior of the hierarchical model
Google or a one-time email code. No password.