Predicting Rider Conversion in Sparse Data Environments with Bayesian Trees

Zammit Alban30 Mar 2026

Problem

Lyft needs a real-time prediction of whether a rider who has entered a destination and seen a price and ETA will request a ride. The decision depends on many contexts at once: place, time, supply and demand, and the rider's history. Slicing the data that finely leaves few examples per context. A gradient boosted model fitted on 10 sessions with 8 conversions can confidently predict 80%, which is likely noise. Neural or LLM models add too much inference time for live use.

Example from the article: a business traveler leaving the Detroit suburbs at 4:00 AM on a Tuesday for an airport 30 minutes away may match only about ten historical sessions.

Solution in brief

Organize sessions into a tree by a hierarchy of context keys and fit one small parametric model per node, top-down. Each child is fitted with an L2 penalty that pulls its parameters toward its parent's, which is a Gaussian Bayesian prior. Sparse leaves fall back to the parent's behavior, and rich ones follow their own data.

Questions this case answers
  • How to predict a binary user decision for thousands of fine-grained contexts when most contexts have only a few observations and predictions must be served with very low latency
  • How to make a sparse-data model's output respect known directional relationships between a feature and the outcome
  • How to predict whether a user will convert in a very narrow context that has only a handful of historical examples?
  • How to serve conversion predictions in real time without a heavy neural network or LLM?

Tools and methods

  • Hierarchical tree of partition keys — splits sessions from broad to specific contexts.
  • Gaussian prior centered on the parent (L2 penalty) — shrinks a child's parameters toward the parent's.
  • Logistic regression per node — a simple model that allows sign constraints.
  • Monotonic constraint — keeps predictions consistent with domain knowledge.
Continue reading — it's free

Google or a one-time email code. No password.