Getting the Most Out of ML Models When There Is Too Little Data

Olesya Noritsyna23 May 2026

Problem

A marketing agency's clients ask how many sales a channel (a billboard, TV, paid search) brought, so they can reallocate budget. The data is thin: hourly TV, radio and outdoor stats are unrepresentative and kept for a week, metro data is monthly, so models run on weekly data. Ads run in flights (2–3 weeks on air, then 5–6 weeks off), so three years give about 150 rows. Regression and boosting give negative channel weights or random SHAP signs under multicollinearity, and the business stops trusting the model.

Example from the article: "We have weekly data and flights two or three times a year, here are your 150 points, do whatever you want with them."

Solution in brief

Fit a Bayesian marketing mix model in PyMC-Marketing. Its built-in adstock (carry-over of ads) and saturation (diminishing returns) modules encode the two basic rules of advertising. Priors from business knowledge stabilize estimates on 150 points. Four validation stages follow, and lift tests calibrate the model where the client allows them.

Questions this case answers
  • How to estimate each marketing channel's contribution to an outcome from very few weekly observations with multicollinear inputs
  • How to build a model whose channel effects stakeholders trust by encoding carry-over and diminishing-returns rules as priors
  • How to estimate the sales contribution of each advertising channel when there are only about 150 weekly data points
  • How to build a marketing mix model that business stakeholders trust

Tools and methods

  • PyMC-Marketing (PyMC), Python, PostgreSQL — the team's stack.
  • Bayesian regression with priors — gives a distribution of each channel's contribution instead of a single coefficient.
  • Adstock and saturation transforms — carry-over and diminishing returns per channel.
  • Sampler and residual diagnostics — divergences, R-hat, ESS, R2, MAPE, residual tests.
  • Lift-test calibration, cross-validation, holdout — checks of predicted effects.
Continue reading — it's free

Google or a one-time email code. No password.