Bandits for Marketing Optimization

Tilman Drerup, Toban Wiebe, Fang Guo, Rustin Partow26 Jun 2024

Problem

A company that buys paid marketing wants to maximize Return minus Cost for each campaign. It needs the performance curve: how the objective changes with a marketing action such as the target cost per action (CPA). With the curve known, the best action is easy to read off. Past spend data is a poor basis for the curve, because earlier actions were never random. Competitor spend, for example, raises cost-per-click and lowers returns.

Example from the article: marketers lean into holidays in anticipation of customer interest, and competitors raise spend, so observational curves of past actions get the sign of the relationship wrong.

Solution in brief

Add controlled random perturbations to marketing actions. This gives variation that is independent of marketer judgment, so the performance curve can be estimated causally. A continuous loop refits the curve with inverse-propensity-weighted (IPW) regression and picks the next action with Thompson sampling. Randomization shrinks as the model gets more certain.

Questions this case answers
  • How to learn a response curve of an outcome to a continuous decision variable when historical choices of that variable were not random
  • How to balance exploration and exploitation when each action also has to be chosen automatically in a repeated loop
  • How to find the spend level of a paid marketing campaign that maximizes return minus cost when past spend decisions are confounded?
  • How to get a credible performance curve for a campaign without running a fully randomized experiment?

Tools and methods

  • Adaptive experiment — randomizes across actions with a bias toward better ones, between a full A/B test and no test
  • Inverse-propensity-weighted regression — reweights history to remove the bias from favoring actions believed optimal
  • Bayesian parametric curve — Beta-pdf-shaped hump curve that starts at zero spend
  • Thompson sampling — draws one curve from the posterior and plays its optimal action
Continue reading — it's free

Google or a one-time email code. No password.