An ML based approach to proactive advertiser churn prevention

Erika Sun, Ogheneovo Dibie9 May 2023

Problem

Pinterest sales teams usually contact an advertiser only after the advertiser has stopped spending. It is very hard to win back an advertiser who has left. The team wanted to find small and medium business (SMB) advertisers likely to leave soon and let account managers (AMs) act in time.

Example from the article: in the managed North America SMB segment, AMs in the treatment group saw a churn risk tier and a churn reason category for each account, and the churn rate of the high tier pods was 24% lower than in the control group.

Solution in brief

A gradient boosting model predicts, once a day, whether an active advertiser will have no spend in a 7-day window starting 14 days later. SHAP values show the main drivers of the score, which are grouped into coarse churn reasons. Accounts are split into high, medium and low risk tiers with thresholds chosen together with sales, and the effect was tested in a randomized experiment.

Questions this case answers
  • How to predict which paying customers will stop spending within a fixed horizon so that account managers can intervene before they leave
  • How to turn a churn score into risk tiers and driver explanations that a non-technical team can act on
  • How to measure whether exposing churn predictions to account managers causes lower churn, when the model scores cannot be tested in isolation
  • How to find advertisers who are about to stop spending before they leave, instead of calling them after they have gone?

Tools and methods

  • Gradient boosting decision trees (GBDT) — snapshot model for the 14-day churn probability on tabular features.
  • SHAP — contribution of each feature to the churn probability, used to name the drivers shown to sales.
  • Risk tiers with thresholds — high, medium and low categories fitted to the precision and recall that sales asked for.
  • Randomized experiment by account — treatment sees predictions, control keeps the usual process.
Continue reading — it's free

Google or a one-time email code. No password.