How We Grew Sales Team Efficiency with Customer Ranking

Roman Zakharov24 May 2025

Problem

Avito's sales managers each handle 100+ sellers and cannot tell what to offer whom, so heuristics spread. Internal evaluation put the team's ROI close to zero. Managers also judge a call by revenue growth, which favors large clients that grow without any call. The real effect is revenue with the call minus revenue without it, and it cannot be observed for one client.

Example from the article: a small client goes from 10,000 to 15,000 RUB after a call and a large one from 100,000 to 115,000 RUB. The manager sees +5,000 vs +15,000 RUB. Without the call they would have had 8,000 and 110,000 RUB, so the real effects are +7,000 vs +5,000 RUB.

Solution in brief

An uplift model predicts each client's revenue with and without a manager's offer, and clients are ranked by the difference. Training data comes from a randomized test/control split sized to managers' capacity. The ranking is validated against random order on held-out test and control, then confirmed online with two groups of managers.

Questions this case answers
  • How to rank customers for manual outreach by the incremental effect of the contact rather than by their growth when the contact capacity is limited
  • How to measure the effect of a ranking model on revenue with an A/B test when the treatment is rarely delivered to the treated group
  • How to rank customers for sales-manager calls by incremental revenue rather than by revenue growth
  • How to measure the effect of a ranking model on revenue with an A/B test when managers cannot reach many clients

Tools and methods

  • S-learner uplift model — one model on client features and the offer flag
  • CatBoost, Random Forest, XGBoost, LightGBM — call-success models that select comparable clients
  • Cumulative uplift curve with an exponential fit — ML ranking vs random order
  • Difference of differences — online experiment, ML vs baseline ranking
Continue reading — it's free

Google or a one-time email code. No password.