ABsalute: A/B Testing Automation

Maria Shabalkova23 May 2026

Problem

X5 ships hundreds of changes across dozens of digital products, including the mobile apps of Pyatyorochka, Chizhik and Perekrestok, with more than 80 million loyal customers. Even a 0.1% effect on per-user retail turnover (RTO) is a large sum per year, so each change needs an online A/B test. The team needed one platform for design, splitting and results. Splitting must not slow down apps that handle sales, experiments must not collide, and results must catch very small effects.

Example from the article: two overlapping tests must never show one person a blue button on a blue background.

Solution in brief

X5 built an in-house platform from separate services: a Go realtime splitter, a Python backend with a React UI, and an Airflow and PySpark calculation service. The splitter uses layers, two hashing salts and a Bloom filter for speed. Results are computed in two stages: cheap daily readouts to catch harmful changes, and a final recalculation with CUPED for sensitivity.

Questions this case answers
  • How to run many overlapping online experiments on a high-traffic mobile product without conflicts while the traffic splitter adds no perceptible latency
  • How to get cheap daily experiment readouts for catching harmful changes and a more sensitive final estimate from the same data
  • How to build an in-house online A/B testing platform for mobile apps
  • How to split app traffic into experiments fast enough not to slow the app down

Tools and methods

  • Layered traffic splitting — a layer usually corresponds to an app screen
  • Hash bucketing with two salts — stable, independent assignment by layer and by experiment ID
  • Bloom filter — fast membership check for fixed user lists
  • Ratio metric linearization and weighted linear regression — daily results
  • CUPED — final recalculation with higher sensitivity
  • Go, Python, Airflow, PySpark, Kafka, PostgreSQL, Kubernetes — the platform stack
Continue reading — it's free

Google or a one-time email code. No password.