Method

Bayesian A/B test

Experiment analysis with posteriors and probability-to-beat-control instead of p-values.

7 cases from 7 companies apply Bayesian A/B test. 1 was checked in an online A/B test.

Typical tasks: Avoid false wins in A/B tests, Get experiment results faster, Predict long-term effect from short tests, Experiment when users affect each other and 1 more. Companies: Spotify, Sber, Khan Academy, Netflix and 3 more. Often combined with: Bootstrap, FDR correction, Guardrails, Sequential testing, t-test and 19 more.