Problem
A multi-armed bandit with too many items needs a candidate-selection step that sends only the top items to it. Improving that step needs an A/B test, and the test is biased twice. Treatment bandits start from scratch on new items while control bandits have already learned their best item. Analyzing only converged bandits fixes this but selects units by a post-treatment variable. A naive read can reject a better method.
Example from the article: Yelp picks photos for advertisers who choose "Let Yelp optimize". A formula or a light embedding model picks top photos, a Beta-Bernoulli bandit chooses among them (impression = pull, click = success), and the experiment is diverted by advertiser_id.
Solution in brief
Compare only bandits that count as converged under a fixed rule (the 80-80 rule). Build the experiment as matched pairs of advertisers, with random assignment inside each pair. If either bandit in a pair has not converged, drop both (pairwise deletion), and run a sensitivity analysis.
Questions this case answers
- How to compare two upstream selection methods when the downstream adaptive allocator in the treatment group starts with no history and the control group's allocator has already learned
- How to analyze a randomized experiment when the units usable for analysis are chosen by a post-treatment convergence criterion
- How to A/B test a new candidate-selection step that feeds a multi-armed bandit without reading a false negative?
- How to decide when a freshly started bandit has converged enough to be compared with an established one?
Tools and methods
- Thompson sampling — the bandit policy, updated nightly.
- A/B test diverted by advertiser — the whole system, selection step plus bandit, is the subject.
- 80-80 convergence rule — decides when a bandit's data can enter the analysis.
- Matched pairs with pairwise deletion — control observed confounders (Mahalanobis distance for numeric features) and handle unconverged bandits.
Google or a one-time email code. No password.