Efficient Optimization With Ax, an Open Platform for Adaptive Experimentation

Miles Olson, Max Balandat, Eytan Bakshy18 Nov 2025

Problem

Teams that tune AI models, infrastructure or hardware designs work with many interacting parameters. Testing a single configuration is slow and costly in compute or time, so every evaluation has to count. Real goals also rarely reduce to one metric: they mean balancing several objectives under constraints and guardrails.

Example from the article: researchers had to shrink natural language models to fit the first generation of Ray-Ban Stories and needed models that trade off size against performance.

Solution in brief

Run the experiment adaptively: fit a surrogate model on the configurations tried so far, let it propose the most promising next configuration, evaluate it, and repeat until the budget runs out. Ax packages this Bayesian optimization loop, built on BoTorch, with support for multiple objectives, constraints and analysis of the results.

Questions this case answers
  • How to find the best configuration of a black-box system with many interacting parameters when each evaluation is costly and the evaluation budget is small
  • How to optimize several competing objectives under constraints while keeping other metrics from regressing
  • How to tune a system with many parameters when each evaluation is slow or expensive?
  • How to optimize one metric without breaking guardrail metrics?

Tools and methods

  • Bayesian optimization — sequential search that balances exploring new configurations and refining good ones
  • Gaussian process (GP) — surrogate model that predicts the metric with uncertainty, effective with very few data points
  • Expected improvement (EI) — acquisition function that picks the next candidate
  • Ax — open-source (MIT) platform for adaptive experiments, installed with pip install ax-platform
  • BoTorch — library behind Ax's Bayesian optimization components
Continue reading — it's free

Google or a one-time email code. No password.