Evaluating AI-generated images (Blackbox optimization)

Jong Woo Han, Suman Bae, Heechan Kim23 Jun 2025

Problem

LY Corporation (LINE) wanted a text-to-image model that draws in the company's minimalist style from a prompt alone, to cut repetitive illustration work for designers. The same fine-tuned model gives on-style or off-style images depending on seed, guidance strength and LoRA scale. Manual or grid search costs too much time and GPU, and automating it requires turning "a good image" into a number.

Example from the article: for the prompt "A woman is holding a picket sign. The sign has the words 'hold the LINE' written on it." the same model gave a style-compliant or a non-compliant image depending on the settings.

Solution in brief

Pick a good seed per prompt with an attention-based loss. Score every image by a normalized average of four metrics (CLIP score, VQA score, HPS-V2, PickScore). Run Bayesian optimization over the guidance scales and LoRA scale with that score as the objective: 13 images per prompt and seed replace the grid.

Questions this case answers
  • How to tune the settings of a generative model automatically against a composite quality score when each evaluation is an expensive black-box call
  • How to merge several quality metrics with different scales and distributions into a single objective for automated search
  • How to pick generation hyperparameters automatically instead of a manual grid search when image quality must be scored by a metric?
  • How to choose a good starting seed for a text-to-image model without generating a full image per candidate?

Tools and methods

  • Bayesian optimization — finds good settings in 13 evaluations by modelling the score surface.
  • Seed screening by attention loss — ranks random seeds before the search.
  • Composite image score — CLIP and VQA scores for prompt fidelity, HPS-V2 and PickScore for human preference.
  • Stable Diffusion 3.5 with LoRA — the tuned generator.
  • CLIP — embedding model behind the CLIP score.
Continue reading — it's free

Google or a one-time email code. No password.