Method

Model routing / cascades

Routing requests among models or cascading light to heavy models.

18 cases from 16 companies apply Model routing / cascades. 1 was checked in an online A/B test; 8 report results in production. Includes variants: LLM cascade.

Typical tasks: Reduce ML cost, Evaluate models, Optimize decisions, Answer questions over data and 9 more. Companies: Meta, DoorDash, Netflix, Zomato, Ramp, Databricks and 10 more. Often combined with: LLM cascade, Few-shot prompting, LLM-as-a-judge, Text classification, Named entity recognition and 58 more.