Shanshan Zheng, Databricks (Genie engineering team)
Evaluating AI systems is hard, especially when correctness is not binary. This talk describes how the Genie engineering team evaluates at scale: how "good" is defined, and how automated eval pipelines are built from offline benchmarks, human-in-the-loop validation and production feedback loops, along with the tradeoffs encountered.
Continue reading — it's free
Google or a one-time email code. No password.