Company

Khan Academy

5 cases from Khan Academy (Edtech), each digested from a public talk or engineering post and linked to the original. 1 was checked in an online A/B test; 2 report results in production.

Tasks: Avoid false wins in A/B tests, Evaluate models, Measure impact without an A/B test, Predict long-term effect from short tests, Analyze user behavior and 1 more. Methods: LLM-as-a-judge, Bayesian A/B test, Guardrails, Context engineering, Reflection / self-refinement, Synthetic data generation (LLM) and 5 more. Tools: GPT-4o.

Tech blog: engineering.khanacademy.org