How We Are Deploying AI at a University

Maxim Novikov, Ilya Galyukshev23 May 2026

Problem

A university course gets 10–15 problems per homework from hundreds of students. Assistants cannot grade it all quickly, so feedback takes weeks or months. Plain LLM grading goes wrong in known ways. Over a long assignment the model loses track and gives half the points with a confident "all good". A generic "grade fairly" prompt works badly. Asked to find errors, the model finds them even where none exist.

Example from the article: Central University grading text assignments and notebooks in three courses, plus a separate review of GitLab repositories.

Solution in brief

Split each assignment into tasks and grade each against instructor-written criteria. A worker LLM grades and a supervisor LLM checks it. Assistants review the result in a separate UI; the share they accept without edits is the quality metric. For code, an agent walks a graph of code entities, specialist agents propose remarks, and a verifier filters them before the score is combined with test results.

Questions this case answers
  • How to automate grading of long multi-part submissions with an LLM while experts keep control of the final score
  • How to stop an LLM reviewer from over-reporting issues when it is asked to find flaws instead of checking conformance to criteria
  • How to automate LLM grading of student homework while instructors keep control of the final score
  • How to measure whether LLM grading is good enough: the share of submissions accepted without edits

Tools and methods

  • Admin panel and Grader UI — instructors set criteria and reference solutions; assistants edit the LLM's feedback and points.
  • Worker and supervisor LLM — one grades a task, another validates it and can send it back.
  • Share accepted without edits — the main quality metric.
  • Code entity graph — navigation map for the review agent.
  • Verifier, arbiter and aggregator — filter and merge agent remarks into a score.
  • Playwright MCP — browser checks of UI assignments.
Continue reading — it's free

Google or a one-time email code. No password.