A Personal Mentor Powered by GPT

Andrey Smirnov25 May 2024

Problem

An online education platform has mentors who answer student questions within an SLA, so students wait for help. A mentor also cannot sit next to every student with a code error or an unclear lesson fragment. Yandex Praktikum wanted a chat that answers the questions a student would not bring to a mentor, stays on topic and does not hand over ready solutions.

Example from the article: a student asks the chat about their code on a task, and the chat answers using the task text, the reference solution and the student's current code.

Solution in brief

The chat finds where the student is (course, module, lesson, task), builds a prompt from that context, the chat history and the methodologists' answering rules, checks the question, generates an answer and checks it. To adapt the model, the author gives a ladder: zero-shot, one-shot, few-shot, PTune adapters, full fine-tuning.

Questions this case answers
  • How to build an LLM assistant grounded in the user's current position in a course, without handing over ready solutions
  • How to choose between prompting, adapters and full fine-tuning for an LLM feature by moving up a ladder of costlier options only when the cheaper one fails a test set
  • How to build an LLM tutor chat that answers course questions without a human mentor
  • How to decide between prompting, PEFT adapters and full fine-tuning for an LLM product feature

Tools and methods

  • Task and lesson context in the prompt — keeps answers inside the course.
  • Mini-RAG with a semantic splitter — retrieves the closest parts of a long lesson.
  • Methodological prompt, question and answer checks — answering rules and a toxicity filter.
  • Zero-, one-, few-shot prompting, PTune, full fine-tuning — the adaptation ladder.
Continue reading — it's free

Google or a one-time email code. No password.