Developing an Aspect-Based Sentiment Analysis System for Online Course Reviews Using LLMs

Varvara Kalicheva23 May 2026

Problem

An online school or course platform collects hundreds of thousands of student reviews. Interviews give depth but do not scale, and 1-5 questionnaires show only "good" or "bad" inside the researcher's frame. Reviews are candid and detailed, but nobody can read them all by hand. Classic sentiment analysis gives one score per review, so mixed reviews average out to neutral and the team cannot see which part of the product is the problem.

Example from the article: the review "The material is interesting, but support is delayed, and the price is average" gets one neutral score, although Content is positive, Support is negative and Price is neutral.

Solution in brief

Extract for every review a set of quadruples (aspect term, opinion, sentiment, aspect category) with one few-shot LLM prompt, with no fine-tuning and no large labeled corpus. The prompt fixes exact-substring extraction rules and a closed list of categories. Quality is judged in three modes (strict, partial, semantic), because exact match under-rates LLM output.

Questions this case answers
  • How to extract aspect-level sentiment quadruples from short free-text feedback with an LLM and no fine-tuning
  • How to evaluate LLM span extraction when exact string match is too strict
  • How to extract aspect-level sentiment (aspect, opinion, polarity, category) from customer reviews with an LLM and no fine-tuning
  • How to find out what exactly users praise and criticize in thousands of free-text reviews

Tools and methods

  • Aspect sentiment quad prediction (ASQP) — joint extraction of aspect term, opinion, polarity and category
  • Few-shot prompting — role, rules, three steps, self-check and 10 hard examples
  • GPT-4.1 mini, YandexGPT 5.1 Pro, DeepSeek V3.2 — compared models
  • Rouge-L on lemmas (pymorphy3), FRIDA embeddings — partial and semantic evaluation
Continue reading — it's free

Google or a one-time email code. No password.