How to Find Students' Opinions and Not Drown in Spam?

Polina Basina24 May 2025

Problem

A research team at Tomsk State University studies how students perceive education. Every year they collect about 5 million posts and comments from public VKontakte university communities. At most 1% of that is a student opinion or evaluation about studying; the rest is sale ads, event announcements, lost-and-found notices and plain questions. Analysts clean the data with heuristics and clustering, then assessors still discard most of what they receive, so 90% of project time goes into cleaning and manual categorization.

Example from the article: "How does Ivanov run exams?" is junk, but "the lecturer does not suit me, what should I do" is a relevant opinion. Structurally the two messages are almost the same.

Solution in brief

Put an automatic binary relevant-versus-irrelevant filter in front of manual labelling. The team compared a semantic router built from clustered examples and a neural classifier trained on the data accumulated over the years. The classifier was faster and more accurate, and it cut analysts' extra cleaning time by 40–45%.

Questions this case answers
  • How to find the rare relevant items in a huge noisy text stream before manual labelling when relevant and irrelevant items look structurally alike
  • How to choose between a semantic router and a trained classifier as a first-pass filter when the category scheme keeps changing
  • How to find the few relevant opinions in a huge noisy social-media stream before manual labelling
  • How to cut the share of junk that human labellers have to review

Tools and methods

  • Community registry — keyword search for communities, automatic filtering, then manual validation
  • Semantic router — routes for relevant and irrelevant texts, built from clusters of two years of data
  • Neural classifier — binary filter trained on 102 thousand posts and 91 thousand comments
  • Double-assessor labelling — category and sentiment assigned only when two assessors agree
Continue reading — it's free

Google or a one-time email code. No password.