Automating Topic Modeling with AutoTM

Maria Khodorchenko27 May 2023

Problem

A team has a large unlabeled text collection and must find out what it is about, split it into categories or turn each text into a vector, then explain the result to a client. Additive regularization of topic models (ARTM) fits: it is interpretable, fast, and lets you impose properties on topics. The price of that flexibility is a huge number of hyperparameters. Tuning them by hand for each corpus takes so long that the method is rarely used.

Example from the article: a jobs site needed a feed for new users known only by a cookie or hash. Topic modeling over the postings gave collections such as "jobs for women", "warehouse jobs" and "jobs with daily payment", pre-computed and shown instead of a random feed.

Solution in brief

AutoTM treats the manual tuning loop as a "learning strategy", a sequence of regularization stages, and searches for the best strategy with a genetic algorithm. Coherence plus the weakest topic's coherence is the fitness function. A surrogate regression model replaces some fitness evaluations, which gives a 1.8x speedup.

Questions this case answers
  • How to cluster a large unlabeled text collection into interpretable topics without tuning many model hyperparameters by hand
  • How to recommend content to a user about whom nothing is known using topic collections computed in advance
  • How to cluster a large unlabeled text collection into interpretable topics without tuning the model by hand
  • How to tune a topic model with many hyperparameters automatically

Tools and methods

  • ARTM / BigARTM — topic model with additive regularizers, C++ implementation
  • Genetic algorithm — searches the space of learning strategies
  • Surrogate model — regression that predicts fitness to skip model training
  • Coherence, NPMI — topic quality metrics
  • Iterated racing — tunes the genetic algorithm itself
  • Toloka — crowd annotation of topic quality
  • MLflow, Celery, Kubernetes — logging and distributed fitness computation
Continue reading — it's free

Google or a one-time email code. No password.