Problem
Gazprombank's analysts work with a mart of client payments, mostly from government organizations. The only signal is a free-text description with no template, and there are no flags, only regex-based tags. The team needs the probability that a payment belongs to each business category, as features for other bank models. Rules cannot cover all wordings, and one payment often fits several categories.
Example from the article: a fine for unpaid parking is tagged "Auto" by the rules, while a model tags it "Traffic Police fines". Both are right, and the label scheme allows only one.
Solution in brief
Start with regex rules for a rough split, then train a classifier on those labels and study its errors to fix the rules. Because single-class labels cannot teach multi-label prediction, relabel the data with one logistic regression per class (One-vs-Rest). Train a stack of the three best models, chosen by F1, on the result.
Questions this case answers
- How to assign several categories to free-text records when the only labels are noisy single-class rules
- How to use a classifier's disagreements with rule-based labels to find errors in the rules and rare classes
- How to categorize free-text payment descriptions into business categories when there are no flags or labels
- How to get several categories per payment (multi-label) when the only labels are single-class regex rules
Tools and methods
- Regular expressions and SQL LIKE — first labeling into tax, fine, duty, payment
- TF-IDF, fastText, LDA, k-means, HDBSCAN — clustering attempts that did not work
- CatBoost and logistic regression (eli5) — classifiers on regex pseudo-labels, used to find labeling errors
- One-vs-Rest relabeling — turns single-class labels into multi-label ones
- Stacking of three best models by F1 — final multi-label classifier
Google or a one-time email code. No password.