Problem
First-line support staff in an IT service management (ITSM) system must pick one of about 2,000 request templates for each ticket. The template sets the fields and the routing to a working group, so a wrong pick means a wrong group and a wrong answer to the client. The first line picks the right template in 60% of cases. The template set keeps changing, so a plain classifier over template IDs misses rare and new templates.
Example from the article: the service takes a ticket text and returns at most 10 templates sorted by relevance. Data: 5,740,031 tickets and 2,076 templates.
Solution in brief
Use classical NLP instead of BERT. A linear SVM on TF-IDF covers templates with enough history. Cosine similarity between the TF-IDF vectors of the ticket and of the template text covers rare and new templates. Both results are merged into one top-10 list, and a grid search keeps the model tuned.
Questions this case answers
- How to recommend the top-k matching templates for a free-text request from a large, changing set of items when many items have few or no historical examples
- How to choose between a transformer and a linear model with sparse features for a short-text ranking task under a CPU budget
- How to recommend the top-10 matching ticket templates for a free-text support request
- How to rank items when the item set changes and new items have no history
Tools and methods
- TF-IDF with lemmatization — text features after cleaning and replacing personal data with tokens
- Linear SVM — classifier on request-to-template history
- Cosine similarity — match between ticket text and template text, no history needed
- Grid search with block cross-validation — tuning on top-1, top-5 and top-10 accuracy
- BentoML, FastAPI — packaging and serving the model on CPU
Google or a one-time email code. No password.