Problem
CFT runs speech analytics for its call center: a model transcribes the operator's audio track, then a checking block verifies that the operator says what is required and avoids what is forbidden. The department that monitors operators asked for a new check: find operators who shout, because shouting counts as emotional pressure on the customer. Recordings run from a few seconds to about 10 minutes, and labeled shouts were almost absent.
Example from the article: the monitoring department's reports held only about 5 shouting recordings, and staged shouting does not match real anger.
Solution in brief
Tap the first transformer layer of the speech-to-text model already in production and train a small fully connected "shout / no shout" classifier on its outputs. Mine training candidates with a public 7-emotion audio model and let the monitoring department label them. Tune the threshold to shrink the pile of recordings to review, not to catch every shout.
Questions this case answers
- How to detect a rare acoustic event in long audio recordings when almost no labeled examples exist and a production speech model must be reused
- How to choose which layer of a pretrained speech model to use as features for a new classifier without running a second large model
- How to detect shouting operators in call recordings when almost no labeled examples exist
- How to add a new audio classifier without deploying another large model
Tools and methods
- wav2vec-style speech model (already in production) — frozen feature extractor, no new large model to deploy
- Public 7-emotion audio classification model — mines shout candidates from conversation history
- t-SNE — visual comparison of class separation across layers
- Small multilayer perceptron — binary classifier on first-layer outputs
- Probability threshold — controls the balance between missed shouts and false positives
Google or a one-time email code. No password.