How We Teach Alice to Activate on Intonation

Dmitry Solodukha24 May 2025

Problem

A voice assistant wakes up on a fixed word ("Alice"), which users must repeat before every request. The team wants a spotter that reacts to the intonation of a request addressed to the device, so a dialogue continues without the name. There is no production traffic for this feature, no labels, and the old metrics do not fit. The model must run on the device and find where the request starts, so the server can take over.

Example from the article: after "Alice, what's the weather?" the user says just "And in Minsk?". The spotter must fire on that request, and stay silent when the same person talks to someone else in the room.

Solution in brief

Take audio from the assistant's "follow-up listening" mode, where the device keeps listening after an answer without the name. Label it with ASR output and session heuristics, turned into pseudo-labels by a CatBoost ensemble. Train a larger streaming Conv1D model with a loss that pushes the activation point left, then quantize it for the NPU.

Questions this case answers
  • How to train an on-device streaming detector that fires at the start of a request addressed to the device when there is no labeled or production data for the new task
  • How to build pseudo-labels for whether a request was addressed to the device from automatic transcripts and session heuristics
  • How to train an on-device voice activation model when there is no production data and no labels for the new task
  • How to build pseudo-labels for 'was this request addressed to the device' from ASR output and session heuristics

Tools and methods

  • Follow-up listening audio — training data with real left context and no "Alice".
  • Pseudo-labels (ASR, session heuristics, CatBoost) — "request addressed to the device" labels.
  • Latency Control Loss — fires at the start of the request.
  • Weighted Scenario Recall, False Activation per Device — offline and online metrics.
  • Quantization, TFLite, streaming windows — deployment on the NPU.
Continue reading — it's free

Google or a one-time email code. No password.