Problem
A family shares one smart speaker, mostly to play music. People often say just "play some music" and expect their own recommendations, but the speaker does not know who is talking, so it cannot pick the right account. Scenarios such as recommendations and blocking need different error rates. The model also has to run on the device, because enrollment voice data cannot be processed or stored in the cloud, and it must cope with noise, distance and similar voices.
Example from the article: A family of four has enrolled. Someone says "Play music". The model picks the closest enrolled person, or reports an unknown speaker if even the best similarity is below the threshold.
Solution in brief
A small on-device embedding model compares the request with the registration phrases of every enrolled person, and below a similarity threshold the speaker counts as unknown. It is trained in two stages: classification pretraining on single-speaker device logs, then pairwise verification with labels from a large offline teacher. Quality is measured on data from households who use the devices freely and label their own recordings.
Questions this case answers
- How to identify which enrolled member is speaking on a shared device and decide when the speaker is unknown
- How to get trustworthy labels and test data for speaker identification when annotators cannot tell similar voices apart
- How to give each member of a household their own recommendations on a shared device
- How to identify a speaker by voice on a shared smart speaker and decide when the speaker is unknown
Tools and methods
- Similarity ranking with an unknown-speaker threshold — pick the closest enrolled person or reject.
- Classification pretraining and pairwise verification — two training stages for speaker embeddings.
- Knowledge distillation — a large offline model teaches the small device model.
- Quantization and model streaming — fit a few million parameters on the device.
Google or a one-time email code. No password.
