1. Keyword
Loading eSpeak NG …
2. Audio
Loading the model …
3. Live output
- Phones
- Keyword score
- –
- Detections
- Speed
- –
What you're seeing
The model reads 16 kHz audio and returns 50 frames per second. It is causal, so it can run while you speak. As the wake-word plugin does, the page keeps the last 1.5 s of audio and runs the model on it after every 80 ms of new sound.
Voice activity is the model's probability, for each 20 ms frame, that someone in the foreground is speaking. Background talk and music are trained towards zero. It tends to drop in the short pauses between words.
Phones are the most likely IPA symbol in each frame, with repeats merged and the CTC blank removed: plain greedy decoding, nothing learned on top. The blank penalty makes the decoder less eager to output nothing; 2 is the value tuned on LibriSpeech. The phone output describes the audio about 100 ms after the fact, so each phone is drawn 100 ms earlier than the frame that produced it. This output was trained on isolated words. On sentences the phones are rough: many are missed and some are wrong.
Keyword spotting needs no recordings of the word. Your word is converted to the model's IPA symbols in the browser with eSpeak NG in the language you choose (American English by default), and each 1.5 s window is scored by the best path of those symbols through the phone posteriors, compared with the best symbol in each frame and divided by the number of phones. A score of 0 is a perfect match. The default threshold, −1.437, kept false activations to one per hour for “jarvis” on 31 hours of speech and noise; other words need their own, and thresholds were checked on English words only. Words whose sounds are rare in everyday speech work best. Common sound sequences match everywhere.
The keyword can be text in any language eSpeak NG supports, phonemised with that language's voice, or IPA typed directly and split into the model's own symbols. How well it works varies by word and by language, and phones the model's table lacks are spelled with its nearest symbols or dropped.