Speaker similarity

The optional Speaker Similarity tile reports a cosine similarity between the speaker embedding of the current recording and the embedding of an enrolled reference you set up earlier. It is a progress-tracking cue — "how close does this recording sound to the reference, according to a neural speaker-embedding model?" — not speaker verification, not a judgement of the voice, and not a substitute for the acoustic measures on the rest of the dashboard.

Note: Speaker similarity is available only in the installed app. The browser version does not include it yet, so the tile, reference controls and Home similarity chart do not appear there.

Caveat: Speaker-similarity confidence is capped at moderate even in ideal conditions. Embedding models are sensitive to room acoustics, mic, and prosody; a cosine-similarity swing of ±0.05 between two takes of the same sentence is normal.

Setting up a reference

Enrolling a reference is a deliberate action. You can do it from the dashboard's reference controls, from the CLI, or through the local API:

uv run vocal-analyzer enroll-reference <reference.wav> --name <my-target>

The web UI shows enrolled references in the sidebar's Speaker reference selector when any exist, and the sidebar's reference section creates or deletes enrollments separately from the normal analyze upload flow. If no references are enrolled, the selector stays hidden and the tile does not render. The HTTP API exposes the same create/list/delete reference endpoints for local clients.

Enrollment JSON is stored locally under the app data directory (or the Docker vocal-data volume). It contains the embedding, model metadata, source path, source sha256, and timestamp; it does not retain a copy of the source reference audio after enrollment. Delete enrollments you no longer use, and treat the embedding file as sensitive local data.

See docs/neural/wespeaker.md in the repository for the full enrollment workflow and the supported embedding models, and see Storage and privacy for data lifecycle details.

Similarity over time on Home

If the WeSpeaker extra and model are available, completed web analyses save a private per-session embedding. After two compatible sessions, Home offers a Similarity chart toggle. It compares each compatible take with the earliest saved voice in the newest compatible model cohort, labels higher values as “more alike,” and explicitly keeps the series separate from target bands and gender scoring.

Sessions analyzed without the optional dependency or model remain valid and simply do not appear in this series. The API can also calculate the trajectory against one of your named enrolled references. Raw embeddings stay local and are not returned by the trajectory endpoint.

What the value means

The cosine similarity is a number between −1 and 1:

• 1.0 = identical embedding. The earliest point in the Home trajectory is exactly 1.0 because it is compared with itself; separate recordings rarely reach it.
• ~0.7–0.85 = typical high-similarity range for two recordings of the same voice saying different things.
• ~0.4–0.7 = moderate similarity — some embedding-model features overlap, some don't.
• Below ~0.4 = dissimilar. The embedding model thinks these are probably different voices or the recording conditions are too different to compare reliably.

These ranges are model- and corpus-dependent. Use the value as a relative tracking cue — "is this recording closer to the reference than my last one?" — rather than as an absolute match score.

Why it is not speaker verification

Speaker verification systems decide whether two recordings are from the same speaker. They are calibrated for security use cases — low false-accept rates, low false-reject rates, against adversarial inputs. Vocal Analyzer's similarity tile:

• Does not make a decision. It reports the raw cosine and leaves interpretation to you.
• Does not calibrate against attack types. A recorded reference played back at the mic will compare highly even though that is a different situation from actually matching the reference speaker.
• Is not robust to channel effects. A reference recorded on one mic and a new recording on another mic will score noticeably lower than two recordings from the same mic, regardless of how similar the voices are.

Supported backend

The similarity pipeline uses the WeSpeaker ResNet34 model (via ONNX Runtime) by default. The neural backend is loaded on first use; if the model is unavailable the similarity tile degrades to unavailable with a clear reason code.

Caveats

• Room and mic. Enroll and compare under similar conditions. Large environmental changes shift the embedding even for the same speaker.
• Short references. An enrolled reference shorter than a few seconds of voiced speech is unreliable; longer references give more stable embeddings.
• Do not infer identity. A high similarity score is not evidence that two recordings are legally or identity-linked. The tool is for voice-training progress tracking, not forensic use.

References

• Wang, H., Liang, C., Wang, S., Chen, Z., Zhang, B., Xiang, X., Deng, Y., & Qian, Y. (2023). WeSpeaker: A research and production oriented speaker embedding learning toolkit. ICASSP 2023.
• Snyder, D., Garcia-Romero, D., Sell, G., Povey, D., & Khudanpur, S. (2018). X-vectors: Robust DNN embeddings for speaker recognition. ICASSP 2018.