Speaker Labels in AI Transcripts: How Accurate Are They?

A transcript of an interview without speaker labels is a wall of text: you can read what was said, but not who said it. SubGrab's AI transcripts now mark each voice — Speaker 1, Speaker 2 and so on — whenever it hears more than one person. This post explains how that works and, more usefully, how often it's right, because we measured it.

What You Get

• In the transcript, the speaker's label appears wherever the voice changes, in its own colour.
• In the downloads: TXT names the speaker on every line ([00:12] Speaker 1: …), SRT names them where the speaker changes, and VTT uses the standard voice tag (<v Speaker 1>), which many players can show.
• Speakers are numbered in the order they first speak. They are numbers, not names — a voice doesn't tell anyone who a person is — so swap in the names yourself.
• A recording with one speaker gets no labels. A narrated tutorial labelled "Speaker 1" on every line is noise, so labels appear only when at least two voices each have real speech.

Labels come with every AI transcript at the usual 1 credit per started hour. Transcripts taken from a video's own captions don't have them, and neither do recordings over 5 hours.

How Accurate Are the Labels?

We measured them before shipping, against recordings where people had already marked who speaks when. Accuracy here is the share of speech time given to the right person.

The interviews came from published podcast transcripts that mark each speaker's turns; the meeting is from the AMI Meeting Corpus, whose speaker annotation is a research standard. These are a handful of recordings, run several times each — enough to show the pattern and the spread, not a guarantee for every file.

The pattern is the one you would expect. Two people taking turns is easy: the voices differ, and the questions and answers give it away. A meeting is hard: four people, interruptions, people talking over each other. On the meeting, one run split a single person into two speakers. That is why the transcript says the labels are added by AI, and why it's worth a quick check before you quote anyone.

How It Works

SubGrab first transcribes the audio with Whisper, which gives the words and their timestamps — the 95% accurate on clear English speech part. Then a second model, Google's Gemini, listens to the same audio alongside that transcript and marks where each speaker's turn begins. The words and timestamps are never changed; only the labels are added, and if that step fails for any reason you still get the transcript, without labels.

We tested several models. The one we use labelled two-person interviews almost perfectly; a cheaper one scored no better than guessing on the meeting, and asking for turns rather than a label per sentence made the meeting results much better.

When Labels Help Most

• Interviews and podcasts: quote the guest, not the host, with a timestamp. See our interview transcription workflow.
• X Spaces and panels: follow a long conversation without losing track of who is talking — see how to transcribe an X Space.
• Meetings: the labels make minutes faster to write, but check them — this is where they slip.

Try it on an interview — every new account gets 2 free credits.