Why a three-hour interview comes back with eleven speakers

You record a three-hour panel. Four people on stage. You upload it, wait, and get back a transcript with eleven speakers in it. Speaker 3 in the first hour is Speaker 7 by the third. One person has been split into three. Two people have been merged into one.

This is the single most common complaint about automatic transcription of long recordings, and the explanation you usually get — "the model struggled with your audio" — is mostly wrong. The audio is often fine. What happened is structural, and once you know the shape of it you can work around most of it.

Speaker labels are clustering, not recognition

Start with what the feature actually is. Speaker separation — diarization, if you want the technical word — does not identify people. It has never heard of you. What it does is take every stretch of speech, turn each one into a vector that encodes what that voice sounds like, and then group the vectors that sit close together. Each group gets a number. That's it.

So "Speaker 1" doesn't mean a person. It means cluster one, in this request. The numbering is an artifact of the sorting, not a name.

This matters because of what comes next.

The twenty-minute wall

If you want a timestamp on every individual word — not just on each paragraph — most batch transcription systems put a ceiling on how much audio they'll do it for in one go. In our case, the engine returns word-level offsets only for audio up to about twenty minutes. Past that, you can have the words or you can have the timings, but not both.

Word-level timings are not a nice-to-have. They're what makes a transcript checkable: click a line, hear that second. Without them you have a wall of text that claims to be what was said and gives you no way to confirm it. So we keep the timings, which means every long file has to be cut into pieces.

A three-hour recording is roughly nine pieces. Nine separate requests.

Nine requests, nine sets of numbers

Here's the part that produces eleven speakers.

Each piece is clustered on its own. Piece one finds four voices and calls them 1, 2, 3, 4. Piece two also finds four voices and also calls them 1, 2, 3, 4 — but its numbering has no relationship to the first piece's. The model isn't confused. It was never asked the question. Nobody told it that piece two is a continuation of piece one, because as far as the API is concerned, it isn't.

Stitch the pieces together naively and you get exactly what people report: speaker numbers that reset, people who change identity at regular intervals, a count that climbs with the length of the recording. The drift isn't gradual, either — that's a tell. It happens at the seams. If you look at where a transcript's speaker labels go wrong and the answer is "every eighteen minutes or so", you're looking at slice boundaries.

What we do about it

Two things, and neither is exotic.

Overlap the slices and match across the seam. Rather than cutting the audio into clean non-overlapping pieces, the pieces share some audio at each boundary. The same speech gets transcribed twice, once at the end of one piece and once at the start of the next. That shared region is a Rosetta stone: whoever is Speaker 2 in the overlap of piece one is the same voice as whoever is Speaker 4 in the overlap of piece two, so the labels can be mapped onto each other and carried forward.

Put the cut in the widest pause. Slicing at exactly eighteen minutes will sometimes land in the middle of a word, which gives both pieces a fragment to misread and often invents a speaker at the boundary. So the cut goes looking for the longest silence near where it wants to land and cuts there instead. A cut in a pause is a cut nobody hears.

Together these fix most of it. Not all of it.

Where it still fails, and why we'd rather say so

The overlap trick needs someone to speak in the overlap. If a participant says nothing for twenty minutes — the fourth person on the panel who only answers questions at the end, the client who listens through the whole demo — there's no shared speech to match them on. When they speak again, they're a new cluster, and a new number.

Voices that genuinely sound alike get merged, and a single person on a bad connection can drift far enough from their own baseline to split. Heavy crosstalk degrades everything, because a vector that encodes two people talking at once doesn't sit near either of them.

And 12 of the 24 language variants we offer don't have speaker separation at all. Those transcribe with full timings and punctuation, but everything arrives attributed to one speaker. We list which is which on the pricing page rather than burying it, because finding out after you've uploaded a two-hour recording is worse than knowing first.

What actually helps

In rough order of how much difference it makes:

Rename speakers once, early. In txscribe, renaming a speaker propagates through the whole transcript, so the fix is one edit rather than three hundred. Do it in the first few minutes, where you can still tell who's who from context, and the rest of the document inherits it.

Record people on separate channels if you can. Nothing in software competes with physically separating the voices. A remote interview recorded locally on both ends, or a multitrack field recorder, makes the whole problem disappear — there's nothing to cluster.

Name the language instead of leaving it on auto. Auto-detection costs accuracy, and in bilingual recordings it can go badly wrong in a way that's worth its own post.

Don't judge the labels by the count. A transcript with nine speaker labels for four people is four renames away from correct. A transcript with four labels that merged two quiet participants into one is harder to fix, because you have to find every boundary yourself.

The honest version

Every transcription product has this problem, including the ones that don't mention it. It's a consequence of how the APIs are shaped, not a difference in how much anyone cares. What differs is whether a product is built on the assumption that its output is final.

We assume ours isn't. That's why renaming propagates, why editing a line doesn't destroy the timings, and why you can click any sentence and hear the audio behind it. A transcript you can correct in five minutes beats a transcript that's slightly better out of the box and traps you in a text box.