A Systematic Way to Fix Bad Auto-Captions

Automatic captions arrive mostly correct, and the incorrect part is never distributed evenly. It sits on the names, the product terms, the numbers, and the moment where one person starts talking before the other has finished. That distribution is the useful fact: errors are predictable enough that you can go looking for them instead of reading a thousand lines hoping something jumps out.

What makes the job harder than it sounds is that recognition failures are grammatical. The system returns a fluent, plausible sentence that happens to be the wrong one, so proofreading instincts trained on typing errors find nothing. This guide sets out the five classes of error worth hunting separately, the order to correct them in, and the preparation that stops most of them from appearing in the first place.

Why It Fails Where It Fails

A recognition pass is choosing the most likely sequence of words for the sound it received. Two things follow from that, and they explain nearly every error you will see.

Improbable words lose. A surname, a product name or a piece of domain jargon is rare in ordinary language, so a common word that sounds similar will usually win the comparison. The system is not failing to hear it; it is deciding against it.

Context arrives late. The model resolves ambiguity using surrounding words, which means the first words of a sentence, the first line after a long pause, and anything spoken over another voice all get less context to work with. That is why openings and interruptions are disproportionately wrong.

Audio quality sets the ceiling for all of it. A recording with room reflection, a distant microphone or a compressor pumping on background noise gives the pass less to work with, and no amount of correction afterwards recovers what was never captured. Where the source is repairable, cleaning it before transcription pays for itself — the sequence for that is in the order to clean up bad audio.

The Five Error Classes

Treat these as five separate passes rather than one read-through. Each has a different search method, and that is what makes the work fast.

Substitution is the one that reaches publication. The others announce themselves by looking wrong; a substituted word looks entirely normal and is only detectable by someone who knows what was actually said.

Correct in One Direction

The order matters because each stage invalidates the one before it. Change the wording after you have set the cue breaks and the breaks are now in the wrong places. Retime before you have finished condensing and you will retime twice.

The measurements that govern the last two steps — characters per second, minimum duration, where a line may break — are covered in the timing rules that decide whether subtitles are readable.

A Vocabulary List That Earns Its Keep

Most transcription tools accept a list of terms to bias the pass toward. It is the highest-leverage thing in this whole process and the most commonly skipped, because it happens before the work rather than during it.

Keep the list short and specific. Feeding in a hundred general words dilutes it; twenty genuinely unusual ones — the client's name, the product, the two people speaking, three pieces of jargon — measurably changes the output. Include the spellings you want, not just the words, because a name with two accepted spellings will otherwise arrive in whichever form the model prefers.

Maintain the list per project and reuse it across the series. A recurring podcast, a course, or a channel with consistent subject matter builds a genuinely valuable list within a few episodes, and it keeps paying out on every subsequent one. Where the same content will also be translated, that list becomes the seed of a terminology glossary, which is the thing that stops eight language versions from disagreeing about what the product is called — the surrounding process is in managing subtitles across many languages.

Speakers, Overlaps and Crosstalk

Automatic speaker separation works well on clean turn-taking and degrades quickly when people talk over each other, which is exactly when the labels matter most.

Check the seconds around every speaker change. The common failures are a short interjection being absorbed into the previous speaker's block, and a label flipping for a single cue in the middle of an otherwise correct run. Both are quick to fix and both are invisible in the text alone — you have to look at the label column.

Where speech genuinely overlaps and cannot be separated, do not attempt to caption both simultaneously. Caption the speaker who carries the meaning and, if the other voice matters, indicate it rather than transcribing it. Two competing lines on screen are less useful than one line plus an indication that someone interrupted.

Numbers, Units and Punctuation

Numbers deserve their own pass because they are wrong in a way that reads as correct. A model may render the same spoken figure as digits in one cue and as words in the next, drop a decimal point, or convert a unit into a homophone.

Decide the house rules up front and apply them consistently: when digits are used rather than words, how times and dates are written, whether units are spelled out. Consistency matters more than which convention you choose, because inconsistency is what a reader notices.

Punctuation is where the pass makes its most confident mistakes. Missing question marks are the frequent one, since a rising intonation is not always detected, and a question rendered as a statement changes how the whole exchange reads.

When Repair Is the Wrong Call

Sometimes the pass has not given you a draft, it has given you a rough time map with words attached. The signal is qualitative rather than numerical: if you are rewriting whole sentences rather than fixing words, the transcript has stopped being a starting point.

At that stage the timing data is still worth keeping. Retype the text against the existing cue boundaries rather than starting a blank file, so you keep the segmentation and only replace the content. And treat it as a recording problem to solve at the source, because a second video from the same setup will produce the same result.

A related decision arrives at the same moment: whether the captions should match the audio exactly or match a cleaned-up version of it. If the plan is to cut the stumbles out of the video, the captions follow the edit rather than the raw take, and the two jobs should be done together — the technique for that is in removing filler words without chopping the speech.

Next Steps

Take the last automatic transcript you accepted without a full review and run one search: every proper noun you expect to be in it. Count how many the search fails to find. That number is a fair estimate of what has been published with errors in it.

Then write the vocabulary list you should have supplied and keep it with the project files. The next episode starts with fewer errors, and the pass after that starts with fewer still. The rest of the caption pipeline — timing, styling, and how the file is delivered — sits under captions and subtitles.