How Accurate Is Whisper AI? WER, Real Limits, and Tips (2026)

By whispertranscribe.ai Editorial Team · Last updated: November 2026 · 13 min read · Category: Transcription accuracy

How accurate is Whisper? On clean, read English, OpenAI's paper reports a 2.5% word error rate (WER) for Whisper large-v2, so about 97 to 98 words in 100 are right. Harder audio scores worse: 13.6% on phone calls, 24.9% at a noisy dinner party, and 39.9% in a meeting recorded by one distant microphone.

Whisper is OpenAI's open-source speech recognition model and the engine behind many transcription apps, including ours. This guide covers accuracy only: what WER measures, what the benchmarks say by audio type and language, where Whisper fails, and how to test it yourself. For the model's background, see our explainer on what Whisper AI is.

Most figures come from the Whisper paper (Radford et al., 2022), which tested every model "zero-shot", with no fine-tuning on the test sets. The newer large-v3 model card reports 10% to 20% fewer errors than large-v2 across many languages, but without a full per-dataset table. whispertranscribe.ai publishes this article. We didn't run a benchmark for it, and no number here is our own test result.

Whisper accuracy at a glance

No single test set answers the question, because the audio moves the number more than anything else. The table shows OpenAI's results for large-v2 with beam search and temperature fallback (appendix Table 9), scored after OpenAI's text normalizer.

Audio type Test set in the paper Whisper large-v2 WER Rough share of words right
Audiobook, clear reader LibriSpeech test-clean 2.5% About 97.5 in 100
Read news text WSJ 2.6% About 97.4 in 100
TED talks TED-LIUM 3 3.7% About 96 in 100
Audiobook, harder speakers LibriSpeech test-other 4.9% About 95 in 100
Volunteers on home microphones Common Voice 5.1 8.2% About 92 in 100
Phone conversations Switchboard 13.6% About 86 in 100
Long-form sociolinguistic interviews CORAAL 14.2% About 86 in 100
Phone calls between family and friends CallHome 16.4% About 84 in 100
Meetings, each person on a headset mic AMI IHM 17.4% About 83 in 100
Dinner parties, far-field mics CHiME-6 24.9% About 75 in 100
Meetings, one distant table mic AMI SDM1 39.9% About 60 in 100

"Share of words right" is 100 minus WER, a shortcut, since WER also counts added words.

You'll often see 2.7% quoted instead: that's the same model with simpler greedy decoding (the paper's Table 2). Also note the AMI rows are the same meetings recorded two ways. Moving the mic from each speaker's head to the middle of the table more than doubles the error rate.

What is word error rate (WER), and how is it calculated?

Word error rate is the share of words a transcript gets wrong compared with a human-checked reference. Count three kinds of error and divide by the words in the reference:

WER = (substitutions + deletions + insertions) ÷ words in the reference

  • Substitution: a wrong word in place of the right one ("lunch" for "launch").
  • Deletion: a spoken word that's missing from the transcript.
  • Insertion: a word in the transcript that nobody said.

Here's a self-made example, scored with the open-source jiwer library:

Line Text
Reference (22 words) we moved the product launch to march ninth because the vendor missed two deadlines and doctor patel wants a new test plan
Transcript we moved the product lunch to march ninth because vendor missed two deadlines and doctor battle wants a new test plan today
Errors 2 substitutions (lunch, battle), 1 deletion (the), 1 insertion (today)
WER 4 ÷ 22 = 18.2%

Three things about WER catch people out:

  1. It can go above 100%. Insertions have no ceiling, and a few low-resource languages in the paper's Fleurs results score above 100%.
  2. Normalization changes the score. "Dr." versus "doctor" counts as an error unless both texts are cleaned the same way. OpenAI says its normalizer cut WER by up to 50% on some datasets, mostly by removing formatting differences.
  3. Some languages use character error rate (CER). For Chinese, Japanese, Thai, Lao, and Burmese, which don't separate words with spaces, the paper scores characters instead.

An accuracy claim without a named test set and scoring method isn't a measurement, whoever makes it, us included.

How accurate is Whisper in English?

Whisper is very accurate on clear English and stays usable on messy English. In the paper, it made 55.2% fewer errors across 12 other English datasets than a supervised wav2vec 2.0 model with almost the same LibriSpeech score.

Long recordings

Whisper reads audio in 30-second windows, so the paper tested long files separately (appendix Table 16, large-v2):

Long-form test set What's in it WER
TED-LIUM 3 11 full TED talks 3.5%
Meanwhile 64 late-night TV segments, heavy on unusual words 5.1%
Kincaid46 46 mixed recordings from a 2018 comparison 8.8%
Earnings-21 Company earnings calls 9.7%
Rev16 16 podcast episodes 11.3%
Earnings-22 More earnings calls, many accents 12.6%
CORAAL 231 full interviews in African American English 19.6%

Whisper versus human transcribers

OpenAI had 25 Kincaid46 recordings transcribed by four human services and one computer-assisted service. The computer-assisted service scored 1.15 percentage points better than Whisper, and the purely human services only a fraction of a point better. The paper's verdict: "not perfect but very close to human-level accuracy."

Model size matters

Whisper comes in several sizes, and apps don't always say which one they run (appendix Table 9):

Model Parameters LibriSpeech clean Common Voice 5.1 Switchboard CHiME-6
tiny 39M 6.7% 26.1% 18.3% 48.0%
base 74M 4.9% 19.4% 15.6% 40.0%
small 244M 3.3% 12.8% 13.3% 30.9%
medium 769M 2.7% 9.7% 13.2% 27.6%
large-v2 1550M 2.5% 8.2% 13.6% 24.9%

The gap grows as audio gets harder: about 4 points on clean audiobooks, 23 points at the dinner party. The newer turbo model, an optimized large-v3, accepts what OpenAI's GitHub README calls a "minimal degradation in accuracy" for much faster transcription.

How accurate is Whisper in other languages?

Outside English, accuracy depends mostly on how much training audio Whisper had in that language. The paper found a squared correlation of 0.83 between log training hours and log WER, and estimates WER halves for every 16-fold increase in data. Most of the 96 non-English languages had under 1,000 hours.

Selected Fleurs results (read sentences, large-v2, appendix Table 13):

Language Fleurs WER Note
Spanish 3.0% Lowest in the table
English 4.2%
Portuguese 4.3%
German 4.5%
Japanese 5.3% Character error rate
Russian 5.6%
French 8.3%
Korean 14.3% Flagged as an outlier
Chinese 14.7% Character error rate
Arabic 16.0%
Hindi 21.5%
Hebrew 27.1% Flagged as an outlier
Swahili 39.3%
Telugu 99.0% Flagged as an outlier

The paper flags Hebrew, Telugu, Chinese, and Korean as doing worse than their training data predicted, possibly because of linguistic distance, a tokenizer that fits those scripts poorly, or data quality. Fleurs is read speech, so expect higher error rates on conversation. The README has a newer chart for large-v3 and large-v2 on Common Voice 15 and Fleurs.

Spanish results are encouraging if you transcribe Spanish audio to text. For lower-resource languages, test a short clip before committing a whole project.

What affects Whisper's accuracy?

Recording conditions matter more than anything else. Evidence from the paper and model card:

Factor What the evidence shows What to do
Background noise Under pub noise below 10 dB signal-to-noise ratio, Whisper beat all 14 comparison models, but every model got worse as noise rose Record in a quiet room, away from fans and music
Microphone distance Same AMI meetings: 17.4% on headset mics, 39.9% on one table mic Keep the mic close to whoever is talking
Conversation and crosstalk Phone calls and dinner parties score 13.6% to 24.9%, against 2.5% for read speech One speaker at a time; one mic per speaker
Accents and dialects The model card reports uneven accuracy across accents and dialects; long CORAAL interviews scored 19.6% Test with speakers who sound like your real recordings
Rare words and jargon Good on the jargon-heavy Meanwhile set (5.1%), yet names and technical terms remain common errors Check names and terms against a list
Language resources Fleurs WER runs from 3.0% (Spanish) to 99.0% (Telugu) Choose the language yourself; plan more review for low-resource languages
Model size tiny scores 48.0% on CHiME-6, large-v2 24.9% Ask which model a tool runs, or use the largest you can
File format and sample rate Whisper's code resamples all audio to 16 kHz mono A 48 kHz WAV gains nothing over a clean 16 kHz file

Converting an MP3 to a big WAV doesn't restore lost detail. A clean recording at a modest bitrate beats a noisy one at a high bitrate.

Known Whisper problems: hallucinations, loops, and timestamps

OpenAI documents each of these failure modes, which matter more than their share of WER suggests.

Hallucinated text

Whisper can write words nobody said. The model card says predictions "may include texts that are not actually spoken in the audio input," likely because the model also predicts probable next words. The paper's limitations section even mentions "complete hallucination," output unrelated to the audio.

An independent study, Careless Whisper (Koenecke et al., 2024), ran 13,140 audio segments through OpenAI's Whisper API in April and May 2023. About 1% of transcriptions (1.4% on average) contained whole invented phrases or sentences, and 38% of those included harmful content such as violence or false associations. Speakers with longer pauses, here people with aphasia, triggered more. The API has changed since, so treat this as a 2023 measurement.

Repetition loops and dropped words

The model card says Whisper is "prone to generating repetitive texts," which beam search and temperature scheduling reduce "but not perfectly." The paper also lists skipping the first or last words of a segment. Its decoding fixes cut average long-form WER from 11.0% to 10.0%: better, not solved.

Silence, music, and filler

Invented text tends to appear in long silences and music. The paper found Whisper's own "no speech" signal unreliable alone, and the open-source code now has an option to skip long silences when a likely hallucination is detected.

Timestamp drift and long files

Whisper moves through long audio one 30-second window at a time, using its own predicted timestamps to place the next window, so the paper notes a mistake in one window can carry into the next. Word-level timestamps are marked "experimental" in the code. For subtitles, spot-check timing around pauses.

During development, OpenAI also saw Whisper guess speaker names that were "almost always incorrect." The released models were fine-tuned to curb this, but check any name in the text.

10 ways to improve Whisper transcription accuracy

Most gains come before you press transcribe. Ordered roughly from biggest effect to smallest:

  1. Put the microphone close. Distance alone more than doubled the error rate in the AMI meetings.
  2. Cut noise at the source. Close windows, turn off fans, pause music.
  3. Avoid crosstalk. One person speaking at a time beats any setting.
  4. Trim long silences and music intros, where invented text most often appears. Cut at pauses, not mid-sentence.
  5. Set the language yourself. Whisper identified the language correctly 80.3% of the time across 82 Fleurs languages, so auto-detect can miss on short or accented clips.
  6. Use the largest model available, especially for noisy recordings.
  7. Prompt with names and terms if you self-host. Open-source Whisper accepts an --initial_prompt with the spellings you expect.
  8. Upload the original file. Re-recording through speakers or re-compressing repeatedly throws detail away.
  9. Run a term check. Search for every name, number, date, and acronym, and verify it against the audio.
  10. Review in two passes. First fix errors at normal speed. Then scan for hallucination signs: repeated lines, sentences over silence, sign-offs like "Thanks for watching." For legal, medical, or published work, have a person review everything.

How to measure Whisper's accuracy on your own audio

The best estimate for your work is a 30-minute test on your own recordings. These steps are for you to run; they aren't a whispertranscribe.ai test.

  1. Pick three 2-minute clips like your real work: one easy, one typical, one hard.
  2. Write a reference transcript by hand, word for word. Set rules first: keep or drop "um," digits or words for numbers.
  3. Transcribe each clip with the tool you're testing and save plain text.
  4. Normalize both texts the same way. OpenAI's normalizer ships with openai-whisper.
  5. Calculate WER with jiwer:
# pip install jiwer openai-whisper
import jiwer
from whisper.normalizers import EnglishTextNormalizer

norm = EnglishTextNormalizer()
ref = norm(open("reference.txt", encoding="utf-8").read())
hyp = norm(open("transcript.txt", encoding="utf-8").read())

out = jiwer.process_words(ref, hyp)
print(f"WER {out.wer:.1%}  S={out.substitutions} D={out.deletions} I={out.insertions}")
  1. Count critical errors separately. Five wrong fillers and five wrong client names give the same WER but aren't the same problem.
  2. Log results so you can compare tools or settings:
Clip Conditions Reference words S D I WER Critical errors
Clip A Quiet room, one speaker 310
Clip B Video call, two speakers 285
Clip C Café, accented speaker 296

Word counts are placeholders. A caution on step 4: when we ran OpenAI's English normalizer on a self-made sentence, it matched "Dr." to "doctor" and "two" to "2" but still scored "9th" against "ninth" as an error. Read flagged differences before trusting the score.

How accurate is whispertranscribe.ai?

whispertranscribe.ai is a web-based AI transcription tool powered by Whisper that supports 130+ languages, with uploads up to 5GB and 10 hours. Accuracy reaches up to 99% on clear audio; background noise, strong accents, and crosstalk lower it, as the benchmarks above show for Whisper itself. We don't claim to match or beat the paper's figures, and we haven't published our own benchmark, so test your own clips with the steps above.

You can start free with no account required. Paid plans start from $9.99/month billed annually; see pricing.

FAQs

Is Whisper 99% accurate?

Not in general. Whisper's best published result is a 2.5% word error rate on LibriSpeech test-clean, a set of clearly read audiobooks, which is about 97.5% of words correct. On phone calls, meetings, and noisy rooms, OpenAI's own figures range from about 13% to 40% WER. A "99%" claim only means something if it names the audio it was measured on. Tools built on Whisper, including whispertranscribe.ai, should be judged on your recordings, not on a headline percentage.

Which Whisper model is the most accurate?

The large models are the most accurate. In OpenAI's paper, large-v2 had the lowest error rates on most test sets, and the large-v3 model card reports 10% to 20% fewer errors than large-v2 across many languages. The turbo model is an optimized large-v3 that OpenAI says loses only a little accuracy in exchange for much more speed. The smallest model, tiny, is far less accurate on hard audio: 48.0% WER at a noisy dinner party, against 24.9% for large-v2.

Why does Whisper write words that weren't said?

Whisper can hallucinate because it's trained to predict likely text as well as transcribe sound, and OpenAI's model card warns that it may output text that isn't in the audio. It happens most often during long silences, music, or noise. A 2023 independent study found hallucinated phrases in about 1% of transcriptions, and pauses made them more likely. Trimming silence, setting the language, and reviewing quiet stretches of the transcript reduce the risk.

How accurate is Whisper with accents?

Whisper handles accents better than many older systems, but not evenly. OpenAI's model card says accuracy varies across accents and dialects of the same language. In the paper, long interviews in African American English (CORAAL) scored 19.6% WER, compared with 3.5% for full TED talks, though recording conditions differ too. The reliable approach is to test a clip with speakers who sound like your real recordings, and to plan for more review when accents are strong.

Is Whisper as accurate as a human transcriber?

For English, Whisper comes close but doesn't match the best. OpenAI compared it with five professional services on 25 recordings. A computer-assisted service had an aggregate WER 1.15 points lower than Whisper's, and four fully human services were a fraction of a point better. Humans still tend to handle names, crosstalk, and unclear speech better. For anything quoted, filed, or published, a person should review the transcript against the recording.

How accurate is Whisper for languages other than English?

It varies widely by language. On the Fleurs benchmark, large-v2 scored 3.0% WER in Spanish, 4.5% in German, and 8.3% in French, but 27.1% in Hebrew and 99.0% in Telugu. Chinese and Japanese are scored by character error rate: 14.7% and 5.3%. The paper found that accuracy tracks how much training audio each language had. For lower-resource languages, test a short clip first and budget more time for corrections.

Test accuracy on your own recording

Benchmarks describe other people's audio. Upload a clip to whispertranscribe.ai in any of 130+ languages and score it with the steps in this guide. Start free, no account required, or compare plans.

Related articles: What is Whisper AI? · Best free transcription software · Best TurboScribe alternatives · How to transcribe a YouTube video · What is an SRT file? · Hard subtitles vs soft subtitles

References

Ready to turn your recordings into text?

Transcribe audio and video with whispertranscribe.ai.

Start transcribing ↗