November 04, 2026

Whisper, Deepgram, Who Keeps the Audio?

Somewhere there's a folder of audio waiting to become words- a meeting, an interview, a podcast, voice notes piling up on a phone.. Two names keep coming up for that job, OpenAI's Whisper and Deepgram, usually in a head to head that lands on "it depends on your needs".

The muddle starts a step before that verdict, since "Whisper" means three different things right now and Deepgram sells one of them too.

Three Whispers

Whisper the download is code plus model weights, both under the MIT licence, free to run on our own machine with no account and no meter. The audio stays on the machine doing the work.

Whisper the API is OpenAI's hosted copy, whisper-1, scheduled for removal on February 26, 2027. The replacements it names are gpt-transcribe at $0.0045 a minute and gpt-live-transcribe at $0.017 a minute. A shutdown of the API doesn't reach the download.. the licence and the weights already sitting on our drive stay put.

Whisper the rental is everybody else hosting those same weights- Groq, fal.ai, Together, Replicate... and Deepgram itself, with a "Whisper Cloud" priced at $0.0048 a minute. Deepgram's docs don't oversell their own listing:

All non-Whisper models will return results faster and scale to higher load.

Pay as you go customers get five concurrent Whisper requests on that service, against 150 concurrent streams for its regular speech-to-text.

Running It for Free

The openai-whisper package wants Python plus ffmpeg, the command line tool that handles audio formats. On a Mac with Homebrew it comes down to three lines:

brew install ffmpeg
pip install -U openai-whisper
whisper audio.mp3 --model turbo

VRAM, the memory on the graphics card, is where the cost lands- the project's own table runs from about 1 GB for tiny to about 10 GB for large, with turbo at about 6 GB and roughly eight times the speed of large. For English-only audio the .en models tend to do better, most of all at the small end. turbo isn't trained for translation either, so it hands back the original language even when told to translate.. medium or large do that job.

Smaller machines have two other doors, both free. whisper.cpp runs the tiny model in about 273 MB of memory and treats Apple silicon as a first-class citizen. faster-whisper claims up to four times the speed of the original at the same accuracy with less memory- that's the project's own number, on its own benchmark. For a quick note with nothing to install at all, dictation.io in Chrome still does the trick, though Whisper plays no part in that one.

Live Audio Is the Awkward Bit

Whisper's command line works on finished files. Words appearing while someone is still talking takes something built on top. The whisper.cpp README describes its own whisper-stream tool as "a naive example of performing real-time inference", sampling the microphone every half a second. The build steps for it are in Talk to It, Don't Type. WhisperLive goes further, wrapping faster-whisper in a server that several clients can connect to.. all of it ours to run and babysit.

Deepgram built its product around that live case. Nova-3 streams at $0.0077 a minute in a single language, and $0.0092 multilingual. Its Flux model costs the same for English and adds end-of-turn detection, meaning the model itself decides when the caller has finished talking. That matters for the relay style of voice agent, where a pause before every answer is already baked in.

The Numbers Don't Agree

Deepgram's docs say Nova-3 delivers a 54.2% reduction in word error rate on streaming and 47.4% on batch "compared to competitors". Artificial Analysis runs its own index over about eight hours of audio. For finished files, that index doesn't back the claim up against Whisper. Lower is better on the error rate, and speed means seconds of audio handled per second:

Whisper Large v2 on OpenAI- 4.1% error, 28.5x speed, $6.00 per 1,000 minutes
Whisper Large v3 on fal.ai- 4.1% error, 100.4x speed, $1.15 per 1,000 minutes
Whisper Large v3 on Together- 4.5% error, 274.3x speed, $1.50 per 1,000 minutes
Nova-3 on Deepgram- 5.2% error, 561.2x speed, $4.30 per 1,000 minutes
Whisper Large v3 on Replicate- 10.1% error, 2.9x speed, $4.23 per 1,000 minutes

Nova-3 trails the better Whisper hosts on errors by about a point while running rings around them on speed. The same large-v3 model scores 4.1% on one host and 10.1% on another, so the host counts for as much as the name on the model! Neither of them leads the index either.. the top spot among its 62 models sits at 1.7%. Those are all batch figures, which leaves Deepgram's streaming claim untested here.

Who Keeps the Audio

Running Whisper locally answers the privacy question by itself, as long as the machine is ours. Deepgram's docs say its cloud stores "fractional increments of data" for model improvement under a program it calls the Model Improvement Partnership. The way out is a parameter added to each request we want left out of it, with that data then kept only as long as the request takes to process:

curl \
  --request POST \
  --header 'Authorization: Token YOUR_DEEPGRAM_API_KEY' \
  --header 'Content-Type: audio/wav' \
  --data-binary @youraudio.wav \
  --url 'https://api.deepgram.com/v1/listen?mip_opt_out=true'

Calling Deepgram cloud-only doesn't hold up either, since it self-hosts on NVIDIA GPUs, on its Enterprise plan through Docker or Kubernetes, or through an Amazon SageMaker listing that needs only an AWS account.

Whisper Invents Things

A 2024 study titled Careless Whisper ran 13,140 audio segments through OpenAI's hosted Whisper in 2023 and found roughly 1% of the transcriptions contained whole phrases or sentences nobody said. Of those, 38% carried explicit harms- violence, made-up associations, false authority. Very, very bad news for anyone feeding a transcript into a medical chart or a legal file. The hallucinations landed disproportionately on speakers with long pauses, a common symptom of aphasia, a condition that makes speech harder. When the researchers re-tested in December 2023, 12 of the 187 troublesome segments still hallucinated... many had cleared up after an update, not all. Google, Amazon, Microsoft, AssemblyAI and RevAI produced nothing comparable on those same 187 segments. Deepgram wasn't among the services tested, so the paper says nothing either way about Nova-3.

Whisper has a flag that skips long silent stretches when a hallucination looks likely.. word timestamps have to be switched on for it to work. The 2 here is seconds, a starting number to play with:

whisper audio.mp3 --model turbo --word_timestamps True --hallucination_silence_threshold 2

faster-whisper does a similar job with a built-in voice activity filter, vad_filter=True, cutting the parts with no speech before the model ever hears them.

What It Comes To

Deepgram does cost money eventually, though not at the start.. it hands out $200 of credit with no credit card and no expiry, covering roughly 43,000 minutes of Nova-3 by its own count, with pre-recorded audio running $0.0043 a minute. Napkin math, ours: an hour a day for a thirty day month is 1,800 minutes, so $7.74 for finished files or $13.86 streaming. Speaker labels add $0.0020 a minute on top, and a ten minute stereo file bills as twenty minutes since each channel counts.

Local Whisper has no meter, just the hardware and the power bill.. the sums only favour it when a capable machine is already sitting there.

That free credit plus a free download makes the honest test cost nothing. Take the worst audio we actually deal with, five minutes of crosstalk or a bad phone line, run it through both, then read the results against the recording by hand. Watch the names, the numbers and the quiet stretches, since a leaderboard score won't warn us about any of those.

Has Whisper ever put words in somebody's mouth in your own recordings? What did it make up, and did anybody catch it? Leave a comment, would love to hear 'em.

Glossary

Aphasia — a condition, often after a stroke, that makes producing or understanding speech harder
Batch / pre-recorded — transcribing a finished audio file, as opposed to a live stream
End-of-turn detection — working out when a speaker has finished, so a voice agent knows when to answer
Hallucination — text in a transcript that was never spoken in the audio
Model weights — the trained numbers that make up the model, the part downloaded to run it
Speaker labels (diarization) — marking which person said which line
VAD (voice activity detection) — finding which parts of the audio contain speech
VRAM — memory on a graphics card
Word error rate (WER) — the share of words a transcript gets wrong

Sources