Whisper: OpenAI's AI that transcribes better than any human

AI audio transcription isn't new: Google, Amazon, Microsoft and others had offered services for a long time. But in September 2022, OpenAI released a model that changed everything: Whisper.

Three years on, Whisper remains the reference standard in transcription, used everywhere in the industry. This article explains why, how it works, and how to use it or spot it in the tools you use day to day.

Why Whisper landed like a bombshell

Before Whisper, AI transcription had three major problems: quality collapsed as soon as there was background noise, non-native accents were handled poorly, and each language needed a separate, specifically trained model.

Whisper solved all three at once thanks to a massive, original training approach. OpenAI trained the model on 680,000 hours of multilingual audio collected from the web, in very varied conditions: podcasts, YouTube videos, conferences, noisy audio, accents of every kind.

The result: a model that handles 99 languages, resists noise, and understands non-native accents almost as well as native speakers. And all of that with a single architecture, not one model per language.

The real-world quality on English

On "clean" audio in standard English (a podcast, a properly recorded conference), Whisper reaches a word error rate of around 4-6%. For comparison, a non-specialist professional human is around 4-5%. So Whisper is at a near-human level on that ground.

On harder audio (a WhatsApp voice note recorded in the street with wind, a fast delivery, a strong regional accent), quality drops, but stays largely usable. You're typically between 8 and 15% errors, which means you understand the meaning of the text perfectly despite a few approximate words.

The real limits

Whisper isn't magic. Three areas where it still falls short.

Rare proper nouns. If your voice note mentions "Hadj Hamou" or "Boulanger-Schmitt", Whisper will probably make a phonetic approximation. Not a disaster for general understanding, but frustrating if the whole point is to recover a name precisely.

Multi-speaker conversations. Whisper does no native "diarisation", meaning it doesn't distinguish who's speaking. If three people talk at once or a conversation switches quickly, you get continuous text with no indication of who said what. Third-party tools add that layer in post-processing.

Very long audio. Whisper has a limit of 30 seconds per processing "chunk". For a 2-hour recording, the tool splits it into segments and stitches them together. This technique works, but can produce inconsistencies at the joins, especially with proper nouns that end up transcribed differently across segments.

Whisper local vs API

Whisper has a peculiarity: OpenAI released the model's weights as open source. You can run it on your own computer if you have a decent graphics card, without sending any data to OpenAI.

This openness created a whole ecosystem: optimised forks (whisper.cpp for CPU, faster-whisper for GPU), integrations into open-source software, services that host their own instance.

If you use Whisper via OpenAI's API, it's billed at around $0.006 per minute of audio. For personal use, that stays affordable. For a large-scale service, it's worth evaluating a self-hosted instance.

Where you use Whisper without knowing it

Whisper has become so standard that many products use it in the background without mentioning it.

• WhatsApp's native transcription probably uses a Whisper variant on Meta's side (Meta has its own model, but one largely inspired by it).
• Many meeting tools (Otter, Tactiq, Fathom) use Whisper under the hood.
• Podcast tools (Riverside, Descript) build in Whisper for automatic transcription.
• Several voice assistants and accessibility tools use it.
• Abrège uses Whisper to transcribe WhatsApp voice notes before summarising.

If you use a modern transcription tool, there's a 70% chance there's Whisper in it.

Transcription vs summary

An important nuance: Whisper does transcription, not summarising. It turns audio into text word for word. For a 5-minute voice note, that produces around 800 words of raw transcription, which still takes time to read.

To turn that transcription into a usable summary (2-3 sentences that capture the essentials), you need a second step with a language model like GPT. It's this combination that delivers real efficiency in practical use: Whisper to transcribe, GPT to summarise.

That's exactly what Abrège does under the hood for WhatsApp voice notes: Whisper transcription + GPT summary. The user doesn't see the machinery, they just get a short, relevant summary.

Where is Whisper heading?

OpenAI released Whisper-large-v3 in 2023, an improved version with better quality on less common languages and better noise resistance. It then launched gpt-4o-transcribe and gpt-4o-mini-transcribe in 2025, models that perform even better for transcription, with a particular focus on English and European languages.

The evolution follows two directions: better quality on hard cases (noisy audio, strong accents), and deeper integration with language models for features like simultaneous translation or native summarising.

In short

Whisper democratised a technology that used to be expensive and limited. Today, anyone can transcribe audio at near-human quality for a few cents or for free. This commoditisation created a whole wave of smart products that use audio as raw material, and Abrège is one of them.

Knowing Whisper, its strengths and its limits, means better understanding what the tools you use can and can't do. Transcription has become a commodity. The value, now, is in what you do with it.