Murmel v2: 30% fewer errors in Dutch speech-to-text

Murmel v2 is here, and it makes roughly 30% fewer errors than the first version. A few months ago I introduced Murmel, a Dutch speech-to-text model I built because the existing options weren't good enough at Dutch. The response was overwhelming: municipalities, universities, ministries and companies lined up to test it on their own audio. All that feedback helped improve the interface. Alongside that, I worked on collecting more data, improving the architecture and investing in more compute. On several widely used public benchmarks, v2 now ranks third overall against the best Dutch ASR systems, ahead of everything from OpenAI, Mistral, NVIDIA, Cohere and Qwen. The only two models still ahead of Murmel are heavily funded, proprietary frontier systems, and that's what I'm aiming at next.

A public benchmark, measured the same way for everyone

When I launched v1, I published my own benchmark on parliamentary audio. It was honest work, but it remained my benchmark, run by me. Readers rightly asked: how does this hold up when the model is measured against a shared, public standard?

These results answer exactly that. Every model runs through the same three standardised public Dutch test sets — FLEURS, VoxPopuli and Multilingual LibriSpeech — with the same evaluation pipeline, so the comparison is like-for-like on common data. One fair caveat applies to every benchmark like this: some models may be tuned on data that overlaps with the test sets, so read the numbers as indicative rather than definitive, and I always recommend verifying any number that matters to you on your own audio.

Where v2 stands

The average word error rate across the three benchmark datasets tells the story. Word error rate is the percentage of words the model gets wrong, so lower is better.

I'm proud to sit just behind the two models ahead of Murmel v2 — Resonate-1 and ElevenLabs scribe_v2, both from well-funded, respected companies. Murmel ranks third, ahead of Mistral's Voxtral, Cohere, NVIDIA's Parakeet, Qwen and OpenAI's Whisper-large-v3, which lands at the very bottom in tenth place with almost double Murmel's error rate.

How far v2 has come from v1

Compared to my own previous release, the improvement is consistent across every dataset. I evaluated v1 and v2 on the same three benchmarks with the same setup, so this is a clean, apples-to-apples comparison.

That's an average drop of roughly 3 percentage points — about a third fewer errors — with the biggest gain on VoxPopuli, the hardest of the three. And none of it costs speed: v2's real-time factor is around 0.006, meaning it transcribes an hour of audio in about twenty seconds of compute, comfortably faster than several of the proprietary systems ranked above it.

The data sources Murmel is tested on

A benchmark is only as trustworthy as the data behind it. The three test sets used here — FLEURS, VoxPopuli and Multilingual LibriSpeech — are established, publicly available datasets, each with a Dutch portion, and each stresses the model in a different way. That spread is what makes the average meaningful.

FLEURS (Few-shot Learning Evaluation of Universal Representations of Speech) is Google's multilingual benchmark, built on read-aloud sentences from the FLoRes translation set. The Dutch portion is relatively clean, well-articulated speech from a range of speakers. It comes closest to a best-case scenario, which is why error rates here are lowest, and it tells you how good a model is when the audio cooperates. Murmel v2 scores 4.67% here, a short distance behind the proprietary leaders.

VoxPopuli is a large corpus drawn from European Parliament recordings: real plenary debates and speeches. This is spontaneous, accented, sometimes overlapping political speech, recorded in a noisy chamber, often by non-native speakers. It's genuinely hard, which is exactly why it's the most relevant test for the public sector Murmel is built for. The high absolute numbers for every model reflect the difficulty of the material, not a weakness specific to Murmel, and v2's jump from 15.44% to 10.59% over v1 is the largest improvement in the whole comparison.

Multilingual LibriSpeech (MLS) is the Dutch portion of a large corpus derived from LibriVox audiobooks — people reading published books aloud. It sits between the other two: cleaner than a parliamentary debate, but with longer, more varied sentences and a wide range of voices and recording conditions. It's a good measure of how a model handles sustained, narrative speech. Murmel v2 scores 6.25% here.

Together these three cover a useful spread: clean read speech (FLEURS), messy real-world institutional speech (VoxPopuli) and long narrative speech (MLS). A model that improves on all three at once improves in a real, generalisable way, rather than overfitting to one style.

One fair caveat, the same one I gave with v1: these are all relatively prepared or read forms of speech. Fully spontaneous audio — a medical consultation, a field interview, a call centre recording — remains an active area of work, and thorough evaluation on those domains is something I'm happy to take on with the right partners.

Built in Europe, for Europe

The result reinforces the idea behind Murmel from day one: a purpose-built Dutch model beats general-purpose alternatives, and the gap grows with every training round. On a public benchmark, the only two models still ahead of Murmel are well-funded, proprietary frontier systems, and v2 closed roughly a third of its remaining error rate in a single iteration.

Murmel runs on infrastructure in the Netherlands, designed for high-volume transcription pipelines (increasingly a requirement rather than a preference in the public sector). Since the launch of v1, more than 500 users have signed up to transcribe their Dutch audio with Murmel, and that feedback is what made v2 better.