First test results with users: Better than Mistral on Dutch. And it understood a mumbling interviewee.
Last month I launched Murmel V1, a Dutch speech-to-text model built and hosted entirely in the Netherlands. Since then, I opened up the API, SDK and Studio to users across a wide variety of sectors like healthcare, technology, and the cultural sector.
The feedback has been great!
It understood "murmelen"
One user shared something that stuck with me. They'd been transcribing an interview where the subject was mumbling: not occasionally, but throughout the entire recording. The kind of audio that makes you wince when you press play.
Murmel got it correctly. Of course the name implies it is good in this situation, but it is great the first model is already delivering on the promise to real users.
It beat Mistral on Dutch
A second user was running a proper evaluation before committing to a production integration. They built a custom benchmark and lined Murmel up against Voxtral from Mistral, and tested them head to head on their use case.
Murmel won with a better WER-score than the competitor! As a big fan of Mistral that was very cool to hear.
Fast enough for real conversations
Accuracy matters. So does speed, especially if you're a developer building something where a user is actually speaking and waiting for a response.
We noticed that short audio clips were sometimes slower than they should have been. The model itself was fast. The infrastructure around it was adding unnecessary lag. So for the last week this has been a point of improvement, and now also shorter files are processed very rapidly.
In order to test that, we evaluated on the Google's FLEURS dataset, a dataset consisting of short audio fragments.
- Character Error Rate: 1.6%
- Word Error Rate: 5.9%
- Average end-to-end latency: 1.6s
- Pipeline overhead: 0.99s
- Speed on short snippets: ~6× faster than real-time
Fast enough for conversational interfaces. Fast enough for agents. On longer audio files the infrastructure has been processing ~37x faster than real-time.
Processed in the Netherlands
The goal of Murmel to not just be an option that is fully built and hosted in the Netherlands, it is also to make a great transcription experience. So very grateful for all the early adopters that help with just that!
Dutch organisations should not have to choose between a model that performs well and one they're allowed to use. Murmel is both.
What we're still working on
A few things came up in early user feedback that we're actively improving. Screen reader accessibility needs work, something we're taking seriously and fixing properly.
And when audio quality gets genuinely bad, heavy background noise, overlapping speakers, a microphone that's seen better days, the model occasionally inserts words that weren't there. Not wrong words for the right ones. Extra words, appearing where the signal gets murky. For example 'eh' or 'ja ja'.
Although this is a common occurrence for STT models, including the ones from the biggest labs, it is definitely a point to work on for the next versions.
Try it today, for free
There's a free tier with 60 minutes of transcription included — no credit card, no waitlist. Sign up at murmel.eu, upload a file directly in the Studio, or grab an API key and have results in minutes.
If you want to test more extensively, or you're building something for production, reach out. I would love to hear what you're working on.