Personal project by Ulf Fäldt Reifenberg, Gothenburg. First run on 2 October 2026 as a private probe; published on 20 November 2026 for an application to Interpreto, with the page built by coding agents. Unofficial: not affiliated with Interpreto, KBLab, Klang, OpenAI or Google. It uses public audio only and says nothing about any company's product.

A 5% word error rate hides a 25% name error rate

Five open speech-to-text models on 60 Swedish clips, every one run locally on a laptop CPU with no API calls. The best model gets about one word in twenty wrong, and one name in four.

What the 60 clips show Sixty clips of clean read speech is a first look, not a ranking. See the limits at the bottom.

Word error rate

Lower is better. The bar is the 95% bootstrap interval over clips (4,000 resamples). Scored after the Swedish normaliser described below; speed is audio time divided by wall time on 12 CPU threads with int8 weights, model loading included.

Paired comparisons on the same clips (lowercase, no punctuation): KB-Whisper Large against Klang Pianissimo −1.9 points (CI −3.3 to −0.6); against OpenAI large-v3-turbo −3.9 (CI −5.4 to −2.4). KB-Whisper Small against OpenAI large-v3-turbo −1.2 (CI −3.0 to +0.6): no detectable difference. Check: KBLab publishes 5.4% for KB-Whisper Large on FLEURS and this sample gives 5.4% under the same plain normalisation.

Where the errors land

Share of reference words transcribed exactly right, split by type: 45 name words (capitalised inside a sentence), 34 number words, 1,077 other words.

Typical misses: Asunción → "association", Chişinău → "kishinau", Sagittariusgalaxen → "galaxen", Duvall → "Duval", Fernández → "Fernandez". Some are spellings a reader would accept and several are not. In a case note or a meeting record, the names are the part that must be right, and an average word error rate does not show whether they are.

Scoring Swedish numbers fairly

The models hear the numbers and write them differently from the reference: "1990" comes back as "nittonhundranittio", "80 km" as "åttio kilometer", "30 procent" as "30%". A generic digits-to-words step does not reconcile these, so I wrote a small normaliser (about 60 lines): number compounds to digits, years as read, ordinals before a month, units and symbols, digit grouping and clock times. It does not handle decimals, fractions, numbers above 999 written as several words, or currency.

FLEURS has few numbers, so the effect is small and the order does not change. On audio full of dates, amounts and times it would be larger; that is not measured here.

A simulated telephone line

The same clips pushed through 8 kHz sampling, a 300–3400 Hz band-pass and 8-bit μ-law companding, then back to 16 kHz.

On the phone channel KB-Whisper Small and OpenAI large-v3-turbo are still too close to call (+0.3 points, CI −1.6 to +2.2). This is a mild simulation: no real codec, no line noise, no speakerphone, no overlapping or spontaneous speech.

Read the transcripts

Every clip, every model, word by word against the reference (after the normaliser). wrong (expected) missing added

What this does not show

How it was run

Whisper-family models through faster-whisper, Klang Pianissimo through onnx-asr, all int8 on CPU. Scoring with jiwer; intervals by resampling clips. Audio and reference text: the Swedish test set of FLEURS (Google, CC BY 4.0). Models: KBLab KB-Whisper, Klang Pianissimo, OpenAI Whisper. The raw numbers behind this page are in data.json.