BlasBench

Why a benchmark

Low-resource speech research is full of numbers that cannot be compared because everyone evaluates on different audio with different scoring. BlasBench fixes the audio, fixes the reference transcripts, and fixes the metric, so a word-error-rate from one model means the same thing as a word-error-rate from another.

How it works

• A held-out set of Irish audio with verified reference transcripts.
• A scoring script that reports word and character error rates with consistent normalisation.
• A baseline recipe so a new system can be dropped in and compared on equal terms.

We hold our own recogniser, Blas Voice, to this bar before it goes near a learner. See the research notes for how it fits the wider speech effort.