Do AI Models Know Scottish Gaelic (Gàidhlig)?
Models read Gaelic much better than they write or fix it. Across nine models tested on 280 Scottish Gaelic items, scores average 80.6% (71.2% to 86.6%). Translation drops 17.0 points out of English, and tutor accuracy averages 72.3%.
When you ask a chatbot for Scottish Gaelic help, it rarely hesitates. Yet writing natural Gaelic and catching errors is much harder for AI than reading Gaelic.
We ran Teanga Bench on nine AI models on 15 and 16 November 2026. Each model answered 280 Scottish Gaelic items three times across five tasks.
The short answer
Across all nine models, Scottish Gaelic averages 80.6%. Claude Fable 5.2 scores highest at 86.6%, but its 95% range overlaps five models down to DeepSeek V4.1 Flash 1017 (81.8%). Models average 97.0% on lessons, yet drop to 72.3% on tutor accuracy and 72.2% on round-trip translation.
What each score measures
The Scottish Gaelic slice has 280 items across five tests. Every item was asked three times:
- Lesson exercises (60 items): multiple choice, tile drills and mutations from the Blas Scottish Gaelic course, marked by accepted answers.
- Word forms (60 items): giving copula words, numbers, plurals, genitives and prepositional pronouns, marked against attested forms.
- Translation (60 items): translating 30 sentences into English and 30 out of English, scored by character match (chrF++).
- Round-trip translation (40 items): translating Scottish Gaelic into English and back again, then checking how close it stays.
- Tutor accuracy (60 items): finding one mistake in a sentence, fixing only that span and leaving the rest alone.
Each score comes with a 95% range. When two ranges overlap, the two models are not clearly different.
Scottish Gaelic scores across nine models
Overall Gaelic score, 95% range and test scores (9-model averages: 97.0% lessons, 85.8% word forms, 75.8% translation, 72.3% tutor, 72.2% round-trip):
| Model | Overall | 95% range | Lessons | Word forms | Translation | Round-trip | Tutor |
|---|---|---|---|---|---|---|---|
| Claude Fable 5.2 | 86.6% | 84.0% to 89.2% | 100.0% | 96.7% | 77.3% | 77.8% | 81.1% |
| Gemini 4 Hydrogen | 86.1% | 83.5% to 88.6% | 100.0% | 91.7% | 76.0% | 74.9% | 87.8% |
| GPT-6 Astra | 83.8% | 80.7% to 86.6% | 98.3% | 91.7% | 77.0% | 70.8% | 81.1% |
| Gemini 4 Argon | 82.5% | 79.3% to 85.5% | 98.3% | 87.8% | 76.6% | 74.9% | 75.0% |
| Kimi K3.1 Pro | 82.1% | 79.1% to 85.0% | 97.2% | 87.8% | 77.5% | 69.3% | 78.9% |
| DeepSeek V4.1 Flash 1017 | 81.8% | 78.6% to 84.7% | 98.3% | 91.1% | 76.4% | 74.8% | 68.3% |
| GPT-6 Luna | 78.4% | 75.0% to 81.8% | 96.1% | 86.7% | 75.8% | 68.3% | 65.0% |
| Claude Haiku 5.5 | 73.2% | 69.7% to 76.7% | 90.6% | 73.3% | 70.5% | 69.5% | 62.2% |
| Qwen3.8 Flash | 71.2% | 67.5% to 74.5% | 94.4% | 65.6% | 74.9% | 69.2% | 51.7% |
Gemini 4 Hydrogen leads tutor accuracy on Scottish Gaelic at 87.8%. Claude Fable 5.2 and GPT-6 Astra tie behind it at 81.1%, while Qwen3.8 Flash drops to 51.7%.
Reading Gaelic vs writing Gaelic
Scottish Gaelic has the steepest direction gap of the three core languages. Models average 84.3% into English, against 67.3% out of English (a 17.0-point drop):
| Model | Into English (30) | Out of English (30) | Gap (pts) |
|---|---|---|---|
| Claude Fable 5.2 | 82.0% (73.1% to 89.2%) | 72.6% (62.9% to 81.7%) | +9.4 |
| Gemini 4 Hydrogen | 82.6% (75.7% to 89.1%) | 69.4% (60.4% to 78.7%) | +13.2 |
| GPT-6 Astra | 88.9% (82.2% to 94.2%) | 65.2% (55.6% to 74.5%) | +23.7 |
| Gemini 4 Argon | 83.8% (76.1% to 90.5%) | 69.4% (59.8% to 78.5%) | +14.4 |
| Kimi K3.1 Pro | 86.1% (78.6% to 92.4%) | 68.9% (60.8% to 76.9%) | +17.2 |
| DeepSeek V4.1 Flash 1017 | 84.0% (76.7% to 90.9%) | 68.9% (60.0% to 77.7%) | +15.1 |
| GPT-6 Luna | 87.3% (81.2% to 92.9%) | 64.4% (55.0% to 73.6%) | +22.9 |
| Claude Haiku 5.5 | 79.3% (71.8% to 86.2%) | 61.7% (52.7% to 70.6%) | +17.6 |
| Qwen3.8 Flash | 85.0% (78.7% to 90.6%) | 64.9% (56.2% to 73.6%) | +20.1 |
Every model scores higher into English than out of English. On five models (GPT-6 Astra, GPT-6 Luna, Qwen3.8 Flash, Claude Haiku 5.5 and Kimi K3.1 Pro), the 95% ranges do not overlap.
Concrete patterns from public sample items
Public sample items and failure logs show three common slips in Scottish Gaelic:
- Dative lenition after
anns a</code> and <code>aig a: OnTha biadh anns a' cidsin.(needs lenitedchidsin), DeepSeek V4.1 Flash 1017 said the text was right. On cleanNach robh iad aig a' mhuir tron oidhche?, it flaggedaig a' mhuirin JSON while calling the line right in its note. - Copula particles and counting numbers: On
___ do Dhia a bhuineas slàinte.(needs'S ann), GPT-6 Astra, GPT-6 Luna and Gemini 4 Argon wroteIs ann, while Gemini 4 Hydrogen and DeepSeek wroteIs. For counting 2 (dhà), Claude Fable 5.2, GPT-6 Astra and Gemini 4 Argon gavea dhà. - Idiomatic drift out of English: For
He is a good carpenter.(referenceTha e na dheagh shaor.), top models wrote'S e saor math a th' ann.orIs saor math e.OnShe used to drink beer.(referenceB' àbhaist dhi leann òl.), models switched toBhiodh i ag òl leann.
What this means when you learn Gaelic
Scottish Gaelic has the lowest tutor average (72.3%) of the three languages. Check model edits with care:
- Use AI for reading help first: Models average 84.3% into English and read Gaelic text fairly well. Treat any Gaelic prose they write (67.3% average) as a rough draft.
- Check lenition and case endings yourself: Check feminine and dative lenition after articles,
deaghand prepositions with our Gaelic mutations table and dictionary. - Check copula and past idioms in a course: Models often flatten Gaelic idioms (
Tha e na...,B' àbhaist dhi...) into literal forms. Check your lines with the grammar checker and Scottish Gaelic lessons.
Honest limits of this test
Keep these limits in mind when you read the Scottish Gaelic scores:
- No human check on gold answers: Every item has
human_checked: false. Scores measure match with the Blas course, dictionaries and parallel text. - Strict reference matching: Valid phrases like
'S e saor math a th' ann.ora dhàlose marks when they differ from the stored reference. - Public sentences and wide ties: Translation items come from public text, and six models share overlapping 95% ranges at the top overall.
- Synthetic tutor errors and reruns: Most tutor faults come from course distractors. Four models hit 2,048-token cutoffs and were rerun at 16,000 tokens.
See the full data
Every run file, score and public sample is open:
- Browse the Blas leaderboard, language breakdown and findings page.
- Read the benchmark methodology for scoring rules, ranges and truncation reruns.
- View the code and public samples in the Teanga Bench repository.
What to do next
We will test new models as they launch. Until AI handles lenition and idioms well out of English, use chatbots for reading help. Rely on reference tables and our Scottish Gaelic course when you write.
Editorial notes
- Review methodology
- Every number in this post comes from the published Teanga Bench Blas-suite run files (research.blasapp.com, results as of 16 November 2026). All concrete examples come solely from public sample files in samples/blas/ and published model pages; no private test items are quoted. No human reviewer checked the gold answers (human_checked is false) or this post.
Ready to make this stick?
blas. is the language app for adults coming back to Irish or Welsh. Grammar, vocabulary, mutations, conversation — all with spaced repetition so you actually remember it.
Keep reading
How Well Do AI Models Know Celtic Languages?
We tested nine AI models on Irish, Welsh, Scottish Gaelic, Manx, Breton and Cornish. See the scores and what they mean if you learn with AI.
Scottish Gaelic Grammar for Beginners: Where to Start
Scottish Gaelic grammar basics in plain English: verb-first order, tha vs is, lenition, the article and prepositional pronouns, with sourced examples.
Irish vs Scottish Gaelic: How Different Are They Really?
Same roots, different languages. Side-by-side spelling, pronunciation, grammar, and vocabulary comparison. Plus: can speakers understand each other?
