Do AI Models Know Scottish Gaelic (Gàidhlig)?

Models read Gaelic much better than they write or fix it. Across nine models tested on 280 Scottish Gaelic items, scores average 80.6% (71.2% to 86.6%). Translation drops 17.0 points out of English, and tutor accuracy averages 72.3%.

2026-11-17·6 min read

When you ask a chatbot for Scottish Gaelic help, it rarely hesitates. Yet writing natural Gaelic and catching errors is much harder for AI than reading Gaelic.

We ran Teanga Bench on nine AI models on 15 and 16 November 2026. Each model answered 280 Scottish Gaelic items three times across five tasks.

The short answer

Across all nine models, Scottish Gaelic averages 80.6%. Claude Fable 5.2 scores highest at 86.6%, but its 95% range overlaps five models down to DeepSeek V4.1 Flash 1017 (81.8%). Models average 97.0% on lessons, yet drop to 72.3% on tutor accuracy and 72.2% on round-trip translation.

What each score measures

The Scottish Gaelic slice has 280 items across five tests. Every item was asked three times:

  • Lesson exercises (60 items): multiple choice, tile drills and mutations from the Blas Scottish Gaelic course, marked by accepted answers.
  • Word forms (60 items): giving copula words, numbers, plurals, genitives and prepositional pronouns, marked against attested forms.
  • Translation (60 items): translating 30 sentences into English and 30 out of English, scored by character match (chrF++).
  • Round-trip translation (40 items): translating Scottish Gaelic into English and back again, then checking how close it stays.
  • Tutor accuracy (60 items): finding one mistake in a sentence, fixing only that span and leaving the rest alone.

Each score comes with a 95% range. When two ranges overlap, the two models are not clearly different.

Scottish Gaelic scores across nine models

Overall Gaelic score, 95% range and test scores (9-model averages: 97.0% lessons, 85.8% word forms, 75.8% translation, 72.3% tutor, 72.2% round-trip):

ModelOverall95% rangeLessonsWord formsTranslationRound-tripTutor
Claude Fable 5.286.6%84.0% to 89.2%100.0%96.7%77.3%77.8%81.1%
Gemini 4 Hydrogen86.1%83.5% to 88.6%100.0%91.7%76.0%74.9%87.8%
GPT-6 Astra83.8%80.7% to 86.6%98.3%91.7%77.0%70.8%81.1%
Gemini 4 Argon82.5%79.3% to 85.5%98.3%87.8%76.6%74.9%75.0%
Kimi K3.1 Pro82.1%79.1% to 85.0%97.2%87.8%77.5%69.3%78.9%
DeepSeek V4.1 Flash 101781.8%78.6% to 84.7%98.3%91.1%76.4%74.8%68.3%
GPT-6 Luna78.4%75.0% to 81.8%96.1%86.7%75.8%68.3%65.0%
Claude Haiku 5.573.2%69.7% to 76.7%90.6%73.3%70.5%69.5%62.2%
Qwen3.8 Flash71.2%67.5% to 74.5%94.4%65.6%74.9%69.2%51.7%

Gemini 4 Hydrogen leads tutor accuracy on Scottish Gaelic at 87.8%. Claude Fable 5.2 and GPT-6 Astra tie behind it at 81.1%, while Qwen3.8 Flash drops to 51.7%.

Reading Gaelic vs writing Gaelic

Scottish Gaelic has the steepest direction gap of the three core languages. Models average 84.3% into English, against 67.3% out of English (a 17.0-point drop):

ModelInto English (30)Out of English (30)Gap (pts)
Claude Fable 5.282.0% (73.1% to 89.2%)72.6% (62.9% to 81.7%)+9.4
Gemini 4 Hydrogen82.6% (75.7% to 89.1%)69.4% (60.4% to 78.7%)+13.2
GPT-6 Astra88.9% (82.2% to 94.2%)65.2% (55.6% to 74.5%)+23.7
Gemini 4 Argon83.8% (76.1% to 90.5%)69.4% (59.8% to 78.5%)+14.4
Kimi K3.1 Pro86.1% (78.6% to 92.4%)68.9% (60.8% to 76.9%)+17.2
DeepSeek V4.1 Flash 101784.0% (76.7% to 90.9%)68.9% (60.0% to 77.7%)+15.1
GPT-6 Luna87.3% (81.2% to 92.9%)64.4% (55.0% to 73.6%)+22.9
Claude Haiku 5.579.3% (71.8% to 86.2%)61.7% (52.7% to 70.6%)+17.6
Qwen3.8 Flash85.0% (78.7% to 90.6%)64.9% (56.2% to 73.6%)+20.1

Every model scores higher into English than out of English. On five models (GPT-6 Astra, GPT-6 Luna, Qwen3.8 Flash, Claude Haiku 5.5 and Kimi K3.1 Pro), the 95% ranges do not overlap.

Concrete patterns from public sample items

Public sample items and failure logs show three common slips in Scottish Gaelic:

  • Dative lenition after anns a</code> and <code>aig a: On Tha biadh anns a' cidsin. (needs lenited chidsin), DeepSeek V4.1 Flash 1017 said the text was right. On clean Nach robh iad aig a' mhuir tron oidhche?, it flagged aig a' mhuir in JSON while calling the line right in its note.
  • Copula particles and counting numbers: On ___ do Dhia a bhuineas slàinte. (needs 'S ann), GPT-6 Astra, GPT-6 Luna and Gemini 4 Argon wrote Is ann, while Gemini 4 Hydrogen and DeepSeek wrote Is. For counting 2 (dhà), Claude Fable 5.2, GPT-6 Astra and Gemini 4 Argon gave a dhà.
  • Idiomatic drift out of English: For He is a good carpenter. (reference Tha e na dheagh shaor.), top models wrote 'S e saor math a th' ann. or Is saor math e. On She used to drink beer. (reference B' àbhaist dhi leann òl.), models switched to Bhiodh i ag òl leann.

What this means when you learn Gaelic

Scottish Gaelic has the lowest tutor average (72.3%) of the three languages. Check model edits with care:

  • Use AI for reading help first: Models average 84.3% into English and read Gaelic text fairly well. Treat any Gaelic prose they write (67.3% average) as a rough draft.
  • Check lenition and case endings yourself: Check feminine and dative lenition after articles, deagh and prepositions with our Gaelic mutations table and dictionary.
  • Check copula and past idioms in a course: Models often flatten Gaelic idioms (Tha e na..., B' àbhaist dhi...) into literal forms. Check your lines with the grammar checker and Scottish Gaelic lessons.

Honest limits of this test

Keep these limits in mind when you read the Scottish Gaelic scores:

  • No human check on gold answers: Every item has human_checked: false. Scores measure match with the Blas course, dictionaries and parallel text.
  • Strict reference matching: Valid phrases like 'S e saor math a th' ann. or a dhà lose marks when they differ from the stored reference.
  • Public sentences and wide ties: Translation items come from public text, and six models share overlapping 95% ranges at the top overall.
  • Synthetic tutor errors and reruns: Most tutor faults come from course distractors. Four models hit 2,048-token cutoffs and were rerun at 16,000 tokens.

See the full data

Every run file, score and public sample is open:

What to do next

We will test new models as they launch. Until AI handles lenition and idioms well out of English, use chatbots for reading help. Rely on reference tables and our Scottish Gaelic course when you write.

Editorial notes

Review methodology
Every number in this post comes from the published Teanga Bench Blas-suite run files (research.blasapp.com, results as of 16 November 2026). All concrete examples come solely from public sample files in samples/blas/ and published model pages; no private test items are quoted. No human reviewer checked the gold answers (human_checked is false) or this post.

Ready to make this stick?

blas. is the language app for adults coming back to Irish or Welsh. Grammar, vocabulary, mutations, conversation — all with spaced repetition so you actually remember it.

Keep reading