Why ழ் is the hardest sound to grade
The word தமிழ் — “Tamil” — ends in ழ், a retroflex approximant so distinctive that speakers call their language “the tongue of ழ.” It barely exists outside Dravidian languages, and children spend years learning to place the tongue for it.
A pronunciation tutor for Tamil literacy must be able to tell a correct ழ் from its near neighbours ள் and ல். All three are voiced, all three are made near the same place in the mouth, and — as we will show — their acoustics overlap almost completely. This is precisely the discrimination that off-the-shelf speech technology, tuned for English and other high-resource languages, was never built to make.
Tamil has ~85 million speakers and a strong schools push for mother-tongue literacy, yet almost no open speech resources for pronunciation teaching. A working evaluator — and the corpus behind it — is public-good infrastructure other Tamil ed-tech can build on.
What we have already built
Over the course of this project we shipped tamil.ai: a running service that scores a spoken Tamil letter against a target. It combines an acoustic-feature engine (formants, spectral bands, dynamic-time-warping against references) with small convolutional-neural-network “arbiters” for the confusable consonant pairs.
247 letter forms
18 consonants (மெய்), 12 vowels (உயிர்), and 216 compound letters (உயிர்மெய்).
Engine + 6 CNN arbiters
Acoustic engine for all letters; trained binary CNNs guard the six hardest confusable pairs.
Deployed & iterating
Versioned, measured against real volunteer recordings, and improved release over release.
Isolated consonants (the inherent-அ forms) were being rubber-stamped. We rerouted them through the CNN arbiters and lifted combined accuracy from 32% to 57% — a 25-point gain, validated live and shipped. The team can move the numbers; the ழ்/ள்/ல் wall is a different kind of problem.
The engine already performs well where sounds are acoustically distinct — isolated-consonant accuracy sits near 93%. The failure is specific and structural, and isolating it is what the rest of this document does.
The measurement that stops us
| Letter | Sound | TPR ↑ | FAR ↓ | Accuracy | Reading |
|---|---|---|---|---|---|
| ழ் | retroflex approx. | 8% | 0% | 69% | rejects real ழ் |
| ள் | retroflex lateral | 33% | 12% | 69% | unreliable |
| ல் | alveolar lateral | 100% | 83% | 44% | accepts anything |
| Overall (3 letters) | 47% | 32% | 61% | not usable | |
Two opposite failures, one root: ழ் rejects itself, ல் accepts everyone.
ழ் is over-guarded — its CNN is so unsure of genuine ழ் that it sends nearly every real attempt back as “try again” (8% pass), while perfectly blocking impostors (0% false-accept).
ல் is unguarded — it has no CNN at all, so it leans on a loose acoustic band and waves through ழ் and ள் audio 83% of the time. Same underlying confusion, opposite symptom.
Root cause: the sounds overlap, and the data is thin
Finding A — acoustically, these three are nearly the same sound
The single feature that should separate them — the third formant (F3), the frequency dip that a curled retroflex tongue produces — overlaps across all three letters. There is no clean line to draw.
Measured F3 range per letter (engine formant tracker, eval population). The bars sit almost on top of one another — a threshold that admits real ழ் also admits ள் and ல்.
Finding B — the model has heard only four voices
Every CNN arbiter was trained on a corpus of just four speakers — two children, one 19-year-old, one 45-year-old man, and no women at all. None of them appear in the population the system is actually used on. The model is being asked to grade strangers.
4 speakers
adult · child · child · man — ~80 samples/letter, zero female voices, one dialect.
10+ speakers
Entirely different people — fully out-of-distribution from what the model learned.
ழ்’s 8% and ல்’s 83% are the same fact seen twice: three near-identical sounds, and a model that learned four voices instead of the sounds themselves.
We ruled out every shortcut — rigorously
Before asking for anything, we tested whether the fix could be free. It cannot. Each experiment below was run against the live system or with leave-one-speaker-out (LOSO) validation — the honest test of whether a model generalises to a voice it has never heard.
-
Loosen the acoustic threshold for ழ் Reverted
Trusting the acoustic band over the unsure CNN would have lifted ழ் to ~83% — but a safety sweep showed it would newly accept 46% of wrong letters, including vowels. Written, tested, and reverted the same session.
band-confirm is not discriminative for ழ் → 40 / 87 impostors leak
-
Retrain on the 4-speaker data ≈ chance
A 3-way ழ்/ள்/ல் model, leave-one-speaker-out: 35% accuracy — barely above the 33% you would get by guessing. The binary ல்-guard: 30% recall at 30% false-accept, net worse than today.
LOSO 4 folds → 28% · 28% · 52% · 30%
-
Pool in the 10 evaluation speakers Helps, not enough
Adding the eval population (≈1 sample each) lifted 3-way LOSO only 35% → 41%; on truly-unseen speakers it stayed at 39%. One sample per new speaker is diluted by the four heavy voices.
cloud-holdout: 3-way 39% · ל-guard 42% TPR / 54% FAR
-
The conclusion that survived Data is the fix
Diversity moves the numbers in the right direction — but you need many speakers each contributing many samples, balanced across gender, age, and dialect. That corpus does not exist for Tamil — building it is the next step.
need ≈ 15–20+ speakers × 10–15 samples/letter, balanced
We are not guessing. We measured exactly where the cheap fixes stop — and then found what works: pretrained speech models. The remaining gap is a known, well-scoped problem, not a research gamble.
The breakthrough — and what we now target
The dead-ends above ruled out the shortcuts. Then pretrained speech models — trained on thousands of speakers — broke through the speaker-generalisation wall. Our contribution is the system that combines them: we built an ensemble that fuses AI4Bharat's IndicWav2Vec with three other pretrained encoders into one model that no single component beats. Recognition of ழ், Tamil's hardest sound, climbed 8% → 42% → 53% → 75% as the models became more Tamil-aware. The reframed picture: not “nothing works,” but “a working model exists — the corpus finishes it.”
| Metric | Now | Target |
|---|---|---|
| Vowels (12) | 94% | ≥ 95% |
| Distinct-place consonants | 83–98% | ≥ 92% |
| Retroflex/alveolar sonorant cluster (ண் ன் ர் ற் ல் ள் ழ்) | 53–67% | ≥ 80% |
| Signature sound ழ் | 75% | ≥ 85% |
| Overall (30 letters) | 61% | ≥ 85% |
| Safe-verdict rate — never marks a wrong sound “correct” | — | ≥ 95% |
We deliberately do not claim 95% everywhere — ள்↔ல் and the nasal set overlap acoustically, and even human listeners confuse some as isolated letters. The ≥80% sonorant-cluster target extrapolates a slope we have already shown (ழ் 8%→75%). The gains are data-limited, not method-limited: the approach works; a balanced, labelled corpus plus fine-tuning closes the gap.
Measured on a held-out, demographically-balanced test set (speakers, gender, age, dialect unseen in training), reported per-category with confusion analysis — not a single averaged number that hides weak spots.
See how every Tamil letter scores All 30 letters, cross-speaker — with the confusion matrix and phonetic-family error breakdown. The Full Scorecard →What’s next
The bottleneck is a resource that does not exist: a balanced, openly-licensed corpus of the Tamil retroflex and lateral sonorants — ழ் ள் ல் and the near neighbours needed to tell them apart — spoken by many diverse voices. Building it is what makes the wall fall.
Balanced by design
| Speakers | 60–100, gender-balanced |
| Age spread | children → elders |
| Dialects | multiple regional |
| Samples / letter | 10–15 per speaker |
| Target set | ழ் ள் ல் |
| + Discriminators | ர் ற் ண் ன் |
Deliverables
CC-licensed, the first of its kind for Tamil pronunciation teaching.
The three sonorants solved and validated speaker-out — a repeatable template for the next confusable set.
Published LOSO protocol + a public accuracy benchmark others can target.
| Participant recruitment & compensation data | diverse speaker panel, ethics & consent |
| Recording & collection tooling build | web recorder (built), QA & sync pipeline |
| Annotation & validation data | native-speaker labelling, quality control |
| Model training & compute ml | retrain, LOSO evaluation, iteration |
| Open release & documentation public good | corpus hosting, licence, benchmark, paper |
Live system
A deployed evaluator and a 25-point shipped improvement already on record.
Proven method
Speaker-out validation and adversarial safety testing already in practice.
Public good
An open corpus every Tamil ed-tech project can build on, not a private asset.
Build the corpus, and the language named for ழ் finally gets a machine that can hear it.