Low-resource language data for AI
Text, speech and evaluation data in languages that have little content online, written, recorded and checked by native speakers. We check that we can staff a language properly, then run a pilot before you commit to volume.
- Script and spellingOdia script, spelling standard agreed with you
- WritersNative speakers screened with a written test
- ReviewersSecond reviewer confirmed for every batch
- GuidelinesTranslated and explained in Odia
- PilotPilot batch delivered for your team to check
Illustrative example of the checks we run before a pilot.
- Data
- Written text, translation, speech recordings, transcripts and evaluation
- People
- Native speakers, screened with a language test for each project
- Variants
- Dialects, script variants and mixed-language text
- First step
- A feasibility check, then a pilot batch
Why these languages need special handling
A low-resource language is one with little digital text or speech available for training AI models, even if millions of people speak it. Many languages of Africa and South and Southeast Asia fall in this group. Models trained mostly on web data understand them poorly and often answer in a stiff, translated style.
The data has to be made by people. That means finding enough qualified native speakers, agreeing on spelling and script, and checking quality with reviewers who know the language as well as the writers do.
What we provide
The same services we run in major languages, planned for a smaller pool of people.
Native text writing
Prompts, responses, articles and dialogs written in the language, not translated into it. More on SFT data.
Translation from English
Parallel data and translated test sets, adapted so the content makes sense locally.
Speech data
Read and spontaneous speech recorded by native speakers, with transcripts. More on speech data.
Evaluation
Model answers rated by native speakers for accuracy, fluency and whether they sound natural.
Mixed-language text
Text the way people write online, such as Hindi and English mixed in Latin script.
Guidelines in the language
Your annotation or writing guidelines translated and explained, so writers don't work from English alone.
How a low-resource language project runs
We tell you whether we can staff the language before anything starts.
Feasibility
We check how many qualified native speakers we can recruit for your volume.
Standards
Script, spelling and dialect choices agreed with you.
Screening
Writers and reviewers pass a language test for the task.
Pilot
A pilot batch reviewed by you and by our second reviewer.
Pilot sign-off QAVolume
Production with ongoing review, at the pace the team can sustain.
Related AI language services
Questions about low-resource languages
What is a low-resource language?
A low-resource language is one with little digital text, speech or labeled data available for training AI models. Many have millions of speakers but a small presence online.
Which languages can you cover?
It depends on the task and the volume. Tell us the language, the dialect and how much data you need, and we'll check whether we can recruit enough qualified native speakers before a pilot is agreed.
How do you check quality when few reviewers exist?
Every writer and reviewer passes a language test for the task. A second native speaker reviews each batch, and your own team or experts can review the pilot before volume starts.
Can you handle dialects and mixed languages?
Yes. We agree the dialect, script and spelling rules with you first, and we can write or collect mixed-language text the way people actually use it.
Ask about a language
Tell us the language, dialect, data type and volume, or book a call. We check whether we can staff it before a pilot is agreed.
- Phone
- +44 20 3290 0116
- Phone hours
- Weekdays 9:30 to 16:00, weekends 9:30 to 12:00, UK time