Multimodal training data
Image, video and audio paired with text written by native speakers: detailed captions, question and answer pairs, region descriptions and video summaries, in the languages your model has to work in. Every caption is checked against what's actually in the picture.
- Media
- Photos, screenshots, charts, documents, video and audio
- Text
- Captions, questions and answers, region descriptions and summaries
- Writers
- Native speakers of each language, trained on your style rules
- Output
- JSONL with media references, region coordinates and metadata
What multimodal data is
Multimodal training data pairs more than one kind of input, usually an image, video or audio clip with text that describes it or answers questions about it. Vision-language models learn from these pairs to describe pictures, read charts and answer questions about what they see. Captions that guess, skip details or describe things that aren't there teach the model to do the same.
Most image-text data online is in English. We write descriptions natively in each language, so the model learns how people there name objects, read signs and describe scenes.
What we create
For your media, or media collected to your spec.
Detailed captions
Short and long image descriptions, written to your length and detail rules.
Visual question answering
Questions about each image with checked answers, from simple counting to reasoning.
Region descriptions
Boxes or points on objects and text, each with its own description. More on image annotation.
Video descriptions
Timestamped descriptions of actions and events, and summaries of whole clips.
Charts and documents
Questions and answers about charts, tables, forms and screenshots, in many scripts.
Caption review
Existing or model-generated captions checked and corrected for accuracy and language.
How a multimodal project runs
The writing rules are agreed with examples before any volume work.
Spec
Media types, tasks, languages, length and detail rules.
Samples
A small set per task and language for your team to approve.
Writing
Native-speaker writers describe and annotate each item.
Visual check
A reviewer compares the text with the image or clip, detail by detail.
Second review QADelivery
JSONL with media IDs, coordinates and language tags.
Related AI data services
Questions about multimodal data
What is multimodal training data?
Multimodal training data combines different types of input, such as images, video or audio with text. Common examples are image captions, visual question and answer pairs, and video descriptions used to train vision-language models.
What makes a good image caption dataset?
Captions that describe only what is visible, follow consistent length and detail rules, and are written in natural language. Each caption should be checked against the image by a second reviewer.
Can you write captions in other languages?
Yes. Native speakers write captions and answers directly in each language instead of translating from English.
Do you provide the images?
We work on your images and video, or we collect new media to your specification with the rights you need. More on data collection.
Send sample media
Send a few images or clips with the tasks and languages you need, or book a call. We write samples for you to approve.
- Phone
- +44 20 3290 0116
- Phone hours
- Weekdays 9:30 to 16:00, weekends 9:30 to 12:00, UK time