AI data labeling

We label datasets for model training and evaluation: we write the guidelines and taxonomy, label four modalities, resolve annotator answers through consensus, and hand over a QA report per class.

We prepare labeled text, images, audio, and video against agreed rules with two review passes.

Pricing

from ₽300,000

per project

This is the lower bound of a project. We scope the exact quote after a brief: it depends on volume and the number of integrations.

What's included

Capabilities

Guidelines and taxonomy

We build the class tree, definitions, and borderline examples. Guidelines fix what counts as a label and cut the spread between annotators.

Four modalities

Text, images, audio, and video run through one pipeline: each modality gets its own tool, shared acceptance rules, and a decision log.

QA and consensus

Every tenth item gets a second pass. We resolve disagreements into consensus and compute agreement per class, then keep it in the report.

RLHF and preference pairs

We collect answer pairs for RLHF: an annotator picks the better option against a rubric, and disputed pairs go to an expert for arbitration.

How we work

From request to result

  1. 1

    Taxonomy audit

    We review the current classes, examples, and borderline cases. We check that each class has a definition and an observable signal.

  2. 2

    Guidelines and pilot batch

    We write the instruction with examples and label a pilot of 200–500 items. We measure agreement and fix ambiguous spots before the stream starts.

  3. 3

    Labeling pipeline

    We set up queues, roles, and acceptance. Batches run on schedule, progress shows in a dashboard, and each label keeps its author and version.

  4. 4

    QA and handover

    We run the sample through two passes, resolve disagreements, and assemble the report. We hand over the dataset in the agreed format along with the guidelines.

Who it fits

Check if it fits you

Good fit

  • + 10,000 items

    A volume of 10,000 items or more pays for the guidelines and pipeline setup: batches run as a stream, and quality rests on the sample.

  • A domain expert on hand

    An expert sets the labeling reference: the other annotators and QA use their decisions.

Not a fit

  • No taxonomy

    Without a class tree and definitions, labeling drifts: the audit and guidelines come first, then the batches.

  • PII without a policy

    Personal data needs a retention and anonymization policy: until you agree on it, batches with PII stay on hold.

Track record

What you get

10,000

10,000 items in the base

The base batch covers 10,000 items and gives enough examples for training and a measurement on a held-out split.

text

Text as the base modality

The text modality goes first: text acceptance takes one QA pass, and the taxonomy template carries over to images, audio, and video.

2

Two QA passes

Two QA passes give agreement per class and show which definitions are worth tightening before scaling.

Scope of work

What the delivery includes

Guidelines and taxonomy

We hand over the taxonomy, class definitions, borderline examples, and an acceptance rubric. The document records its version and is the reference for annotators.

A labeled pilot batch

We deliver 200–500 items with labels, versions, and a decision log. The pilot batch shows agreement on real data before scaling.

A QA report

We assemble a report on agreement, disagreements, and class coverage. The QA report shows where the rules hold and which classes need more work.

How it works

How the system is built

  1. Data intake

    We accept files in batches, check the format and completeness, route them into queues, and assign an owner to each batch. At intake we filter broken files and duplicates, so the annotator receives a clean set.

  2. Labeling

    Annotators label against the guidelines and rubric, leave a comment on a borderline case, and pass the item to a second pass. Each label keeps its author, time, and instruction version, so the decision history can be restored.

  3. QA and consensus

    A second annotator reviews the sample, disagreements go to expert arbitration, and the final label is fixed through consensus. We compute the agreement metric per class and return disputed definitions to the guidelines.

Limitations and cost

Where the solution pushes back

  • 01

    The quality of the guidelines

    The quality of the guidelines sets the label spread: precise definitions and examples raise agreement and cut fixes at acceptance.

  • 02

    Expert availability

    Expert availability sets the arbitration speed: borderline cases wait for the expert decision, so the expert schedule sits in the batch plan.

  • 03

    The PII policy

    The PII policy defines the workflow: we agree on anonymization and retention before the start, and that adds a step to batch preparation.

Stack

Technologies

The technologies we use in our work and their role in a project.

Labeling pipeline

We write validation and metric scripts in Python, store tasks and statuses in PostgreSQL, keep dataset files in MinIO, run the job queue through Kafka, and hold batch progress in Redis.

We write validation, agreement scoring, and dataset export: the scripts check the schema and catch broken files before handover.It stores tasks, statuses, and edit history: the log shows who changed a label and when, and agreement is computed from it.It stores images, audio, and video next to the labels: the dataset stays reproducible, and access runs through signed links.It passes jobs between annotator queues and the quality service: the buffer holds the batch pace under peak load.It caches progress and task locks: the annotator sees the current status, and one item is issued to a single worker.
Budget estimate

Cost calculator

This is an indicative range, not a quote. We scope the exact estimate after a brief: it depends on volume and the number of integrations.

Items to label

10000

Modality

Quality level

Guidelines

Add-ons

Support

Year one: updates and monitoring, a reply within 2 business days.

Service

AI data labeling

The estimate covers our team's work. Hardware, third-party service licences and travel are added on request.

Estimated cost

₽400,000

Timeline

2 wk

Questions

Frequently asked questions

How do you control labeling quality?

Every tenth item gets a second pass, we resolve disagreements into consensus, and we compute agreement per class. We fix the acceptance threshold in the guidelines before the batch starts.

Can you label under an NDA and on our own infrastructure?

Yes. We sign an NDA, work in your cloud or on an on-prem deployment, grant file access by role, and keep the action log on your side.

Which modalities do you label?

Text, images, audio, and video. Each modality has its own tool and acceptance rules, while the taxonomy and QA stay shared.

How long does a pilot take?

The pilot runs 1–2 weeks: a taxonomy audit, the guidelines, and a batch of 200–500 items with an agreement measurement. After the pilot we quote the timeline for the full volume.

What teams add to this service

Ready to discuss your task?

Leave a request – we'll get back with a proposal and a preliminary estimate.

Contact us