Data Labeling
Producing the correct answers (labels) that supervised models learn from, by people, by rules or by other models, and making sure those labels are consistent.
Also annotation · labeling · crowdsourcing · inter-annotator agreement · label noise · ground truth
Before a computer can learn what a stop sign is, someone has to go through thousands of photos and draw a box around every stop sign. That patient work is labeling.
Where it fits
+All relationships as a list
How it works
Supervised learning needs labels: the right answer for each training example. Someone or something has to produce them.
A typical labeling workflow:
- Write guidelines. Is a person on a bicycle a “pedestrian” or a “cyclist”? Ambiguity here becomes noise in your data.
- Annotate. With domain experts (doctors, lawyers), trained in-house teams or crowd workers, depending on difficulty.
- Check agreement. Have several people label the same examples. Low agreement means the task or guidelines need work.
- Audit. Even famous benchmarks contain a few percent of wrong labels.
Ways to need fewer human labels:
- Active learning: label only the examples the model is unsure about.
- Semi-supervised learning: let a few labels spread to similar unlabeled examples.
- Model-assisted labeling: a model pre-labels and humans correct, which is much faster.
- LLM labeling: large models can label text at scale, but need spot checks.
Labeling is not only for classic ML. The human comparisons behind RLHF are labels too, and their quality directly shapes how assistants behave.
Everyday use cases
- Case 01
Self-driving datasets
Annotators draw boxes and masks around cars, pedestrians and lanes in millions of frames.
- Case 02
Chatbot training
People write ideal answers and rank model responses to teach assistants what good looks like.
Strengths, weaknesses and when to use it
Strengths
- Good labels are often the single biggest lever on model quality.
Weaknesses
- Slow, expensive and sometimes psychologically taxing work; ambiguous guidelines produce inconsistent labels.
When to use itEvery supervised project. Invest in clear guidelines and agreement checks before scaling up.
Compare with
Check yourself
- Q01
Two annotators disagree on 30% of examples. What is the most useful first step?
Show answer
A: Clarify the labeling guidelines and review the disagreements