AI Alignment
Making AI systems reliably pursue the goals and values their designers and users actually intend, including as they become more capable.
Also value alignment · outer alignment · inner alignment · scalable oversight
A genie grants exactly what you say, not what you mean. Alignment is about building a genie that understands and cares about what you meant.
Where it fits
+All relationships as a list
How it works
Training optimizes a measurable objective. Alignment asks whether that objective, and the behavior that emerges from optimizing it, matches what people actually want.
It breaks into a few questions:
- Specifying the goal: rewards and instructions are always incomplete. Optimizing hard against an imperfect target leads to reward hacking.
- Learning the goal: even with a good objective, training may produce a system that pursued a convenient shortcut that happened to score well, and that behaves differently in new situations.
- Keeping oversight: as systems handle tasks people cannot easily check, how do we evaluate them? Ideas include using AI to help humans supervise AI and interpretability to inspect what models are doing internally.
Practical alignment work is already everywhere in today’s assistants: RLHF and Constitutional AI (used for Claude) shape models toward helpful, honest and harmless behavior, while red teaming and evaluations probe for failures such as deception, harmful advice or susceptibility to prompt injection.
Everyday use cases
- Case 01
Helpful, honest chat assistants
Post-training methods teach models to follow intent, admit uncertainty and refuse harmful requests.
- Case 02
Red teaming
Teams try to provoke unsafe behavior before release, then fix what they find.
Famous examples
Strengths, weaknesses and when to use it
Strengths
- Practical alignment techniques made today's assistants far more useful and safer than raw pretrained models.
Weaknesses
- Hard to verify; good behavior in tests does not guarantee good behavior everywhere.
When to use itFor any system trained to optimize an objective, and especially for capable, autonomous ones.
Compare with
Check yourself
- Q01
Which is an example of misalignment?
Show answer
A: A model rewarded for user approval learns to tell people what they want to hear