Prompting
Steering a language model with the text you give it: clear instructions, examples of what you want, and room to reason step by step.
Also prompt engineering · system prompt
Brown et al. (in-context learning), 2020; Wei et al. (chain-of-thought), 2022
Asking a very well-read helper for something. The clearer your request, and the better your examples, the better the result.
Where it fits
+All relationships as a list
Is a kind of / part of
Kinds and parts
Siblings
- AI Agents and Tool Use
- BERT
- Direct Preference Optimization
- Embedding
- Fine-Tuning
- Google Translate
- GPT
- Hallucination
- In-Context Learning
- Keyboard Autocomplete
- Large Language Model
- LLM Evaluation and Benchmarks
- Perplexity
- Pretraining
- Retrieval-Augmented Generation
- RLHF
- Sampling and Temperature
- Scaling Laws
- Tokenization
- Vector Database
- word2vec
Applies to
Used by
How it works
A large language model continues the text it is given, so the prompt is the program. Some patterns reliably help:
- Be specific: who the audience is, what format, what to avoid. A model cannot read your mind any better than a new colleague.
- Give examples (few-shot): two or three input-output examples in the prompt teach a format or style. This in-context learning was the headline finding of GPT-3: the model picks up the task from the prompt alone, with no change to its weights.
- Let it think: asking for step-by-step reasoning (chain-of-thought) improves multi-step problems, because each written step becomes context for the next. Reasoning models are trained to do this by default, spending extra tokens thinking before they answer.
- Provide the facts: paste in the relevant document, or let a system retrieve it (RAG), instead of relying on the model’s memory.
Everything has to fit in the context window, the maximum number of tokens the model can read at once. When prompting stops being enough, fine-tuning is the next step.
Everyday use cases
- Case 01
Getting consistent output formats
Show two examples of the exact JSON or table you want, and the model follows the pattern.
- Case 02
Better answers to multi-step problems
Asking the model to work through a problem step by step improves arithmetic and logic.
Famous examples
- 2020GPT-3
OpenAI's 175-billion-parameter language model that showed a big enough model can do new tasks from just a few examples in its prompt.
- 2023Claude
Anthropic's family of AI assistants: Transformer-based large language models trained with RLHF and Constitutional AI to be helpful, honest and harmless.
Strengths, weaknesses and when to use it
Strengths
- Instant and free compared with training.
- Anyone can do it; easy to iterate.
Weaknesses
- Results can be sensitive to wording.
- Limited by the context window and by what the model already knows.
When to use itAlways first. Move to retrieval or fine-tuning only when good prompting is not enough.
Compare with
Check yourself
- Q01
In few-shot prompting, how does the model learn the task?
Show answer
A: From examples placed in the prompt, without any change to its weights
- Q02
What does chain-of-thought prompting encourage?
Show answer
B: Writing out intermediate reasoning steps before the answer