GPT-3
OpenAI's 175-billion-parameter language model that showed a big enough model can do new tasks from just a few examples in its prompt.
Brown et al., OpenAI, 2020
Show it two examples of turning English into French, then give it a new English sentence, and it continues the pattern. Nobody trained it specifically for translation. It just learned from reading a lot.
Where it fits
+All relationships as a list
How it works
GPT-3 kept the GPT recipe (a decoder-only Transformer predicting the next token) and scaled it up to 175 billion parameters, more than 100 times its predecessor GPT-2.
The surprise was in-context learning. Put a task description and a few examples in the prompt, and GPT-3 would often perform the task on a new input, with no fine-tuning at all. Its paper was titled accordingly: “Language Models are Few-Shot Learners”.
GPT-3 was still a raw text predictor. The step from it to a helpful assistant came from instruction tuning and RLHF, which produced InstructGPT and then the models behind ChatGPT.
Everyday use cases
- Case 01
How you've probably met it
ChatGPT launched in late 2022 on a model from the GPT-3.5 series, a refined descendant of GPT-3.
Strengths, weaknesses and when to use it
Strengths
- Demonstrated in-context learning at scale, changing how people use language models.
Weaknesses
- Before instruction tuning and RLHF it often ignored instructions, and it could produce false or toxic text.
When to use itAs a milestone reference. Current models are far more capable.
Go deeper
Sources
Facts checked as of 2026-10-11