Large Language Model
A very large Transformer trained to predict the next token on vast amounts of text, then tuned to follow instructions. The engine behind chat assistants.
Also LLM · foundation model · language model
Someone who has read a giant library and became amazingly good at guessing what word comes next. Guessing well enough turns out to require knowing grammar, facts and even some reasoning.
Where it fits
Not shown: 2 more kinds and parts, 18 more it builds on. See the full list below.
+All relationships as a list
Is a kind of / part of
Kinds and parts
Siblings
- AI Agents and Tool Use
- BERT
- Direct Preference Optimization
- Embedding
- Fine-Tuning
- Google Translate
- GPT
- Hallucination
- In-Context Learning
- Keyboard Autocomplete
- LLM Evaluation and Benchmarks
- Perplexity
- Pretraining
- Prompting
- Retrieval-Augmented Generation
- RLHF
- Sampling and Temperature
- Scaling Laws
- Tokenization
- Vector Database
- word2vec
Uses / built on
Trained with
Techniques that apply here
- Chain-of-Thought
- Constitutional AI (RLAIF)
- Direct Preference Optimization
- Fine-Tuning
- Hallucination
- In-Context Learning
- Instruction Tuning (SFT)
- LLM Evaluation and Benchmarks
- LoRA (Low-Rank Adaptation)
- Mechanistic Interpretability
- Perplexity
- Pretraining
- Prompt Injection
- Prompting
- Quantization
- Red Teaming
- Retrieval-Augmented Generation
- RLHF
- Scaling Laws
How it works
A large language model is, at heart, a next-token predictor. Given some text, it outputs a probability for every possible next token. Pick one (see sampling), append it, repeat, and the model writes.
How it is built:
- Pretraining. A decoder-only Transformer reads trillions of tokens of text and code and learns to predict each next token. This is self-supervised: the text provides its own answers. To predict well, the model has to absorb grammar, facts, styles and patterns of reasoning.
- Instruction tuning. The raw model continues text but does not follow instructions well. Fine-tuning on examples of good assistant answers teaches the format.
- Preference tuning. Methods like RLHF train the model toward answers that people (or AI feedback guided by principles) rate as helpful, honest and harmless.
“Large” matters: as models, data and compute grow, the loss falls smoothly and predictably, and new abilities show up, such as following complex instructions or writing working code.
Limits to remember: the model generates plausible text, which is usually but not always true. It also only knows what was in its training data. Retrieval-augmented generation and tool use address both.
Everyday use cases
- Case 01
Chat assistants
Answer questions, explain concepts, draft emails, brainstorm.
- Case 02
Coding help
Suggest code, explain errors, write tests.
- Case 03
Summarizing and extracting
Turn a 40-page report into five bullet points, or pull names and dates out of contracts.
Famous examples
- 2022ChatGPT
OpenAI's chat assistant, launched as a free research preview on 30 November 2022, which brought large language models to the general public.
- 2023Claude
Anthropic's family of AI assistants: Transformer-based large language models trained with RLHF and Constitutional AI to be helpful, honest and harmless.
- 2023Llama
Meta's family of language models whose weights are downloadable, which made it a foundation for a huge ecosystem of open models, fine-tunes and local AI.
- 2025DeepSeek-R1
An open-weight reasoning model from DeepSeek, released in January 2025 under the MIT license, trained with large-scale reinforcement learning to think step by step before answering.
- 2020GPT-3
OpenAI's 175-billion-parameter language model that showed a big enough model can do new tasks from just a few examples in its prompt.
- 2023Mistral AI Models
Language models from the Paris start-up Mistral AI, known for efficient open-weight releases such as Mistral 7B and the mixture-of-experts Mixtral 8x7B.
Strengths, weaknesses and when to use it
Strengths
- One model handles thousands of different tasks from a plain-language instruction.
- Learns new tasks from a few examples in the prompt (in-context learning).
Weaknesses
- Can state false things confidently (hallucination).
- Knowledge is frozen at training time unless connected to tools or retrieval.
- Expensive to train and run.
When to use itFor language tasks that are open-ended or hard to specify with rules. Pair with retrieval when facts must be current or private.
Compare with
Check yourself
- Q01
What is the core pretraining task of a GPT-style LLM?
Show answer
B: Predicting the next token
- Q02
Why can an LLM "hallucinate"?
Show answer
A: It generates plausible text, which is not always checked against facts