Foundations
The shared ideas behind every ML model: data, a loss to minimize, an optimizer to minimize it, and checks that the model generalizes.
Before you learn any sport you learn to run, catch and keep score. These are the running and catching of machine learning: every model, from a straight line to ChatGPT, uses them.
Where it fits
Not shown: 13 more kinds and parts. See the full list below.
+All relationships as a list
Inside this domain
- Accuracy
- Bias-Variance Trade-off
- Cross-Validation
- Curse of Dimensionality
- Data, Features and Labels
- Gradient Descent
- Hyperparameters
- Linear Algebra Basics
- Log Loss (Cross-Entropy)
- Loss Function
- Overfitting
- Precision, Recall and F1
- Probability Basics
- Regression Metrics
- Regularization
- ROC Curve and AUC
- Train, Validation and Test Sets
Overview
Almost every machine learning method follows the same recipe:
- Collect examples. Inputs (features) and, often, the right answers (labels).
- Pick a model. A function with adjustable knobs (parameters), such as a line or a neural network.
- Define what “wrong” means. A loss function turns mistakes into a single number.
- Turn the knobs to reduce the loss. Usually with gradient descent.
- Check it works on new data. Hold back validation and test sets. A model that only remembers its training examples is overfitting; regularization and held-out test sets keep it honest. Metrics like accuracy and precision and recall measure the result.
The topics in this domain are the vocabulary for the rest of the atlas. If a term elsewhere feels slippery, it probably lives here.
Everyday use cases
- Case 01
Reading any ML paper or blog post
Loss, gradient, overfitting and accuracy show up everywhere. Knowing them makes every other topic easier.
- Case 02
Debugging a model
Most real problems (a model that memorizes, a training run that explodes) are foundations problems, not architecture problems.