Cross-Validation
Estimating how well a model will do on new data by training and testing it several times on different splits of the same dataset.
Also k-fold cross-validation · CV · leave-one-out · stratified k-fold
Instead of one practice exam, you split your study cards into five piles. Five times, you study four piles and quiz yourself on the fifth. Your average quiz score is a much fairer measure of how ready you are.
Where it fits
+All relationships as a list
Is a kind of / part of
Siblings
- Accuracy
- Bias-Variance Trade-off
- Curse of Dimensionality
- Data, Features and Labels
- Gradient Descent
- Hyperparameters
- Linear Algebra Basics
- Log Loss (Cross-Entropy)
- Loss Function
- Overfitting
- Precision, Recall and F1
- Probability Basics
- Regression Metrics
- Regularization
- ROC Curve and AUC
- Train, Validation and Test Sets
Uses / built on
Used by
How it works
A single train/test split gives one number, and that number depends on which examples happened to land in the test set. With small datasets the luck of the split can matter more than the model.
k-fold cross-validation fixes this:
- Shuffle the data and cut it into equal folds (5 or 10 is typical).
- For each fold: train on the other folds, evaluate on this one.
- Report the average score (and its spread).
It is the standard way to tune hyperparameters on small data. The rule still applies: once you have chosen, evaluate one final time on a test set that played no part in any of this.
Choose the right flavor:
- Stratified: keeps the share of each class equal in every fold, important for rare classes like fraud.
- Grouped: keeps all records from the same patient or user in the same fold, so the model cannot cheat by recognizing them.
- Time-series: folds move forward in time; you always train on the past and test on the future.
Everyday use cases
- Case 01
Picking hyperparameters on a small dataset
Choosing the regularization strength for a medical study with 300 patients, where a single split would be too noisy.
- Case 02
Comparing models fairly
Deciding between logistic regression and gradient boosting on the same folds.
Strengths, weaknesses and when to use it
Strengths
- Uses every example for both training and validation, so estimates are less noisy.
- The spread across folds shows how stable the model is.
Weaknesses
- k times more expensive than a single split; rarely used for large deep learning models.
- Wrong for time series or grouped data unless you use the matching variant, otherwise it leaks information.
When to use itSmall to medium datasets with fast-to-train models. Still keep a final test set untouched.
Compare with
Check yourself
- Q01
In 5-fold cross-validation, how many times is the model trained?
Show answer
C: 5
- Q02
You forecast sales. Why is plain shuffled k-fold a bad idea?
Show answer
B: It trains on the future to predict the past