Feature Engineering
Turning raw data into the input columns a model learns from: cleaning, combining, scaling and encoding information so patterns are easy to find.
Also feature scaling · one-hot encoding · normalization · data pipeline
Before you can judge a house price, someone has to measure the rooms, count the bathrooms and note the neighborhood. Feature engineering is deciding what to measure and writing it down neatly.
Where it fits
+All relationships as a list
How it works
Models only see numbers in columns. Feature engineering decides which columns exist.
Common moves:
- Clean: fill or flag missing values, fix units, remove impossible entries.
- Encode categories: turn “red / green / blue” into numbers, for example one column per color (one-hot encoding).
- Scale: put numbers on comparable ranges, which matters for distance-based models like kNN and for gradient descent.
- Create: ratios, differences, counts, time since the last event, rolling averages. This is where domain knowledge pays off.
Two classic traps:
- Leakage: a feature that secretly contains the answer, such as “was refunded” when predicting fraud, makes a model look brilliant in testing and useless in production.
- Training-serving skew: if production computes a feature differently from training, accuracy silently drops. Production systems compute features in one shared pipeline (often a “feature store”) for both.
Deep learning learns many features itself from raw pixels or text, which is a big part of its appeal, but for spreadsheet-like data with classical models, good features still matter most.
Everyday use cases
- Case 01
Fraud detection
"Amount compared with this card's usual spending" is far more telling than the raw amount.
- Case 02
Demand forecasting
Day of week, holidays and weather turn a date into useful signals.
Strengths, weaknesses and when to use it
Strengths
- Often the biggest accuracy gain for tabular models.
- Encodes domain knowledge the model could not discover alone.
Weaknesses
- Time-consuming and easy to get subtly wrong (for example leaking future information).
- Less central for deep learning on raw images, audio and text.
When to use itAlways for tabular data and classical models; lightly for deep learning on raw signals.
Compare with
Check yourself
- Q01
What is "data leakage" in feature engineering?
Show answer
A: Using information in a feature that would not be available at prediction time