😱 The big idea in one sentence: when you make a model too simple it makes big mistakes
(high bias), and when you make it too complicated it memorizes the noise (high variance).
The best model is the one in the middle — the "sweet spot" where the total test error is smallest.
📊 The graph from the quiz
This learning check gives you a picture with three lines and asks you to match each line to
its name. Here is a real Monte Carlo simulation I ran (fitting polynomials of growing flexibility to
noisy data) that produces the exact same shapes:
Line A — Test MSE Line B — Squared Bias Line C — Variance
🎨 How to read this: x-axis = "flexibility" (how complicated the model is).
Red (A) dips then rises = U-shape (Test MSE).
Teal (B) starts high, drops to ~0 (Squared Bias).
Orange (C) starts ~0, climbs (Variance).
1️⃣ Question 1 — Line A is…
Line A is…
Options: Squared Bias · Variance · Test MSE ✅
✅ Answer: Line A = Test MSE
Line A is the one that goes down, then up (a "U" shape). That's the classic signature of
test error: too simple = high (left), too complicated = high (right), and just right = low (middle).
2️⃣ Question 2 — Line B is…
Line B is…
Options: Squared Bias ✅ · Variance · Test MSE
✅ Answer: Line B = Squared Bias
Line B starts high and drops toward 0 as flexibility grows. That's squared bias: a simple
model can't capture the pattern (big mistakes, high bias), but as the model gets more flexible it fits
the true shape better and its "systematic mistake" shrinks.
3️⃣ Question 3 — Line C is…
Line C is…
Options: Squared Bias · Variance ✅ · Test MSE
✅ Answer: Line C = Variance
Line C starts near 0 and climbs as flexibility grows. That's variance: a very complicated
model changes wildly depending on which random training data it saw — so its predictions "wobble" a lot
(high variance, i.e. overfitting).
🧪 Why these shapes? Where bias & variance come from
🎨 How to read this: three ways to fit noisy dots with a polynomial.
Left (flexibility=1): a straight line — misses the curves (HIGH BIAS). Middle (flexibility=3): a
good fit (balanced). Right (flexibility=10): a wild squiggly line that hugs every noise point
(HIGH VARIANCE / overfitting).
🎯 Dartboard analogy: Bias is how far your darts land from the bullseye on average
(a "systematic miss"). Variance is how spread out your darts are (are they clustered or all over the
place?). A simple model throws darts in a small cluster far from center (low variance, high bias).
A super-complex model throws darts wildly scattered (high variance). The sweet spot clusters darts
near the center.
This is the famous decomposition. The total test error is the sum of three parts. As flexibility
grows, bias goes down but variance goes up — so the total dips to a minimum then climbs.
🎨 How to read this: the red line (Test MSE) is exactly the sum of the teal
(Bias²) and orange (Variance) plus the gray noise line. Watch bias fall and variance rise as flexibility grows.
🎬 The U-shape animated
🎬 Animated: the Test MSE curve builds up as flexibility increases, tracing
the U-shape — high on the left (underfitting), low in the middle (sweet spot), high on the right (overfitting).
🧪 Try it yourself — a live "fit" game!
Drag the flexibility slider and watch how the fitted curve changes.
Too low = misses the pattern (bias). Too high = chases noise (variance). Find the sweet spot!
3
loading…
📋 Quick Answer Sheet
Line
Shape
Answer
A
U-shaped (down then up)
Test MSE
B
Starts high → drops to ~0
Squared Bias
C
Starts ~0 → climbs
Variance
🧠 Checklist — know these cold
Bias = systematic mistake (simplification error). Decreases with flexibility.
Variance = how much the model changes with different training data. Increases with flexibility.
Test MSE = Bias² + Variance + noise → U-shaped, minimum at the sweet spot.
Underfit (left): high bias, low variance, high test error.
Overfit (right): low bias, high variance, high test error.
Sweet spot = lowest test error = balance of bias and variance.
