💡 The big idea in one sentence: spectra have a few hundred nearly-duplicate columns, so ordinary regression chokes — PLS squeezes X into a handful of scores that are deliberately steered by y, and the only knob you must turn is how many components to keep.
0 · Setup: 8 soils, 3 channels, 1 y
The paper works with calibration (X → measured spectra) and prediction (Y → concentrations). Let's recreate that with a tiny soil-spectroscopy dataset computed live on this page — 8 soil samples, 3 reflectance channels (think 1400 / 1900 / 2200 nm), 1 lab measurement (water lost after 24 h drying, %).
n = 8 samplesm = 3 channels
corr(ch1,ch2) = 0.997 ⚠️PC1 = 99.8% of variance
Data were rounded to 0.01. Means: ch1 9.859, ch2 4.878, ch3 0.901, y −0.322. Everything below first subtracts these means (the paper's "mean-centering" step).
Imagine asking three friends who always say the same thing to estimate your exam grade. Their three answers carry ~one piece of information, not three. A regression that treats them as three independent witnesses starts making things up — that is exactly what 99.7% correlation between channels does to a regression.
1 · Why Multiple Linear Regression breaks (paper §MLR)
The paper first shows the classic formula and its Achilles heel:
y = Xb + e, solved by b = (X′X)⁻¹X′y. It only works if X′X can be inverted — i.e. columns NOT collinear, and more samples than variables (m < n). With spectra: m ≫ n and X′X is nearly singular. 💥
b = (X′X)−1X′y ← needs (X′X)−1, dies on collinearity
Don't believe it? Here is the fitted model on our data, then re-fitted 8 times, each time dropping one sample (leave-one-out):
| model | b₁ | b₂ | b₃ |
|---|
| all 8 samples | +1.56 | +0.75 | −1.76 |
| drop #2 & refit | +4.15 | −5.44 | +0.34 |
| drop other samples (6 of 8 vary too) | b's keep swinging — see the widget below |
| LOO spread (max−min) | 2.96 | 7.40 | 2.91 |
The true slopes of all three channels are positive (every channel rises with the same underlying driver). Yet MLR hands you −1.76 for channel 3 — and after removing just ONE sample, b₂ flips from +0.75 to −5.44 (and b₃ flips sign!). With correlated columns the b's become unstable credit-shuffling: one column "explains away" another. In-sample R² looks great (0.995!); prediction is a lottery. This is exactly the m > n / singularity wall the paper warns about.
2 · The escape hatch: PCA by NIPALS (paper §PCA)
Idea: don't regress on the raw columns. Rewrite X as a sum of rank-1 outer products — big "directions" of shared variation:
X = t₁p′₁ + t₂p′₂ + … — each term is one score vector t (n×1, one number per sample: "where the sample sits on this direction") times one loading p (1×m, "the recipe of the direction"). NIPALS finds them one pair at a time, subtracts each from X, and repeats on the remainder (deflation).
start: t = any column of X → p′ = t′X / t′t, normalize → t = Xp / p′p → repeat until t stops moving → E = X − tp′ → next component
3 · PCR — and its sneaky flaw (paper §PCR)
Principal Component Regression = PCA first, then regress y on the kept scores:
Y = TB + E, solved on the first few orthogonal scores. The inversion problem disappears (scores are orthogonal), and dropping tiny components removes noise. ✅ This is the exact method the Chang-et-al 2001 soil paper used (with local n=30 calibration sets).
But PCR never looks at y while building components. Its components are picked purely by X-variance. So useful-y information can hide inside a component you threw away, and pure-noise components can survive just because they carry X-variance. The paper calls PCR "a two-step method with a risk". PLS fixes exactly this.
4 · PLS: same deflation, but y holds the steering wheel (paper §PLS)
PLS keeps the outer relations for both blocks, then ties them with an inner relation:
X = TP′ + E (outer, X side) · Y = UQ′ + F (outer, Y side) · inner relation: u = b·t — each Y-score regressed on each X-score. During the iteration the two blocks swap scores (u drives X, t drives Y), so every component is rotated toward predicting y, not just describing X.
w′ = u′X / u′u → t ← X-weights → q′ = t′Y/t′t → u ← Y-weights → … until t converges (see Appendix steps 1–13)
For a single y (PLS1) the very first weight simply points at the X-spectra's covariance with y:
w₁ = X′y / ‖X′y‖ = [47.09, 23.74, 12.31]/54.16 = [0.870, 0.438, 0.227] → t = Xw
How tight is the inner relation u ≈ b·t per component? Plot u against t for the first two components of our data:
Component 1: points almost perfectly on a line (r = 0.996) — it is pure signal for y. Component 2: mush (r = 0.60) — mostly leftover X-structure y barely cares about. That is why 1 component already captures 99.1% of y's variance on this data.
5 · The ONLY knob: number of components a
Drag the slider. Bars = regression coefficients β per channel (computed live — PLS coefficients from a-components + mean vector; more on "how" below). Table = prediction ŷ per sample.
Read the a-slider like a dimmer: a = 1 → the single "y-relevant direction", coefficients small-but-stable; a = 2 → refines residuals, best prediction; a = 3 → mathematically identical to full MLR — maximum in-sample fit, minimum stability. "How many components?" is not a detail. It IS the regularization.
6 · Choosing a with cross-validation (the paper's Fig. 6)
The paper says: compute PRESS (Prediction Residual Sum of Squares) by cross-validation and keep the a at its minimum. In-sample fit would just say "more components = better" — PRESS is the honest test. Here is leave-one-out CV, computed live for all 4 models:
7 · Prediction machine (calibration → new sample)
The second half of the tutorial: save w·p·b per component, then for a NEW sample run the same deflation on X and build up Y. Try it — presets are the 4 test inputs:
8 · "Robust" means: shake y, coefficients barely move
The paper's key sales pitch: PLS model parameters do not change much when new calibration samples are taken. Here y gets extra noise (fixed pattern z, amplitude α). Compare how each model's coefficients react:
| check | MLR | PLS (w₁) |
|---|
| drop-one-sample spread | 7.40 | 0.014 |
| what it means | coefficients ~500× less stable | the paper's "robust", measured ✔ |
📋 Quick Answer Sheet (the 1986 paper in 10 lines)
1 · Two jobs: calibration (fit model on X→Y) and prediction (apply saved w, p, q to new X). Mean-center everything first; scale only if units differ.
2 · MLR b=(X′X)⁻¹X′y needs m<n and no collinearity — spectra violate both; b's swing wildly.
3 · PCA/NIPALS: X = Σ t·p′ over orthogonal components, built one at a time by deflation; converges in a few iterations from any start.
4 · PCR = PCA + MLR on scores: stable, but components never look at y → may discard predictive directions.
5 · PLS: same deflation, but weights are chosen to maximize covariance with y (inner relation u = b·t; blocks exchange scores each iteration).
6 · Prediction from saved pairs: tₕ = w′x (then E −= t·p′, ŷ += b·t, per component) + ȳ.
7 · Number of components = the regularization: linear truth → exactly rank-of-model components needed; noise → fewer is safer.
8 · Choose a by cross-validated PRESS minimum (Fig. 6), never by in-sample fit.
9 · Property check: scores t orthogonal & centered; loadings ‖p‖=‖q‖=1 per paper's scaling; ‖F‖ shrinks per component.
10 · Diagnostics: per-variable & per-object sums-of-squares (Figs. 7–8) flag useless variables and outlier samples, component by component.
🧠 Checklist — can you…
…say why MLR fails on spectra in one sentence?
Nearly-duplicate columns ⇒ X′X almost singular ⇒ the b's divide by noise and shuffle credit unpredictably (sign flips, ±1 sample wobble).…explain NIPALS in one breath?
Guess t from a column → project to get p → re-project X to refresh t → repeat until stable → subtract tp′ → next component.…state PCR's flaw and PLS's fix?
PCR picks components blind to y; PLS rotates each component toward y covariance (outer relations + inner relation u = b·t).…name the ONE knob and the honest way to set it?
Number of components a; choose by cross-validated PRESS minimum (here: a=2), never by in-sample R².…connect this to soil spectroscopy (Chang 2001)?
Chang used PCR with local n=30 neighbor calibration sets, RPD categories A/B/C. Same math: components + regression; PLSR is the version that lets y steer the extraction — and became the soil-spectroscopy workhorse (Stenberg 2010, OSSL/MLSP pipelines).…spot the invariant that PLS3 = MLR?
a=3 components span the full column space of X, so PLS's β equals OLS exactly — a free unit test for any PLS implementation (this page runs it live in section 5).