Skip to main content

The Elements of Statistical Learning — Hastie, Tibshirani & Friedman

Review of ESL/ESLII: the classic graduate-level reference on statistical learning, its mathematical depth, and how it fits into the modern ML landscape.

finished certainty: likely importance: 8

The Elements of Statistical Learning (200125ya; 2nd edition 200917ya) by Trevor Hastie, Robert Tibshirani, and Jerome Friedman—universally known as “ESL” or “ESLII”—is one of those rare textbooks that defined its field. If you have spent any time in machine learning or applied statistics, you have either read it, been told to read it, or felt guilty about not reading it. It is freely available as a PDF from the authors’ website, which is both generous and dangerous—generous because it removes all barriers to access, dangerous because it removes all excuses.

What It Is

ESL is a graduate-level textbook covering supervised and unsupervised learning from a statistical perspective. The coverage is remarkably broad: linear methods, basis expansions, kernel smoothing, model selection, SVMs, trees, boosting, random forests, neural networks, graphical models, high-dimensional problems, and more. The unifying theme is the bias-variance tradeoff—nearly every method is analyzed through this lens, which gives the book a conceptual coherence that many ML textbooks lack.1

The writing style is dense but precise. Hastie, Tibshirani, and Friedman are statisticians, and it shows: everything is framed in terms of estimation, risk, and loss functions. This is a feature, not a bug. The statistical framing gives you tools to reason about why methods work, not just how to run them. The book assumes comfort with linear algebra, multivariate calculus, and basic probability theory—roughly the level of a first-year statistics PhD student.2

What’s Great

The treatment of the bias-variance tradeoff and model selection (chapters 7–8) is definitive. I have not found a better explanation anywhere of why cross-validation works, when it fails, and how to think about the effective number of parameters. The discussion of the bootstrap in chapter 8 is similarly authoritative—unsurprising, given that Efron (who invented the bootstrap) is a Stanford colleague.

The chapters on additive models, trees, and ensemble methods (chapters 9–10, 15–16) are outstanding. The boosting chapter in particular provides a clear derivation of AdaBoost as forward stagewise additive modeling, connecting what initially seems like an ad hoc algorithm to a principled statistical framework. Gradient boosting, which Friedman himself developed, gets the most thorough treatment you’ll find anywhere.3 This matters practically: gradient-boosted trees (XGBoost, LightGBM) remain the dominant method for tabular data in 2026, and understanding the theory behind them is genuinely useful.

Chapter 3 on linear methods is a masterclass. You might think there’s nothing new to learn about linear regression—you’d be wrong. The geometric interpretation of OLS, ridge, and lasso; the explicit connection between ridge regression and PCA; the derivation of the lasso from a Bayesian perspective—all of this is handled with an elegance that makes the familiar feel fresh.4

What’s Dated or Difficult

The neural networks chapter (chapter 11) is thin and dated. Written from a pre-deep-learning perspective, it covers single-hidden-layer networks and briefly mentions backpropagation. This was already somewhat behind the times in 200917ya; by 2026, it is purely historical. If you want neural networks, look elsewhere—Goodfellow et al. or d2l.ai will serve you far better.

The book’s density is a real obstacle. This is not a gentle introduction; it is a reference work that happens to be ordered pedagogically. Many readers bounce off chapters 5–6 (basis expansions and kernel smoothing), which are mathematically heavy and less immediately applicable than the tree-based methods that follow. My advice: skip what’s hard on first pass and come back to it when you need it.5

The lack of code is a deliberate choice—the authors wanted the focus on concepts, not implementation—but it does make ESL harder to learn from actively. This is partly addressed by An Introduction to Statistical Learning (ISL/ISLR), the junior companion book by Hastie, Tibshirani, James & Witten, which covers a subset of the same material at a lower mathematical level with R (and now Python) labs.6

Who Should Read Which Parts

  • Everyone in ML: Chapters 2–4 (overview, linear methods, linear classification), Chapter 7 (model assessment and selection). These are foundational.

  • Practitioners working with tabular data: Chapters 9–10 (additive models, boosting), Chapter 15 (random forests). This is where the practical money is—these methods still dominate Kaggle and industry.

  • Aspiring researchers: Chapters 5–6 (smoothing, kernels), Chapter 12 (SVMs), Chapter 14 (unsupervised learning). Deeper theory that informs how you think about function approximation.

  • Skip or skim: Chapter 11 (neural networks—outdated), Chapter 13 (prototype methods—rarely used), parts of Chapter 17 (undirected graphical models—unless specifically needed).

  • ISL first, then ESL: If chapters 2–4 of ESL feel impenetrable, read ISL first. It covers the same ground at a friendlier level and will prepare you for the real thing.

How It Fits in 2026

ESL occupies an interesting position. The methods it covers best—trees, ensembles, regularized linear models—are exactly the methods that remain dominant for structured/tabular data, where deep learning has still not convincingly won.7 The statistical perspective it teaches—thinking in terms of bias, variance, risk, and degrees of freedom—is more important than ever in an era where people throw transformers at problems that a well-tuned gradient boosting model would solve better and cheaper.

At the same time, ESL says essentially nothing about deep learning, transformers, large language models, or the representation learning paradigm that now dominates computer vision and NLP. It is not a book about modern AI in the popular sense. It is a book about the mathematical theory of learning from data, with an emphasis on interpretable and well-understood methods.

Rating: 8/10. A genuine classic. Dense and demanding, but the payoff—a deep understanding of the statistical foundations of machine learning—is worth it. Read ISL first if you need to; but eventually, read this.8

Further Reading