Skip to main content

Pattern Recognition and Machine Learning — Bishop

Review of Bishop’s PRML: the Bayesian perspective on machine learning, its thorough treatment of probabilistic models, and where it stands in 2026.

finished certainty: likely importance: 7

Pattern Recognition and Machine Learning (200620ya) by Christopher Bishop is the Bayesian counterpart to ESL. Where Hastie, Tibshirani & Friedman approach machine learning from a frequentist statistical tradition, Bishop builds nearly everything on Bayesian inference: prior distributions, posterior computation, marginal likelihoods, and model comparison via Bayes factors. If ESL teaches you to think in terms of bias and variance, PRML teaches you to think in terms of priors and posteriors. Both perspectives are valuable; which book you prefer often reveals which tribe you belong to.1

What It Covers

PRML proceeds through a carefully structured sequence: probability distributions and their properties (chapters 1–2), linear models for regression and classification (chapters 3–4), neural networks (chapter 5), kernel methods (chapter 6), sparse kernel machines/SVMs (chapter 7), graphical models (chapter 8), mixture models and EM (chapter 9), approximate inference (chapter 10), sampling methods (chapter 11), continuous latent variables/PCA (chapter 12), sequential data and HMMs (chapter 13), and combining models (chapter 14).

The Bayesian thread runs throughout. Linear regression is introduced via maximum likelihood, then immediately generalized to Bayesian linear regression with conjugate priors. Neural networks get a Bayesian treatment. Even SVMs—typically presented as a purely frequentist optimization problem—are connected to the Bayesian framework through the relevance vector machine (chapter 7).2

What’s Great

The graphical models chapter (chapter 8) is the single best textbook introduction to the topic I’ve encountered. Bishop explains directed and undirected graphical models, d-separation, factor graphs, and message passing with exemplary clarity. The figures are particularly well-designed—you can practically learn the material from the diagrams alone.3

The EM algorithm chapter (chapter 9) is another highlight. Bishop derives EM from first principles as coordinate ascent on a variational lower bound, then shows how K-means emerges as a special case of EM for Gaussian mixtures in the hard-assignment limit. This derivation is elegant and illuminating—once you see EM this way, the algorithm stops feeling like a magic trick and starts feeling inevitable.

Chapters 10–11, covering variational inference and MCMC, are thorough and clearly written. These are hard topics that many textbooks either skip or botch. Bishop’s treatment of variational Bayes as a generalization of EM, and his clear exposition of Hamiltonian Monte Carlo, are both excellent. If you’re working with probabilistic programming frameworks like Stan or PyMC, the theoretical background in these chapters is directly relevant.

What’s Weak or Dated

The neural networks chapter (chapter 5) is historically interesting but practically obsolete. It covers single-hidden-layer networks, backpropagation, and some regularization, but nothing about modern architectures, deep learning, or the training techniques that make contemporary neural networks work. This was already somewhat dated in 200620ya; by 2026, it is a museum piece.4

The sequential data chapter (chapter 13), focused on HMMs and linear dynamical systems, has aged poorly in the age of transformers and modern sequence models. HMMs remain relevant in specific domains (computational biology, speech processing), but the chapter’s implicit framing of them as the approach to sequences is no longer defensible.

More broadly, PRML predates the deep learning revolution entirely. There is no treatment of representation learning, attention, or the empirical successes that have shifted the field’s center of gravity. The book’s Bayesian perspective actually has a lot to say about deep learning in principle—Bayesian neural networks, uncertainty quantification, the connection between dropout and variational inference5—but none of this is in the book because it hadn’t happened yet.

Reading Guide: What to Prioritize

  • Essential: Chapters 1–2 (probability refresher, distributions), Chapter 3 (Bayesian linear regression), Chapter 8 (graphical models), Chapter 9 (EM). These form the conceptual core and remain highly relevant.

  • Very valuable: Chapter 10 (variational inference), Chapter 12 (PCA and latent variable models). Important for anyone working with generative models or probabilistic programming.

  • Good background: Chapters 4, 6–7 (classification, kernel methods, SVMs). Solid treatments, though you may prefer ESL’s coverage of some of this material.

  • Skip unless specifically needed: Chapter 5 (neural networks—outdated), Chapter 13 (sequential data—superseded by modern approaches), Chapter 14 (combining models—brief and less insightful than the rest).

PRML vs. ESL

The comparison is unavoidable. ESL is broader in coverage and more practically oriented; PRML is deeper on probabilistic methods and more theoretically unified. ESL’s strength is ensemble methods and model selection; PRML’s strength is graphical models and approximate inference. If you work primarily with tabular data and tree-based methods, ESL is more useful. If you work with probabilistic models, generative models, or anything where uncertainty quantification matters, PRML is essential.

The ideal is to read both—they complement each other surprisingly well. ESL gives you the frequentist tool set; PRML gives you the Bayesian tool set. Together, they cover classical machine learning theory more thoroughly than any other two-book combination I know of.6

The Verdict

PRML is a beautiful textbook that teaches a powerful way of thinking about machine learning. The Bayesian perspective it develops is increasingly relevant as the field grapples with uncertainty quantification, out-of-distribution detection, and calibration—problems that the Bayesian framework addresses naturally.7 The graphical models and approximate inference chapters alone justify the book.

Its main limitation is the same as every pre-2012 ML textbook: it says nothing about modern deep learning. But unlike a book that teaches only techniques, PRML teaches a framework—and that framework generalizes well to architectures Bishop could not have anticipated.

Rating: 7/10. A superb probabilistic ML textbook that rewards careful study. Pair with Goodfellow et al. for modern neural networks and Murphy’s Probabilistic Machine Learning for an updated Bayesian treatment.8

Further Reading