Introduction to Machine Learning Systems

A principle-driven textbook that teaches students and practitioners to reason quantitatively about machine learning systems, from data pipelines to deployment.

Machine learning has crossed from research into engineering practice, yet the field lacks a comprehensive treatment of principles, vocabulary, and quantitative reasoning tools. Filling that gap, this innovative textbook treats machine learning systems not as a collection of tools and frameworks, but as an engineering discipline governed by physical constraints. Introduction to Machine Learning Systems develops quantitative frameworks that decompose system performance into measurable components, giving readers the ability to diagnose bottlenecks, predict trade-offs, and design systems that work—by reasoning from first principles, not recipes.
Organized in four parts—Foundations, Build, Optimize, and Deploy—the book covers the complete ML systems lifecycle: data engineering, neural network computation and architectures, framework internals, training infrastructure, data selection, model compression, hardware acceleration, benchmarking, serving systems, ML operations, and responsible engineering including fairness, privacy, security, and sustainability. The scope encompasses systems from embedded devices to cloud-based accelerators on a single compute node, the fundamental unit of ML computation and the prerequisite for everything built on top of it.
Develops quantitative reasoning tools that let readers diagnose system bottlenecks and predict trade-offs Covers the full ML systems lifecycle end-to-end, from data pipelines through training, optimization, deployment, and operations Teaches enduring principles rather than current tools Treats fairness, privacy, security, and environmental sustainability as engineering problems with measurable solutions Features rich pedagogy including learning objectives, self-check questions, worked calculations, and real-world production failure case studies Is based on the authors popular Harvard course and the TinyML edX program Offers interactive labs, lecture slides, and the companion TinyTorch educational framework