Machine Learning Systems

English • 中文 (earlier version) • 日本語 (earlier version) • 한국어 (earlier version)

Why This Curriculum Exists

A model that scores well on a benchmark still has to fit in memory, meet a latency budget, stay within a power limit, and fail safely in front of real people. Those constraints decide the architecture, and they follow laws that outlast any framework or chip. Memory bandwidth, network latency, energy, and failure rates govern a microcontroller with 256 KB of memory and an accelerator in a datacenter alike, even though the two differ in scale by many orders of magnitude. I learned this while working on TinyML at Google, and it became the idea that runs through every volume: constraints drive architecture.

AI engineering is the discipline of building efficient, reliable, and safe intelligent systems that operate in the real world, not just models in isolation. My aim is to help establish it alongside software engineering and computer engineering, and to teach it the way computer architecture has long been taught, from first principles and with numbers a student can recompute.

Everything lives in one repository because each part teaches something the others cannot. Students who only read do not internalize, and students who only code do not generalize. The textbook builds the mental model, TinyTorch makes it concrete by having students build the machinery themselves, and the hardware kits and the MLSys·im simulator show where the model breaks, on devices students can hold and at scales they cannot rent.

The work began in a classroom. In 2023, the students in CS249r at Harvard wrote the first class notes together, one topic per group, and we published them on GitHub. Those notes have been rewritten every semester since, and the HarvardX TinyML course that preceded them reached more than 100,000 learners. The goal now is one million learners by 2030.

There is also a personal reason, which the FAQ tells in full. As an undergraduate, I wrote to a professor I had never met and received a VHS tape in the mail that changed what I studied. This curriculum is my way of sending that package back out.

What You Will Learn

Each chapter connects an idea a reader may already know from machine learning to the systems that make it work in practice.

The Four Volumes

Each volume widens the boundary of the system and raises the cost of failure. The distinction between volumes is scope, not depth; every volume is equally quantitative.

Each volume answers a question the previous one leaves open:

• Vol I → Vol II, when one machine is not enough. Volume I covers single-machine execution, memory walls, and kernel efficiency. Volume II carries those foundations across thousands of accelerators, collective communication, fault-tolerant networks, and datacenter orchestration.
• Vol II → Vol III, when a stateless response is not enough. Volume II serves stateless requests at scale. Volume III adds stateful, multi-step loops that manage context memory, call tools, run in sandboxes, and recover from nondeterministic failures.
• Vol III → Vol IV, when software acts on the physical world. Volume III governs software environments. Volume IV crosses into physical systems, where computational latency becomes distance traveled and a failure cannot be undone.

⚠️ Volumes III and IV are open drafts. Chapters are added, reorganized, and rewritten as I iterate, so please do not cite or teach from them yet. Feedback is welcome through the book feedback issue forms.

Start Here

The pieces are designed to work together, but none of them requires the others.