AI from Scratch in C
By the last chapter you will have written a language model in C that trains and generates text, and you will understand every line of it, because you derived the mathematics yourself and then wrote the code that implements it. The book starts at a single neuron and adds one idea at a time until the whole thing is standing.
What you walk away with
Most machine learning books teach you to call a function that somebody else wrote, which works right up until the abstraction leaks and you find you have nowhere to stand. Build each piece yourself and the ground stops moving, and you end up doing it in the language the fast implementations are written in anyway.
You know where the numbers come from
Every gradient is derived on the page before it is written out in code, so when a model misbehaves you are reasoning about arithmetic you already understand instead of guessing at what a library decided on your behalf.
The code still builds later
Each listing is a single C file that compiles on its own, so there is nothing to install before you can run it and nothing that will rot the week a package decides to move on without you.
The last program grew out of the first
Each chapter works on the code from the one before it rather than starting over, which is why the neuron you write on the first day is still recognisable inside the language model you finish with.
It starts with two lines
A weighted sum followed by an activation is the entire forward pass of a single neuron, and every network in the book turns out to be a variation on it, which is why you will find the same shape at the end of the book running inside multi head attention, a KV cache and a transformer that trains and generates text.
/* chapter 1, the whole forward pass */ float z = w1 * x1 + w2 * x2 + b; float y = step_function(z); /* later on, the same shape scaled up */ for (j = 0; j < n; j++) acc += w[j] * x[j];
The whole book, free to read
Every chapter is on this site. Start at the beginning and work forward, or go straight to whatever you came looking for.
Every piece from the previous chapters assembled into one program that trains and then generates text.
If you write C, you can follow this
The mathematics arrives immediately before it gets used, so the notation and the thing it describes show up together instead of in separate chapters, and what you need to follow along is a C compiler and the willingness to run the programs and change the numbers to see what breaks.
Embedded and systems engineers
You already know pointers, memory layout and fixed point arithmetic, which is the background that makes the quantization and KV cache chapters read as engineering rather than theory.
People tired of the abstraction
If you have trained models for years through a framework and never seen a backward pass written out, this is the book that shows you the arithmetic underneath.
Students and self teachers
A laptop and a compiler are the whole requirement, since nothing here wants a GPU or a cloud account or a dataset you have to download first, and that matters a great deal if you are learning somewhere the cloud bill is not an afterthought.
Prefer a permanent copy?
The book is free to read from beginning to end and it stays that way, but if you would rather have a polished offline copy you can keep, or you simply want to support the work and the editions that come after it, the complete PDF is available.
- The complete book as a formatted PDF
- Every program, ready to compile
- Read offline, print it, keep it
- Supports future editions
One neuron to a working GPT
Hear about corrections, new chapters and whatever gets written next.