The model generates.
The harness decides.
Inference engineering and harness engineering, taught from first token to production. Real math, custom diagrams, and workbenches that let you break the system on purpose.
- 27 lessons
- 2 tracks
- 3 labs
- No login · no tracking
Inference makes the tokens. The harness makes them count.
Learn the whole system, not one layer.
Inference decides how fast and cheaply a model thinks. The harness decides what it is allowed to do with that thought. Most production failures live in the seam between them.
Inference engineering
Turn trained weights into a reliable stream of tokens. Follow a request through prefill and decode, then learn to size the memory, batch the work and defend an SLO.
Harness engineering
Turn tokens into bounded, observable work. Design the loop, tools, context, permissions, state and evals that make an agent trustworthy.
Every layer constrains the one above it.
Hardware sets the physical budget. The model spends it. Serving schedules it. The harness gives it authority. The application makes a promise to a user.
Application
The product surface: chat, IDE, API, CLI, or an automated workflow.
Harness
The runtime that gives a model memory, tools, permissions, and a stopping condition.
Serving
The scheduler and runtime that turn model weights into tokens under a latency budget.
Model
The weights and the computation: attention, MLPs, experts, activations.
Hardware
The physical budget: SMs, HBM, bandwidth, NVLink, power, and money.
Workbenches, not calculators.
Every number is an educational estimate. Change the model, the batch, the policy or the context — then look at the trace and ask what actually broke.
Serving Workbench
Size a KV cache, simulate batching, walk the roofline and see what a GPU-hour actually buys.
Harness Workbench
Give an agent tools, approval modes and a context policy. Run a mission. Read the trace and the debrief.
Tokenizer Lab
See how text becomes token IDs, how vocabularies differ, and why token counts are not word counts.
From first token to a design review.
Each track builds a vocabulary, then applies it. The capstones ask you to defend a design, not recite a definition.
Inference engineering
Mechanism before vocabulary.
You should be able to explain why a system is slow before you know the name of the optimization. Definitions come after the picture.
Numbers with error bars.
Every calculator here is an educational model. Constants are rounded, real kernels vary, and your workload is the only one that matters. Measure.
Trust is architecture.
A prompt is not a boundary. A benchmark is not a capacity plan. A confident answer is not evidence. Build the check, then trust the check.
See the whole system.
Read the primer for the mental model. Then pick a track, open a workbench, and change one variable at a time.