L4 · 08 · Level 4 Labs

Seven labs, one per page. Level 4 is where you stop treating the model as an API and start understanding the machinery — so these labs involve implementing pieces yourself and measuring the results. Expect 10–16 hours.

Setup

• Everything from earlier levels. Python with uv.
• For labs 2–3 and the capstone: uv add torch (CPU is fine for a tiny model) and a small text file (any book/docs).
• Optional for lab 4: a local model server (vLLM, llama.cpp or Ollama) to benchmark.

Lab 1 · Implement the six maths primitives

1. Reimplement matmul, softmax, cross_entropy from page 01 without looking.
2. Add a numerically-stable softmax test with logits [1000, 1001].
3. Extend gradient descent to fit y = w*x + b (two parameters).
4. Verify against numpy if available (np.allclose).

Success: all pass tests you wrote, including the overflow case and a non-trivial gradient fit.

Lab 2 · Attention, then a tiny GPT

1. Extend page 02's attention.py to multi-head: split d into h heads, attend per head, concatenate.
2. Print the weight matrix for a 5-token input. Describe which token attends where.
3. Install PyTorch and implement (or read nanoGPT's) a 4-layer GPT; train on a small text until loss drops clearly.
4. Generate 50 tokens. Note what it learned (character-level? word-level?).

Success: a tiny model generates recognisable text patterns, and you can say what each line of the model does.

Lab 3 · SFT vs. preference: feel the difference

1. Take a base model and its instruct sibling (L2·11). Give both 5 instruction prompts; tabulate who follows instructions.
2. Write 10 preference pairs (two answers each). Note how often the "better" one is obvious vs. judgement call.
3. If you did L3·06: compare your LoRA fine-tune to the base model on instruction-following.

Success: a written comparison showing you can explain why the instruct model behaves differently.

Lab 4 · Benchmark a serving setup

1. Serve a model (vLLM/llama.cpp/Ollama). Measure TTFT and tokens/s for a 20-token and a 2000-token prompt.
2. Fire 1, then 5, then 10 concurrent requests. Plot throughput and per-request latency.
3. Estimate KV-cache memory for your model at 4k and 64k context; check actual memory.
4. Try a smaller quantisation. Re-measure.

Success: numbers showing prefill vs. decode bottlenecks and the latency/throughput trade-off.

Lab 5 · Measure the reasoning dial

1. Pick 10 hard problems (maths/logic/multi-step) and 10 easy ones.
2. Run on a standard model, then a reasoning model (or low vs. high effort).
3. Compare accuracy and total tokens. Where does reasoning pay off? Where is it waste?
4. Try self-consistency (5 samples, majority) on the standard model — how close does it get?

Success: a table showing you know when reasoning is worth the cost — on your own tasks, not a blog's.

Lab 6 · Design an AI gateway

1. Design a gateway for a real product: routing rules, per-feature budgets, guardrail hooks, caching, fallback model.
2. Implement a thin version (Python): a function that routes by task, logs cost, enforces a budget, falls back on error.
3. Add a second provider behind a config flag. Show a failover works.

Success: a working gateway where swapping the model is a one-line change and every call is traced and budgeted.

Lab 7 · Write a model card + risk review

1. Write a one-page model card for the Store Assistant (or a feature you know): purpose, data, evals, limitations, oversight.
2. Classify it under the EU AI Act tiers. What changes if it acts autonomously?
3. Walk the OWASP LLM Top 10 + your MRM checklist: top 3 gaps, and what you'd fix first.

Success: documentation a competent outsider could use to reproduce your risk assessment.

Then the capstone