L4 · 05 · Reasoning Models & Test-time Compute

For a decade, making a model better meant a bigger model and more training data. Reasoning models added a second lever: let it think longer at inference time. Instead of answering immediately, they generate long internal chains of thought — exploring, checking, backtracking — and accuracy climbs with the tokens spent. This page is about how that works, when to use it, and what it costs.

The second scaling axis

Pre-training scale hit diminishing returns; test-time compute opened a new frontier. A model that "thinks" for 30 seconds can beat a bigger model that answers instantly — on the right tasks.

Where reasoning comes from

You saw this on the last page: RLVR. Train a model with reinforcement learning on tasks with checkable answers — maths with a numeric answer, code that must pass tests. Reward correct final answers, and the model discovers that long intermediate reasoning raises its reward. Emergent behaviours follow: self-verification ("wait, let me check"), backtracking ("that's wrong, try another approach") and exploration of alternatives. Nobody hand-coded those; they fall out of the objective.

Reasoning vs. non-reasoning: pick deliberately

Using a reasoning model to classify tickets burns 10–100× the tokens for no gain. Route by task (L3·07): reasoning for hard steps, fast models for the rest.

Test-time strategies (you can use some yourself)

Controlling the effort

Most reasoning APIs expose an effort/budget setting. Use it as a cost-and-latency dial:

HARD = {"maths", "debug", "plan", "prove"}
MEDIUM = {"analyse", "compare", "design"}
# everything else -> low


def route(task_kind: str) -> dict:
    """Pick model + reasoning effort per task."""
    if task_kind in HARD:
        return {"model": "reasoning-model", "effort": "high"}
    if task_kind in MEDIUM:
        return {"model": "reasoning-model", "effort": "medium"}
    return {"model": "fast-model", "effort": "low"}     # don't over-think lookups


def estimate_cost(kind: str, out_tokens: int, price_per_1k: float) -> float:
    """Thinking tokens are billed too — a 'high' effort call can emit thousands."""
    thinking = {"high": 4000, "medium": 1200, "low": 50}
    cfg = route(kind)
    total = thinking[cfg["effort"]] + out_tokens
    return round(total / 1000 * price_per_1k, 4)


if __name__ == "__main__":
    for kind in ["classify", "analyse", "maths"]:
        print(kind, "->", route(kind), "cost ~$", estimate_cost(kind, 200, 0.01))
    # reasoning tokens dwarf the visible answer — that's the trade-off

Does thinking actually help here?

Measure it, don't assume. Take 20 hard cases from your golden set (L3·04), run them at low vs. high effort, compare accuracy and token cost. Sometimes medium effort is 90% of the gain at 30% of the cost.

• Reasoning helps most when there's a checkable right answer.
• On open-ended tasks, extra thinking often just produces a longer-sounding answer.
• Thinking traces aren't guaranteed faithful — the shown reasoning can be a rationalisation. Don't treat it as an audit log.
• Latency: deep thinking means TTFT of seconds–minutes. Don't put it in the interactive path.