L2 · 01 · Advanced Prompt Patterns

Level 1 taught the anatomy of a good prompt. Level 2 is about systems of prompts: breaking work into steps, chaining outputs, making the model check itself, and building a prompt library you can version and test like code.

From one prompt to a pipeline

A single prompt that tries to "read 200 support tickets, find the top issues, propose fixes and write a report" will be shallow and hard to debug. Split it into steps, where each step has one job and a checkable output.

Rules for good chains

• One responsibility per step. Classification, extraction, analysis and writing are different jobs.
• Structured hand-offs. Steps exchange JSON or typed objects, not free prose (page 02).
• Deterministic glue. Anything that code can do exactly (filter, count, join, sort), do in code.
• Inspect every intermediate. When the report is wrong, you can see which step failed.
• Right-size models per step. A small, fast model can classify. Use a stronger one for analysis.

Pattern catalogue

1. Plan → execute

Ask for a plan as structured output, review or validate it, then execute each step with a focused prompt.

You are planning a data migration for the store inventory service.
Goal: move stock levels from the legacy table `inv_old` to `inventory` (new schema below).
Return ONLY a JSON array of steps: [{"id":1,"action":"...","risk":"low|medium|high","verify":"..."}]
Max 8 steps. Every step must include how to verify it.

Here is the approved plan: <plan JSON>. Execute step 3 only: "<action>".
Produce the SQL and a verification query. Do not perform other steps.

2. Generator → critic → reviser

One call writes, another critiques against explicit criteria, a third revises. Separate calls (or at least separate roles) work better than "write it and check it" in one breath, because the critic is not anchored on defending its own text.

You are reviewing a customer-facing reply drafted by another assistant.
Criteria (score each 1-5 and quote the problem): accuracy vs. the order data provided,
policy compliance (returns within 14 days, no promises about delivery dates), tone (friendly,
concise), and clarity. Then list concrete edits. Do not rewrite the reply yourself.
Order data: <json>
Draft reply: <text>

3. Self-consistency (sample and vote)

For classification or reasoning where one answer is right, run the same prompt N times at moderate temperature and take the majority. Disagreement is a useful signal: route low-agreement cases to a human.

from collections import Counter

def classify_with_vote(ticket: str, ask, n: int = 5) -> tuple[str, float]:
    """ask(prompt, temperature) -> str. Returns (label, agreement)."""
    prompt = f"Classify this support ticket as one of: PICKUP, RETURN, PAYMENT, PRODUCT, OTHER.\nAnswer with the label only.\n\n{ticket}"
    votes = Counter(ask(prompt, temperature=0.7).strip().upper() for _ in range(n))
    label, count = votes.most_common(1)[0]
    return label, count / n          # e.g. ("RETURN", 0.8); below 0.6 -> human review

4. Extract → then reason

Ask the model first to extract the relevant facts (quotes, numbers, IDs) from long input, and only then answer using the extracted facts. This reduces hallucination on long documents and makes the reasoning auditable.

Step 1: From the policy document below, quote verbatim every sentence relevant to returning opened items.
Step 2: Using ONLY those quotes, answer: can a customer return opened headphones after 10 days?
If the quotes don't answer it, say "Not covered by policy".

5. Routing

A cheap first call decides which prompt or model handles the request: order-status questions go to a tool-using flow, policy questions go to RAG, and complaints go to a human. Routing keeps every downstream prompt small and specialised.

6. Role-separated perspectives

Ask for the same artefact reviewed from several perspectives (security, performance, store-operations, accessibility), each as a separate call, then merge. It surfaces issues one generic review misses.

7. Ask for uncertainty explicitly

For each field you extract, include "confidence": "high" | "medium" | "low".
Use "low" if the value is inferred rather than stated. Never invent an order ID.

8. Meta-prompting

Use a strong model to improve your prompt: "Here is my prompt and 5 inputs where it failed, with expected outputs. Diagnose why, and propose a revised prompt." Then test the revision (see below). Don't just trust it.

System prompts that hold up

A production system prompt is read on every request, so it deserves the same care as code.

# Role
You help retail store staff answer customer questions about orders, pickup and returns.

# Facts you may rely on
Only the data in the <context> block and results from tools. If something is not there,
say you don't know and suggest checking the order system.

# Policies
- Returns: 14 days from pickup/delivery, receipt or order ID required.
- Never promise delivery dates; give the "estimated" date from the order data only.
- Never reveal another customer's data.

# Style
Friendly, concise (max 120 words), plain language. Use the customer's first name if known.

# Output
Reply text only. If you need a tool, call it rather than guessing.

• Sections with headings (or XML-style tags) help models follow long instructions.
• State priorities when rules can conflict ("privacy over helpfulness").
• Give the reason behind non-obvious rules, so the model generalises correctly.
• Keep examples realistic and diverse. Models over-copy a single example.
• Put variable data in clearly delimited blocks (<context>...</context>) separate from instructions. This also helps against prompt injection.

Prompts as code

Once prompts drive features, treat them like source code:

//> using scala 3
final case class PromptTemplate(name: String, text: String):
  private val Placeholder = """\{\{(\w+)\}\}""".r

  /** Fails loudly if a variable is missing or unused, so typos never reach the model. */
  def render(vars: Map[String, String]): Either[String, String] =
    val needed  = Placeholder.findAllMatchIn(text).map(_.group(1)).toSet
    val missing = needed -- vars.keySet
    val extra   = vars.keySet -- needed
    if missing.nonEmpty then Left(s"$name: missing ${missing.mkString(", ")}")
    else if extra.nonEmpty then Left(s"$name: unused ${extra.mkString(", ")}")
    else Right(Placeholder.replaceAllIn(text, m => scala.util.matching.Regex.quoteReplacement(vars(m.group(1)))))

@main def demo(): Unit =
  val classify = PromptTemplate(
    "classify_ticket.v3",
    """Classify the ticket between the tags as PICKUP, RETURN, PAYMENT, PRODUCT or OTHER.
      |Answer with the label only.
      |<ticket>{{ticket}}</ticket>""".stripMargin
  )
  println(classify.render(Map("ticket" -> "My pickup order says ready but the store can't find it")))
  println(classify.render(Map("tiket" -> "typo")))

import json, pathlib

def run_golden(classify, path="golden/classify_ticket.jsonl", min_accuracy=0.9) -> None:
    cases = [json.loads(l) for l in pathlib.Path(path).read_text().splitlines() if l.strip()]
    failures = [c for c in cases if classify(c["input"]) != c["expected"]]
    acc = 1 - len(failures) / len(cases)
    for f in failures[:10]:
        print(f"FAIL expected={f['expected']!r} input={f['input'][:70]!r}")
    print(f"accuracy={acc:.0%} on {len(cases)} cases")
    assert acc >= min_accuracy, "prompt regression"

Long inputs: map → reduce

When input is bigger than you want in one context (a month of tickets, a large codebase), map a prompt over chunks (extract or summarise each), then reduce the partial results with a final prompt. This is the same pattern as Spark's map/reduce, applied to language.