L2 · 02 · Structured Output & JSON Schema

When code consumes model output, free text is a liability. This page shows how to get typed, validated data out of a model and into Scala case classes and Python models, with a repair loop for the cases where it still goes wrong.

The problem

A customer emails the store: "Hi, order ORD-482913. The left earbud from the headphones is dead and I'd also like to send back the phone case, wrong colour. Refund please. I can drop them at the Riverside store." Your returns system needs this:

{
  "orderId": "ORD-482913",
  "items": [
    { "sku": "HEADPHONES", "quantity": 1, "reason": "DAMAGED" },
    { "sku": "PHONE_CASE", "quantity": 1, "reason": "WRONG_ITEM" }
  ],
  "wantsRefund": true,
  "storeId": "RIVERSIDE"
}

A model answering in prose ("Sure! Here's the extracted info: …") breaks your parser. Even "JSON only" prompts sometimes add code fences, comments, trailing commas, or a different field name. You need guarantees.

Four levels of strictness

Validation is still your job

Schema enforcement guarantees shape, not truth. The model can still return a well-formed but invented order ID, a wrong reason, or a quantity of 0. Always run domain validation in code.

Designing schemas models get right

• Use enums for categories ("reason": {"enum": ["DAMAGED", ...]}). Never free text when the set is known.
• Describe every field. Descriptions are part of the prompt: "description": "Order ID, format ORD-123456. Do not guess."
• Mark required fields, and set "additionalProperties": false.
• Represent "unknown" explicitly. Use a nullable field (["string", "null"]) rather than letting the model invent a value.
• Keep it flat and small. Deeply nested, huge schemas reduce accuracy. Split into multiple calls if needed.
• Put reasoning first when needed. An optional "reasoning" string before the answer fields lets the model think before committing (it generates in order).
• Use one naming convention that matches your code (camelCase for Scala/JSON here).

{
  "type": "object",
  "additionalProperties": false,
  "required": ["orderId", "items", "wantsRefund", "storeId"],
  "properties": {
    "orderId":     { "type": ["string", "null"], "description": "Order ID in format ORD-123456. null if not stated. Never guess." },
    "items": {
      "type": "array",
      "description": "Items the customer wants to return",
      "items": {
        "type": "object",
        "additionalProperties": false,
        "required": ["sku", "quantity", "reason"],
        "properties": {
          "sku":      { "type": "string",  "description": "Product code from the catalogue list provided" },
          "quantity": { "type": "integer", "description": "Number of units, at least 1" },
          "reason":   { "type": "string",  "enum": ["DAMAGED", "WRONG_ITEM", "NOT_AS_DESCRIBED", "CHANGED_MIND", "OTHER"] }
        }
      }
    },
    "wantsRefund": { "type": "boolean", "description": "true for refund, false for exchange/store credit" },
    "storeId":     { "type": ["string", "null"], "description": "Store ID from the store list if the customer named one, else null" }
  }
}

The request (OpenAI-compatible shape)

{
  "model": "<model>",
  "temperature": 0,
  "messages": [
    { "role": "system", "content": "Extract return requests from customer emails. Valid SKUs: HEADPHONES, PHONE_CASE, CHARGER, TABLET. Valid stores: RIVERSIDE, CENTRAL, AIRPORT." },
    { "role": "user",   "content": "<email>Hi, order ORD-482913 ...</email>" }
  ],
  "response_format": {
    "type": "json_schema",
    "json_schema": { "name": "return_request", "strict": true, "schema": { "...": "the schema above" } }
  }
}

Notice the allowed values in the system prompt: the model maps "phone case, wrong colour" to PHONE_CASE / WRONG_ITEM only because you told it the valid codes. Page 03 sends this request from real code.

Scala 3: decode, validate, retry

This example runs offline: it feeds sample "model outputs" through the same pipeline you would use in production.

//> using scala 3
//> using dep io.circe::circe-core:0.14.15
//> using dep io.circe::circe-parser:0.14.15
import io.circe.{Decoder, Encoder}
import io.circe.parser.decode
import io.circe.syntax.*

enum ReturnReason:
  case DAMAGED, WRONG_ITEM, NOT_AS_DESCRIBED, CHANGED_MIND, OTHER

object ReturnReason:
  given Decoder[ReturnReason] = Decoder.decodeString.emap: s =>
    values.find(_.toString == s).toRight(s"unknown reason '$s'")
  given Encoder[ReturnReason] = Encoder.encodeString.contramap(_.toString)

final case class ReturnItem(sku: String, quantity: Int, reason: ReturnReason) derives Decoder, Encoder.AsObject
final case class ReturnRequest(
    orderId: Option[String],
    items: List[ReturnItem],
    wantsRefund: Boolean,
    storeId: Option[String]
) derives Decoder, Encoder.AsObject

val KnownSkus   = Set("HEADPHONES", "PHONE_CASE", "CHARGER", "TABLET")
val KnownStores = Set("RIVERSIDE", "CENTRAL", "AIRPORT")
val OrderIdFmt  = """ORD-\d{6}""".r

/** Business rules the schema cannot express. Returns ALL problems, not just the first. */
def validate(r: ReturnRequest): Either[List[String], ReturnRequest] =
  val errors =
    r.orderId.filterNot(OrderIdFmt.matches).map(id => s"orderId '$id' must match ORD-123456").toList ++
    Option.when(r.items.isEmpty)("items must not be empty").toList ++
    r.items.filter(_.quantity < 1).map(i => s"quantity for ${i.sku} must be >= 1") ++
    r.items.filterNot(i => KnownSkus(i.sku)).map(i => s"unknown sku '${i.sku}'") ++
    r.storeId.filterNot(KnownStores).map(s => s"unknown storeId '$s'").toList
  if errors.isEmpty then Right(r) else Left(errors)

def parse(raw: String): Either[List[String], ReturnRequest] =
  decode[ReturnRequest](raw).left.map(e => List(s"invalid JSON for schema: ${e.getMessage}")).flatMap(validate)

/** ask(feedback) calls the model; feedback is None on the first attempt. */
def extractWithRetry(ask: Option[String] => String, maxAttempts: Int = 3): Either[List[String], ReturnRequest] =
  def loop(attempt: Int, feedback: Option[String]): Either[List[String], ReturnRequest] =
    parse(ask(feedback)) match
      case ok @ Right(_)                     => ok
      case Left(errs) if attempt < maxAttempts =>
        loop(attempt + 1, Some(s"Your previous answer had these problems:\n- ${errs.mkString("\n- ")}\nFix them and answer again."))
      case failed                            => failed // escalate to a human
  loop(1, None)

@main def demo(): Unit =
  // Simulated model: first answer has a bad SKU and quantity, second is corrected
  val answers = Iterator(
    """{"orderId":"ORD-482913","items":[{"sku":"EARBUDS","quantity":0,"reason":"DAMAGED"}],"wantsRefund":true,"storeId":"RIVERSIDE"}""",
    """{"orderId":"ORD-482913","items":[{"sku":"HEADPHONES","quantity":1,"reason":"DAMAGED"},{"sku":"PHONE_CASE","quantity":1,"reason":"WRONG_ITEM"}],"wantsRefund":true,"storeId":"RIVERSIDE"}"""
  )
  val result = extractWithRetry { feedback =>
    feedback.foreach(f => println(s"--- feedback sent to model ---\n$f\n"))
    answers.next()
  }
  println(result.map(_.asJson.spaces2))

Points worth noticing:

• Enums decode strictly. An unknown reason fails decoding instead of silently becoming a string.
• Option maps to nullable, matching ["string","null"] in the schema.
• Validation accumulates all errors, so the model can fix everything in one retry.
• The feedback is specific: exact problems, not "try again".

Python: pydantic does both jobs

Pydantic v2 models give you validation and can generate the JSON Schema to send to the model, so there is one source of truth.

from enum import Enum
from typing import Callable
from pydantic import BaseModel, ConfigDict, Field, ValidationError, field_validator

KNOWN_SKUS = {"HEADPHONES", "PHONE_CASE", "CHARGER", "TABLET"}
KNOWN_STORES = {"RIVERSIDE", "CENTRAL", "AIRPORT"}


class ReturnReason(str, Enum):
    DAMAGED = "DAMAGED"
    WRONG_ITEM = "WRONG_ITEM"
    NOT_AS_DESCRIBED = "NOT_AS_DESCRIBED"
    CHANGED_MIND = "CHANGED_MIND"
    OTHER = "OTHER"


class ReturnItem(BaseModel):
    model_config = ConfigDict(extra="forbid")
    sku: str = Field(description="Product code from the catalogue list provided")
    quantity: int = Field(ge=1, description="Number of units, at least 1")
    reason: ReturnReason

    @field_validator("sku")
    @classmethod
    def known_sku(cls, v: str) -> str:
        if v not in KNOWN_SKUS:
            raise ValueError(f"unknown sku '{v}'")
        return v


class ReturnRequest(BaseModel):
    model_config = ConfigDict(extra="forbid")
    orderId: str | None = Field(pattern=r"^ORD-\d{6}$", description="ORD-123456, null if not stated")
    items: list[ReturnItem] = Field(min_length=1)
    wantsRefund: bool
    storeId: str | None

    @field_validator("storeId")
    @classmethod
    def known_store(cls, v: str | None) -> str | None:
        if v is not None and v not in KNOWN_STORES:
            raise ValueError(f"unknown storeId '{v}'")
        return v


SCHEMA = ReturnRequest.model_json_schema()   # send this in response_format


def extract_with_retry(ask: Callable[[str | None], str], max_attempts: int = 3) -> ReturnRequest:
    feedback = None
    for attempt in range(1, max_attempts + 1):
        raw = ask(feedback)
        try:
            return ReturnRequest.model_validate_json(raw)
        except ValidationError as e:
            problems = [f"{'.'.join(map(str, err['loc']))}: {err['msg']}" for err in e.errors()]
            feedback = "Your previous answer had these problems:\n- " + "\n- ".join(problems) + "\nFix them and answer again."
    raise RuntimeError(f"extraction failed after {max_attempts} attempts; escalate to a human")


if __name__ == "__main__":
    answers = iter([
        '{"orderId":"ORD-482913","items":[{"sku":"EARBUDS","quantity":0,"reason":"DAMAGED"}],"wantsRefund":true,"storeId":"RIVERSIDE"}',
        '{"orderId":"ORD-482913","items":[{"sku":"HEADPHONES","quantity":1,"reason":"DAMAGED"}],"wantsRefund":true,"storeId":"RIVERSIDE"}',
    ])

    def fake_ask(feedback: str | None) -> str:
        if feedback:
            print("--- feedback ---\n" + feedback + "\n")
        return next(answers)

    print(extract_with_retry(fake_ask).model_dump_json(indent=2))

Common failure modes