Structured outputs

When a model's output feeds into code — a database, a UI, another API, an agent's next step — free-form text is a liability. Structured outputs force the model to return data in an exact, machine-readable shape, usually JSON that matches a schema. This lesson covers every level of enforcement, from prompt instructions to schema-constrained decoding, and shows how to validate and repair outputs with Pydantic.

Last updated: 01 Nov, 2026 · Pydantic v2

Definitions

Definition · Structured output

Model output that conforms to a predefined, machine-readable format — most often a JSON object with specific keys, types, and allowed values — so code can consume it without fragile text parsing.

Definition · JSON Schema

A standard vocabulary for describing the shape of JSON data: which properties exist, their types, which are required, and allowed values (enums). Model providers accept JSON Schema to constrain output.

Definition · Constrained decoding

A generation technique where, at every step, tokens that would break the required schema are excluded before sampling. The output is therefore guaranteed to be valid against the schema.

Four levels of enforcement

Level	How	Guarantee
1. Prompt only	“Reply with JSON containing keys …”	None — usually works, sometimes adds prose, fences, or wrong keys.
2. JSON mode	`response_format={"type": "json_object"}`	Valid JSON syntax, but keys and types are not enforced.
3. Structured outputs (schema)	`response_format` with a JSON Schema, `strict: true`	Output matches your schema (constrained decoding).
4. Tool / function calling	Define a tool whose parameters are your schema	Arguments match the schema; useful when the model also decides whether to call it.

Use the strongest level your provider supports, and still validate in code — a schema guarantees shape, not correctness. A perfectly formatted "urgency": "low" can still be the wrong judgement.

Defining the schema with Pydantic

Definition · Pydantic

The most widely used Python library for data validation. You declare a class with typed fields; Pydantic validates incoming data against it, converts types where safe, and produces clear errors. It can also export a JSON Schema.

Python

from typing import Literal

from pydantic import BaseModel, Field


class Ticket(BaseModel):
    category: Literal["billing", "technical", "account", "other"]
    urgency: Literal["low", "medium", "high"]
    summary: str = Field(description="One sentence, max 25 words")
    customer_name: str | None = Field(default=None, description="Null if not mentioned")


schema = Ticket.model_json_schema()
print(list(schema["properties"]))
print(schema["properties"]["urgency"])
print(schema["required"])

['category', 'urgency', 'summary', 'customer_name'] {'enum': ['low', 'medium', 'high'], 'title': 'Urgency', 'type': 'string'} ['category', 'urgency', 'summary']

Literal restricts a field to fixed values (an enum), which is the single most useful constraint for classification tasks. Field(description=...) is included in the schema and acts as an inline instruction to the model.

Validating model output

Python

from typing import Literal

from pydantic import BaseModel, ValidationError


class Ticket(BaseModel):
    category: Literal["billing", "technical", "account", "other"]
    urgency: Literal["low", "medium", "high"]
    summary: str
    customer_name: str | None = None


good = '{"category": "technical", "urgency": "high", "summary": "Backups failed overnight.", "customer_name": "Asha"}'
bad = '{"category": "tech", "urgency": "urgent", "summary": "Backups failed."}'

ticket = Ticket.model_validate_json(good)
print(ticket.category, ticket.urgency, ticket.customer_name)

try:
    Ticket.model_validate_json(bad)
except ValidationError as err:
    for e in err.errors():
        print(e["loc"][0], "->", e["msg"])

technical high Asha category -> Input should be 'billing', 'technical', 'account' or 'other' urgency -> Input should be 'low', 'medium' or 'high'

The validation errors are precise and readable — which makes them perfect feedback to send back to the model.

Native structured outputs with the OpenAI SDK

The SDK can take a Pydantic class directly, convert it to a strict JSON Schema, and parse the response back into an instance:

Python

from typing import Literal

from openai import OpenAI
from pydantic import BaseModel

client = OpenAI()


class Ticket(BaseModel):
    category: Literal["billing", "technical", "account", "other"]
    urgency: Literal["low", "medium", "high"]
    summary: str
    customer_name: str | None


completion = client.chat.completions.parse(
    model="gpt-4.1-mini",
    messages=[
        {"role": "system", "content": "Classify the support email into a ticket."},
        {"role": "user", "content": "Hi, this is Asha. Our nightly backups failed and a client folder is missing!"},
    ],
    response_format=Ticket,
)

message = completion.choices[0].message
if message.refusal:
    print("model refused:", message.refusal)
else:
    ticket = message.parsed            # a Ticket instance
    print(ticket.model_dump())

{'category': 'technical', 'urgency': 'high', 'summary': 'Nightly backups failed and a client folder is missing.', 'customer_name': 'Asha'}

Note

Strict schema mode has rules: every field must be listed as required (use str | None for optional values rather than defaults), and some JSON Schema features are unsupported. Other providers offer equivalents — Gemini's response_schema, Anthropic's tool-use with an input schema, and grammar-constrained decoding in local runtimes such as Ollama and llama.cpp.

When native schemas are not available: validate and retry

With models or providers that only support prompting or JSON mode, wrap the call in a loop: parse, validate, and if it fails, send the error back and ask for a corrected version. The fake model below fails once, then fixes itself after seeing the error — exactly what real models usually do.

Python

import json
from typing import Literal

from pydantic import BaseModel, ValidationError


class Sentiment(BaseModel):
    label: Literal["positive", "negative", "mixed"]
    confidence: float


RESPONSES = iter([
    'Sure! {"label": "Positive", "confidence": "high"}',   # attempt 1: prose + bad values
    '{"label": "positive", "confidence": 0.91}',            # attempt 2: corrected
])


def fake_llm(messages: list[dict]) -> str:
    return next(RESPONSES)


def extract_json(text: str) -> str:
    start, end = text.find("{"), text.rfind("}")
    return text[start:end + 1] if start != -1 else text


def structured_call(messages, model_cls, max_attempts=3):
    for attempt in range(1, max_attempts + 1):
        raw = fake_llm(messages)
        try:
            return model_cls.model_validate_json(extract_json(raw))
        except (ValidationError, json.JSONDecodeError) as err:
            print(f"attempt {attempt} invalid: {str(err).splitlines()[0]}")
            messages = messages + [
                {"role": "assistant", "content": raw},
                {"role": "user", "content": f"That output was invalid:\n{err}\n"
                                            "Reply with ONLY corrected JSON matching the schema."},
            ]
    raise RuntimeError("could not get valid output")


result = structured_call([{"role": "user", "content": "Review: Loved the speed!"}], Sentiment)
print(result)

attempt 1 invalid: 2 validation errors for Sentiment label='positive' confidence=0.91

Schema design tips

Tip	Why
Use enums (`Literal`) for categories	Eliminates spelling variants like “Tech”, “technical issue”.
Name fields clearly and add descriptions	Field names and descriptions are instructions the model reads.
Put a `reasoning` field before the answer field	The model generates fields in order, so it reasons before committing — improving accuracy.
Make “not found” representable	Nullable fields or an `"unknown"` enum value stop the model inventing data.
Keep nesting shallow	Deeply nested schemas are harder for models and for humans to debug.
Validate business rules in code	E.g. a refund amount must not exceed the order total.

Python

from typing import Literal
from pydantic import BaseModel, Field


class RefundDecision(BaseModel):
    reasoning: str = Field(description="Brief step-by-step check against the policy")
    eligible: bool
    policy_clause: Literal["annual_30_day", "monthly_no_refund", "not_covered"]
    refund_amount: float | None = Field(description="Null if not eligible")


print(list(RefundDecision.model_json_schema()["properties"]))

['reasoning', 'eligible', 'policy_clause', 'refund_amount']

Try it

Define a Pydantic model for extracting an invoice: vendor, invoice number, date, currency (enum), total, and a list of line items.
Use client.chat.completions.parse to extract it from a pasted invoice text.
Feed it an invoice with a missing date and confirm the model returns null rather than inventing one.

Key takeaways

Structured outputs turn model responses into data your code can trust in shape.
Levels of enforcement: prompt-only, JSON mode, schema-constrained structured outputs, and tool calling.
Define schemas with Pydantic: Literal enums, nullable fields, and descriptive field names.
Always validate; when native schemas are unavailable, send validation errors back and retry.
A schema guarantees format, not correctness — evaluate the values too.