Validating AI Inputs and Outputs with Pydantic Models

Good to see you again. In the previous lesson, you used type annotations and dataclasses to make internal Python contracts explicit: a GenerationConfig object has known fields, and a typed function tells its caller what it expects.

This lesson moves to a less trustworthy boundary. HTTP request bodies, environment-derived payloads, queue messages, and especially LLM responses arrive as dictionaries, strings, or JSON that your type hints do not enforce at runtime. Pydantic turns an annotated Python model into a runtime parser and validator, so the rest of your application receives a well-defined object or a clear validation failure.

For an AI engineer, this is a core reliability practice: validate data before it enters a prompt-building pipeline, and validate a model’s structured result before it reaches a database, UI, tool, or downstream workflow.

---

Pydantic: runtime contracts at system boundaries

A dataclass is a strong representation for data your own code constructs. It does not, by itself, parse untrusted JSON or verify that a runtime dictionary contains valid values. Pydantic models do.

A Pydantic model inherits from BaseModel. Its annotations define expected fields, and constructing or validating the model runs parsing and validation.

uv add pydantic

Here is a small model for a search-like customer request:

from typing import Annotated, Literal

from pydantic import BaseModel, ConfigDict, Field


NonEmptyQuestion = Annotated[
    str,
    Field(min_length=1, max_length=4_000),
]


class CustomerQuery(BaseModel):
    model_config = ConfigDict(
        extra="forbid",
        str_strip_whitespace=True,
    )

    question: NonEmptyQuestion
    account_id: Annotated[
        str,
        Field(pattern=r"^acct_[A-Za-z0-9]+$"),
    ]
    locale: Literal["en", "es"] = "en"
    max_results: Annotated[
        int,
        Field(ge=1, le=10),
    ] = 5

This class is simultaneously:

• a readable data definition for Python developers;
• a runtime validator;
• a source of structured validation errors;
• a serializer for data that has passed validation;
• a generator of JSON Schema, which later integrations can use to describe expected structured output.

Create a validated instance from Python data with model_validate():

incoming_request = {
    "question": "  What is the refund policy?  ",
    "account_id": "acct_7K91",
    "locale": "en",
    "max_results": "3",
}

query = CustomerQuery.model_validate(incoming_request)

print(query.question)
# What is the refund policy?

print(query.max_results)
# 3

print(type(query.max_results))
# <class 'int'>

A few things happened:

1. str_strip_whitespace=True normalized the leading and trailing whitespace in question.
2. Pydantic converted "3" to the integer 3.
3. The regular expression checked the account identifier’s format.
4. The Literal annotation restricted locale to the declared values.
5. query is now a CustomerQuery, not an unstructured dictionary.

The last point matters most. Prompt construction can access query.question and query.max_results with an established contract rather than repeatedly inspecting arbitrary dictionary keys.

Given your FastAPI experience, this should look familiar: FastAPI uses Pydantic models for request and response bodies. In an AI service, use the same discipline around the model boundary. The difference is that an LLM result is often less predictable than a conventional service response, so validation failures must be an expected operational path.

---

Validation is more than checking whether JSON parses

People often say “validate the model JSON,” but there are several distinct checks involved.

1. JSON syntax: Is the text valid JSON at all?
2. Structure: Does it have the expected objects, lists, and field names?
3. Types: Can values be parsed as the declared Python types?
4. Constraints: Are lengths, numeric bounds, patterns, and allowed values respected?
5. Cross-field rules: Do fields make sense together?

For example, this request is valid JSON but is not a valid CustomerQuery:

invalid_request = {
    "question": "   ",
    "account_id": "customer-7",
    "locale": "fr",
    "max_results": 100,
    "debug": True,
}

It fails for several independent reasons:

• after whitespace stripping, question is empty;
• account_id does not match the required format;
• "fr" is not an allowed locale;
• 100 exceeds the result limit;
• debug is unexpected.

The extra="forbid" setting makes unknown fields fail validation. This is often the right default for AI application contracts because a misspelled or invented field should not silently disappear.

For example, if your application expects citations but a model emits citation, silently ignoring that field could turn an evidence-backed answer into an answer that appears uncited. Failing early exposes the integration mismatch.

By contrast, Pydantic’s default extra-field behavior is "ignore". That can be suitable when consuming a third-party payload that intentionally contains many fields you do not care about, but choose it consciously.

---

Read the contract, then make it precise

Pydantic defaults to pragmatic conversion. It might parse numeric text into an integer or turn a tuple into a list when the declared field is list[int]. That is useful at many boundaries, including form-style data and some model outputs.

But conversion is a policy decision, not a fact about correctness.

Suppose a model response includes:

{
  "confidence": "0.91"
}

Converting that string to a float may be harmless. On the other hand, accepting "yes" for a field that is supposed to be a boolean could obscure an upstream format violation. Decide which fields benefit from flexible parsing and which should be strict.

For a particularly important field, request strict validation:

class EvaluationResult(BaseModel):
    passed: Annotated[bool, Field(strict=True)]
    score: Annotated[float, Field(ge=0.0, le=1.0)]

With this definition, passed=True is valid, while passed="true" is rejected. Meanwhile, score can still use Pydantic’s default conversion behavior unless you also make it strict.

Do not make every field strict automatically. For example:

• Public API input may reasonably coerce "20" to 20, depending on how clients submit data.
• An internal event schema may be strict because a producer you own should send the exact documented types.
• LLM output often benefits from a tight schema, but whether coercion is acceptable depends on what the field controls. A numeric UI score is different from a monetary amount or an authorization decision.

---

Design an input model around the application’s actual needs

A useful schema does not mirror every detail from an incoming client payload. It represents the data your specific component needs and the rules that component owns.

For example, a retrieval endpoint might accept a validated CustomerQuery, then create a small prompt-oriented representation:

def build_retrieval_prompt(query: CustomerQuery) -> str:
    return (
        "Find policy passages relevant to this customer question.\n"
        f"Question: {query.question}\n"
        f"Locale: {query.locale}\n"
        f"Maximum passages: {query.max_results}"
    )

This function does not need to accept dict[str, object]. Its caller has already done validation, so the function can operate on a stable model.

However, schema validation is not authorization. Checking that account_id looks like acct_7K91 does not establish that the caller owns that account. Authentication, tenant isolation, and access checks remain separate security controls.

Field constraints communicate policy

Field() provides constraints that turn assumptions into executable checks:

from typing import Annotated

from pydantic import BaseModel, Field


class GenerationRequest(BaseModel):
    prompt: Annotated[str, Field(min_length=1, max_length=8_000)]
    temperature: Annotated[float, Field(ge=0.0, le=2.0)] = 0.2
    max_output_tokens: Annotated[int, Field(ge=1, le=2_048)] = 512

The Annotated[...] pattern separates two ideas cleanly:

• the Python type, such as str or int;
• Pydantic metadata, such as length and range constraints.

A static type checker still sees prompt as a str, while Pydantic sees the extra validation metadata. This gives you both editor and type-checker support, plus runtime enforcement.

---

Validate LLM output before code consumes it

An LLM may return prose, malformed JSON, a valid JSON object with missing keys, or data that has the right keys but fails constraints. Even if a provider offers a “JSON mode” or structured-output feature, keep application-side validation. The model or provider integration reduces the chance of failure; Pydantic decides whether your program accepts the received data.

Consider a document-answering service. Its downstream UI and persistence layer need an answer status, answer text, and traceable citations.

from typing import Annotated, Literal

from pydantic import (
    BaseModel,
    ConfigDict,
    Field,
    model_validator,
)


ShortText = Annotated[str, Field(min_length=1, max_length=2_000)]


class Citation(BaseModel):
    model_config = ConfigDict(extra="forbid")

    document_id: Annotated[str, Field(min_length=1, max_length=100)]
    chunk_id: Annotated[str, Field(min_length=1, max_length=150)]
    excerpt: Annotated[str, Field(min_length=1, max_length=500)]


class GroundedAnswer(BaseModel):
    model_config = ConfigDict(extra="forbid")

    status: Literal["answered", "insufficient_evidence"]
    answer: ShortText
    citations: list[Citation] = Field(default_factory=list, max_length=5)

    @model_validator(mode="after")
    def answered_response_needs_citations(self) -> "GroundedAnswer":
        if self.status == "answered" and not self.citations:
            raise ValueError(
                "an answered response must contain at least one citation"
            )
        return self

This model demonstrates three levels of contract.

Level	Example rule	Purpose
Field type	citations: list[Citation]	The output must contain a list of structured citation objects.
Field constraint	excerpt has max_length=500	Bounds output size and makes storage and UI behavior predictable.
Model invariant	An "answered" result needs a citation	Prevents a logically incomplete “grounded” answer.

The nested Citation model is not merely documentation. Given a JSON object in citations, Pydantic validates every nested field and constructs Citation instances. Application code can then safely use answer.citations[0].document_id rather than defensively walking raw dictionaries.

A valid raw model response might be:

raw_model_output = """
{
  "status": "answered",
  "answer": "The policy allows refunds within 30 days of purchase.",
  "citations": [
    {
      "document_id": "refund-policy-2025",
      "chunk_id": "refund-policy-2025:chunk-04",
      "excerpt": "Customers may request a refund within 30 days of purchase."
    }
  ]
}
"""

Because the source is JSON text, use model_validate_json():

answer = GroundedAnswer.model_validate_json(raw_model_output)

print(answer.status)
# answered

print(answer.citations[0].document_id)
# refund-policy-2025

Use model_validate() when you already have Python data, such as a dictionary returned by a framework or deserialized earlier:

raw_python_output = {
    "status": "insufficient_evidence",
    "answer": "The available documents do not establish a refund exception.",
    "citations": [],
}

answer = GroundedAnswer.model_validate(raw_python_output)

The two methods express an important engineering distinction:

Method	Use when you have	What it handles
Model.model_validate(data)	Python objects such as dict and list	Python-data validation and parsing
Model.model_validate_json(text)	Raw JSON str or bytes	JSON decoding plus validation
model_dump()	A validated Pydantic object	Conversion to ordinary Python data
model_dump_json()	A validated Pydantic object	Conversion to JSON text

For example, serialize a validated result for an HTTP response or event payload:

payload: dict[str, object] = answer.model_dump()
json_payload: str = answer.model_dump_json()

---

Treat validation failure as a normal branch

Pydantic raises ValidationError when data cannot satisfy the schema. It collects all detected validation failures rather than stopping at the first one. That is useful for client-facing request feedback and for diagnosing an LLM integration.

Keep the raw-output boundary narrow:

import logging

from pydantic import ValidationError

logger = logging.getLogger(__name__)


def parse_model_output(raw_json: str) -> GroundedAnswer | None:
    try:
        return GroundedAnswer.model_validate_json(raw_json)
    except ValidationError as error:
        logger.warning(
            "Rejected model output: %s",
            error.errors(include_url=False),
        )
        return None

The important design rule is: do not let unvalidated output pass into the next business step.

A caller can make the failure behavior explicit:

answer = parse_model_output(raw_model_output)

if answer is None:
    safe_response = {
        "status": "unavailable",
        "message": "The answer could not be processed safely.",
    }
else:
    safe_response = answer.model_dump()

For production systems, the exact response policy depends on the task:

• reject the client request with a detailed 4xx response when input is invalid;
• show a safe fallback to the user when an LLM response is invalid;
• retry a generation under a bounded, idempotent policy;
• send failures to observability tooling with model, prompt version, and schema version;
• route sensitive or high-value failures to a human review path.

Avoid logging full raw inputs or outputs by default. AI traffic may contain user data, internal documents, or secrets. Logging structured error locations and request correlation IDs is usually safer than logging entire payloads.

Validation also has limits. A schema can establish that document_id is a non-empty string, but it cannot prove that the cited document exists, that the excerpt truly appears in it, or that the answer is factually supported. Those require retrieval provenance checks, evaluation, and authorization-aware data access, which the later RAG and safety modules will develop.

---

A compact boundary module

The following is a useful starting point for your project. It validates one incoming request and one outgoing model response while keeping raw data at the edge.

import logging
from typing import Annotated, Literal

from pydantic import (
    BaseModel,
    ConfigDict,
    Field,
    ValidationError,
    model_validator,
)

logger = logging.getLogger(__name__)

NonEmptyQuestion = Annotated[
    str,
    Field(min_length=1, max_length=4_000),
]

ShortText = Annotated[
    str,
    Field(min_length=1, max_length=2_000),
]


class CustomerQuery(BaseModel):
    model_config = ConfigDict(
        extra="forbid",
        str_strip_whitespace=True,
    )

    question: NonEmptyQuestion
    account_id: Annotated[
        str,
        Field(pattern=r"^acct_[A-Za-z0-9]+$"),
    ]
    locale: Literal["en", "es"] = "en"
    max_results: Annotated[
        int,
        Field(ge=1, le=10),
    ] = 5


class Citation(BaseModel):
    model_config = ConfigDict(extra="forbid")

    document_id: Annotated[str, Field(min_length=1, max_length=100)]
    chunk_id: Annotated[str, Field(min_length=1, max_length=150)]
    excerpt: Annotated[str, Field(min_length=1, max_length=500)]


class GroundedAnswer(BaseModel):
    model_config = ConfigDict(extra="forbid")

    status: Literal["answered", "insufficient_evidence"]
    answer: ShortText
    citations: list[Citation] = Field(default_factory=list, max_length=5)

    @model_validator(mode="after")
    def answered_response_needs_citations(self) -> "GroundedAnswer":
        if self.status == "answered" and not self.citations:
            raise ValueError(
                "an answered response must contain at least one citation"
            )
        return self


def parse_customer_query(data: object) -> CustomerQuery:
    return CustomerQuery.model_validate(data)


def parse_model_output(raw_json: str) -> GroundedAnswer | None:
    try:
        return GroundedAnswer.model_validate_json(raw_json)
    except ValidationError as error:
        logger.warning(
            "Model output rejected: %s",
            error.errors(include_url=False),
        )
        return None


def main() -> None:
    request_data = {
        "question": " What is the refund policy? ",
        "account_id": "acct_7K91",
        "max_results": "3",
    }

    model_json = """
    {
      "status": "answered",
      "answer": "The policy allows refunds within 30 days of purchase.",
      "citations": [
        {
          "document_id": "refund-policy-2025",
          "chunk_id": "refund-policy-2025:chunk-04",
          "excerpt": "Customers may request a refund within 30 days."
        }
      ]
    }
    """

    query = parse_customer_query(request_data)
    answer = parse_model_output(model_json)

    print(query.model_dump())

    if answer is not None:
        print(answer.model_dump_json(indent=2))


if __name__ == "__main__":
    main()

Run it from the project environment:

uv run python main.py
uv run mypy main.py

Keep model definitions near the boundary they protect. For instance, a schemas/ package might contain requests.py, outputs.py, and events.py, rather than one giant application-wide “models” file. Give schemas stable, domain-specific names such as CustomerQuery and GroundedAnswer; generic names such as Response quickly become ambiguous in a service with multiple boundaries.

---

Key takeaways

Pydantic turns Python annotations into runtime contracts. Use BaseModel when data crosses a trust boundary: user input, external services, queues, files, and LLM responses.

The central methods are:

• model_validate() for Python dictionaries and objects;
• model_validate_json() for raw JSON text;
• model_dump() and model_dump_json() for exporting a validated model.

Use Field constraints and Annotated to make real policies executable: bounds, allowed values, lengths, patterns, and nested structures. Use ConfigDict(extra="forbid") when unexpected fields should be treated as integration failures rather than silently ignored. Catch ValidationError narrowly and ensure invalid model output cannot reach downstream code.

Next, you will apply the same boundary discipline to persistent data: reading and writing text, JSON, JSONL, and CSV safely with pathlib and context managers.