Testing Python Components with pytest Fixtures, Parameterization, and Mocks

Good to see you again. In the previous lesson, you made failure behavior explicit at file and network boundaries: missing artifacts, malformed JSON, timeouts, and unusable HTTP responses should produce distinct, meaningful exceptions. Those contracts now need executable evidence.

This lesson uses pytest to test such components without relying on your real filesystem state or a live upstream service. You will use fixtures to create isolated setup, parameterization to express a family of cases compactly, and mocks to control external dependencies while checking both outcomes and interactions. These are the testing patterns that keep AI-service changes safe when prompt code, ingestion logic, model clients, and schemas evolve together.

---

Tests as executable contracts

A unit test should make a narrow claim about behavior:

• Given a valid manifest, the loader returns the expected dictionary.
• Given malformed JSON, it raises ArtifactContentError.
• Given a model-registry timeout, the client raises UpstreamUnavailableError.
• Given an HTTP 404 that the product defines as an acceptable absence, the client returns None.

A useful test has three properties:

1. Deterministic: its result does not depend on whether a third-party endpoint happens to be up.
2. Isolated: its setup does not leak state into another test.
3. Diagnostic: when it fails, the test name and assertion say what contract was broken.

For the project structure established earlier, a conventional layout is:

your-project/
├── app/
│   ├── __init__.py
│   ├── artifacts.py
│   └── model_cards.py
├── tests/
│   ├── conftest.py
│   └── unit/
│       ├── test_artifacts.py
│       └── test_model_cards.py
└── pyproject.toml

Pytest collects files named test_*.py or *_test.py, then collects functions named test_*. From the project root, run the suite with:

pytest -q

During focused work, run one file or one named test:

pytest tests/unit/test_artifacts.py -q
pytest tests/unit/test_model_cards.py -k timeout -q

A minimal project-level configuration keeps discovery predictable:

# pyproject.toml
[tool.pytest.ini_options]
testpaths = ["tests"]
addopts = "-ra"

-ra adds a concise summary of skipped, expected-failure, and related outcomes. Do not use a test run merely as a final gate before merging; run it after meaningful changes to implementation contracts.

---

Fixtures: controlled setup through dependency injection

A fixture is pytest-managed setup. A test requests one by listing its name as a parameter; pytest resolves the fixture, runs it, and passes its value into the test. This is dependency injection based on function signatures rather than explicit calls.

The prior lesson’s load_manifest_json() is a good first testing target. It reads an actual file, so use pytest’s built-in tmp_path fixture rather than a path in your repository or home directory.

# tests/unit/test_artifacts.py
from pathlib import Path

import pytest

from app.artifacts import ArtifactContentError, load_manifest_json


@pytest.fixture
def manifest_path(tmp_path: Path) -> Path:
    return tmp_path / "manifest.json"

tmp_path is supplied by pytest. It creates an isolated temporary directory for this test invocation, and pytest cleans it up. Our manifest_path fixture derives a single predictable file location from it.

A test that requests manifest_path gets a fresh path:

def test_load_manifest_returns_json_object(manifest_path: Path) -> None:
    manifest_path.write_text(
        '{"corpus_id": "handbook", "document_count": 2}',
        encoding="utf-8",
    )

    result = load_manifest_json(manifest_path)

    assert result == {
        "corpus_id": "handbook",
        "document_count": 2,
    }

Notice what the fixture does not do:

• The test does not call manifest_path() itself.
• It does not hard-code /tmp, which differs across operating systems and is shared state.
• It does not reuse a repository fixture file that another test could overwrite.
• It does not clean up manually. tmp_path owns that responsibility.

By default, a fixture has function scope: pytest constructs it independently for each test that requests it. That is almost always the correct scope for mutable application state.

Pytest also supports class, module, package, and session scopes. Wider scope can reduce expensive setup, but it also makes state sharing easier to introduce accidentally. A session-scoped object can be reasonable for immutable configuration or an expensive local test server, but it is usually the wrong choice for mutable stores, caches, or model-client state.

Fixture teardown with yield

When your test setup opens or modifies a resource that pytest does not already manage, write a yield fixture:

from collections.abc import Iterator
from pathlib import Path

import pytest


@pytest.fixture
def staged_document(tmp_path: Path) -> Iterator[Path]:
    document = tmp_path / "incoming.txt"
    document.write_text("temporary test content", encoding="utf-8")

    yield document

    # Teardown goes here.
    # tmp_path itself is already cleaned up by pytest.

Everything before yield is setup. Pytest provides the yielded value to the test. When the test finishes, even if an assertion fails, pytest resumes the fixture after yield to run teardown.

Use this pattern for resources you own, such as a temporary database schema, spawned subprocess, or test container. Prefer the resource’s own context manager when one exists; for example, a fixture that opens a file should generally use with, rather than manually calling close().

---

Parameterization: express a set of behaviors, not copy-pasted tests

A fixture controls environment. Parameterization varies the case under test.

For a loader, valid JSON objects may vary in shape. The assertion logic is the same, so write it once and provide a small, purposeful set of inputs:

# tests/unit/test_artifacts.py

@pytest.mark.parametrize(
    ("raw_json", "expected"),
    [
        pytest.param(
            '{"corpus_id": "handbook", "document_count": 2}',
            {"corpus_id": "handbook", "document_count": 2},
            id="typical-manifest",
        ),
        pytest.param(
            '{"corpus_id": "empty", "document_count": 0}',
            {"corpus_id": "empty", "document_count": 0},
            id="empty-corpus",
        ),
    ],
)
def test_load_manifest_returns_json_object(
    manifest_path: Path,
    raw_json: str,
    expected: dict[str, object],
) -> None:
    manifest_path.write_text(raw_json, encoding="utf-8")

    assert load_manifest_json(manifest_path) == expected

Pytest runs this as two independently reported test cases. If the empty-corpus behavior fails, the output identifies the empty-corpus case rather than reporting an opaque failure in a loop.

The same approach makes failure contracts explicit:

@pytest.mark.parametrize(
    ("raw_json", "message"),
    [
        pytest.param("{", "invalid JSON", id="invalid-json"),
        pytest.param("[]", "Expected a JSON object", id="wrong-top-level-shape"),
    ],
)
def test_load_manifest_rejects_bad_content(
    manifest_path: Path,
    raw_json: str,
    message: str,
) -> None:
    manifest_path.write_text(raw_json, encoding="utf-8")

    with pytest.raises(ArtifactContentError, match=message):
        load_manifest_json(manifest_path)

pytest.raises() ensures that the specified operation raises the expected exception. match is a regular expression checked against the exception message, so use a stable, meaningful fragment rather than asserting the entire string including a temporary path or JSON line number.

Choose cases by behavior, not by volume

Parameterization is not a reason to add dozens of weakly distinct inputs. For an artifact loader, a compact behavior-oriented set is stronger:

Behavior category	Representative case
Valid ordinary object	Required fields with typical values
Valid boundary object	An empty collection or zero count that remains valid
Invalid representation	Truncated or malformed JSON
Invalid top-level shape	JSON array when an object is required
Unavailable artifact	Missing path, tested separately as an access failure

Avoid mutating parameter values such as shared dictionaries or lists. Pytest passes values as supplied, not copied. If a test mutates a parameter object, later cases can inherit that mutation. Prefer immutable values, freshly constructed values, or fixtures for mutable setup.

---

Mocks: isolate dependencies you do not intend to test

A unit test for your model-card client should not need:

• network access,
• a registry account or API key,
• a stable upstream endpoint,
• a real timeout lasting ten seconds.

Those are concerns for a separate integration test. In a unit test, replace the external call with a mock and control its outcome.

The most important patching rule is:

Patch the name where the code under test looks it up, not necessarily where the dependency was originally defined.

Suppose app/model_cards.py contains:

import json
from typing import Any

import requests


class UpstreamUnavailableError(Exception):
    """The upstream service could not be reached reliably."""


def fetch_model_card(url: str) -> dict[str, Any] | None:
    try:
        with requests.get(url, timeout=10) as response:
            if response.status_code == 404:
                return None

            response.raise_for_status()
            body = response.content

    except requests.Timeout as error:
        raise UpstreamUnavailableError(
            "Timed out while requesting model card"
        ) from error

    except requests.ConnectionError as error:
        raise UpstreamUnavailableError(
            "Could not connect to model registry"
        ) from error

    payload: object = json.loads(body)

    if not isinstance(payload, dict):
        raise ValueError("Model card must be a JSON object")

    return payload

Because that module executes import requests and later looks up requests.get through its own module namespace, patch this target:

"app.model_cards.requests.get"

Patching "requests.get" may sometimes appear to work, but it is less precise and can affect unrelated code that also uses requests. If the production module instead used from requests import get, the target would be "app.model_cards.get".

A reusable patch fixture

Python’s standard library provides unittest.mock, so no additional pytest plugin is necessary for this style:

# tests/unit/test_model_cards.py
from collections.abc import Iterator
from unittest.mock import MagicMock, Mock, patch

import pytest
import requests

from app.model_cards import UpstreamUnavailableError, fetch_model_card


@pytest.fixture
def mocked_get() -> Iterator[Mock]:
    with patch("app.model_cards.requests.get", autospec=True) as get:
        yield get

This is a yield fixture with a particularly valuable cleanup property. The patch is active while the test runs; when pytest tears down the fixture, execution leaves the with patch(...) block and restores the original requests.get, even if the test fails.

Now configure a successful HTTP response without creating a socket:

def test_fetch_model_card_returns_parsed_payload(
    mocked_get: Mock,
) -> None:
    response = MagicMock()
    response.status_code = 200
    response.content = b'{"id": "small-model", "context_length": 4096}'

    mocked_get.return_value.__enter__.return_value = response

    card = fetch_model_card("https://registry.example/models/small-model")

    assert card == {
        "id": "small-model",
        "context_length": 4096,
    }
    mocked_get.assert_called_once_with(
        "https://registry.example/models/small-model",
        timeout=10,
    )
    response.raise_for_status.assert_called_once_with()

There are two kinds of assertions here:

• State assertion: card contains the expected parsed data.
• Interaction assertion: the client called the dependency with the configured timeout, exactly once.

The nested .__enter__.return_value matters because production code uses:

with requests.get(...) as response:

The patched requests.get() returns an object used as a context manager; its __enter__() result is the response visible inside the with block. MagicMock supports those context-manager magic methods.

Test the intentional 404 contract separately:

def test_fetch_model_card_returns_none_for_missing_card(
    mocked_get: Mock,
) -> None:
    response = MagicMock()
    response.status_code = 404
    mocked_get.return_value.__enter__.return_value = response

    card = fetch_model_card("https://registry.example/models/missing")

    assert card is None
    response.raise_for_status.assert_not_called()

Finally, use side_effect to simulate a dependency that raises rather than returns:

def test_fetch_model_card_translates_timeout(
    mocked_get: Mock,
) -> None:
    mocked_get.side_effect = requests.Timeout("upstream did not respond")

    with pytest.raises(
        UpstreamUnavailableError,
        match="Timed out while requesting model card",
    ) as captured:
        fetch_model_card("https://registry.example/models/small-model")

    assert isinstance(captured.value.__cause__, requests.Timeout)

This test verifies the external behavior—the stable application-level exception—and also checks that exception chaining from the prior lesson was preserved. The original requests.Timeout remains available as __cause__ for logs and debugging.

---

When not to mock

Mocking is for isolating a dependency, not for pretending that every layer works together.

For the artifact loader, using tmp_path and a real temporary file is appropriate. The filesystem interaction is small, local, deterministic, and part of the component’s behavior. Mocking Path.open() would require reproducing Python’s file semantics while proving less.

For the model-card client, mocking the HTTP call is appropriate in unit tests because the network is slow, unreliable, external, and not the behavior under test.

A healthy testing portfolio separates concerns:

Test level	Typical dependency choice	Example
Unit test	Mock remote services; use lightweight real local resources	fetch_model_card() translates a timeout
Integration test	Real components in controlled environments	A local service returns a real HTTP response
End-to-end test	Deployed dependencies and user-facing flow	API request reaches retrieval and generation pipeline

Over-mocking can make a test verify only your mock configuration. Under-mocking makes unit tests slow and flaky. The boundary is the question: is this dependency’s real behavior what this test is trying to validate?

Design also affects testability. A function that directly constructs a client, reads an environment variable, calls a network service, parses data, and persists state is difficult to test cleanly. Separating a model client from the service that uses it lets you pass a narrow fake or mock client directly. That is the same dependency-boundary discipline used in backend service design, applied consistently to Python AI components.

---

Key takeaways

pytest turns your file and network error contracts into fast, repeatable checks.

• Fixtures are pytest-managed dependencies requested by name in a test signature.
• Use the built-in tmp_path fixture for isolated filesystem tests instead of shared local paths.
• Function scope is the safe default for mutable setup; use broader fixture scopes only with a clear sharing rationale.
• Put teardown after yield; pytest runs it after the test, including after failed assertions.
• Use @pytest.mark.parametrize when multiple cases share the same test logic, and give behavior-focused cases readable IDs.
• Use pytest.raises() to assert specific exception contracts, optionally matching a stable part of the message.
• Mock external systems in unit tests, but keep deterministic local behavior real when it is part of the component under test.
• Patch the dependency where the production module looks it up.
• Assert both returned behavior and critical interactions, such as timeout configuration or whether an error-status check was invoked.

Next, you will use asyncio to run multiple model-style requests under a concurrency limit. The same principles carry forward: controlled dependencies, deterministic tests, and explicit checks around timeout and failure behavior become even more important once requests overlap.