Validation, Errors, and API Testing

Published

Aug 2026

  • ID: MD-L06
  • Type: Applied
  • Audience: Intermediate
  • Theme: Test success, invalid input, and failure behaviour

A prediction API is reliable only when its failures are as deliberate as its successful responses. A model may predict correctly for valid data and still be unsafe to deploy if malformed requests produce confusing messages, unexpected status codes, or silent coercions.

This chapter develops a practical testing strategy for the FastAPI service introduced in Serving Predictions with FastAPI. The goal is to verify the public API contract: which requests are accepted, which are rejected, and what every response communicates to a client.

Learning objectives

By the end of this chapter, you should be able to:

  • distinguish request validation from model inference errors;
  • interpret common HTTP status codes returned by a prediction API;
  • design positive, negative, and boundary test cases;
  • test a running API with a repeatable Python program;
  • write focused in-process tests with FastAPI’s TestClient;
  • diagnose failures without exposing internal implementation details.

Validation is part of the API contract

The request schema defines the boundary between an external client and the prediction system. In FastAPI, a Pydantic model can validate field presence, data types, ranges, and selected business rules before inference begins.

Consider a request model with explicit constraints:

from pydantic import BaseModel, ConfigDict, Field


class PredictionRequest(BaseModel):
    model_config = ConfigDict(extra="forbid")

    mean_radius: float = Field(gt=0)
    mean_texture: float = Field(gt=0)
    mean_perimeter: float = Field(gt=0)
    mean_area: float = Field(gt=0)
    mean_smoothness: float = Field(ge=0, le=1)
    mean_compactness: float = Field(ge=0, le=1)
    mean_concavity: float = Field(ge=0, le=1)
    mean_concave_points: float = Field(ge=0, le=1)

These constraints prevent several invalid states:

  • missing required fields;
  • values with incompatible types;
  • non-positive size measurements;
  • proportions outside their supported range;
  • unexpected fields when extra="forbid" is enabled.

Validation should reflect the conditions under which the model was designed and evaluated. A field being numeric does not mean that every numeric value is meaningful.

Separate client errors from server errors

An API should distinguish a request the client can correct from a failure inside the service.

Status Meaning in this workflow Example
200 OK The request was valid and inference succeeded A prediction response is returned
400 Bad Request The request is understandable but violates an application rule An unsupported category combination
404 Not Found The requested route or resource does not exist An unknown model version
422 Unprocessable Content FastAPI/Pydantic could not validate the request body A required field is missing
500 Internal Server Error An unexpected failure occurred in the service An unhandled preprocessing failure
503 Service Unavailable The service is running but cannot currently predict The model artifact failed to load

Do not return 200 OK with an error hidden inside the response body. Status codes allow clients, monitoring tools, and automated tests to respond correctly.

Design a small but meaningful test matrix

Testing only one valid example establishes very little. A useful first test suite covers four behaviours.

Test category Example Expected result
Valid request All required fields contain supported values 200; prediction fields are present
Missing field Omit mean_texture 422; validation details are returned
Invalid boundary Send a negative mean_radius 422; inference is not attempted
Malformed JSON Send an incomplete JSON document 422 or another documented client error

Add domain-specific cases as the contract matures: minimum and maximum accepted values, unknown categories, null values, extra fields, and values close to decision thresholds.

Test a running API

Start the API in one terminal using the Chapter 05 command. In a second terminal, run the Chapter 06 test program:

python scripts/python/06-test-api.py \
  --base-url http://127.0.0.1:8000 \
  --output results/06-api-test-results.json

The program sends health, valid-prediction, and invalid-prediction requests. It checks status codes and response structure, writes a machine-readable report, and exits with a non-zero status if any check fails. This makes the same script useful locally and in continuous integration.

The Bash wrapper provides a shorter equivalent command:

bash scripts/bash/06-test-api.sh

The VALID_PAYLOAD fields match the Chapter 05 breast-cancer request schema. Keep invalid cases derived from this known-valid payload so each negative test isolates one validation rule.

Add fast in-process tests

Live tests verify the assembled service over HTTP. During development, in-process tests provide faster feedback and do not require a separately running server.

from fastapi.testclient import TestClient

from app.main import app


client = TestClient(app)


def test_health_endpoint():
    response = client.get("/health")

    assert response.status_code == 200
    assert response.json()["status"] == "ok"


def test_predict_rejects_missing_field():
    payload = {
        "mean_radius": 17.99,
        # mean_texture is deliberately omitted
        "mean_perimeter": 122.80,
        "mean_area": 1001.0,
        "mean_smoothness": 0.11840,
        "mean_compactness": 0.27760,
        "mean_concavity": 0.30010,
        "mean_concave_points": 0.14710,
    }

    response = client.post("/predict", json=payload)

    assert response.status_code == 422
    assert "detail" in response.json()

Save project-specific tests under tests/ and run them with:

pytest -q

Prefer assertions about the public contract. Testing private helper functions too closely can make harmless refactoring unnecessarily difficult.

Test response structure, not only status

A 200 response can still be unusable. Tests should verify the stable fields and basic types promised to clients.

def test_prediction_response_contract():
    payload = {
        "mean_radius": 17.99,
        "mean_texture": 10.38,
        "mean_perimeter": 122.80,
        "mean_area": 1001.0,
        "mean_smoothness": 0.11840,
        "mean_compactness": 0.27760,
        "mean_concavity": 0.30010,
        "mean_concave_points": 0.14710,
    }

    response = client.post("/predict", json=payload)
    body = response.json()

    assert response.status_code == 200
    assert isinstance(body["prediction"], int)
    assert 0.0 <= body["probability"] <= 1.0
    assert isinstance(body["model_version"], str)

Avoid asserting an exact probability unless the test intentionally locks a particular model artifact and dependency environment. Otherwise, a legitimate model retraining can break the test even though the API contract remains valid.

Return controlled application errors

Known operational failures should become deliberate HTTP responses. For example, a service that has not loaded its model is temporarily unable to predict:

from fastapi import HTTPException


@app.post("/predict", response_model=PredictionResponse)
def predict(request: PredictionRequest) -> PredictionResponse:
    if model is None:
        raise HTTPException(
            status_code=503,
            detail="Prediction service is not ready.",
        )

    return run_inference(request)

The client receives a useful, stable message without seeing a Python traceback, filesystem path, or model internals. Detailed exceptions should go to protected service logs, not public responses.

Diagnose a failing test systematically

When a test fails, inspect the layers in order:

  1. Confirm that the API process is running and the base URL is correct.
  2. Read the status code before interpreting the response body.
  3. Compare the sent JSON keys with the current request schema.
  4. Check the API log for model-loading or preprocessing errors.
  5. Reproduce the request through /docs to separate client-code errors from server behaviour.
  6. Confirm that the model and API are using the same feature names and preprocessing pipeline.

A 422 response usually indicates a schema mismatch. A 500 response indicates that the request passed validation but an unhandled server-side failure occurred.

Testing layers before deployment

No single test style is sufficient:

  • function tests verify preprocessing and inference helpers;
  • in-process API tests verify routes, schemas, and error responses quickly;
  • live smoke tests verify the running service and its configuration;
  • deployment checks verify the packaged service in its target environment.

The test cases can overlap, but each layer should have a clear purpose. Start with a small contract-focused suite and add regression tests whenever a defect is discovered.

Chapter checklist

Before proceeding, confirm that:

  • valid requests return a documented success response;
  • missing, invalid, and unexpected inputs are rejected consistently;
  • response bodies contain the promised fields and types;
  • known operational failures use deliberate status codes;
  • unexpected errors do not expose sensitive internal details;
  • automated tests exit unsuccessfully when a contract check fails;
  • the test payloads match the deployed request schema.

Key takeaway

Reliable API testing is not a final check added after deployment. It is an executable definition of the service contract. By testing both success and failure behaviour, the prediction service becomes easier to integrate, monitor, debug, and change safely.