How to Handle Structured Output Validation Errors

Separate Structural from Semantic Errors

The first step is recognizing that your error handling strategy depends on the type of error. Structural errors (wrong type, missing field, invalid JSON) should never occur with schema-enforced structured output. If you see them, something is misconfigured: you might be using JSON mode instead of strict schema mode, or the provider might be having an issue. Structural errors are infrastructure problems, not application problems.

Semantic errors are application-level problems where the structurally valid response fails your business rules. These come in several flavors:

Range violations: A number field is outside the expected range. A sentiment score of 1.5 when the valid range is 0.0 to 1.0. A word count of -3. A price of $0.00 when the document clearly states $49.99.

Format violations: A string field does not match the expected format. An "email" field containing "contact the user at their email" instead of an actual email address. A "date" field containing "next Tuesday" instead of "2026-07-22".

Content quality issues: The field has a valid type and format but the content is poor. A one-sentence summary when you needed a paragraph. A generic extraction that misses the specific detail in the document. A classification that chose the wrong category.

Cross-field inconsistencies: Individual fields are valid but they contradict each other. A status of "completed" with a null completion date. A discount percentage of 20% but a discounted price that does not reflect a 20% reduction.

Your validation library (Pydantic or Zod) catches the first two categories automatically through field validators and refinements. Content quality and cross-field consistency require custom validators or model validators. Design your validation layers to distinguish between these categories because the recovery strategy differs for each.

Implement Retry with Error Context

The most effective recovery strategy for semantic errors is feeding the validation error back to the model and asking it to try again. This works because the error message tells the model exactly what went wrong, giving it specific information to correct. A message like "price must be greater than 0, got -5.99" gives the model clear direction. A message like "email must match pattern, got 'the user prefers email contact'" tells the model to extract an actual email address or return null.

The retry flow is: send the original prompt, receive a response that passes schema validation, run semantic validation, catch the validation error, format the error as a follow-up message, call the model again with the original prompt plus the error context, validate the new response. The Instructor library automates this entire flow, but you can implement it manually with any provider.

Key principles for effective retry:

Include the specific error. "Field 'price' failed validation: value must be positive, received -5.99" is actionable. "Validation failed" is not. The more specific the error message, the more likely the retry succeeds.

Include the failed response. Show the model its previous response so it can see the full context of what it produced, not just the field that failed. This prevents the model from fixing one field while breaking another.

Limit retries. Set a maximum of 2-3 retries. If the model cannot produce a valid response after 3 attempts, more retries are unlikely to help and will only add latency and cost. Move to a fallback strategy instead.

Track retry rates. A retry rate above 5% on a given schema indicates a systemic issue: the schema might be asking for something the model cannot reliably produce, the prompt might be ambiguous, or the validation rules might be too strict. Retry should be a safety net, not a core part of the pipeline.

Build a Fallback Chain

When retries exhaust without producing a valid response, you need a fallback. The right fallback depends on the criticality of the task and the cost tolerance of the application.

Model escalation: Retry with a more capable model. If the initial attempt used a fast, cheap model (GPT-4o-mini, Claude Haiku), retry with a frontier model (GPT-5, Claude Opus). More capable models are better at following complex constraints and handling ambiguous inputs. This adds cost per failed request but keeps the pipeline fully automated.

Schema simplification: Retry with a simpler schema that asks for less. If a 15-field extraction fails, try a 5-field extraction with just the most critical fields. The model is more likely to succeed with less to fill, and partial data is better than no data for most use cases.

Human review queue: Route the failed extraction to a human reviewer who can fill in the fields manually. This is appropriate for high-value, low-volume tasks where accuracy matters more than automation rate. The queue should include the original input, the model's failed response, and the specific validation errors so the reviewer has full context.

Safe defaults: Return a predefined default response when the extraction fails. This is appropriate only for non-critical fields or low-stakes classifications where a default is better than no answer. Be careful with defaults in data extraction, because a wrong value that looks correct can be worse than an obvious failure.

Skip and log: Skip the record entirely and log it for later processing. This is appropriate for batch pipelines where individual failures are acceptable as long as the overall success rate stays above a threshold.

The most robust production systems chain these strategies: retry 2-3 times with error context, escalate to a more capable model for one attempt, and if that fails, route to a human review queue. The chain ensures that the vast majority of requests succeed automatically, while the small percentage that fail get human attention rather than producing wrong data.

Set Error Budgets and Monitor

In production, every validation failure is a data point about your system's reliability. Track failure rates per schema, per field, and per error type. Set alerting thresholds that notify your team when failure rates exceed expected levels.

A reasonable error budget for a well-designed schema with a capable model is 1-3% before retries and under 0.1% after retries. If your failure rate exceeds these benchmarks, investigate the cause:

High pre-retry failure rate (over 5%): The schema is asking for something the model struggles to produce consistently. Review the schema for overly strict validators, ambiguous field requirements, or constraints that conflict with common model behavior. Consider loosening validation rules or adding the reasoning field pattern to improve accuracy.

High post-retry failure rate (over 1%): The model cannot recover from its errors even with specific error messages. This usually indicates a fundamental mismatch between what the input contains and what the schema requires. Review whether the schema is appropriate for the input data, whether the prompt gives the model enough context, or whether the task exceeds the model's capability.

Sudden spike in failures: A previously stable schema suddenly failing more often indicates either a change in the input data distribution (new formats, new edge cases) or a model regression from a provider update. Compare recent inputs to historical inputs to identify the cause.

Logging should capture the original input, the model's response, the specific validation errors, the number of retries, and whether the final result was a success, a fallback, or a failure. This data is essential for debugging and for improving your schemas over time. The LLM evaluation pillar covers broader monitoring patterns for AI systems in production.