Structured Outputs

Goal

Define a small output schema, distinguish valid syntax from valid meaning, and validate model-produced structured data before downstream code trusts it.

Free-form prose is convenient for humans but awkward for software. If the next program step needs an item name, quantity, and confidence flag, a structured object is easier to validate than a paragraph.

Start with an explicit contract

Suppose the required output is:

{
  "item": "battery",
  "quantity": 2,
  "needs_review": false
}

A simple schema can require:

• item is a non-empty string;
• quantity is an integer greater than or equal to 1;
• needs_review is a boolean;
• no required field is missing.

Now the application has something concrete to check.

Valid JSON is not the same as valid output

This is valid JSON:

{"item": "", "quantity": -4, "needs_review": "maybe"}

But it violates the intended schema. Likewise, a schema-valid object can still contain a factual error:

{"item": "battery", "quantity": 9, "needs_review": false}

when the source document clearly says quantity 2. There are therefore at least three layers:

1. syntax validity — can the data be parsed?
2. schema validity — do fields and types satisfy the contract?
3. semantic/grounding validity — are the values supported by the source and task rules?

Do not stop at layer 1.

Validation belongs before side effects

Imagine structured output controls a later action:

model output
→ create refund

The workflow should not perform the action merely because the text looks JSON-like. A safer path is:

model output
→ parse
→ schema validation
→ business-rule checks
→ authorization/human approval where required
→ action

The model proposes data. The application decides whether that data is acceptable for the next step.

Repair and retry need limits

If validation fails, a workflow can sometimes retry with the validation error:

quantity must be an integer >= 1

But retries should be bounded. An endlessly retrying model loop is not reliability. Record which validation failed, how many attempts were made, and what happens after the retry budget is exhausted.

Distinguish missing, null, and empty values

Structured contracts become clearer when absence has one deliberate representation. These three values can mean different things:

{}
{"answer": null}
{"answer": ""}

An omitted field may mean the producer violated the schema. A null field can deliberately mean “no supported answer.” An empty string may mean a present-but-empty value—or may simply be a malformed answer. Choose the meaning explicitly. For an evidence-grounded workflow, a strong contract might say:

supported=true  → answer and source_id must be non-empty
supported=false → answer=null and source_id=null

Now downstream code does not need to guess whether an empty string means abstention.

Schemas evolve, so consumers need a compatibility rule

Suppose version 1 returns:

{"item": "battery", "quantity": 2}

and version 2 adds:

{"item": "battery", "quantity": 2, "needs_review": false}

Will old consumers ignore the new field, reject it, or require an explicit schema version? There is no single answer for every system, but silent assumptions are risky. When structured output becomes an API between components, treat schema changes like ordinary interface changes: version them, test them, and decide which changes are backward-compatible.

Structure gives software a boundary it can enforce

Structured generation is most useful when the next component needs dependable fields rather than prose that “looks about right.” Parsing answers one question: is this valid JSON? Schema validation answers a second: are the required keys and types present? Business and grounding checks answer further questions: is the value allowed, is a cited source among the supplied sources, and is the claim actually supported?

Keeping those checks separate makes failures easier to understand. A model can produce syntactically perfect JSON with a fabricated value, or a factually correct sentence that violates the expected schema. Neither should be silently accepted. Treat model output as untrusted data crossing an interface: parse it, validate its shape, validate task-specific rules, and only then allow downstream side effects.

Predict

A model returns parseable JSON with quantity=-3. What has been proven?

Only that the syntax can be parsed; schema/business checks can still failThat the output is safe for downstream actionsThat the model used the correct source evidence

Complete the Lab validator

The starter contains a small dictionary validator.

1. Run it unchanged and observe the failing checks.
2. Complete validate_order so it checks required fields, types, and quantity range.
3. Confirm that the valid case passes.
4. Confirm that negative quantity and string-valued needs_review fail.
5. Add one new invalid case and explain which contract rule it tests.

Loading lab…

Quick Check

1. What is the difference between JSON validity and schema validity?

JSON validity checks syntax; schema validity checks the required structure and typesJSON validity checks required keys and value types, while schema validity checks punctuation and bracesSchema validity checks whether decoding stayed within a constrained format, not whether required fields have the right types

2. Why validate before a downstream side effect?

Because valid structured output is already safe to execute, so validation can focus on formatting before the side effectBecause validation also improves how the model writes future outputsBecause model-produced data should not be trusted merely because it is well formatted

3. Does schema-valid output prove the facts are correct?

Yes, if the output was produced with constrained decodingYes; valid structure means field values can be trustedNo; grounding and business-rule checks may still be needed

0 of 3 questions answered.

Explain it back

Define a three-field schema for a task of your choice. Give one example that fails syntax, one that passes syntax but fails schema, and one that passes schema but is factually unsupported.

Key Takeaways

• Structured outputs make model results easier for software to inspect.
• Parsing, schema validation, and grounding are different checks.
• Validate before side effects or privileged actions.
• Hard business rules belong in application logic when possible.
• Retries should be bounded and their failures recorded.

Next Lesson

Complete the mini checkpoint, then move to context windows and the problem of relevant information being present but poorly used.

References

• JSON Schema, Draft 2020-12.