Why Does AI Ignore My Format Instructions?

TL;DR: Asking for a format in prose is the weakest mechanism available. OpenAI, Anthropic and Google all ship a schema-enforced mode that constrains decoding instead, and OpenAI states its Structured Outputs guarantee outright. When AI ignores format, the fix is usually to stop asking and start enforcing.

Why does AI ignore my format instructions?

Because "reply in JSON" is a request, not a constraint. The model weighs it against everything else in your prompt and emits the most likely continuation. Usually that is your format. Sometimes it is your format with a friendly sentence in front of it, wrapped in a code fence, or with the third key quietly renamed to something more natural.

The part most articles on this query miss: for the three largest vendors there is a mechanism that removes the choice. OpenAI, Anthropic and Google all ship a mode that constrains generation to a JSON Schema you supply. That is not better prompting. It is a different category of thing.

So the honest answer has two halves. One is that you used the weak mechanism when a strong one existed. The other is that your prompt carries a specific, repairable defect. Both are below, with blocks you can paste.

Which format mechanisms actually bind the model?

Four, in ascending order of how much they bind. Exactly one of them is a guarantee.

The gap between rows two and four is the whole subject. Everything else here either makes row two behave, or survives honestly when row four is unavailable.

Prefill is worth a note because it used to be the reflex fix and is now partly gone. Anthropic's consistency guide now carries a note above that section saying prefilling is not supported on Claude 4.6 and later, then still explains the technique below it, saying to "Prefill the Assistant turn with your desired format" because it "bypasses Claude's friendly preamble and enforces your structure". The note above that section says something else.

What is the difference between JSON mode and structured outputs?

One guarantees your schema. The other only guarantees that something parses. OpenAI documents the distinction in its own words.

Structured Outputs, per OpenAI's guide, is "a feature that ensures the model will always generate responses that adhere to your supplied JSON Schema, so you don't need to worry about the model omitting a required key, or hallucinating an invalid enum value." JSON mode, on the same page, is the older and weaker thing: "JSON mode will not guarantee the output matches any specific schema, only that it is valid and parses without errors." OpenAI summarises the pair as "While both ensure valid JSON is produced, only Structured Outputs ensure schema adherence" (developers.openai.com, read August 27, 2026).

JSON mode has one more trap worth knowing before you reach for it. OpenAI warns that you must still instruct the model to produce JSON, because "If you don't include an explicit instruction to generate JSON, the model may generate an unending stream of whitespace and the request may run continually until it reaches the token limit."

# JSON mode: valid JSON, no schema. Note the instruction is still required.
curl https://api.openai.com/v1/responses \
  -H "Authorization: Bearer $OPENAI_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "gpt-5.6",
    "input": [
      {"role": "developer", "content": "Reply with a JSON object. No prose."},
      {"role": "user", "content": "Summarise this ticket: printer offline since Tuesday."}
    ],
    "text": { "format": { "type": "json_object" } }
  }'

How do you turn on schema enforcement in each API?

Three vendors, three parameter names, one idea. Each block below is the minimum that works.

# OpenAI: Structured Outputs via text.format
curl https://api.openai.com/v1/responses \
  -H "Authorization: Bearer $OPENAI_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "gpt-5.6",
    "input": [{"role": "user", "content": "Summarise this ticket: printer offline since Tuesday."}],
    "text": {
      "format": {
        "type": "json_schema",
        "name": "ticket_summary",
        "strict": true,
        "schema": {
          "type": "object",
          "properties": {
            "title": { "type": "string" },
            "severity": { "type": "string", "enum": ["low", "medium", "high"] },
            "owner": { "type": ["string", "null"] }
          },
          "required": ["title", "severity", "owner"],
          "additionalProperties": false
        }
      }
    }
  }'

Two constraints trip people up. OpenAI requires every field to be listed in required, and additionalProperties: false on every object. An optional field becomes a union with null, which is why owner above is typed ["string", "null"] rather than omitted from required.

# Anthropic: JSON outputs via output_config.format
curl https://api.anthropic.com/v1/messages \
  -H "content-type: application/json" \
  -H "x-api-key: $ANTHROPIC_API_KEY" \
  -H "anthropic-version: 2023-06-01" \
  -d '{
    "model": "claude-opus-5",
    "max_tokens": 1024,
    "messages": [{"role": "user", "content": "Summarise this ticket: printer offline since Tuesday."}],
    "output_config": {
      "format": {
        "type": "json_schema",
        "schema": {
          "type": "object",
          "properties": {
            "title": { "type": "string" },
            "severity": { "type": "string", "enum": ["low", "medium", "high"] }
          },
          "required": ["title", "severity"],
          "additionalProperties": false
        }
      }
    }
  }'

Anthropic describes the mechanism rather than just the promise: structured outputs "guarantee schema-compliant responses through constrained decoding". Its docs also list what breaks the guarantee anyway, which is worth handling. A refusal returns stop_reason: "refusal" and may not match your schema. Hitting the ceiling returns stop_reason: "max_tokens", where "The output may be incomplete and not match your schema". And one detail almost nobody publishes: Anthropic states that "Structured outputs don't guarantee the capitalization of string enum and const values", advising you to compare enum values case-insensitively (platform.claude.com, read August 27, 2026).

# Google Gemini: response_format on the interactions endpoint
curl -X POST "https://generativelanguage.googleapis.com/v1beta/interactions" \
  -H "x-goog-api-key: $GEMINI_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "gemini-3.7-flash",
    "input": "Summarise this ticket: printer offline since Tuesday.",
    "response_format": {
      "type": "text",
      "mime_type": "application/json",
      "schema": {
        "type": "object",
        "properties": {
          "title": { "type": "string", "description": "One-line ticket title." },
          "severity": { "type": "string", "enum": ["low", "medium", "high"] }
        },
        "required": ["title", "severity"]
      }
    }
  }'

Google's framing is the same: "You can configure Gemini models to generate responses that adhere to a provided JSON Schema." All three support only a subset of JSON Schema, so check the vendor's supported-features list before assuming a keyword works.

Where should the format instruction go in the prompt?

Not in the middle. A format rule sandwiched between three paragraphs of context is competing with everything around it, and the vendors say so in slightly different ways.

OpenAI's GPT-4.1 prompting guide is the most specific: "If you have long context in your prompt, ideally place your instructions at both the beginning and end of the provided context, as we found this to perform better than only above or below." Anthropic's advice points the other direction for the data itself, telling you to "Place your long documents and inputs near the top of your prompt, above your query, instructions, and examples", and noting that "Queries at the end can improve response quality by up to 30 percent in tests." Those are compatible once you separate the two things: long data near the top, your instruction near the query, and for very long prompts, the format rule repeated at both ends.

# Before: format instruction buried
You are a research assistant. Here is the transcript.
[3,000 words of transcript]
Return the three key decisions as JSON with keys decision, owner, due_date.
Also consider the tone of the meeting and whether follow-up is needed, and
note any risks the team raised, and summarise the overall sentiment.

# After: contract first, data in the middle, contract repeated last
Return ONLY a JSON array. Each element has exactly these keys:
decision (string), owner (string), due_date (string, YYYY-MM-DD or null).

<transcript>
[3,000 words of transcript]
</transcript>

Reminder: output is a JSON array with keys decision, owner, due_date. No prose.

Do conflicting instructions cause format drift?

Yes, and this is the single most common self-inflicted case. Two rules that cannot both be satisfied force the model to pick one, and OpenAI documents which one it picks: "If there are conflicting instructions, GPT-4.1 tends to follow the one closer to the end of the prompt."

A word count fighting a table is the classic. Six table rows cannot also be 200 words of flowing prose, so one rule loses. The same collision is why word counts get ignored, which is the page to read if length rather than shape is your real complaint.

# Before: two rules that cannot both hold
Write a 200-word summary of the vendor comparison.
Present it as a markdown table with columns Vendor, Price, Verdict.
Keep it conversational and add a closing recommendation paragraph.

# After: one shape, one budget, stated as a contract
Output exactly two parts, in this order:

1. A markdown table. Columns: Vendor | Price | Verdict. One row per vendor.
   Each Verdict cell is at most 12 words.
2. A single closing paragraph of 40 to 60 words, plain prose, no bullets.

Do not output anything before part 1 or after part 2.

Why do my examples override my stated format?

Because examples are the stronger signal, and if yours disagrees with your prose you have effectively overwritten your own instruction. Anthropic states the strength plainly: examples are "one of the most reliable ways to steer Claude's output format, tone, and structure", and its consistency guide adds that showing the output "is more effective than abstract instructions".

That is excellent news until your example carries a stray key, a different date format, or a trailing comment. It will be copied. Audit the example against the rule before you audit the rule.

# Before: instruction and example disagree
Return JSON with keys: name, role, start_date (YYYY-MM-DD).

Example:
{ "name": "Ada", "title": "Engineer", "start": "01/03/2026", "notes": "" }

# After: the example IS the schema
Return JSON with keys: name, role, start_date (YYYY-MM-DD).

Example:
{ "name": "Ada", "role": "Engineer", "start_date": "2026-03-01" }

Why does the format decay halfway through a long output?

There are two different failures wearing the same costume, and they need different fixes.

The first is truncation. The response stopped because it ran out of room, so the tail is missing rather than malformed. Anthropic names the signal directly, returning stop_reason: "max_tokens" when it happens. Raise the output ceiling and the problem disappears. If you are counting tokens in your head, count the closing braces too, because a schema with fifty items needs room for all fifty.

The second is genuine drift. By item forty, the format rule is thousands of tokens behind and the thirty-nine previous items are a nearer, louder pattern. OpenAI flags a related behaviour in the GPT-4.1 guide's caveats: "In some isolated cases we have observed the model being resistant to producing very long, repetitive outputs, for example, analyzing hundreds of items one by one." The fix is not a firmer sentence. It is fewer items per call.

# Before: one call, sixty items, hope
Convert all 60 rows below into JSON objects with keys id, name, amount.
[60 rows]

# After: batched, with the contract restated per batch
Convert rows 1-15 into a JSON array. Each object has exactly:
id (integer), name (string), amount (number, no currency symbol).
Output the array only.

[rows 1-15]

Why is my JSON wrapped in a code fence, with a paragraph in front of it?

Because both are the most likely way to present code in chat-shaped text, and nothing in your prompt forbids either. OpenAI lists the preamble as a known failure mode: "Without specific instructions, some models can be eager to provide additional prose to explain their decisions, or output more formatting in responses than may be desired."

Schema enforcement solves this structurally: the constrained output is the JSON document, so there is no room for a sentence in front of it. Without it you need both an instruction and a defensive parse. Anthropic's migration guidance for former prefill users says the same, recommending "Respond directly without preamble" and adding that if one slips through, strip it in post-processing.

# Before
Give me the results as JSON.

# After
Output a single JSON object and nothing else. No preamble, no explanation,
no markdown code fence, no trailing commentary. The first character of your
response must be { and the last must be }.

Should you say what not to do, or what to do?

What to do. Anthropic puts this first in its list of ways to steer output formatting, under the heading "Tell Claude what to do instead of what not to do", with the worked pair: instead of "Do not use markdown in your response", try "Your response should be composed of smoothly flowing prose paragraphs."

The reason is mechanical. A negative instruction names a shape without supplying a replacement, so the model invents the target itself. A positive one specifies it. Anthropic notes a subtler lever on the same page: "The formatting style used in your prompt may influence Claude's response style", which means a prompt written in heavy markdown is quietly asking for heavy markdown back.

# Before: negative, vague, easy to satisfy badly
Don't use bullet points. Don't be too formal. Don't add headers.

# After: positive, specific, checkable
Write in continuous prose paragraphs of three to five sentences each.
Use a plain, direct register, as if explaining to a colleague.
Structure the piece as: situation, complication, recommendation.

For anything longer than a paragraph, promote the rule from an instruction to a named block. Anthropic's own house example wraps it in an XML-style tag, which also makes it reusable across prompts.

<output_format>
Write in flowing prose using complete paragraphs and sentences. Reserve
markdown for inline code and simple headings. Avoid bold and italics.

Do not use ordered or unordered lists unless the content is genuinely a set of
discrete items, or the user explicitly asks for a list or ranking. Incorporate
items into sentences instead.

Goal: readable text that guides the reader through ideas rather than
fragmenting them into isolated points.
</output_format>

Should you lower the temperature for reliable formatting?

On current Claude models you cannot, and the advice is now stale wherever you find it. Anthropic's Messages API reference marks the parameter deprecated and states: "Models released after Claude Opus 4.6 do not support setting temperature. A value of 1.0 of will be accepted for backwards compatibility, all other values will be rejected with a 400 error." The typo in the middle of that sentence is Anthropic's, not ours.

Confusingly, the same field description still carries the old guidance to "Use temperature closer to 0.0 for analytical / multiple choice" a few lines below. Both sentences were live on the same page on August 27, 2026. Treat the deprecation as operative, since it is the half the API enforces. Our explainer on sampling parameters predates the change and is on the list to update.

The wider point stands regardless of vendor. Temperature was never a formatting control. It changes how the next token is sampled, not which keys exist in your object. Reaching for it to fix a missing field treats a structural problem as a statistical one.

What do you do when there is no enforced mode?

Validate, then retry once with the error attached. That is the honest fallback, and the vendors recommend it themselves. Google is blunt about the residual risk: "While output is syntactically correct JSON, always validate values in your application", and it tells you to "Implement robust error handling for schema-compliant but semantically incorrect outputs."

That warning applies to the enforced path too. Schema conformance is not correctness. OpenAI says so directly: "Structured Outputs can still contain mistakes", and warns that "The model will always try to adhere to the provided schema, which can result in hallucinations if the input is completely unrelated to the schema." A perfectly shaped object full of invented values passes every parser you own.

import json, re
from jsonschema import validate, ValidationError

FENCE = re.compile(r"^\s*`{3}(?:json)?\s*|\s*`{3}\s*$")

def parse_strict(raw, schema):
    """Strip a stray fence, parse, validate. Returns (ok, value_or_error)."""
    cleaned = FENCE.sub("", raw).strip()
    try:
        value = json.loads(cleaned)
    except json.JSONDecodeError as e:
        return False, f"Invalid JSON at position {e.pos}: {e.msg}"
    try:
        validate(instance=value, schema=schema)
    except ValidationError as e:
        return False, f"Schema violation at {list(e.path)}: {e.message}"
    return True, value

def ask_with_repair(client, prompt, schema, max_attempts=2):
    """One repair attempt, with the validator's own message fed back."""
    messages = [{"role": "user", "content": prompt}]
    for _ in range(max_attempts):
        raw = client.complete(messages)
        ok, result = parse_strict(raw, schema)
        if ok:
            return result
        messages += [
            {"role": "assistant", "content": raw},
            {"role": "user", "content":
             f"That response failed validation: {result}. "
             f"Return corrected JSON only, matching the schema exactly."},
        ]
    raise ValueError("Model did not produce schema-valid output")

Two attempts, not ten. A model that fails schema validation twice on one prompt is usually being asked for something the schema cannot express, and a third call will not discover that for you.

The five-line diagnosis

Run these in order next time a format is ignored. Most cases resolve on line one or two.

1. Is a schema-enforced mode available on this API? If yes, use it and stop. Structured output is a guarantee; nothing you write in prose is.
2. Is the format rule the last thing before the answer? If it is buried mid-prompt, move it, and repeat it at both ends for long inputs.
3. Do any two of your rules collide? A word count against a table, a tone against a schema. Delete one.
4. Does your example match your rule exactly? Key names, date format, no extra fields. The example wins if they disagree.
5. Is the output truncated or genuinely drifting? Truncation needs a higher ceiling. Drift needs smaller batches.

Here is the reusable version. Fill the four slots and keep it wherever you keep prompts.

<output_contract>
SHAPE: [one JSON object | a JSON array | a markdown table | prose paragraphs]
FIELDS: [name (type), name (type), name (type)] — all required, no extras
LIMITS: [max length per field, date format, allowed enum values]
FORBIDDEN: nothing before the first character, nothing after the last

If a field cannot be filled from the input, use null. Do not invent a value.
</output_contract>

Then save the version that worked. That is the step people skip, which is why the same fix gets rediscovered every fortnight. A format contract is a reusable asset, not a sentence you improvise under deadline, which is the argument behind our prompt enhancer and a saved library with variables in it.

Two closing honesties. First, none of this makes the content right, only the container predictable, the same distinction that makes AI rewrite code you did not ask it to touch: a confidently wrong answer in a perfect schema is still wrong. Second, the strongest formatting decision usually happens earlier, when you choose the output format at all. OpenAI's long-context testing found "JSON performed particularly poorly" for feeding many documents into a prompt, and recommends markdown as the starting point, noting that JSON "can be more verbose, and require character escaping that can add overhead". If you want JSON because it looks rigorous rather than because something downstream parses it, the format was the problem all along. JSON prompts have a proper use, and decoration is not it.