Why Does My JSON Output Keep Breaking?

TL;DR: Trailing commas, stray code fences and half-finished objects are symptoms, not the disease. The three real causes are no schema enforcement, a truncated response, and drift over a long list. OpenAI, Anthropic and Google each document a mode that removes the first cause entirely. The other two need a different fix, covered below.

Why does my JSON output keep breaking?

A trailing comma after the last array item. A stray markdown code fence wrapped around an otherwise-valid object. A response that just stops, mid-string, with no closing brace anywhere. A list of forty records where the first thirty-nine are clean and the fortieth is missing a field nobody asked it to drop. These look like four different bugs. They are three causes wearing different costumes.

The first cause is the most common and the most fixable: nothing in your request actually enforces JSON. A prompt that ends in "respond only in JSON" is a request the model weighs against everything else it has generated so far, not a rule it is structurally unable to break. Anthropic states the resulting failure mode directly: "Without structured outputs, Claude can generate malformed JSON responses or invalid tool inputs that break your applications" (docs.claude.com, read September 3, 2026). Every major vendor now ships something that removes this cause outright, and that mode is not new advice dressed up: it is a different category of guarantee than a well-worded prompt.

The second cause is truncation, and it is arguably the more common one in practice, because it looks identical to a model failure. OpenAI's own guide names it directly, alongside the other honest way a schema-enforced call can still miss: "This can happen in the case of a refusal, if the model refuses to answer for safety reasons, or if for example you reach a max tokens limit and the response is incomplete" (developers.openai.com, read September 3, 2026).

The third is schema drift over a long response, which is a different failure from either of the above and needs a different fix. We cover the full four-tier hierarchy of format-enforcement mechanisms (prose instruction, worked example, prefill, schema-enforced mode) in Why Does AI Ignore My Format Instructions?. This page goes one layer deeper and JSON-specific: the exact syntax failures, how to tell truncation from drift before you reach for a JSON fixer, and a validation loop that checks the right thing first.

Is my JSON actually malformed, or did it just get cut off?

Check this before you touch a JSON repair function, because fixing syntax on a truncated response wastes the call. The object was never going to close, no matter how you patch it. Both OpenAI and Anthropic expose a machine-readable signal for exactly this, and it is a different field than a syntax error.

On OpenAI's Responses API, a truncated call reports its own status rather than throwing:

{
  "status": "incomplete",
  "incomplete_details": { "reason": "max_output_tokens" }
}

Anthropic marks the identical situation with stop_reason: "max_tokens" on the message object, and its guidance for what to do next is direct: retry with a higher max_tokens value to get the complete structured output, rather than attempting to salvage the partial text (docs.claude.com, read September 3, 2026). Neither vendor treats this as an error you catch in a try block. It is a normal, successful response that happens to be short of what you asked for, which is exactly why so many integrations miss it. If your parser is the first thing that notices a truncated JSON response, you are finding out one step later than the API already told you.

Token ceilings themselves, and exactly what each vendor calls the parameter that sets them, get their own dedicated treatment in Max Tokens vs Max Output Tokens; the point that matters here is narrower: check the finish signal first, every time, before you assume the JSON itself is broken.

The exact syntax errors that break JSON.parse

JSON's grammar is stricter than the JavaScript and Python object literals it resembles, and a model trained on enormous amounts of both will occasionally borrow their looser habits. Three of these account for most of the "invalid JSON" reports that turn out to be a one-character problem:

// Invalid — trailing comma, single quotes, an unquoted key
{
  'name': "Ada",
  role: "Engineer",
  "skills": ["Rust", "Go",],
}

// Valid — double-quoted keys and strings, no trailing commas
{
  "name": "Ada",
  "role": "Engineer",
  "skills": ["Rust", "Go"]
}

A trailing comma after the last item, single-quoted strings, and an unquoted object key are all legal in a JavaScript object literal and all illegal in JSON. None of them will raise a warning from the model. The two languages look almost identical, and a model producing JSON from a prose request is really producing text shaped like the JSON it has seen in training, which includes the JS-flavoured version far more often than the strict one. A schema-enforced mode does not have this problem, because the constrained decoder cannot emit a character the grammar forbids: trailing commas, single quotes and comments are simply not reachable outputs.

This page assumes you already have a shape in mind and are debugging why it keeps breaking. If you're still designing that shape, from scratch, JSON Prompts Explained covers when JSON is the right ask at all and how to write the schema before any of the failures above become relevant.

Does asking nicely for JSON still work in 2026?

Only as a fallback, and it was never a guarantee. All three major vendors now document a mode that constrains decoding to a schema you supply, and the differences between them matter more than the similarity.

The Gemini row is worth a second look if you learned this from an older tutorial. The field most guides still show (generateContent with a responseSchema in generationConfig) is marked deprecated on Google's own reference page today, alongside its JSON-Schema alternative _responseJsonSchema. The documented current path is the newer Interactions API's response_format object, carrying a type, a mime_type of application/json, and the schema itself. That is a genuinely different request shape from what most existing tutorials show, not a rename.

# Gemini's currently documented shape — Interactions API, not generateContent
curl -X POST "https://generativelanguage.googleapis.com/v1beta/interactions" \
  -H "x-goog-api-key: $GEMINI_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "gemini-3.8-flash",
    "input": "List the fields for a support ticket: title, severity, owner.",
    "response_format": {
      "type": "text",
      "mime_type": "application/json",
      "schema": {
        "type": "object",
        "properties": {
          "title": { "type": "string" },
          "severity": { "type": "string", "enum": ["low", "medium", "high"] },
          "owner": { "type": ["string", "null"] }
        },
        "required": ["title", "severity", "owner"]
      }
    }
  }'

Even the enforced path has a documented ceiling. Google's own limitations note is two sentences, and both matter: schemas support only a subset of JSON Schema, and "Very large or deeply nested schemas may be rejected" (ai.google.dev, read September 3, 2026) outright rather than partially honored. OpenAI's version of the same ceiling is a number: a schema may have up to 5,000 object properties total, with up to 10 levels of nesting, before the API rejects it rather than complying loosely.

What happens when the model refuses instead of answering?

It still returns successfully, which is the part that surprises people wiring up error handling for the first time. A refusal under a schema-enforced call is not an exception, a 4xx status, or a malformed body. It is a normal response that simply is not the JSON you asked for.

The fix is the same shape as the truncation check above: read the field the vendor gives you for exactly this case (OpenAI's refusal content type, Anthropic's stop_reason: "refusal") before you assume a parse failure means your schema or your prompt was wrong. Sometimes it means the request itself was the problem, and no amount of JSON repair fixes that.

Why does a long list break partway through when the first items were fine?

Because a schema-enforced mode constrains the shape of each object, not the consistency of the whole response, and a model generating item forty is pattern-matching against the thirty-nine items already written more than it is re-reading your original instruction. This is a variant of the same failure that makes AI rewrite code nobody asked it to touch: without an explicit, restated boundary, the model uses its own judgment about what "close enough" means, and that judgment drifts the further it gets from your original words.

The practical fix is not a firmer instruction repeated at the model. It is fewer items per call. Ask for records 1 through 15, validate that batch, then ask for 16 through 30 with the same schema restated. A batch that fails validation costs you one retry on fifteen records instead of one retry on the whole list, and a schema restated every batch has far less context competing with it than one stated once at the top of a 60-item request.

A validation loop that checks the right thing first

Most JSON-repair code jumps straight to JSON.parse and catches the exception. That order is backwards: check the completion status first, because no amount of syntax repair fixes a response that never finished.

def get_valid_json(client, prompt, schema, max_attempts=2):
    """Diagnose before repairing: truncation and refusal first, syntax second."""
    for attempt in range(max_attempts):
        response = client.respond(prompt, schema=schema)

        # 1. Truncated — no amount of parsing fixes a response that never finished.
        if response.status == "incomplete":
            if response.incomplete_details.reason == "max_output_tokens":
                raise ValueError("Truncated: raise max_output_tokens and retry, don't repair text.")

        # 2. Refused — a 200 with no JSON to recover.
        if response.content_type == "refusal":
            raise ValueError(f"Model refused: {response.refusal_text}")

        # 3. Only now does a syntax or schema problem mean what it looks like it means.
        try:
            value = parse_and_validate(response.text, schema)
            return value
        except (ValueError, KeyError) as e:
            prompt = f"{prompt}\n\nPrevious attempt failed validation: {e}. Return corrected JSON only."

    raise ValueError("Did not produce schema-valid JSON in the allotted attempts")

Two attempts, not ten. If a model fails schema validation twice against the same prompt, the third call is unlikely to discover something the first two didn't. The more useful move is usually a smaller batch or a clearer schema, not a longer retry loop.

What if I only have a chat window, not the API?

Then there is no enforced mode, because none of the consumer chat apps (ChatGPT, Claude.ai, the Gemini app) expose a schema parameter to a person typing into a text box. Your only levers are the same prose-level ones that apply to any ignored format instruction: name the exact keys, state the types, and forbid prose and fences explicitly rather than hoping the model infers the ban.

Return exactly one JSON object and nothing else.
Keys, in this order: title (string), severity ("low"|"medium"|"high"), owner (string or null).
No markdown fence. No text before the opening brace or after the closing one.
If a value is unknown, use null. Do not invent one.

That gets you closer, but treat anything copied out of a chat window the same way you would treat a form submission from a stranger: validate it before your code trusts it, exactly as you would for the API path. A word count in a prompt is a soft target for the same underlying reason a chat-window JSON request is a soft target: nothing downstream is actually enforcing either one.