Writing Idempotent Prompts (Same Input, Same Output)

TL;DR: No mainstream LLM API guarantees byte-identical output for the same input, even at temperature 0 with a seed set — the vendors say so themselves. "Idempotent prompting" in practice means engineering for stable structure and checkable correctness, not chasing a determinism guarantee that doesn't exist.

The word "idempotent" comes from systems where it has a precise meaning: calling an operation once or ten times produces the same end state. An HTTP PUT is supposed to be idempotent. A database upsert is supposed to be idempotent. An LLM completion is not, and the honest starting point for this topic is that no major provider claims otherwise, no matter how carefully you set the parameters.

Does Temperature 0 Actually Make a Prompt Deterministic?

Not according to the two vendors who've said so most directly, and they disagree with each other. Anthropic's own API reference states plainly that "even with temperature of 0.0, the results will not be fully deterministic." Google's mainline Gemini docs take the opposite position, calling temperature 0 deterministic outright. But Google's own Vertex AI documentation for the same underlying models walks that back, describing temperature 0 as mostly deterministic with a small amount of variation still possible. Google is contradicting Google here, not just disagreeing with Anthropic, which is worth knowing before you treat either page as settled.

One more Anthropic detail changes the practical answer. As of September 17, 2026, its Messages API reference marks temperature, top_p and top_k as deprecated: models released after Claude Opus 4.6 accept temperature only at 1.0 and return a 400 error for any other value. So on current Claude models, "set temperature to 0" is not an option. Stability there has to come from the prompt, the output schema and validation, which is what the rest of this post covers.

OpenAI takes a different position again: its Chat Completions reference makes no determinism claim for temperature at all, and treats the seed parameter, not temperature, as the relevant lever, with an explicit caveat attached.

Does a Seed Parameter Guarantee Repeatable Output?

No, and the vendors that offer one say so alongside the parameter itself. OpenAI's API reference frames seed as a best-effort setting and is explicit that determinism isn't guaranteed with it — the same seed and the same parameters can still occasionally produce a different completion. Google documents a seed option on its models but attaches no guarantee to it either. Anthropic doesn't expose a seed parameter on its Messages API at all, so the question doesn't apply there in the first place.

Why Isn't It Just GPU Race Conditions?

The popular explanation for LLM nondeterminism has been "floating-point operations race on the GPU and the order of additions changes the rounding." Researchers at Thinking Machines examined this directly and found it's mostly wrong for the case that matters: individual GPU kernels are typically run-to-run deterministic, and a single forward pass through the kind of transformer architecture behind most LLMs usually contains no atomic adds for the race-condition story to apply to.

Their actual explanation is batch-invariance: your request's output can shift depending on what other requests the server happens to be batching alongside it at that moment, because batch size and composition change the numerics of certain operations even when your own input is identical. In their framing, "the other concurrent users are not an 'input' to the system but rather a nondeterministic property of the system." A separate measurement (arXiv:2506.09501) found swings of up to 9% in accuracy and thousands of tokens of length variation driven by nothing but GPU count, GPU type, and batch size, with the actual prompt held fixed. That's a genuinely different failure mode than the one most people reach for when they explain this, and it's not something a temperature=0 setting or a seed value has any power over, because it lives in the server's batching behavior, not in your request.

What Can You Actually Control, Then?

Given that byte-identical output isn't a guarantee any vendor makes, the useful target shifts from "the same text" to "a checkable, stable result."

Extract the following fields from the support ticket below as JSON
matching this exact schema:

{
  "category": "billing" | "technical" | "account" | "other",
  "urgency": "low" | "medium" | "high",
  "summary": "one sentence, under 20 words"
}

Return only the JSON object, no other text. If a field cannot be
determined from the ticket, use null rather than guessing.

Ticket:
[paste ticket text]

A prompt built this way doesn't need identical wording between two runs to be useful. It needs the same schema every time, which you can check with actual schema validation rather than a string comparison, and values that fall within the same small set of valid categories, which you can verify programmatically. This is what "idempotent" should mean in practice for an LLM call: the same input produces output that passes the same checks, even if the exact phrasing of a free-text field like summary drifts slightly between runs.

Three concrete techniques get you most of the way there:

1. Structured output with an enforced schema, so the shape of the answer is fixed even when the exact prose isn't.
2. A verification step, checking the output against explicit pass/fail criteria before anything downstream trusts it, rather than assuming this run matched the last one.
3. Caching the first trusted response, rather than regenerating on every call and hoping it lands in the same place. If you need the same answer for the same input across a day of traffic, storing and reusing the first verified result is more reliable than re-prompting and expecting consistency the API itself doesn't promise.

A Worked Example: Making a Classification Call Idempotent Enough to Trust

Take the ticket-classification prompt above and put it behind an actual check, rather than assuming a schema in the prompt is the same thing as a schema you've verified.

import json
from jsonschema import validate, ValidationError

SCHEMA = {
    "type": "object",
    "properties": {
        "category": {"enum": ["billing", "technical", "account", "other"]},
        "urgency": {"enum": ["low", "medium", "high"]},
        "summary": {"type": ["string", "null"]},
    },
    "required": ["category", "urgency", "summary"],
    "additionalProperties": False,
}

def classify_ticket(ticket_text: str, call_model) -> dict:
    raw = call_model(ticket_text)  # your actual API call (temperature=0 where the model allows it)
    try:
        parsed = json.loads(raw)
        validate(instance=parsed, schema=SCHEMA)
    except (json.JSONDecodeError, ValidationError) as e:
        raise ValueError(f"Model output failed schema check: {e}") from e
    return parsed

This doesn't make the model deterministic. It makes a non-deterministic call safe to depend on, because anything that doesn't match the schema fails loudly at the point of the call, rather than silently downstream where it's harder to trace back to a malformed classification. That's the practical difference between chasing idempotency as a property of the model and engineering for it as a property of your system.

Logging What You Actually Sent, Not Just What You Got Back

A subtler failure than sampling variation is not knowing which parameters produced a given output in the first place. If a result looks wrong three weeks later, you need the temperature, the seed (if any), and the exact model ID that produced it, not just the output text. Ask your prompt-management setup, or a wrapper around your API calls, to log all three alongside every response. Without that, "why did this answer differ from last month's" becomes a question you can't actually investigate, because you don't know if the difference came from sampling, a model update, or a change you made to the prompt itself in the meantime.

Model Version Changes Are a Separate Source of Drift

Even with temperature at 0, a seed set, and a schema enforced, output can still shift between a call made today and one made next month, for a reason that has nothing to do with sampling: the model behind an API endpoint can change. Anthropic's own model documentation is explicit that a dateless alias resolves to a pinned dated snapshot, and separately notes that the infrastructure serving a fixed set of weights can itself change over time even when the weights don't. If your pipeline depends on a specific model snapshot's behavior, pin the dated model ID where the provider offers one, and treat an undated alias as a moving target regardless of how carefully you've tuned sampling parameters around it.

Where this compounds is downstream: if your prompt's output feeds directly into a schema another system expects, designing output schemas that models actually follow covers the schema-design half of this problem, while this post covers the sampling-and-determinism half. The two are meant to be used together, not as substitutes for each other — a well-designed schema doesn't help if you're expecting the free-text portions inside it to be byte-identical, and a correctly configured seed doesn't help if the schema itself is loose enough to drift in shape.

When "Close Enough" Isn't Good Enough

Some use cases genuinely can't tolerate the wording drift a schema check alone allows through. A generated legal disclaimer, a piece of copy that goes through a compliance review once and then ships unchanged for months, or a system prompt embedded in a product all fall into this category: the exact wording matters, not just the shape.

For those, the right move isn't a better sampling setting, it's removing the model from the repeat path entirely. Generate the text once, have a human review it, and store the approved version as a literal string or a versioned asset your application reads from, rather than re-generating it on every request and hoping the model's non-guaranteed reproducibility holds. This sounds obvious stated plainly, but it's a common mistake in systems that grew organically: a prompt that was originally meant to produce a one-off piece of copy quietly becomes part of a live request path, and nobody notices that "the same text every time" was never actually promised by anything in the stack.

A Short Checklist for "Idempotent Enough" Prompting

1. Are you checking for a stable structure (a schema, a fixed set of valid values), not exact string equality?
2. If you set temperature and seed, do you know which vendor page's determinism claim you're relying on, and have you read the caveat next to it?
3. Is anything downstream pinned to a dated model ID, or silently riding an alias that can change?
4. Do you have a verification step that checks the result against explicit criteria, rather than trusting that "it looks the same as last time"?
5. For a value you genuinely need to be fixed, are you caching the first verified response instead of re-generating and hoping?