Temperature vs Top-P vs Top-K (and Models That Ignore Them)

TL;DR: Temperature, top-p (top_p) and top-k (top_k) are the three sampling parameters that control how random an LLM's output is. Temperature is the main dial; top-p and top-k are tighter filters you usually leave at default. In 2026, the newest reasoning models, including Claude Opus 5.5 and OpenAI models running with reasoning effort turned on, tell you to omit these parameters and steer with reasoning effort and prompting instead.

Contents: Which models let you set these, Side-by-side, Definitions, Settings by task, Balancing creativity and consistency, API examples, Models that ignore them.

Which Models Even Let You Set Temperature, Top-P and Top-K?

Last verified October 2026 against Anthropic's Claude Opus 5.5 migration guide and OpenAI's latest-model guide. Vendors change this every few months, so check your model's page before you hardcode a value.

More on each vendor in the 2026 shift section below. For effort settings, see reasoning effort settings and reasoning models vs chat models.

Temperature vs Top-P vs Top-K: Side-by-Side

Top-k means "only consider the K most likely next tokens." The top_p parameter means "only consider the smallest group of tokens that together cover P of the probability." For per-task values see the table below, and for the full reference see temperature settings by task.

What are temperature, top-p, and top-k in AI?

Temperature, top-p, and top-k are the three sampling parameters that control randomness when a large language model picks each next token. Temperature rescales the whole probability distribution (higher means more random). Top-p (nucleus sampling) keeps only the smallest set of tokens whose cumulative probability reaches a threshold. Top-k keeps only the K most-probable tokens. Tune temperature first; leave the others at default.

If you have ever wondered why the same prompt produces a slightly different answer every time you hit send, the answer lives in these parameters. The AI temperature and top-p settings are the difference between a model that returns the exact same JSON on every call and one that writes ten genuinely different ad headlines. Understanding them is one of the highest-leverage skills in prompt engineering, because the right setting can swing accuracy, reliability, and creativity without changing a single word of your prompt.

This guide explains what each parameter does, gives you copy-pasteable defaults per task, walks through real API examples for OpenAI and Anthropic, and covers the big 2026 shift: frontier reasoning models that no longer let you set temperature at all.

What is sampling, and why does it make output random?

When an LLM generates text, it does not simply "know" the next word. At each step, the model produces a probability distribution over its entire vocabulary — every possible next token gets a score. Vocabularies are large, tens of thousands of tokens or more, so at every step the model is choosing one token out of tens of thousands of candidates. Sampling is the process of picking from that distribution. If the model always picked the single highest-probability token, output would be deterministic — and often dull and repetitive. Instead, it samples, and sampling parameters control how it samples.

Here is the mental model. Imagine the model is about to finish the sentence "The capital of France is ___." The distribution might look like this:

With a factual prompt like this, you almost always want "Paris." But with a creative prompt — "Write the opening line of a noir novel" — a flat distribution where many tokens are plausible is exactly what you want. Sampling parameters let you reshape and trim that distribution to match the job.

What does temperature do in an LLM?

Temperature scales the probability distribution before the model samples from it. A low temperature sharpens the distribution so the model strongly favors the top tokens. A high temperature flattens it, giving unlikely tokens a real chance. Mathematically, the model divides each token's logit by the temperature value before applying softmax — small divisor, sharper peaks; large divisor, flatter curve.

Here is how the common values behave:

• Temperature 0 — Deterministic. Always picks the most probable token. Same input produces the same output (in theory; see the FAQ on floating-point caveats).
• Temperature 0.3 — Mostly safe choices with a little variation. Good for code and factual answers.
• Temperature 0.7 — Balanced randomness. The traditional everyday default.
• Temperature 1.0 — The full distribution as the model was trained. The default in OpenAI's and Anthropic's classic APIs.
• Temperature 1.5+ — Amplified randomness. The model reaches into the long tail of unlikely tokens. More creative, less reliable.
• Temperature 2.0 — Often produces broken, off-topic, or garbled text.

The range differs by provider. OpenAI's Chat Completions API accepts temperature from 0 to 2, with a default of 1.0. Anthropic's classic Messages API accepts 0.0 to 1.0, also defaulting to 1.0, and recommends values closer to 0 for analytical tasks and closer to 1 for creative ones.

Temperature settings by task

Here is the cheat sheet most teams converge on. Treat it as a starting point, not gospel — always test against your own prompts.

For the full by-task reference, see temperature settings by task.

The practical rule: when there is one right answer, set temperature 0. When variety helps the outcome, raise it. If you are building structured JSON output, temperature 0 paired with a strict schema is the gold standard for parseable, repeatable results.

How Do You Balance Creativity and Consistency?

Match the setting to how many right answers the task has. One right answer (extraction, classification, code): use the low end of the table above. Many acceptable answers (copy, ideas): go higher. Tune one parameter, usually temperature, and run the same prompt five times to see whether the variation fits the job. If the outputs differ more than you can accept, lower it; if they feel samey, raise it.

What is top-p (nucleus sampling)?

Top-p, also called nucleus sampling, restricts which tokens are eligible before sampling. The model sorts tokens by probability and keeps the smallest set whose cumulative probability reaches the threshold P, discarding the long tail. A top-p of 0.9 keeps just enough of the top tokens to cover 90% of the probability mass.

The key difference from top-k: top-p adapts the candidate pool to each step. When the model is confident (one token at 0.95), top-p might keep only that one token. When the model is uncertain (the top 30 tokens each around 3%), top-p keeps a wide set. It is "dynamic" where top-k is "fixed."

How the common values behave:

• Top-p 0.1 — Only the most probable handful of tokens. Very narrow, near-greedy.
• Top-p 0.5 — Considers tokens that together account for 50% of probability mass.
• Top-p 0.9 — A common balanced setting. Trims the unlikely tail.
• Top-p 0.95 — A frequent default. Keeps almost everything except the most improbable tokens.
• Top-p 1.0 — Full distribution, no truncation. The default value on OpenAI's API.

Range: 0 to 1.

When to tune top-p

For most users, the honest answer is: rarely. Tuning top-p moves the needle less than tuning temperature. Microsoft's Azure OpenAI guidance and the wider community both echo the provider rule that you should alter temperature or top_p, but not both, since they interact in ways that are hard to reason about together.

Some useful pairings, if you do experiment:

Temperature 0   + top_p 1.0  = pure greedy (most-probable token each step)
Temperature 0.7 + top_p 0.9  = balanced, slightly trimmed tail
Temperature 1.0 + top_p 0.5  = creative but bounded (won't go off the rails)

That last pairing is genuinely useful: a high temperature gives you exploration, while a tight top-p prevents the model from sampling truly bizarre low-probability tokens. It is one of the few cases where tuning both can be justified — you are using top-p as a safety rail on a high-temperature setup.

What is top-k sampling?

Top-k sampling restricts the candidate pool to the K most-probable tokens, regardless of how much probability mass they cover. Where top-p keeps "as many tokens as it takes to reach 90%," top-k keeps "exactly the top 50, no matter what."

How the common values behave:

• Top-k 1 — Greedy. Always pick the single most-probable token. Equivalent to temperature 0.
• Top-k 40 — Consider only the top 40 tokens. A common default in open-source models.
• Top-k 50 — Slightly wider; another frequent default.
• Top-k 1000+ — Effectively no truncation for most models, since the useful probability mass is concentrated in far fewer tokens.

Range: 1 to vocabulary size.

When to use top-k

Top-k is the parameter you are least likely to touch. It is a coarser, less adaptive version of top-p, and OpenAI's Chat Completions API never exposed it. You will encounter it mainly in:

• Open-source models (Llama, Qwen, Mistral) where the default top-p is not well tuned for your use case.
• Reproducing research that specified an exact top-k value.
• Anthropic's classic API, where top_k is available as a way to remove long-tail, low-probability responses.

For most people in 2026, the entire top-k conversation is academic. Tune temperature, occasionally touch top-p, and leave top-k alone.

How do temperature, top-p, and top-k interact?

They apply in a specific sequence during token selection. Knowing the order helps you predict what happens when more than one is set.

The LLM sampling pipeline runs like this:

1. The model produces a raw probability distribution over all vocabulary tokens.
2. Temperature scales the distribution — higher flattens, lower sharpens.
3. Top-k truncates to the K most-probable tokens. (In Anthropic's pipeline, the top_k filter runs first, discarding all but the K highest-probability tokens and renormalizing.)
4. Top-p truncates to the nucleus — the smallest set whose cumulative probability reaches P.
5. The model samples one token from whatever survives.

Here is the same idea as a table, showing what each stage does:

If both top-p and top-k are set, both filters apply and you get whichever is more restrictive at each step. Usually one is enough — and on the newest frontier models, as you will see, you can set none of them.

What are the best default settings for each use case?

Here are the configurations that cover the overwhelming majority of real-world needs. Copy whichever matches your task.

For everyday balanced tasks:

temperature: 0.7
top_p: 0.95        (or just omit — leave at default)
top_k: 50          (only if your model supports it; otherwise omit)

For deterministic output (extraction, classification, structured data):

temperature: 0
top_p: 1.0

For creative variation (marketing, ideation, fiction):

temperature: 1.0
top_p: 0.95

For maximum randomness (rarely used in production):

temperature: 1.5
top_p: 1.0

Notice that you are only ever really moving temperature. That is intentional. For the vast majority of applications, temperature is the only sampling parameter worth tuning, and everything else stays at default.

Real API examples: OpenAI and Anthropic

Concrete code beats abstract advice. Here are two common scenarios with working request shapes. (These use the classic, sampling-enabled models; the next section covers the newest models that reject these parameters.)

Example 1: Extracting entities from an email

You want a deterministic, parseable shape every single time. Temperature 0 plus structured output is the right call.

// Production extraction — determinism is critical
const response = await openai.chat.completions.create({
  model: "gpt-4o",
  messages: [
    { role: "system", content: "Extract contact details as JSON." },
    { role: "user", content: emailBody },
  ],
  response_format: { type: "json_schema", json_schema: contactSchema },
  temperature: 0, // same input -> same output
});

Example 2: Generating 10 ad variants

Here you want diversity. Crank temperature up and request multiple completions in one call.

// Creative variation desired
const response = await openai.chat.completions.create({
  model: "gpt-4o",
  messages: [
    { role: "user", content: "Write a punchy headline for a running shoe." },
  ],
  temperature: 1.0, // variety wanted
  n: 10, // 10 distinct completions in a single request
});

If you find yourself hand-setting temperature per task every time, that is exactly the kind of repetitive setup a prompt library with per-task presets is meant to eliminate.

The Big 2026 Shift: Models That Won't Let You Set Temperature

This is the most important update since this guide was first written, and it changes how you should think about sampling. The newest frontier reasoning models are removing temperature, top-p, and top-k. Last verified October 2026.

OpenAI's reasoning models

OpenAI's latest-model guide says: "When reasoning effort is not none, remove temperature, top_p, and top_logprobs." The same guide notes that some current models (GPT-6 Astra and GPT-6.1 Sol) do not support the none effort level, while others (GPT-6.1 Terra, GPT-6 Sol, GPT-6 Luna) do. Earlier GPT-5 reasoning models went the same way, returning an error like "Unsupported value: 'temperature' does not support 0.2 with this model. Only the default (1) value is supported," as reported in developer threads. To steer output, set reasoning_effort (or reasoning.effort in the Responses API) and prompt for the length and style you want.

Anthropic's Claude Opus models

Anthropic went the same direction. The Claude Opus 5.5 migration guide lists, among the requirements for every request: "Omit temperature, top_p, and top_k. Use prompting to guide the model's behavior." On the earlier Claude Opus 4.7 and 4.8, Anthropic's documentation says non-default values return a 400 error. The SDK still defines these fields for backward compatibility, so your code type-checks, but the request is rejected server-side at runtime. Instead, Claude uses the effort parameter (Opus 5.5 defaults to medium) with adaptive thinking to control reasoning depth.

# Old way (Claude Opus 4.6 and earlier): sampling parameters accepted
response = client.messages.create(
    model="claude-opus-4-6",
    messages=[...],
    temperature=0.3,
    top_p=0.95,
)

# New way (Claude Opus 5.5): omit sampling parameters, set effort
response = client.messages.create(
    model="claude-opus-5-5",
    messages=[...],
    output_config={"effort": "high"},
)

What this means for your mental model

The takeaway is not "temperature is dead." Plenty of models, including older chat models and essentially the whole open-source ecosystem, still expose temperature, top-p, and top-k. But the frontier is clearly moving toward reasoning-effort controls instead of sampling controls. When you pick a model, check whether it accepts sampling parameters before you build logic that depends on them, and have a fallback path for models that reject them. The compatibility table at the top of this page has the current picture.

Self-consistency: turning temperature into accuracy

One of the best reasons to raise temperature on purpose is a technique called self-consistency, and the numbers behind it are striking.

Self-consistency was introduced by Wang et al. at Google Research (ICLR 2023). Instead of taking a single greedy answer, you sample several diverse reasoning paths at a non-zero temperature, then take the majority vote across them. The diversity comes from temperature — typically around 0.7 — which lets the model explore different chains of reasoning that often converge on the correct answer even when any single chain might slip.

The accuracy gains reported in the original work are large. According to a breakdown of the paper's benchmarks, self-consistency improved:

• GSM8K (grade-school math) by 17.9%
• SVAMP (arithmetic word problems) by 11.0%
• AQuA (algebraic reasoning) by 12.2%
• StrategyQA (multi-hop reasoning) by 6.4%
• ARC-challenge (science questions) by 3.9%

Here is the pattern in code — five samples at temperature 0.7, then a majority vote:

// Self-consistency: sample N reasoning paths, take the consensus
const responses = await Promise.all(
  Array.from({ length: 5 }).map(() =>
    openai.chat.completions.create({
      model: "gpt-4o",
      messages: [{ role: "user", content: chainOfThoughtPrompt }],
      temperature: 0.7, // diversity drives the gains
    })
  )
);

const answers = responses.map((r) => extractAnswer(r));
const consensus = mode(answers); // majority vote

This pairs naturally with chain-of-thought prompting: you elicit explicit reasoning, sample it several times at moderate temperature, and let the majority correct one-off mistakes. For high-stakes math and logic, it is one of the most reliable accuracy boosts you can get without changing models. The obvious trade-off is cost and latency — you are paying for N completions instead of one — so reserve it for the tasks where correctness genuinely matters.

Common mistakes with sampling parameters

Even experienced builders trip over the same handful of issues. Here are the ones worth memorizing.

1. Setting temperature 0 for creative tasks. Deterministic output is repeatable but bland. Use 0.7-1.0 for marketing copy and 1.0+ for ideation. If your "creative" outputs feel samey, your temperature is probably too low.

2. Setting temperature 1.5+ in production. Very high temperatures reach into the long tail of unlikely tokens, where output degrades into broken or off-topic text. Cap production temperatures around 1.2 for stability.

3. Tuning both top-p and temperature. Providers explicitly recommend changing one or the other, not both, because their combined effect is hard to predict. Pick temperature and leave top-p alone unless you have a specific reason.

4. Forgetting that temperature is free. Temperature does not change cost or model tier — only which token is chosen. Tune it per task, not once per app. There is no budget reason to use the same temperature everywhere.

5. Shipping random temperature into critical paths. The same prompt at temperature 0.7 returns different output on every call. For operations that must be reproducible — audits, idempotent pipelines, regression tests — use temperature 0 plus a fixed seed where the API supports one.

6. Assuming temperature works on every model. As covered above, current reasoning models reject or tell you to omit sampling values. Code that hardcodes temperature=0.3 can throw an error on those models. Branch by model capability.

7. Treating temperature 0 as perfectly deterministic. It is the most deterministic setting available, but floating-point math on GPUs, mixture-of-experts routing, and hardware load-balancing can still introduce rare variation. Add a seed when you truly need bit-for-bit reproducibility.

What Changed Across 2025-2026

• Frontier models dropped sampling controls. GPT-5 reasoning models fixed temperature at 1, Claude Opus 4.7 and 4.8 began rejecting temperature, top_p, and top_k outright, and Claude Opus 5.5 and OpenAI's current reasoning-effort models tell you to omit them.
• Structured output matured. JSON-schema modes plus temperature 0 became the standard for production extraction on models that still accept temperature.
• Open-source kept the full toolkit. Llama, Qwen, and Mistral families still expose all three, so the classic knowledge remains essential when you self-host.

What to do next

You do not need to memorize the math behind softmax to use these parameters well. You need a small set of habits.

1. Audit your production prompts. If you are using temperature 0.7 for everything, you are probably leaking accuracy on deterministic tasks and starving creative tasks of variety. Right-size per task.
2. A/B test by temperature. Run the same prompt at 0, 0.5, and 1.0. Note which value suits each job, then standardize.
3. Implement self-consistency for high-stakes reasoning. Five samples at 0.7 plus a majority vote is a proven accuracy boost for math and logic.
4. Pair temperature 0 with structured output for extraction. It is the most reliable production pattern for parseable data.
5. Check model capabilities before you ship. Branch your code so reasoning models that reject sampling parameters fall back to reasoning effort and prompting.

Tools that ship per-task temperature presets — like the Prompt Architects Chrome extension and prompt library — save you the manual setup on every call. But the understanding matters more than the tool: once you know what each parameter does, you can get the right behavior from any model, whether it exposes a temperature dial or a reasoning-effort knob.

By Nafiul Hasan — Founder of Prompt Architects, builder of prompt-engineering tooling used across ChatGPT, Claude, and Gemini. Last updated: October 27, 2026.