Self-Consistency Prompting (Sample, Then Vote)

TL;DR: Self consistency prompting means running the same chain-of-thought prompt several times at a non-zero temperature and keeping whichever final answer comes up most often. Wang et al.'s 2022 paper measured it lifting PaLM-540B's grade-school-math accuracy from 56.5% to 74.4%. It costs several times the tokens of one run, so it's worth reserving for answers you actually can't afford to get wrong.

What Is Self-Consistency Prompting?

Self-consistency prompting is a decoding strategy, not a phrasing trick. Instead of asking a model for one reasoning path and trusting whatever comes back, you sample several independent reasoning paths for the same question and keep whichever final answer the paths agree on most.

It comes from a specific paper: Self-Consistency Improves Chain of Thought Reasoning in Language Models (Xuezhi Wang, Jason Wei, Dale Schuurmans, Quoc Le, Ed Chi, Sharan Narang, Aakanksha Chowdhery, and Denny Zhou, Google Research), first posted to arXiv in March 2022 and published at ICLR 2023. The paper describes the method plainly: it "first samples a diverse set of reasoning paths instead of only taking the greedy one, and then selects the most consistent answer by marginalizing out the sampled reasoning paths."

That's the whole idea, stripped of jargon. You don't ask the model to think harder in a single pass. You ask it to think several times, independently, and you trust the answer that survives disagreement.

One naming note worth flagging up front: this has nothing to do with visual style consistency in image generation. If you're trying to keep a character or art style consistent across a batch of AI images, that's a different problem, usually solved with LoRAs or reference images rather than sampling and voting — see LoRAs and style consistency if that's what brought you here.

How Does Self-Consistency Actually Work?

The paper lays it out as three steps: prompt the model with chain-of-thought examples as usual, then sample instead of greedily decoding a single path (the model generates several different reasoning chains for the same question), and finally "marginalize out the reasoning paths and aggregate by choosing the most consistent answer in the final answer set." In practice that third step is a majority vote: whichever final answer appears most often across your sampled runs wins, and the reasoning text that produced it is discarded.

The intuition behind it, in the authors' own words, is that a complex reasoning problem "typically admits multiple different ways of thinking leading to its unique correct answer." Correct reasoning tends to converge on the same answer even when the wording of the reasoning differs; wrong reasoning tends to scatter. Sample enough independent attempts and the correct answer usually has the largest cluster.

How Much Does Self-Consistency Actually Improve Accuracy?

The headline comparison in the paper, run on Google's PaLM-540B model: standard chain-of-thought prompting reached 56.5% accuracy on GSM8K, a grade-school math word-problem benchmark; adding self-consistency raised that to 74.4%, an 18-point jump on the exact same prompts and the exact same model. The only thing that changed was sampling multiple reasoning paths and voting instead of taking one.

That wasn't a one-off result on a single benchmark. Here's the same PaLM-540B comparison across every arithmetic benchmark the paper reports it on:

The paper's own abstract summarizes gains across a broader set of benchmarks, some measured on other models in the study rather than PaLM-540B specifically: "GSM8K (+17.9%), SVAMP (+11.0%), AQuA (+12.2%), StrategyQA (+6.4%) and ARC-challenge (+3.9%)." Note the SVAMP figure there (+11.0) differs from the PaLM-540B-specific SVAMP gain in the table above (+7.6) — the abstract is reporting the best result across the four models tested, not a single fixed comparison, which is worth knowing before you quote either number out of context.

Does Self-Consistency Help More on Bigger Models?

Yes, and the paper says so directly, which matters if you're deciding whether the technique is worth the extra cost on whatever model you're actually using. Tested across four models of different scale, the improvement from self-consistency grew with model size: "the gains become more significant when the language model’s scale increases, e.g., we see +3%-6% absolute accuracy improvement over UL2-20B but +9%-23% for LaMDA-137B and GPT-3."

That's a genuinely useful caveat for anyone trying to decide whether this is worth doing on a smaller or lighter-weight model. The paper's own data suggests a small model gets a smaller lift from the same technique, not because self-consistency stops working, but because a smaller model's reasoning chains may already be closer to converging (or already wrong often enough that voting doesn't rescue much).

When Should You Actually Use Self-Consistency?

Reach for self-consistency when both of these are true at once: the task has a single checkable final answer (a number, a category, a yes/no), and getting that specific answer wrong is expensive enough to justify running the prompt several times instead of once.

That combination shows up in a narrower set of situations than reasoning task generally suggests:

• A specific calculation you're about to act on. A cost estimate, a unit conversion, a date calculation feeding a deadline. Running it 5 times and checking agreement is cheap insurance against a one-off arithmetic slip.
• A classification decision with real downstream consequences. Routing a legal document, flagging a support ticket as urgent, categorizing a transaction for compliance purposes. If a wrong label triggers a costly downstream action, the extra tokens are worth it.
• A single high-stakes yes/no judgment. Does this contract clause conflict with this other clause? is exactly the kind of fixed-answer-set question self-consistency is built for.

It's overkill for anything exploratory, anything creative, or anything you're going to read and sanity-check yourself anyway. If you'd catch a wrong answer just by glancing at it, the extra API calls buy you very little.

How Do You Run Self-Consistency by Hand?

You don't need code or an API loop to get the core benefit. Start from a normal chain-of-thought prompt:

[Your reasoning question — include 1-3 worked examples if the task
benefits from few-shot chain-of-thought]

Let's think step by step.

Run that exact prompt 3-5 separate times (fresh conversations, or regenerate if your interface allows a non-zero temperature), and each time write down only the final answer, not the reasoning. Then look at what you collected:

Run 1: 46
Run 2: 46
Run 3: 51
Run 4: 46
Run 5: 46

Majority answer: 46 (4 of 5 runs agree)

If four or five runs land on the same number, you can be reasonably confident in it. If the runs split roughly evenly, that's real information too: it tells you the model is not confidently converging on this question, and you should either simplify the prompt, add a worked example, or verify the answer another way before trusting it.

Is Self-Consistency the Same as Asking the Model to Double-Check Itself?

No, and the difference matters. Asking a model are you sure? or double-check your work in the same conversation means the model is looking back at the reasoning it already committed to, in the same context, with a strong pull toward agreeing with itself. That's a real technique (self-critique and reflexion-style prompting), but it's a different mechanism from self-consistency, which never lets the model see its own earlier attempt at all.

Self-consistency runs the prompt in independent passes, each with no visibility into the others. That independence is the entire point: it's what makes agreement across runs meaningful evidence, rather than the model politely confirming whatever it already said. A model that made an arithmetic slip in run one and gets asked to double-check immediately after often just re-confirms the same slip, because nothing in that follow-up forces it to actually redo the calculation from scratch. Five genuinely separate attempts don't have that problem — a slip in one run has no way to contaminate the other four.

That's also why self-consistency needs a non-zero temperature to work at all. At temperature zero, the model's decoding is deterministic, so running the same prompt five times just produces the same reasoning path five times over, and voting on five identical answers tells you nothing you didn't already know from one. The diversity in the sampled paths is what makes the vote informative; without it, self-consistency degrades into an expensive way of asking once.

If you're deciding between these for a specific problem, the practical rule is: use self-consistency when you can define a single correct final answer and want to guard against one bad sample; use self-critique when the task benefits from the model reviewing and revising its own reasoning in place, which reflexion and self-critique prompting covers in full.

Does Self-Consistency Still Apply to 2026 Reasoning Models?

This is worth being honest about, because the original paper is from March 2022 and its largest tested model, PaLM-540B, predates the reasoning-native models that now dominate frontier releases. None of the four models in the study (UL2-20B, GPT-3-175B, LaMDA-137B, PaLM-540B) do internal chain-of-thought search by default the way current reasoning models do; a plain chain-of-thought prompt on those older models was closer to one shot at reasoning correctly than it is on a model that already explores multiple internal paths before answering.

That difference matters for how much self-consistency still adds. A reasoning-native model already does something structurally similar internally before it commits to an answer, so the marginal benefit of layering an external sample-and-vote loop on top is plausibly smaller than the 18-point GSM8K jump measured in 2022. No published benchmark in this paper, or any source checked for this piece, measures that gap on a current frontier model — this is a reasoned caveat about why the original number probably doesn't transfer unchanged, not a documented finding. The underlying logic still holds regardless of model generation: any single sampled output, however it was produced, can still be wrong, and running an important question more than once and checking agreement remains a cheap way to catch that.

What Are Self-Consistency's Limits?

Two limits are worth knowing before you reach for this on the wrong task.

The second limit is cost, and the paper's own authors flag it rather than glossing over it: "One limitation of self-consistency is that it incurs more computation cost." Their own recommendation is modest — "people can try a small number of paths (e.g., 5 or 10) as a starting point to realize most of the gains while not incurring too much cost" — which is a useful anchor if you're wondering how many runs is enough. More than 10 is rarely buying you much on top of what 5 or 10 already captured in the original research.

The Bottom Line

Self-consistency prompting is not a rewording of your prompt — it's a decision to run the same reasoning prompt several times and trust agreement over any single answer. The original research measured a real, sizable accuracy gain on math and logic benchmarks, largest on the biggest models tested, and it costs roughly what you'd expect: N times the tokens for N samples.

Use it narrowly. It earns its cost on a fixed-answer question where being wrong is expensive, not on every prompt you write. For a related but different failure mode, where a single pass loses the thread entirely rather than landing on a slightly-wrong number, self-ask prompting is worth knowing alongside this one.

By Nafiul Hasan — Founder of Prompt Architects, building prompt tooling used across ChatGPT, Claude, and Gemini, writing from hands-on testing of production prompts.