Generating Multiple Completions (n and Best-of)

TL;DR: A multiple completions API call asks one model for several answers at once. OpenAI supports it through n on Chat Completions and best_of on the legacy Completions endpoint only; Gemini uses candidateCount; Claude and OpenAI's Responses API have no equivalent, so you send parallel requests. Every extra candidate is billed. Decide how you will choose before you generate.

Asking for several answers instead of one sounds like a free quality upgrade. It is not free, and it is only an upgrade if something picks the best answer afterwards. This guide covers the multiple completions API surface as it stands today at OpenAI, Anthropic and Google, what each option really costs, and the selection strategies that make the extra spend worth it.

Every parameter below was read on the vendor's own API reference on 17 September 2026. These fields move. OpenAI dropped n from its newest endpoint, and Anthropic has deprecated the sampling controls that used to make Claude's candidates differ. Check the linked references before you ship.

What does a multiple completions API call actually do?

It runs the same prompt through the model several times and returns every result. The candidates are independent samples, so they differ only as much as the sampling settings and the prompt allow.

There are three distinct mechanisms, and articles on this topic blur them constantly:

1. Return N candidates. You ask for N and get N back. OpenAI's n and Google's candidateCount work this way. You do the choosing.
2. Generate N, return the best. The server generates several candidates, scores them, and sends back only the winner. OpenAI's best_of is the only published example, and it lives on a legacy endpoint.
3. Send N requests. No parameter at all. Your code fires the same request several times, usually in parallel. This is the only option on Claude and on OpenAI's Responses API.

The first two save you plumbing. The third works everywhere and gives you more control, at the price of more requests and more rate-limit headroom.

Which providers support n or best-of in 2026?

Here is the short version. The LLM parameter cheat sheet has the full parameter matrix; this table only covers candidate generation.

Checked 17 September 2026 against each vendor's API reference.

The pattern is worth noticing. The newest surface at each of OpenAI and Google has no candidate count, and Anthropic's Messages API has none either. Built-in multi-candidate generation is becoming a legacy convenience, and the send-N-requests pattern is where new code should start.

How does OpenAI's n parameter work?

On Chat Completions, n is an integer from 1 to 128, default 1, and each candidate arrives as its own entry in the choices array with an index. OpenAI's specification is blunt about the bill: "Note that you will be charged based on the number of generated tokens across all of the choices."

A minimal call looks like this:

from openai import OpenAI

client = OpenAI()

resp = client.chat.completions.create(
    model="YOUR_MODEL_ID",
    n=3,
    messages=[
        {"role": "system", "content": "You write product taglines. Under 8 words."},
        {"role": "user", "content": "A reusable water bottle that tracks how much you drink."},
    ],
)

for choice in resp.choices:
    print(choice.index, choice.message.content)

Two practical points. First, if you are starting a new project, OpenAI itself steers you to the Responses API, and n does not exist there. OpenAI's migration guide puts it plainly: "In Responses, we've removed this param, leaving only one generation." Its comparison table says multiple generations are "Not available in Responses; make separate requests if you need multiple candidate outputs". So n is a reason to stay on Chat Completions for this one job, not a reason to build new work around it.

Second, n does not make candidates different. If your temperature is low, three candidates can come back word for word. The temperature, seeds and determinism guide explains what each knob controls.

Does best_of still exist?

Yes, but only on the older Completions endpoint, the one whose named models in the specification are gpt-3.5-turbo-instruct, davinci-002 and babbage-002. There, best_of generates candidates server-side and returns the one "with the highest log probability per token". It accepts up to 20, and the specification adds: "Results cannot be streamed."

When you combine the two, best_of sets how many candidates are generated and n sets how many come back, and the specification requires best_of to be larger than n. You pay for every candidate generated, including the ones you never see.

The scoring rule matters more than the parameter. Highest average log probability means the answer the model was most confident about, token by token. That is a reasonable proxy for fluency and a poor proxy for correctness or originality. For creative work it actively selects the blandest option. That is one reason the technique never made it onto the chat endpoint, and why the selection strategies further down use a different rule.

If you want best-of behaviour on a current model, build it: generate N candidates with n or parallel requests, then score them yourself.

How do you get multiple completions from Claude?

You send several requests. Anthropic's Messages API reference, checked 17 September 2026, lists no candidate-count parameter among its request fields. Parallel requests are the documented-by-absence answer:

import asyncio
from anthropic import AsyncAnthropic

client = AsyncAnthropic()

ANGLES = [
    "Lead with the health benefit.",
    "Lead with the tracking feature.",
    "Lead with the environmental angle.",
]

async def one(angle: str) -> str:
    msg = await client.messages.create(
        model="YOUR_MODEL_ID",
        max_tokens=100,
        system="You write product taglines. Under 8 words.",
        messages=[{
            "role": "user",
            "content": f"A reusable water bottle that tracks how much you drink. {angle}",
        }],
    )
    return msg.content[0].text

async def main() -> None:
    results = await asyncio.gather(*(one(a) for a in ANGLES))
    for i, text in enumerate(results):
        print(i, text)

asyncio.run(main())

Notice what that code does differently: each request carries a different angle. That is deliberate, and on current Claude models it is close to mandatory. Anthropic's reference now marks temperature, top_p and top_k as deprecated, and states that "Models released after Claude Opus 4.6 do not support setting temperature." A value of 1.0 is still accepted for backwards compatibility; anything else returns a 400. The same page restricts top_p and top_k on those models too.

The practical effect is that you cannot raise or lower randomness to control how different your candidates are. Variety has to come from the prompt. Varying the angle, audience or constraint per request produces genuinely different candidates, which is usually what you wanted from multiple completions in the first place.

If the answers are not needed in real time, the Message Batches API is the cheaper route. Anthropic's batch documentation says: "All usage is charged at 50% of the standard API prices." OpenAI's and Google's batch documentation each describe the same half-price arrangement for asynchronous work.

What about Gemini's candidateCount?

Gemini's generateContent supports it directly. The field is candidateCount in REST and candidate_count in the Python SDK, and Google's reference describes it as "Number of generated responses to return." It defaults to 1 when unset.

from google import genai
from google.genai import types

client = genai.Client()

resp = client.models.generate_content(
    model="YOUR_MODEL_ID",
    contents="Write a tagline for a reusable water bottle that tracks how much you drink.",
    config=types.GenerateContentConfig(
        candidate_count=3,
        max_output_tokens=60,
    ),
)

for cand in resp.candidates:
    print(cand.index, cand.content.parts[0].text)

Billing follows the same logic as OpenAI's. The usage metadata field candidatesTokenCount is defined as "Total number of tokens across all the generated response candidates."

The caveat is the surface. Google is moving developers towards its Interactions API, and that API's generation configuration, checked on the same date, carries fields for output length, seed, speech and stop sequences but nothing for candidate count. Code written against candidateCount today is tied to generateContent.

What does generating multiple completions really cost?

Output scales with the number of candidates, and output is the expensive side of most price lists. Input depends on the mechanism.

The vendors document how output is counted across candidates; how the input side of a multi-candidate request is billed is not spelled out on the reference pages I checked, so measure it from the usage fields on your own account before you rely on it.

For the parallel-request pattern, the repeated input is where money leaks. A long system prompt sent five times is five times the input bill unless the provider can cache it. If your candidates share a large fixed prefix, the prompt caching parameters are the first thing to set up. Put the shared material first and the per-candidate variation last, so the prefix is identical across requests.

A rough rule for budgeting: estimate the cost of one good answer, multiply by N, then ask whether the selection step raises quality enough to justify that multiple. For a tagline brainstorm where a human picks one, three to five candidates is plenty. For a batch classification job, you almost never want more than one per item unless you are voting.

How should you pick the winning completion?

This is the part that decides whether the spend was worth it. Five selection strategies cover almost every real case.

1. Let a human choose. The simplest and often the best. Brainstorming, headlines, subject lines and image prompts all benefit from three options on screen. No scoring code, and the human's taste is the objective function.

2. Filter with hard checks first. Before any clever scoring, discard candidates that fail something you can test mechanically: invalid JSON, a missing required field, a word count over the limit, a banned phrase. This is ordinary schema validation, and it is cheap. Often only one candidate survives, and you are done.

3. Vote on the answer. For questions with one correct answer (a number, a category, yes or no), sample several reasoning paths and take the answer most of them reach. This is self-consistency, and it is the strongest reason to generate multiple completions at all. It only works when answers can be compared exactly, so ask for the final answer in a fixed format on its own line.

4. Ask a judge model. For open-ended text, send the candidates to a model with an explicit rubric and ask it to pick one and explain why. Keep the rubric concrete, shuffle the candidate order between runs, and never let the judge see which candidate came from which prompt variant.

5. Score with a heuristic. Length closest to target, reading level, keyword coverage. Crude, fast and deterministic. Useful as a tiebreaker after steps 2 and 3.

Here is a compact pattern that combines the hard filter and the vote:

import json
from collections import Counter

def pick(candidates: list[str]) -> str | None:
    labels = []
    for text in candidates:
        try:
            data = json.loads(text)
        except json.JSONDecodeError:
            continue
        label = data.get("label")
        if label in {"billing", "bug", "feature_request", "other"}:
            labels.append(label)
    if not labels:
        return None
    label, votes = Counter(labels).most_common(1)[0]
    # Require a real majority, otherwise send the ticket to a human.
    return label if votes > len(candidates) / 2 else None

The last line matters. A plurality of two out of five is not agreement. When candidates disagree, that disagreement is useful information, and routing the case to a person is a better outcome than picking a winner at random.

How many candidates should you request?

Fewer than you think. The useful number depends on the selection strategy, not on the model.

• Human picks: three to five. Beyond that, people skim rather than compare, and the extra options stop earning their cost.
• Hard-check filter: enough that at least one usually passes. If one attempt in five fails validation, two or three candidates covers most requests; if half fail, fix the prompt instead of adding candidates.
• Voting: an odd number, so a two-way split cannot tie. Five is a sensible starting point for classification. Measure how often the vote changes the answer compared with a single sample, and drop back to one if it rarely does.
• Judge model: three to four. A judge comparing many long candidates at once gets less reliable, and you pay for the judge's input as well.

Whatever you choose, log the candidates and the chosen one for a week. If the first candidate wins nearly every time, the extra samples are not doing any work, and you can cut N without losing quality.

When should you not ask for multiple completions?

When nothing downstream uses the extras. The most common waste is code that requests n=3 and then reads choices[0], usually because someone copied an example.

Skip it, too, when you need identical output for identical input. Multiple candidates are the opposite of that goal. If repeatability is what you are after, the sibling guide on writing idempotent prompts is the better starting point.

And skip it when the prompt is the problem. Five samples of an underspecified prompt give you five mediocre answers and a judge choosing the least bad one. A clear role, context and output format usually does more for quality than any amount of resampling. Where the instructions live matters too, and system prompt handling differs by provider.

Where Prompt Architects fits

Prompt Architects does not call these APIs for you and has no public REST API. What it does is the step before: turning a rough request into a structured prompt with a clear role, context and output format, which is what makes candidates worth comparing. If you work in Claude Code, Cursor or Codex, our MCP server exposes the same improve and refine tools inside the editor where you are writing the sampling code. The Free plan includes 5 prompt enhancements per day, according to our FAQ page.