Why Is My API Bill So High?

TL;DR: Your API bill runs higher than the visible text suggests because reasoning tokens are billed as output even when hidden, four different fields cap "max tokens" across three vendors, and a request can burn its entire budget before producing anything you see. Caching and batching are the two levers that actually move the number.

Why does my bill not match what I can see on screen?

Because you are not only paying for the answer. Reasoning models from OpenAI and Anthropic generate an internal reasoning process before they write the visible response, and both vendors bill that internal process as output tokens.

OpenAI's own reasoning guide is direct about it: "While reasoning tokens are not visible via the API, they still occupy space in the model’s context window and are billed as output tokens." Anthropic's extended thinking documentation says the same thing about its own architecture in different words: "Thinking tokens count toward the max_tokens limit for the turn", so the budget for a turn has to leave room for both. Neither vendor treats reasoning as a free, invisible pre-step. It is metered exactly like the text you can read.

This is the single most under-understood line item on a modern LLM invoice, and it explains why two requests that return identically short answers can carry wildly different costs. A hard problem that makes the model reason for four thousand tokens before writing a two-sentence answer bills for the four thousand plus the two sentences.

Can a request actually run out of budget before writing anything?

Yes, and this is the sharper version of the same problem. OpenAI's guide describes exactly this failure mode: "If the generated tokens reach the context window limit or the max_output_tokens value you’ve set, you’ll receive a response with a status of incomplete and incomplete_details with reason set to max_output_tokens. This might occur before any visible output tokens are produced, meaning you could incur costs for input and reasoning tokens without receiving a visible response."

In other words, a tightly capped max_output_tokens on a hard problem can spend the entire budget "thinking" and return nothing usable, and you are still billed for it. OpenAI's own mitigation is blunt: reserve real headroom. The guide recommends "reserving at least 25,000 tokens for reasoning and outputs when you start experimenting with these models." From there, narrow the buffer once you know your actual usage pattern.

Checking for this is a few lines, and it is worth adding to any reasoning-model call rather than assuming a response is always complete:

response = client.responses.create(
    model="gpt-5.6",
    reasoning={"effort": "medium"},
    input=prompt,
    max_output_tokens=300,
)
if response.status == "incomplete" and response.incomplete_details.reason == "max_output_tokens":
    print("Ran out of tokens")
    if response.output_text:
        print("Partial output:", response.output_text)

Google's Gemini API handles the accounting differently, and this is a real difference worth knowing rather than assuming. Its usage metadata reports thoughtsTokenCount as its own separate integer, distinct from the visible answer, and its own reference states totalTokenCount is defined as the sum of "prompt + thoughts + response candidates". So Google clearly tracks and bills thinking tokens as their own bucket. What Google's documentation does not state, in the same explicit terms as OpenAI and Anthropic, is whether maxOutputTokens itself can be exhausted purely by thinking before any visible candidate text appears. Treat that specific behavior as unconfirmed rather than assuming it works identically to the other two vendors.

Why are there four different "max tokens" settings?

Because three vendors built three different APIs, and one of them built two. None of the four caps your prompt; every one of them only limits how much the model is allowed to generate.

That last row is the trap. A team that migrates a prompt from Anthropic to Google, or from OpenAI's Chat Completions to its Responses API, and reuses the old field name silently gets a request rejected or a limit that means something subtly different, because the parameter names do not line up one-to-one across vendors. If you want the complete parameter-by-parameter table, including sampling and tool-choice fields beyond token caps, post 114 is the full reference; this page is specifically about what drives cost.

Does Google's "minimal" thinking level actually turn thinking off?

Not reliably, and this is a real cost trap for anyone trying to save money by dialing reasoning down to its lowest setting. Gemini's ThinkingLevel enum documents exactly four values: THINKING_LEVEL_UNSPECIFIED as the default, then MINIMAL, LOW, MEDIUM and HIGH. Google's own field description for the lowest tier reads simply: "MINIMAL Little to no thinking."

"Little to no" is not "none." A setting that reads as an off switch is documented as a reduction, not a guarantee of zero reasoning tokens. If you are budgeting for a specific number of thinking tokens rather than a qualitative reduction, that gap matters, and it is worth testing your own workload's actual thoughtsTokenCount at each level rather than assuming MINIMAL means zero.

The four-value structure itself is also easy to get wrong from memory. It is not a binary on/off switch, and it is not a three-value low/medium/high scale either. Support varies by model, so check the current model page before assuming a given level is even available. Our reasoning effort settings reference covers the equivalent OpenAI and Anthropic parameters if you are comparing across vendors.

Can prompt caching actually cut this in half?

Often by more than half, and it is the biggest single lever available for a bill driven by repeated context. Both major vendors document it directly, though the mechanics differ.

OpenAI's prompt caching guide states the benefit plainly: cached input tokens are "discounted up to 90%." Separately, "Prompt caching is enabled by default for supported OpenAI models". So you are not opting into a separate feature so much as structuring your prompt to take advantage of one that is already running. The same guide documents the eligibility floor precisely: "The minimum cacheable prompt length is 1,024 tokens for GPT-5.6 and later and 2,048 tokens for models older than GPT-5.6."

Anthropic's approach is explicit rather than automatic-by-default in the same way, using a cache_control field, but the discount is comparable: "Cache read tokens are 0.1 times the base input tokens price", meaning a cache hit costs roughly a tenth of a fresh read. Anthropic's own docs describe "explicit breakpoints with 5-minute or 1-hour TTLs" at different write-side prices, so the right choice depends on how often your traffic actually reuses the same prefix within that window.

The practical implication for cost is the same on both vendors: put the parts of your prompt that repeat across requests (a long system prompt, a big document, a tool definition list) first and unchanged, and put the part that varies per request last. A prompt that reorders its stable content to the front on every single call gets none of this discount, because the cached prefix has to match exactly to hit. How to give ChatGPT context without repeating yourself walks through the prompt-construction side of the same problem.

Is OpenAI's Batch API actually half price?

Yes, and the published rate card confirms the discount is exactly what OpenAI advertises rather than a smaller real-world number. OpenAI's guide states the offer directly: "Learn how to use OpenAI’s Batch API to send asynchronous groups of requests with 50% lower costs, a separate pool of significantly higher rate limits, and a clear 24-hour turnaround time." A dedicated bullet elsewhere on the same page adds, "Fast completion times: Each batch completes within 24 hours (and often more quickly)".

The current published pricing table makes the arithmetic checkable rather than a marketing round number. On the GPT-5.6 Sol model, standard short-context output is $20.00 per million tokens; batch short-context output on the same model is $10.00. That is precisely half, not "up to" half. On the other end, OpenAI's Fast mode (renamed from Priority processing on July 30, 2026, reachable via service_tier: "fast" or the older "priority" value) prices the same output at $40.00, exactly double standard. So the same model on the same day spans a 4x price range purely on how fast you need the response.

Batch is a real option for anything that does not need an immediate reply: bulk classification, evaluation runs, embedding a document set, or any job you would otherwise queue and poll. If your workload can tolerate the wait, it is the least risky cost lever on this whole page, because it changes nothing about your prompt or your output quality.

What else is quietly inflating the number?

Beyond the mechanics above, four ordinary engineering habits show up on real invoices more often than any single parameter:

• Retrying a truncated response pays for the failure twice. If a request comes back incomplete, resending the same prompt from scratch bills the input tokens again on top of whatever the first attempt already spent on reasoning and partial output. Handle the partial output and extend the budget instead of blindly retrying.
• A long system prompt resent on every turn of a conversation adds up fast. If that system prompt is not eligible for caching, or the conversation is structured so the cacheable prefix keeps shifting, you are paying full input price for the same instructions dozens of times per session.
• Images and documents get tokenized, not just described. OpenAI's vision guide is explicit: "Vision models convert image inputs into billable input tokens", using a patch-based formula for higher-detail modes, and "Image tokens also count toward your tokens per minute (TPM) limits" as well as your bill. PDF uploads carry a second hidden cost: OpenAI's file-input guide notes that "PDF parsing includes both extracted text and page images in context, which can increase token usage", so a ten-page PDF can bill for both the words and a rendered image of every page.
• An agent loop with no real stopping condition keeps calling tools and re-reading context on every iteration. If a bill spikes with no matching change in traffic, a runaway agent loop is worth checking before a pricing change. Why does my agent loop forever? walks through the five most common causes.

How do you actually audit and fix this?

Start with the parts of this page that are pure mechanics, not judgment calls, because they cost nothing to check: confirm you are reading status and incomplete_details on every reasoning-model response, confirm your caching-eligible content sits at the front of the prompt and stays byte-identical between calls, and confirm which of the four "max tokens" fields your current API actually uses.

Then move to the judgment calls: does anything in your pipeline genuinely need xhigh or max reasoning effort, or does a lower effort level on your current model do the job for less? Could a batch job replace a synchronous call anywhere in a nightly pipeline? Do you actually need to resend the whole conversation history on every turn, or would a compacted summary plus the caching-eligible instructions do the same job for less?

None of this requires switching vendors or downgrading model quality. It requires reading the same fields your provider is already returning on every response. Our AI subscription audit covers the adjacent question of what you are paying for beyond the API itself, which is worth running alongside this one.