Why Do I Keep Hitting Rate Limits?

TL;DR: A 429 error is not one thing. It can mean a per-minute request ceiling, a per-minute token ceiling, a daily cap, a sudden-growth "ramp" penalty, or a monthly spend cap that a retry cannot fix at all. OpenAI, Anthropic and Google each measure and structure these differently. Diagnose which one you hit before you touch your retry logic.

What actually triggers a 429?

The HTTP status code itself is generic by design. MDN's own definition covers every vendor at once: "the client has sent too many requests in a given amount of time." MDN adds that the practice of throttling requests this way is commonly known as rate limiting. That single status code gets reused by every AI API for at least four or five structurally different conditions, and the fix for one often does nothing for another.

Here is the practical problem with most advice on this topic: it treats "rate limit" as one number to raise, when in reality you are usually hitting one of several independent ceilings, only one of which is a classic per-minute rate limit in the traditional sense.

RPM, TPM, TPD: the metrics, not the numbers

OpenAI's own rate-limits documentation gives the cleanest enumeration of what gets measured, and it is worth reading exactly as written: "Rate limits use metrics such as RPM (requests per minute), RPD (requests per day), TPM (tokens per minute), TPD (tokens per day), IPM (images per minute), and audio minutes per minute for some streaming audio models." Then the sentence that actually explains most confusing 429s: "Rate limits can be hit across any of the options depending on what occurs first."

OpenAI's own illustration makes this concrete: "you might send 20 requests with only 100 tokens to the ChatCompletions endpoint and that would fill your limit (if your RPM was 20), even if you didn’t send 150k tokens (if your TPM limit was 150k) within those 20 requests." Twenty tiny requests can exhaust your request ceiling while your token ceiling sits almost untouched. This is why being nowhere near your token limit is not proof you are not rate limited — you may be tripping a completely different metric.

One scoping detail worth knowing before you go looking for the culprit: OpenAI states plainly that "Rate limits are defined at the organization level and at the project level, not user level." A single high-volume integration on a shared project key can exhaust a limit that no individual caller ever approached alone.

Anthropic splits your token limit into two directions

Where OpenAI's TPM is one combined number, Anthropic's Messages API measures tokens in both directions separately. Its documentation states this directly: "The rate limits for the Messages API are measured in requests per minute (RPM), input tokens per minute (ITPM), and output tokens per minute (OTPM) for each model class." That is a genuinely different shape from a single TPM figure: a request with a huge prompt and a tiny response draws mostly against ITPM, while a short prompt producing a long generation draws mostly against OTPM.

Anthropic also documents a mechanism with no real equivalent elsewhere: prompt caching narrows what counts against your limit. Its docs state it plainly: "For most Claude models, only uncached input tokens count toward your ITPM rate limits." Breaking the accounting down further: freshly-read tokens from cache do not count toward ITPM, while tokens newly written to cache and genuinely fresh input do. Heavy reuse of a cached system prompt or reference document can raise your effective ceiling without Anthropic raising your tier at all.

Anthropic is also explicit about how enforcement actually behaves moment to moment, which explains why bursts fail even under a limit that looks generous on paper: "a rate of 60 requests per minute (RPM) might be enforced as 1 request per second." That is a deliberately smoothed pace rather than letting you spend the whole minute's allowance in one burst. The mechanism behind that is named directly: "The API uses the token bucket algorithm to do rate limiting. This means that your capacity is continuously replenished up to your maximum limit, rather than being reset at fixed intervals." That is a smoother, continuously-refilling model than a hard reset every sixty seconds.

Google adds a fourth dimension: spend, not just volume

Google's Gemini API layers a genuinely different kind of limit on top of the usual volume metrics, and this is the clearest illustration in any of the three vendors' docs of rate limiting and billing protection sharing one mechanism. Google states that "the Gemini API enforces spend-based rate limits to protect against unexpected charges." This applies on top of its ordinary RPM and TPM limits, and Google specifies the window this way: "These limits are evaluated on a rolling 10-minute window." Hit one, and "the API returns a 429 RESOURCE_EXHAUSTED error", using the same status code as every other rate limit on the platform, distinguished only by its error string.

That rolling 10-minute spend window is a meaningfully different animal from a classic per-minute request ceiling: it exists specifically to catch a sudden spike in expensive calls, not high request volume as such. Google's advice for resolving it reflects that framing directly — retry after a short wait, or "Reduce the rate of expensive requests, for example by using smaller context windows or shorter outputs" — a fix aimed at cost per call, not call count.

Is a rate limit the same thing as a quota?

No, and conflating the two is the single most expensive mistake in this whole topic, because the fix for one does nothing for the other.

A genuine rate limit is temporary. It is measured over seconds or minutes, and it resets on its own without you doing anything. A quota, by contrast, is a spend or usage cap tied to a longer period (often a full billing cycle), and it does not resolve itself on any short timer. Both can surface as the identical HTTP 429 status code, which is exactly why "just retry" is unsafe advice to follow blindly.

Anthropic's documentation shows the quota case in unusually concrete terms. Once an organization crosses its tier's spend cap, "API usage pauses until 00:00 UTC on the first day of the next month, unless you request a higher limit sooner." And the response header that would normally tell you how long to wait is deliberately absent for this specific case — Anthropic's own header reference notes its retry-after field is "Not sent with the spend-cap 429" — because there is no short wait that fixes it.

OpenAI states the general principle even more directly, and it applies just as much to Anthropic's and Google's quota-shaped 429s as to its own: "Retry-After may be present on 429 responses caused by a temporary rate limit. It does not mean that quota, billing, or other errors that require user action can be resolved by retrying." If your error message mentions a monthly threshold, a spend cap, or an account-level limit rather than a request or token rate, no amount of backoff will help. The fix is raising the cap or waiting for the billing period to turn over, not tuning your retry loop.

How tier ladders actually work — three different graduation models

All three vendors raise your limits automatically as you use the API more, but the criteria genuinely differ, and knowing which model applies to you tells you what actually moves the needle.

OpenAI graduates by cumulative spend. Its own phrasing: "As your spend on our API goes up, we automatically graduate you to the next usage tier." That progression moves from Free through five numbered paid tiers as your total paid amount crosses fixed thresholds.

Anthropic graduates by usage history and account standing, not a stated dollar figure. Anthropic's own docs state that new organizations "may start in the Evaluation tier, with limits below the standard limits shown on this page while account history is established." From there, organizations move through named tiers (Start, Build, Scale) as that history accumulates, rather than at a specific spend milestone you can predict in advance.

Google ties tiers to linked billing plus elapsed time, and moves faster than either of the others once you qualify: "Tier upgrades from the Free to Tier 1 will typically take effect instantly, and subsequent tier upgrades will take effect within 10 minutes" of meeting the stated billing and payment-history criteria for that tier.

Three genuinely different models: spend-triggered, history-triggered, and billing-plus-time-triggered. If you are trying to get a higher limit quickly, the lever that works depends entirely on which vendor you are asking.

The ramp-rate trap: a "too fast," not just a "too much"

This is the failure mode that catches people who read every other section of this post and still get confused, because it looks identical to a normal rate limit but is not one. OpenAI documents it directly: "A slow_down error can occur even when your traffic is within its requests-per-minute and tokens-per-minute limits. It reflects how quickly traffic increased, not whether you exhausted those limits."

In other words, you can be well under every published ceiling and still get throttled, because the system is reacting to the shape of your growth curve, not just its current level. OpenAI's own rule of thumb for avoiding it: "once your traffic reaches 1 million input tokens per minute (TPM), increase it by no more than 50% every 15 minutes." Launching a new integration at full production volume on day one, rather than ramping into it, is a textbook way to hit this even on a generously-provisioned account.

Fixing it for real: backoff, jitter, and knowing what not to retry

Once you know which of the above you actually hit, the fix logic is simple and near-universal for the genuinely temporary cases.

Exponential backoff with jitter is the standard shape: wait briefly after a failed request, increase that wait after each subsequent failure, and add a small random offset so that many simultaneous callers do not all retry at exactly the same instant and re-trigger the same limit together. OpenAI's own guidance is blunt about the one thing this does not fix: "Don’t retry quota, billing, or other errors that require you to take action." And it flags a subtler trap that catches naive retry loops: "unsuccessful requests contribute to your per-minute limit, so continuously resending a request won’t work" — hammering a closed door faster only spends more of the same budget you are trying to recover.

import random
import time

def call_with_backoff(fn, max_retries=5, base_delay=1.0, max_delay=60.0):
    """Retry a rate-limited call with exponential backoff and jitter.
    fn should raise a RateLimitError-style exception on a 429."""
    for attempt in range(max_retries):
        try:
            return fn()
        except RateLimitError as e:
            if attempt == max_retries - 1:
                raise
            retry_after = getattr(e, "retry_after_seconds", None)
            if retry_after is not None:
                delay = retry_after
            else:
                delay = min(max_delay, base_delay * (2 ** attempt))
            delay += random.uniform(0, delay * 0.25)  # jitter
            time.sleep(delay)

For workloads that do not need an immediate answer (bulk classification, batch summarisation, large-scale data processing), both OpenAI and Google offer a structurally different path around all of this rather than a retry loop at all. OpenAI's framing: "If your use case does not require immediate responses, you can use the Batch API to more easily submit and execute large collections of requests without impacting your synchronous request rate limits." A batch job runs against its own separate limit entirely, so it stops competing with your live traffic for the same per-minute ceiling.

The one-line version

Identify which ceiling you hit — requests, tokens (and on Anthropic, which direction), a daily cap, a sudden-growth penalty, or a spend cap — before you touch your retry code, because only some of those respond to patience. A genuine rate limit resets on a clock and rewards a well-built backoff loop. A quota resets on a billing cycle and rewards raising the cap or waiting it out. Confusing the two just burns retries against a door that a timer, not a delay, will open.

If your requests are also getting truncated rather than just rejected, that is a different parameter entirely — max tokens vs max output tokens covers the output-length ceiling this post does not. If it is specifically your input tokens per minute that keep getting exhausted, an oversized prompt is often the real culprit; why is my prompt too long covers trimming it down. And if two runs of the same prompt keep coming back different regardless of rate limits, why do I get a different answer every time covers that separately.

Sources and access dates

Every claim above traces to a primary document, accessed September 3, 2026:

• General HTTP behaviour: MDN's 429 Too Many Requests reference and MDN's Retry-After header reference.
• OpenAI: the rate limits guide.
• Anthropic: the rate limits reference.
• Google: the Gemini API rate limits guide.

Exact RPM/TPM figures per tier and per model were deliberately left out of this post because they change on the order of months; the mechanics above — what gets measured, how tiers are earned, and the rate-limit-versus-quota distinction — are the durable part. Check your account's own limits page for current numbers.