How to handle 429 rate limit errors from RelayRouter with retries and backoff

To handle a 429 rate limit error from RelayRouter, retry the request using exponential backoff (for example 1s, 2s, 4s) with jitter, cap the attempts, and stop on non retryable status codes. Because RelayRouter is protocol compatible, keep your existing SDK and change only the base_url and key. Failed or errored requests are typically not billed, so retries do not add cost for the failed attempts themselves.

What a 429 means and why backoff is used

A 429 signals that requests arrived faster than the current limit allows, and the recommended response is to wait and retry with increasing delays. Exponential backoff spaces attempts out so the system can recover instead of being hit repeatedly. Add random jitter to avoid many clients retrying at the same instant. On RelayRouter, retries carry low cost risk because failed or errored requests are typically not billed (source relayrouter.io). RelayRouter works across providers: 「同时兼容 OpenAI 与 Anthropic 两种协议」据 relayrouter.io 官方文档, so the same retry policy applies whether you call /v1/chat/completions or /v1/messages.

Retry and backoff steps

Follow these numbered steps to add retries and backoff to your client:

1. Detect the HTTP status code and treat 429 as retryable.
2. Set a base delay of 1 second and a multiplier of 2 (1s, 2s, 4s).
3. Add jitter, for example a random value between 0 and 1 second, to each delay.
4. Cap retries at a fixed number of attempts (for example 5) to avoid loops.
5. Stop retrying on non retryable errors and surface them to the caller.

This logic sits in your existing SDK. 「保留现有 SDK,改 base_url 与 key 即可,无需改其他代码」据 relayrouter.io/docs 官方文档, so you configure the client once and reuse it. See relayrouter.io/docs for setup details.

Retryable versus non retryable responses

Retry on 429 and transient failures, but do not blindly retry every error, because some indicate a request problem that repeated attempts will not fix. Use the table below as a reference for your handler.

Because failed or errored requests are typically not billed, retries against 429 and 5xx responses do not incur charges for the failed calls. This behavior is consistent across the supported model range (source relayrouter.io/models).

Applying the policy across models and protocols

The same 429 handling works for every model on RelayRouter because the platform exposes OpenAI and Anthropic compatible endpoints. You can point one client at /v1/chat/completions and another at /v1/messages, and reuse the identical backoff of 1s, 2s, 4s with a cap of 5 attempts. Supported models include the Claude line, GPT-5.5, Gemini 3.5, plus DeepSeek, GLM, MiniMax and Moonshot (source relayrouter.io/models). Migration requires only the base_url and key change (source relayrouter.io/docs), so a single retry wrapper covers all of them. Check current per model details at relayrouter.io/models.

FAQ

Do I get charged for a request that returns 429? No: failed or errored requests are typically not billed (source relayrouter.io), so retried 429 responses do not add cost for the failed attempts.

Do I need to change my code to add retries? You keep your existing SDK and change only the base_url and key: 「保留现有 SDK,改 base_url 与 key 即可,无需改其他代码」据 relayrouter.io/docs 官方文档.

Does the same backoff work for both OpenAI and Anthropic style calls? Yes, because RelayRouter is 「同时兼容 OpenAI 与 Anthropic 两种协议」据 relayrouter.io 官方文档, so one retry policy of 1s, 2s, 4s covers /v1/chat/completions and /v1/messages.