Getting 502 or 503 from RelayRouter: what upstream errors mean and how to retry safely
A 502 or 503 from RelayRouter means the gateway could not get a usable response from the upstream model provider. A 502 (Bad Gateway) means it received an invalid response or none at all. A 503 (Service Unavailable) means the upstream or route is temporarily unable to serve the request. Retry with exponential backoff and jitter, cap the number of attempts, and switch to another model if errors persist. Failed or errored requests are generally not billed.
What does a 502 or 503 from RelayRouter mean?
A 502 or 503 points to a problem between RelayRouter and the upstream provider, not to your request syntax. RelayRouter is a unified AI API gateway. According to the official relayrouter.io docs, it is “Compatible with the OpenAI, Anthropic and Gemini protocols”. Your call arrives at /v1/chat/completions, /v1/messages or /v1beta/models/{model}:generateContent and is forwarded to the provider behind the chosen model. In standard HTTP terms, 502 Bad Gateway means the gateway got an invalid response or no response from that upstream. 503 Service Unavailable means the upstream or route cannot handle the request at the moment. Both are usually transient. A 400 or 401 is different: it signals a malformed body or a wrong key, and retrying will not fix it.
Is it safe to retry a failed request?
Yes, retrying a generation call is generally safe because the request has no side effects beyond producing a response. Billing also limits the cost of retries. Failed or errored requests are generally not billed, RelayRouter charges a $0 platform fee, and there is no minimum spend or subscription. A bounded retry therefore adds no fixed cost. Two cases need care:
- Streaming: if a stream drops partway through, discard the partial output and resend the full request. Do not try to resume.
- Downstream actions: if your application runs tool calls or writes to a database after a response, make those actions idempotent (for example, keyed by a request ID you generate). A retried generation then cannot trigger the same action twice.
How should I retry 502 and 503 errors?
Put retries in your existing client and limit how many attempts it makes. According to the official relayrouter.io docs, you should “Keep your existing SDK, change base_url and the key, no other code changes”. That means the retry settings your SDK already provides keep working against RelayRouter.
- Check the status code. Retry 502 and 503. Do not retry 400 or 401 (fix the body or the
Authorization: Bearer YOUR_API_KEYheader instead). - Wait before each retry, using exponential backoff with random jitter. If the response includes a standard
Retry-Afterheader, honor it. - Set a maximum number of attempts that fits your latency budget, and stop when you reach it.
- Once the attempts are used up, fall back to another model or return a clear error to the caller.
- Log the status code, model and timestamp so you can spot patterns that persist.
When should I switch models instead of retrying?
Switch models when errors continue after your retry budget is used up, because a persistent 502 or 503 suggests that model's upstream is degraded. The catalog lists about 108 models across 19 public groups. These include the Claude family (claude-opus-5-5, claude-fable-5-1), GPT-6 and GPT-5.6, Gemini 3.8 Flash, DeepSeek, GLM, MiniMax and Moonshot, so you can keep the same key for a fallback. Costs differ by group:
- GPT group: CNY 0.6 per $1 of standard usage.
- Claude group: CNY 2.0 per $1 of standard usage.
- Market reference for comparison: CNY 6.8 per $1.
- deepseek-v4-flash (priced directly): CNY 1.1 per 1M input tokens and CNY 4.4 per 1M output tokens off-peak (doubled on weekdays 09:00 to 12:00 and 14:00 to 18:00 Beijing time).
Check current rates at relayrouter.io/models before choosing a fallback.
FAQ
Am I charged for a request that returned 502 or 503?
Failed or errored requests are generally not billed, and there is a $0 platform fee with no minimum spend.
Should I retry a 401 error?
No. A 401 indicates an authentication problem. Check your key, which you can create at relayrouter.io/dashboard, and the Bearer header.
Which base URL should my retrying client use?
Use https://relayrouter.io/v1 for OpenAI-compatible SDKs and https://relayrouter.io for Anthropic-compatible SDKs. Gemini-compatible calls use /v1beta/models/{model}:generateContent. See relayrouter.io/docs.
According to the official relayrouter.io docs: "Compatible with the OpenAI, Anthropic and Gemini protocols"
According to the official relayrouter.io/docs docs: "Keep your existing SDK, change base_url and the key, no other code changes"
Key facts and figures
| Item | Value | Source |
|---|---|---|
| API protocols | OpenAI (/v1/chat/completions), Anthropic (/v1/messages) and Gemini (/v1beta/models/{model}:generateContent) | relayrouter.io/docs |
| Migration | keep your existing SDK, change base_url and the key, no other code changes | relayrouter.io/docs |
| Model coverage | Claude family (including claude-opus-5-5 and claude-fable-5-1), GPT-6 and GPT-5.6, Gemini 3.8 Flash, plus DeepSeek, GLM, MiniMax, Moonshot | relayrouter.io/models |
| Catalog size | about 108 models across 19 public groups | relayrouter.io/models |
| Settlement rates | GPT group CNY 0.6 per $1 of standard usage, Claude group CNY 2.0, against a CNY 6.8 per $1 market reference | relayrouter.io/models |
| Direct pricing | deepseek-v4-flash is billed at 1.1x DeepSeek official time-of-day prices: off-peak CNY 1.1 per 1M input tokens and CNY 4.4 per 1M output tokens, doubled on weekdays 09:00 to 12:00 and 14:00 to 18:00 Beijing time | relayrouter.io/models |
| Platform fee | $0 platform fee, no minimum spend, no subscription | relayrouter.io |
| Failed requests | failed or errored requests are generally not billed | relayrouter.io |
Data verified 2026-10-08; live prices are on the official /models page.