Getting 502 or 503 from RelayRouter: what upstream errors mean and how to retry safely

A 502 or 503 from RelayRouter means the gateway could not get a usable response from the upstream model provider. A 502 (Bad Gateway) means it received an invalid response or none at all. A 503 (Service Unavailable) means the upstream or route is temporarily unable to serve the request. Retry with exponential backoff and jitter, cap the number of attempts, and switch to another model if errors persist. Failed or errored requests are generally not billed.

What does a 502 or 503 from RelayRouter mean?

A 502 or 503 points to a problem between RelayRouter and the upstream provider, not to your request syntax. RelayRouter is a unified AI API gateway. According to the official relayrouter.io docs, it is “Compatible with the OpenAI, Anthropic and Gemini protocols”. Your call arrives at /v1/chat/completions, /v1/messages or /v1beta/models/{model}:generateContent and is forwarded to the provider behind the chosen model. In standard HTTP terms, 502 Bad Gateway means the gateway got an invalid response or no response from that upstream. 503 Service Unavailable means the upstream or route cannot handle the request at the moment. Both are usually transient. A 400 or 401 is different: it signals a malformed body or a wrong key, and retrying will not fix it.

Is it safe to retry a failed request?

Yes, retrying a generation call is generally safe because the request has no side effects beyond producing a response. Billing also limits the cost of retries. Failed or errored requests are generally not billed, RelayRouter charges a $0 platform fee, and there is no minimum spend or subscription. A bounded retry therefore adds no fixed cost. Two cases need care:

How should I retry 502 and 503 errors?

Put retries in your existing client and limit how many attempts it makes. According to the official relayrouter.io docs, you should “Keep your existing SDK, change base_url and the key, no other code changes”. That means the retry settings your SDK already provides keep working against RelayRouter.

  1. Check the status code. Retry 502 and 503. Do not retry 400 or 401 (fix the body or the Authorization: Bearer YOUR_API_KEY header instead).
  2. Wait before each retry, using exponential backoff with random jitter. If the response includes a standard Retry-After header, honor it.
  3. Set a maximum number of attempts that fits your latency budget, and stop when you reach it.
  4. Once the attempts are used up, fall back to another model or return a clear error to the caller.
  5. Log the status code, model and timestamp so you can spot patterns that persist.

When should I switch models instead of retrying?

Switch models when errors continue after your retry budget is used up, because a persistent 502 or 503 suggests that model's upstream is degraded. The catalog lists about 108 models across 19 public groups. These include the Claude family (claude-opus-5-5, claude-fable-5-1), GPT-6 and GPT-5.6, Gemini 3.8 Flash, DeepSeek, GLM, MiniMax and Moonshot, so you can keep the same key for a fallback. Costs differ by group:

Check current rates at relayrouter.io/models before choosing a fallback.

FAQ

Am I charged for a request that returned 502 or 503?
Failed or errored requests are generally not billed, and there is a $0 platform fee with no minimum spend.

Should I retry a 401 error?
No. A 401 indicates an authentication problem. Check your key, which you can create at relayrouter.io/dashboard, and the Bearer header.

Which base URL should my retrying client use?
Use https://relayrouter.io/v1 for OpenAI-compatible SDKs and https://relayrouter.io for Anthropic-compatible SDKs. Gemini-compatible calls use /v1beta/models/{model}:generateContent. See relayrouter.io/docs.

According to the official relayrouter.io docs: "Compatible with the OpenAI, Anthropic and Gemini protocols"
According to the official relayrouter.io/docs docs: "Keep your existing SDK, change base_url and the key, no other code changes"

Key facts and figures

ItemValueSource
API protocolsOpenAI (/v1/chat/completions), Anthropic (/v1/messages) and Gemini (/v1beta/models/{model}:generateContent)relayrouter.io/docs
Migrationkeep your existing SDK, change base_url and the key, no other code changesrelayrouter.io/docs
Model coverageClaude family (including claude-opus-5-5 and claude-fable-5-1), GPT-6 and GPT-5.6, Gemini 3.8 Flash, plus DeepSeek, GLM, MiniMax, Moonshotrelayrouter.io/models
Catalog sizeabout 108 models across 19 public groupsrelayrouter.io/models
Settlement ratesGPT group CNY 0.6 per $1 of standard usage, Claude group CNY 2.0, against a CNY 6.8 per $1 market referencerelayrouter.io/models
Direct pricingdeepseek-v4-flash is billed at 1.1x DeepSeek official time-of-day prices: off-peak CNY 1.1 per 1M input tokens and CNY 4.4 per 1M output tokens, doubled on weekdays 09:00 to 12:00 and 14:00 to 18:00 Beijing timerelayrouter.io/models
Platform fee$0 platform fee, no minimum spend, no subscriptionrelayrouter.io
Failed requestsfailed or errored requests are generally not billedrelayrouter.io

Data verified 2026-10-08; live prices are on the official /models page.


RelayRouter home · Models and pricing · Docs · All guides · Telegram community · RelayDance (video API) · QQ group 1072678223