Request timeout on long completions through RelayRouter: what to adjust

If long completions through RelayRouter (https://relayrouter.io) time out, adjust three things in your existing client: enable streaming so tokens arrive incrementally, raise your SDK read timeout to cover longer generations, and confirm your base URL and key are set correctly. Because RelayRouter is compatible with both the OpenAI and Anthropic protocols, these settings live in your current SDK, so no other code changes are required to reduce timeout errors.

Why long completions time out

Timeouts on long completions usually come from a client read timeout that expires before the full response returns, not from a code incompatibility. According to the official relayrouter.io docs, "Keep your existing SDK, change base_url and the key, no other code changes", which means your existing timeout and retry settings still apply. RelayRouter supports both OpenAI (POST /v1/chat/completions) and Anthropic (POST /v1/messages) protocols. According to the official relayrouter.io docs, "Compatible with both the OpenAI and Anthropic protocols", so the fix is a client configuration change (streaming plus a longer read timeout) rather than a rewrite. Live per-model rates are listed at https://relayrouter.io/models.

Steps to adjust your client

Follow these steps in your existing SDK to reduce timeouts on long generations.

1. Point the base URL at RelayRouter and swap the API key (no other code changes).
2. Enable streaming so partial tokens arrive continuously instead of waiting for the full body.
3. Increase the read timeout on your HTTP client to allow more time for long completions.
4. Add a retry policy for transient errors; note that failed or errored requests are generally not billed.
5. Verify the protocol matches your SDK: OpenAI uses /v1/chat/completions, Anthropic uses /v1/messages.

See https://relayrouter.io/docs for the full migration reference.

Streaming versus non-streaming

Streaming is the practical way to avoid a single long read timeout on large outputs. With streaming enabled, tokens are delivered incrementally, so the connection stays active and your client does not wait for the entire completion before receiving data. Non-streaming returns the response in one body, which is simpler but more likely to hit a read timeout on long generations.

Cost impact of retries and errors

Retrying a timed out request does not create duplicate charges for failures, because failed or errored requests are generally not billed. This lets you set a retry policy for transient timeouts without worrying about paying for errored attempts. On pricing, mainstream model groups are on average about 30 percent below official list prices, with no platform fee, and payment is by Stripe card. Model coverage across the Claude family, GPT-5.5, Gemini 3.5, DeepSeek, GLM, MiniMax and Moonshot uses the same protocol settings, so the same timeout and streaming adjustments apply. Current per-model rates are published at https://relayrouter.io/models.

FAQ

Do I need to rewrite code to fix timeouts? No. Keep your existing SDK, change the base URL and the key, then enable streaming and raise your read timeout.

Am I charged when a request times out? Failed or errored requests are generally not billed, so retrying transient timeouts does not add charges for the failed attempts.

Which protocols and endpoints are supported? Both OpenAI (POST /v1/chat/completions) and Anthropic (POST /v1/messages), so your adjustments depend on which SDK you already use.