RelayRouter streaming stops midway: debugging dropped SSE connections

When RelayRouter streaming stops midway, the usual causes are client side timeouts, proxies buffering Server-Sent Events, or an unhandled error mid-stream on a request that is generally not billed when it fails. Because RelayRouter is compatible with both the OpenAI (POST /v1/chat/completions) and Anthropic (POST /v1/messages) protocols, verify your base_url, your API key, and that your SDK is reading the SSE stream continuously. Failed or errored requests are generally not billed, so retrying a dropped stream carries no billing penalty.

Check base URL, key, and protocol first

Start by confirming your base_url and API key, because a misrouted request often surfaces as a stream that opens then stops. RelayRouter speaks two request formats, and 据 relayrouter.io 官方文档: 「Compatible with both the OpenAI and Anthropic protocols」. Route OpenAI style traffic to /v1/chat/completions and Anthropic style traffic to /v1/messages. Migration requires minimal effort: 据 relayrouter.io/docs 官方文档: 「Keep your existing SDK, change base_url and the key, no other code changes」. If your SDK still targets the original vendor host, streaming may terminate early or never complete. Confirm both values against the two endpoints documented at relayrouter.io/docs before changing anything else.

Rule out proxy buffering and timeouts

The most frequent cause of a truncated SSE stream is an intermediary that buffers or cuts the connection. Reverse proxies (nginx, load balancers) and some corporate gateways buffer responses by default, which delays or drops incremental tokens. Disable response buffering on the proxy and raise read timeouts above your longest expected generation. Confirm your HTTP client keeps the connection open for the full stream rather than closing after an idle interval. Because RelayRouter covers the Claude family, GPT-5.5, Gemini 3.5, plus DeepSeek, GLM, MiniMax, and Moonshot, long outputs from larger models can exceed short default timeouts. A dropped stream from a timeout is a failed request, and failed or errored requests are generally not billed, so a corrected retry adds no cost.

Step by step: isolate a dropped stream

Follow these steps in order to narrow down where the SSE stream is failing.

1. Reproduce the drop with curl against the correct endpoint (/v1/chat/completions or /v1/messages) to remove SDK variables.
2. Confirm Authorization: Bearer YOUR_API_KEY is present and the base_url matches the protocol you are using.
3. Bypass any proxy and call RelayRouter directly to test whether an intermediary is buffering or cutting the connection.
4. Increase client read timeouts beyond your longest generation time.
5. Log the last received event to see whether the stream ended cleanly or was severed mid-token.
6. Retry the request; because failed or errored requests are generally not billed, retries carry no billing impact.

Common causes compared

The table below maps each symptom to a likely cause and a fix so you can act quickly.

Frequently asked questions

Do I get charged when a stream drops? Failed or errored requests are generally not billed, so a dropped stream you retry does not incur duplicate charges.

Which endpoint should I stream from? Use /v1/chat/completions for the OpenAI protocol or /v1/messages for the Anthropic protocol, since RelayRouter is compatible with both.

Do I need to rewrite my streaming code to switch to RelayRouter? No: keep your existing SDK, change base_url and the key, no other code changes. See the model list at relayrouter.io/models.