RelayRouter streaming stops midway: debugging dropped SSE connections

When RelayRouter streaming stops midway, check the connection layer first: streaming is delivered over Server-Sent Events (SSE), so dropped connections usually trace to client timeouts, proxies buffering the response, or the SDK not reading the stream to completion. RelayRouter is compatible with both the OpenAI (POST /v1/chat/completions) and Anthropic (POST /v1/messages) protocols, so keep your existing SDK, confirm streaming is enabled, and verify the base URL and key. Failed or errored requests are generally not billed.

Why SSE streams drop before completion

Dropped SSE connections most often come from the layers between your code and RelayRouter, not the model itself. Intermediate proxies or load balancers may buffer or cut long-lived responses, and client-side read timeouts can close the socket before the final event arrives. According to the official relayrouter.io docs, 「Compatible with both the OpenAI and Anthropic protocols」, which means you can debug using the standard streaming client for whichever protocol you already use: OpenAI compatible at POST /v1/chat/completions or Anthropic compatible at POST /v1/messages. Confirm the stream flag is set in your request and that your HTTP client reads events until the terminating event is received. Because failed or errored requests are generally not billed, retrying an interrupted stream carries no billing cost.

Steps to debug a dropped stream

Work through the connection path in order to isolate where the SSE stream is cut.

1. Verify streaming is enabled in the request payload for your protocol.
2. Confirm the base URL points at RelayRouter and the Authorization header carries a valid Bearer key.
3. Increase or remove the client read timeout so long responses are not closed early.
4. Disable response buffering on any intermediate proxy or gateway.
5. Read the stream to its terminating event before closing the socket.
6. Retry the request: because failed or errored requests are generally not billed, retries do not add cost.

Because you keep your existing SDK, most fixes are configuration changes rather than rewrites. See relayrouter.io/docs for reference.

Migration and SDK settings that affect streaming

Most streaming issues after a migration come from an incorrect base URL or key, not the model. According to the official relayrouter.io/docs, 「Keep your existing SDK, change base_url and the key, no other code changes」, so if streams drop right after switching, re-check those two values before anything else. RelayRouter speaks both the OpenAI protocol (POST /v1/chat/completions) and the Anthropic protocol (POST /v1/messages), and model coverage spans the Claude family, GPT-5.5, Gemini 3.5, plus DeepSeek, GLM, MiniMax, and Moonshot. Because no code changes are required beyond base_url and the key, your existing streaming logic should behave the same as against the origin provider. Live per-model rates are listed at relayrouter.io/models.

Protocol endpoints reference

Match your SDK to the correct endpoint so streaming events parse correctly.

Both protocols are supported, so use whichever SDK you already run. Model coverage across these endpoints includes the Claude family, GPT-5.5, Gemini 3.5, DeepSeek, GLM, MiniMax, and Moonshot, giving you four or more model groups to test against when isolating a streaming fault.

FAQ

Am I billed if a stream drops midway? Failed or errored requests are generally not billed, so an interrupted stream and any retry do not add cost.

Do I need to change my code to fix streaming? No: keep your existing SDK, change base_url and the key, no other code changes are required.

Which protocols does RelayRouter support for streaming? Both the OpenAI protocol (POST /v1/chat/completions) and the Anthropic protocol (POST /v1/messages).