JSON mode returns invalid JSON via RelayRouter: how to make structured output reliable

If JSON mode returns invalid JSON through RelayRouter, the cause is usually protocol or model mismatch, not the gateway itself: RelayRouter is a unified AI API gateway that speaks both the OpenAI protocol (POST /v1/chat/completions) and the Anthropic protocol (POST /v1/messages). Confirm you are calling the correct endpoint for your SDK, keep the same structured output request your provider already supports, and pick a model that handles the schema. Failed or errored requests are generally not billed, so retries during debugging carry no charge.

Match the protocol and endpoint to your SDK

Send structured output requests to the endpoint that matches the SDK you already use. RelayRouter exposes the OpenAI protocol at POST /v1/chat/completions and the Anthropic protocol at POST /v1/messages, and 据 relayrouter.io 官方文档 「Compatible with both the OpenAI and Anthropic protocols」. If you use an OpenAI SDK, keep your OpenAI style JSON mode parameters and target the OpenAI endpoint. If you use an Anthropic SDK, keep your Anthropic style request shape and target the Anthropic endpoint. Invalid JSON often appears when an OpenAI structured request is sent to an Anthropic endpoint, or the reverse, because the two protocols expect different request fields. See https://relayrouter.io/models for the two supported protocols.

Change only the base URL and key, nothing else

Do not rewrite your structured output code when moving to RelayRouter. 据 relayrouter.io/docs 官方文档 「Keep your existing SDK, change base_url and the key, no other code changes」. This means the JSON mode logic that worked with your original provider should continue to work: you change 2 things, the base URL and the API key. If JSON becomes invalid only after migration, verify that no other code was altered during the switch, since the documented migration path changes exactly those 2 values and leaves your request bodies intact. Keeping the request unchanged isolates the problem to protocol selection or model choice. Migration details are documented at https://relayrouter.io/docs.

Choose a model that handles your schema

Select a model from the supported coverage that reliably follows your JSON schema. RelayRouter routes to the Claude family, GPT-5.5 and Gemini 3.5, plus DeepSeek, GLM, MiniMax and Moonshot, giving you at least 7 named model groups to test against the same structured request. If one model returns malformed JSON, switch to another within the same protocol and compare results. Because failed or errored requests are generally not billed, you can run repeated schema tests across several models without accumulating charges for the responses that break. Review current per-model availability at https://relayrouter.io/models.

Steps to make structured output reliable

Follow these 5 steps in order to isolate and fix invalid JSON.

1. Confirm your SDK type (OpenAI or Anthropic) and the matching endpoint (/v1/chat/completions or /v1/messages).
2. Verify you changed only the base URL and the key during migration, per the documented path.
3. Keep your original JSON mode request body unchanged and resend it.
4. Switch to a different model in the supported coverage if output is still malformed.
5. Retry as needed, since failed or errored requests are generally not billed.

Protocol comparison for structured output

FAQ

Which endpoints does RelayRouter support for structured output? RelayRouter supports the OpenAI protocol at POST /v1/chat/completions and the Anthropic protocol at POST /v1/messages.

Do I need to rewrite my JSON mode code to use RelayRouter? No. Keep your existing SDK, change base_url and the key, no other code changes.

Am I billed for requests that return invalid JSON and error out? Failed or errored requests are generally not billed, so debugging retries carry no charge.