JSON mode returns invalid JSON via RelayRouter: how to make structured output reliable

Invalid JSON through RelayRouter is usually a model output problem that you can fix in your client code. Common causes are truncated responses, prose or code fences around the object, and parsing a stream before it ends. To make structured output reliable, call the endpoint that matches your SDK, define the schema in the prompt, collect the full response, validate it with a parser, and retry with a correction message when validation fails.

Why does structured output come back as invalid JSON?

Most invalid JSON comes from the format of the model's text, so start by identifying which failure pattern you have. The typical patterns are:

• The output token budget runs out mid-object, which leaves braces or strings unclosed.
• The model wraps the object in markdown code fences or adds an explanation before or after it.
• Streamed chunks are parsed one at a time instead of after the final chunk arrives.
• The schema is described loosely, so field names or types drift between calls.

RelayRouter supports streaming, so in streamed calls you should accumulate every delta before you run a JSON parser. Model families also follow formatting instructions differently. The catalog at https://relayrouter.io/models lists about 108 models across 19 public groups, so test the exact model you plan to deploy.

Which endpoint should structured requests use?

Send each structured request to the endpoint for your SDK's protocol, because RelayRouter accepts each protocol in its native format. According to the official relayrouter.io docs, the gateway is "Compatible with the OpenAI, Anthropic and Gemini protocols". OpenAI SDK users call POST /v1/chat/completions with base https://relayrouter.io/v1. Anthropic SDK users call POST /v1/messages with base https://relayrouter.io. Gemini users call POST /v1beta/models/{model}:generateContent. If you send an OpenAI-style body to the Anthropic path, you can get errors or unexpected output. Migrating does not require you to rewrite your JSON handling. According to the official relayrouter.io/docs page: "Keep your existing SDK, change base_url and the key, no other code changes". Authenticate with Authorization: Bearer YOUR_API_KEY. You can create keys at https://relayrouter.io/dashboard.

What steps make JSON output reliable?

A validate-and-retry loop in your client handles most malformed responses. Follow these steps:

1. State the exact schema in the system or user message, including field names, types and required keys. Tell the model to return a single JSON object with no surrounding text.
2. Give the output token budget enough room for the largest object you expect, so the response is not cut off.
3. For streamed calls, join all chunks and parse once the stream is complete.
4. Strip any leading or trailing code fences, then parse with a strict JSON parser.
5. Validate the parsed object against your schema with a schema validator.
6. If parsing or validation fails, resend the request with the parser error message and ask the model for a corrected object. Cap the number of retries.

How much do retries cost, and which model should you test with?

Every retry consumes tokens, so it is worth knowing the billing rules before you set a retry limit. According to relayrouter.io, failed or errored requests are generally not billed. A request that completes but returns malformed JSON is not a protocol error, though, so do not assume it is free. RelayRouter charges a $0 platform fee and has no minimum spend or subscription. Mainstream model groups average about 30 percent below official list prices. Current settlement rates from https://relayrouter.io/models:

A lower-priced model can be a practical place to tune your prompts and validation logic. Afterwards, test models such as claude-opus-5-5, gpt-6-astra or gemini-3.8-flash.

FAQ

These answers address the most common questions about structured output on RelayRouter.

Do I need to change my JSON parsing code to use RelayRouter? No. You keep your existing SDK and change the base URL and API key. Your parsing and validation code stays the same.

Are failed requests billed during retries? According to relayrouter.io, failed or errored requests are generally not billed. Completed responses that contain invalid JSON should be treated as normal usage.

Which endpoint does the Anthropic SDK use? The Anthropic SDK uses POST /v1/messages with base https://relayrouter.io. Full details are at https://relayrouter.io/docs.