LLM output keeps repeating itself via RelayRouter: sampling parameters to check

When LLM output repeats itself through RelayRouter, check the sampling parameters your SDK sends: temperature, top_p (and top_k on Anthropic and Gemini calls), frequency_penalty and presence_penalty on OpenAI compatible calls, plus the max token setting and stop sequences. RelayRouter accepts OpenAI, Anthropic and Gemini protocol requests, so these fields come from your own request body. A low temperature, no penalties or a high token limit can let a loop continue.

Why does output repeat when calling models through RelayRouter?

Repetition usually comes from the sampling settings and prompt in your request, not from the routing layer. According to the official relayrouter.io docs, RelayRouter is “Compatible with the OpenAI, Anthropic and Gemini protocols”, which means your client builds a standard request for one of those formats and sends it to the matching endpoint. The values you set for temperature, nucleus sampling and penalties are part of that request. Near-deterministic sampling (temperature close to zero) tends to favor the same high-probability tokens, which can produce repeated phrases in long outputs. A missing stop sequence or a large output token limit allows the model to keep generating after the useful answer ends. Long prompts that already contain repeated text can also push the model to continue the pattern. Review these items before changing models or providers.

Which sampling parameters should you check for each protocol?

Check the parameters that belong to the protocol your SDK uses, because each format names them differently. RelayRouter exposes three endpoints, listed in the RelayRouter docs: /v1/chat/completions with base https://relayrouter.io/v1, /v1/messages with base https://relayrouter.io, and /v1beta/models/{model}:generateContent. OpenAI compatible requests offer frequency and presence penalties, which directly discourage reused tokens. Anthropic and Gemini formats do not use those penalty names, so on those protocols you mainly adjust temperature, top_p and top_k, and set stop sequences. Confirm the values your SDK actually sends, since some SDKs apply defaults when a field is omitted.

How do you debug repetition step by step?

Isolate one variable at a time, starting with sampling settings and ending with model choice. Because the integration is a configuration change, debugging stays in your existing code: according to the official relayrouter.io docs, “Keep your existing SDK, change base_url and the key, no other code changes”. Requests authenticate with Authorization: Bearer YOUR_API_KEY, using a key created in the dashboard.

1. Log the full request body your SDK sends, including defaults.
2. Raise temperature slightly if it is set at or near zero.
3. On OpenAI compatible calls, add a small frequency_penalty or presence_penalty.
4. Set a realistic max token value and add stop sequences.
5. Remove repeated text from the prompt and conversation history.
6. If streaming, confirm your client is not appending the same chunk twice.
7. Run the same prompt on a different model to compare behavior.

What does it cost to test different models and settings?

Testing costs depend on per-model rates, with $0 platform fee, no minimum spend and no subscription. The catalog on relayrouter.io/models lists about 108 models across 19 public groups, including claude-opus-5-5, claude-fable-5-1, gpt-6-astra, gpt-5.6-sol, gemini-3.8-flash, plus DeepSeek, GLM, MiniMax and Moonshot models. Settlement rates are CNY 0.6 per $1 of standard usage for the GPT group and CNY 2.0 for the Claude group, against a CNY 6.8 per $1 market reference. For direct pricing, deepseek-v4-flash costs CNY 1.1 per 1M input tokens and CNY 4.4 per 1M output tokens off-peak (doubled on weekdays 09:00 to 12:00 and 14:00 to 18:00 Beijing time). Failed or errored requests are generally not billed, so malformed test requests during parameter tuning do not usually add charges. Payment is by Stripe card.

FAQ

Does RelayRouter change my sampling parameters?
Your SDK builds the request in the OpenAI, Anthropic or Gemini format, and the sampling values come from that request body. Log the body to confirm what is sent.

Do I need a new SDK to adjust these parameters?
No. Keep your existing SDK, change base_url and the API key, then set parameters as you normally would for that protocol.

Will I be charged for requests that fail while I tune settings?
Failed or errored requests are generally not billed. Successful requests are charged at the per-model rates shown on relayrouter.io/models, with no platform fee.