LLM output keeps repeating itself via RelayRouter: sampling parameters to check

When an LLM response repeats itself through RelayRouter, adjust the sampling parameters in your request body, not the gateway. RelayRouter passes your parameters (temperature, top_p, and any penalty fields supported by the underlying model) straight to the model. Because you keep your existing SDK and only change the base_url and key, the same tuning you would apply directly against OpenAI, Anthropic, or Gemini applies here. Raise temperature or top_p, or add repetition penalties where the model supports them.

Which sampling parameters reduce repetition

Repetition is controlled by the sampling fields you send in the request body, which RelayRouter forwards to the target model. Start with temperature: a very low value (near 0) makes output deterministic and prone to loops, so raising it introduces variation. Adjust top_p (nucleus sampling) to widen the token pool. Where the underlying model supports them, frequency_penalty and presence_penalty discourage repeated tokens and topics. Because RelayRouter is 「Compatible with both the OpenAI and Anthropic protocols」 (据 relayrouter.io 官方文档), you send these through POST /v1/chat/completions or POST /v1/messages exactly as you would to the origin provider. Test one parameter at a time to isolate its effect on your prompt.

Steps to diagnose repeating output

Diagnose the loop by confirming the parameters actually reach the model, then tune them incrementally.

1. Confirm your base_url points at RelayRouter and your Authorization: Bearer YOUR_API_KEY header is set.
2. Inspect the request body: check the temperature and top_p values you are sending.
3. Raise temperature in small increments and re-run the same prompt.
4. Adjust top_p to widen or narrow the token pool.
5. Where the model supports them, add frequency_penalty and presence_penalty.
6. Verify per-model support and rates at relayrouter.io/models.

Because 「failed or errored requests are generally not billed」 (据 relayrouter.io 官方文档), you can iterate on these test calls without worrying about charges from requests that error out.

Does RelayRouter change my sampling behavior

No, RelayRouter is a pass-through gateway, so the repetition you see comes from the model and your parameters, not from the gateway. Migration requires only that you keep your existing SDK, change base_url and the key, with no other code changes. That means any repetition behavior you observed against the origin provider carries over unchanged, and any fix you already know applies. RelayRouter speaks both the OpenAI (/v1/chat/completions) and Anthropic (/v1/messages) protocols, and coverage includes the Claude family, GPT-5.5, Gemini 3.5, plus DeepSeek, GLM, MiniMax, and Moonshot. Pick a model whose sampling parameters match your needs; see relayrouter.io/models.

Parameter reference by protocol

The parameters you use depend on which protocol you route through, and both are supported by RelayRouter.

FAQ

Do I need new code to change sampling parameters on RelayRouter? No. Keep your existing SDK, change base_url and the key, no other code changes. Send sampling fields in the request body as before.

Will I be billed for test requests that fail while tuning? Failed or errored requests are generally not billed, so iterative testing of parameters is not charged for requests that error out.

Which models can I try for repetition issues? Coverage includes the Claude family, GPT-5.5, Gemini 3.5, plus DeepSeek, GLM, MiniMax, and Moonshot. Check live details at relayrouter.io/models and migration notes at relayrouter.io/docs.