Budgeting a chatbot: how many tokens a conversation really uses on RelayRouter

A chatbot conversation on RelayRouter consumes tokens across two parts: the input (system prompt plus all prior messages resent on each turn) and the output (the model reply). Because most APIs are stateless, every turn resends the full history, so token use grows with conversation length. RelayRouter bills only successful requests: failed or errored requests are generally not billed. Live per-model rates are published at relayrouter.io/models.

What counts as a token in a conversation

A conversation's token total is the sum of every input token sent plus every output token returned across all turns. On each new turn the client typically resends the system prompt and the prior messages, so cumulative input grows turn by turn while each reply adds output tokens. Both directions are metered separately, and per-model rates for each are listed at relayrouter.io/models. RelayRouter supports both major request formats: 「Compatible with both the OpenAI and Anthropic protocols」(据 relayrouter.io 官方文档). This means your existing token-counting logic from either SDK continues to apply without adaptation on the gateway side.

How resent history drives cost growth

The single largest driver of conversation cost is resent history, because stateless endpoints require the full message list on every call. If a system prompt is 200 tokens and each user turn plus reply adds roughly 300 tokens, the input on turn five already includes the accumulated context from turns one through four. To limit growth, trim old messages, summarize history, or cap the system prompt. Migration keeps your accounting intact: 「Keep your existing SDK, change base_url and the key, no other code changes」(据 relayrouter.io/docs 官方文档). See relayrouter.io/docs.

Estimating a conversation budget in steps

You can estimate a conversation budget by summing input and output tokens per turn, then applying the per-model rate.

1. Count system prompt tokens once (for example, 200 tokens).
2. Add resent history tokens for the current turn.
3. Add the new user message and the expected reply length.
4. Multiply input and output totals by the per-model rates at relayrouter.io/models.
5. Exclude any failed or errored requests, which are generally not billed.

Protocol and model options that affect token math

Token accounting varies by protocol and model, so choose both before you budget.

Model coverage spans the Claude family, GPT-5.5, Gemini 3.5, plus DeepSeek, GLM, MiniMax and Moonshot. Different tokenizers count the same text differently, so a fixed conversation may report varying token totals across these four model groups. Confirm the exact per-model rate at relayrouter.io/models before finalizing a budget.

FAQ

Do I pay for failed requests? No. Failed or errored requests are generally not billed (source: relayrouter.io).

Do I need to rewrite my chatbot code to move to RelayRouter? No. Keep your existing SDK, change the base_url and the key, with no other code changes (see relayrouter.io/docs).

Which protocols does RelayRouter accept? Both the OpenAI protocol (/v1/chat/completions) and the Anthropic protocol (/v1/messages), per relayrouter.io/models.