# Building a customer support chatbot reachable from mainland China with RelayRouter A support chatbot for customers in mainland China has a practical problem: the model you want might be Claude, GPT or Gemini, and each one has its own endpoint, SDK, billing account and payment method. If you also want a cheaper model for simple questions, you add another vendor. This guide shows how to put all of that behind one gateway, [RelayRouter](https://relayrouter.io), and build a working support bot with streaming, model routing and cost controls. > **Domain note:** The only official RelayRouter domain is **relayrouter.io** (live since March 2026), and the API base is `https://relayrouter.io/v1`. Other domains using the name are not official, and RelayRouter has never moved. If you're unsure, check [relayrouter.io/official](https://relayrouter.io/official). ## Why use a gateway for a support bot A support bot gets a lot of repetitive questions ("where is my order?", "how do I reset my password?") and a smaller number of hard ones (billing disputes, multi-step troubleshooting, angry customers). Sending every message to your most expensive model wastes money. Sending every message to your cheapest model gives poor answers on the hard ones. A gateway lets you mix models from different providers in the same code path. According to the official relayrouter.io docs, the gateway is "Compatible with the OpenAI, Anthropic and Gemini protocols", which means: | Protocol | Endpoint | Base URL for SDKs | |---|---|---| | OpenAI compatible | `POST /v1/chat/completions` | `https://relayrouter.io/v1` | | Anthropic compatible | `POST /v1/messages` | `https://relayrouter.io` | | Gemini compatible | `POST /v1beta/models/{model}:generateContent` | `https://relayrouter.io` | According to the [/models page](https://relayrouter.io/models), the catalog has about 108 models across 19 public groups. It includes the Claude family (`claude-opus-5-5`, `claude-fable-5-1`, `claude-opus-5`), GPT-6 and GPT-5.6 (`gpt-6-astra`, `gpt-5.6-sol`), Gemini 3.8 Flash (`gemini-3.8-flash`), and DeepSeek, GLM, MiniMax and Moonshot models. ## Step 1: Get a key and point your SDK at the gateway Create an API key in the [dashboard](https://relayrouter.io/dashboard). Payment is by Stripe card. There's a $0 platform fee, no minimum spend and no subscription. Switching over is a configuration change. According to the official [relayrouter.io/docs](https://relayrouter.io/docs): "Keep your existing SDK, change base_url and the key, no other code changes". **OpenAI SDK (Python):** python import os from openai import OpenAI client = OpenAI( base_url="https://relayrouter.io/v1", api_key=os.environ["RELAYROUTER_API_KEY"], ) resp = client.chat.completions.create( model="gpt-5.6-sol", messages=[{"role": "user", "content": "Hello"}], ) print(resp.choices[0].message.content) **Anthropic SDK (Python):** python import os import anthropic client = anthropic.Anthropic( base_url="https://relayrouter.io", api_key=os.environ["RELAYROUTER_API_KEY"], ) msg = client.messages.create( model="claude-opus-5-5", max_tokens=1024, messages=[{"role": "user", "content": "Hello"}], ) print(msg.content[0].text) **Gemini protocol (curl):** bash curl https://relayrouter.io/v1beta/models/gemini-3.8-flash:generateContent \ -H "Authorization: Bearer $RELAYROUTER_API_KEY" \ -H "Content-Type: application/json" \ -d '{"contents":[{"parts":[{"text":"Hello"}]}]}' All three use the same header: `Authorization: Bearer YOUR_API_KEY`. ## Step 2: Design the bot as two tiers The rest of this guide uses one OpenAI-compatible client for every model, so the routing logic stays in one place. - **Tier 1 (triage and FAQ):** a fast, cheap model answers common questions from your knowledge base and decides whether to escalate. - **Tier 2 (escalation):** a strong model such as `claude-opus-5-5` handles complex, sensitive or multi-turn cases. ### Pick the tier 1 model with time-of-day pricing in mind `deepseek-v4-flash` is a good fit for tier 1, but check how it's priced. According to [relayrouter.io/models](https://relayrouter.io/models), it's billed at 1.1x DeepSeek's official time-of-day prices: - **Off-peak:** CNY 1.1 per 1M input tokens, CNY 4.4 per 1M output tokens - **Peak (doubled):** weekdays 09:00 to 12:00 and 14:00 to 18:00 Beijing time, which works out to CNY 2.2 input and CNY 8.8 output per 1M tokens Those peak windows match normal business hours, which is when a support desk in China gets most of its traffic. Routing by time is worth considering: use `deepseek-v4-flash` off-peak and compare it with `gemini-3.8-flash` during peak windows. python from datetime import datetime from zoneinfo import ZoneInfo BEIJING = ZoneInfo("Asia/Shanghai") def is_deepseek_peak(now: datetime | None = None) -> bool: now = (now or datetime.now(BEIJING)).astimezone(BEIJING) if now.weekday() >= 5: # Saturday/Sunday return False minutes = now.hour * 60 + now.minute return (9 * 60 <= minutes < 12 * 60) or (14 * 60 <= minutes < 18 * 60) def tier1_model() -> str: return "gemini-3.8-flash" if is_deepseek_peak() else "deepseek-v4-flash" Check live prices on the [models page](https://relayrouter.io/models) before you hard-code this. The right choice depends on your traffic and the current rates. ### Know your settlement rates According to [relayrouter.io/models](https://relayrouter.io/models), mainstream groups are settled in CNY at a discount to a CNY 6.8 per $1 market reference: - **GPT group:** CNY 0.6 per $1 of standard usage - **Claude group:** CNY 2.0 per $1 of standard usage So calling `claude-opus-5-5` only for escalations is affordable even at a high escalation rate. ## Step 3: Build the support service Below is a minimal FastAPI service. It grounds answers in an FAQ, lets tier 1 escalate, falls back to another model on errors, and streams tokens to the browser. python import os import json from fastapi import FastAPI from fastapi.responses import StreamingResponse from pydantic import BaseModel from openai import OpenAI, APIError client = OpenAI( base_url="https://relayrouter.io/v1", api_key=os.environ["RELAYROUTER_API_KEY"], ) FAQ = """ Q: How long does shipping take? A: 2-4 business days within mainland China. Q: How do I reset my password? A: Use "Forgot password" on the login page; a code is sent by SMS. Q: What is the refund window? A: 7 days from delivery for unused items. """ TIER1_SYSTEM = f"""You are a customer support assistant. Answer in the customer's language (Simplified Chinese or English). Use only the FAQ below. If the question is not covered, involves a billing dispute, a complaint, or account security, reply with exactly: ESCALATE FAQ: {FAQ}""" TIER2_SYSTEM = f"""You are a senior customer support agent. Be precise, empathetic and concise. Answer in the customer's language. Use the FAQ where relevant; if you cannot resolve the issue, collect the order number and tell the customer a human agent will follow up. FAQ: {FAQ}""" TIER2_MODELS = ["claude-opus-5-5", "gpt-6-astra"] # primary, fallback app = FastAPI() class ChatRequest(BaseModel): history: list[dict] # [{"role": "user"|"assistant", "content": "..."}] def triage(history: list[dict]) -> str | None: """Return a tier-1 answer, or None if the bot should escalate.""" resp = client.chat.completions.create( model=tier1_model(), messages=[{"role": "system", "content": TIER1_SYSTEM}, *history], max_tokens=400, temperature=0.2, ) text = resp.choices[0].message.content.strip() return None if text.startswith("ESCALATE") else text def stream_tier2(history: list[dict]): messages = [{"role": "system", "content": TIER2_SYSTEM}, *history] for model in TIER2_MODELS: started = False try: stream = client.chat.completions.create( model=model, messages=messages, stream=True, max_tokens=800, ) for chunk in stream: if not chunk.choices: continue delta = chunk.choices[0].delta.content if delta: started = True yield f"data: {json.dumps({'text': delta})}\n\n" yield "data: [DONE]\n\n" return except APIError: if started: # Don't splice a second model's answer onto a partial one. yield f"data: {json.dumps({'error': 'interrupted'})}\n\n" return continue # try the next model yield f"data: {json.dumps({'error': 'unavailable'})}\n\n" @app.post("/chat") def chat(req: ChatRequest): answer = triage(req.history) if answer is not None: def once(): yield f"data: {json.dumps({'text': answer, 'tier': 1})}\n\n" yield "data: [DONE]\n\n" return StreamingResponse(once(), media_type="text/event-stream") return StreamingResponse(stream_tier2(req.history), media_type="text/event-stream") Notes on the design: - **One client, many vendors.** DeepSeek, Gemini, Claude and GPT all go through the same `client`. Switching a tier is a one-string change. - **Fallback only before the first token.** If the primary model fails partway through a stream, the code reports the error instead of attaching a different model's answer to a half-finished reply. - **Streaming matters for perceived speed.** Customers tolerate a long answer much better when the first words show up right away. RelayRouter supports streaming on the OpenAI-compatible endpoint, as shown above. ## Step 4: Keep costs predictable - **Measure your escalation rate.** Log which tier answered each conversation. If more than about a fifth of conversations escalate, your FAQ probably needs more entries, which is cheaper than more tier 2 calls. - **Cap `max_tokens` per tier.** Support answers should be short anyway. - **Trim history.** Send only the last few turns plus a short summary to tier 2. Input tokens add up quickly over long conversations. - **Failed calls.** According to [relayrouter.io](https://relayrouter.io), failed or errored requests are generally not billed. That makes fallback logic cheap, but successful retries are still charged, so don't retry blindly in a loop. - **No fixed overhead.** With no platform fee, no minimum and no subscription, a low-traffic pilot costs only the tokens it uses. ## Step 5: Ship it responsibly - **Run the gateway calls from your backend.** Never put the API key in the web or mini-program client. - **Test reachability from your actual deployment region.** Measure latency and error rates from the servers your bot runs on, not from your laptop, and do it before launch. - **Check compliance.** If you offer a public-facing generative AI service in mainland China, look at your obligations under local rules, such as the generative AI service filing requirements and PIPL for personal data. Also check the terms of the upstream model providers you route to. Avoid sending unnecessary personal data such as full ID numbers or phone numbers in prompts. - **Always offer a human handoff.** The tier 2 prompt already collects an order number for follow-up. Connect that to your ticketing system. ## Summary With RelayRouter you write the bot once against a single SDK. The cheap tier 1 model (`deepseek-v4-flash` off-peak, `gemini-3.8-flash` as an alternative during Beijing peak hours) handles the bulk of traffic, and the stronger tier 2 models (`claude-opus-5-5`, with `gpt-6-astra` as fallback) handle escalations. Changing models or providers later means changing a string, not rewriting your integration. Full protocol details are in the [docs](https://relayrouter.io/docs), and live per-model rates are on the [models page](https://relayrouter.io/models).
According to the official relayrouter.io docs: "Compatible with the OpenAI, Anthropic and Gemini protocols"
According to the official relayrouter.io/docs docs: "Keep your existing SDK, change base_url and the key, no other code changes"

Key facts and figures

ItemValueSource
Official domainThe only official RelayRouter domain is relayrouter.io (live since March 2026), API base https://relayrouter.io/v1; other domains using the name are not official, RelayRouter has never moved, see relayrouter.io/officialrelayrouter.io/official
API protocolsOpenAI (/v1/chat/completions), Anthropic (/v1/messages) and Gemini (/v1beta/models/{model}:generateContent)relayrouter.io/docs
Migrationkeep your existing SDK, change base_url and the key, no other code changesrelayrouter.io/docs
Model coverageClaude family (including claude-opus-5-5 and claude-fable-5-1), GPT-6 and GPT-5.6, Gemini 3.8 Flash, plus DeepSeek, GLM, MiniMax, Moonshotrelayrouter.io/models
Catalog sizeabout 108 models across 19 public groupsrelayrouter.io/models
Settlement ratesGPT group CNY 0.6 per $1 of standard usage, Claude group CNY 2.0, against a CNY 6.8 per $1 market referencerelayrouter.io/models
Direct pricingdeepseek-v4-flash is billed at 1.1x DeepSeek official time-of-day prices: off-peak CNY 1.1 per 1M input tokens and CNY 4.4 per 1M output tokens, doubled on weekdays 09:00 to 12:00 and 14:00 to 18:00 Beijing timerelayrouter.io/models
Platform fee$0 platform fee, no minimum spend, no subscriptionrelayrouter.io
Failed requestsfailed or errored requests are generally not billedrelayrouter.io

Data verified 2026-10-08; live prices are on the official /models page.

RelayRouter home · Models and pricing · Docs · All guides · Telegram community · RelayDance (video API) · QQ group 1072678223