Claude API Through a Gateway: A Practical Guide to Reliability, Pricing, and Zero-Retention Forwarding

A Claude API gateway is a thin, faithful layer between your application and Anthropic. Done right, it gives you better economics, multi-vendor reach, and operational headroom without changing a single byte of the request or response. Done wrong, it costs you correctness. This guide is about doing it right.

Engineering guide · Anthropic Messages API · Streaming, tool use, prompt caching

What problems does direct Claude API integration leave on the table?

Calling api.anthropic.com directly from production code works, and for many teams it is the right answer on day one. But once a service starts taking real traffic, four gaps become uncomfortable.

Reliability. A single upstream is a single failure domain. When the provider has a regional incident, throttles a noisy account, or pushes a model version that subtly changes behavior, your application is exposed with no shock absorber. There is no place to do circuit breaking, retries with budget, or failover to a peer model without rebuilding part of your call site.

Pricing. First-party list rates are the same for everyone: Opus at $5 input and $25 output per million tokens, Sonnet at $3 and $15, Haiku at $1 and $5. There is no negotiation surface for small and mid-sized accounts. A gateway aggregates demand and resells at a lower per-token rate. BUZZ pricing sits well below upstream rates while preserving the same shape (input, output, and cache multipliers), so cost modeling stays familiar.

Multi-model reach. Your application probably wants more than Claude. Maybe Gemini for long-context cheap drafting. Maybe GPT for a specific eval. Maybe Grok for a niche workload. Each direct integration adds a credential, a billing relationship, an SDK, and a different error shape. One gateway, one key, one error vocabulary collapses all of that.

Audit and observability. Token-accurate billing telemetry, per-feature usage attribution, per-team rate limits, and a single place to revoke a key are all things you would otherwise build yourself. A gateway gives you those by default.

None of these gaps are hypothetical. The first time a launch coincides with an upstream incident, or finance asks why two product surfaces share a single line item, or a security review wants to know exactly which keys reached which models last quarter, the gap shows up as engineering work nobody scoped. A gateway is the place where that work has already been done.

What a clean gateway should and should not do

The temptation when building a relay is to "help." Inject a safety preamble. Trim what looks like an oversized system prompt. Quietly route Opus traffic to Sonnet under load. Cache responses to save tokens. Every one of those choices breaks the contract that the application thought it had with the model.

A correct gateway is boring. It accepts the request, forwards the bytes, returns the bytes. The four properties that matter:

• Transparent forwarding. The request body, including system, messages, tools, tool_choice, temperature, top_p, top_k, metadata, and any cache control markers, goes upstream byte-for-byte. The response goes back byte-for-byte. No prompt rewriting. No silent model substitution. No injected instructions.
• Zero data retention. Request and response bodies are never written to disk, databases, or logs. Only billing metadata persists: model name, input and output token counts, timestamps, and a user identifier for attribution. If a regulator or a customer asks "what did you keep," the answer is a short list.
• No buffering on streams. Server-Sent Events from Anthropic are forwarded chunk by chunk. The first token reaches the client as soon as the upstream emits it. A gateway that buffers a complete response before flushing is invisible in benchmarks but felt by every user with a chat UI.
• Stable error semantics. Upstream 4xx and 5xx responses are returned with their original status and body. The gateway adds its own errors only for gateway-specific failures (auth, rate limit, billing) and uses distinguishable codes for them.

BUZZ implements all four. If you want the contract in one sentence: the only difference between calling BUZZ and calling Anthropic directly should be the URL, the key, and the invoice.

Anti-patterns to recognize in any gateway, including ours

When evaluating any AI relay, ask the operator three questions and watch for hedging:

1. Do you log request bodies anywhere, even temporarily, even encrypted, even for "abuse detection"? "Yes but" is a no.
2. Do you ever route a request to a different model than what was specified, for any reason, including capacity, latency, or cost? Silent substitution destroys evals.
3. Do you transform the request body, including stripping fields, normalizing casing, or adding system instructions? Anything beyond TLS termination is a transformation.

Code walkthrough: from direct Anthropic to a gateway

The migration path is intentionally trivial. The Anthropic Python SDK accepts a base_url argument. That is the entire diff for most codebases.

Direct, before

import anthropic

client = anthropic.Anthropic(api_key="sk-ant-...")

resp = client.messages.create(
    model="claude-opus-4-8",
    max_tokens=1024,
    messages=[{"role": "user", "content": "Summarize the CAP theorem in one paragraph."}],
)
print(resp.content[0].text)

Through BUZZ, after

import anthropic

client = anthropic.Anthropic(
    api_key="buzz-...",
    base_url="https://buzzai.cc",
)

resp = client.messages.create(
    model="claude-opus-4-8",
    max_tokens=1024,
    messages=[{"role": "user", "content": "Summarize the CAP theorem in one paragraph."}],
)
print(resp.content[0].text)

Two lines change: the key value and the base_url. The model identifier is the same string Anthropic publishes, because the gateway forwards it verbatim.

Streaming

Streaming uses the SDK's standard helper. The gateway forwards SSE chunks without buffering, so the perceived latency to first token is whatever the upstream returns plus a small fixed network hop.

with client.messages.stream(
    model="claude-sonnet-4-6",
    max_tokens=2048,
    messages=[{"role": "user", "content": "Write a haiku about idempotency."}],
) as stream:
    for text in stream.text_stream:
        print(text, end="", flush=True)
    final = stream.get_final_message()
    print()
    print("usage:", final.usage)

Tool use

Tool definitions and tool_choice pass through unchanged. The response shape, including tool_use blocks and the eventual tool_result round-trip, is identical to the direct Anthropic surface.

tools = [{
    "name": "get_weather",
    "description": "Get current weather for a city.",
    "input_schema": {
        "type": "object",
        "properties": {
            "city": {"type": "string"},
            "unit": {"type": "string", "enum": ["c", "f"]},
        },
        "required": ["city"],
    },
}]

resp = client.messages.create(
    model="claude-opus-4-8",
    max_tokens=1024,
    tools=tools,
    tool_choice={"type": "auto"},
    messages=[{"role": "user", "content": "What's the weather in Reykjavik in celsius?"}],
)

for block in resp.content:
    if block.type == "tool_use":
        print("call:", block.name, block.input)

Prompt caching

Anthropic prompt caching is preserved end to end. Mark a long, stable prefix (a system prompt, a document, a tool catalog) with cache_control and the gateway forwards the directive without modification. The first request writes the cache; subsequent requests within the TTL read it at a small fraction of the input rate.

system = [
    {
        "type": "text",
        "text": LONG_STABLE_RULES_DOCUMENT,
        "cache_control": {"type": "ephemeral", "ttl": "1h"},
    }
]

resp = client.messages.create(
    model="claude-sonnet-4-6",
    max_tokens=512,
    system=system,
    messages=[{"role": "user", "content": "Apply the rules to this transcript: ..."}],
)

print(resp.usage)  # cache_creation_input_tokens / cache_read_input_tokens

Billing follows Anthropic's published model: 5-minute cache writes at 1.25x the base input rate, 1-hour writes at 2x, reads at 0.1x. Those multipliers apply on top of the gateway's discounted base rate, which means cached workloads compound the savings.

OpenAI SDK compatibility

If your codebase already standardizes on the OpenAI Python or TypeScript SDK, you can call Claude through the OpenAI-compatible surface by pointing at https://buzzai.cc/v1. This is convenient for teams that want one client library across model families, although the native Anthropic SDK gives you full fidelity for tools and streaming events.

from openai import OpenAI

client = OpenAI(
    api_key="buzz-...",
    base_url="https://buzzai.cc/v1",
)

resp = client.chat.completions.create(
    model="claude-opus-4-8",
    messages=[{"role": "user", "content": "Hello, Claude."}],
)
print(resp.choices[0].message.content)

Claude Code in one command

For interactive coding sessions, Claude Code can be pointed at the gateway in a single step:

curl -fsSL https://buzzai.cc/sh/claudecode.sh | bash

The installer configures the CLI to authenticate with a BUZZ key and route through the gateway. Existing projects pick up the new endpoint with no per-repo changes. The script source lives at https://buzzai.cc/sh/claudecode.sh if you prefer to read it before piping it.

Pricing model

The pricing surface mirrors Anthropic's so cost modeling does not change shape. Each model has a base input rate and a base output rate, both denominated per million tokens. Prompt caching applies multipliers on top of the input rate. The gateway's headline number is that base rates are significantly below first-party rates, with the exact discount floating with upstream pricing and capacity.

For reference, Anthropic's published list pricing is:

BUZZ rates sit well below those numbers. Because the discount moves with the market, the canonical figures live on the live pricing page rather than this article: https://buzzai.cc/api/pricing. The full list of currently routable models is at https://buzzai.cc/models, including the long form identifiers (claude-opus-4-5-20251101, claude-sonnet-4-5-20250929, claude-haiku-4-5-20251001) for teams that pin to a specific snapshot.

Cache multipliers are unchanged from upstream: 1.25x base input for the 5-minute write, 2x for the 1-hour write, 0.1x for reads. Multiplying a smaller base by the same multiplier produces a smaller absolute number, which is the part that matters for the budget.

Operational concerns

Rate limits and backpressure

The gateway enforces per-key request and token budgets, separate from upstream limits. When a key approaches its allotment, the gateway returns 429 with a Retry-After header. Upstream 429s and 529s (overloaded) are passed through with their original semantics so that SDK retry logic continues to behave the way Anthropic documents it.

Retries

The Anthropic SDK already retries idempotent failures with exponential backoff. The gateway does not double-retry on the server side, because that would multiply load during incidents. If you want failover across model families (for example, Claude to Gemini when Anthropic is overloaded), implement it explicitly in your application using the same BUZZ key.

Error handling

The gateway preserves Anthropic error envelopes. A 400 from Anthropic stays a 400 with the original error.type and error.message. The gateway adds its own errors only for gateway-layer issues, distinguishable by an error.type value prefixed with gateway_ so that downstream code can branch cleanly.

Observability

Per-request billing metadata is available through the dashboard and an export endpoint. The metadata is intentionally narrow: model, input and output token counts (including cache write and cache read counts), timestamp, and the user identifier the request was attributed to. There is no body, no prompt prefix, no response excerpt. If you need to log request content for your own audit purposes, do it in your own application before the request leaves your process.

Key hygiene

Keys are scoped per project and revocable in one click. Because the gateway accepts the same key across Claude, GPT, Gemini, and Grok, rotating one credential after a leak rotates access to all of them at once, which is usually what you want.

Cost attribution across teams

One pattern that recurs in real deployments: a single application surface (a chat assistant, a code reviewer, an internal Q&A tool) ends up calling several models for different jobs. Drafts go to Haiku, hard reasoning to Opus, long-context retrieval to Sonnet. Without a gateway, the bill arrives as a flat line item per provider and has to be reconstructed from logs. With a gateway, every request carries a small set of tags (key, user identifier, model) and the dashboard rolls them up into per-feature spend without storing any prompt content. That is enough to answer the question every quarter: which features moved the cost line, and is the unit economics still positive.

Versioning discipline

Pinning to a snapshot identifier such as claude-opus-4-5-20251101 is a deliberate choice for evaluation pipelines and regression suites. The gateway forwards snapshot identifiers verbatim, which means your eval harness will keep producing the same scores until you choose to move it. For interactive product surfaces, the family alias (claude-opus-4-8) is usually the right call, because it picks up upstream improvements automatically. Mixing the two intentionally, snapshots in CI and aliases in production, is a common and reasonable shape.

FAQ

Conclusion

The right mental model for a Claude API gateway is plumbing, not middleware. It should accept your request, deliver it intact, return the upstream response intact, and bill for the tokens that crossed the wire. Everything else, transformation, retention, silent rerouting, is a feature you did not ask for and a correctness risk you do not want.

BUZZ is built around that contract. Same SDK, same model names, same response shape, lower per-token cost, one key across model families, and no copy of your data left behind. Start at https://buzzai.cc, check current rates at https://buzzai.cc/api/pricing, and see the routable model list at https://buzzai.cc/models.