# qache

> qache is a transparent ZDR gateway for production LLM traffic. It
> automatically runs cache operations on your prompts and can route each request
> to the best model for your quality bar. Drop-in: prefix any provider's base
> URL with https://gateway.qache.cc/ and add an X-Qache-API-Key header. Provider
> credentials travel unchanged on their normal headers.

Cut spend and hold quality on every LLM request. Start with automatic caching in minutes; qache learns your traffic and pays for repeated work just once. When you're ready, add a custom router that sends each request to the right model for your quality bar. Prompts are processed in memory with zero data retention by default.

The best cache strategy depends on both shape and timing. One route may repeat in short bursts while another repeats slowly over an hour. The right cache TTL depends on when requests arrive. Stable instructions mix with changing user input, retrieved snippets, identifiers, and runtime values in different places. qache learns your traffic patterns and automatically chooses the cache strategy that delivers the best savings and speed for each workload.

## Gateway

- Host: https://gateway.qache.cc/
- Prefix rule: replace `https://` in the provider's native base URL with `https://gateway.qache.cc/`
- Auth header: `X-Qache-API-Key: <key>`, a `qk_` dashboard key or an `ak_` agent key (see "API keys" below)
- Diagnostic response headers: `x-qache-mode`, `x-qache-analysis-mode`, `x-qache-rewrite-case`, `x-qache-request-id`
- Failure-source header: `x-qache-error: <code>` is set on every qache-internal failure response (auth, draining, queue, malformed, upstream unreachable). It is absent from a clean upstream pass-through, including upstream LLM error responses, so clients can tell a qache failure apart from an LLM failure on the wire.
- Caller-set cache markers are respected: a request that already carries Anthropic `cache_control`, OpenAI `cache_control` / `prompt_cache_key`, or Gemini `cachedContent` is forwarded byte-for-byte. The gateway will not add, move, or strip your markers. Decision is per request; other requests still get automatic breakpoint placement.

## API keys

Two kinds of key authenticate the same `X-Qache-API-Key` header:

- **Dashboard key (`qk_`)**, for humans. Sign in at https://qache.cc/dashboard, create a tenant, and generate a key. Full dashboard observability from the first request; prompt bodies are not retained by default (ZDR).
- **Agent key (`ak_`)**, for autonomous agents, no sign-in. `POST https://operations.qache.cc/api/v1/public/agent-keys` (no auth, empty body) returns an `ak_` key you can use immediately. Agent keys enable every feature, so prompt bodies ARE retained by default to power suggestions, the one exception to qache's ZDR default. An agent key has no dashboard until a human claims it.
- **Claim an agent key:** a signed-in human can attach an existing `ak_` key to their account at https://qache.cc/dashboard/api-keys (paste the key, name the tenant). The key keeps working unchanged; claiming adds dashboard observability + management and lets you turn prompt retention off. A key must have been used at least once before it can be claimed.

## Provider routes

- OpenAI: https://gateway.qache.cc/api.openai.com/v1
- Anthropic: https://gateway.qache.cc/api.anthropic.com
- Gemini API: https://gateway.qache.cc/generativelanguage.googleapis.com
- OpenRouter: https://gateway.qache.cc/openrouter.ai/api/v1
- Grok (xAI): https://gateway.qache.cc/api.x.ai/v1
- Vertex AI: https://gateway.qache.cc/LOCATION-aiplatform.googleapis.com/v1/projects/PROJECT_ID/locations/LOCATION/endpoints/openapi
- Amazon Bedrock: https://gateway.qache.cc/bedrock-mantle.REGION.api.aws/v1

## Recommended integration patterns

Three paradigms, ranked from best to most verbose. All work; pick the one that minimizes per-call-site changes in your codebase.

1. **Configure once at SDK client construction (preferred for SDK users).** Every supported provider SDK accepts a base URL + default headers at construction time. Set them once and every call routes through qache. Example in [the integration guide](https://qache.cc/docs.md).
2. **HTTP transport wrapper (preferred for raw HTTP code).** For code that builds requests by hand (e.g. Go `http.NewRequest`, Python `requests`/`httpx`, JS `fetch`), wrap the HTTP client's transport once instead of editing every call site. Drop-in snippets for Go, Python, and JS are in [the client middleware guide](https://qache.cc/docs/clients.md), with an opt-in fallback to the original upstream if qache is unreachable.
3. **Per-call-site URL swap.** If you only call one endpoint, gating on `if QACHE_KEY != ""` and rewriting the URL + adding the header inline is fine. Beware: it multiplies the diff by every call site.

## Defensive integration (fallback)

qache is in beta. Clients that want to fail open (sending requests directly to the upstream LLM if the gateway is unreachable) should use the transport wrapper from [the client middleware guide](https://qache.cc/docs/clients.md). The wrapper falls back when (a) the gateway returns no response (TCP/DNS error) or (b) the response carries an `x-qache-error` header (a qache-internal failure). Upstream LLM errors pass through unchanged, so a real LLM 5xx is never silently retried.

## Routing (beta)

Model routing (beta) lets qache pick the model for each request: a cheaper model where it holds up, a stronger one where it's needed, within the egress you already authenticated. Opt in per request with the `X-Qache-Router: r_…` header.

- **Aggregator egresses only.** Routing applies only when you're going through an aggregator egress (OpenRouter, Vertex AI, or Amazon Bedrock), because picking a model reuses that one aggregator credential across models. A `X-Qache-Router` named on the `direct` single-provider egress (or a URL-prefix forward-form request) is rejected with `routing_unsupported_egress`; routing is never applied silently on an unsupported egress.
- **Platform routers need no sign-in.** Any key, including a no-login `ak_` agent key, can use a platform-global router by adding the header on an aggregator route. (Creating your own router needs the dashboard.) Public `r_…` ids for platform routers are listed in the dashboard; this doc names no concrete id.
- **What runs on a miss.** On a confident classification the router runs the model its eval assigned to that kind of request. On a miss (no confident match, or the classifier is briefly unavailable) it falls back, in order, to the model you sent on the request, then a default model configured for your key or tenant; if none is set the request fails fast with `400 router_no_fallback_model` rather than going to an unintended model.
- **Overload returns 529, not your model.** If the gateway is overloaded it returns `529` with a `Retry-After` header before any model runs; back off and retry (Anthropic SDKs already do). A `529` is "retry shortly", not a passthrough of your original model.
- **Read the decision off the wire.** Routed responses carry an `x-qache-router-decision` response header: `routed` (matched a shape, ran the assigned model) or `routed_default` (no confident match, ran the fallback model). Both are normal 2xx traffic.
- **Platform routers are self-serve; custom routers are not.** The platform routers above are ready to use with just the header. A router tuned to your own quality bar on your own traffic is built with the qache team from your evals; see "Working with the qache team" below.

## Working with the qache team

Two qache capabilities are set up with us directly rather than self-serve, because each is tuned to your specific workload:

- **Custom routing for production traffic.** Beyond the self-serve platform routers above, qache can build a router tuned to your own evaluation set: one that holds your quality bar on your real traffic, sending each request to the cheapest model that clears it. This starts as a guided pilot.
- **Self-hosted deployment.** For teams that need prompts and traffic to stay in-house, qache can run entirely inside your own infrastructure so data never leaves your perimeter. Arranged case by case.

To get started with either, email info@qache.cc and include: (1) a short description of your use case (what you're routing, or your deployment environment); (2) for routing, a representative evaluation set: examples of your traffic plus what a good response looks like for your workload (or ask us and we'll help you build one); and (3) a point of contact we can follow up with.

You can also start from the website: https://qache.cc/enterprise#routing for routing, or https://qache.cc/enterprise#self-hosted for self-hosted.

## Documentation

- [Integration guide](https://qache.cc/docs.md): quickstart, prefix rule, provider matrix
- [Client middleware guide](https://qache.cc/docs/clients.md): Go / Python / JS transport snippets with opt-in fallback
- [FAQ](https://qache.cc/faq.md): caching behavior, data handling, current limits
- [Status](https://qache.cc/status.md): service availability
- [Full docs](https://qache.cc/llms-full.txt): integration guide + middleware guide + FAQ + status as one file