Busbar
The fastest performing AI Control Plane · 100% Rust · every provider & capability

Your AI Control Plane

Busbar is a single binary you run in your own infrastructure, sitting between your applications and every AI provider. One endpoint to route and fail over chat, embeddings, images, audio, and rerank across providers, enforce hard budget caps, provision or revoke access, and see every request’s cost, latency, and traffic.

156k req/s · 33 µs added latency · 10.25 MB idle ~9× the throughput of Bifrost ~394× lower p99 than LiteLLM ~56× less idle memory than Kong ~540× faster streaming TTFT than Kong →
ChatEmbeddingsImagesAudioModerationsRerank
Any client · 6 protocols · 6 capabilities
OpenAI · Anthropic
Gemini · Bedrock
Cohere · Responses
TLS · mTLS
Busbar
TranslateRouteFailover
GovernanceSecurityObservabilityAI Middleware
one endpoint · :8080
Hooks: AI middleware on the path
Backends · any provider
Anthropic · OpenAI
Gemini · Bedrock
+ more
OpenAIAnthropicGeminiBedrockCohereResponses
Get started Download

Adopting it is a two-line swap.

Already speak OpenAI? Point the SDK at busbar and pick a pool by name. The request, left as OpenAI, may have been served by Anthropic and came back as OpenAI, translated both ways.

Already speak Anthropic? Point the Anthropic SDK at busbar and pick a pool by name. The request, left as Anthropic, may have been served by OpenAI and came back as Anthropic, translated both ways.

Already speak Gemini? Point the google-genai SDK at busbar and pick a pool by name. The request, left as Gemini, may have been served by Bedrock and came back as Gemini, translated both ways.

Already speak Amazon Bedrock? Point boto3 at busbar and pick a pool by name. The request, left as Bedrock, may have been served by Anthropic and came back as Bedrock, translated both ways.

Already speak Cohere? Point the Cohere SDK at busbar and pick a pool by name. The request, left as Cohere, may have been served by Gemini and came back as Cohere, translated both ways.

On the OpenAI Responses API? Same SDK, same two lines. The request, left as a Responses call, may have been served by Cohere and came back as Responses, translated both ways.

The model name is a config value, not a code dependency.

app.py 
# Already speak OpenAI? Just swap the base URL.
- client = OpenAI(api_key=OPENAI_KEY)
+ client = OpenAI(
+     api_key=BUSBAR_TOKEN,
+     base_url="http://busbar:8080",
+ )

  # "fast" = a pool you define in config:
  # 80% Claude / 20% GPT-4o, Gemini on failover.
  client.chat.completions.create(
      model="fast",
      messages=[{"role": "user", "content": "Hi!"}],
  )
app.py 
# Already speak Anthropic? Just swap the base URL.
- client = Anthropic(api_key=ANTHROPIC_KEY)
+ client = Anthropic(
+     api_key=BUSBAR_TOKEN,
+     base_url="http://busbar:8080",
+ )

  # "fast" = a pool you define in config:
  # 80% Claude / 20% GPT-4o, Gemini on failover.
  client.messages.create(
      model="fast", max_tokens=1024,
      messages=[{"role": "user", "content": "Hi!"}],
  )
app.py 
# Already speak Gemini? Just swap the base URL.
- client = genai.Client(api_key=GEMINI_KEY)
+ client = genai.Client(
+     api_key=BUSBAR_TOKEN,
+     http_options={"base_url": "http://busbar:8080"},
+ )

  # "fast" = a pool you define in config:
  # 80% Claude / 20% GPT-4o, Gemini on failover.
  client.models.generate_content(
      model="fast",
      contents="Hi!",
  )
app.py 
# Already speak Bedrock? Just swap the endpoint.
- bedrock = boto3.client("bedrock-runtime")
+ bedrock = boto3.client(
+     "bedrock-runtime",
+     endpoint_url="http://busbar:8080",
+ )

  # "fast" = a pool you define in config:
  # 80% Claude / 20% GPT-4o, Gemini on failover.
  bedrock.converse(
      modelId="fast",
      messages=[{"role": "user", "content": [{"text": "Hi!"}]}],
  )
app.py 
# Already speak Cohere? Just swap the base URL.
- co = cohere.ClientV2(api_key=COHERE_KEY)
+ co = cohere.ClientV2(
+     api_key=BUSBAR_TOKEN,
+     base_url="http://busbar:8080",
+ )

  # "fast" = a pool you define in config:
  # 80% Claude / 20% GPT-4o, Gemini on failover.
  co.chat(
      model="fast",
      messages=[{"role": "user", "content": "Hi!"}],
  )
app.py 
# The Responses API? Same swap, same client.
- client = OpenAI(api_key=OPENAI_KEY)
+ client = OpenAI(
+     api_key=BUSBAR_TOKEN,
+     base_url="http://busbar:8080",
+ )

  # "fast" = a pool you define in config:
  # 80% Claude / 20% GPT-4o, Gemini on failover.
  client.responses.create(
      model="fast",
      input="Hi!",
  )

A control plane you rent is still someone else’s. Busbar is one static binary in your infrastructure: inspect it, pin it, run it air-gapped. Nothing has to leave your network.

What Busbar does for you

Observability

Every request lands in Prometheus counters and latency histograms broken out by pool, protocol, and outcome. Label a key team: growth at mint and its token and spend gauges carry that label straight into your dashboards. A live /stats snapshot shows every model lane’s health, budget, and in-flight load, and OTLP traces or a request-log webhook plug into whatever you already run.

Enables See usage, cost, tokens, and latency by team or project without wiring your own metering.

Governance

Every virtual key binds to a budget group, and groups nest: a person under a team, a team under the org. A request is admitted only if every limit in that chain — request rate, token rate, concurrency, dollar budget — is under cap, checked and charged as one atomic step before the call is forwarded. Mint a key scoped to a group, watch its spend live, revoke it instantly.

Enables Budgets enforced before a request runs, not discovered on next month’s invoice.

Credential scoping

Every app gets a Busbar token, never your provider’s raw API key. By default Busbar holds the real credential and signs each upstream call itself, so a leaked app token can spend against its budget but can’t walk off with your OpenAI or Anthropic secret. Rotating a vendor key is one restart in one place, not a deploy sweep across every service that embedded it.

Enables Provider keys move out of your apps and into one place you control.

Operational reliability

Every provider lane carries its own circuit breaker, and every failure is attributed to a cause, not just counted: your bad request, a rate limit, a real outage, and a context overflow are each handled differently instead of retried into a wall. A failed attempt reroutes to another lane before the first byte reaches your client.

Enables A flaky provider gets pulled and probed back gently; one outage doesn’t stall your pipeline.

Universal API

Busbar speaks six wire protocols — OpenAI, Anthropic, Gemini, Bedrock, Cohere, and the Responses API — natively on both sides, bridged through one internal format rich enough to hold every protocol’s features. System prompts, tool calls, structured-output schemas, and Bedrock’s own tool-use blocks survive the hop losslessly, in either direction.

Enables Point whichever SDK your code already speaks at one URL and reach every model through it.

No model list to be on.

Busbar isn’t another proxy with a curated model catalog. You define a model name and the backends behind it. There’s no privileged protocol and no allowlist to wait for. Providers speak about six languages between them; Busbar speaks all six, so a new model works the day it ships, not the day we add it. See for yourself: pick your models, get the config.

One endpoint.
No fast-lane to opt into.

Point your SDK at Busbar in whatever language your code already speaks, and Busbar translates to any upstream and picks the most performant path for each request itself. There’s no separate “passthrough” route to learn and no provider to hard-code in your URL: same-protocol calls forward without translation automatically, and your cross-provider routing and failover stay in the path either way. Choosing the fastest route is the control plane’s job, not something we push into your client code.

Already committed to one AI provider?

Busbar pays for itself before the second provider exists. Run it as a straight passthrough and your posture improves the same day: the provider key moves out of every app deployment, a runaway loop hits your cap instead of your bill, and every request shows up in your metrics. When the constraint moves, a new backend is a config lane, not a rewrite. Why add a control plane now · the day it paid off

Built reliability-first

Route: a programmable request path

Routing is the first hook. Pick a built-in policy (weighted, cheapest, fastest, least_busy, usage) or bring your own as a webhook in any language, or as a sandboxed Rhai script. Policies see each member’s cost, latency, live concurrency, budget, and rate headroom. A slow or broken hook falls back; it never blocks a request.

Enables The same fail-safe machinery is built to carry PII-steering, audit, and guardrails: your own logic on the request path, in any language.

Security: native TLS & mutual TLS

Busbar terminates TLS itself (cert and key in config, no reverse proxy required). Turn on mTLS and a client must present a certificate signed by your CA; anyone without one is rejected at the handshake, before any bearer-token check is reached.

Enables Zero-trust ingress without standing up a service mesh.

A single Rust binary you own

One static binary with no runtime, no sidecar, and no interpreter in the request path. It ships for Linux and macOS (Intel and ARM) and Windows (Intel).

Enables Your infrastructure, your data path — nothing else to deploy or patch.

Program the path.

Reliability earns Busbar a spot between your app and your models. Hooks are what that spot becomes: a tap that watches, or a gate that decides: reject, restrict, route, or rewrite. Because hooks fire on Busbar’s normalized IR, you write one once and it runs against all six protocols and every provider, with failover and circuit breaking underneath it, in one hop. Busbar is where your AI middleware runs.

Tap: watch every request

Fire-and-forget observers on the path. A tap sees the full request and response and never blocks the hop: feed a SIEM, a compliance archive, custom alerting, and more. It runs alongside the request, so nothing waits on it.

Enables A SIEM feed or compliance archive that sees every call, across every provider, from one place.

Gate: decide on the path

Fire-and-wait deciders that run before the request leaves. A gate can reject a call, restrict it to certain models, re-order or route it, or rewrite the request body outright: a PII-redaction guard, a cost-and-latency router, or Headroom compressing context before it ships.

Enables Your policy enforced before the request runs, not reconstructed from logs after.

Write once, run anywhere

Hooks fire on Busbar’s normalized IR, the one internal representation every protocol is translated into, losslessly. Write a hook once and it runs against all six wire protocols and every provider, with failover and circuit-breaking underneath it, in one hop.

Enables One hook covers OpenAI, Anthropic, Gemini, Bedrock, Cohere, and Responses at once, with no per-provider plumbing to maintain.

Running in 60 seconds

One command pulls the binary and the provider catalog. Write a minimal config, point an SDK at it, and you’re serving. No sudo, no runtime, no virtualenv.

Full walkthrough →
install 
curl -fsSL https://getbusbar.com/install.sh | sh
config.yaml 
providers:
  anthropic:
    api_key_env: ANTHROPIC_KEY

models:
  claude-sonnet:
    provider: anthropic
    max_concurrent: 10
run 
export ANTHROPIC_KEY=sk-ant-...
BUSBAR_PROVIDERS=./providers.yaml \
BUSBAR_CONFIG=./config.yaml busbar   # serving on :8080
request 
curl localhost:8080/v1/chat/completions \
  -d '{"model":"claude-sonnet",
       "messages":[{"role":"user","content":"hi"}]}'
One binary
No Python runtime
Hardened
SSRF guards · constant-time auth
Cross-platform
Linux · macOS · Windows · ARM
Measured
33 µs added latency · verifiable per-request

Don't take our benchmark's word for it: onthebench.ai tests every AI gateway identically, Busbar included, and anyone can re-run it.

Using Busbar? We’d love to put your logo here. [email protected]