Why Busbar

Busbar is the reliability layer for your AI traffic: the breaker-and-failover control plane that sits between your application and every provider it calls. This page is for the person deciding whether to adopt it: the specific problems it solves, what it enables that you would otherwise have to build yourself, and an honest comparison with the tools you are probably weighing it against.

It shares an arena with multi-provider proxies and hosted routers, but the priorities are different. Where those forward requests and list models, Busbar is built reliability-first. It knows whose fault a failure is. It fails over inside the request, before your user sees a byte. And it translates losslessly across six wire protocols, so you never trade away a provider’s native features to get portability.

The problems Busbar solves

One provider’s bad day becomes your outage

If your application calls a single provider directly, any upstream incident (rate limits, degraded capacity, a regional outage) surfaces immediately to your users. The usual fix is defensive try/except blocks scattered across the codebase. Those blocks are not failover; they are error handling. They do not retry on a different provider, they do not skip a circuit that is already open, and they do not restore service while the original provider recovers.

Busbar provides genuine in-flight failover. Requests that have not yet received a first byte from the upstream are automatically retried against the next available lane in the pool. A configurable per-request deadline and hop cap (defaults: 120 seconds, 3 hops) bound worst-case latency. The breaker for the failed lane opens independently of every other lane, so a rate-limit storm on one provider does not suppress traffic to healthy ones.

Vendor lock-in is in the SDK call, not the contract

Every provider ships an SDK that speaks its own wire format. Migrating from Anthropic to OpenAI, or adding a second provider as a fallback, means updating call sites across your codebase. In practice, teams don’t do this until they have to, and by then the migration is a project.

Busbar presents a single endpoint to your application. You configure which provider or pool of providers lives behind it. Swapping or adding a provider is a config change, not a code change. Because Busbar translates losslessly between all six supported wire protocols (Anthropic, OpenAI, OpenAI Responses, Gemini, Amazon Bedrock, Cohere), your application does not need to know or care which model answered the request.

Cost control requires a control plane

Direct API usage gives you a bill at the end of the month. It does not give you enforced budget caps, rate limits applied in real time, or auditability of which workload consumed what. Without a control plane, cost control means trusting every developer and every deployment to self-police.

Busbar’s governance layer issues virtual keys: signed, expiring bearer tokens scoped to a set of pools and bound to at most one enforcement group. A group carries generic limits: request, token, and budget caps per window (minute, hour, day, month, or total) plus an instantaneous concurrency cap, all priced from an operator-declared rate card and a flat per-request fee. Groups nest under a parent, and admission walks the whole chain so a request passes only when every limit of the key’s group AND every ancestor group is under cap, with the rejection naming the exact bucket that blocked. A key for a staging environment can be capped at a daily budget and restricted to a cheaper model pool; an internal tool can be rate-limited independently of the production path. That is how per-team or per-project limits are enforced, not just observed. Keys expire (default 90 days) and revoke via a fleet-wide denylist; usage is tracked per key and queryable via the admin API.

Costs are computed from tokens, not stored as dollars. The store accumulates immutable per-model token counts; every spend figure is derived at read time as tokens x rate_card + requests x per_request_fee. Correcting a mispriced model is a config edit and a reload, and every historical and current window re-prices on the next read, with no re-billing and no migration. The rate numbers are abstract cost units, so Busbar attaches no currency and does pure integer math; symbols and FX are your dashboard’s concern.

One operational caveat worth stating plainly. RPM limits are enforced precisely (the counter is incremented synchronously on admission). Budget caps are hard and atomic: at admission Busbar derives every bucket’s spend fresh from its token ledger and admits only if the key and every ancestor budget group are under cap, charging the whole chain or nothing in one critical section. A 429 insufficient_quota names which bucket blocked. Because admission is in-memory and never touches the durable store, the cap holds even when the store is unreachable. That in-memory speed carries one honest limit: the hard cap is enforced per node. With N Busbar nodes splitting traffic behind a shared store, each node enforces from its own counters and reconciles durably through additive flushes, so between flushes the fleet can admit up to roughly N times a configured cap. The cap is not a synchronous cluster-wide gate. TPM limits remain best-effort under concurrency, since token usage is only known after the response.

Your data path is also your security perimeter

When your application calls a provider directly, your traffic and your API keys are visible to any process or infrastructure component in that path. More practically: every provider SDK in your codebase is a secret-handling surface. Rotating a key means finding and updating every deployment that holds it.

Busbar holds provider keys in one place: the process that reads the config file at startup. Your application carries only a Busbar virtual key (or a client token). Rotating an upstream provider key is a Busbar restart, not a deployment sweep. The request path itself is security-hardened: SSRF guards on all configured URLs, constant-time token comparison that closes list-position timing oracles, and native-protocol error envelopes that reveal no Busbar internals to callers.

”We only have one provider” is not a reason to wait

Busbar is valuable before the second provider exists. Run it as a straight same-protocol passthrough in front of your one provider, no features enabled, and your posture improves the same day: the provider key moves out of every app deployment and into one process, so a leaked app credential is no longer a leaked provider account, and rotating the key is a restart, not a deployment sweep. max_concurrent caps a runaway loop at your ceiling instead of your bill. Every request becomes visible in /metrics and /stats where direct SDK calls are invisible. The request path is hardened whether you asked for it or not: SSRF-guarded upstreams, constant-time token checks, body caps, secrets never logged. And the cost of the extra hop is tens of microseconds, measurable per-request on your own traffic via the opt-in Server-Timing header.

Then the constraint moves. A team whose compliance posture locks them to one provider today (a BAA that covers AWS Bedrock but not yet Anthropic, say) points their existing SDK at Busbar and changes nothing else. The day the second provider becomes possible (a better price, a better model, a signed BAA, a bad outage), it’s a new lane in config.yaml. The application code never changes and never learns which backend answered. Without the control plane already in place, that same day starts with rewriting every AI call site you own. The endpoint swap is a one-line change; the option it buys is the point.

How Busbar compares

The detailed comparisons live on their own pages, with every claim about the other tools cited to their own documentation and re-checked against it: Busbar vs LiteLLM and Busbar vs OpenRouter.

The short version. LiteLLM is a Python-native router with a big preconfigured catalog; if you want an in-process Python library, it’s the right call. Busbar is a different kind of tool: no privileged protocol (all six first-class, in both directions), a fault-attributed circuit breaker rather than a cooldown timer, and a gateway whose own cost is measured in microseconds and megabytes. OpenRouter is a hosted service, and the deciding question is whether prompts passing through a third party is acceptable for your application: if it is, OpenRouter is excellent; if it isn’t, Busbar runs in your infrastructure and nothing enters the data path but the providers you configure.

The operational story

Busbar ships as a single static binary. Deployment is:

Write a config.yaml (providers + models, optionally pools and governance).
Set the environment variables your config references (one per provider key).
Run the binary.

There is no Python environment to manage, no Node runtime, and no database to provision: governance defaults to an in-memory store (ephemeral RAM, zero setup), and durability is an opt-in you point at a sqlite, postgres, or redis store plugin. No sidecar is required. Health, metrics, and management traffic all pass through the same process on the same port.

Observability is built in. Prometheus metrics are exposed at /metrics with bounded cardinality: metric labels use configured pool names and fixed enumerations, never raw model strings from client requests. OTLP trace export and a request-log webhook are both optional and configurable. The /healthz endpoint is side-effect-free (it never steals a recovery probe) and safe for high-frequency load balancer probing. Note that /metrics and /stats are not auth-exempt, they go through the same auth check as request traffic, since telemetry is itself a fingerprinting surface.

Security defaults are strict. Provider base_url values are guarded against cloud-metadata/IMDS endpoints (including alternate IP encodings). Loopback, RFC-1918, and CGNAT addresses are permitted for local-model upstreams (e.g. Ollama/vLLM); plain http:// is only allowed for those private/loopback hosts: public hosts must use https://. Observability sink URLs (OTLP endpoint, request-log webhook) apply a stricter guard that additionally blocks loopback, link-local, RFC-1918, and broadcast addresses. Auth failures return native-protocol error envelopes with no Busbar vocabulary, an Anthropic SDK sees an Anthropic 401, an OpenAI SDK sees invalid_api_key, a Gemini SDK sees a 400 INVALID_ARGUMENT, and a Bedrock SDK sees a 403 AccessDeniedException. Admin endpoints are separately guarded by an admin token and disabled entirely if none is configured.

The request body cap is 32 MiB (DefaultBodyLimit), enforced before handler code runs, with protocol-native 413 responses (not bare text).

One auth note for Bedrock: Busbar signs outbound Bedrock requests with AWS SigV4 AND verifies inbound SigV4 (when governance is enabled). Under governance, a Bedrock-SDK client authenticates with a minted aws_access_key_id + aws_secret_access_key pair: Busbar verifies the signature and enforces budgets / rate limits exactly like a bearer-token client. Without governance, Bedrock ingress requires auth.chain: [], with upstream_credentials: passthrough to forward the credentials upstream, or plain chain: [] to ignore them.

Who Busbar is for

Busbar is a good fit if:

You run your own infrastructure and want to own the full request path to AI providers.
You use more than one provider and want failover, load distribution, or the ability to swap providers without code changes.
You need per-key, per-team, or per-project cost control enforced at the control plane through nestable budget groups, before the request runs.
Your existing applications use different provider SDKs (OpenAI, Anthropic, Gemini, Bedrock, Cohere) and you want to standardize the routing layer without rewriting call sites.
You have data residency, compliance, or internal security requirements that exclude third-party traffic routing.

Busbar is not the right fit if:

You need a provider that speaks a wire protocol Busbar doesn’t support, one that’s neither one of the six protocols nor OpenAI-compatible. That single case needs new translator code (contribute it, or wait for it). Note what this is not: adding any other provider isn’t a code change and doesn’t wait on anyone. A provider is just an entry in your own providers.yaml: its protocol, base_url, the env var holding its key, and its error-code mappings. providers.yaml is your config file, not something the Busbar project hosts or gatekeeps; the verified catalog ships as a starting convenience you own and extend, not a list you’re limited to.
You are prototyping and want zero infrastructure overhead, use a hosted service.
You specifically want an in-process Python router library (imported into your code) rather than a network service, that’s LiteLLM’s design, and the more natural fit. To be clear, “I do ML” is not a reason to skip Busbar: LangChain, LlamaIndex, and the rest already work with it today, point their OpenAI/Anthropic client at Busbar’s base URL and you keep your whole framework stack, you just get failover and translation underneath it.
You want a hosted managed service with no operational responsibility.

Current status

Busbar is licensed Apache-2.0. The wire protocol translation, the data-plane HTTP surface, and the circuit breaker model are stable under Semantic Versioning; the config format is an operator artifact with a tooled migration path between releases, and the admin API carries its own contract version. The test suite covers over 2,300 test cases across the protocol translators, breaker FSM, auth middleware, governance enforcement, and config validation.

Apache-2.0 is permissive: use it commercially, modify it privately, redistribute it, with an explicit patent grant and no copyleft obligations.