Getting Started with Busbar

Busbar is your AI control plane: a single self-hosted, static Rust binary that sits between your application and your AI providers. Point any SDK at it (OpenAI, Anthropic, Gemini, Cohere, Bedrock, or the OpenAI Responses API) and Busbar routes each request to the provider (or pool of providers) you configured, translating between wire protocols when the ingress and egress differ.

It does the same job as LiteLLM or OpenRouter (one API in front of every model and provider) and goes further: it speaks six provider protocols natively (so any vendor’s SDK can point at it, not just OpenAI-shaped ones), ships as one binary with no Python runtime and no third party in your data path, and gives you per-(pool, lane) circuit breaking with in-flight failover. You run it in your own infra and it holds your keys.

This guide takes you from zero to a working request in about five minutes.

---

What you need

• An API key for at least one supported provider (Anthropic, OpenAI, Gemini, Cohere, or AWS Bedrock credentials)
• The Busbar binary (see below)
• curl or any LLM SDK

---

Step 1: Get the binary

One-line install (macOS / Linux). It detects your platform, downloads the latest release binary and the provider catalog into the current directory, and prints the next steps:

curl -fsSL https://getbusbar.com/install.sh | sh

Drops busbar and providers.yaml where you run it (no sudo). To install onto your PATH instead: BUSBAR_INSTALL_DIR=/usr/local/bin curl -fsSL https://getbusbar.com/install.sh | sh.

Or download manually: grab the archive for your platform from the latest release (Linux x86_64/aarch64, macOS Intel/Apple Silicon, Windows x86_64), plus the provider catalog from getbusbar.com/providers.yaml. The binary is self-contained (no runtime, no virtualenv, no dependencies):

tar -xzf busbar-*.tar.gz   # extracts the `busbar` binary
chmod +x busbar
./busbar --version

Or use Docker: a tiny FROM scratch image (the static binary plus the provider catalog, amd64 + arm64, see the image-size badge on the repo for the current compressed size), cosign-signed with build provenance:

docker run -d -p 8080:8080 \
  -e ANTHROPIC_KEY \
  -v "$PWD/config.yaml:/etc/busbar/config.yaml:ro" \
  getbusbar/busbar

The provider catalog ships inside the image at /etc/busbar/providers.yaml, so you only mount config.yaml (written in Step 2). Pin an exact version (getbusbar/busbar:1.5.0) or ride latest. If you use a durable store, give it a writable volume (e.g. -v busbar-data:/var/lib/busbar with store.settings.db_path: /var/lib/busbar/governance.db).

Or build from source (requires Rust 1.97+):

cargo build --release      # binary at target/release/busbar

---

Step 2: Write a minimal config

Prefer clicking to typing? The config builder generates this file: pick your models, copy the YAML, come back for Step 3.

Busbar reads two YAML files:

• providers.yaml: the shipped provider catalog (protocol, base_url, error maps). You almost never edit this. The one-line installer fetches it for you, or grab it from getbusbar.com/providers.yaml.
• config.yaml, your deployment: which providers to activate, their key env-var names, your models, and optionally pools.

Important: keys are never written into config. api_key: { env: VAR } is a secret REFERENCE naming the environment variable that holds a provider’s key; Busbar resolves it at startup (a { file: /path } reference or a secret plugin work the same way). (Separately, ${VAR} tokens elsewhere in config.yaml are expanded from the environment at load time.) An unset referenced variable is a loud startup failure, not a silent skip.

Minimal config.yaml (one provider, one model, no auth)

This is the smallest config that boots and serves requests. Use it on a local machine to try Busbar quickly.

# config.yaml (dev/minimal, no client auth gate)
providers:
  anthropic:
    api_key: { env: ANTHROPIC_KEY }   # the NAME of the env var to read the key from, NOT the key itself

models:
  claude-sonnet:
    provider: anthropic

provider is the only required field on a model. max_concurrent (a per-lane concurrency limiter) is optional and defaults to unbounded; add it only when you want to cap in-flight requests to a model.

The key itself is never in this file. api_key: { env: ANTHROPIC_KEY } tells Busbar “read this provider’s key from the $ANTHROPIC_KEY environment variable at startup”, so you set the real secret in your environment (Step 3), and config.yaml stays safe to commit and share.

Save this as config.yaml in your working directory.

providers.yaml must also be present. The one-line installer fetches it for you. If you built from source it lives in the repo root; otherwise grab it from getbusbar.com/providers.yaml.

What the fields mean

Field	What it does
providers.<name>.api_key	A secret reference to this provider’s API key ({ env: VAR } / { file: /path } / a secret plugin)
models.<name>.provider	Which provider entry in the providers block this model calls
models.<name>.max_concurrent	Optional per-lane concurrency limiter: max simultaneous in-flight requests to this model. Omit for unbounded (the default); set a value ≥ 1 to cap.

providers and models are the only required sections. listen defaults to 0.0.0.0:8080. auth defaults to an empty chain (chain: []), an open relay, when omitted, fine for local dev, not for production.

---

Step 3: Set environment variables and run

# the actual secret, this is what `api_key: { env: ANTHROPIC_KEY` } in config.yaml points at
export ANTHROPIC_KEY=sk-ant-...

BUSBAR_PROVIDERS=./providers.yaml BUSBAR_CONFIG=./config.yaml ./busbar

Busbar logs a startup event indicating the listen address (busbar listening, with the bound address as a field). It accepts requests immediately: Prometheus/TSC calibration is deferred to a background thread, so it never blocks the hot path at boot.

Check liveness:

curl -s http://localhost:8080/healthz
# → ok

/healthz is always unauthenticated and returns 200 ok when at least one lane is ready, 503 no usable lanes when every lane’s circuit breaker is open. It is side-effect-free and never steals a recovery probe.

---

Step 4: Send a request

Via curl: Anthropic-format ingress

The model name goes in the URL path: POST /<model-name>/v1/messages. Busbar resolves <model-name> against your configured pools first, then your models, and routes the request to the matching lane.

curl -s http://localhost:8080/claude-sonnet/v1/messages \
  -H "Content-Type: application/json" \
  -d '{
    "max_tokens": 256,
    "messages": [{"role": "user", "content": "What is a busbar?"}]
  }' | jq .

You get back a standard Anthropic Messages response. Because both ingress and egress are Anthropic here, Busbar relays it as a native same-protocol passthrough. Routing keys off the name in the URL, not the model field in the body.

Via curl, OpenAI-format ingress

The model name goes in the request body: POST /v1/chat/completions. This works with any OpenAI SDK or tool that targets http://localhost:8080 as the base URL.

curl -s http://localhost:8080/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "claude-sonnet",
    "messages": [{"role": "user", "content": "What is a busbar?"}]
  }' | jq .

Busbar translates the request from OpenAI format to Anthropic format on the way out, and the response from Anthropic format back to OpenAI format on the way in, through its intermediate representation, transparently. Your client receives a standard chat.completion object, with the answer in choices[0].message.content.

Via the OpenAI Python SDK

from openai import OpenAI

client = OpenAI(
    api_key="unused",          # no client auth gate in the minimal config
    base_url="http://localhost:8080",
)

response = client.chat.completions.create(
    model="claude-sonnet",     # busbar model name, not OpenAI's
    messages=[{"role": "user", "content": "What is a busbar?"}],
)
print(response.choices[0].message.content)

The OpenAI SDK has no idea it’s talking to an Anthropic backend. Swap model="claude-sonnet" to any model or pool name you configured; no other change required.

Via the Anthropic Python SDK

import anthropic

client = anthropic.Anthropic(
    api_key="unused",
    base_url="http://localhost:8080",
)

message = client.messages.create(
    model="claude-sonnet",     # the model name from your config
    max_tokens=256,
    messages=[{"role": "user", "content": "What is a busbar?"}],
)
print(message.content[0].text)

The Anthropic SDK sends x-api-key; Busbar accepts it on the /v1/messages routes.

---

Step 5: Add a second provider and a pool

Once the single-provider setup is working, extend the config to introduce a pool. A pool is a named group of models with weighted load balancing, per-member circuit breaking, and automatic failover.

# config.yaml, two providers, two models, one pool, with client auth
auth:
  chain:
    - keys                                 # callers present minted signed keys
  admin_auth:
    - admin-tokens: { token: { env: BUSBAR_ADMIN_TOKEN } }

providers:
  anthropic:
    api_key: { env: ANTHROPIC_KEY }
  openai:
    api_key: { env: OPENAI_KEY }

models:
  claude-sonnet:
    provider: anthropic
    max_concurrent: 20
  gpt-4o:
    provider: openai
    max_concurrent: 20

pools:
  smart:
    members:
      - model: claude-sonnet
        weight: 2
      - model: gpt-4o
        weight: 1

Set the additional environment variables, restart Busbar, and mint a caller key (shown once):

export ANTHROPIC_KEY=sk-ant-...
export OPENAI_KEY=sk-...
export BUSBAR_ADMIN_TOKEN=your-admin-token

BUSBAR_PROVIDERS=./providers.yaml BUSBAR_CONFIG=./config.yaml ./busbar

# Mint a signed key for your app (expires in 90 days by default):
BUSBAR_CLIENT_TOKEN=$(curl -s -X POST http://127.0.0.1:8081/api/v1/admin/keys \
  -H "Authorization: Bearer $BUSBAR_ADMIN_TOKEN" -H "Content-Type: application/json" \
  -d '{"name":"quickstart"}' | jq -r .token)

Now call the pool by name. Both ingress styles work against a pool:

# OpenAI ingress, model field selects the pool
curl -s http://localhost:8080/v1/chat/completions \
  -H "Authorization: Bearer $BUSBAR_CLIENT_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"model": "smart", "messages": [{"role": "user", "content": "Hello!"}]}'

# Anthropic ingress, pool name in the URL path
curl -s http://localhost:8080/smart/v1/messages \
  -H "Authorization: Bearer $BUSBAR_CLIENT_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"max_tokens": 256, "messages": [{"role": "user", "content": "Hello!"}]}'

With weight: 2 on claude-sonnet and weight: 1 on gpt-4o, Busbar distributes via smooth weighted round-robin, roughly two of every three requests go to Claude and one to GPT-4o. If claude-sonnet’s breaker trips (on upstream 5xx, 429, timeout, or network errors), Busbar fails the request over to gpt-4o, provided the failure happens before the first byte of the response reaches your client. After the first byte, in-flight failover is no longer possible.

What “cross-protocol” means here

claude-sonnet speaks Anthropic; gpt-4o speaks OpenAI. A client using OpenAI-format ingress (/v1/chat/completions) is making an OpenAI request. When Busbar routes that request to claude-sonnet, it translates the request from OpenAI to Anthropic format, and the response back, losslessly, through its intermediate representation. When it routes to gpt-4o, it passes through natively. Your client never needs to know.

---

Checking health and stats

Liveness (always unauthenticated):

curl -s http://localhost:8080/healthz

Returns 200 ok if any lane is ready, 503 no usable lanes otherwise.

Per-lane topology (/stats):

curl -s http://localhost:8080/stats \
  -H "Authorization: Bearer $BUSBAR_CLIENT_TOKEN" | jq .

/stats goes through the auth middleware, so with a non-empty auth.chain it requires a valid key; under chain: [] it is open. It returns a per-lane snapshot: model, provider, max_concurrent, inflight, free_slots, ok/err/client_fault counts, usable, dead, dead_reason, cooldown_remaining_s, streak, and budget. A key restricted to specific allowed_pools only sees the pools and lanes it can reach.

Prometheus metrics (/metrics) — opt-in; add a metrics: block with a required buffer_seconds retention window first, otherwise the route is not mounted:

curl -s http://localhost:8080/metrics \
  -H "Authorization: Bearer $BUSBAR_CLIENT_TOKEN"

Prometheus scrape exposition. Like /stats, /metrics is subject to the auth middleware (it is not auth-exempt, telemetry is a fingerprinting surface), so it requires a key with a non-empty chain and is open under chain: []. Key metrics: busbar_requests_total, busbar_upstream_failures_total, busbar_breaker_trips_total, busbar_request_duration_seconds, busbar_translations_total.

---

Common setup variations

auth: none (local dev, open relay)

Omit the auth block entirely, or set chain: []. No Authorization header required. Do not use in production.

auth: passthrough (forward your own key)

auth:
  chain: []
  upstream_credentials: passthrough

The caller’s own token (Authorization: Bearer, x-api-key, or x-goog-api-key) is forwarded directly to the upstream provider. Use this when each caller has their own provider key and you want Busbar purely for routing and protocol translation, not credential management.

Note: with passthrough Busbar forwards the caller’s credential and holds no upstream keys; a caller with a bad key can hard-down a lane for everyone (30 minutes), so use it deliberately.

Bedrock egress (Busbar signs requests with SigV4)

Add a Bedrock provider. The key env var holds ACCESS_KEY_ID:SECRET_ACCESS_KEY (or with an optional third segment :SESSION_TOKEN):

providers:
  bedrock:
    api_key: { env: AWS_BEDROCK_CREDS }

models:
  claude-bedrock:
    provider: bedrock
    max_concurrent: 10

export AWS_BEDROCK_CREDS="AKIAIOSFODNN7EXAMPLE:wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY"

Busbar signs each outbound request with SigV4 (region parsed from the host); your clients call Busbar normally with a Busbar token. OpenAI-format clients can reach Bedrock backends this way with no SDK changes.

Bedrock ingress (acting as a Bedrock endpoint for native AWS SDK clients) has two tracks:

• Open/passthrough (auth.chain: [], optionally with upstream_credentials: passthrough): Busbar does not verify the inbound SigV4 signature. The credential is forwarded upstream (passthrough) or ignored (plain chain: []).
• With keys (auth.chain: [keys]): Busbar verifies the inbound SigV4 signature natively (crates/busbar/src/auth/mod.rs verify_bedrock_sigv4). Mint a virtual key with "issue_aws_credential": true via POST /api/v1/admin/keys; the response includes aws_access_key_id + aws_secret_access_key (shown once). Configure your Bedrock SDK with those credentials: Busbar verifies the signature, then enforces the key’s group limits and pool ACL. No passthrough required.

Injecting max_tokens for cross-protocol calls

If you route OpenAI-format requests to an Anthropic backend, Anthropic’s API requires a max_tokens field that OpenAI clients often omit. Busbar injects a default only on cross-protocol translation to a backend that requires max_tokens (Anthropic Messages) when the source omitted it. The default is 4096 unless you override it per model:

models:
  claude-sonnet:
    provider: anthropic
    max_concurrent: 20
    default_max_tokens: 8192

A caller-supplied max_tokens is always preserved; this only applies when the field is absent and the egress requires it. It has no effect on same-protocol passthrough.

---

Production checklist

Before taking Busbar out of dev mode:

• Set auth.chain: [keys] and mint a signed key per caller (keys expire; default 90 days)
• Enable inbound TLS: add a tls block (cert + key secret references) so the client-to-Busbar hop is encrypted, and, for zero-trust deployments, set client_ca to require client certs (mTLS). See docs/operations.md#inbound-tls--mutual-tls-mtls
• Consider setting max_concurrent on models where you want to cap in-flight load to your provider tier (optional; omitted = unbounded)
• Set max_requests to -1 (unlimited lifetime budget) or a finite positive budget per model
• Run busbar --validate (in CI and before every deploy/reload): parses and validates both YAML files with no server, no network, and no secrets required. Exit 0 = valid, 1 = errors. See operations.md#validating-configuration-busbar---validate
• Verify /healthz returns 200 and /stats shows all lanes usable: true before routing production traffic
• Consider health.mode: dead on providers you care about (re-probes tripped lanes so they recover faster after an outage clears)
• Set RUST_LOG=info (the default); increase to debug only temporarily for diagnostics

---

What’s next

• Deploy it: running Busbar for real — process configuration, TLS termination, the two-listener model, and running multiple instances (docs/operations.md)
• Full config reference: every field, default, and validation rule (docs/configuration.md)
• Pools, breakers, and failover: weighting, breaker tuning, session affinity, context-length failover, and exhaustion policies (docs/configuration.md#pools)
• Running in production: TLS termination, systemd, Docker, /stats monitoring, and breaker diagnosis (docs/operations.md)
• Governance: signed expiring keys, group limits, and the /admin API (docs/operations.md)
• Architecture: how the IR works, the six-protocol model, and why f64 instead of f32 (docs/architecture.md)