Comparison · same box, run it yourself

Busbar vs LiteLLM Rust

LiteLLM shipped the Beta of their Rust AI Gateway, with GA expected December 2026. It's the right idea, and on raw speed it's a tie. Both are compiled Rust cores, within a few percent on our own same-box benchmark. I'll show you those numbers. They're near the bottom of the page.

But speed was never the point, we just knew it would be a positive side effect of developing in a modern language like Rust. A funded team, three years in, built a Rust gateway that matches the performance of what a 3-month-old startup ships in a single binary. Take that as the floor. Here's the pitch.

What an OpenAI-normalized Rust passthrough can't do

1 · Any protocol in, any protocol out. That's the shape, not the language

Busbar's core is a six-protocol superset IR. OpenAI, Anthropic, Gemini, Bedrock, Cohere, and Responses all map into it and back out, both directions. A signed Bedrock boto3 client can reach a Gemini backend and get a valid Bedrock eventstream back. An OpenAI SDK can reach an Anthropic model and get OpenAI-shaped JSON. Every SDK speaks its native dialect in, every backend gets its native dialect out, and not one line of your application changes. How translation works →

LiteLLM normalizes everything to an OpenAI hub. Their Bedrock ingress load-balances across Bedrock deployments and nothing else. Point that boto3 client at a Gemini model: it can't. Point a Gemini client at a Bedrock model: same answer. This is not a speed problem you fix with Rust. A Rust rewrite of an OpenAI-normalized core is still an OpenAI-normalized core. The shape is the ceiling, and Busbar was built without it.

2 · Governance compiled into the binary, not bolted on in Python

Busbar's Rust core is the control plane. Virtual keys with total, daily, and monthly budgets. RPM and TPM limits. Per-pool ACLs, audit, usage metering. All of it in-process, on the hot path, in one binary.

LiteLLM's Rust gateway carries none of it. It's a bare passthrough, and their governance stays behind in the Python proxy they aren't rewriting. So with LiteLLM you choose one: fast Rust, or governance in Python. Busbar refuses the trade. It ties their latency while it authenticates, meters, and routes on every single request. They only match our speed because they do less on the way through.

And Busbar's governance needs no database to do it: enforcement runs from in-memory counters, so a single node is one binary, no PostgreSQL, no Valkey (Redis-protocol compatible). When you want that state durable or shared across a cluster, 1.5.0 adds drop-in SQLite, Postgres, and Valkey store plugins (signed archives Busbar verifies and loads; no recompile). Optional, not structural. LiteLLM's stack keeps PostgreSQL and Redis as prerequisites of the Python proxy where its governance lives.

3 · Shipped and complete today, not a beta waiting on 2026

All six protocols. Native in-flight failover across protocol families. A plugin system. One static ~11 MB binary. In production, stable, right now.

LiteLLM's Rust core is a single Anthropic /v1/messages endpoint serving only the azure_ai provider, with GA expected December 2026. Everything else still routes through the Python sidecar [1]. AI moves fast. You need the right control plane for your infrastructure today, not in six months.

On raw speed: a tie, both Rust, within a few percent. But speed was never the gap. A gateway forwards bytes. Busbar is a control plane: on every request it decides whose fault a failure was, whether you're inside budget and rate limits, which backend serves it, and whether to fail over mid-stream, all in the compiled hot path at that same speed. Six protocols in and out, governance in the core, shipped today. That isn't a faster gateway, it's a different category. So why run the one that only forwards?

The numbers, for completeness. Speed is a tie

I put both on one box and measured everything. Latency and throughput came out even; they're both compiled Rust, and I'm not going to pretend otherwise. Memory is the one gap, and it's Busbar's: it idles at 7.1796875 MB against LiteLLM Rust's ~253 MB.

Memory: Busbar serves at ~35× lighter idle

Idle RSS, serving the endpoint. Their floor is high because the serving config embeds CPython and the full litellm package. That's the "Python in the request path" cost their launch post warns about, still there. Why 253, not 21.8 → below.

Added latency & throughput: a tie

p99 added over the upstream, concurrency 1, 85k to 130k samples. Within ~10 µs. A tie.

And Busbar hits that number while authenticating, metering, and routing on every request. Their bare passthrough carries none of it. We match their speed doing strictly more work.

Sustained req/s at p99 < 1 s, zero errors. Max proxy is raw forwarding. @ 20 ms is concurrency under a realistic model delay. A tie.

Why ~253 MB, not the 21.8 MB in their launch post. From the branch their README links (litellm_rust_gateway_v1_messages_route):

/v1/messages serves only azure_ai; their own unit test asserts anthropic returns None.
It serves only under the python-config reader (embedded CPython + the full litellm package). The lean 21.8 MB config returns HTTP 400, so it doesn't serve.
The config that does serve idles at ~252.83203125 MB on a fresh Graviton box.

Fair's fair: their latency is good. Point me at a branch that serves anthropic natively at a lower footprint and I'll re-run it.

Where LiteLLM Rust is the right call

You already run the LiteLLM proxy and your traffic is Anthropic /v1/messages-shaped. Opting into their Rust core on the endpoints it covers is a low-friction win. No new tool to adopt.
You want the LiteLLM ecosystem (its admin UI, plugin catalog, and provider list), and the Rust core makes the hot path lighter over time without changing the surface you know.
You're validating the compiled-gateway idea on a narrow slice, and you're fine running beta software in that path until their GA (December 2026). If pre-GA code in front of production traffic is acceptable to you, it's a reasonable early adopt.

If that's your shape, it's a sensible choice, and I say so without hedging.

If this layer carries all your AI traffic (many protocols in their native dialects, one compliant backend, real failover, governance on the path, one thing to run), that's a control plane, not one accelerated endpoint. Get started in five minutes, read why Busbar exists, or if you're on LiteLLM today, here's how to migrate.

How this page is measured. Unless a figure says otherwise, every number here is from the neutral OSS harness (GetBusbar/benchmarking), an AWS m7g.2xlarge (Graviton3, 4-core pin), run the same way as /performance, measured 2026-07-31. Figures tagged "their published" are that project's own numbers; a core-scaling sweep, where one appears, is a separate test and is labelled as such. And you don't have to take our framing of the results: onthebench.ai publishes the field neutrally, every AI gateway measured identically, side by side, and anyone can re-run it.

References

LiteLLM Rust facts checked against their launch post on 2026-07-20. Something out of date? Tell us and we'll fix it.

LiteLLM Rust launch post: the announcement, early-beta status, staged rollout, Anthropic + Azure-Anthropic /v1/messages only (early beta), and the Python sidecar.
GetBusbar/benchmarking: the public, one-command harness behind every number here. Gateways, mock, load generator, methodology, raw JSON, charts.
Busbar protocols & translation: the six-protocol matrix and how cross-protocol requests are re-encoded natively both ways.

← Performance overview · Busbar vs LiteLLM (Python) · Busbar vs Bifrost