Comparison

Busbar vs Bifrost

Bifrost is a real, working Go gateway, and we tested it fairly. Busbar sustains 48,236 req/s to Bifrost's 5,341 under a realistic 20 ms model delay (~9x), adds 111 µs p99 vs Bifrost's 904 µs, and peaks at 314.28515625 MiB vs Bifrost's 0.9 GB. Where Bifrost is good, we say so; where its own published numbers didn't reproduce, we show our work.

Head-to-head

	Busbar	Bifrost
Added latency, p99 gateway minus direct	111 µs in-band on every response	claimed 11 µs measured 904 µs ~8x slower see below
Sustained RPS @20ms model delay	48,236 req/s	5,341 req/s ~9x less
Max proxy RPS instant mock, raw forwarding	48,236 req/s	5,938 req/s ~8x less
Peak memory sustained load	314.28515625 MiB idle: 7.1796875 MiB	0.9 GB idle: 219.73046875 MiB

Why a gap this wide between two compiled proxies? The two systems don't do the same amount of work per request. Most of an LLM proxy's cost is (de)serializing JSON bodies, and Go's encoding/json is reflection-based and allocation-heavy, so every request pays GC pressure a Rust proxy doesn't. Busbar's edge is bigger still on same-protocol traffic, where it passes the body through byte-for-byte with no re-encoding. And every figure here uses unique bodies on both sides, so both gateways actually proxy (fire identical requests and a gateway can answer from cache without proxying, which is not a real proxy number).

Memory

Busbar idles at 7.1796875 MiB and peaks at 314.28515625 MiB; Bifrost idles at 219.73046875 MiB and peaks at 0.9 GB, a ~3x peak difference on the same box and load. Bifrost's pools (Go GC, documented initial_pool_size 15000) fill under sustained load and don't release. Busbar runs on jemalloc with a background purge thread: buffers are freed when requests complete, and RSS falls to idle when the load stops.

Their "11 µs": and what the neutral harness measured

Bifrost publishes a "11 µs overhead." We quote it because hiding it would be the dishonest move, but we could not reproduce it, and neither can you: their own benchmark tool never emits it and their /metrics can't resolve it. Their docs say it "excludes JSON marshalling and the HTTP call," but for a network proxy that IS the work, and you can't measure a request's real cost without it from outside the process.

Added latency p99 (gateway minus direct, concurrency 1, instant mock) is 904 µs for Bifrost vs 111 µs for Busbar. That puts Bifrost at roughly ~82x its own published 11 µs, and ~8x Busbar on the same axis.

We reached out to the Bifrost team asking how the 11 µs is instrumented and whether it's reproducible from their tooling. We will update if we hear anything.

Busbar's own compute is reported in-band on every response: Server-Timing: busbar;dur, ~33 µs p99 on the 16-core c7g scaling run, a different box than the m7g head-to-head above (how end-to-end and busbar;dur relate).

And Busbar scales

A separate core-scaling test, not the m7g head-to-head above: same status as the 16-core anchor on /performance. One c7g.8xlarge (32 vCPU Graviton3) (Graviton, so one vCPU is one real core), gateway pinned to N cores, a dedicated load generator and fast mock on their own cores, and a unique body on every request so both gateways actually proxy. We swept 2 to 16 cores and recorded req/s at 100% success. Busbar 1.4.1 and Bifrost 1.6.4, measured 2026-07-19.

req/s at 100% success vs cores, same box, real proxy traffic. Both lines are straight: Busbar at ~9750+ req/s per core, Bifrost at ~1,147. Raw per-core CSVs and the load generator are archived under bench/. Hover any point for its numbers.

Where Bifrost is genuinely good

It scales cleanly at low-to-moderate load. 100% success across the throughput sweep, linear scaling, low latency when it isn't saturated or filling its pools. It is a working gateway. It just isn't "The Fastest LLM Gateway" its own banner claims: Busbar sustains ~9x its throughput, adds ~8x less latency, and holds ~3x less peak memory.
It scales vertically. Add cores, it adds throughput in a straight line, same as Busbar. The difference is efficiency per core, not whether it scales.
Bundled governance/telemetry out of the box. Some of its per-request cost buys features Busbar keeps opt-in. If you want that always-on, it's there.

How this page is measured. Unless a figure says otherwise, every number here is from the neutral OSS harness (GetBusbar/benchmarking), an AWS m7g.2xlarge (Graviton3, 4-core pin), run the same way as /performance, measured 2026-07-31. Figures tagged "their published" are that project's own numbers; a core-scaling sweep, where one appears, is a separate test and is labelled as such. And you don't have to take our framing of the results: onthebench.ai publishes the field neutrally, every AI gateway measured identically, side by side, and anyone can re-run it.

← Performance overview · Busbar vs LiteLLM · Reproduce it