Changelog
All notable changes to Busbar are documented here.
The format is based on Keep a Changelog, and this project adheres to Semantic Versioning.
Every release uses the same section headings, in this order: Added, Changed, Deprecated, Removed, Fixed, Security. Migration steps for a breaking change appear as a bold Migration item under Changed.
Z],
August 1, 2026
The config / identity / cost redesign release. 1.5.0 is a deliberate, tooled, breaking-for-operators
step: the config format changed shape (run busbar --migrate-config), and every 1.4.x virtual key
stops working and must be re-minted. It is still a minor version because the SemVer contract is
now stated honestly (see Changed): the frozen, SemVer-protected surface is the runtime an
application integrates against — the data-plane HTTP surface plus the six wire protocols — and that
surface is unaffected by this release: an app posting to /v1/chat/completions before the upgrade
gets a byte-identical response after it. What does change is auth: every existing virtual key must be
re-minted, a one-time credential rotation operators need to perform. That rotation is also the
release’s security headline: 1.x keys were bearer secrets that never expired, so a leaked key stayed
valid forever; 1.5.0 keys are signed tokens with a built-in expiry, so a leaked key eventually stops
working on its own.
-
The clean 1.5.0 config. One governing principle: the object that owns a concept is the only place it is defined, and the same kind of thing is expressed the same way everywhere.
module+settingsfor every loadable unit (store, secret, auth, hook); one limit shape; one secret shape; reference fields name the referenced thing (model,group,provider,module); windows are nouns (minute|hour|day|month|total);on_Xhandlers are keyword-bare or structured refs; an omitted list means “all” and an explicit[]means “none”, everywhere. Every operator struct rejects unknown fields (a typo fails boot, never a silent no-op). The canonical example lives atexamples/clean-config-1.5.0.yamland boots underbusbar --validate. -
groups:- the one limit tree. A group is a named enforcement bucket:{ parent?, enabled, limits: [...] }, forming an acyclic chain (any depth). A limit is generic:{ requests|tokens|budget: <amount>, per: <window> }, or{ concurrent: <n> }(instantaneous in-flight gauge, no window). Admission walks the chain up throughparentand ANDs every limit of every group atomically (all-or-nothing charging); the rejection names the exact blocking bucket (group + metric + window) withRetry-Afterfor rolling windows.enabled: falsefreezes a group (and every descendant) while keeping its history. Requests are enforced precisely; tokens are best-effort post-paid (the old TPM posture); budget derives at check time from the token ledger x the current rate card + the flat fee;concurrentholds release automatically when the response stream completes. -
Pool-scoped limits (
pool:qualifier) - the per-tier budget split. A windowed limit may carrypool: <name>, so it accounts and enforces per(group, pool)instead of group-wide:{ budget: 5000, per: month, pool: frontier }+{ budget: 5000, per: month, pool: value }carve one team’s spend across model tiers, each in its own ledger bucket (group:<name>@<window>#<pool>). Exhausting the frontier budget blocks only frontier traffic (the rejection names the pool - the caller’s expensive calls stop while cheap ones continue); group-wide limits still AND over all traffic. Admission charge, token accrual, and non-2xx refunds share one participation predicate, so what was charged is exactly what refunds. The named pool must exist (validated at boot /--validate/ Admin API); the hook seam’sBudgetBucketStateand the Admin groups read (LimitView) carry the pool scope. Also gone: the arbitrary 8-level group-depth ceiling (the cycle check is what bounds the walk; hierarchy depth is the operator’s call). -
Budgets that teach (
on_exhaust: downgrade). A pool-scoped budget limit may declareon_exhaust: downgrade, downgrade_to: <pool>: when it runs dry, the request is re-admitted and dispatched through the downgrade pool instead of refused - the dev’s expensive calls get cheaper, not blocked (teach by gravity; omit for today’s hard rejection, teach by friction). The charge lands on the effective pool’s buckets (accounting follows the traffic), the key’s pool ACL re-runs on every hop (an exhaustion can never route a key into a pool it may not use), and cascades are cycle-bounded. Where several budgets merge into one bucket, the most restrictive cap’s behavior governs. Validated at the door:downgradeneeds a different, existingdowngrade_topool, apool:scope, and thebudgetmetric. -
Keys are pure auth, and they expire. A minted key is a busbar-signed token
{sub, exp, kid}(ed25519). Verify = signature + expiry (stateless) + a small revocation denylist; policy (the boundgroup,allowed_pools) is resolved from the store bysub, so policy is mutable without re-issuing the credential. Keys carry no limits - a key resolves to a group, and a key with no group is authed + unlimited (access only). Mint body:{ name, group?, allowed_pools?, labels?, expires_in|expires_at?, issue_aws_credential? }(default lifetime 90 days). Revoke = denylist entry, live across every store backend. The signing key isauth.signing_key(a secret reference, fleet-shared); absent, busbar generates one 0600 on first boot. Rotating it revokes every outstanding key. -
Secrets are plugins (
kind: secret). Every secret value in config is a secret reference:{ env: VAR },{ file: /path }(the built-in secret modules), or{ module: <secret-plugin>, settings: {...} }for third-party sources (vault, cloud secret managers) loaded through the same signed-plugin trust pipeline. Applies to providerapi_key,auth.signing_key, the admin token, and every TLS cert/key/CA. No*_envsuffix fields remain. -
Hooks are plugins - no
hooks:registry block. A hook instance is referenced inline where it runs: inpools.<p>.hooks(ordered) andglobal_hooks(ordered). A bare name is a built-in ordering strategy (weighted | cheapest | fastest | least_busy | usage); everything else is a module ref{ module: webhook|socket|<kind: hook plugin>, settings: {...}, kind?, timeout_ms?, on_error?, prompt?, user?, priority?, at? }. The socket/webhook transports are built-in hook modules (settings.path/settings.url), so out-of-process hooks persist; the registry and theglobal:/default:flags are gone (subsumed by the two lists).module: webhook|socketrefs are the out-of-process transport modules; akind: hookplugin name routes through the plugin loader and runs in-process over the hybrid ABI (see the Unified plugin model entry below). -
Runtime-mutable groups on the Admin API - self-service governance. The
groups:limit tree is now editable live over the Admin API, so per-team and per-user budgets change without a restart:GET /api/v1/admin/groups+GET/POST/PUT/PATCH/DELETE /api/v1/admin/groups/{name}. A write validates at the door (the whole tree is re-checked - parent exists, acyclic, depth - so an invalid edit is a 400 that changes nothing), then the enforcement projection is rebuilt in place (limits live on the next request) while the token ledger survives, so past accrual is preserved.PATCHis the ergonomic per-field verb (“raise Alice’s budget” = send justlimits; “freeze a team” =enabled: false). Because a group is{ parent?, enabled, limits, child_default? }and org/team/user are the same primitive, a user is just a leaf group parented to their team: a personal budget is the leaf’s own limits, always sub-capped by the team ceiling (the chain ANDs, so over-allocating personal budgets can never sum past the team pool). Every mutation is audited, bumpsconfig_version(optimistic concurrency viaIf-Match), and is written to the config overlay so it survives a restart; a base-config group is file-owned (a 409 - edit config.yaml). New optionalgroups.<g>.child_defaultseeds the limits of children auto-provisioned under a group (nearest-ancestor-wins). -
Per-group read surface (§6d).
GET /api/v1/admin/groups/{name}/usagereturns the group’s derived current-window usage, one row per(window, pool?)enforcement bucket its limits materialize:{group, enabled, buckets: [{window, pool?, requests, tokens, spend_cents, ...caps and budget_remaining_cents...}], as_of}. Spend is repriced at read time from the token ledger x the currentrate_card(nothing dollar-shaped is stored);bucketsis empty for a group with only aconcurrentlimit.GET /api/v1/admin/keys?group=<name>lists the keys bound to a group — a leaf group’s keys are one person’s keys; a team group’s are the team’s (exact bound-group match; no existence check, so dangling references remain findable). -
Self-service mint: auto-provision + the delegated
mintscope.POST /api/v1/admin/keystakes an optionalparent: whengroupnames a leaf that does not yet exist andparentis an existing group, the leaf is auto-provisioned under it (limits stamped from the nearest-ancestorchild_default, inherit-only when none) through the same validate-at-the-door group-write path - so the first self-mint materializes auser:<sub>personal budget bucket, binds the key, and the new leaf is live in the enforcement chain (leaf ∩ team ∩ org). If the group already exists,parentmust match its actual parent (a 409 - a mint never re-homes an existing group). A new delegatedmintadmin scope lets a customer’s self-service portal mint keys (and auto-provision) without god-modefull.mintandhooks-registerare siblings, not ladder rungs: a mint credential cannot register hooks and a hooks-register credential cannot mint (the authorization check is a diamond lattice, not>=). New optionallimits.max_keys_per_principalcaps how many keys may bind to one group (= one principal, since a user is a leaf group) - an over-cap self-mint is a 409; absent/0= unlimited (today’s behavior). -
Config overlay substrate (
BUSBAR_CONFIG_OVERLAY). Admin-API config mutations layer onto a busbar-owned overlay file (never the operator’s baseconfig.yaml); the effective config = base- overlay, re-merged at boot and re-validated on every hot-apply. Atomic write (temp + rename), per-section tombstones for deletions, and a loud refusal to overwrite a corrupt overlay. This is what makes the runtime-mutable groups (above) durable across restarts.
-
Per-section overlay reset - the audited revert-to-config.yaml front door.
DELETE /api/v1/admin/overlay/{section}(section ∈groups|hooks) discards every overlay mutation for that one section and reverts it to what baseconfig.yamldeclares: agroupsreset restores the base limit tree (cost model rebuilt), ahooksreset restores base hooks (registry/gates/rewrites rebuilt), each leaving the other section’s runtime mutations untouched. Full scope,If-Matchoptimistic concurrency, audited + versioned, and the cleared overlay is persisted so the revert survives a restart. A section with no overlay state is an idempotent no-op (changed: false, version unchanged); an unknown section is a 400. The revert re-reads disk truth (the same boot pipelineconfig/reloadruns), so an ephemeral busbar with no config files has nothing to revert to and 400s. -
auth.role_bindingsnested by module.role_bindings.<module>.<role> -> { allowed_pools?, group?, admin_scope? }- pure auth. A role asserted by one module can never ride another module’s binding (ad.platform!=oidc.platform); an unbound role grants nothing (fail closed). Admin access = a role’sadmin_scope(ceilinged by the asserting module’smax_admin_scope) OR theadmin-tokensoperator credential, now a secret reference under the module entry itself. -
The 1.5.0 cost model: tokens are the ledger, dollars are derived. The store accumulates an immutable token ledger per (bucket, window, model, tier) - input, output, cache-read, cache-write - and every spend figure (enforcement, admin reads, metrics, hooks) is computed at read time as
tokens x rate_card + requests x per_request_fee. Nothing dollar-shaped is stored or crosses the store wire, so correcting a rate is a config edit + reload: historical and future derived spend become right on the next read, with no re-billing and no data migration. (Honest limit: repricing cannot un-make past admit/reject decisions taken under a wrong rate.) -
Top-level
rate_card:- the only cost source. Per-model, per-tier token rates in micro-units per token of an abstract cost unit (busbar attaches no currency). All-or-nothing: absent = every model’s tokens price at 0 (budgets count onlyper_request_fee); present = authoritative and complete - every configured model must have an entry or boot/--validatefail with a paste-ready zeroed stub. A request for an arbitrary passthrough model with no rate is rejected pre-forward. Routing’scheapeststrategy derives its scalar from the card; pool members carry no cost. -
busbar --migrate-config <old.yaml>. Mechanically converts the deterministic 1.4.x changes (governance:->store/rate_card/per_request_fee/groups/advanced;group_map->role_bindingswith inline caps moved into a generatedgroups:entry;api_key_env->api_key: { env: ... }; membertarget->model; thehooks:registry -> inline refs; breaker/failover aliases;price_per_1k_tokens_cents-> synthesizedrate_card;otlp_endpoint->otlp_url) and prints TODO comments where a human must decide, with a loud warning on everyallowed_pools: []occurrence (its meaning flipped: it used to mean all pools, it now means none). Zero side effects: the new YAML goes to stdout, the change summary to stderr. -
Loud fail-closed boot on a 1.x config. Boot and
--validatedetect the 1.x structural markers (agovernance:block,auth.group_map:,auth.mode:, a top-levelhooks:block,*_envsecret fields,target:in a pool member) and refuse to start: “this looks like a busbar 1.x config; runbusbar --migrate-configand review the flagged items.” Nothing from 1.x can boot-and-silently-flip semantics. -
Store plugins: durable backends load from a dynamic library, and the default binary is lean. SQLite, Postgres, and Redis stores ship as signed plugin tarballs loaded in-process at boot over a versioned store C ABI when
store.modulenames them (memory, the compiled-in RAM store, is the zero-setup default). New workspace crates:busbar-plugin-abi,busbar-plugin-sdk(export_store_plugin!),busbar-plugin-loader(all FFIunsafeisolated; the engine keepsforbid(unsafe_code)). Store calls are write-behind, off the request hot path. Postgres/Redis are the shared multi-node stores (keys, usage, audit, denylist cluster-shared); the budget hard cap remains per-node between additive flushes (documented fleet honesty). -
The dynamic plugin system: one signed tarball per plugin, a top-level
plugins.*block, and a three-phase fail-closed load pipeline. Store, secret, auth, and hook plugins share one artifact format, one trust model, one loader, discriminated by the manifestkind. Phase 1 structural (in-memory unpack, manifest completeness, sha256, abi gate - any invalid tarball in an enabled dir aborts boot naming file + reason); phase 2 trust (busbar’s release key is embedded - first-party plugins verify with zero config and are auto-floored at the binary’s version;trust.publishersallowlists third-party ed25519 keys;trust.allow_unsigned/trust.allow_third_partyare explicit opt-ins, both defaultfalse, and an untrusted plugin is logged and skipped - neverdlopened); phase 3 conflict (no two loadable plugins share a name/alias). The loader maps exactly the verified bytes (memfd_createon Linux; a private0700staging dir elsewhere) - a pre-existing on-disk library is never loaded, closing the verify-then-load TOCTOU.plugins.min_versionsadds per-plugin anti-downgrade floors. -
Auth plugins load over the same signed hybrid ABI as store and secret plugins. A
kind: authplugin is now a real runtime identity provider: name it inauth.chain(anything but the built-inkeys) and the engine resolves it against the plugins directory and loads it in-process at boot throughregistry.open_auth— the identical trust posture, loader, and fail-closed pipeline that back store and secret plugins. Store, secret, and auth plugins now genuinely load over the one signed hybrid ABI (an earlier note that implied a single loader already carried all plugin kinds was ahead of the code forauth; it is now true for these three). The plugin returns identity only (Principalid + roles); busbar maps those roles throughauth.role_bindings.<module>— keyed by the plugin’s runtimename()— to pools, group limits, and an admin scope capped byauth.chain.<module>.max_admin_scope. Fail-closed everywhere: a configured auth plugin that cannot load (missing/untrusted tarball, wrong kind,plugins.enabled: false, or an ABI failure) is a hard boot/apply error — a dropped front-door module can never silently open the door.busbar --validatecatches it manifest-only ahead of boot, andGET /api/v1/admin/plugins?type=authreports a loaded auth plugin. The bundledoidcmodule (busbar-auth-oidc-plugin,auth.chain: [oidc]) is the first such plugin. Hook plugins are in-processdlopenplugins too (1.5.0 retired the out-of-process socket/webhook hook transports): a hook is now a signedkind: hookplugin loaded in-process as a routing policy, sharing the same artifact, trust, and inventory machinery as the store/auth plugins. -
Admin plugin API. The admin surface manages the plugin catalog over its own versioned contract: list/inspect installed plugins (manifest, signature verdict, load status), install a signed tarball, and remove one - with the same trust pipeline as boot (an untrusted upload is refused, never loaded) and same-name/different-file conflicts rejected.
-
busbar --validatecovers the whole new surface with paste-ready fixes: rate-card completeness (zeroed stubs of exactly the missing models), groups tree faults (missing parent stub, cycle path, depth), role_bindings module/role checks, secret-module resolvability, key group existence, plugin pre-flight (consistency, trust, three-phase scan, store resolution) - the same code path boot runs, so a clean--validatemeans a clean boot. Newbusbar --list-pluginsprints the manifest-only inventory without loading plugin code. -
Limit-dimension metrics. Scrape-time gauges are keyed by the new enforcement dimensions:
busbar_bucket_spend_cents/busbar_bucket_budget_remaining_cents/busbar_bucket_tokens{model,tier}carry{bucket, group, window}labels, one series per (group, window) bucket, all derived from the token ledger at the current rate card at scrape time. Mint-time keylabels(e.g.{"team": "growth"}) echo onto per-key series so external dashboardssum by (team)without busbar knowing what a team is. -
Budget state on the routing hook seam.
RoutingContext.budget(and the webhook/socket wire’scontext.budget) carries{bucket_id, budget_group?, spend_micros_at_current_rate, remaining_micros?, window_start, budget_period}for the caller key and every ancestor group bucket, so a hook can downshift to a cheaper model as a bucket nears its cap. -
PGO release builds. Host-native release binaries are built through a profile-guided optimization pipeline (instrument, replay a representative traffic profile, rebuild), as part of the standard release workflow.
-
Upstream client sharding (perf). The outbound HTTP client pool is sharded
min(cores, 16)ways (rounded to a power of two) and each worker thread is pinned to one shard on first use, so the shared pool lock is contended by ~1/N threads. Warm connections and TLS sessions stay worker-local; per-host idle budgets are divided across shards so the total kept-alive connections are unchanged. The shard count is machine-derived, not configurable. -
Semaphore skip for unbounded lanes (perf). When
max_concurrentis omitted on a model (the default — unbounded), the admission path skips the semaphore’s shared atomics entirely. The/statsinflightcounter for those lanes reads a per-lane fast path instead, so the hot path for the common case never touches a shared atomic. -
Per-thread telemetry bank (perf). Hot-path metrics (request counters, histogram samples) now accumulate in per-thread cells (the telemetry bank) and are drained into the recorder at scrape time, taking span and counter updates off the shared atomic path on every request.
-
Unified plugin model — hooks are
kind: hookdlopen plugins. The old socket/webhook hook transport is retired as the only mechanism; hooks now load as signed in-process plugins over the frozen hybrid ABI (the same six kind-neutral symbols:busbar_abi/busbar_plugin_kind/busbar_open/busbar_call/busbar_free/busbar_close). Kind is bound at load via the signed manifestkindfield cross-checked againstbusbar_plugin_kind(). Trust is signature-based, not process-isolation. The socket and webhook transports remain as built-in hook modules for operators who want out-of-process hooks. A plugin cannot self-grant access: the signed manifestneedsfield declares intent at pack time; the core enforces the actual projection.busbar_plugin_sdk::export_hook_plugin!is the SDK macro. -
Headroom and webrequest — the two shipped 1.5.0 hook plugins. Headroom (
busbar-headroom-hook) is a first-partykind: hookprompt-compression rewrite gate that reduces context before dispatch, saving tokens and latency. Webrequest (busbar-webrequest-hook) is a first-party signed HTTP-forwarder hook — the isolation story for code you don’t want in-process: it forwards the routing projection to an operator-run HTTPS sidecar, so the operator gets out-of-process isolation without running an untrusted library in busbar’s address space. Both plugins are signed by release CI and auto-trusted by the embedded key. -
Full-config coverage (
GET/PUT /api/v1/admin/config/settings). EveryRootCfgsection is now API-settable and overlay-persisted.GETreturns the current overlay root-section state;PUTaccepts any subset of the root config (rate_card,per_request_fee,security,limits,observability,advanced,metrics,health,routing,listen,tls,admin_listen,admin_tls,store, …), merges it into the overlay, re-resolves, and swaps in.config.yamlis never written (persistence is the busbar overlay, atomic temp+rename). The response names which fields are live (rate_card, per_request_fee, security, limits, observability, advanced, metrics, health, routing — hot-applied immediately) and which are restart-to-apply (listen/admin_listen socket binds, tls/admin_tls binds, admin_insecure, store backend — bound once at process start, store reused across hot reloads; stored durably but take effect on next restart). -
POST /api/v1/admin/restartapplies the restart-to-apply settings above without a shell: drains through the same path a signal takes (final budget flush, state snapshot, tracing shutdown), responds202before the drain begins, then exits. Body is optional ({"confirm": bool?}; absent ={}); refuses with409 conflictwhen no process supervisor is detected andconfirmwas not set, or when the process cannot restart itself. Full scope. -
Hot plugin reload + explicit rollback.
POST /api/v1/admin/plugins/reloadis now a live hot swap: re-runs the fail-closed plugin pipeline from disk+overlay, rebuilds the registry andkind: hooktransports, and old libraries drain then unmap — no restart. Fail-closed: a bad artifact leaves old plugins serving.POST /api/v1/admin/plugins/rollbackis an explicit, audited, If-Match-guarded rollback (body:{"file": "<tarball-filename>"}) that pins a prior version and lowers the anti-downgrade floor only for the operator’s action — automatic silent downgrade stays refused. Both are Full scope and audited. Store-backend live swap is not in 1.5.0; a store module change still requires a restart.
Changed
Section titled “Changed”- MSRV corrected to Rust 1.97. The declared
rust-version(1.87, set whenu32::is_multiple_offirst required it) had drifted silently: no CI job ever built at that pin, and a transitive dependency (redis) had since raised its own floor past it, socargo buildon a real 1.87 toolchain has failed for some time. CI has always floated ondtolnay/rust-toolchain@stable(currently 1.97), sorust-versionis corrected to match what is actually verified rather than an unverified, already-false claim. - The SemVer contract is redefined (and this is what makes 1.5.0 a minor). The stable,
SemVer-protected contract is the runtime: the data-plane HTTP surface an application
integrates against and the six wire-protocol contracts. The
config.yamlis an operator deployment artifact (nginx/postgres/envoy precedent), explicitly outside the SemVer freeze: it may change between releases, always with a migration path (busbar --migrate-config) and a loud fail-closed boot on an outdated config - never a silent behavior change. The admin API is its own versioned contract (/api/v1/admin); a break there is expressed by the admin contract version, not the binary version. Existing keys stopping is a credential rotation (see the release header), not an API break. README/docs state the same scope. - Migration (operator steps). 1)
busbar --migrate-config old-config.yaml > config-1.5.yaml, review every WARNING/TODO comment (especially eachallowed_pools: []occurrence - the all->none flip),busbar --validate. 2) Re-mint every virtual key (POST /api/v1/admin/keys) and roll the new signed tokens out to callers; 1.4.x bearer secrets and staticclient_tokensno longer authenticate. 3) A durable store is dropped and recreated on first open (schema v3; 1.5.0 shipped no earlier stable schema): usage history resets with the schema. - The admin
keysresource is pure auth.key_metais{id, name, allowed_pools, group, enabled, created_at, labels}(allowed_pools: null= all pools,[]= none).PATCH /keys/{id}takes{enabled?, group??}(three-state group: absent = unchanged,null= unbind, value = rebind to an existing group); the 1.4.x cap fields are rejected.GET /keys/{id}/usagereports the key’s all-time attribution bucket + chain-derivedrate_headroom. - Store contract + plugin ABI v3 (breaking, pre-release). v2 (this cycle): the scalar
Usagebecame the per-(model, tier) token ledger (UsageLedger/UsageDelta), re-keyed fromkey_idtobucket_id, with additive fleet flushes. v3 (this release):VirtualKeyis pure auth - inline limits dropped,budget_grouprenamedgroup,allowed_poolsre-encoded as nullable (C6), and the denylist surface added. sqlite (PRAGMA user_version), postgres (busbar_schema), and redis (busbar:schema) stamp schema v3 and drop a pre-v3 dev schema on open (1.5.0 unreleased: no stable schema existed to migrate). - Governance is always available; enforcement is in-memory on the request path. No on/off
switch: governance is inert until keys exist. The per-request admission is an atomic in-memory
check-and-charge over the whole group chain (no store round-trip, no await); the durable store
is a write-behind layer (boot-hydrate, periodic additive flush, final flush on graceful
shutdown). Group ledger buckets are per-(group, window):
group:<name>@<window>. - Docs corrected alongside (audited overstatements). Budget accuracy is stated as derived-from-the-rate-card; scoping is per-key attribution / per-group enforcement; the store default is documented as in-memory; the budget hard cap carries the per-node fleet caveat everywhere it is described.
prompt: ro/rwhook projections now include reasoning/thinking text (behavior change, not just a bugfix). Anthropicthinking, BedrockreasoningContent.reasoningText, and Responsesreasoningblock text now flow into the flattenedsystem/messagesprojection like any other text block, closing a screening bypass (see Security). This widens what an already-opt-in,full-scope-gated grant exposes: an operator who wiredprompt: robefore this release for PII/DLP screening will now also see replayed chain-of-thought text in that projection (and, for aprompt: rwhook that echoes its projection verbatim rather than abstaining, that text — or the redacted-reasoning marker — can be promoted into a real, visible content block on the outgoing request via the pre-existing, not-index-alignedapply_rewrite_to_bodywrite-back; seedocs/hooks.md’srewritearm). Awareness, not a migration step: no config change is required or possible to opt back out — the previous behavior was the bug. If your hook logs or forwards thepromptprojection somewhere less trusted than the hook itself, review that path against reasoning content now being present. Opaque redacted reasoning (Anthropicredacted_thinking, BedrockredactedContent, a Responsesreasoningitem carrying only an opaqueencrypted_contentblob) is unaffected by this widening — it still never appears as plaintext, only as the fixed[busbar:redacted_reasoning]marker.
Removed
Section titled “Removed”- Hooks-as-socket/webhook as the only hook model retired. The retired out-of-process-only hook model (where every hook ran in a separate process) is replaced by the unified plugin model. Socket and webhook persist as built-in transport modules but are no longer the only option.
- The
governance:block. Its contents dissolved:governance.store/db_path->store: { module, settings };governance.rate_card-> top-levelrate_card;governance.price_per_request_cents->per_request_fee;governance.budget_groups->groups(broadened to the generic limit tree);governance.admin_token-> theadmin-tokensmodule’stokensecret ref;governance.rate_sweep_interval/usage_flush_interval_ms->advanced.governance.enabledandgovernance.budget_on_store_errorare gone (always-on, and admission never touches the store). - Static tokens. The
tokens/static-tokensallowlist module andauth.client_tokensare removed. Data-plane auth = the built-inkeyssigned-token verifier + IdP auth modules. - Per-key limits.
rpm_limit,tpm_limit,max_budget_cents,budget_periodare gone from mint, PATCH, the store schema, and the key metadata: every limit lives on the bound group. The per-keybusbar_key_budget_remaining_centsgauge is gone with them (use the group bucket gauges). - The top-level
hooks:registry block and the hookglobal:/default:flags (inline refs inpools.<p>.hooks/global_hookssubsume all three). - Config aliases.
window_s(->window_secs),n(->consecutive_n),deadline_secs(->timeout_secs),cap(->max_hops),otlp_endpoint(->otlp_url), membertarget(->model),api_key_env(->api_key: { env: ... }),auth.mode(->auth.chain/auth.upstream_credentials). One canonical name each; unknown keys fail boot. cost_per_mtokon pool members andgovernance.price_per_1k_tokens_cents.rate_cardis the only cost source (--migrate-configsynthesizes equivalent card entries from the flat price and flags them for review).
- Budget-cell straddle wipe. A request straddling a window boundary could rewind a newer live budget cell to its older admission window, zeroing the live window’s accrued tokens/requests and flush baselines; the atomic chain charge and accrual paths now only reset genuinely stale cells.
- Derived-spend integer wrap. An adversarially large ledger (u64-scale tokens x a large
configured rate) could push the cent total past
i64::MAXand wrap negative, deriving as free and bypassing every budget cap; derivation now saturates ati64::MAX(blocks, fail-closed). - Requests-limit refund escape. The flat per-request fee bills 2xx only (a non-2xx outcome
refunds it), while the
requestslimit counts admissions. Backing both with one counter would let a caller escape the requests cap by hammering failing requests (each refunds its own slot, so the cap only ever counted successes). The ledger now tracks the admission count (never refunded, the requests-limit truth) separately from the billable count (the fee base, refunded on non-2xx). - Boot fail-open on budget hydration. A store error while hydrating accrued ledgers at boot silently started with empty cells (a maxed-out key could spend its whole cap again); boot now fails loudly instead of resuming unenforced.
- Bucket-namespace collisions. IdP-influenced principal ids shaped like
vk_...orgroup:...could alias a real key’s or group’s ledger bucket; both prefixes are reserved and such principals are refused a synthetic key (fail closed). - mTLS typo hole. Operator structs (
tls:, providers, limits, metrics, routing…) now reject unknown fields, so a typo likeclient_c:forclient_ca:fails boot instead of silently disabling mTLS. - Write-behind flush lost-update. Concurrent budget flushes could double-send an in-flight delta against an un-advanced baseline; flushes are serialized (skip-if-still-flushing; shutdown waits for the in-flight flush).
- Durable-audit gaps + unbounded restore. Transient durable-write failures self-heal, and the boot restore reads only the bounded tail it keeps.
- Store correctness batch. Redis:
append_auditupserted on the wrong key (duplicate audit entries), non-idempotent writes are no longer retried, the schema-wipe guard cannot fire on an evicted marker, atomic multi-key writes + reconnect + TLS support. Postgres: a transient version-read error is never conflated with a fresh database;GREATESTparameters carry explicit::bigintcasts. All stores scrub DSN passwords (raw and percent-decoded) from every error string. - Admin hardening batch. Mint-time
labelsare validated at the ingress (protecting Prometheus exposition); a same-name-different-file plugin install is rejected; plugin-scan errors propagate instead of vanishing; a malformedon_exhaustedis caught by--validate; the adminUsageViewregained its documentedcurrencyfield.
Security
Section titled “Security”- Keys expire and revoke (the release headline). 1.x virtual keys never expired: a leaked bearer secret was valid forever unless an operator noticed. 1.5.0 keys are signed tokens with a mandatory expiry (default 90 days), stateless verification, and a durable revocation denylist that is enforced fleet-wide and survives restarts. Rotating the signing key revokes everything at once. Upgrading forces the rotation.
- Plugin supply-chain hardening. Untrusted plugin code is never executed: unsigned,
tampered, or unknown-publisher tarballs are logged and skipped without being
dlopened (trust.allow_unsigned/trust.allow_third_partyare explicit opt-ins, both defaultfalse- safe by default). The loader maps exactly the verified bytes (memfd / private staging dir), closing the verify-then-load TOCTOU; first-party plugins are automatically floored at the binary’s version and third-party floors are configurable (plugins.min_versions), closing signed-but-old downgrade replays. - Env-var YAML injection closed. Interpolated
${VAR}values are rejected if they carry YAML-structural control characters, so a compromised environment cannot splice extra config nodes (e.g. widen an auth allowlist) through a quoted scalar. - Remaining flow-collection env-var YAML injection closed. The control-character check above
only stops the newline-based breakout; a value containing a bare
,,", or'could still splice extra structure into a YAML flow collection ({ }/[ ], e.g. this project’s own documentedclient_tokens: [ "${VAR}" ]pattern) or an opaquesettings:map with no newline at all — a flow sequence has no schema defense against an extra element, and a genericserde_json::Mapsettings block has nodeny_unknown_fieldsequivalent against an injected sibling key. Rather than enumerate more forbidden characters (which is both under-inclusive — misses anchor/alias redefinition — and over-inclusive — breaks legitimate LDAP DNs, JSON blobs, and Windows paths), interpolation now runs a structural-equivalence check: the template is interpolated a second time with each${VAR}replaced by an inert placeholder, both results are parsed, and the two trees must have identical map keys, sequence lengths, and node kinds at every position (scalar leaf content and inferred type are exempt, since real values are expected to differ there). Any shape difference is rejected, naming the responsible variable where it can be isolated. - Reasoning-block
prompt: ro/rwhook-visibility bypass closed.flatten_content/total_text_chars/system_text_chars(proxy/hooks.rs) probed content blocks only for atextfield, so Anthropicthinking, BedrockreasoningContent.reasoningText, and Responses summary-onlyreasoningblocks were invisible to aprompt: roscreening gate even though they ship to the provider in full — an operator-owned PII/DLP/guardrail policy-enforcement point silently passing content it was deployed to inspect. Two of the three paths need no forged/valid signature at all to reach the provider: Bedrock’s writer has no unsigned-drop filter (unlike Anthropic’s egress path), and the Responses reader admits areasoningitem on either real text or a non-emptyencrypted_contentblob alone. All three now route through a protocol-dispatched extractor (keyed on the parsed ingress protocol, not on which JSON field happens to be present) and are projected/counted like any other text. Opaque redacted reasoning (redacted_thinking/redactedContent, which busbar cannot decrypt) is never exposed as plaintext — it projects as a fixed, non-authenticated presence marker instead of silently vanishing. A follow-up audit found theencrypted_content-only half of the Responses OR was itself still a bypass after the above: areasoningitem admitted purely on its opaqueencrypted_contentblob (nocontent[]/summary[]text at all) produced an empty string from the text extractor and read as “nothing here” instead of the redacted-reasoning marker, and separately,content: [](present but an empty array, not absent) on areasoningitem bypassed the reasoning-item dispatch in bothtotal_text_charsandbuild_prompt_projectionentirely (it took the empty-array walk instead of ever reaching the reasoning extractor). Both are now fixed: the extractor treats anencrypted_content-only item the same as the other two redacted-reasoning shapes, and both callers check the item’stypebefore itscontent’s shape, socontent: []no longer shadows the reasoning dispatch. Also fixes a related role-inference bug: a Responsesreasoningitem was misattributed touserinstead ofassistant, contradicting the protocol reader’s own mapping on the same body. See Changed for the resulting widenedpromptgrant scope.
July 20, 2026
- Published OpenAPI schema per release. Every tagged release now attaches the admin API’s OpenAPI 3.1
document as a release asset (
busbar-openapi-<tag>.json), emitted in CI from the sameopenapi_doc()that servesGET /api/v1/admin/openapi.jsonand stamped with the release version. Downstream tooling can generate a client or diff the API surface across releases without running the gateway.
Changed
Section titled “Changed”- Repository now at
github.com/GetBusbar/busbar(older links redirect). Release binaries, the GHCR image (ghcr.io/getbusbar/busbar), and build-provenance attestation are published under this repository; verify this release’s artifacts with--repo GetBusbar/busbar. Docker Hub (getbusbar/busbar) is unchanged.
July 19, 2026
-
jwt-beareregress auth (OAuth 2.0 JWT-bearer grant, RFC 7523): the 5th auth mechanism. A provider withauth: jwt-bearerself-mints a short-lived bearer by signing a JWT assertion (RS256) and posting it to the token endpoint, then refreshes in the background ahead of expiry. A Google service-account JSON is a recognized credential container (client_email→iss,private_key→signing key,token_uri→aud);scopedefaults tocloud-platformand is overridable. Any RFC 7523 provider works via config, no new code. -
oauth-client-credentialsegress auth (OAuth 2.0 client-credentials grant, RFC 6749 §4.4): the 6th auth mechanism.auth: oauth-client-credentialswithtoken_url+scopeexchanges aclient_id:client_secretfor a bearer and refreshes it. This is what authenticates Azure OpenAI via Entra ID (AAD). It shares the self-minting bearer-refresh machinery withjwt-bearer; the two differ only in the mint call. -
path_baseprovider knob: overrides a URL-model protocol’s hardcoded base segment while keeping the/{model}:verbsuffix, so one config line can reach a non-standard base path (e.g. Vertex’s/v1/projects/{project}/locations/{location}/publishers/{publisher}/models). -
Google Vertex AI, delivered as configuration. Gemini on Vertex (
protocol: gemini+path_base+auth: jwt-bearer) and Claude on Vertex (protocol: anthropic+path_base+auth: jwt-bearer; the model moves into the:rawPredict/:streamRawPredictURL and the body carriesanthropic_versionin place ofmodel). No new protocol: Vertex speaks Gemini and Anthropic on the wire, so it is a config entry, not code. Azure OpenAI via Entra ID lands the same way (protocol: openai+auth: oauth-client-credentials). -
token_urlandscopeprovider fields, consumed by the OAuth auth mechanisms above. -
Oracle OCI Generative AI, delivered as configuration. The
oci-genaicatalog entry targets OCI’s OpenAI-compatible surface (/openai/v1/chat/completions, serving OCI’s hosted OpenAI/Llama/xAI/Google/ Cohere models). Since Jan 2026 OCI issues plain API keys (Bearer), so no OCI request-signing is needed:protocol: openai+auth: bearerwith the OCI regionalbase_url. No new protocol or code; itsTooManyRequests/QuotaExceeded/LimitExceeded-in-a-400 quirk is handled by the catalogerror_map.The support surface is now 6 protocols × 6 auth mechanisms. To be clear on direction: these are egress auth mechanisms: how Busbar authenticates outward to each upstream AI provider (Busbar → provider), configured per provider in
providers.yaml. They are unrelated to how clients authenticate inward to Busbar (client → Busbar), which is the separateauth:client-token / virtual-key layer. Any upstream speaking one of the six wire protocols and one of the six egress auth styles is a config entry, no code change.
Changed
Section titled “Changed”- Default worker-thread count now scales to the box. The async worker pool defaulted to
min(cores, 4)in 1.3.1–1.3.3 (1.3.0 used Tokio’s all-cores default), which pinned the CPU-bound request path (parse, translate, serialize) to ~4 cores no matter how large the machine: throughput plateaued regardless of core count. The default is now one worker per available core (available_parallelism, which respects CPU affinity and the cgroup cpuset but not the CFScpu.maxbandwidth quota, which it cannot see), so throughput scales linearly with cores out of the box: ~9,750 req/s per core, ~156k req/s on 16 cores at 100% success in the published benchmark, with added latency flat at ~33 µs. On a quota-limited orchestrator (e.g. a k8s pod with a CPU limit on a many-core node) this defaults to the node’s core count and oversubscribes the quota. PinBUSBAR_WORKER_THREADSto your CPU limit there. Each worker carries a thread stack and its own allocator arena, so idle memory grows slowly with the count; footprint-sensitive sidecars should setBUSBAR_WORKER_THREADS=1(or2). No config or API change. For back-compat, an operator who pinned the standardTOKIO_WORKER_THREADSon 1.3.0 (honored by 1.3.0’s#[tokio::main]runtime) still gets that pool size: it is read as a fallback whenBUSBAR_WORKER_THREADSis unset; an explicitly-set-but-invalid value warns instead of being silently ignored. The reproducible throughput/scaling harness and raw per-core data live in the external benchmark repoGetBusbar/benchmarking. - Allocator: jemalloc with a background purge thread. The request hot path holds a few copies of each
request body while it is parsed and forwarded, so peak RSS tracks
peak concurrency × payload size. The system allocator (glibc) almost never returns freed pages to the OS, so after a big-payload burst RSS stayed pinned at the peak indefinitely: memory read as a one-way ratchet even though the live set had collapsed. Busbar now uses jemalloc withbackground_threadenabled: freed pages return to the OS after a short decay, so memory plateaus under sustained load and falls back toward idle when the load subsides (measured: a ~1.2 GB plateau under a 5-minute 150 KB-payload soak drops to ~250 MB within ~30 s of the load stopping). It remains bounded, a function of in-flight work, never unbounded growth. Cost: ~450 KB of binary and four new dependency crates (tikv-jemallocator/tikv-jemalloc-ctl/tikv-jemalloc-sysplus thepastebuild-macro dep), all vendored under the Apache-2.0/MIT compatible set. Reproduction harness in the external benchmark repoGetBusbar/benchmarking. Windows (msvctarget) keeps the system allocator: jemalloc’s C build is incompatible with the MSVC toolchain, so it is compiled in only for non-msvc targets (Linux/macOS, incl. the published container and static musl builds). Windows binaries build and run unchanged on the system allocator and do not get the plateau/fall-back-to-idle behavior. ringandbase64are now direct dependencies (both were already in the lockfile via rustls), used for the RS256 JWT signature and the PKCS#8/base64url handling injwt-bearer. No new crates enter the tree.- Streaming: for a cross-protocol stream whose backend reports token usage in a separate trailing chunk
(the OpenAI
include_usageconvention), the terminal usage frame is now deferred and the trailing usage folded into it, so a non-OpenAI client (Anthropic/Gemini/Cohere/Responses) receives the real prompt/ completion counts on its terminal frame instead of zeros. Delivery is uniform across the SSE and gemini-json-array paths (the response body now streamsfinish()’s content through the json-array framer, which previously discarded it). Behavior-preserving for OpenAI ingress (which still receives the separate usage chunk) and Bedrock ingress (which carries usage in itsmetadataframe). Wire-shape note: a Gemini JSON-array (non-SSE) client on a cross-protocol stream now receives one additional trailing array element carrying the terminalusageMetadatathat 1.3.0 silently dropped, spec-correct for native Gemini streaming, but a client that counted or hashed raw array elements will see N+1 elements. - Upgrade hint for the removed
auth.mode:key. A config that still carries the pre-auth.chainauth.mode:key now fails to boot with a targeted migration hint (mode: none→ an empty/omittedchain:;mode: token/apikey→chain: [tokens];mode: passthrough→auth.upstream_credentials: passthrough) instead of a bare serde “unknown field” error. Still fail-closed.
-
Redis store atomicity.
delete_key’s cascade (key row, index, usage windows, SigV4 credentials and their indexes) andput_key_with_aws_credentialnow run as single atomicMULTI/EXECtransactions — a mid-cascade failure can no longer orphan a SigV4 credential behind a deleted key. The Redis store also gains transparent one-shot reconnect after a dropped connection,rediss://TLS support (rustls), and password scrubbing in error strings; its unbounded no-TTL growth posture is documented (retention is the operator’s schedule). -
Fleet-budget flushes are additive. The write-behind usage flush now writes the delta since the last acknowledged flush (
Store::add_usage, atomic accumulate in every backend) instead of an absolute overwrite, so multiple nodes sharing one durable store sum to the true fleet total instead of last-writer-wins clobbering each other. -
Security: OAuth egress SSRF hardening (config-time and runtime): the
token_urlaoauth-client-credentialsprovider POSTs the client secret to now runs through the same SSRF/cloud-metadata denylist and case-insensitive https requirement asbase_url(previously only a case-sensitivehttp://check with NO metadata guard, so a typo’d/templatedtoken_urlpointing at IMDS ormetadata.google.internalcould leak the secret). Additionally: (a) both self-minting OAuth clients (jwt-bearer,oauth-client-credentials) now refuse HTTP redirects and carry connect/overall timeouts, the credential rides in the POST body, so a 307/308 from a compromised token endpoint would otherwise re-POST the plaintextclient_secret/ signed assertion to a redirect target the boot-time URL check never saw; (b) thejwt-bearerservice-accounttoken_urinow gets the same https + metadata denylist vetting astoken_url; and (c)busbar --validatenow dry-run-validates ajwt-bearercredential’s SA JSON + PKCS#8 key (when the env var is set), instead of surfacing malformed key material only at boot/apply. -
An aborted cross-protocol gemini JSON-array stream now emits exactly one trailing error element (a mid-cycle change had it wrap the native error frame and append a second one).
busbar --validateno longer reports false errors on a config that env-templates itsbase_url/token_url(${VAR}) when the variable is unset (a Lenient-mode placeholder was failing the URL/https checks it will pass at boot). Streaming Cohere→X hops now preservemessage.tool_plan(the pre-tool-call reasoning), matching the non-stream path. -
jwt-bearernow honors a configuredscope:(it was dropped, so the defaultcloud-platformscope always won). JWT claims are built with a JSON serializer instead of string interpolation (a"/\inclient_email/scopecan no longer malform or inject into the claim set). -
Health probers are re-spawned on config reload/apply and hold a
Weak<App>: reloaded lanes are now probed, and the previous generation exits instead of leaking one task-set per reload (which also pinned the orphaned snapshot and wrote breaker outcomes into a store no longer serving traffic). -
A mid-stream upstream transport error no longer token-bills the partial usage accumulated before the cut (symmetric with the terminal-error / translate-abort no-bill gates).
-
Translation fidelity: the Cohere reader surfaces
message.tool_plan(the assistant’s pre-tool reasoning was silently dropped on any Cohere→X hop); the Cohere and Responses writers emit a raw-string tool argument verbatim instead of JSON-encoding it a second time; a prompt-level GeminiRECITATIONblock maps tosafety(consistent with the candidate-level mapping). -
busbar --validatenow catches a model whosecontext_maxconflicts across pools (previously rejected only at real boot, so a clean--validatecould stilldieon start). -
The shared upstream HTTP client sets a
connect_timeoutand TCP keepalive; the virtual-key secret is widened to 256 bits; the self-minting bearer credential recovers from a poisoned lock on the request hot path rather than panicking. -
License hygiene: the three OAuth source files that were mislabeled
AGPL-3.0-or-laterare corrected toApache-2.0(the project license); a test now fails on any non-Apache SPDX header in first-party source.
July 16, 2026
busbar --validate: validate a config file without booting or binding a socket (thenginx -tworkflow). Reports structural/reference errors and, in lenient env mode, records unset${VARS}as placeholders (structure, not secrets) so a config can be checked in CI without the runtime environment.
Changed
Section titled “Changed”- Egress auth is now fully separated from the protocol writers: a
CredentialProviderowned by the lane (resolved once at boot from protocol + auth style) produces each request’s outbound auth headers vialane.credential.headers_for(...), replacing per-writerauth_headers/sign_request. Behavior-preserving (every protocol emits byte-identical auth) and it sets up a self-minting (OAuth/Vertex) credential later. - Release profile is
opt-level = 3(wass). NewBUSBAR_WORKER_THREADSenv knob hard-caps the Tokio worker pool (defaultmin(cores, 4), ~5% lower RSS on many-core hosts). Deduped busbar’s owngetrandomusage onto 0.3 (dropped the 0.4 copy);ringstill vendors its own 0.2 line, so twogetrandommajors remain in the tree. - All workspace crates are now versioned in lockstep at
1.3.3. The internalpublish = falsesupport crates (busbar-api,busbar-auth-tokens,busbar-auth-admin-tokens,busbar-hooks-ranking) had lagged at 1.3.0 across the last few releases; they now track the binary’s version.
- Tap (fire-and-forget hook) spawns are now bounded by a semaphore (cap 1024) so a slow hook can’t grow
unbounded in-flight tasks; over-cap taps are dropped and counted (
tap_notifications_dropped_total). - Config overlay no longer risks tombstone-loss: an unreadable overlay file is refused rather than overwritten (which previously could silently drop persisted admin state).
- Request rewrite-on-failover now fails closed: if a queued rewrite can’t be re-applied to the retried upstream, the request is rejected (500) instead of silently forwarding the un-rewritten body.
Security
Section titled “Security”- SSRF egress guard extended to block Azure WireServer (
168.63.129.16) and Oracle Cloud IMDS (192.0.0.192) alongside the existing cloud metadata ranges.host_from_basestrips userinfo, folds backslashes, and drops any path before deriving the SigV4-signed host, so the signed host can never desync from the host actually dialed.
July 14, 2026
- Finished greening the CI matrix 1.3.1 began; three still-red checks are now clean with no binary change.
fmt: committed the rustfmt reformatting of
render_histogramthat had been applied locally but never committed. Windows:PolicyOnError(used only by the unix-only socket-gate tests) is now imported under#[cfg(unix)]. cargo-deny: workspace member crates are versionless path deps; marking thempublish = false(they never go to crates.io) letsallow-wildcard-pathsapply, sobanspasses while external wildcards stay denied.
Changed
Section titled “Changed”scripts/preflight.shnow fails on an uncommitted working tree (CI tests the committed state; an uncommittedcargo fmtis how the fmt red slipped past 1.3.1) and runs cargo-deny (the Security job) when installed.- Dependency bumps (Dependabot):
bytes1.12.0 → 1.12.1; CI/release actionsdocker/build-push@7,docker/setup-buildx@4,docker/metadata@6,docker/login@4,actions/upload-artifact@7,actions/download-artifact@8.
July 14, 2026
A maintenance release: no binary behavior change, a clean CI matrix, and a pre-release gate so the config-specific breakage caught here can’t recur.
- Restored a clean CI matrix: config-specific dead code that only
-D warningsrejects under--no-default-featuresor on Windows now compiles cleanly.PROBE_TIMEOUTmoved inside its#[cfg(unix)]scope; theadmin_token_hashgetter and aWarnCapturetest helper are gated to the features that use them; the admin-listener split test is gated toauth-admin-tokens.
scripts/preflight.sh: a pre-release gate mirroring the full CI matrix locally (fmt, structure-lint, clippy + build + test on both the default and--no-default-featuresfeature sets, and a best-effort Windows type-check). Run it before tagging so a release can’t ship red CI.
July 13, 2026
The API release: everything you could only do by editing YAML and restarting, you can now do over an authenticated, audited API. The routing hook grew into a hook system: gates and taps on every request.
This release reshapes how hooks and policies are configured. Hooks are now defined once by name and referenced
everywhere; the old inline policy: block and transport-named route: values are replaced. Existing
configs need a one-time update — see the 1.2.x → 1.3 migration guide. It is a clean
cut with no silent fallbacks: an old-form key reports a clear startup error telling you exactly what to write
instead.
- FinOps metering, built for third parties.
GET /api/v1/admin/usagereports per-model and per-key consumption as the raw token split (input / output / cache-read / cache-creation, each prices differently) in fixed UTC-day buckets, withspend_micros(micro-USD, integer math) derived at read time from your configured prices. Busbar exposes the inputs of cost, not just its own number, so a consumer with negotiated per-model pricing reconstructs cost exactly from the split.?window=selects past buckets; over-cap key lists carry anothersremainder so every unit stays attributable;window/as_of/currencylabel the numbers. - Hooks are control-plane citizens. A hook self-reports its observed settings and its own operational
metrics over the new
statuswire message, andGET /api/v1/admin/hooks/{name}/statussurfaces it with a desired-vs-reported drift verdict, so a dashboard built on Busbar sees what every plug is doing without each hook needing its own dashboard. - One professional wire contract, audited to zero. Three independent contract-audit rounds on the Admin
API and two on the hook wire, all findings fixed pre-freeze: one error envelope everywhere with a frozen
code taxonomy (including
unauthorized,method_not_allowed, and retryableversion_conflictsplit from terminalconflict), one{items, next_cursor}list envelope with opaque cursors, one optimistic-concurrency mechanism (RFC-7232If-Match/ETag on every mutable resource),Idempotency-Keyon both secret-minting POSTs,Retry-Afteron 429s, machine-readable query params + scope annotations inopenapi.json, and explicit per-requestopdiscrimination + append-only evolvability rules on the hook wire. - The Admin API is a full config plane. Anything the config file can express, the API can do: read the running config, apply a validated change atomically, roll back to any previous version, register hooks, adjust pools, budgets, and rate limits. Drive Busbar from Terraform, Ansible, or CI: no SSH, no file edits, no restarts.
- Config overlay. API-applied changes persist to a Busbar-owned overlay file; your hand-written
config.yamlis never touched. The effective config is base plus overlay, both human-readable, so “who set this” is always answerable. - Admin audit log. Every admin mutation records who changed what, when. Scoped admin tokens let you mint credentials that can, for example, only register hooks or only read.
- Named hooks. Define a hook once under
hooks:, reference it anywhere: in a pool’shooks: [...]list or viaglobal_hooks:to run on every request. One list carries both jobs: a pool names its ranking strategy (weighted, cheapest, fastest, least_busy, usage) and any gates together, e.g.hooks: [cheapest, pii-guard]. The oldroute:values and inlinepolicy:block are removed; an old-form key is a clear startup error naming its replacement (a clean cut, no silent fallback). See the 1.2.x → 1.3 migration guide. - Gates and taps. A
gateis a blocking hook that can reject a request or restrict which pool members may serve it; atapis fire-and-forget observation (request, route, per-attempt, and completion stages) that can never delay or fail a request. Routes rank, gates decide, taps watch. - The restrict verb. A gate can reply “only members carrying these tags may serve this”: compliance-constrained routing (data residency, BAA-only lanes) without teaching your router about compliance. Restrictions hold across failover.
- Concurrent hooks. All of a request’s hooks fire at once, so added latency is the slowest hook, not the sum. Any reject wins; restrictions intersect; the route ranks what survives.
- Pluggable auth. Authentication is now an ordered chain of modules: each identifies the caller, rejects, or passes to the next. Token auth is the first module and the default, and it is removable: list only your own module and tokens are gone. External modules speak the same hook transports; validated identities are cached (with instant admin flush), and auth always fails closed. Budgets, rate limits, pool access, and audit all follow the authenticated principal, whoever issued it.
- Admin API lockdown. The admin API authenticates through its own pluggable chain, with scoped principals
(read-only, hooks-register, full) replacing the single shared admin token, and every mutation in the audit
log attributed to the person who made it. The chain itself is live-mutable (
PUT /api/v1/admin/admin-auth) and guarded so a change that would lock the caller out is refused instead of applied. - The rewrite verb. A trusted gate (
prompt: rw) can replace the request body before dispatch: context compression and redaction, across all six protocols at once, because it fires on the normalized form. Rewrites persist across failover, token accounting uses the rewritten body (the savings are real and measured), and a malformed or slow rewrite proceeds with the original body untouched; a broken compressor can never corrupt a request. - Live hook settings. Push a settings map to a running hook over the admin API; the change commits only when the hook acknowledges it, and a restarted hook always receives its current settings before any traffic. Hooks can also describe their own settings schema, served verbatim by the API.
- Config reload, and health that survives everything.
POST /api/v1/admin/config/reloadre-reads your config files and applies them atomically, and lane health (circuit breakers, cooldowns, learned latency) is carried across by model identity, not list position, so a reorder or added model never resets what Busbar has learned. That health state now persists across restarts too: kill Busbar, fix the config, start again, and sub-second it comes back remembering which lanes were misbehaving.--safe-modeboots from your base config alone when an API-applied overlay is the problem. - Group-based governance.
group_map:maps identity-provider groups to authority in one place: admin scopes and data-plane access (allowed pools, rate limits, budgets), governed by exactly the machinery a virtual key uses. Per-module caps bound what any auth module can assert: an allowlist of groups it may claim and a ceiling on the admin scope obtainable through it. - The admin API runs on its own listener, always. The management surface (
/api/v1/admin/…) is served on a dedicatedadmin_listenand is never mounted on the datalisten: the public bind cannot serve/api/v1/admin/*at all. It carries its ownadmin_tls:(cert + optionalclient_ca_filefor client-certificate mTLS), so the control plane can require client certs, bind, and firewall independently of public LLM traffic.admin_listendefaults to loopback (127.0.0.1:8081), so a zero-config deployment boots with admin reachable only on-host; point it at an exposed address (with mTLS) to manage Busbar remotely.
Changed
Section titled “Changed”- The management API lives under one root:
/api/v1/admin/…. Every Busbar-native API mounts under/api/<version>/<area>/, cleanly separated from data-plane protocol paths (dictated by the vendor SDKs, which don’t move). The key-management endpoints previously at/admin/keys*are now/api/v1/admin/keys*; scripts calling the old paths need a one-line URL update. Future surfaces (and a futurev2) slot in under the same root without new top-level paths. - Completion telemetry now carries usage for every operation type (chat tokens, embeddings, images, audio, rerank) plus a request id that correlates a request across hook stages.
Removed
Section titled “Removed”- The inline
policy:block and transport-namedroute:values. A pool’sroute:now takes a hook name (defined once underhooks:) or a native policy name (weighted/cheapest/fastest/least_busy/usage); the oldroute: socket/route: webhook+policy:form is replaced. Each removed key reports a startup error with the exact replacement. See the migration guide. - The embedded Rhai script routing policy (
route: script). Available only behind an opt-in build flag and deprecated in 1.2.1, it is gone. A compiled hook over a socket or an HTTP webhook does the same job with real process isolation; if you want scripting, run a hook that embeds it.
Security
Section titled “Security”- Exposed admin plane requires mTLS, fail closed. A network-exposed
admin_listen(any non-loopback bind) refuses to boot unless protected by client-certificate mTLS (admin_tls.client_ca_file). A loopback admin bind is exempt (unreachable off-host), and an operator fronting admin with a mesh that terminates mTLS can waive the guard explicitly withadmin_insecure: true. The management plane is never silently published behind a bearer token alone.
July 11, 2026
A hardening release, plus the routing hook layer growing up: a faster transport and the payload and verbs that make screening hooks possible.
- The socket routing hook (
route: socket). Your routing policy as a compiled binary on a local Unix domain socket. Same wire contract as the HTTP webhook (a hook moves between the two without changing its logic), same hard deadline andon_errorfail-safe, no HTTP stack in between: measured end to end against a real external Rust hook, a decision costs about 8 microseconds median. Busbar never spawns or supervises the hook binary; you (or your init system) run it, Busbar connects lazily, keeps the connection alive, and reconnects transparently across hook restarts. Kill the hook mid-traffic and requests keep flowing on the pool’s fallback. Unix-only; on other platforms useroute: webhook. - Hook payload opt-ins:
policy.send_promptandpolicy.send_user. The hook payload stays shape-only by default; two per-pool booleans (both defaultfalse) extend it.send_promptadds the flattened prompt content (request.system+request.messagesas{role, text}) so a trusted hook can screen content: PII, guardrails, audit.send_useradds caller identity (request.user: the governance virtual-keyid/nameplus the body’s end-user field) so a hook can route by who is asking. The caller’s secret/token is never in the payload, under any configuration. Both transports carry the same fields; a pool that sets neither flag sends the exact pre-1.2.1 payload. - Member
tagsin the hook payload. Each candidate now carries its operator-declared free-formtags(team names, regions, compliance labels), omitted when the member declares none. - The hook
rejectverb. A hook may reply{"reject": {"status": 451, "message": "..."}}instead of an order: no upstream is dispatched and the caller receives a dialect-native error. Fail-closed and bounded: the status is clamped to 400–499 (default 403) and picks the typed error class the SDK sees, the message is sanitized, and a malformed reject still rejects, never silently routes. Counted in the newbusbar_route_policy_rejections_totalmetric. Combined withsend_prompt, this is the PII-screen primitive: a hook that sees content can stop a request before it leaves your network.
Changed
Section titled “Changed”- Default hook deadline is now 1 ms (
policy.timeout_ms, was 150). The default says hooks are fast: a co-located socket hook decides in ~8 µs and a co-located webhook in ~34 µs. Raise it when your hook legitimately does I/O or crosses the network; on timeout the decision falls back peron_errorand the request proceeds regardless.
Deprecated
Section titled “Deprecated”route: script(Rhai). The embedded interpreter costs ~100x more per decision than a compiled socket hook for the same logic, and its sandbox is a weaker isolation story than a separate process. It still works behind thescript-policyfeature but warns at startup; migrate toroute: socket(compiled hook) orroute: webhook(any language).
- Hardened throughout. Multiple rounds of extensive adversarial testing and code review over the full 1.2.0 change set and the new hook layer surfaced and fixed a broad batch of defects: protocol-translation edge cases, input validation and sanitization, error handling, and observability gaps. Every fix shipped with the regression test that catches it; the suite grew by several hundred tests this release.
July 10, 2026
Busbar now speaks more than chat. Five new operations land on top of chat: Embeddings, Moderations, Image generation, Audio (transcription and speech), and Rerank. Every one is cross-protocol, carried by the same lossless translation that already carried chat: a Gemini client can call embeddings on an Amazon Bedrock backend, an OpenAI client can route images and audio to Google Gemini, and every answer comes back in the caller’s own dialect, lossless in both directions, errors included. Chat itself is byte-for-byte unchanged: it is simply the first operation now, not a special case.
- Embeddings (
/v1/embeddingsand each protocol’s native surface), cross-protocol. An OpenAI-dialect embeddings request routes to OpenAI, Amazon Bedrock (Titan), Cohere (v2/embed), or Google Gemini (embedContent) and returns in the caller’s own dialect. Vectors, token/usage accounting, and errors all survive the hop. - Per-attempt hang detection (
attempt_timeout_ms). Some providers fail by hanging: the connection opens and response headers never come back, silently eating the whole failover budget on one member.attempt_timeout_mscaps a single attempt’s time to response headers; on expiry the attempt is recorded as a transient breaker failure and the request fails over to the next pool member within the same request. Set it on a model as that model’s default and override per pool member, so the same model can carry a 10s cap in a batch pool and a 50ms cap in a latency-critical one. The cap covers connect and headers only (a stream that has started answering is never cut off) and is always floored by the request’s remainingfailover.timeout_secs.0is a startup error. Observable asdisposition="attempt_timeout"onbusbar_upstream_failures_totalandreason="attempt_timeout"onbusbar_failovers_total. - Cross-protocol logprobs (OpenAI ↔ Gemini), buffered and streaming. Per-token log probabilities are now a
first-class IR concept. The ask crosses the seam either direction (OpenAI
logprobs/top_logprobs↔ GeminigenerationConfig.responseLogprobs/logprobs) and the response comes back in the caller’s own shape (choices[].logprobs.content[]↔candidates[].logprobsResult, including chosen tokens, top alternatives, and synthesized UTF-8byteswhere OpenAI’s shape requires them), buffered and per-chunk in streams. Backends with no logprobs concept (Anthropic, Bedrock) never receive the ask; Cohere logprobs stay same-protocol-only (its wire shape carries bare token IDs under its own tokenizer). Live-validated against real OpenAI from a Gemini-dialect client, buffered and streaming. - Two new cross-protocol carries:
userandparallel_tool_calls. OpenAI’suserand Anthropic’smetadata.user_idare the same end-user identifier; OpenAI’sparallel_tool_callsand Anthropic’stool_choice.disable_parallel_tool_useare the same switch, inverted. Both are now first-class in the IR and translate between the two protocols in both directions instead of being dropped at the seam. The documented field-survival table (docs/protocols.md) is now measured against real egress capture, not asserted. - Moderations (
/v1/moderations), cross-protocol. Content-classification requests translate through the IR and return in the caller’s dialect, so a moderation call is not pinned to one vendor’s endpoint. - Image generation, cross-protocol. An OpenAI-dialect image request routes to OpenAI, Google Gemini (Imagen), or Amazon Bedrock (Titan) and comes back in the caller’s dialect.
- Rerank (
/v2/rerankand Bedrock rerank models), cross-protocol. The sixth operation: Cohere v2 rerank and Amazon Bedrock rerank models (viaInvokeModel, detected by thequery+documentsbody) translate exactly in both directions: the two wires share the same result shape (index+relevance_score), so a Cohere-dialect client can rerank on a Bedrock backend and vice versa, with pools, failover, and breakers like every other operation. The other four protocols ship no rerank surface and answer with the standard dialect-native 404. - Audio, cross-protocol. Transcription (speech-to-text, including speech-to-English translation) and Speech (text-to-speech) against OpenAI and Google Gemini backends, translated to and from the caller’s dialect like every other operation.
- Clean, dialect-native 404 for an operation a backend lacks. Calling an operation a backend does not implement (e.g. image generation on an Anthropic backend) returns a well-formed 404 in the caller’s own protocol dialect: never a crash, never a malformed body, and never taking the lane down for other traffic.
- Cross-protocol reasoning/thinking carry (opt-in per lane). The reasoning ask now translates between the
three protocols that model it: OpenAI
reasoning_effort/ Responsesreasoning.effort(words), Anthropicthinking.budget_tokensand GeminithinkingConfig.thinkingBudget(token budgets). Number to number is a straight copy (a Claude/Gemini thinking pool loses nothing); words and numbers convert through a configurable effort table (limits.reasoning_effort_budgets, defaults 1024/4096/8192/16384). Because thinking support is per-model, not per-protocol, the carry is gated by an operator flag:reasoning: trueon a model (overridable per pool member) declares “this backend accepts thinking params”. Without the flag the ask is dropped at the seam with a warn and the request proceeds normally, so a non-reasoning model can never 400 from translation. Budgets are clamped to fitmax_tokens(Anthropic requires it), and Anthropic-incompatible sampling knobs (temperature, top_k) are omitted with a warn when thinking is emitted. The response-side thinking content was already lossless and is unaffected. Gemini’s dynamic-1round-trips to Gemini and projects elsewhere asmedium.
Changed
Section titled “Changed”- License: Apache 2.0. Busbar 1.2.0 and onward is licensed under the Apache License, Version 2.0: permissive, commercial-friendly, with an explicit patent grant, no copyleft obligations: use, modify, and redistribute privately or commercially.
- Every operation is lossless across protocols, errors included. Responses and error envelopes always come back in the caller’s own protocol dialect, and token/usage accounting survives the cross-protocol round trip on every operation, not just chat.
- Four-layer operation architecture (internal). The request path is now Router → RequestHandler → OperationHandler → IR, where each operation is a small codec over the shared reliability engine and chat is operation #1 rather than a special case. Adding an operation is a codec, not a change to routing, failover, or the breaker. No user-visible behavior change to chat.
- Billing is now a polymorphic data model (internal). Usage is metered as tokens, duration, characters, images, or a flat unit depending on the operation, so non-chat operations meter on their natural axis. A pricing engine that turns these units into cost is planned for 1.3.
- Gemini streamed thinking no longer leaks into answer text on cross-protocol streams. The Gemini stream
reader routed
thought: trueparts into the answer text, so a Gemini backend’s streamed reasoning was concatenated into the visible reply for every cross-protocol client (the buffered path was already correct). Thought parts now stream as proper thinking blocks (signature included) with balanced block framing on every terminal path. Caught by the new offline streaming-reasoning harness rows.
July 9, 2026
GET /v1/modelsandGET /v1beta/models: the list-models surface. Returns every routable name: configured pools first, then model entries, each sorted. This is the first callclient.models.list()and self-hosted UIs (Open WebUI, LibreChat) make to build a model picker; it previously returned 404. Three protocols put list-models on the same noun, so Busbar answers in the caller’s dialect by protocol fingerprint: ananthropic-versionheader gets the Anthropic envelope,x-goog-api-keyor the/v1betapath gets Gemini’s, otherwise OpenAI’s. Governance-scoped like/stats: a virtual key restricted byallowed_poolssees only the pools it may target and the models reachable through them.
Changed
Section titled “Changed”- Operations are now a first-class axis of the forward engine (internal). The request path is generic over
an operation spec (
OpSpec) rather than hardcoding chat’s assumptions (stream intent, upstream path, usage extraction, affinity, egressAccept). Chat is spec #1 and its behavior is byte-for-byte unchanged: the full test suite passes unmodified. Groundwork that lets a future release add non-chat operations (embeddings, moderations, images, audio) as small spec files with no change to the reliability engine. No user-visible behavior change.
/metricsis no longer empty before the first request. The unlabeled counter family is pre-registered at startup and per-lanebusbar_lane_stategauges are now also emitted for direct-model (pool-less) lanes (labeled with the model name aspool, matching the counter convention), so a freshly booted gateway exposes a live exposition to Prometheus immediately. Both issues were found by the user-emulated acceptance harness on its first run./statsoutput is now deterministic across restarts. Lanes are built sorted by model name (previously inHashMapiteration order, randomized per process), and/statsserializes pools in sorted key order. The lane/pool ordering (and therefore metric lane-series identity) is now stable boot to boot, so scrapes, dashboards, and tests are reproducible.
June 30, 2026
upstream_modelconfig field: decouples a model’s config key (operator alias) from the model id sent to the provider on the wire. Lets the same model run behind two providers in one failover pool (e.g. Claude 3.5 Sonnet via Anthropic and Bedrock), where the keys must differ but each provider needs its own model string. Threaded through body rewriting, URL generation, and health probes (probes hit the same wire id as real traffic). Metrics, breaker cells, and logs continue to key off the config key. Feature contributed by @lguzzon (adopted asupstream_model; the resolver isLane::wire_model()).
- Documentation drift surfaced by @lguzzon and a deep doc-vs-code audit: removed
dead
UsageTapreferences, corrected same-protocol passthrough to the IR-unified model, fixed thebilling_truncatedmetric and budget-atomicity descriptions, updated all route notation to axum 0.8{param}/{*rest}syntax, andwindow_s→window_secs.
June 30, 2026
First hardened maintenance release. No request-path behavior change; the binary is functionally identical to 1.0.0, and the API, config schema, and six wire-protocol contracts are unchanged.
Changed
Section titled “Changed”- Dependency upgrades. axum 0.7 → 0.8: route path-param syntax migrated (
:id→{id},*rest→{*rest}), no behavior change. getrandom 0.2 → 0.3 (getrandom()→fill(), same OS-CSPRNG). rcgen 0.13 → 0.14 (test-only). All build-verified: 1,667 tests pass, clippy-D warningsclean. A new credential-generator contract test pins the bearer / AWS-AKID / AWS-secret wire shapes so a future dependency change that alters them fails loudly.
Security
Section titled “Security”- Dependency scanning gate. A
cargo-denyCI workflow checks every dependency against the RustSec advisory DB and enforces a license allow-list, crates.io-only sources, and a duplicate-version ban: on dependency changes and on a weekly schedule (an advisory can be filed after a dep is merged). - Signed, inventoried releases. Each release now ships a CycloneDX SBOM and a keyless (Sigstore/OIDC)
build-provenance attestation, so a downloaded artifact can be verified with
gh attestation verify <file> --repo GetBusbar/busbar.
June 21, 2026
First stable release. 1.0.0 keeps the 1.0.0-rc.7 architecture (all traffic through the superset IR with a
verbatim serialize short-circuit, IR-metered billing) and ships an extensive hardening pass on top of it. The
HTTP API, configuration schema, and the six wire-protocol contracts are stable under Semantic Versioning: no
breaking change without a major-version bump. See the rc entries below for the full pre-1.0 history.
Changed
Section titled “Changed”- Typed-IR completeness.
response_format,stop_reason, image source, and redacted-reasoning are first-class IR fields rather than passthrough blobs, so each survives a cross-protocol hop losslessly and no off-spec value reaches a wire. - Containment refactor. Per-protocol logic moved fully behind the reader/writer vtable so the agnostic core names no protocol module; load-bearing literals named as consts; in-module-only items privatized.
- OpenAI-family module split.
proto/openai.rs→openai_chat.rs,proto/responses.rs→openai_responses.rs, with shared error/auth/id helpers inopenai_family.rs. The protocol names (openai,responses) are unchanged: internal layout only. - Reproducible builds. CI and release builds run with
--locked. - Migration (rc.7 → 1.0.0):
governance.rate_sweep_intervalmust now be>= 1;0is rejected at boot (rc.7 silently disabled the rate-map idle-entry sweep on0). No other config change for a default deployment.
- Cross-protocol fidelity. Two Bedrock egress shapes that returned 400 on a valid request; consecutive
same-role turn coalescing on Bedrock; Anthropic
cache_controlcarried through on thinking/image blocks; unknownstop_reasonnormalized on egress; a streaming-Responses refusal data-loss. - Billing precision. Sub-cent carry attribution, billing of cancelled mid-stream requests, and no token-billing of a translate-aborted stream.
Security
Section titled “Security”- A slow-loris header-read bound on both the TLS and plain-HTTP listeners; the SigV4 inbound body buffer capped independently of the body-limit layer; circuit-breaker probe-leak / streak-inflation / jitter hardening.