Routing

Every Busbar pool has an ordering strategy. The default (weighted smooth round-robin (SWRR)) costs nothing and is the right choice for most pools. When you need a different selection strategy, you name one in the pool’s hooks: list (hooks: [cheapest]) and everything else in Busbar (the circuit breaker, failover loop, concurrency semaphore, session affinity) is unchanged. The strategy only determines the order in which healthy candidates are tried.

Routing is one verb of Busbar’s Hooks system: a programmable request path. A hook is your own code — a signed kind: hook plugin loaded in-process, or the first-party busbar-webrequest-hook forwarder to an HTTP sidecar in any language — that sees a projection of the request and the live candidate signals and replies with a decision. An ordering hook replies with the order arm: a ranked preference list. The same machinery (timeout, on_error fallback, transparency headers) carries every hook, so a broken or slow hook never blocks or fails a request. This page covers ordering; the full hook contract (taps, the restrict and rewrite arms, live settings) lives in the Hooks guide.

Cross-references: Pools (how to define a pool and its members) · Hooks (the full hook model) · Configuration (full field reference) · Reliability & Failover (breaker, failover, and exhaustion behavior). Coming from a 1.2.x config (route: / policy: keys)? See the 1.3 migration guide. The old keys are hard startup errors that name the fix.

The model: ordering as ranked preference
How ordering composes with the breaker and failover
Native strategies
The routing signals
External ordering hooks
Fail-safety: on_error
A fleet-wide default ordering
Full examples
Observability

The model: ordering as ranked preference

An ordering hook does one thing: given the current request and the set of healthy candidates, it returns an ordered preference list. Busbar’s existing failover loop walks that list (trying the first candidate, then the second if the first fails, and so on) using the circuit breaker at every step to skip any lane that is tripped, at capacity, or already tried this request.

The design consequence: a hook ranks; the breaker decides health. A hook cannot resurrect a tripped lane, and a hook that omits a healthy lane does not strand it; omitted candidates are appended to the end of the preference list, not excluded. A broken hook (timeout, error, empty response) falls back per its on_error (for an ordering hook, back to weighted SWRR) rather than failing the request.

A pool names its ordering in one hooks: [...] list: at most one strategy (a native name or an external gate that returns order) plus any number of other gates. Pools with no hooks: list run the zero-overhead case: today’s unchanged SWRR code, with no projection built and no hook object constructed. Adding a hook to one pool adds no overhead to pools that do not use one.

How ordering composes with the breaker and failover

The sequence for every request routed through a pool with a non-default ordering:

The ordering hook runs once, before the failover loop. It receives the healthy candidate set (as a projected read-only view) and returns a ranked list of candidate indices. When several gates apply (the pool’s own plus any globals), they fire concurrently and reconcile deterministically (see Hooks for the reconcile rules).
The failover loop walks the ranked list. It tries candidates in the preferred order, skipping any that are tripped, at capacity, or already tried.
Candidates not in the ranked list are tried last. If the hook emits a subset of candidates, the omitted ones are appended after the ranked set in an unspecified order. They are reachable; a hook can never permanently exclude a healthy lane by omission.
On hook failure, on_error takes over. A timeout or error coerces the decision to the hook’s on_error (for an ordering hook, weighted, as if no hook were configured). An explicit abstain is different: it always means “no opinion” and Busbar proceeds as it normally would, regardless of on_error; abstaining is not a failure.

This composition means an ordering hook’s job is deliberately narrow. You declare a preference; Busbar’s existing reliability machinery handles the rest.

Native strategies

Native strategies are compiled into Busbar and have no runtime dependencies. They are sync, never do I/O, and sort the candidate set by a single live signal. Name one directly in the pool’s hooks: list; a pool may name at most one strategy.

weighted (default)

The default selection strategy when a pool names no strategy. Uses Nginx-style smooth weighted round-robin (SWRR) across healthy members, proportional to each member’s weight field. Writing hooks: [weighted] gives byte-identical behavior to omitting it entirely.

Use weighted explicitly only when you want to name the strategy in config for documentation clarity. There is no behavioral difference from the default.

cheapest

Prefers the member with the lowest cost, derived from the top-level rate_card. In 1.5.0 members carry no cost fields; the cost scalar for each model comes from its rate_card entry (there is one cost source). A member whose model has no rate-card entry is demoted to the end of the preference list but is still reachable. If no candidate has a priced model, the strategy abstains and SWRR takes over.

Signal: the model’s rate_card entry (input/output/cache per-token rates). You price the model once; Busbar ranks on it.

rate_card:
  claude-sonnet: { input_utok: 3.0,  output_utok: 15.0 }
  gpt-4o:        { input_utok: 5.0,  output_utok: 15.0 }
  gpt-4o-mini:   { input_utok: 0.15, output_utok: 0.6 }

pools:
  cost-optimized:
    hooks: [cheapest]
    members:
      - { model: claude-sonnet }
      - { model: gpt-4o }
      - { model: gpt-4o-mini }

Traffic flows to gpt-4o-mini first, then gpt-4o, then claude-sonnet. If gpt-4o-mini is tripped, the breaker skips it and gpt-4o becomes the first attempt.

fastest

Prefers the member with the lowest measured round-trip latency, tracked as a rolling EWMA updated after each request. Members with no latency sample yet (new lanes, recently restarted) are demoted but reachable. If no candidate has latency data, the strategy abstains.

Signal: rolling EWMA latency in milliseconds, accumulated from organic traffic. No configuration required.

This is a good choice when your members have meaningfully different tail latencies and you want Busbar to track and prefer the faster one automatically over time.

least_busy

Prefers the member with the most available concurrency headroom; the lane with the most free slots in its semaphore. Unlike fastest, least_busy always has data (concurrency is always known) and never abstains.

Signal: free concurrency permits on each lane’s semaphore at decision time.

Use this when your members have different max_concurrent limits or when you want to avoid piling requests onto an already-saturated backend before the breaker trips it.

usage

Prefers the member with the most remaining rate-limit headroom (requests/tokens headroom computed from the caller key’s group-chain rate-limit counters). Candidates with no headroom signal (e.g. when governance is disabled or no rate limit is set) are demoted to last but remain reachable. Abstains only when every candidate lacks the signal (no rate limit in play), falling back to SWRR.

The routing signals

Every ordering decision (native and external) sees the same projection of each candidate:

Signal	Field	Notes
Per-member cost	`cost_per_mtok`	Derived from the model’s top-level `rate_card` entry (members carry no cost fields in 1.5.0). `None` if the model is unpriced.
Per-lane latency	`latency_ms`	Rolling EWMA in ms, updated per request. `None` until first request.
Live concurrency	`available_concurrency`	Free semaphore slots. Always populated.
Budget remaining	`budget_remaining`	Remaining budget on the caller key’s group chain. `None` = unlimited.
Rate headroom	`rate_headroom`	Remaining requests/tokens headroom from the group-chain counters, as a fraction (most headroom first). `None` when governance is disabled or the group has no rate limit.
Labels	`tier`, `tags`	Your operator-declared member labels.

External hooks also receive the request projection: pool, ingress_protocol (one of anthropic, openai, gemini, bedrock, cohere, responses), message_count, has_tools, total_chars, max_tokens (null if the caller set no output-token limit), and stream.

Token counts are not available pre-dispatch. The upstream response carries token usage, but that comes after a lane is chosen. Use total_chars (sum of all text chars across system + messages; not a token count) with the rule of thumb of ~4 chars/token for size-based decisions.

Every signal a native strategy ranks on is on the wire, so an external hook can implement any of them identically, and then go further.

External ordering hooks

External hooks let you run routing logic outside Busbar, in any language, with access to any data Busbar cannot see. Busbar sends a lightweight projection of the request and candidates to your hook and receives a ranked candidate list back. The same timeout, fallback, and safety machinery applies as for native strategies: a slow or broken hook never fails a request.

Inline instances (no registry)

In 1.5.0 there is no top-level hooks: registry, and a hook is a signed kind: hook plugin loaded in-process over the hybrid ABI. A hook instance is defined inline, as a module ref, right where it runs: in a pool’s hooks: [...] list or in the top-level global_hooks: [...] list. An ordering hook is a kind: gate (fire-and-wait; a tap only observes and can never appear in a pool list; in a pool list an unmarked ref defaults to kind: gate). A module ref’s module: names a loaded kind: hook plugin by its signed-manifest name/alias; settings: is the plugin’s opaque config. Using any hook plugin requires plugins.enabled: true and the signed tarball installed in the plugins directory.

plugins:
  enabled: true
  dir: /etc/busbar/plugins            # holds the signed kind:hook plugin tarballs

pools:
  smart:
    hooks:
      - { module: busbar-webrequest-hook,
          settings: { url: "https://router.internal/rank" },
          kind: gate, timeout_ms: 5, on_error: weighted }   # a broken ordering hook falls back to the weighted floor
    members:
      - { model: claude-opus,   tier: large }
      - { model: claude-sonnet, tier: small }

One list carries both jobs: a pool may name a base strategy and gates, e.g. hooks: [cheapest, { module: busbar-webrequest-hook, settings: { url: "https://router.internal/rank" } }]. All of a request’s gates fire concurrently and reconcile deterministically (any reject wins; restricts intersect; with several orders, the last in the priority chain wins). The priority field on a ref orders that chain; ties keep globals first, then config order. Adding a ref to global_hooks: (or global: true on a registered hook) attaches it to every request. Full reconcile rules in Hooks.

Misconfiguration is a hard startup error, never a silent fallback: a dangling built-in name, a plugin ref with the plugin subsystem disabled or the tarball missing, a tap in a pool list, or more than one strategy in one list all fail the boot with a message naming the fix.

Out-of-process hooks (`busbar-webrequest-hook`)

For routing logic in another language or address space, the first-party busbar-webrequest-hook plugin is the sanctioned path: it forwards the request/candidate projection over HTTPS to an operator-run sidecar and returns the sidecar’s ranked reply, so your logic runs out-of-process without an untrusted library inside Busbar. The artifact is signed and auto-trusted; forwarding is SSRF-guarded (loopback allowed; RFC-1918 / link-local / CGNAT / cloud-metadata blocked; remote must be https://); and the sidecar’s reply rides the same op-discriminated JSON contract. You (or your init system) run the sidecar; Busbar never spawns or supervises it, and falls back per on_error if it is slow or unreachable.

Settings. The sidecar URL is a plugin setting (settings.url); it is operator-config-only, never derived from a request header or body. Loopback sidecars are allowed; remote URLs must be https:// and are SSRF-guarded. Replies are read under a size cap and a depth-guarded JSON parse.

In-process alternative. For the lowest latency and the tightest integration, write your own signed kind: hook plugin and load it in-process by its manifest name — the same ABI and trust model as the store, auth, and secret plugins. See the Hooks guide and Plugins for the plugin SDK, packing, and trust model. A worked example hook ships in the repo under examples/.

The decision payload

Request payload (Busbar → hook):

{
  "request": {
    "pool": "smart",
    "ingress_protocol": "anthropic",
    "message_count": 12,
    "has_tools": true,
    "total_chars": 41200,
    "max_tokens": 8192,
    "stream": true
  },
  "candidates": [
    {
      "idx": 0,
      "model": "claude-opus",
      "tier": "large",
      "cost_per_mtok": 15.0,
      "latency_ms": 320.5,
      "available_concurrency": 14,
      "budget_remaining": null,
      "rate_headroom": 0.82
    },
    {
      "idx": 1,
      "model": "claude-sonnet",
      "tier": "small",
      "cost_per_mtok": 3.0,
      "latency_ms": 95.2,
      "available_concurrency": 18,
      "budget_remaining": 5000,
      "rate_headroom": 0.55,
      "tags": ["sonnet"]
    }
  ]
}

Field notes:

candidates[*].tier; null if not set on the member config. cost_per_mtok is derived from the model’s rate_card entry; null if the model is unpriced.
candidates[*].latency_ms; null until the lane has served at least one request.
candidates[*].budget_remaining; the remaining budget on the caller key’s group chain. null = unlimited.
candidates[*].rate_headroom; null when governance is disabled or the group has no rate limit.
candidates[*].tags; the member’s operator-declared free-form tags array (team names, regions, compliance labels). Omitted entirely when the member declares none.

By default the payload contains only this projection: shapes, not content, no prompt text, no message bodies, no caller identity. A routing decision is a shape decision. Two per-hook grants (below) extend it for hooks you trust with more.

Access grants: prompt and user

The grants live on the hook definition, both default off, and work identically on both transports:

Grant	Levels	Adds
`prompt:`	`no` (default) · `ro` · `rw`	`ro` adds the request’s content to `request`: a `system` string (flattened system-prompt text; absent when the request has none) and a `messages` array of `{"role": "...", "text": "..."}` (text flattened; images and other binary blocks skipped). The switch for content-screening hooks: PII detection, guardrails, audit. `rw` additionally lets a gate return the `rewrite` reply arm (Hooks).
`user:`	`no` (default) · `ro`	Adds `request.user`: the governance virtual key’s `id` and `name` (when the caller authenticated with one) and the body’s end-user field, each absent when unknown. The switch for route-by-who hooks: team lanes, per-user denies. The caller’s secret/token is never in the payload, under any configuration: the identity projection is built from the resolved key record, not the credential.

Grants are immutable after registration and enforced both directions: a hook is never sent, and can never return, a field it wasn’t granted.

pools:
  guarded:
    hooks:
      - { module: busbar-guard-hook,   # your signed kind:hook plugin
          kind: gate,
          prompt: ro,        # this hook screens content
          user: ro,          # ... and routes by caller
          on_error: reject }  # a security gate fails closed

The reply

Ranked preference; most preferred first:

{ "order": [1, 0] }

Or abstain (no opinion; Busbar proceeds as it normally would):

{ "abstain": true }

Or reject the request outright. No upstream is dispatched and the caller receives a dialect-native error:

{ "reject": { "status": 451, "message": "Request blocked: contains an unredacted SSN." } }

Rules:

order is the only ranking key. Unknown idx values are dropped; duplicates are deduplicated preserving first-seen order. Omitted candidates are demoted, not excluded. An absent or empty order (including a bare {}) is treated as abstain.
reject wins over order and abstain if sent together, and the verb is fail-closed: any reject value except an explicit false (or null) is a rejection, even a malformed one. A mis-typed detail degrades to the defaults (403, a generic message), never to “silently route the request”. status is clamped to 400–499 (a hook cannot mint a success, a redirect, or a 5xx) and picks the dialect error type the SDK sees (401 → authentication, 429 → rate-limit, …); message is sanitized. A rejection is a deliberate decision, not a failure: on_error does not apply. Combined with prompt: ro this is the PII-screen primitive: see content, say no, before it leaves your network.
A gate has two more arms, restrict (pin the candidate set to members carrying given tags, persisting across failover) and rewrite (replace the request body; requires prompt: rw), documented in Hooks.
Any non-2xx response, malformed JSON, or timeout applies on_error.

Fail-safety: on_error

The decision is bounded by the hook’s timeout_ms (default 1 ms; the default says hooks are fast; raise it when your hook does I/O or crosses the network). On timeout, crash, garbage reply, or a dead hook, Busbar coerces the decision to the hook’s on_error:

`on_error`	Behavior
`nothing` (default)	The failing gate does not participate: it drops out of the decision entirely and can never displace another gate’s verdict. The right posture for gates whose job is orthogonal to routing.
`weighted`	Falls back to the weighted floor: a broken hook is indistinguishable from no hook. Behaviorally identical to `nothing` in the reconcile; the two names exist so a config reads correctly: `weighted` for ordering hooks, `nothing` for everything else.
`first`	First member in config order; deterministic.
`reject`	Fail closed with a 503, for security gates, where an unscreened request is worse than none. Set this on every security gate.
`<hook-name>`	A named fallback: when this hook fails, that hook fires in its place (projected per its own grants). Its own `on_error` chains further; Busbar proves at boot that every chain terminates (an unknown name, a tap, or a cycle is a startup error). A strategy name (`cheapest`, …) is also a valid, infallible fallback.

Kill an ordering hook mid-traffic and requests keep flowing on the weighted floor. A dropped candidate is demoted, not excluded, so a buggy ranking never strands a healthy model.

A fleet-wide default ordering

A hook registered over the admin API (or a named hook) with default: true becomes the base ordering for every pool that named no strategy of its own, replacing the built-in weighted floor. At most one hook may be the default (a second is an error naming both); a pool that named its own base keeps its choice, and a pool’s own gates layer on top of whatever base it has. No default set means the zero-cost inline weighted backstop. Since 1.5.0 dissolved the top-level hooks: config block, the simplest approach is to name the base strategy per pool (hooks: [cheapest], etc.); reserve default: true for a fleet-wide ordering hook you register and manage over the Admin API.

Full examples

Cost-optimized pool

Route all traffic to the cheapest healthy member. Useful for background jobs or batch workloads where latency is not the primary concern.

rate_card:
  gpt-4o-mini:   { input_utok: 0.15, output_utok: 0.6 }
  claude-sonnet: { input_utok: 3.0,  output_utok: 15.0 }
  gpt-4o:        { input_utok: 5.0,  output_utok: 15.0 }

pools:
  batch:
    hooks: [cheapest]
    failover:
      timeout_secs: 60
      max_hops: 3
    members:
      - { model: gpt-4o-mini }
      - { model: claude-sonnet }
      - { model: gpt-4o }

gpt-4o-mini is tried first. If it is tripped, the breaker skips it and claude-sonnet becomes the first attempt for this request. The failover loop and breaker are unchanged.

Latency-sensitive pool

Route to the fastest-responding member, measured over real traffic. New members start with no latency data and are tried last until they accumulate samples.

pools:
  realtime:
    hooks: [fastest]
    members:
      - { model: claude-sonnet }
      - { model: gpt-4o }
      - { model: gemini-1.5-flash }

Tier-based external hook

Route large requests to a capable model and smaller requests to a cheaper one, using your own hook. Forward the decision to a sidecar with busbar-webrequest-hook, or write your own signed kind: hook plugin.

plugins:
  enabled: true
  dir: /etc/busbar/plugins

rate_card:
  claude-opus:   { input_utok: 15.0, output_utok: 75.0 }
  claude-sonnet: { input_utok: 3.0,  output_utok: 15.0 }

pools:
  smart:
    hooks:
      - { module: busbar-webrequest-hook,
          settings: { url: "https://127.0.0.1:8731/route" },
          kind: gate,
          timeout_ms: 5,               # raise for hooks that do I/O
          on_error: weighted }          # a broken hook falls back to SWRR, never fails
    failover:
      timeout_secs: 60
      max_hops: 3
    members:
      - { model: claude-opus,   tier: large, tags: ["opus"] }
      - { model: claude-sonnet, tier: small, tags: ["sonnet"] }

Your hook receives the request projection (including total_chars, max_tokens, and each candidate’s tier and cost_per_mtok) and returns {"order": [0, 1]} or {"order": [1, 0]} depending on request size (~4 chars/token rule of thumb; 24000 chars ≈ 6k tokens).

Coming from a 1.2.x embedded Rhai script (route: script)? That transport was removed in 1.3; the same logic runs as a small socket hook binary, same ranked-order wire contract, ~100x faster. See the migration guide.

Observability

Response headers. Every request with a non-default ordering emits two headers:

x-busbar-route-policy: <hook>; the hook or strategy that made the decision (e.g. size-router, cheapest)
x-busbar-route-target: <chosen-lane-model>; the model of the chosen lane (e.g. claude-sonnet, gpt-4o-mini)

Prometheus metrics:

Metric	Labels	Description
`busbar_route_policy_selections_total`	`policy`, `pool`	Count of requests where a non-default ordering produced a usable ranked order (incremented once per selection).
`busbar_route_policy_rejections_total`	`policy`, `pool`, `status`	Count of requests deliberately rejected by a hook’s `reject` verb (no upstream dispatched, 4xx to the caller).