Governance

Governance gives you a control plane over who can use which models and how much. Enforcement happens before the request is forwarded: the budget check and the charge are a single atomic operation, and a request that would exceed its cap or RPM limit is rejected at the gate, not reconciled after the bill. It’s off by default; turn it on when you need budgets, rate limits, spend visibility, or access control. Each capability below has two views; read What it does for the why, flip to Configure it for the exact YAML and commands.

Busbar issues virtual keys: scoped bearer tokens your applications present instead of a raw client token. Each key carries its own budget (a spend cap over a total, daily, or monthly window) and its own spend tracking (running spend, tokens, and requests).

Spend is computed from a price you set: a flat charge per request, plus a per-1,000-token charge accrued from each response’s usage. The flat per-request fee is an atomic hard cap: charged in a single conditional UPSERT, so concurrent requests cannot race past the budget; only a single admitted request’s post-response token cost can marginally overshoot. If the budget store itself errors, Busbar fails open by default (your traffic keeps flowing) and can be configured to fail closed.

Why it matters: you get per-key cost attribution and caps without trusting every developer and deployment to self-police. A staging key can be capped at a monthly budget; a production key runs uncapped. Secrets are stored only as SHA-256 hashes and shown in plaintext exactly once.

Scope: control is per key. Busbar doesn’t model users, teams, or per-model budgets; the virtual key is your unit of control.

Enable governance and set pricing + the admin token in config.yaml:

governance:
  enabled: true
  admin_token: "${BUSBAR_ADMIN_TOKEN}"   # guards /admin; required when enabled; boot fails if unset/blank
  price_per_request_cents: 1             # flat charge per request (default 1)
  price_per_1k_tokens_cents: 20          # + per 1k input+output tokens (default 0)
  db_path: busbar-governance.db          # SQLite store (default)

Mint a key; budget and period are set at creation:

curl -s http://localhost:8080/admin/keys \
  -H "Authorization: Bearer $BUSBAR_ADMIN_TOKEN" \
  -H "content-type: application/json" \
  -d '{"name":"staging-app","max_budget_cents":50000,"budget_period":"monthly"}'
# → 201 with the secret, shown once. Store it now; only its SHA-256 is kept.

For a Bedrock-SDK client (inbound SigV4 auth), add "issue_aws_credential": true to the same request; details under Mint a key.

The fail-open default lives in config too: set governance.budget_on_store_error: deny to reject requests on any budget-store error, a hard budget guarantee for regulated deployments.

Check spend any time (the full key lifecycle, PATCH and DELETE included, is under Admin API):

curl -s http://localhost:8080/admin/keys/<id>/usage \
  -H "Authorization: Bearer $BUSBAR_ADMIN_TOKEN"
# → { "id": "...", "spend_cents": ..., "tokens": ..., "requests": ... }

Rate limits

What it does
Configure it

Each virtual key can carry a requests-per-minute (RPM) and a tokens-per-minute (TPM) cap. RPM is enforced precisely: the counter increments synchronously on admission. TPM is best-effort under concurrency, since token counts are known only after the response.

Why it matters: you can rate-limit an internal tool independently of the production path, or keep a noisy batch job from starving interactive traffic; per key, without writing limiter code.

Set limits at key creation (or update them later). Omit a field to leave it unlimited:

curl -s http://localhost:8080/admin/keys \
  -H "Authorization: Bearer $BUSBAR_ADMIN_TOKEN" \
  -H "content-type: application/json" \
  -d '{"name":"internal-tool","rpm_limit":120,"tpm_limit":200000}'

Model access

What it does
Configure it

A key’s allowed_pools list restricts which pools it may target. An empty list means all pools are allowed. Access is scoped to pools: the named routing targets you define; not to individual models or named access groups.

Why it matters: a cheap-staging key can be locked to a low-cost pool; a partner key can be fenced to exactly the pools you intend to expose, with no way to reach the rest.

curl -s http://localhost:8080/admin/keys \
  -H "Authorization: Bearer $BUSBAR_ADMIN_TOKEN" \
  -H "content-type: application/json" \
  -d '{"name":"partner-key","allowed_pools":["public-fast","public-smart"]}'

The key now returns 403 if it targets any pool outside that list.

Admin API

All /admin routes require the configured admin token, sent as Authorization: Bearer <admin_token> or X-Admin-Token: <admin_token>. These are not virtual keys, and they are not the vendor SDK carriers (x-api-key / x-goog-api-key); the admin token is a single static credential set in governance.admin_token.

Mint a key

curl -s -X POST http://localhost:8080/admin/keys \
  -H "Authorization: Bearer $BUSBAR_ADMIN_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
        "name": "team-ml",
        "allowed_pools": ["fast", "overflow"],
        "max_budget_cents": 50000,
        "budget_period": "monthly",
        "rpm_limit": 600,
        "tpm_limit": 200000
      }'

Response includes id (vk_<16hex>), secret (sk-bb-<32hex>, shown once), and all attributes. Store the secret immediately.

To also issue an AWS credential pair for Bedrock-SDK clients, add "issue_aws_credential": true to the request body. The 201 response then additionally includes aws_access_key_id and aws_secret_access_key: both shown once and never returned by any subsequent read API. Configure your Bedrock SDK with those credentials; Busbar verifies the inbound SigV4 signature and enforces the key’s governance controls. See Bedrock ingress.

Create-key field validation: budget_period must be total, daily, or monthly (400 otherwise); max_budget_cents must be ≥ 0; rpm_limit and tpm_limit must each be ≥ 1 when set. allowed_pools that name no configured pool logs a warning but does not fail.

List keys

curl -s http://localhost:8080/admin/keys \
  -H "Authorization: Bearer $BUSBAR_ADMIN_TOKEN"

Returns key metadata. Secret hashes are never returned.

Check usage

curl -s "http://localhost:8080/admin/keys/vk_abc123def456.../usage" \
  -H "Authorization: Bearer $BUSBAR_ADMIN_TOKEN"

Returns { "spend_cents": ..., "tokens": ..., "requests": ... } for the current budget window. 404 if the key does not exist.

Update a key

curl -s -X PATCH "http://localhost:8080/admin/keys/vk_abc123def456..." \
  -H "Authorization: Bearer $BUSBAR_ADMIN_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{ "rpm_limit": 1200, "max_budget_cents": null }'

Absent fields are left unchanged. null clears a cap to unlimited. Providing a value sets it. Same validation as create. 404 if the key does not exist.

Revoke a key

curl -s -X DELETE "http://localhost:8080/admin/keys/vk_abc123def456..." \
  -H "Authorization: Bearer $BUSBAR_ADMIN_TOKEN"

Returns 404 if the key does not exist (delete is not idempotent). After deletion, the key’s sk-bb-… secret is immediately rejected with a native 401.

Enforcement model and precision

Understanding where each limit is precise and where it is approximate is important for setting realistic budgets:

RPM: precise. The counter is incremented synchronously on admission, before the request is forwarded. A request that would exceed the limit is rejected before any upstream call is made.

TPM: best-effort. Token counts are recorded post-response, after the upstream reports them. In-flight concurrent requests do not yet contribute to the window. The first request of each new 60-second window is always admitted regardless of the previous window’s total. If multiple large concurrent requests arrive simultaneously at the window boundary, all may be admitted before the TPM limit kicks in.

Budget: atomic hard cap. The over-budget check and the spend charge are performed as a single atomic SQLite UPSERT (charge_within_budget). Concurrent requests cannot cause an overshoot; if the cap is reached mid-burst, the excess requests are rejected. On a store error during the admission check, behavior is controlled by governance.budget_on_store_error: the default allow fails open (preserves availability); set deny for a hard budget guarantee that rejects on any store error. A definitive over-budget result always rejects regardless of this setting. Budget is a true spending ceiling when deny is used, and a reliable guardrail with allow.

allowed_pools ACL: precise and pre-forwarding. A key with allowed_pools: ["fast"] trying to target overflow or any direct model route not in the list gets 403 before any upstream call.

Rate windows are per-process, in-memory. Running multiple Busbar instances against the same governance database means RPM/TPM windows are per-instance, not shared. For distributed rate limiting, run one instance per governance database or front multiple instances with an upstream rate limiter.