Benchmark

“How much latency does Busbar add?” has one honest answer: its added latency, the microseconds Busbar itself spends parsing your request, translating it, and serializing the response, not the network and not the model. Busbar measures exactly that and reports it in-band on every response (when enabled; see below), in a standard header:

Server-Timing: busbar;dur=0.087

busbar;dur is total request time minus the upstream round-trip: everything Busbar did, nothing the upstream did. It’s a per-request number on Busbar’s own clock, so it is valid at the tail (each request is its own paired measurement) and independent of the upstream’s speed, which is what lets us measure the same number against an instant mock and against the real Anthropic API.

Every result below changes exactly one variable. If a benchmark moves two things at once, none of its numbers mean anything, so we don’t.

Result

The setup (identical for every row)

The units, up front, so every number reads the same way:

• µs: microsecond. 1 µs = 1/1,000,000 of a second (and 1 ms = 1,000 µs). All latency is in µs.
• tok: token, the unit LLM APIs bill in (roughly 4 characters of text each). All payload sizes are in tok.
• p50 / p99 / p99.9: percentiles. p50 is the median (half are faster); p99 means 1 in 100 is slower; p99.9 is the tail, 1 in 1,000.

One AWS c7g.2xlarge (8 vCPU, Graviton3) in us-east-1, Amazon Linux 2023, running the released v1.0.0 aarch64 binary. Busbar is taskset-pinned to 2 dedicated cores; the load generator and the mock get different cores. The headline metric is Busbar’s own Server-Timing: busbar;dur= value, captured per request; Busbar’s CPU is logged from /proc. Each cell is thousands of requests at concurrency 1 (the per-request floor).

Result 1: added latency, swept by payload (Anthropic, vs mock)

Hold the protocol (Anthropic) and the upstream (instant mock) fixed; vary the payload. Two paths: a same-protocol passthrough, and a full cross-protocol translation (Anthropic in → OpenAI upstream → converted both ways). busbar;dur, in µs:

Path	Payload	p50	p99	p99.9
Same protocol (Anthropic→Anthropic)	~8 tok	38	45	51
	~1k tok	41	49	57
	~12k tok	93	137	147
Cross-protocol translate (Anthropic↔OpenAI)	~8 tok	48	55	62
	~1k tok	52	60	65
	~12k tok	84	158	174

It scales with bytes, not surprises. A tiny call is 38µs; a full cross-protocol translation of a ~12k-token (about 32 KB) body (parse, internal representation, re-serialize into a different protocol, both directions) is still 84µs. A fixed per-request floor plus the bytes Busbar parses and serializes; no allocation cliff. Busbar’s CPU stays ~0.5 of one core at concurrency 1.

The tail stays tight. Busbar is a single Rust binary with no garbage collector, so nothing in the request path pauses to sweep memory. Its p99.9 is only ~1.3 to 1.6× its p50 on small/medium payloads (and under ~2× even on the 12k-token translate); the added latency does not blow out into a tail. A garbage-collected proxy (Python/Node/JVM) pays an occasional GC pause on some requests, swelling p99/p99.9 by far more even when the median looks fine; we report p99.9 precisely so that’s visible.

Throughput, so “under one core” isn’t misread as a cap: saturating its 2 pinned cores Busbar sustained 19,505 req/s (~9,750 per core), and the Tokio runtime scales across cores.

Result 2: added latency by protocol (all six, vs mock)

Now hold the upstream (mock) and payload (~1k tok) fixed and vary only the wire protocol: same-protocol passthrough on each of Busbar’s six. This explains why it isn’t a single number: different formats cost different amounts to parse and serialize. busbar;dur p50 / p99, µs:

Protocol	p50	p99	note
OpenAI	37	43
Anthropic	42	49
Responses	37	44
Cohere	37	44
Gemini	40	47
Bedrock	63	76	+ SigV4 request signing

They cluster in the same tens-of-µs band; Bedrock is the outlier because it’s the one protocol that cryptographically signs every upstream request (AWS SigV4, an HMAC-SHA256 chain over the body), which is real CPU the others don’t pay. That’s an honest cost of speaking Bedrock natively, surfaced rather than hidden.

Result 3: invisible in production (paired, vs real Anthropic)

Change the upstream to the real api.anthropic.com and measure it honestly: direct and through-Busbar requests are interleaved (direct, Busbar, direct, Busbar, …) in the same window, so Anthropic’s minute-to-minute drift hits both equally and cancels in the delta. Small ~8 tok calls to claude-haiku-4-5, median client wall-clock:

Path (paired, interleaved)	p50 wall-clock
Direct (client → Anthropic)	807ms
Through Busbar (client → Busbar → Anthropic)	790ms
Difference	−17ms

Through Busbar was 17ms faster at the median. That of course doesn’t mean Busbar speeds up Anthropic. It means Busbar’s cost is below the provider’s own run-to-run noise: in a paired test where drift cancels, the difference lands on either side of zero. The interleaving is the whole point. Measure the two sides in separate passes and the provider’s own minute-to-minute drift masquerades as Busbar overhead; alternate them request-by-request and that drift hits both equally and subtracts out. We report only the median: at p99 the paired sample (n≈150 real calls) is dominated by provider-tail jitter, not Busbar.

And on those same requests, Busbar’s own self-reported number: busbar;dur p50 = 84µs. That’s the actual compute, about 0.01% of a ~790ms model call, and you can read it on your own traffic.

The one thing we won’t hide: Busbar is a proxy, so it adds one network hop (client to Busbar to provider). Here client and Busbar are co-located, so that hop is sub-millisecond and lost in the noise above; co-locate Busbar with your app in production and it stays sub-millisecond. We don’t claim zero. We claim small, and measured both ways.

Reproduce

One self-cleaning script on one named instance type; the upstream is the only thing that changes between Result 1, Result 2, and Result 3.

1. Launch a c7g.2xlarge (Graviton3, arm64), Amazon Linux 2023, us-east-1.

2. Build the released binary, stand up a tiny instant mock that speaks all six protocols, pin the cores:

curl -fsSL https://getbusbar.com/install.sh | sh   # or build the tagged source
# mock: a small Go HTTPS server returning a valid response for each of the six wire formats,
#       sized to the request's max-tokens; served over a real Let's Encrypt cert on a loopback host
#       (127-0-0-1.local-ip.sh → 127.0.0.1) so the release binary's public-CA-only upstream TLS trusts it.
taskset -c 0,1 ./busbar ... &     # Busbar -> 2 dedicated cores
taskset -c 6,7  ./mock  ... &     # mock   -> its own cores

3. Fire requests pinned to the remaining cores and read the Server-Timing: busbar;dur= header off each response; that is the metric. For Result 3, interleave direct and through-Busbar requests so provider drift cancels. The full self-contained script (launch → pin → measure → log CPU → tear down) is published with the release.

Why these choices

• Server-Timing, not a client stopwatch: Busbar’s own clock excludes the upstream round-trip and the client’s jitter, and it’s a W3C-standard header you can read in DevTools or any APM tool, on your own production traffic.
• A fixed cloud instance, pinned cores, logged CPU: reproducible by anyone; the number is the cost of one path of work, not a big box. “I used my MacBook” is not a citable number.
• Interleaved for the real-provider comparison: the only way to separate Busbar’s cost from a provider whose own latency drifts minute to minute.

Why a standard header. Server-Timing is W3C-standard, so you can read busbar;dur in browser DevTools or any APM tool, on your own production traffic, with no special tooling. The overhead number isn’t a one-time lab claim; it’s something you can verify continuously, on your own requests.

Enabling it. The Server-Timing header is opt-in (default off) to avoid fingerprinting Busbar on unauthenticated responses. Set observability.emit_server_timing: true in config.yaml to turn it on.