Comparison · same box, run it yourself

Busbar vs LiteLLM Rust

LiteLLM shipped the Beta of their Rust AI Gateway, with a staged rollout they've said targets December 2026 for the full server migration. It's the right idea, and on raw speed Busbar comes out ahead: 1.3× lower added latency, 1.4× the throughput, on our own same-box benchmark. I'll show you those numbers. They're near the bottom of the page.

But speed was never the whole point, we just knew it would be a positive side effect of developing in a modern language like Rust. A funded team, three years in, built a Rust gateway that a 3-month-old startup's single binary now beats on speed and covers with six protocols instead of one. Take Busbar's speed as the floor, not the pitch. Here's the pitch.

What an OpenAI-normalized Rust passthrough can't do

1 · Any protocol in, any protocol out. That's the shape, not the language

Busbar's core is a six-protocol superset IR. OpenAI, Anthropic, Gemini, Bedrock, Cohere, and Responses all map into it and back out, both directions. A signed Bedrock boto3 client can reach a Gemini backend and get a valid Bedrock eventstream back. An OpenAI SDK can reach an Anthropic model and get OpenAI-shaped JSON. Every SDK speaks its native dialect in, every backend gets its native dialect out, and not one line of your application changes. How translation works →

LiteLLM normalizes everything to an OpenAI hub. Their Bedrock ingress load-balances across Bedrock deployments and nothing else. Point that boto3 client at a Gemini model: it can't. Point a Gemini client at a Bedrock model: same answer. This is not a speed problem you fix with Rust. A Rust rewrite of an OpenAI-normalized core is still an OpenAI-normalized core. The shape is the ceiling, and Busbar was built without it.

2 · Governance compiled into the binary, not bolted on in Python

Busbar's Rust core is the control plane. Virtual keys with total, daily, and monthly budgets. RPM and TPM limits. Per-pool ACLs, audit, usage metering. All of it in-process, on the hot path, in one binary.

LiteLLM's Rust gateway carries none of it. It's a bare passthrough, and their governance stays behind in the Python proxy they aren't rewriting. So with LiteLLM you choose one: fast Rust, or governance in Python. Busbar refuses the trade. It beats their latency while it authenticates, meters, and routes on every single request. Their number is only close because they do less on the way through.

And Busbar's governance needs no database to do it: enforcement runs from in-memory counters, so a single node is one binary, no PostgreSQL, no Valkey. When you want that state durable or shared across a cluster, 1.5.0 adds drop-in SQLite, Postgres, and Valkey store plugins (signed archives Busbar verifies and loads; no recompile). Optional, not structural. LiteLLM's stack keeps PostgreSQL and Redis as prerequisites of the Python proxy where its governance lives.

3 · Shipped and complete today, not a beta waiting on 2026

All six protocols. Native in-flight failover across protocol families. A plugin system. One static ~12 MB binary. In production, stable, right now.

LiteLLM's Rust core is a single Anthropic /v1/messages endpoint serving only the azure_ai provider, with the full server migration targeted for December 2026. Everything else still routes through the Python sidecar [1]. AI moves fast. You need the right control plane for your infrastructure today, not in six months.

On raw speed: Busbar wins, 1.3× lower added latency and 1.4× the throughput, both compiled Rust. But speed was never the whole gap. A gateway forwards bytes. Busbar is a control plane: on every request it decides whose fault a failure was, whether you're inside budget and rate limits, which backend serves it, and whether to fail over mid-stream, all in the compiled hot path, faster than theirs. Six protocols in and out, governance in the core, shipped today. That isn't just a faster gateway, it's a different category. So why run the one that only forwards, and forwards slower?

The numbers, for completeness. Busbar wins on speed too

I put both on one box and measured everything. Latency and throughput both favor Busbar; they're close (both compiled Rust), but not equal, and I'm not going to round that off to a tie. Memory is where they really aren't close. Busbar idles at 7.4 MB against LiteLLM Rust's ~253 MB.

Memory: Busbar serves at ~34× lighter idle

Idle RSS, serving the endpoint. Their floor is high because the serving config embeds CPython and the full litellm package. That's the "Python in the request path" cost their launch post warns about, still there. Why ~253 MB, not their sub-100 MB target → below.

Added latency & throughput: Busbar's ahead on both

p99 added over the upstream, concurrency 1, on the public harness. Busbar is 1.3× lower.

And Busbar hits that lower number while authenticating, metering, and routing on every request. Their bare passthrough carries none of it. We beat their speed doing strictly more work.

Sustained req/s at p99 < 1 s, zero errors. Max proxy is raw forwarding. @ 20 ms is concurrency under a realistic model delay. Busbar pushes 1.4× the req/s.

Why ~253 MB, not the sub-100 MB they're targeting in their launch post. From the branch their README links (litellm_rust_gateway_v1_messages_route):

/v1/messages serves only azure_ai; their own unit test asserts anthropic returns None.
It serves only under the python-config reader (embedded CPython + the full litellm package). The lean config aimed at their sub-100 MB target returns HTTP 400, so it doesn't serve.
The config that does serve idles at ~252.6 MB on a fresh Graviton box.

Fair's fair: their latency is good. Point me at a branch that serves anthropic natively at a lower footprint and I'll re-run it.

Where LiteLLM Rust is the right call

You already run the LiteLLM proxy and your traffic is Anthropic /v1/messages-shaped. Opting into their Rust core on the endpoints it covers is a low-friction win. No new tool to adopt.
You want the LiteLLM ecosystem (its admin UI, plugin catalog, and provider list), and the Rust core makes the hot path lighter over time without changing the surface you know.
You're validating the compiled-gateway idea on a narrow slice, and you're fine running beta software in that path while their rollout continues toward its December 2026 target. If pre-GA code in front of production traffic is acceptable to you, it's a reasonable early adopt.

If that's your shape, it's a sensible choice, and I say so without hedging.

If this layer carries all your AI traffic (many protocols in their native dialects, one compliant backend, real failover, governance on the path, one thing to run), that's a control plane, not one accelerated endpoint. Get started in five minutes, read why Busbar exists, or if you're on LiteLLM today, here's how to migrate.

How this page is measured. Unless a figure says otherwise, every number here is from the neutral OSS harness (GetBusbar/benchmarking), an AWS m7g.4xlarge (Graviton3, 4-core pin), run the same way as /performance, on Busbar v1.5.5, measured 2026-08-26. Figures tagged "their published" are that project's own numbers; a core-scaling sweep, where one appears, is a separate test, on its own Busbar build, and is labelled as such. And you don't have to take our framing of the results: onthebench.ai publishes the field neutrally, every AI gateway measured identically, side by side, and anyone can re-run it.

References

LiteLLM Rust facts checked against their launch post on 2026-07-20. Something out of date? Tell us and we'll fix it.

LiteLLM Rust launch post: the announcement, early-beta status, staged rollout, Anthropic + Azure-Anthropic /v1/messages only (early beta), and the Python sidecar.
GetBusbar/benchmarking: the public, one-command harness behind every number here. Gateways, mock, load generator, methodology, raw JSON, charts.
Busbar protocols & translation: the six-protocol matrix and how cross-protocol requests are re-encoded natively both ways.

← Performance overview · Busbar vs LiteLLM (Python) · Busbar vs Bifrost