An LLM gateway is the routing layer between your application and the dozen-plus providers you might want to call — a single endpoint that handles multi-provider routing, failover, caching, cost tracking, and (sometimes) observability or governance on top. As production teams have stopped betting on a single model, the gateway has quietly become one of the more load-bearing pieces of an AI stack.
Below are the five gateways we'd actually pick from, ranked by what they're best at — not by who paid for the placement.
Single OpenAI-compatible endpoint to 500+ models across 60+ providers, billed pay-as-you-go.
The default answer for most teams. OpenRouter has the widest model catalog of any gateway (500+ models, 60+ providers) behind a single OpenAI-compatible endpoint, with prepaid pay-per-token billing and automatic failover when a provider goes down.
If your problem is "we want to call any model from one place, today," nothing else gets you there faster. The tradeoff is shallow on observability and governance — pair it with a real eval/observability tool for those.
Open-source Python SDK and proxy that translates requests across 100+ LLM providers into the OpenAI format.
The OSS standard for self-hosted gateway deployments. LiteLLM ships as both a Python SDK and a proxy server that translates 100+ providers into OpenAI-compatible calls. Virtual keys, per-user budgets, and native exporters into Braintrust, Langfuse, and OpenTelemetry mean you don't rebuild instrumentation when you adopt it.
The cost is operational: Redis, Postgres, and the proxy itself are real infrastructure. Worth it if your team has DevOps capacity and your priority is control. Less worth it if you're a five-person product team that just wants routing to work.
Eval-driven dev platform combining traces, datasets, scorers, and a playground in one product.
The right pick if eval-tied routing is the workflow you actually want. Braintrust's gateway exists because their core eval and observability product needed one — so routing, caching, traces, and CI-gated evals all live in one platform. Production traffic flowing through the gateway shows up in the same trace view your dev-time evals run against.
Currently in beta, free during the beta period. Catalog is narrower than OpenRouter or Portkey — but if you're already using Braintrust for evals, the integration is the differentiator and you should adopt this before evaluating standalone gateways. If you're not, OpenRouter or LiteLLM are sharper picks for the routing-only job.
Full-stack AI gateway with the broadest model catalog, built-in guardrails, and enterprise-grade governance.
The pick when "gateway" needs to come with guardrails, governance, and a compliance posture. 1,600+ models — broadest catalog of any gateway with this feature set — plus content moderation, output validation, virtual keys, audit trails, and SOC2 Type 2 / ISO 27001 / GDPR / HIPAA on enterprise tiers.
Pricing is log-based, which means you'll want to model traffic carefully before committing. For regulated industries or platform teams running shared LLM infrastructure across multiple product teams, the platform overhead is justified.
Proxy-based LLM observability — drop in by changing the base URL, no SDK changes needed.
The pick if you want a gateway and basic observability in one product, without taking on a full platform. Helicone is OSS, dual-deployment (cloud or self-hosted), and gives you logging + cost tracking + caching out of the box.
Limited evaluation features — Helicone is observability-first, not eval-first. Free up to 10K requests/month; paid plans from $79/month. The right call when you've outgrown "no gateway" but haven't grown into "we need governance."
How to choose
- Default answer: OpenRouter. Most teams should start here.
- OSS / self-host? LiteLLM.
- Already on Braintrust for evals? Braintrust Gateway.
- Enterprise / regulated industry? Portkey.
- Need a gateway with built-in observability? Helicone.
What we didn't rank
The two adjacent categories worth a mention: cloud providers (Vercel AI Gateway, Cloudflare AI Gateway) bundle gateway features with their existing edge platforms, and may be the right pick if your infrastructure already lives there. And every major model lab (OpenAI, Anthropic, Google) ships first-party gateway-ish features inside their own platforms — fine if you're committed to one provider, less interesting if you're not.
The deeper point: the gateway question and the eval question are separate, but they're more connected than they look. When your gateway and your eval system share a data model, "this prompt regression came from the new model we routed to last Tuesday" becomes a one-click trace. When they don't, it becomes a forensics project.