AI Gateway
An API Evangelist landscape index of AI gateways — the LLM routers, prompt firewalls, model fallback proxies, cost-control planes, and policy engines that sit between applications and AI providers. AI gateways unify access across OpenAI, Anthropic, Google, AWS Bedrock, Azure OpenAI, and self-hosted models behind a common interface and apply caching, routing, guardrails, observability, rate limiting, budgets, RBAC, and audit controls. This index catalogs commercial SaaS gateways, open-source projects, API gateway AI plugins, and cloud-provider AI proxies, with a shared schema and vocabulary for describing model routes, fallbacks, guardrails, and budgets across vendors.
Limited machine-readable signal and partial portal coverage — documentation a human can read, but little a machine or agent can consume without scraping.
API Evangelist profiles AI Gateway the way a machine reads it — 84 machine-readable artifacts across 38 APIs, pulled from the provider's own public surface and indexed so a developer, an analyst, or an AI agent can evaluate it against every other provider on the network.
Every provider in the network is reduced to the same set of machine-readable artifacts — OpenAPI contracts, event specifications, GraphQL schemas, runnable collections, pricing and rate-limit signals, security posture, OAuth scopes, and the agent surfaces (MCP servers and skills) that let software drive the API on its own. We profile them because the interface is the part of a company you can actually inspect: it is a truer signal of what a provider does than any marketing page. From those artifacts we compute the Kin Score — AI Gateway scores 37.9/100 (thin), with a separate agent-readiness read of 48/100 (agent ready). The full breakdown is below, followed by every artifact we hold — each card links through to its machine-readable definition on apis.io.
Kin Score
This is the API Evangelist rating — a single, repeatable read computed from the artifacts on this page. Green fill is points earned; the red track is points possible, so every bar shows earned-versus-possible at a glance.
Kin Score
How this is scored →scored 2026-07-27 · rubric v0.5
How we profile AI Gateway
Each block below is one kind of artifact we hold for AI Gateway. For each we say what it is and why it earns a place in the profile, then list every one we've indexed — capped at two rows, scroll within the panel for the rest.
APIs 38
Each API is captured as its own OpenAPI definition — every operation, parameter, and response. This is the single most useful machine-readable description of what an API does, and it's what lets us score, lint, mock, and generate against it without asking the provider for anything.
Individual APIs this provider publishes, each with its own machine-readable definition.
Portkey
Portkey is a production-grade AI gateway and control plane that fronts 1,600+ LLMs with unified routing, fallbacks, semantic caching, guardrails, cost attribution, and prompt ma...
OpenRouter
OpenRouter is a unified inference marketplace exposing 400+ models from 60+ providers behind one OpenAI-compatible API, with automatic provider fallback, pay-as-you-go credits, ...
LiteLLM
LiteLLM (BerriAI) is an open-source LLM gateway that exposes 100+ LLM providers — OpenAI, Anthropic, Azure, Bedrock, Gemini — through a single OpenAI-compatible API. The LiteLLM...
Helicone
Helicone is an open-source AI observability and routing platform centered on requests, sessions, prompts, datasets, rate limits, and alerts. Integrates with OpenAI, Anthropic, G...
Cloudflare AI Gateway
Cloudflare AI Gateway is an edge-deployed proxy that fronts AI providers — Workers AI, Anthropic, Google Gemini, OpenAI, Replicate, and more — with caching, rate limiting, analy...
Kong AI Gateway
The Kong AI Gateway is delivered as the AI Proxy plugin for Kong Gateway, transforming and proxying requests across 16+ providers including OpenAI, Azure OpenAI, Anthropic, Amaz...
Apache APISIX AI Proxy
The Apache APISIX ai-proxy plugin streamlines integration with LLMs by converting plugin settings into the appropriate request format for OpenAI, DeepSeek, Azure OpenAI, Anthrop...
Tetrate Agent Router Service
Tetrate Agent Router Service is an Envoy AI Gateway-as-a-service from the creators of Envoy, providing an approved LLM catalog, unified model access, automatic fallback, cost ma...
NVIDIA NIM
NVIDIA NIM is a set of inference microservices for streamlined AI model deployment, prebuilt and optimized for low-latency, high-throughput inference on NVIDIA-accelerated infra...
Traefik AI Gateway
Traefik AI Gateway is an enterprise, self-hosted, Kubernetes-native AI gateway with safety and governance (NVIDIA Safety NIMs, jailbreak detection, content filtering across 22+ ...
Together AI
Together AI is a full-stack AI Native Cloud for inference, fine-tuning, and GPU clusters powered by research, exposing serverless inference, batch processing, dedicated model an...
Anyscale
Anyscale is the production-scale AI platform built on Ray by the creators of Ray, supporting LLM inference and other data-intensive AI workloads across distributed GPU clusters....
LangDB
LangDB is an enterprise AI gateway for routing and governing LLM traffic across providers, with observability, cost tracking, and policy enforcement. Public homepage was unreach...
Envoy AI Gateway
Envoy AI Gateway is an open-source extension to Envoy Proxy and Envoy Gateway, providing a Kubernetes-native AI traffic plane for routing, governing, and observing LLM calls acr...
Gentrace
Gentrace was an AI evaluation and observability product; the company has shut down and its codebase is now MIT-licensed open source on GitHub. Included here for historical compl...
AI Gateway Analytics API
The Analytics API from AI Gateway — 2 operation(s) for analytics.
AI Gateway APIKeys API
The APIKeys API from AI Gateway — 1 operation(s) for apikeys.
AI Gateway Assistants API
The Assistants API from AI Gateway — 1 operation(s) for assistants.
AI Gateway Audio API
The Audio API from AI Gateway — 3 operation(s) for audio.
AI Gateway Batches API
The Batches API from AI Gateway — 2 operation(s) for batches.
AI Gateway Chat API
The Chat API from AI Gateway — 1 operation(s) for chat.
AI Gateway Completions API
The Completions API from AI Gateway — 1 operation(s) for completions.
AI Gateway Configs API
The Configs API from AI Gateway — 1 operation(s) for configs.
AI Gateway Embeddings API
The Embeddings API from AI Gateway — 2 operation(s) for embeddings.
AI Gateway Feedback API
The Feedback API from AI Gateway — 1 operation(s) for feedback.
AI Gateway Files API
The Files API from AI Gateway — 2 operation(s) for files.
AI Gateway FineTuning API
The FineTuning API from AI Gateway — 2 operation(s) for finetuning.
AI Gateway Guardrails API
The Guardrails API from AI Gateway — 1 operation(s) for guardrails.
AI Gateway Images API
The Images API from AI Gateway — 1 operation(s) for images.
AI Gateway Integrations API
The Integrations API from AI Gateway — 2 operation(s) for integrations.
AI Gateway Logs API
The Logs API from AI Gateway — 2 operation(s) for logs.
AI Gateway MCP API
The MCP API from AI Gateway — 2 operation(s) for mcp.
AI Gateway Policies API
The Policies API from AI Gateway — 2 operation(s) for policies.
AI Gateway Prompts API
The Prompts API from AI Gateway — 6 operation(s) for prompts.
AI Gateway Responses API
The Responses API from AI Gateway — 1 operation(s) for responses.
AI Gateway Threads API
The Threads API from AI Gateway — 3 operation(s) for threads.
AI Gateway VirtualKeys API
The VirtualKeys API from AI Gateway — 1 operation(s) for virtualkeys.
AI Gateway Workspaces API
The Workspaces API from AI Gateway — 4 operation(s) for workspaces.
Scroll within the panel for all 38 ·
Open Collections 1
Open, tool-agnostic collections carry the same runnable value as Postman without locking you to one client — the portable, forkable form of the same exercise.
Open, tool-agnostic API collections (OpenAPI-derived and Bruno).
Portkey AI Gateway API
OPEN COLLECTIONFeatures 13
The notable capabilities this provider advertises, captured as structured features so they can be searched and compared instead of read one landing page at a time.
Notable capabilities this provider offers.
Provider Abstraction
A unified, typically OpenAI-compatible API surface that lets clients call any supported LLM provider without provider-specific SDK juggling.
Model Routing
Route requests to the right model and provider based on alias, header, request content, identity, time-of-day, cost, or latency.
Fallback and Failover
Automatically retry failed requests against backup providers or models when a primary upstream is degraded, rate-limited, or down.
Load Balancing and Fanout
Distribute traffic across multiple providers or replicas using weighted, priority-based, or RPM/TPM-aware load balancing.
Response Caching
Exact-match and semantic caching of model responses to cut latency and provider spend; some gateways claim 40-70 percent cost savings.
Cost Controls and Budgets
Per-user, per-team, per-key, per-project budgets, spend tracking, and hard or soft caps on token consumption.
Rate Limiting and Quotas
RPM, TPM, concurrency, and per-key quotas enforced at the gateway, decoupled from each upstream provider's limits.
Guardrails and Prompt Firewall
Prompt injection detection, jailbreak filtering, content moderation, PII redaction, and topic control applied to requests and responses.
Observability
Request, response, token, cost, latency, error, and trace data exported via OpenTelemetry, Langfuse, Phoenix, Langsmith, or built-in dashboards.
Authentication and RBAC
Virtual keys, JWT, OAuth2, SSO, and role-based access control over which clients can use which models with which budgets.
BYOK and Secret Management
Bring-your-own provider API keys, with the gateway holding and injecting them so clients never see upstream credentials.
Multi-Tenant Governance
Per-tenant isolation of keys, budgets, logs, and policies for platform teams serving multiple internal product teams.
MCP Federation
Some AI gateways also front Model Context Protocol servers, aggregating tools and exposing a single MCP endpoint to agents.
Scroll within the panel for all 13 ·
Semantic Vocabularies 1
JSON-LD contexts give the data shared meaning across APIs. We profile them because semantics are what let a machine reconcile 'customer' here with 'customer' somewhere else.
JSON-LD contexts and semantic vocabularies used across these APIs.
Ai Gateway Context
JSON-LDSpectral Rules 1
Governance rulesets we run against this provider's specs — the automated checks behind parts of the score. Profiling them makes the quality bar explicit and re-runnable, not a matter of opinion.
AI Gateway API Rules
SPECTRALJSON Schema 3
Standalone JSON Schema definitions describe the data models behind the API. We profile them so the shapes are validatable on their own — useful long after a single request is forgotten.
Standalone JSON Schema definitions for this provider's data models.
JSON Structure 3
JSON Structure captures the data shapes in a form built for tooling — a complement to JSON Schema that keeps the model machine-legible.
JSON Structure definitions describing this provider's data shapes.
Ai Gateway Policy Structure
JSON STRUCTUREAi Gateway Provider Structure
JSON STRUCTUREAi Gateway Route Structure
JSON STRUCTUREExamples 6
Real request and response payloads are what turn a spec from abstract into obvious — and they're one of the twelve things an agent needs to call an API correctly on the first try.
Example request and response payloads for these APIs.
Ai Gateway Provider Example
EXAMPLEAi Gateway Route Example
EXAMPLESecurity Posture 2
Authentication, domain security, vulnerability disclosure, and trust-center signals — the evidence that a provider takes security seriously enough to document it. We profile it because you can't govern what you can't see.
Authentication, domain security, vulnerability disclosure, and trust-center signals.
Agentic Access 1
An x-agentic-access contract marks which operations are safe for an agent to run on its own and which need a human in the loop. It is the difference between an API an agent can use and one it can use safely.
Recommended x-agentic-access execution contracts for AI agents.
Use Cases 6
What developers actually build with this provider — captured so the catalogue answers 'what is this for', not just 'what does this expose'.
What developers build with this provider.
Provider-Agnostic LLM Access
Front many LLM providers behind one API so application teams can switch models without changing client code.
Cost Containment for AI
Apply caching, routing to cheaper models, and per-team budgets to keep generative-AI spend predictable.
Reliability and Failover
Survive single-provider outages by automatically failing over to backup models when the primary degrades.
Centralized AI Governance
Enforce content, PII, and policy controls in one place for every AI request leaving the organization.
Observability and FinOps
Attribute cost and latency to teams, projects, and users; expose token-level metrics to FinOps and SRE.
Multi-Tenant AI Platforms
Build internal AI platforms where each product team gets its own virtual keys, budgets, and logs.
Integrations 9
Pre-built integrations with other platforms tell you where this provider already fits in a stack.
Pre-built integrations with other platforms and tools.
OpenAI
Front OpenAI's GPT, embeddings, and image models behind the gateway with virtual keys and budgets.
Anthropic
Route Claude requests through the gateway for fallback, caching, and central observability.
Google Gemini and Vertex AI
Proxy Google Gemini and Vertex AI calls with OpenAI-format translation where supported.
AWS Bedrock
Bridge OpenAI-format clients to Bedrock-hosted Anthropic, Mistral, Cohere, Meta, and Amazon models.
Azure OpenAI
Route to Azure-hosted OpenAI deployments with per-region failover and key rotation.
Ollama and vLLM
Front self-hosted Ollama and vLLM inference servers for hybrid cloud and on-prem inference.
OpenTelemetry
Export request, token, cost, and trace data to any OTel-compatible observability backend.
Langfuse and Phoenix
Stream prompts, completions, and evaluations to Langfuse and Arize Phoenix for prompt and model analytics.
Model Context Protocol
Some AI gateways federate MCP servers alongside LLM routes, exposing a unified agent endpoint.
Scroll within the panel for all 9 ·
Resources
Every other property we hold for AI Gateway — documentation, portals, status pages, policies, and corporate surface — grouped by the job it does, following the integrator's arc from getting started to running in production.
Get Started 1
Portal, sign-up, and the first successful call
Documentation 3
Reference material describing how the API behaves
Agent Surfaces 1
MCP servers, agent skills, and machine-readable catalogs
Design & Contract 5
Pagination, idempotency, versioning, errors, and events
Build 1
SDKs, sample code, and the tooling you integrate with
Access & Security 2
Authentication, authorization, and security posture
Company 1
The organization behind the API
← All providers · Data indexed from github.com/api-evangelist/ai-gateway · machine-readable index on apis.io