Promptfoo
Promptfoo is an open-source LLM evaluation and red-teaming framework distributed as a TypeScript CLI and Node.js library under the MIT license. Developers use it to evaluate prompts, models, and RAG pipelines side by side, run automated red team attacks against LLM applications, scan code for LLM vulnerabilities in IDE and CI, and proxy Model Context Protocol traffic. Promptfoo also operates a commercial Enterprise platform at promptfoo.app for teams that need centralized governance, remediation reports, and shared evaluation history.
More than an index entry, but the surface is still mostly links rather than artifacts — the cohort most likely to move a full band from modest, well-targeted work.
API Evangelist profiles Promptfoo the way a machine reads it — 33 machine-readable artifacts across 6 APIs, pulled from the provider's own public surface and indexed so a developer, an analyst, or an AI agent can evaluate it against every other provider on the network.
Every provider in the network is reduced to the same set of machine-readable artifacts — OpenAPI contracts, event specifications, GraphQL schemas, runnable collections, pricing and rate-limit signals, security posture, OAuth scopes, and the agent surfaces (MCP servers and skills) that let software drive the API on its own. We profile them because the interface is the part of a company you can actually inspect: it is a truer signal of what a provider does than any marketing page. From those artifacts we compute the Kin Score — Promptfoo scores 29.3/100 (emerging), with a separate agent-readiness read of 7/100 (human only). The full breakdown is below, followed by every artifact we hold — each card links through to its machine-readable definition on apis.io.
Kin Score
This is the API Evangelist rating — a single, repeatable read computed from the artifacts on this page. Green fill is points earned; the red track is points possible, so every bar shows earned-versus-possible at a glance.
Kin Score
How this is scored →scored 2026-07-27 · rubric v0.5
How we profile Promptfoo
Each block below is one kind of artifact we hold for Promptfoo. For each we say what it is and why it earns a place in the profile, then list every one we've indexed — capped at two rows, scroll within the panel for the rest.
APIs 6
Each API is captured as its own OpenAPI definition — every operation, parameter, and response. This is the single most useful machine-readable description of what an API does, and it's what lets us score, lint, mock, and generate against it without asking the provider for anything.
Individual APIs this provider publishes, each with its own machine-readable definition.
Promptfoo CLI
The Promptfoo CLI is the primary entry point for running prompt and model evaluations from the command line or CI. It is installable via npm, Homebrew, pip, or npx and reads a Y...
Promptfoo Node.js Library
The Promptfoo Node.js package exposes the same evaluation engine programmatically so developers can embed evaluations, assertions, and dataset runs directly into JavaScript and ...
Promptfoo Red Team
Promptfoo Red Team generates adversarial test cases against LLM applications targeting prompt injection, jailbreaks, PII leakage, bias, and other OWASP LLM Top 10 categories. Ru...
Promptfoo Enterprise
Promptfoo Enterprise is the commercial SaaS at promptfoo.app providing centralized evaluation history, shared red team findings, remediation reports, role-based access control, ...
Promptfoo MCP Proxy
MCP Proxy is Promptfoo's security gateway for Model Context Protocol traffic, inspecting tool calls and responses flowing between agents and MCP servers to enforce policies and ...
Promptfoo Code Scanning
Promptfoo Code Scanning analyzes source code in IDEs and CI pipelines to find LLM-related vulnerabilities including unsafe prompt construction, missing guardrails, and risky too...
Pricing Plans 1
Pricing is part of the interface. Machine-readable plans tell you what a tier costs and includes before you commit — one of the six things the Kin Score reads for commercial clarity.
Published pricing tiers and plan structures.
Rate Limits 1
Rate limits are the difference between a demo that works and a production integration that doesn't fall over. Publishing them is an operational-transparency signal — and a hard requirement for any agent that plans its own throughput.
Documented rate limits and quota policies.
Promptfoo Rate Limits
RATE LIMITSFinOps 1
Cost, billing, and metering signals let a buyer model the financial operations of an API before it's live. We profile them for the same reason we profile pricing: the money is part of the contract.
Cost, billing, and metering signals for API financial operations.
Promptfoo Finops
FINOPSFeatures 8
The notable capabilities this provider advertises, captured as structured features so they can be searched and compared instead of read one landing page at a time.
Notable capabilities this provider offers.
Side-by-Side Evaluation
Compare prompts and models across providers with assertion-based scoring.
Red Teaming
Automated adversarial test generation across prompt injection, jailbreaks, PII, bias, and more.
RAG Evaluation
Evaluate retrieval-augmented generation pipelines for accuracy, faithfulness, and groundedness.
CI/CD Integration
Run evaluations on every pull request with exit-code-driven gating.
Local Web Viewer
Inspect evaluation runs and diffs in a local browser UI without sending data to a third party.
MCP Proxy
Inspect and govern Model Context Protocol traffic between agents and tool servers.
Code Scanning
Static analysis of LLM application code in IDEs and CI to surface risky patterns.
Self-Hosting
Open source CLI and library run locally with no telemetry to Promptfoo required.
Scroll within the panel for all 8 ·
Security Posture 2
Authentication, domain security, vulnerability disclosure, and trust-center signals — the evidence that a provider takes security seriously enough to document it. We profile it because you can't govern what you can't see.
Authentication, domain security, vulnerability disclosure, and trust-center signals.
Use Cases 5
What developers actually build with this provider — captured so the catalogue answers 'what is this for', not just 'what does this expose'.
What developers build with this provider.
Prompt Iteration
Compare prompt variants against datasets to choose the best-performing version.
Model Selection
Benchmark candidate models across providers before committing to one in production.
RAG Quality Assurance
Evaluate chunking, retrieval, and generation choices in RAG systems.
AI Red Teaming
Probe pre-production LLM applications for adversarial failure modes.
Continuous LLM Testing
Wire evaluations and red team scans into CI so regressions block deploys.
Integrations 9
Pre-built integrations with other platforms tell you where this provider already fits in a stack.
Pre-built integrations with other platforms and tools.
OpenAI
Native provider for OpenAI Chat Completions, Assistants, and Responses APIs.
Anthropic
Native provider for Anthropic Claude models.
Azure OpenAI
Provider for Azure-hosted OpenAI deployments.
AWS Bedrock
Provider for Anthropic, Meta, Mistral, and other models on Bedrock.
Google Vertex AI
Provider for Gemini and other models on Vertex.
Ollama
Provider for locally hosted open source models via Ollama.
LangChain
Evaluate LangChain chains and agents through custom providers.
GitHub Actions
Run promptfoo evaluations in CI on every pull request.
Model Context Protocol
MCP Proxy and MCP provider support for agentic workflows.
Scroll within the panel for all 9 ·
Resources
Every other property we hold for Promptfoo — documentation, portals, status pages, policies, and corporate surface — grouped by the job it does, following the integrator's arc from getting started to running in production.
Get Started 2
Portal, sign-up, and the first successful call
Documentation 2
Reference material describing how the API behaves
Build 1
SDKs, sample code, and the tooling you integrate with
Access & Security 2
Authentication, authorization, and security posture
Operate 2
Status, limits, changes, and where to get help
Commercial 2
Pricing, plans, and the legal terms of use
Company 3
The organization behind the API
← All providers · Data indexed from github.com/api-evangelist/promptfoo · machine-readable index on apis.io