Companies

ML observability platform extended into LLMs, with the open-source Phoenix framework as a popular standalone trace viewer.

Eval-driven dev platform combining traces, datasets, scorers, and a playground in one product.

Open-source LLM evaluation and observability from a mature MLOps team — credible Langfuse alternative.

APM giant with bolted-on LLM observability for OpenAI and Anthropic calls.

pytest-style LLM evaluation framework with synthetic dataset generation and CI/CD-native testing.

Open-source ML and LLM evaluation framework with strong methodology docs — building blocks, not a finished platform.

Enterprise ML governance platform extended to LLMs and generative AI, with audit-ready traces and in-environment evaluations.

Agent reliability platform with cheap, fast evaluators that can run on every request in production.

Open-source platform for building RL environments and evals for computer-use agents — used by frontier labs, ships its own benchmarks.

Open-source data annotation platform with rubric enforcement, escalation workflows, and audit trails — extended to LLM review.

Open-source LLM observability with evals, prompt management, and best-in-class tracing.

Observability and evaluation built by the LangChain team — best-in-class if your stack is LangChain or LangGraph.

Open-source Python SDK and proxy that translates requests across 100+ LLM providers into the OpenAI format.

AI quality evaluation platform with prebuilt and custom scorers, designed to plug into existing observability stacks.

Open-source MLOps standard with LLM tracing, evaluation, and prompt management bolted on top.

Single OpenAI-compatible endpoint to 500+ models across 60+ providers, billed pay-as-you-go.

Full-stack AI gateway with the broadest model catalog, built-in guardrails, and enterprise-grade governance.

Open-source CLI for evaluating LLM prompts and red-teaming applications, with YAML/JSON configs that live next to your code.

Git-style version control for prompts — branch, commit, merge, and CI-gate prompt changes.

Visual prompt editor and version control built for non-technical teams.

Open-source evaluation framework purpose-built for RAG pipelines, with reference-free metrics that became the industry standard.

Annotation platform with strong tooling for measuring and resolving disagreements between human reviewers and automated scorers.

Visual workflow builder with built-in observability for low-code agent development.

LLM tracing, evaluation, and prompt management embedded inside the Weights & Biases ML platform.

Open-source MLOps and LLMOps framework for building reproducible, infrastructure-agnostic AI pipelines.