Documentation
Evals.si evaluates AI before you ship, while it runs, and as you train it. It runs agents, RAG pipelines and models on datasets and sandboxed tasks and fails the build when a gate fails, scores live traces, and turns the same evaluators into RL rewards and checkpoint checks. Agents come first.
Start here
Section titled “Start here”- Get started: install (PyPI, release binaries, charts) and run your first evaluation in a minute.
- Changelog: what changed in each release.
- Examples: runnable configs for runs, agents, guardrails, fine-tuning and more.
- Agent frameworks: score LangGraph, CrewAI, OpenAI Agents SDK, LlamaIndex and Claude Agent SDK traces.
- Trace sources: score the traces you already keep in MLflow, Phoenix or Langfuse.
- Operations: upgrade policy, backup and restore, runbooks and reference cluster values.
Guides
Section titled “Guides”- agent runs
- annotation
- fine tuning
- guardrails
- identity
- kubernetes
- agentgateway kubernetes
- mcp
- plugins
- web ui
- webhooks
- trace sources
- integrate an agent harness
- agent frameworks
- operations
- end to end
- deepeval
- gate an agent in ci
- evaluate rag