MCP Servers
iris-eval/mcp-server
The agent eval standard for MCP — score output quality, catch safety failures, enforce cost budgets
Our verdict
We have not reviewed iris-eval/mcp-server yet
No one at Enterprise DNA has checked this listing against its sources. The description comes from public listings gathered automatically. Any fact below shows where it came from and when it was last checked.
From the listing
About iris-eval/mcp-server
iris-eval/mcp-server provides a standardized evaluation framework for agents using the Model Context Protocol (MCP). It scores output quality, detects safety failures, and enforces cost budgets to help developers assess and control agent behavior.
Best for
Developers building and evaluating agents that use the Model Context Protocol
Use cases
- Benchmark agent output quality against defined criteria
- Automatically catch safety violations during agent execution
- Enforce per-call or cumulative cost limits to prevent budget overruns
Notes
6 stars on GitHub. Last updated 2026-05-25. Licensed MIT.
Pros
- Offers a formal evaluation standard for MCP-based agents
- Combines quality, safety, and cost checks in one tool
- Written in TypeScript for type-safe integration
Cons
- Very low GitHub star count (6) suggests limited community adoption
- Tightly coupled to the MCP ecosystem, not useful outside it
- Requires agent infrastructure already built on MCP
Key facts
What we have checked
- Days since the last commit
- 1
- GitHub forks
- 4
- GitHub stars
- 9
- Days to a first reply on issues
- 42.4
- Open issues
- 3
- Releases in the last 90 days
- 21
Each value shows when we last confirmed it and where it came from. Values we have not confirmed are left out, never guessed.
How to use
How to set it up
As the listing publishes it. We have not run this install.
Install
npx @iris-eval/mcp-server --demoTools exposed
IRIS_TRANSPORTIRIS_HOSTIRIS_PORTIRIS_HOMEIRIS_DB_PATHIRIS_LOG_LEVELIRIS_DASHBOARDIRIS_DASHBOARD_PORTIRIS_DASHBOARD_HOSTIRIS_API_KEYIRIS_ALLOW_UNAUTHENTICATEDIRIS_ALLOWED_ORIGINSIRIS_NO_AUTO_LAUNCHIRIS_ANTHROPIC_API_KEYIRIS_OPENAI_API_KEYIRIS_LLM_JUDGE_MAX_COST_USD_PER_EVALIRIS_CITATION_ALLOW_FETCHIRIS_CITATION_DOMAINSIRIS_OTEL_ENDPOINTIRIS_OTEL_SERVICE_NAME
Clients the listing names (not checked here)
Claude Desktop, Claude Code, Cursor, Windsurf, Cline, Continue, VS Code, ChatGPT
Example client config
{
"mcpServers": {
"iris-eval": {
"command": "npx",
"args": [
"-y",
"@iris-eval/mcp-server",
"--dashboard"
]
}
}
}Install
Add iris-eval/mcp-server to your client
Checked : our lab installed version 0.20.0, started it and listed its 12 tools. How the lab checks
Maker? Show the badge on your own README or site
Our lab passed this server, so it can carry "Server verified on Enterprise DNA". The badge is a live link: it disappears if a later check fails, and it is never for sale.
HTML
<a href="https://enterprisedna.co/directories/mcp/iris-eval-mcp-server"><img src="https://enterprisedna.co/directories/badge/mcp-servers/iris-eval-mcp-server.svg" alt="Server verified on Enterprise DNA, checked 4 Nov 2026" width="300" height="44"></a>Markdown
[](https://enterprisedna.co/directories/mcp/iris-eval-mcp-server)Claude Code
Config checkedRun in your terminal (runs on your computer)
claude mcp add --transport stdio iris-eval-mcp-server -- npx -y @iris-eval/mcp-server@0.20.0Claude Desktop
Config checkedAdd to claude_desktop_config.json (runs on your computer)
{
"mcpServers": {
"iris-eval-mcp-server": {
"command": "npx",
"args": [
"-y",
"@iris-eval/mcp-server@0.20.0"
]
}
}
}Cursor
Config checkedAdd to .cursor/mcp.json (runs on your computer)
{
"mcpServers": {
"iris-eval-mcp-server": {
"command": "npx",
"args": [
"-y",
"@iris-eval/mcp-server@0.20.0"
]
}
}
}VS Code
Config checkedAdd to .vscode/mcp.json (runs on your computer)
{
"servers": {
"iris-eval-mcp-server": {
"type": "stdio",
"command": "npx",
"args": [
"-y",
"@iris-eval/mcp-server@0.20.0"
]
}
}
}Codex
Config checkedAdd to ~/.codex/config.toml (runs on your computer)
[mcp_servers.iris-eval-mcp-server]
command = "npx"
args = ["-y", "@iris-eval/mcp-server@0.20.0"]- ChatGPT: ChatGPT connects only to remote servers that need no API key.
Recipes come from the server's entry in the official MCP registry. Check the package name before you run it, and keep keys out of shared files.
Works with
What iris-eval/mcp-server connects to
5 clients, each with how we know and when we checked.
| Works with | Type | How we know | Checked |
|---|---|---|---|
| Claude Code | Client | Install recipe matches the client's documented formatRuns locally, version 0.20.0Config checked | Claude Code docs |
| Claude Desktop | Client | Install recipe matches the client's documented formatRuns locally, version 0.20.0Config checked | Claude Desktop docs |
| Cursor | Client | Install recipe matches the client's documented formatRuns locally, version 0.20.0Config checked | Cursor docs |
| VS Code | Client | Install recipe matches the client's documented formatRuns locally, version 0.20.0Config checked | VS Code docs |
| Codex | Client | Install recipe matches the client's documented formatRuns locally, version 0.20.0Config checked | Codex docs |
Stars and usage
How iris-eval/mcp-server is growing
9 to 9
Health and security
Is iris-eval/mcp-server looked after, and what do its setup files show
Health score
60 out of 100
| Part | Points | What we saw |
|---|---|---|
| Recent activity | 30 of 30 | Last push 1 days ago |
| Releases | 15 of 15 | 21 releases in 90 days |
| Issue replies | 0 of 20 | First reply in 42.4 days (median) |
| Maintainer | 15 of 15 | Owned by an organisation |
| Official source | 0 of 20 | Community server |
Security check static checks
100 out of 100
- Note The lab has not recorded this server's tool list yet, so tool permissions were not checked.
Alternatives
Alternatives to iris-eval/mcp-server
| Record | Type | How we know | Checked |
|---|---|---|---|
| Agent Coliseum MCPAlternative to this | MCP server | Matched from the listing text | |
| ahmedak/defluffAlternative to this | MCP server | Matched from the listing text | |
| ai-evalAlternative to this | MCP server | Matched from the listing text |
Compare
Compare iris-eval/mcp-server side by side
Sources
Where this comes from
- GitHubRegistry
A source shows what was published when we checked it. It does not certify security, performance or a working installation. Read our method.
Changelog
What changed
No changes recorded yet. Releases, price changes and deprecations appear here, with their dates, once we track them for this record.
Open-source & AI alternatives
Swap-in tools that solve the same job. Weigh the trade-offs before you commit.
Agent Coliseum MCP
nanocorp-hq
Free Streamable HTTP MCP arena for 1v1 AI agent skill battles and Gladiator Badges.
ahmedak/defluff
Various
Deterministic slop detector for AI-generated prose. No model, no API key.
ai-eval
lazymac2x
Cloudflare Workers MCP server: ai-eval. REST + MCP (JSON-RPC 2.0). Live: api.lazy-mac.com/ai-eval
Pairs with
Other entries in the index that connect to this one. Click through to see the chain.
Get the free Developer’s Field Guide
A 27-page field guide to the AI coding workflow with Claude. Claude Code, MCP servers, the prompt patterns that work, and what to delegate. Free.
Enter your work email. We send it straight over, plus a few short notes worth knowing. Unsubscribe any time.