Enterprise DNAEnterprise DNA

MCP Servers

iris-eval/mcp-server

The agent eval standard for MCP — score output quality, catch safety failures, enforce cost budgets

Visit site ↗

Our verdict

Not reviewed yet

We have not reviewed iris-eval/mcp-server yet

No one at Enterprise DNA has checked this listing against its sources. The description comes from public listings gathered automatically. Any fact below shows where it came from and when it was last checked.

How we review records

From the listing

About iris-eval/mcp-server

iris-eval/mcp-server provides a standardized evaluation framework for agents using the Model Context Protocol (MCP). It scores output quality, detects safety failures, and enforces cost budgets to help developers assess and control agent behavior.

Best for

Developers building and evaluating agents that use the Model Context Protocol

Use cases

  • Benchmark agent output quality against defined criteria
  • Automatically catch safety violations during agent execution
  • Enforce per-call or cumulative cost limits to prevent budget overruns

Notes

6 stars on GitHub. Last updated 2026-05-25. Licensed MIT.

Pros

  • Offers a formal evaluation standard for MCP-based agents
  • Combines quality, safety, and cost checks in one tool
  • Written in TypeScript for type-safe integration

Cons

  • Very low GitHub star count (6) suggests limited community adoption
  • Tightly coupled to the MCP ecosystem, not useful outside it
  • Requires agent infrastructure already built on MCP

Key facts

What we have checked

Not checked yet. Dated facts such as prices, releases and repository activity appear here once we have confirmed them from a source.

How to use

How to set it up

As the listing publishes it. We have not run this install.

Install

npx @iris-eval/mcp-server --demo

Tools exposed

  • IRIS_TRANSPORT
  • IRIS_HOST
  • IRIS_PORT
  • IRIS_HOME
  • IRIS_DB_PATH
  • IRIS_LOG_LEVEL
  • IRIS_DASHBOARD
  • IRIS_DASHBOARD_PORT
  • IRIS_DASHBOARD_HOST
  • IRIS_API_KEY
  • IRIS_ALLOW_UNAUTHENTICATED
  • IRIS_ALLOWED_ORIGINS
  • IRIS_NO_AUTO_LAUNCH
  • IRIS_ANTHROPIC_API_KEY
  • IRIS_OPENAI_API_KEY
  • IRIS_LLM_JUDGE_MAX_COST_USD_PER_EVAL
  • IRIS_CITATION_ALLOW_FETCH
  • IRIS_CITATION_DOMAINS
  • IRIS_OTEL_ENDPOINT
  • IRIS_OTEL_SERVICE_NAME

Clients the listing names (not checked here)

Claude Desktop, Claude Code, Cursor, Windsurf, Cline, Continue, VS Code, ChatGPT

Example client config

{
  "mcpServers": {
    "iris-eval": {
      "command": "npx",
      "args": [
        "@iris-eval/mcp-server"
      ]
    }
  }
}

Install

Add iris-eval/mcp-server to your client

Not verified yet

Our lab has not run this server yet. "Config checked" on a client means the recipe matches that client's documented config format; it does not mean we ran it.

Claude Code

Config checked

Run in your terminal (runs on your computer)

claude mcp add --transport stdio iris-eval-mcp-server -- npx -y @iris-eval/mcp-server@0.20.0

Version 0.20.0 · From the Official MCP registry, read · Format checked against Claude Code's config docs

Claude Desktop

Config checked

Add to claude_desktop_config.json (runs on your computer)

{
  "mcpServers": {
    "iris-eval-mcp-server": {
      "command": "npx",
      "args": [
        "-y",
        "@iris-eval/mcp-server@0.20.0"
      ]
    }
  }
}

Version 0.20.0 · From the Official MCP registry, read · Format checked against Claude Desktop's config docs

Cursor

Config checked

Add to .cursor/mcp.json (runs on your computer)

{
  "mcpServers": {
    "iris-eval-mcp-server": {
      "command": "npx",
      "args": [
        "-y",
        "@iris-eval/mcp-server@0.20.0"
      ]
    }
  }
}

Version 0.20.0 · From the Official MCP registry, read · Format checked against Cursor's config docs

VS Code

Config checked

Add to .vscode/mcp.json (runs on your computer)

{
  "servers": {
    "iris-eval-mcp-server": {
      "type": "stdio",
      "command": "npx",
      "args": [
        "-y",
        "@iris-eval/mcp-server@0.20.0"
      ]
    }
  }
}

Version 0.20.0 · From the Official MCP registry, read · Format checked against VS Code's config docs

Codex

Config checked

Add to ~/.codex/config.toml (runs on your computer)

[mcp_servers.iris-eval-mcp-server]
command = "npx"
args = ["-y", "@iris-eval/mcp-server@0.20.0"]

Version 0.20.0 · From the Official MCP registry, read · Format checked against Codex's config docs

  • ChatGPT: ChatGPT connects only to remote servers that need no API key.

Recipes come from the server's entry in the official MCP registry. Check the package name before you run it, and keep keys out of shared files.

Works with

What iris-eval/mcp-server connects to

5 clients, each with how we know and when we checked.

Works withTypeHow we knowChecked
Claude CodeClientInstall recipe matches the client's documented formatRuns locally, version 0.20.0Config checkedClaude Code docs
Claude DesktopClientInstall recipe matches the client's documented formatRuns locally, version 0.20.0Config checkedClaude Desktop docs
CursorClientInstall recipe matches the client's documented formatRuns locally, version 0.20.0Config checkedCursor docs
VS CodeClientInstall recipe matches the client's documented formatRuns locally, version 0.20.0Config checkedVS Code docs
CodexClientInstall recipe matches the client's documented formatRuns locally, version 0.20.0Config checkedCodex docs

Sources

Where this comes from

No sources recorded yet. Each source we check is listed here with the date we checked it.

A source shows what was published when we checked it. It does not certify security, performance or a working installation. Read our method.

Changelog

What changed

No changes recorded yet. Releases, price changes and deprecations appear here, with their dates, once we track them for this record.

Pairs with

Other entries in the index that connect to this one. Click through to see the chain.

Free 27-page guide

Get the free Developer’s Field Guide

A 27-page field guide to the AI coding workflow with Claude. Claude Code, MCP servers, the prompt patterns that work, and what to delegate. Free.

Enter your work email. We send it straight over, plus a few short notes worth knowing. Unsubscribe any time.

No spam. Unsubscribe any time.

← Back to MCP ServersSubmit your own entry →