# MCP Gateway Bench > Do MCP tools behave identically through each gateway as they do directly, and what latency does each gateway add? A public pilot benchmark comparing Speakeasy MCP Gateway with MCPJungle, Docker MCP Gateway, IBM MCP Context Forge. Every gateway routes to the same reference MCP server (official @modelcontextprotocol/server-everything 2.3.0 tools), and each call through a gateway is compared with the same call made directly. - Leaderboard: https://mcpgatewaybench.com/ - Raw results: https://mcpgatewaybench.com/results/latest.json - All runs: https://mcpgatewaybench.com/results - Methodology: https://mcpgatewaybench.com/methodology - Replicate: https://mcpgatewaybench.com/replicate ## Metrics - Tool-call correctness (`conformance_pass_rate`, %): higher is better; primary metric, used for ranking - Median paired added latency (`added_latency_p50`, ms): lower is better - Timed call error rate (`error_rate`, %): lower is better ## Latest results Run: 2026-11-16T03:46:26Z Summary: Pilot run of 2026-11-16, 4 of 4 gateways measured; tool-call results identical to calling the reference server directly: Speakeasy MCP Gateway (hosted) 100%, median added latency 374.6 ms; MCPJungle (local) 100%, median added latency 217.9 ms; Docker MCP Gateway (local) 100%, median added latency -18.1 ms; IBM MCP Context Forge (local) 100%, median added latency 281.2 ms. Ranked on Tool-call correctness (higher is better): 1. Speakeasy MCP Gateway (hosted, ok): Tool-call correctness: 100%; Median paired added latency: 374.6 ms; Timed call error rate: 0% 1. MCPJungle (local, ok): Tool-call correctness: 100%; Median paired added latency: 217.9 ms; Timed call error rate: 0% 1. Docker MCP Gateway (local, ok): Tool-call correctness: 100%; Median paired added latency: -18.1 ms; Timed call error rate: 0% 1. IBM MCP Context Forge (local, ok): Tool-call correctness: 100%; Median paired added latency: 281.2 ms; Timed call error rate: 0% Caveats: - This is a pilot: eight reference cases and paired echo timings, not broad protocol conformance or enterprise readiness. - Added latency is end-to-end: it includes each route's network distance, proxy behaviour and serverless cold starts, not pure gateway processing time. - The harness repository is private, so its code cannot be inspected publicly until its owner makes it public. - Runs are not yet scheduled unattended: the Speakeasy OAuth token must be renewed by an operator before each run. ## Replicate # Replicate MCP Gateway Bench For a person or an AI agent: how to reproduce the published results and check them. ## 1. What you need Node.js 24, npm and git, and network access to each contender's service. A contender that needs credentials reads them from the environment variables named in `.env.example`; without them, the contender is recorded as `skipped`, never scored. - Speakeasy MCP Gateway (`subject`): `SPEAKEASY_GATEWAY_URL`, `SPEAKEASY_OAUTH_JSON` Use your own account on each service. `BENCH_SCALE` sets the dataset's size and `BENCH_REPETITIONS` how many times each query is timed; the defaults are what the published results used. ## 2. Get the code ```sh git clone cd mcp-gateway-bench npm install ``` ## 3. Run it ```sh npm run bench ``` It runs every contender in `bench.json` on every scenario and writes `results/latest.json` and `results/history/.json` (the shape is in `results/README.md`). ## 4. Compare with the published results The published file is `results/latest.json` on `main`. The same results, readable: https://mcpgatewaybench.com Compare each contender's metrics. Run-to-run noise is expected; METHODOLOGY.md says what counts as a real difference. The primary metric is `conformance_pass_rate` (higher is better). ## 5. Report a difference Open an issue on the repo with your `results/latest.json` and the `environment` it recorded.