All results
Every run in results/history/, newest first. The files are written by the harness and never edited by hand.
Pilot run of 2026-11-16, 4 of 4 gateways measured; tool-call results identical to calling the reference server directly: Speakeasy MCP Gateway (hosted) 100%, median added latency 374.6 ms; MCPJungle (local) 100%, median added latency 217.9 ms; Docker MCP Gateway (local) 100%, median added latency -18.1 ms; IBM MCP Context Forge (local) 100%, median added latency 281.2 ms.
| Rank | Gateway | Where it ran | Status | Tool-call correctness higher is better | Median paired added latency lower is better | Timed call error rate lower is better | Notes |
|---|---|---|---|---|---|---|---|
| 1 | Speakeasy MCP Gateway (subject) | Hosted service | Measured | 100% | 374.6 ms | 0% | |
| 1 | MCPJungle | Local, on the bench machine | Measured | 100% | 217.9 ms | 0% | |
| 1 | Docker MCP Gateway | Local, on the bench machine | Measured | 100% | -18.1 ms | 0% | |
| 1 | IBM MCP Context Forge | Local, on the bench machine | Measured | 100% | 281.2 ms | 0% |
Raw file: results/history/2026-11-16.json · Scale: small: the METHODOLOGY.md pilot (20 warmup pairs, then 3 rounds of 100 measured echo pairs)