All results

Every run in results/history/, newest first. The files are written by the harness and never edited by hand.

Pilot run of 2026-11-16, 4 of 4 gateways measured; tool-call results identical to calling the reference server directly: Speakeasy MCP Gateway (hosted) 100%, median added latency 374.6 ms; MCPJungle (local) 100%, median added latency 217.9 ms; Docker MCP Gateway (local) 100%, median added latency -18.1 ms; IBM MCP Context Forge (local) 100%, median added latency 281.2 ms.

RankGatewayWhere it ranStatusTool-call correctness
higher is better
Median paired added latency
lower is better
Timed call error rate
lower is better
Notes
1Speakeasy MCP Gateway (subject)Hosted serviceMeasured100%374.6 ms0%
1MCPJungleLocal, on the bench machineMeasured100%217.9 ms0%
1Docker MCP GatewayLocal, on the bench machineMeasured100%-18.1 ms0%
1IBM MCP Context ForgeLocal, on the bench machineMeasured100%281.2 ms0%

Raw file: results/history/2026-11-16.json · Scale: small: the METHODOLOGY.md pilot (20 warmup pairs, then 3 rounds of 100 measured echo pairs)