mcp-autotest
by strowk
A simple tool that allows you to test your MCP servers using the MCP protocol by defining YAML files with requests and responses. It supports stdio and HTTP transports.
Last updated: N/A
What is mcp-autotest?
mcp-autotest is a language-agnostic tool for testing MCP (Model Context Protocol) servers. It uses YAML files to define test cases, specifying requests and expected responses.
How to use mcp-autotest?
- Install mcp-autotest using npm, GitHub Releases, or build from source.
- Create YAML files with
_test.yamlsuffix, defining test cases with 'in' (request) and 'out' (expected response) keys. - Run the tool using the command:
mcp-autotest run path/to/tests/folder [--] command-to-run-server [server-args]. - Use the
--urlflag for HTTP transport.
Key features of mcp-autotest
Language-agnostic MCP server testing
YAML-based test case definition
Supports stdio and HTTP transports
Dynamic matching with regular expressions
Embedded regex support for partial string matching
Use cases of mcp-autotest
Automated testing of MCP servers
Regression testing of MCP implementations
Validating MCP server behavior against a specification
Integration testing of MCP clients and servers
FAQ from mcp-autotest
How do I define test cases?
How do I define test cases?
Create YAML files with the _test.yaml suffix. Each file can contain multiple test cases separated by ---. Each test case should define an in key for the request and an out key for the expected response.
How do I use HTTP transport?
How do I use HTTP transport?
Use the --url flag followed by the URL of your MCP server, e.g., mcp-autotest run --url http://localhost:8080/mcp testdata go run main.go.
How do I handle dynamic values in responses?
How do I handle dynamic values in responses?
Use the !!re tag in your expected output to match values using regular expressions.
What is embedded regex?
What is embedded regex?
Embedded regex (using !!ere) allows you to treat parts of a string as regular expressions within a larger string. Everything inside slashes / would be treated as regular expression.
How do I escape forward slashes in embedded regex?
How do I escape forward slashes in embedded regex?
Use \/ to escape forward slashes in places of your string where you want to use them, but not designate regular expression.