PyMCPAutoGUI
by kitfactory
PyMCPAutoGUI bridges AI agents with your computer's GUI, enabling screen perception, mouse/keyboard control, and window interaction. It allows AI agents to automate tasks, test GUIs, and act as powerful AI assistants.
Last updated: N/A
What is PyMCPAutoGUI?
PyMCPAutoGUI is a server that allows AI agents to interact with a computer's graphical user interface (GUI) through the Model Context Protocol (MCP). It leverages PyAutoGUI and PyGetWindow to provide comprehensive GUI automation capabilities.
How to use PyMCPAutoGUI?
- Install PyMCPAutoGUI using pip (preferably in a virtual environment). 2. Run the server using
python -m pymcpautogui.server. 3. Configure your MCP client (e.g., Cursor Editor) to connect to the server. 4. Use the exposed functions via your MCP client to automate GUI tasks.
Key features of PyMCPAutoGUI
Empowers AI agents to interact with desktop applications
Seamless integration with MCP-compatible clients like Cursor
Comprehensive GUI control through PyAutoGUI and PyGetWindow
Screen perception with screenshot and image location tools
Window management capabilities (position, size, state)
Use cases of PyMCPAutoGUI
Automating repetitive GUI tasks
Testing GUI applications
Building powerful AI assistants
Controlling desktop applications with AI agents
FAQ from PyMCPAutoGUI
What operating systems are supported?
What operating systems are supported?
Windows, macOS, and Linux (with appropriate dependencies for pyautogui)
What Python versions are supported?
What Python versions are supported?
Python 3.11+
What MCP clients are compatible?
What MCP clients are compatible?
Cursor Editor and any client supporting the Model Context Protocol (MCP)
How do I install PyMCPAutoGUI?
How do I install PyMCPAutoGUI?
Use pip install pymcpautogui (preferably in a virtual environment).
How do I find a list of available tools?
How do I find a list of available tools?
Check the pymcpautogui/server.py file or use @PyMCPAutoGUI list_tools in your MCP client.