pywinassistant

PyWinAssistant is the first open-source artificial intelligence framework that fully controls graphical user interfaces (GUIs) through natural language. It employs visual thinking and chain-of-thought reasoning to trigger spatial reasoning and perception, simulating, planning, and synthesizing HID interactions. The framework does not rely on computer vision, OCR, or pixel-level imaging but instead operates via native Windows UI automation APIs. PyWinAssistant can generate automation scripts, automatically detect and adapt to UI changes, and support cross-context automation tasks.