Andrew Ng Introduces OpenWorker: An Open-Source AI Agent for Desktop That Delivers Results, Not Chat

Andrew Ng introduced OpenWorker, an open-source desktop AI agent that delivers completed work rather than engaging in conversation. Instead of a prompt, the user specifies the expected outcome: a finished document, a Slack response with precise numbers, an updated calendar, or a sorted email inbox. The agent then breaks down the task into steps, works with local files and connected applications, and requests confirmation before any significant action. The architecture comprises four layers: a desktop shell built with Tauri 2 and React 18, a local agent server in Python (FastAPI, uvicorn, port 127.0.0.1:8765), a capabilities and connectors layer (local tools, hosted integrations, MCP), and a model router built on the aisuite library. OpenWorker does not provide its own inference service — the user inserts an API key or runs a local environment. The list of supported models includes 30 options: native providers (OpenAI, Anthropic, Google), OpenAI-compatible ones (GLM, DeepSeek, Kimi, MiniMax, Qwen, Grok, Mistral), open-weight models through Together AI and Fireworks, and fully local ones via Ollama (no key required). A key feature is the permissions engine, which classifies each action into four risk classes (read, write, command execution, external effects) and supports five permission modes. In autonomous mode, notifications are sent to the inbox, and the session is paused until a response. The security policy mandates that the model treat content from tools, logs, web pages, and incoming messages as untrusted data, not instructions. Privacy is ensured by local processing: model calls go directly, connector tokens and keys remain on the machine, and secrets are not included in the model context. The only cloud component is an optional broker for OAuth handshakes (Auth0 with PKCE); tokens are issued directly to the machine and are not stored in the cloud. The application is fully functional without login. The source code is available on GitHub (MIT license).