Andrew Ng has announced OpenWorker, an open-source desktop agent that produces finished work rather than conversation. OpenWorker asks the user for an outcome, not a prompt: a polished document, a Slack reply containing the actual numbers, an updated calendar, a triaged inbox. It then breaks that outcome into steps, works across local files and connected apps, and checks in before anything consequential.
The architecture is four layers, and all of them run on your machine
The repository contains 119 Python files (~32,400 lines) under coworker/
, 149 TypeScript/TSX files under surfaces/gui/
, and 78 backend test modules.
The stack breaks down as follows:
- Desktop shell — a Tauri 2 native window wrapping a React 18 UI. The bundle identifier is
com.openworker.desktop
, and the shell supervises the Python server itself. - Local agent server — Python 3.10+ on FastAPI and uvicorn, binding to
127.0.0.1:8765
by default. The example config caps a turn at 12 model↔tool iterations. - Capability and connector layer — vetted local tools (files, git, ripgrep-backed search, shell, todo) plus hosted integrations plus MCP.
- Model router — one interface over native, OpenAI-compatible, reseller and local providers.
The engine is built on aisuite, Andrew Ng’s provider-agnostic LLM library.
Bring your own model, from a deliberately small curated list
There is no OpenWorker inference service. The user pastes an API key, or points the app at a local runtime.
The curated model matrix contains exactly 30 entries. Native providers cover OpenAI (GPT-5.6 Sol/Terra/Luna and GPT-5.5), Anthropic (Claude Fable 5, Opus 4.8, Sonnet 4.6, Haiku 4.5) and Google (Gemini 3.1 Pro, 3.6 Flash, 2.5 Pro, 2.5 Flash). OpenAI-compatible vendors add GLM-5.2, DeepSeek V4, Kimi K2.6, MiniMax M2.5, Qwen3 Max, Grok 4.3 and Mistral Large. Open-weight models arrive through Together AI and Fireworks, and fully local models through Ollama, which requires no key at all.
The permission engine is the actual engineering story
Most desktop agent projects treat approvals as a UI afterthought. OpenWorker treats them as a typed layer.
Every tool call is classified into one of four risk classes: read
(no side effects), write_local
(mutates the workspace, path-scoped), exec
(runs commands), and external
(side effects off the machine). Five permission modes then decide what happens: discuss
and plan
are read-only, interactive
is the default and asks before writes, commands and external actions, auto
allows everything while remaining path-scoped, and custom
auto-approves a user-listed set of tools.
Two design decisions stand out.
- First, unattended mode does not raise the autonomy ceiling — it only changes where the human is reached. Prompts that would appear inline are routed to an Inbox, and the session suspends until answered.
- Second, task-scoped standing rules are restricted to
external
risk only. Shell commands ask forever, by design.
The built-in ops persona also instructs the model to treat content from tools, logs, the web, files and incoming messages as untrusted data rather than instructions. That is an explicit prompt-injection posture, written into the shipped persona.
Privacy: local-first
Model calls go directly from the machine to the configured provider. Conversations, connector tokens and model keys stay local, and the secret store is designed so that secrets never enter the model’s context, prompts or traces.
The only cloud component is an optional broker that handles OAuth handshakes for one-click connectors, using Auth0 Authorization Code with PKCE. Connector tokens are handed straight to the machine and are never stored in the cloud. The app is fully functional signed out, using manually pasted credentials.
Key Takeaways
- OpenWorker is Andrew Ng’s MIT-licensed desktop AI coworker that returns finished deliverables, not chat replies.
- The stack is a Tauri 2 + React shell over a local Python FastAPI agent server built on aisuite.
- Model access is bring-your-own-key across 30 curated tool-calling models, plus fully local Ollama.
- A typed risk engine (
read
/write_local
/exec
/external
) gates every action across five permission modes.
Check out the GitHub Repo, the project site, and the announcement. All credit for this research goes to the researchers and developers of this project.
Asif Razzaq is the CEO of Marktechpost Media Inc.. As a visionary entrepreneur and engineer, Asif is committed to harnessing the potential of Artificial Intelligence for social good. His most recent endeavor is the launch of an Artificial Intelligence Media Platform, Marktechpost, which stands out for its in-depth coverage of machine learning and deep learning news that is both technically sound and easily understandable by a wide audience. The platform boasts of over 2 million monthly views, illustrating its popularity among audiences.
Facts Only
Andrew Ng announced OpenWorker, an MIT-licensed open-source desktop agent.
The software is designed to produce finished deliverables rather than conversational replies.
The technical stack consists of a Tauri 2 shell with a React 18 UI and a Python 3.10+ FastAPI server.
The agent runs locally on the user's machine.
The repository includes 119 Python files, 149 TypeScript/TSX files, and 78 backend test modules.
The model router supports 30 curated models from providers including OpenAI, Anthropic, Google, and Ollama.
Tool calls are categorized into four risk classes: read, writelocal, exec, and external.
Five permission modes are available: discuss, plan, interactive, auto, and custom.
Model calls travel directly from the machine to the provider; conversations and keys are stored locally.
An optional cloud broker handles OAuth handshakes using Auth0 Authorization Code with PKCE.
Executive Summary
OpenWorker is an open-source AI desktop agent developed by Andrew Ng's team, designed to shift the LLM paradigm from chat-based interaction to outcome-based productivity. Instead of responding to prompts, the agent takes a desired end-state—such as a triaged inbox or a polished document—and autonomously executes the necessary steps across local files and connected applications.
The system prioritizes local execution and privacy, utilizing a Tauri and FastAPI architecture where the user provides their own API keys or uses local models via Ollama. A central feature of the project is its granular permission engine, which classifies actions by risk level (ranging from read-only to external side effects) to ensure human oversight of consequential actions. While the agent can operate in an "unattended" mode, it does not increase the autonomy ceiling; instead, it routes requests for approval to a separate inbox. The only cloud-based component is an optional OAuth broker for third-party integrations, leaving the core logic and secret storage on the user's hardware.
Full Take
The strongest version of this narrative is that OpenWorker represents a meaningful transition from "AI as a chatbot" to "AI as an operator," solving the critical friction point of trust through a typed risk-management layer rather than a simple "on/off" switch for autonomy.
The paradigm driving this development is the "Local-First AI" movement, which assumes that the primary barrier to agentic adoption is not model capability, but security and privacy. By decoupling the inference provider from the agent's orchestration layer, the system attempts to grant the user sovereign control over their data and the "kill switch" for execution. However, there is a subtle tension in the "curated model list." While promoting openness, the reliance on a specific matrix of 30 models suggests that high-agency work still depends on a narrow set of frontier capabilities.
For human agency, this shifts the user's role from "writer/coder" to "manager/reviewer." The second-order consequence is the potential for "review fatigue," where users in "auto" or "custom" mode may begin rubber-stamping complex actions, effectively eroding the safety the permission engine was designed to provide.
Patterns detected: none
If this were part of a coordinated influence campaign, the playbook would involve using the prestige of a known academic figure (Andrew Ng) to normalize the installation of high-privilege "exec" agents on private machines, potentially masking a broader push for telemetry. The actual content does not match this; it is a technical disclosure of an open-source project with explicit local-first controls.
Bridge Questions:
1. How does the "review fatigue" phenomenon affect the actual efficacy of the four-layer risk engine in long-term use?
2. If the "autonomy ceiling" remains fixed, at what point does the friction of human approval negate the productivity gains of an agentic workflow?
Sentinel — Human
The article reads as a detailed, technically informed journalistic piece about a specific software project, exhibiting the density and structure expected from an engineer or reporter documenting a technical announcement.