For the last few years, the story of AI agents has largely been about giving models access to tools. An agent could search the web, call an API, retrieve information from a database, write code, or trigger an action in another system. The idea was simple: give AI enough tools and enough context, and it could start completing parts of a workflow instead of simply answering questions.
Computer-use agents take that idea further. Rather than connecting an agent to every capability through a dedicated API, the agent can interact with the software itself. It can open a browser, navigate a website, read what is on the screen, fill in forms, move between applications, and continue working through a task.
The computer becomes the interface through which the agent can access an increasingly large part of the digital world.
That sounds like a technical improvement. I think it represents something bigger.
AI is moving from using tools to operating environments.
And once an agent can operate inside the same environment where people work, shop, communicate, and manage information, the difficult question is no longer simply whether it can complete a task. It becomes a question of authority: what should the agent be allowed to see, change, send, purchase, or decide while completing it?
From Chat to Computer Use
The progression is easier to understand when you look at how the relationship between people and AI has changed.
Chatbots gave us systems we could ask for information or assistance. Tool-using models allowed AI to retrieve information or perform specific actions through controlled interfaces. Agentic systems introduced another level of autonomy, allowing the model to decide which tools to use and how to combine them to reach a goal.
Computer-use agents introduce a different kind of flexibility because they are not necessarily limited to the capabilities a developer has explicitly exposed through an API.
OpenAI’s computer-use research was an early example of this direction. The model could interpret a graphical interface and interact with it through mouse and keyboard actions. Meta’s Muse is now pushing the concept into a consumer-facing product. Meta says Muse can work across services such as email, travel and shopping, operate a browser, fill out forms, book travel and make purchases, with sensitive actions requiring user approval.
The important part is not that an AI can click a button. Software has been automating clicks for years. The difference is that the system deciding what to do next can now be a general-purpose AI agent that interprets the environment, chooses an action, observes what happened, and continues from there.
The computer starts to look less like a collection of predefined automation steps and more like an environment in which an agent can work.
The Problem With Giving an Agent a Computer
This flexibility also introduces problems that traditional automation largely avoided.
A conventional automation usually follows a workflow that someone has already defined. If the website changes, the automation may break. If a new condition appears, the workflow may need to be updated. The limitation is obvious, but so is the boundary: the system generally does not decide for itself how to reinterpret the entire task.
An agent can.
Imagine an agent asked to arrange a business trip. It finds a flight, opens the airline website, encounters a different booking interface, sees an upgrade offer, checks the user’s calendar, and discovers that the cheapest option has a restrictive cancellation policy. None of these situations necessarily violates the original instruction. But each requires a judgment about what the user actually intended.
This is where capability and authority become different things.
An agent may be capable of making a purchase without being authorized to make every purchase it encounters. It may be able to send an email without being allowed to send every email it drafts. It may be able to access a file without having permission to modify or share it.
The more capable the agent becomes, the less useful a simple “human in the loop” rule becomes. What matters is where the human enters the workflow, which decisions require approval, and what the agent is allowed to do without it.
Muse Shows the Problem in the Real World
Meta’s Muse makes this discussion more concrete because it moves computer-use agents into everyday consumer software. Meta says users can control which services Muse can access, while sensitive actions such as sending an email or making a purchase require confirmation.
The product quickly ran into the kinds of boundary questions that computer-use agents create.
Amazon blocked Muse from accessing Amazon.com, arguing that Meta had not received permission for the agent to interact with its store. Meta disputed the characterization and said Muse does not have access to users’ passwords or payment information. Shopify also announced an integration with Muse, allowing Muse users to purchase products through Shop Pay across participating Shopify stores.
Those reactions are more interesting than the individual companies involved. They show that the rules governing agents are still being established.
When a human visits a website, the website knows how to interact with that human. When an AI agent arrives, several new questions appear. Who is the agent representing? What is it allowed to do? Does the website recognize it as an agent? Can the service distinguish an authorized action from an automated request that should be blocked?
Computer-use agents are creating a new layer between users and software, and companies are still figuring out how that layer should work.
The Agent Is Also a Security Boundary
The security implications make this even more important.
A computer-use agent can be useful precisely because it has access to things a normal chatbot does not. It may have access to a browser session, applications, files, messages, or services. Those permissions allow it to complete tasks, but they also increase the consequences if something manipulates the agent or gains control over its execution environment.
A recent vulnerability discovered in the macOS version of Muse illustrated the problem. Security researcher Patrick Wardle reported a flaw that could allow malicious software already running on a Mac to interfere with Muse’s dictation process and potentially use the agent’s privileges to perform actions on the user’s behalf. Meta subsequently released a fix.
The vulnerability was not a remote attack that could compromise any Muse user over the internet. But it demonstrated why agents with broad permissions create a new security boundary that needs to be protected.
The agent is no longer simply software producing an answer. It can become a pathway to other software, accounts and actions. As its permissions increase, the agent becomes more valuable to both the user and anyone trying to exploit those permissions.
Security therefore cannot be treated as something added after autonomy. It has to be part of the architecture that determines what autonomy actually means.
Capability Is Not Authority
This distinction has become central to the way I think about agentic systems.
An agent’s capability describes what it can technically do. Its authority describes what it is actually permitted to do. Those two things have always been different for humans, and they need to remain different for AI agents.
Consider an employee who technically has access to a company’s financial system. That access does not automatically give the employee authority to approve every payment. Organizations create permissions, approval thresholds and responsibilities around that capability.
Agents will need the same separation.
An agent might be allowed to read an inbox but not send messages. It might be allowed to research a purchase but not complete it. It might be allowed to prepare a transaction but require approval before committing it. In higher-risk workflows, it may need to stop entirely when it encounters a situation outside the conditions it was given.
This gives us a more useful way to think about autonomy.
Capability asks what the agent can technically do.
Authority asks what it is permitted to do.
Verification asks how we know an action was appropriate.
Accountability asks who owns the outcome.
The important question is not whether agents will become more autonomous. They almost certainly will. The more important question is whether our systems for permission, verification and accountability will become as sophisticated as the capabilities we are giving them.
The Visibility Problem
There is another difference between an AI assistant and an agent that deserves more attention.
When AI gives you an answer, you can usually inspect the answer directly. When an agent completes a workflow, you may only see the outcome.
Imagine an agent spending thirty minutes navigating websites, reading documents, changing settings, sending messages and making decisions. At the end, it gives you a short summary saying the task is complete. The summary may be accurate, but it does not necessarily tell you which decisions mattered or why the agent took them.
This creates what I would call an Agent Visibility Gap.
The solution is not to make users watch every click. It is to give them meaningful visibility into the decisions that carry consequences.
A useful audit trail should answer four questions:
What was the agent trying to accomplish?
Which important decisions did it make?
What actions did those decisions trigger?
What was the resulting outcome?
For a human delegating work to another person, we already understand this distinction. You do not need a transcript of every keystroke. You need to know what decision was made, what changed because of it, and who was responsible.
Agents will need the same kind of operational visibility.
Computer Use Changes the Workflow
This is where computer-use agents could have a larger impact on businesses than the technology demo initially suggests.
Traditional automation usually starts with a workflow that humans have already designed. The company identifies the steps, determines the rules, and then automates some of those steps. The system follows a predefined path.
Agentic systems can work differently. A company can define the goal and constraints while allowing the agent to determine the path through the workflow.
That flexibility can be valuable when environments change frequently or when there are too many exceptions to encode manually. But it also means workflow design can no longer stop at deciding which steps should be automated. Organizations have to decide where the agent can make decisions, when it needs approval, what happens when it is uncertain, and how its actions will be reviewed afterward.
The introduction of agents can therefore turn workflow design into authority design.
That is a much bigger organizational change than simply adding an AI assistant to an existing process.
AI Wide Open Take
I don’t think the important development in computer-use agents is that AI can now click buttons. We have had automation that clicks buttons for years.
The important development is that the system deciding which button to click next is becoming increasingly general-purpose.
That changes the relationship between AI and software. Software used to give people capabilities. Now an AI agent can use software on a person’s behalf, sometimes across applications that were never designed to be operated by an autonomous system.
That is why I think the next phase of agent development will be less about adding more capabilities and more about building better boundaries around those capabilities.
An agent that can do everything but cannot clearly communicate its authority is difficult to delegate to. An agent that can complete less work but operates within clear permissions, stops when it reaches uncertainty, and leaves an understandable trail of important decisions may be much easier to trust.
The real measure of agent autonomy may not be how much work AI can do without us. It may be how much responsibility we can safely give it without losing control of the outcome.
The Agent Authority Test
Before giving an AI agent access to a real workflow, there are a few questions worth answering.
What can it see, and what information does it actually need?
What can it change, send or purchase?
Which decisions can it make without approval?
What happens when it encounters something outside the original instructions?
Can you understand why it made an important decision?
And if the outcome is wrong, can you identify where the decision happened and who was responsible for the authority that allowed it?
These questions become more important as agents move from isolated tools into the environments where actual work happens.
What Comes Next
The next stage of agent development will probably involve less attention on whether an agent can use a computer and more attention on the infrastructure around that capability.
We will need more granular permissions that distinguish between reading, recommending, drafting and executing. We will need stronger forms of agent identity so websites and services can understand who an agent represents and what it is authorized to do. We will need approval systems that respond to the risk of an action rather than simply placing a human somewhere in the workflow. And we will need better observability so organizations can understand not only what an agent did, but which decisions actually changed the outcome.
The technology is moving quickly. The organizational model around it is still catching up.
That may be the more important story.
The Bigger Shift
The first generation of AI taught machines to understand our instructions. The next generation is learning to operate inside the digital environments where those instructions need to be carried out.
We are no longer asking only whether AI can use a computer.
We are deciding what happens when AI can use our computer, our accounts, our information and eventually our authority.
Giving an agent a mouse and keyboard is relatively easy. Giving it permission to act on our behalf is the harder problem.
As agents move from answering questions to completing real work, the defining challenge may not be autonomy itself. It may be building systems that make autonomy understandable, bounded and accountable enough to delegate.
AI Wide Open
Every Thursday, AI Wide Open looks beyond the AI headline to understand what is actually changing underneath it.



"Workflow design becomes authority design" is the most useful line here, and the read / recommend / draft / execute split is where I'd start. In a construction office the line is clear in practice: an agent drafting an RFI or a daily site report is cheap to review, but the moment it sends that document it becomes a contractual record, so the approval belongs at the send, not at the draft. Your Agent Visibility Gap shows up there too — a "task complete" summary is useless if it doesn't say which quantity, date or scope item the agent changed along the way. When approvals scale with the risk of the action, who assigns the risk tier in your view: the agent classifying its own next step, or a fixed list a human owns — given that self-classification is itself one of the judgments we're trying to bound?