An agent that occasionally misunderstands a request is a problem. An agent that can misunderstand a request while holding the keys to every account in the building is a management decision with a very cheerful interface.

We spend a remarkable amount of time arguing about the model's personality. Is it obedient? Is it cautious? Does it sound confident? Meanwhile, the permission map has been drawn with a bucket of paint.

OWASP's Excessive Agency guidance identifies excessive functionality, permissions and autonomy as sources of damaging agent actions. That is a useful framing because it puts the surrounding application back into the discussion. What a system is allowed to do matters alongside what its model says.

The latest debate about uncontrolled AI development is often presented as a contest between moving quickly and being a tedious adult. I think a useful permission boundary is one of the things that lets a team move quickly without making every experiment an emergency.

A deliberately ordinary example

Imagine a fictional assistant whose job is to prepare a summary of a team's public documentation. It needs to read a limited set of documents and return a draft. Someone connects a general account because it is already available. That account also manages the documents, sends messages and accesses unrelated material.

The demo still looks like a summarizer. The account does not.

I would begin by listing the task in plain verbs. Read these documents. Produce this draft. Show this source list. Anything beyond that deserves its own explanation. The point is not to forbid useful additions forever. The point is to make the addition a decision somebody can see.

Then I would ask who receives the output and which person owns a consequential next step. A draft shown to the requester has a different consequence from a message automatically sent to somebody else. A proposed update has a different consequence from an edit that has already taken effect. Collapsing those into one friendly button makes the interface simpler by making the decision harder to inspect.

Finally, I would make the failure conversation concrete. Suppose the assistant misunderstood which document the person meant. What could change? How would the team notice? Could the person correct the misunderstanding before it affected someone else? Does the system have a record that explains the action without revealing information the viewer should not receive?

These questions are less glamorous than debating whether the model seems trustworthy. They are also easier to answer with something other than an adjective.

The owner is not the prompt

A carefully written instruction can be useful. It should not become the place where a team hides a decision it could have enforced more clearly in the application. If the tool only needs a narrow capability, granting a broad one and requesting restraint turns restraint into an ongoing operational dependency.

Sometimes broader access really is necessary. Say why. Name the owner. Describe what the person is authorizing and what the system can do after authorization. A difficult case deserves a legible explanation, not a reassuring phrase about having a human somewhere in the vicinity.

I do not expect a permission map to make an imperfect model perfect. I expect it to keep a useful experiment from acquiring unexplained powers. That is a much more modest promise, which is one reason I trust it more.

The robot's pleasant expression is irrelevant to the size of its key ring. Look at the keys.

Anime avatar of Rowan Blake
About the author

Security and privacy writer

I'm based in Toronto, Canada. I write about AI agents, security incidents and who actually owns the blast radius.

A thought worth keeping?