Least-Privilege Access for Tools

by kirupa (https://www.kirupa.com/me/index.htm) | filed under AI Fundamentals (https://www.kirupa.com/ai_fundamentals/index.htm)

Some cars come with two kinds of keys. There's the regular key that does everything, and there's a valet key. The valet key opens the door and starts the engine, but it won't open the trunk or the glovebox. When you hand your car to a stranger at a restaurant, you hand over the valet key. You aren't assuming the valet is a thief. You're just making sure that if something goes wrong, the damage stops at they drove my car around the block:

[Image: A regular key opens the car, the trunk, the glovebox, and the house. The valet key only starts the car; the trunk, glovebox, and house stay locked.] (https://www.kirupa.com/ai_fundamentals/images/least_privilege_tools_keys.png)

That's the principle of least privilege. Jerome Saltzer and Michael Schroeder wrote it down in 1975:

Every program and every user of the system should operate using the least set of privileges necessary to complete the job.

Jerome Saltzer and Michael Schroeder, The Protection of Information in Computer Systems (https://web.mit.edu/Saltzer/www/publications/protection/Basic.html)

When we give an AI agent tools, we're handing it keys, and this 50-year-old principle turns out to fit agents remarkably well.

Onwards!

Why Agents Need Valet Keys

When we write ordinary code, we decide exactly which functions get called with which arguments. With an agent, the model decides. It reads some text, picks a tool, and fills in the arguments. Most of the time it picks well. But the text it reads comes from users, from documents, and sometimes from complete strangers, and the model can be confused, mistaken, or deliberately tricked (we'll dig into that last one in Prompt Injection Defense (https://www.kirupa.com/ai_fundamentals/prompt_injection_defense.htm)).

So the safe assumption is a slightly paranoid one. Sooner or later, the model will call any tool we give it, with any arguments it's able to produce. That turns the important question for every tool into this one: what's the worst thing that happens if the model calls this tool with the worst possible arguments? The answer is the tool's blast radius, and least privilege is the practice of keeping it small.

The Version That Has Every Key

Imagine we're building the support agent for Copper Kettle, an online kitchenware store. A customer named Jun opens the chat and asks why his cast iron pan hasn't arrived, and whether he can get a refund on the lid that came cracked.

Here is Jun's question in the chat on Copper Kettle's website:

[Image: Copper Kettle's support agent sits in a chat panel in the corner of the Copper Kettle kitchenware store's website. The panel header shows a robot avatar with a frying pan badge, the name Copper Kettle, and the tagline Support agent. Jun asks, My cast iron pan hasn't arrived, and the lid came cracked. Can I get a refund on the lid? The agent replies, Sorry about that! Let me look up your order. A handwritten note says The support agent!] (https://www.kirupa.com/ai_fundamentals/images/least_privilege_tools_meet_copper_kettle.png)

The quickest way to build this agent is to give it two very flexible tools. One is run_sql(query), connected to our database with the same admin credentials our app uses. The other is send_email(to, body). With these, the agent can answer almost any question. Here is what those two tools can reach:

[Image: The first version of the agent has an admin-level run_sql tool that can reach every table, including payments and employees, and a send_email tool that can write to any address. Its blast radius is everything.] (https://www.kirupa.com/ai_fundamentals/images/least_privilege_tools_every_key.png)

The blast radius here is the whole company. One bad query could read every customer's address, refund every order, or DROP TABLE refunds. One bad email could send that data anywhere. We'd be relying entirely on the model choosing to be well-behaved, every single time.

Shrinking the Blast Radius

What we are going to do is take that agent and give it a valet key instead, one move at a time.

Move 1: Narrow, Purpose-Built Tools

The first move is to replace the do-anything tools with tools that each do one job. Instead of run_sql, the agent gets get_order(order_id), list_my_orders(), and refund_item(order_id, item_id, amount). Each one has typed parameters and runs a query we wrote and reviewed. The agent can still help Jun, but there is simply no tool that reads the employees table or deletes anything.

send_email gets the same treatment. Our agent only ever needs to email the customer it's talking to, so the tool becomes email_customer(body), and the address isn't a parameter at all.

Move 2: Act on Behalf of the User

Narrow tools still have a problem. What stops the agent from calling get_order("C-9999"), which belongs to somebody else?

The fix is to have every tool run as Jun instead of as the store. When Jun logs in, our app knows who he is. The tool layer (our ordinary code that sits between the model and the database) takes that identity from Jun's session, and it never takes it from the model's arguments:

[Image: The tool layer takes Jun's identity from his logged-in session rather than from the model. A call for his own order C-2231 is returned, and a call for someone else's order C-9999 is denied.] (https://www.kirupa.com/ai_fundamentals/images/least_privilege_tools_on_behalf.png)

This defends against a very old problem with a wonderful name: the confused deputy. Norm Hardy described it in a 1988 paper, The Confused Deputy. A program that holds powerful permissions on behalf of many people (the deputy) can be tricked by one of them into using those permissions on something that person isn't allowed to touch. An agent with admin database credentials is a confused deputy waiting to happen. An agent whose tools only carry Jun's permissions can't be talked into reading someone else's order, because the database itself won't let it.

Move 3: Put the Limits in Code

The next move is to set limits on what each tool will do, even for Jun's own orders. It's tempting to write these rules into the system prompt (https://www.kirupa.com/ai_fundamentals/system_prompts_vs_user_prompts.htm), something like Only issue refunds up to the item's price, and never more than $50 without a manager. That sentence is advice. The model will follow it most of the time, and most of the time isn't a security property. The same rule written in code is a rule:

async function refundItem({ orderId, itemId, amount }, session) {
  const order = await db.orders.get(orderId);
  const item = order?.items.find((i) => i.id === itemId);

  if (!order || order.customerId !== session.customerId) {
    throw new Denied("Not your order");
  }
  if (!item || amount > item.price) {
    throw new Denied("Refund is more than the item cost");
  }
  if (await db.refunds.existsFor(orderId, itemId)) {
    throw new Denied("Already refunded");
  }
  if (amount > 50) {
    return requestHumanApproval({ orderId, itemId, amount, session });
  }

  return payments.refund(orderId, itemId, amount);
}

Notice that session comes from our app, not from the model, and every check runs no matter what the model was told or what it believes. Jun's cracked lid costs $18, so his refund goes straight through. A request for $400 on the same lid is rejected, and a legitimate $80 refund on a dutch oven pauses for a person to approve it (https://www.kirupa.com/ai_fundamentals/human_approval_for_risky_actions.htm).

Here are those three refunds side by side:

[Image: Three refund requests through the refund tool's checks. Jun's cracked lid costs 18 dollars, so the refund goes straight through. A request for 400 dollars on the same lid is rejected. A legitimate 80-dollar refund on a dutch oven pauses for a person to approve it.] (https://www.kirupa.com/ai_fundamentals/images/least_privilege_tools_limits.png)

Move 4: Fewer Tools, for Less Time

The last move is about timing. A conversation that's only asking where's my order? doesn't need the refund tool loaded at all, so we only give the agent the tools that match the task in front of it. The credentials behind the tools should be short-lived and tied to one session, so a leaked access token expires in minutes instead of working forever. And read-only tools can point at a read-only copy of the database, so even a bug in our own query code can't write anything.

Here is where all four moves leave us:

[Image: After the four changes, the agent's narrow tools act as Jun, can only reach Jun's orders and refunds, need human approval above 50 dollars, and can only email Jun. Its blast radius is Jun's stuff.] (https://www.kirupa.com/ai_fundamentals/images/least_privilege_tools_valet_key.png)

What Least Privilege Doesn't Do

Least privilege won't make the agent smarter, and it won't stop it from being tricked. Within Jun's valet key, the agent could still be talked into refunding his lid when it isn't actually cracked. What least privilege does is put a ceiling on how bad any single mistake or trick can get, so the worst case stops at an $18 refund that needs a second look instead of every customer's data emailed to a stranger. And because every tool call (https://www.kirupa.com/ai_fundamentals/tool_calling.htm) now carries a clear identity and a clear decision, it's also exactly what we'll want to record in Audit Logs (https://www.kirupa.com/ai_fundamentals/audit_logs.htm).

The same thinking applies when we plug in tools somebody else wrote, like third-party MCP servers. Before connecting one, it's worth checking what the token we hand it can actually do. A read my calendar integration that asks for full access to the whole email account is a regular key pretending to be a valet key.

Conclusion

We started with an agent that held every key, an admin run_sql and an email tool that could write to anyone, and a blast radius the size of the company. We shrank it in four moves: narrow tools that each do one job, tools that act as the user instead of as the store, limits enforced in code instead of requested in the prompt, and only the tools a task needs, with credentials that expire quickly.

None of this requires trusting the model less in our day-to-day work. It just means that when the model is wrong, or when someone is actively trying to make it wrong, the damage stops at the valet key. In the next article, Prompt Injection Defense (https://www.kirupa.com/ai_fundamentals/prompt_injection_defense.htm), we'll meet the people who are actively trying.