Can You Trust AI to Execute Tools Safely

The Model Does Not Execute Anything

A critical distinction that is often missed in discussions about AI tool safety: the model generates structured requests. Your code executes them. The model cannot access your database, send emails, or delete files on its own. It can only produce a JSON object that says "please call this function with these arguments." Whether that request actually runs is entirely under your control. This means tool safety is an application engineering problem, not an AI problem.

This architecture gives you complete control over what happens. You can validate every argument, check every permission, require approval for every dangerous operation, and block any call that does not meet your safety criteria. The model's intent is visible in the structured call before anything executes, which is actually safer than many traditional application architectures where user actions trigger backend operations more directly.

A Tiered Safety Model

Production systems typically implement tool safety in tiers based on the risk level of each operation. Read-only tools (data lookups, searches, status checks) execute automatically because they cannot change state or cause harm. The worst outcome of a bad read call is wasted tokens and a slightly confusing response. Write tools that create, modify, or update non-critical data (adding notes, updating preferences, creating drafts) may execute automatically for low-risk operations or require confirmation for higher-risk ones. Destructive tools (deleting data, canceling services, revoking access) always require explicit user confirmation. Financial tools (processing payments, issuing refunds, modifying subscriptions) always require confirmation and often require additional authorization.

This tiered approach balances safety with usability. Requiring confirmation for every tool call makes the agent safe but painfully slow. Requiring confirmation for nothing makes the agent fast but potentially dangerous. The right balance matches the confirmation requirement to the actual risk of each operation.

Prompt Injection and Tool Safety

The primary attack vector for tool-using agents is prompt injection: a malicious user (or malicious content in retrieved data) crafts input that tricks the model into making unintended tool calls. An attacker might include hidden instructions in a document that tell the model to "delete all records" or "send this information to external-server.com." Validation and authorization layers are the defense against these attacks.

A well-designed validation layer catches injected tool calls because they typically violate business rules (the user does not have permission to delete all records), fail schema validation (the injected call uses parameters that do not match the schema), or trigger confirmation gates (destructive operations require user approval that the attacker cannot provide through the injected content). Defense in depth means that prompt injection must bypass multiple layers to cause harm, which is difficult when each layer enforces independent constraints.

Real-World Safety Patterns

Consider a financial services assistant that can check balances, transfer funds, and close accounts. The safety architecture might look like this: balance checks execute automatically (read-only, no risk). Fund transfers under $100 execute with a brief confirmation ("Transfer $45 to your savings account?"). Fund transfers over $100 require both confirmation and two-factor authentication. Account closures require confirmation, two-factor authentication, and a mandatory 24-hour cooling period during which the user can cancel. Each tier adds friction proportional to the risk, and the boundaries are enforced in the execution layer, not by hoping the model behaves correctly.

Another common pattern is scoped permissions. An agent that manages multiple customer accounts should only access the account of the currently authenticated user. Rather than trusting the model to restrict itself, the execution layer injects the authenticated user's ID into every tool call, overriding whatever the model provided. If the model generates get_orders(customer_id="X123") but the authenticated user is "X456", the execution layer either replaces the customer_id or rejects the call. This pattern means that even a successful prompt injection cannot access another user's data because the permission is enforced structurally, not through model reasoning.

Rate limiting provides another safety layer. An agent that can send emails should be limited to a reasonable number per conversation (perhaps 3 to 5) to prevent a runaway loop or an injection attack from mass-mailing through the agent. Rate limits on destructive operations (no more than one delete per conversation without explicit re-authorization) prevent cascading damage even if a single malicious call gets through.

Memory Improves Safety

Persistent memory adds another safety dimension. When the agent remembers the established patterns for a user (what tools they normally use, what operations they normally request, what entities they typically interact with), anomalous tool calls stand out. If a user who has only ever queried their own orders suddenly tries to access another customer's data, the deviation from established patterns can trigger additional verification. Adaptive Recall's cognitive scoring naturally highlights these deviations because memories about the user's normal behavior receive high activation, creating an implicit baseline that anomalous requests contrast against.

Memory also supports audit and accountability. When every tool call outcome is stored as a memory with context (who requested it, why, what the result was), the memory system becomes an audit trail that can be searched semantically. "Show me all financial operations for this user in the past month" is a natural language query that returns relevant tool outcome memories, giving compliance teams and administrators visibility into what the agent did and why. This is more useful than raw logs because the memories include context and summaries, not just raw API calls.

The bottom line is that AI tool execution is as trustworthy as you engineer it to be. The model generates requests. Your code decides what to execute. Build validation, authorization, confirmation, sandboxing, rate limiting, and audit into your execution layer, and tool use becomes a controlled, auditable, and safe capability rather than a liability.