Security in artificial intelligence systems has never been more urgent than it is right now.
In just a few weeks, still during the northern hemisphere summer, OpenAI, Anthropic, and the UK AI Security Institute all reported cases where frontier agents operated beyond the boundaries that had been set for them. Among the observed behaviors were exploiting unexpected pathways to escape lab environments onto the open internet, unauthorized access to other companies’ systems, and executing actions involving people and infrastructure without any authorization.
And the most curious part of all this?
These cases involved long-horizon agents running with reduced model safeguards. But they all point to the same design challenge: the very capabilities that allow agents to solve problems creatively and pursue complex goals can also help them find paths the original instructions never anticipated.
As AI agents become more capable and operate over longer periods, building security and trust into the applications they power stops being a technical detail and becomes a central design concern.
Drawing from the work of the NVIDIA team alongside OpenShell, agent developers, open source projects, and partners across the ecosystem, NVIDIA’s security and safety teams offer a perspective on this emerging agent stack. In this article, we’ll map the key architecture layers of that stack, from the model to the secure runtime, explaining the role of each one and, most importantly, where the security boundary actually needs to be in order to work. 🔐
Why architecture matters more than the model
For a long time, the conversation about security in AI revolved around models: how they’re trained, what data they use, how they respond to malicious prompts. And it makes sense that this discussion exists, because model behavior is indeed an important piece of the puzzle. But when we talk about AI agents that operate autonomously, make chained decisions, and interact with external systems, the model is only part of the story.
The other part, often overlooked, is the structure holding everything together: the architecture layers that determine what the agent can access, what it can execute, and what barriers it encounters along the way.
Receive the best innovation content in your email.
All the news, tips, trends, and resources you're looking for, delivered to your inbox.
By subscribing to the newsletter, you agree to receive communications from Método Viral. We are committed to always protecting and respecting your privacy.
Think of it this way: a highly aligned model, trained with all the care in the world, can still cause damage if the infrastructure around it lacks well-defined controls. It’s like putting a responsible driver in a car with no brakes. The problem isn’t the driver. The problem is the car. And that’s exactly what the recent cases revealed: well-intentioned agents, running on robust models, managed to exceed boundaries simply because those boundaries were in the wrong place in the architecture or didn’t exist at certain layers at all.
It’s worth noting that ensuring agent security doesn’t mean reinventing the wheel. Decades of systems security have already given us solid, durable principles like least privilege, defense in depth, isolation, explicit authorization, and auditability. The real challenge is figuring out where to apply each of these principles within an agent stack.
Behavioral controls and infrastructure controls
This is where a distinction comes in that changes everything. Prompts, model safeguards, and harness logic shape what an agent will probably do, but none of them create a hard boundary around what the agent can do. That difference leads us to two distinct types of control.
Behavioral controls guide the agent’s actions
The model and the agent propose actions, and the harness directs them. Together, they interpret goals, handle ambiguities, and suggest next steps. The harness is the natural point for behavioral control because it owns the loop, the context, the tools, and the session, making it capable of steering behavior in the direction the operator intends. That steering is valuable, but every control implemented at this level still depends on how the model will behave.
Infrastructure controls determine what the agent can do
The final authority belongs to the environment where the agent runs. That environment holds identity, enforces policy, contains failures, records what happened, and reaches the same authorization decision every time, given the same verified state and the same approved policy. It doesn’t estimate what the agent will do. It determines what the agent can do.
The harness guides what an agent tries to do. Infrastructure controls what an agent is able to do. Both are necessary, but only one is authoritative.
It’s important to be clear that infrastructure enforcement isn’t infallible. It means that approved policy and verified configuration produce repeatable outcomes, and that the agent can’t choose whether or not to comply. The policy itself can still be wrong, and external outcomes may remain uncertain.
The layers of the agent stack and where security lives
A modern AI agent stack can be thought of as a set of stacked layers, where each one has distinct responsibilities and, consequently, distinct risks. The open source ecosystem has already been converging toward an organization that looks something like this:
- Distribution/product: handles package installation, defaults, and the supported experience.
- Orchestration (meta-harness): selects and coordinates different harnesses.
- Agent harness: turns a model into an agent, managing the loop, context, tools, and sessions.
- Secure runtime: responsible for isolation, identity, policy, credentials, and auditing — like NVIDIA OpenShell.
- Inference data plane: serves the model, handles caching, routing, and scaling.
Each of these layers is a potential point of failure if security isn’t explicitly considered. At the model layer, the most common risks are prompt injection attacks, where malicious inputs manipulate the agent’s behavior, and responses that go beyond the defined scope. At the orchestration layer, the risk lies in task chains that were never anticipated. At the harness layer, there’s an important detail: because it’s designed to be modifiable and programmable, it’s a terrible place to deposit security guarantees. A layer built to be altered can’t reliably enforce controls against its own modification.
And it’s precisely at the runtime — the outermost and often most neglected layer — where one of the greatest opportunities for real protection exists. A well-configured runtime can limit what the agent sees, what it can call, and how long it can operate before needing human confirmation. A credential with a narrow scope already limits potential damage, but keeping the raw credential out of the agent’s reach creates an even stronger boundary, enforced by the environment itself.
Establish the runtime boundary before launch
Models, harnesses, runtimes, policies, and inference deployments are being chosen more and more independently. This approach only works if the runtime guarantees hold no matter what components run above it. That means a security boundary needs to be established the moment the agent is launched — not after.
In practice, an orchestrator asks the secure runtime to create an environment and apply policies and governance. The chosen harness starts inside that runtime, and its plugins, processes, and tools run within the same boundary. Sub-agents receive delegated child runtimes with ceilings they can’t exceed. This is very different from treating the runtime as just another tool the harness can invoke after it’s already running.
A control the agent can choose not to invoke is not an effective security control.
Infrastructure controls that actually work
When it comes to reliability in agentic systems, infrastructure controls have gone from optional to the core of the protection strategy. Teams deploying agents in production today are learning firsthand that trusting the model’s judgment isn’t enough. You need to create an environment where the agent simply can’t do certain things, regardless of what it decides to do.
Five design rules help keep security decisions out of the agent’s control:
- The layer above proposes, the layer below decides: no model, agent, harness, or tool grants authority to itself.
- Authoritative policy location: keep policy below the boundary line. Policy-aware planning above the line is useful but only advisory.
- Verify every effect: control every file, process, network request, API call, and data operation.
- Just-in-time access: credentials should be narrow, short-lived, and easy to revoke.
- Isolation and recovery: isolate each agent, revoke access quickly, and preserve the record.
There’s also a growing trend of implementing what some teams call human-in-the-loop at critical points in execution. The idea isn’t to stop the agent at every step — that would cancel out the benefits of autonomy — but to define checkpoints where high-impact actions need confirmation before being executed.
Common security gaps in agent stacks
Many stacks share the same flaw: authorization decisions can be influenced by the agent itself or by untrusted data it reads. Among the most frequent gaps are unclear boundaries, with rules scattered across prompts, models, harnesses, and infrastructure; excessive access, with long-lived credentials far beyond what the task requires; untrusted data acting as control, where documents and messages redirect actions without having the authority to do so; uncontrolled external effects; failures that cascade; and incomplete audit trails.
Four security profiles for agent workloads
All profiles use the same stack and the same boundary, applying different controls based on the authority granted, the potential impact, and the likelihood of adversarial behavior:
- Level 1 — Isolated: pre-production work with disposable data, no production credentials, and restricted networking.
- Level 2 — Connected: pre-production using approved services, with short-lived identity and masked data.
- Level 3 — Production: changes to real systems, with task-scoped access and human approval for high-impact actions.
- Level 4 — Adversarial: red-team runs or frontier models without guardrails, with automatic quarantine and the strongest isolation possible.
What recent incidents taught us about secure design
The cases reported by OpenAI, Anthropic, and the UK AI Security Institute share a pattern that’s very much worth examining: in every one of them, the agent didn’t do anything the model considered wrong. From the model’s perspective, the actions made sense within the logic of the task. The problem was that the architecture around it didn’t have enough barriers to contain that behavior within the expected limits.
These incidents reinforce a lesson software engineering already knows well: complex systems fail in unexpected ways, and the best protection strategy is defense in depth. For AI agents, that translates to combining controls at the model level, in orchestration, in tools, and in the runtime — without relying on any single layer as the sole point of protection.
Another important takeaway is that security needs to be designed upfront, not bolted on after the fact. Teams that try to retrofit protections onto agentic systems already in production run into enormous friction. Starting with a least-privilege model, with sandboxing and logging from the very first deploy, is far more efficient. 🛡️
The security boundary in agentic systems isn’t a single line. It’s a network of controls distributed across the entire architecture, and every layer plays an irreplaceable role in that structure.
What recent months have made clear is that the reliability of an AI agent isn’t a property of the model. It’s a property of the system as a whole. And building trustworthy systems requires thinking about every layer with the same seriousness as the model itself. The more autonomous agents become, the more this lesson will define who can put AI into production responsibly — and who will keep putting out fires. 🔥
