Agentic Immune System
A reference architecture for running enterprise AI agents without concentrating trust in any one of them. Five layers — identity, least privilege, containment, oversight and recovery — assume that some agent will eventually be wrong or subverted, and make that event survivable instead of catastrophic. The unit of defence is the individual run, not the fleet.
Key concepts
- Assume compromise: the question is not whether an agent will act wrongly but what it can reach when it does.
- Identity per run: every agent execution carries its own short-lived identity, never a shared operator credential.
- Containment over prevention: blast radius is designed, not hoped for — a subverted run affects one task, not the estate.
- Recovery is a design surface: the ability to trace, undo and stop is built in, not improvised during the incident.
Definition
The agentic immune system is a layered runtime architecture that lets AI agents take real actions on enterprise systems while bounding what any single run can reach, observe or damage, through per-run identity, task-scoped privilege, execution and egress containment, risk-based human oversight, and traceable recovery.
Architecture
The architecture inverts the usual question. Instead of asking how to stop an agent from making a mistake — which no guardrail achieves reliably against a system that reasons — it asks what the agent can reach at the moment it makes one. Every layer is a bound on reach, and the layers are independent so that one failing does not open the rest.
Identity is the foundation. Each run is issued a short-lived credential bound to the task, the requesting principal and the tools it declared it needs. Agents do not share a service account: when something goes wrong, the trace names a run, not a fleet, and revocation costs one token rather than a rotation across the estate.
Privilege is scoped to the task and expires with it. The tool broker grants the narrow set of capabilities a run declared up front — read this ticket, write this record — and refuses anything outside it, so a prompt injection that persuades the model to attempt more finds nothing to call.
Containment assumes the first two layers can be defeated. Execution happens in a sandbox with no ambient credentials; outbound network access goes through an allowlist so exfiltration has nowhere to send; and anything the agent produces is encoded at the boundary so its output cannot become the next system's instruction.
Oversight is where a human carries accountability, placed by risk rather than by default: irreversible or regulated actions pause for approval, and low-confidence outcomes escalate with the full run context attached. Gating everything defeats the automation and trains reviewers to click through.
Recovery closes the loop. Every run is traced under a single correlation id across models, tools and retries; actions are designed with a compensating counterpart where the underlying system allows it; and a kill switch stops a class of runs without taking down the platform.
Request flow
- 1. Request: a task arrives with its requesting principal and a declared set of tools and data scopes.
- 2. Issue: the broker mints a short-lived, run-scoped identity bound to that declaration.
- 3. Admit: the run starts in a sandbox with no ambient credentials and an egress allowlist.
- 4. Act: every tool call is authorised against the run's scope; anything outside it is refused and recorded.
- 5. Gate: irreversible or regulated actions pause for human approval with the run context attached.
- 6. Emit: outputs are encoded at the boundary so they cannot be executed as instructions downstream.
- 7. Close: the identity expires, the trace is sealed under its correlation id, and compensating actions stay available.
Components
Reference scenario
- Context
- An illustrative enterprise running dozens of agents across ticketing, finance and internal knowledge, where each agent can both read and write to systems of record.
- Scenario
- A retrieval step ingests a document containing an injected instruction telling the agent to export customer records. The model complies, but the export tool is outside the run's declared scope and is refused; the attempt is recorded, the run is stopped by the anomaly rule, and the correlation id gives the responder the full chain in one query.
- Technology
- Per-run credential broker, capability-scoped tool broker, sandboxed execution, egress allowlist, boundary encoding, risk-based approval gate, correlated tracing.
- Load
- Continuous background automation with bursts around business processes; the rare, high-impact action is a small fraction of calls and the one that carries the risk.
- Results
- Reference target, not a measured outcome: a subverted run is bounded to its declared scope, every refusal is attributable to a run, and any action taken can be traced and — where the underlying system permits — compensated.
Benefits
- A wrong or subverted agent costs one task, not the estate.
- Incidents are attributable to a run rather than to a shared account, so response is targeted and revocation is cheap.
- Prompt injection loses most of its value: persuading the model does not grant it capability it never held.
- Oversight effort concentrates on the actions that are actually irreversible, keeping approval meaningful.
Risks
- Scope declarations drift from what agents actually need, so teams widen them until least privilege is nominal.
- Gating too much trains reviewers to approve without reading, which is worse than not gating at all.
- Sandboxing and brokering add latency and operational surface that small deployments may not justify.
- Compensating actions are impossible for some external effects — a sent email or a paid invoice does not roll back.
KPIs
- Share of runs with a unique, short-lived identity
- The load-bearing control. Anything short of 100% means some path still uses a shared credential.
- Out-of-scope tool attempts, refused
- Counted per run. A rising number is a signal about the environment, not necessarily a failure of the design.
- Blast radius per run
- Number of systems and records a single run could reach if fully subverted. The number the architecture exists to shrink.
- Gated action rate and approval latency
- Both matter: too high a rate defeats automation, too high a latency makes teams disable the gate.
- Trace completeness
- Share of runs reconstructable end to end under one correlation id, including retries and model switches.
- Time to revoke
- From detection to a run's capability being gone. Short-lived credentials should make this close to expiry, not to a rotation.
Cost & scaling
- The identity and tool brokers are on the hot path of every call, so they set the ceiling; they are stateless and scale horizontally, but their latency is paid on every tool use.
- Sandbox start-up dominates cost for short runs; pooling warm sandboxes trades isolation depth for latency and should be an explicit decision.
- Human approval does not scale linearly and is the real constraint — the gated set must stay small as the fleet grows or the queue becomes the outage.
- Trace storage grows with runs times tool calls; sampling is safe for observability but not for audit, so the two retention policies should be separate.
Observed failure modes
- A shared service credential survives somewhere in the stack and silently defeats per-run identity.
- The egress allowlist is bypassed through an approved destination that itself forwards data.
- Traces break across a retry or a model switch, so the incident chain cannot be reconstructed.
- The approval queue backs up and the gate is disabled 'temporarily' to clear a backlog.
- Tool output is treated as trusted input by the next step, turning containment into a single boundary that leaks.
Lessons learned
- Design the blast radius before the capability: what an agent may reach is a harder question than what it may do, and answering it first makes the rest tractable.
- Per-run identity is the load-bearing layer; without it every other control is enforced against a subject you cannot name.
- A gate that fires on everything is a gate that fires on nothing — reserve human approval for the irreversible.
- Assume the model will be persuaded and design so that persuasion is not authorisation.
- Recovery has to be built while the system is calm; nobody designs a compensating action during an incident.
Technologies
Examples
- An injected instruction in a retrieved document asks for a data export; the tool is outside the run's scope and the call is refused and recorded.
- A finance agent drafts a payment and a human approves it before execution, with the run trace attached to the approval.
- A misbehaving agent class is stopped by kill switch while the rest of the fleet keeps running.
- An incident is reconstructed from a single correlation id spanning three models, nine tool calls and two retries.
FAQs
- Why an immune system rather than a firewall?
- A firewall assumes a boundary between inside and outside. An enterprise running agents has no such boundary: the agent is already inside, acting with real credentials. An immune system assumes intrusion is normal and invests in recognition, containment and repair rather than in a perimeter.
- Isn't this just least privilege with extra steps?
- Least privilege is one of the five layers and the most familiar. The architecture's claim is that it is not sufficient alone: without per-run identity you cannot scope privilege to a subject, and without containment and recovery a correctly-scoped run that still goes wrong has no bound and no undo.
- Does this stop prompt injection?
- No, and treating any control as stopping it is the mistake. It makes injection cheap to survive: persuading the model to attempt an action is not the same as the action being authorised, and the refused attempt is itself a signal.
- What is the minimum viable version?
- Per-run identity and task-scoped tool access. Those two give you attribution and a bound. Containment, gating and compensating actions matter more as the actions get more irreversible.
- How does this relate to the governance frameworks?
- It is the runtime expression of what they require. NIST AI RMF and ISO 42001 ask for accountability and traceability; OWASP LLM Top 10 and MITRE ATLAS describe the attacks. This architecture is where those obligations become identities, scopes, sandboxes and traces.