Secure Sovereign AI on EKS, Part 1: Zero Trust Infrastructure

Sovereignty is only worth as much as you can prove, so this build proves its claims rather than asserting them, along three lines:

• Data residency: the weights and every inference stay in eu-west-2.
• Key custody: the root of trust is a Vault you run, unsealed by a KMS key you own in-region, and it both signs every workload identity and holds the cluster's secrets, with Secrets on disk encrypted under that same class of key.
• Traffic control: the model sits behind one governed door and the network is default-deny, so nothing leaves the region by accident.

The sovereignty section sets each of these out in full. Around it sit the layers a zero-trust deployment needs, from the hardened AWS edge to admission control to the one governed door in front of the model. It is all live, and every Solo product is marked in the diagram.

A production-grade, zero-trust architecture for agentic workloads

This is what a production deployment looks like when the workload is an AI model and the agents that call it: every layer hardened, and nothing trusted by default. The topology below is the real eu-west-2 build, from the AWS edge through the cluster to the managed services. Click any component, and its full detail, what it is, what we configured, and why, appears in the panel beside it (on a phone it opens as a card).

every layer deployed and verified on the live cluster dashed = best practice, not deployed in this lab ▸ Click any component to see what it is, what we configured, and why.

The layers, low to high

The topology above groups the stack by where things run. Here is the other way to read it, the way a security review does: by layer. Each control sits at a layer, and the higher the layer, the more it knows about what is actually happening. The network at the bottom sees a packet, an address and a port. The mesh above it sees a workload identity. The gateway at the top sees the whole request: which principal, which model, which tool, how often. That is why there is a control at every layer rather than one wall. Each one enforces something the layer beneath it cannot even see, and an attacker has to clear all of them, not one.

Two controls sit across all of it rather than at one layer. gVisor gives the agents a userspace-kernel boundary, so a workload that does run is boxed in at the syscall level. And observability watches every band, because a control that fires is only useful if someone, or something, can see that it did.

How a request reaches the model

The agent calls the gateway's in-cluster service over the ambient mesh (mutual-TLS), the gateway checks its Keycloak JWT, and only then does the request proceed. The gateway-to-vLLM hop is over the mesh too: ztunnel wraps it in mTLS with hourly SPIFFE certificates, and the model's authorization policy admits the gateway's SPIFFE identity and nothing else. So the request never touches a public IP, never crosses the network unencrypted, and never reaches a workload that has not proven who it is. The NLB out front is the cluster's one public address, and anything arriving there still has to clear the gateway's JWT check before it can reach the model.

The two planes: identity and policy

Read the stack as two planes that cross at every workload: identity (who is this?) and policy (what may they do?).

Identity has two layers. Service identity is the mesh: ztunnel gives every workload a short-lived SPIFFE certificate at L4, signed by a Vault you run, so which workload is talking is cryptographic, not an IP that changes on restart. Caller identity is the IdP: Keycloak issues a JWT the gateway and waypoint check at L7, so which principal (which agent, user or team) is proven too. SPIFFE says the connection is the gateway's workload; the JWT says the request is alice in the platform group. You need both. Elsewhere people split these across two products (SPIFFE plus Okta or Entra); here they are the mesh and Keycloak, both in-region.

Policy is what those identities let you decide. Admission (Pod Security Admission and Kyverno) refuses a bad workload before it runs: no privileged pods, no unsigned or unknown-registry images, no secrets in env vars, and an agent onto kagent only if it came through agentregistry. NetworkPolicy is default-deny at the packet level for the workloads outside the mesh, and ztunnel's L4 authorization does the same job inside it (a k8s NetworkPolicy is a no-op once ztunnel re-originates the traffic). The agentgateway waypoint is L7, where per-agent and per-tool authorization lives, and agentgateway is the one door where authentication, rate limits, guardrails and the audit trail meet. No single control is load-bearing: two independent ones have to fail at once for anything to leak.

What runs where

Everything runs in one EKS cluster in eu-west-2, split across a hardened platform node group and a GPU node group, with the sandboxed agents on their own isolated group. The security stack shares the platform nodes; only the model needs the GPU.

The configuration

Every object that stands this up, straight from the source in the lab repo. Pick a product on the left and a config on the right. Account ids are placeholders and anything secret-shaped is redacted; none of this is hand-typed on the page, it is generated from the source files so it cannot drift from what actually runs. Where a layer is applied by a script rather than a manifest (the mesh, the CA, Vault, kagent, agentregistry), the tab shows that script's exact block.

What makes this sovereign

Sovereignty here is three things you can point at: where the data lives, who holds the keys, and how the traffic is controlled. All three are in-region and provable, not asserted. Below the tiers is the Sovereign Guide: what is sovereign today, point by point.

Not air-gapped: brokered egress

Sovereign does not mean sealed. The agents need the internet to be useful, so they have it, but only through the gateway. Nodes sit in private subnets with a single NAT gateway as the one way out; NetworkPolicy stops the non-mesh workloads reaching it directly, and a Route 53 Resolver DNS firewall stops the mesh ones, whose ztunnel-reoriginated traffic a pod NetworkPolicy would miss. An agent that wants GitHub or a public MCP server goes through agentgateway, which authenticates the call, checks the destination against an allowlist, injects the credential the agent never holds, and logs it. The result is the opposite of a workload dialling anywhere it likes: one back door, brokered, and every outbound call on the record. It is also the only version of internet access you can put in front of an auditor.

Operational readiness

The architecture above is what the cluster is. Running it for real is a set of practices around it, and they are as much a part of the sovereignty story as any control on the diagram. Three are worth naming.

Build and release: CI/CD, GitOps and provenance

Everything on the cluster arrived as an image or an agent, and where those come from is the supply chain the runtime controls can only check, not create. cosign, Trivy and restrict-registries verify an image at admission; they cannot build a trustworthy one. That is the pipeline behind a sovereign deployment: images built from source you control, scanned for vulnerabilities and signed with a key you hold before they go anywhere near production, promoted through environments rather than pushed straight to prod, and released with the provenance to say which commit produced which running artifact. On the cluster side that is GitOps. Because every object here is already declarative YAML, a controller like Argo CD or Flux reconciles the cluster to what is in git, and git becomes the single source of truth: a change is a reviewed, signed pull request with an audit trail, drift is detected and corrected on its own, and a rollback is a git revert rather than a scramble. The whole cluster state, sovereign and reviewable, in a repo you control.

Test the restore

The lab backs up with Velero to an in-region bucket and snapshots the volumes, and that is the easy half. The half that matters is the restore, because a backup you have never restored is a hope, not a plan. So the restore gets rehearsed on a schedule: stand the cluster back up from the backup, bring Vault back from its own raft snapshot, re-attach the volumes, and prove the model answers and the agents run again, all inside the region. Time it and cost it, because that number is your real recovery time. A restore first attempted during an actual incident is how a recoverable outage becomes a permanent one.

Rehearsals and gamedays: kill switches you have pulled before

The point of layered controls is that one of them eventually fires for real, and the time to work out what to do next is not then. A gameday rehearses it. You inject a plausible exploit, a prompt-injected agent reaching for the model, a token that should not exist, a workload trying to leave the region, and you practise the response until it is routine. Part of that is knowing the kill switches and having used them before: cut an agent's egress at the gateway or the network, revoke a Keycloak token or client, scale a suspect agent to zero, isolate a namespace, or rotate the Vault-signed identities so a stolen certificate dies inside the hour it already lives. Every one of those is a control this architecture already has. A gameday is what turns having them into being able to reach for them under pressure.

Those three are practices. The architecture itself would also gain a few things in production that the lab leaves out on purpose, to keep it one region, one cluster and one command. They are worth naming so the gap is explicit rather than assumed.

Keep the AWS traffic in-region: VPC endpoints

The mesh keeps workload-to-workload traffic inside the region, but the cluster also talks to AWS itself: S3 for the weights and the backups, ECR for images, STS and KMS for identity and keys, CloudWatch for logs. In the lab those calls leave through the NAT gateway to the public AWS endpoints. In production they should not leave the VPC at all. An interface VPC endpoint (PrivateLink) for ECR, STS, KMS and CloudWatch, and a gateway endpoint for S3, keeps every AWS API call on the Amazon network inside eu-west-2, off the public internet. For a build whose claim is that nothing leaves the region, the AWS control-plane traffic is part of that claim, and endpoints are how you close it. They also take the NAT out of the path for AWS traffic, which removes a dependency and some data-processing cost.

No single points of failure

The lab runs one of several things a production cluster should run more than one of, because a single NAT gateway, a single-replica Vault and an in-cluster Keycloak are cheaper and easier to read. Production removes each: a NAT gateway per availability zone rather than one, Vault as a three-node raft cluster rather than a single replica so the mesh CA survives a node loss, and Keycloak's database on a managed multi-AZ store rather than a StatefulSet in the cluster it authenticates. The model is the honest exception: it is one replica on one GPU, and giving it real availability means GPU replicas behind the gateway and a recovery time you have measured. The PodDisruptionBudgets are already here; spreading the critical workloads across zones with topology constraints is the piece to add. None of this is in the lab, and all of it is standard EKS.

Run more than one cluster

The lab is one cluster. Ambient mesh does not stop at a cluster boundary: an east-west gateway peers two or more clusters so a workload in one reaches a workload in another over the same mTLS, with the waypoints still enforcing L7 policy in each cluster and the SPIFFE identities recognised across the peering. Global services then give you failover, so if a service's pods are gone in one cluster the traffic shifts to the copy in another. Kept in-region, several clusters spread across the eu-west-2 zones rather than one, this is how the single-cluster point of failure goes away without the data ever leaving the region. Multi-region uses the same mechanism, and for a sovereign build it is the thing you deliberately do not do: the clusters stay in one region on purpose.

A WAF at the gateway

agentgateway is the one door, and a web application firewall is a layer it runs at that door: inspecting each request against a rule set before it reaches the model or an agent, refusing the injection and abuse patterns that a schema check and a prompt guard are not shaped to catch. It sits in front of the JWT check and the rate limit as the coarse filter, so an obviously hostile request is turned away before any of the finer policy runs. In a production sovereign deployment the WAF is part of the edge, enabled on the gateway alongside the auth and rate-limit policies already there.

A few more, in brief

Three that belong on a production account rather than on the architecture diagram, noted here as best practice:

• Threat detection and account posture. GuardDuty for EKS runtime and malware detection you do not run yourself, CloudTrail as the organisation-wide, immutable, validated API audit trail, and AWS Config and Security Hub for drift and posture. These are account-level, so they sit outside this repo, but they are part of a production account in a way a single cluster is not.
• Operator access and audit. Human access to the API server through EKS access entries and SSO rather than long-lived kubeconfig credentials, a break-glass role kept aside for incidents, and a Kubernetes audit policy shipped to an in-region bucket, so who ran what against the cluster is recorded the same way the control-plane logs already are.
• Lifecycle and upgrades. A pre-production cluster to rehearse an EKS, node-group or add-on version bump before it touches production, with the upgrade itself driven through the same GitOps flow rather than by hand.

Running the environment

The GPU is the main cost, at roughly six dollars an hour, and it scales to zero between sessions. It is not the only cost: with the GPU off, the rest of the environment, the EKS control plane, the platform nodes that host the security stack, the NAT gateway and the storage, still idles at a few hundred dollars a month. That floor is the price of keeping the whole zero-trust environment alive with the model switched off. Bringing the model back is a scale-up of the GPU node group and a few minutes for vLLM to load the weights from the in-region volume; the weights never leave eu-west-2, so a rebuild never touches an external model host. Note that inference itself is not metered: once the GPU is up, a token costs nothing extra, which makes this a data-residency choice rather than a cost-saving one.

How to run it

The whole environment is code in the lab repo: the cluster is an eksctl definition, the Kubernetes objects are plain manifests, and an ordered set of scripts stands each layer up. There is no Terraform here because eksctl is already declarative infrastructure-as-code for EKS, but the shape is the same: one config describes the VPC, node groups and addons, and everything above it is versioned YAML and Helm values. Nothing secret is in the repo. The AWS account id and the weights bucket name are derived at run time from whichever profile is in effect, and the Solo Enterprise licence keys are read from your environment, never committed.

The scripts here run each layer by hand so you can read what each does. A production cluster would be driven by GitOps instead, a controller reconciling it to what is in git rather than a person running scripts, which is out of scope for a lab about the security layers themselves. That, along with the pipeline that builds and signs what runs here, and the backup and gameday rehearsals that keep it recoverable, is in Operational readiness above.

Before you start

You provide four things; the scripts create everything else.

The model itself needs nothing from you: Mistral-Small-3.2-24B is Apache-2.0 and not gated, so there is no Hugging Face token. The first run pulls the weights once, mirrors them to an S3 bucket in your account, and every run after that restores from S3 in-region and never contacts Hugging Face again.

One command

The whole environment stands up from a single orchestrator. Set your profile and licence file, then run it; it creates the cluster and all three node groups, then runs every layer in the one order that works. Every phase is idempotent, so a re-run after a failure picks up where it left off.

export SOVEREIGN_AWS_PROFILE=your-sso-profile
export SOVEREIGN_ENV_FILE=~/your-solo-licences.env   # exports SOLO_ISTIO_LICENSE_KEY etc.

./deploy-all.sh                 # empty account -> defended sovereign model with agents

Under the hood it runs eleven phases, each also runnable on its own (./deploy-all.sh <phase>):

cluster     # eksctl: VPC, and all three node groups (platform, gpu-od, sandbox)
model       # network policy, storage, IRSA, the GPU node, the weights, vLLM
ca          # Vault (raft + KMS auto-unseal + PKI), then the ambient mesh signed by it
idp         # Keycloak
gateway     # agentgateway, then TLS on the edge (HTTPS listener, HTTP -> 301)
enrol       # put the workloads in the mesh
policy      # Pod Security Admission + Kyverno
obs         # Prometheus, Grafana, Alertmanager, Mailpit
substrate   # gVisor on the sandbox node group
kagent      # the agent runtime, OIDC to Keycloak
verify      # Mistral answers, over TLS, through the gateway

The GPU meter starts at the model phase; everything before it is cheap. Nothing secret is in the repo: the account id and bucket name are derived at run time and the licence keys come from your environment.

Stopping and starting

The GPU is the main cost, and it scales to zero on its own command; the EKS control plane, platform nodes and networking are the rest, and they idle cheaply. Bring the GPU down at the end of a session and back up for the next one; the weights are already on the in-region volume, so a restart is a few minutes, not a re-pull.

./scripts/gpu.sh down     # stop the GPU meter (the rest idles cheaply)
./scripts/gpu.sh up       # bring the model back for the next session

./scripts/teardown.sh down   # remove everything, after confirming the weights are safe in S3

The full source, the eksctl config and every script in this order are in the lab repo, linked from the top of this page.

That is the infrastructure. Part 2, testing the exploits puts it under test: a rogue agent and real attacks, each refused by a layer and caught in the logs, with real cluster output. The model answering over TLS, a 401 with no token, four admission refusals, Vault surviving a pod kill, an agent denied a tool it is not entitled to, a pod provably sandboxed under gVisor, and the gateway itself detecting an unauthorised call and emailing the SOC.

And if the cloud is Google rather than AWS, Part 3 is the same architecture on GKE, every AWS service swapped for its Google Cloud equivalent. This lab deploys to AWS today; Part 3 is the GKE version of the same diagram, the security unchanged because it lives in the mesh and the gateway, not the cloud underneath.