Auth is the topic where people most often run into trouble keeping the layers straight. This page maps every piece — JWT validation, claims-based authorization, OIDC login at the gateway, and RFC 8693 token exchange — to the component that actually runs it.
Five things to take away
JWT validation, authorization, and OIDC are three different things
Validation = is this token real? (signature, issuer). Authorization = does this token say I'm allowed to do this? (claims). OIDC = the login flow that produced the token in the first place. Different layers, different problems. Keep them separate in your head and most of the confusion goes away.
Validation + authorization are upstream Istio. OIDC + token exchange are Solo Enterprise.
OSS Istio gives you RequestAuthentication and
AuthorizationPolicy — that's enough to validate JWTs and
write rules on their claims. The OIDC login flow at the gateway
and RFC 8693 token exchange are gateway-side features; in OSS
you bolt them on with ext-authz, in Solo Enterprise they're first-class
on agentgateway.
Token exchange = swap a user's token for a backend-scoped token
A user logs in via Okta or Entra and gets a JWT. The gateway can't just forward that JWT to a downstream backend like Snowflake — wrong audience, wrong trust domain. So the gateway calls a token-exchange service: "give me a new token, scoped for this backend, that still represents this user." That's RFC 8693. The pattern of preserving user identity through the swap is called OBO (on-behalf-of).
The runtime: agentgateway → Solo control-plane STS → IdP
The agentgateway data plane calls Solo's control-plane Security Token Service to perform the swap. The STS either signs the new token itself, or delegates the exchange to an external IdP — tested today with Entra for OBO. The gateway-to-STS exchange is RFC 8693 compliant, so any RFC 8693 compliant IdP should plug in.
Token-exchange config is global per agentgateway, not per-route
You can't have one route exchange against Okta and another route exchange
against Entra in the same agentgateway. Two backends needing two IdPs =
two agentgateway instances today. Note this is separate from
OIDC login, which does work per-route via
AuthConfig. Login and exchange are different problems.
The whole flow on one canvas
How to read this: three operations happen at the
edge — OIDC login (purple, left), JWT validation (purple, middle) and
claims-based authorization (purple, right). All three run inside the
same Envoy / agentgateway proxy, but they're separate concerns and
separate CRs. The fourth piece — the amber STS box — only matters
when the edge needs to call a downstream service with a different
audience than the user's token. It accepts the user JWT and returns a
new service-scoped JWT signed by the STS. The waypoint can then
validate that fresh token with its own RequestAuthentication
if you want belt-and-braces.
The CRDs, group by group
🪪 RequestAuthentication security.istio.io/v1
Declares which JWT issuers to trust and where to fetch their JWKS.
The proxy verifies the signature, checks iss,
exp and (if you list them) audiences, and
on success parses claims into request.auth.principal,
request.auth.audiences and
request.auth.claims[...]. It does not deny
anything by itself — an invalid token is rejected (401), a missing
one is allowed through with empty claims. Pair it with an
AuthorizationPolicy to actually require a token.
Often surprising: the proxy doesn't fetch the JWKS — istiod
does, then ships it to every matching waypoint inline via
xDS. The diagram below shows the default flow and the
PILOT_JWT_ENABLE_REMOTE_JWKS=envoy alternative.
Default mode (istiod-fetch).
istiod fetches the JWKS at jwksUri, caches it in memory,
and pushes the JWKS inline with the JwtAuthn filter config
via xDS to every waypoint that has a matching
RequestAuthentication. The waypoint verifies signatures
against the cached JWKS on each request — no IdP call per request,
and only istiod needs egress to the IdP. Refresh
runs on a 20-minute timer (PILOT_JWT_PUB_KEY_REFRESH_INTERVAL).
Envoy-fetch mode.
With PILOT_JWT_ENABLE_REMOTE_JWKS=envoy (or
true) on istiod, istiod stops fetching the JWKS itself.
It ships only the RequestAuthentication config; each
waypoint Envoy then fetches the JWKS directly from the IdP and
refreshes on its own remoteJwks.cacheDuration.
Every waypoint pod now needs egress to the IdP — a
real consideration for airgapped clusters and NetworkPolicy. A
hybrid value splits the two modes by issuer.
RequestAuthentication · validate JWTs from a single OIDC issuer at the waypointattaches to the waypoint Gateway via targetRef
apiVersion: security.istio.io/v1
kind: RequestAuthentication
metadata:
name: payments-jwt
namespace: payments-prod
spec:
# Attach to the namespace waypoint (L7) — selector also works for
# workload-scoped attachment.
targetRefs:
- kind: Gateway
group: gateway.networking.k8s.io
name: waypoint
jwtRules:
- issuer: "https://example.okta.com/oauth2/default"
jwksUri: "https://example.okta.com/oauth2/default/v1/keys"
audiences:
- "api://payments"
forwardOriginalToken: true # leave the Authorization header intact upstream
outputClaimToHeaders:
- header: "x-user-email"
claim: "email"
What each field actually does
targetRefs→ Gateway- Attaches the policy to the waypoint Envoy — that's what makes JWT validation an L7 enforcement point.
selectorwould land it on workload sidecars instead. The two are mutually exclusive: pick one. jwksUri- Where the JWKS lives, but istiod is the one that fetches it (see diagram). The waypoint reads JWKS from its xDS-pushed config, not from this URL. Only one of
jwksUri/jwks(inline) may be set. audiences- Accepted
audclaim values. A token whoseaudisn't in this list fails validation and the principal is left empty — but the request still passes if noAuthorizationPolicyrequires authentication. forwardOriginalToken: true- Default is
false— the proxy strips theAuthorizationheader after validation. Settruewhen the upstream service needs to read the token itself (re-validate, parse claims, propagate). outputClaimToHeaders- Copies a string/int/bool (or nested) claim into a request header for upstream consumption. Each
headermust be unique; pre-existing values are overwritten.
RequestAuthentication on its own never blocks anything. It validates tokens that are present (returning 401 on bad signature / wrong issuer / wrong audience) and ignores requests with no token at all. To actually require authentication, pair it with an AuthorizationPolicy rule whose from.source.requestPrincipals: ["*"] demands a valid JWT.
RequestAuthentication · multiple issuers on the same waypointOkta for humans, internal IdP for service-to-service
apiVersion: security.istio.io/v1
kind: RequestAuthentication
metadata:
name: payments-jwt-multi
namespace: payments-prod
spec:
targetRefs:
- kind: Gateway
group: gateway.networking.k8s.io
name: waypoint
jwtRules:
# End-user tokens, minted by the corporate Okta tenant
- issuer: "https://example.okta.com/oauth2/default"
jwksUri: "https://example.okta.com/oauth2/default/v1/keys"
audiences: ["api://payments"]
# Service tokens, minted by an internal IdP for east-west calls
- issuer: "https://idp.internal/realms/services"
jwksUri: "https://idp.internal/realms/services/protocol/openid-connect/certs"
audiences: ["svc://payments"]
fromHeaders:
- name: "x-service-jwt" # accept on a dedicated header, not Authorization
How Envoy picks the matching rule
- multiple
jwtRules - Envoy selects a rule by matching the inbound token's
issclaim against the rule'sissuer. A token whoseissdoesn't match any listed issuer is treated as if no token was presented. audiencesper rule- Each issuer enforces its own audience. Cross-issuer audience confusion isn't possible because the rule is picked by
issfirst, thenaudis checked against that rule's list only. fromHeaders- Tells Envoy where to look for the token for this rule. The first rule (no
fromHeaders) keeps the defaultAuthorization: Bearer; the second rule readsx-service-jwtinstead — so a single request can carry one of each without collision. - downstream matching by
requestPrincipals - An
AuthorizationPolicycan route on either issuer using the<iss>/<sub>format, e.g.requestPrincipals: ["https://idp.internal/realms/services/*"]for service tokens only.
One token at a time. If a single request carries tokens that match two rules (e.g. both Authorization and x-service-jwt populated with valid JWTs), Envoy's output principal is undefined. Have callers send exactly one.
Easy to miss: list jwtRules entries with distinct
issuer values — Envoy picks the rule whose
iss claim matches the inbound token. Field is enumerated
in the upstream spec at
istio.io/.../request_authentication.
🛡️ AuthorizationPolicy with claims security.istio.io/v1
L4 predicates (source.principals, namespaces, ports)
are evaluated at ztunnel. Anything that reads from
the request — paths, methods, headers, JWT claims — is L7 and only
runs when a waypoint is in front of the destination.
No waypoint, no claim-based authz. The upstream docs spell this out:
HTTP-based attributes "cannot be processed" on TCP rules.
AuthorizationPolicy · allow only if groups claim contains payments-adminL7 — needs a waypoint
apiVersion: security.istio.io/v1
kind: AuthorizationPolicy
metadata:
name: payments-admin-only
namespace: payments-prod
spec:
targetRefs:
- kind: Gateway
group: gateway.networking.k8s.io
name: waypoint
action: ALLOW
rules:
- from:
- source:
requestPrincipals: ["*"] # require *some* valid JWT
when:
- key: request.auth.claims[groups]
values: ["payments-admin"]
What each predicate does — and how they combine
targetRefs→ Gateway- Lands the policy on the waypoint, the only thing in Ambient that parses HTTP and surfaces
request.auth.*. Aselector-attached version on a workload with no waypoint would fail-closed and silently break the workload. requestPrincipals: ["*"]- "Any valid JWT" — i.e. a token that produced a non-empty
<iss>/<sub>principal. Token without asubclaim → no principal → no match. request.auth.claims[groups]- Bracket syntax (no quotes) for the claim name. Chain brackets for nested claims (
[a][b]). When the claim is an array, the rule matches if any array element appears invalues. action: ALLOW- Whitelist semantics. Once any ALLOW policy targets the waypoint, every request that doesn't match an ALLOW rule is denied — adding one ALLOW flips the default posture from allow to deny.
JWT claims surface only inside the waypoint. RequestAuthentication on the same waypoint must already have validated the token — without it, request.auth.claims is empty and nothing matches. A useful debug: istioctl proxy-config listener <waypoint-pod> -o json | jq '..|.name?' | grep -i jwt to confirm the JwtAuthn filter is programmed.
AuthorizationPolicy · deny if the aud claim isn't oursaudience mismatch → 403
apiVersion: security.istio.io/v1
kind: AuthorizationPolicy
metadata:
name: payments-aud-guard
namespace: payments-prod
spec:
targetRefs:
- kind: Gateway
group: gateway.networking.k8s.io
name: waypoint
action: DENY
rules:
- when:
- key: request.auth.audiences
notValues: ["api://payments"]
DENY semantics — read the footnote before shipping
action: DENY- Evaluated before ALLOW. If any DENY rule matches, the request is rejected with 403 regardless of any ALLOW policy on the same target.
notValues- "NOT-IN-LIST". The rule matches when none of the listed values appear in the attribute. At least one of
valuesornotValuesmust be set per condition. request.auth.audiences- The
audclaim from a validated JWT, as a list. When there's no token (or noaud), the attribute is empty — and an empty list satisfiesnotValues, which matches the rule.
This rule over-denies anonymous requests. Because a missing aud matches notValues, anonymous (no-token) traffic hits this DENY too. If the intent is only "deny wrong-audience JWTs, let anonymous through," scope it with from.source.requestPrincipals: ["*"] so the DENY applies only to authenticated callers.
AuthorizationPolicy · mTLS identity AND claim-based scopeworkload identity at L4 · user scope at L7
apiVersion: security.istio.io/v1
kind: AuthorizationPolicy
metadata:
name: payments-write
namespace: payments-prod
spec:
targetRefs:
- kind: Gateway
group: gateway.networking.k8s.io
name: waypoint
action: ALLOW
rules:
- from:
- source:
# L4: only the checkout SA can hit /charge — enforced at ztunnel
principals:
- "spiffe://cluster.local/ns/checkout/sa/checkout"
to:
- operation:
methods: ["POST"]
paths: ["/charge"]
when:
# L7: the user JWT must carry the write scope — enforced at waypoint
- key: request.auth.claims[scope]
values: ["payments:write"]
How from + to + when compose
- fields within one rule
- All AND'd. Source identity, HTTP operation, and claim must all match for the rule to allow. Within a single field (e.g. multiple
principals), entries are OR'd. principalsseen at the waypoint- When attached to a waypoint, the principal check matches the source workload's identity carried through HBONE — i.e. the original caller, not the waypoint itself. The
spiffe://scheme prefix is required; omitting it silently won't match. methods+paths- L7. Only valid at the waypoint. The same policy attached via
selectorto a workload with no waypoint fails-closed: ztunnel sees L7 attributes it can't enforce and denies the request. request.auth.claims[scope]- Requires a validated JWT — anonymous requests have no
request.auth.*, so they can't match. Combined with ALLOW, this means: missing token → no match → denied by default.
Always attach principal-AND-claim policies at the waypoint. If you attach a SPIFFE-principal check at the destination ztunnel, it sees the waypoint's identity (not the original caller's) because the connection from waypoint → destination is a fresh HBONE leg. The waypoint is the only enforcement point that has both pieces of information at once.
Split brain to keep in mind: from.source.principals is
workload identity (SPIFFE / mTLS) and gets enforced
at ztunnel. when.request.auth.claims[...] is
user identity from a JWT and only gets evaluated
inside a waypoint. The
trust & identity page covers
the first one in depth.
🔁 OIDC + token exchange Solo Enterprise
Enterprise
Upstream OSS Istio doesn't run an OIDC login flow at the gateway —
you bolt one on with ext-authz, oauth2-proxy or a custom filter.
Solo Enterprise / agentgateway ships it as a first-class
AuthConfig with PKCE + session cookie support
(Keycloak, Entra ID, Okta) and a built-in Security Token Service
that does RFC 8693 token exchange. The YAML below is verbatim from
the docs at
docs.solo.io/agentgateway/.../oauth/authorization-code
and
.../obo/impersonation.
AuthConfig · OIDC authorization-code login on a Solo gateway, pointing at KeycloakPKCE redirect · session cookie in Redis
apiVersion: extauth.solo.io/v1
kind: AuthConfig
metadata:
name: oauth-authorization-code
namespace: agentgateway-system
spec:
configs:
- oauth2:
oidcAuthorizationCode:
appUrl: "http://${INGRESS_GW_ADDRESS}:80"
callbackPath: /openai
clientId: ${KEYCLOAK_CLIENT}
clientSecretRef:
name: oauth-keycloak
namespace: agentgateway-system
issuerUrl: "${KEYCLOAK_URL}/realms/master/"
scopes:
- email
session:
failOnFetchFailure: true
redis:
cookieName: keycloak-session
options:
host: :6379
headers:
idTokenHeader: jwt # forward id_token as Authorization upstream
---
apiVersion: enterpriseagentgateway.solo.io/v1alpha1
kind: EnterpriseAgentgatewayPolicy
metadata:
name: oauth-authorization-code
namespace: agentgateway-system
spec:
targetRefs:
- group: gateway.networking.k8s.io
kind: Gateway
name: agentgateway-proxy
traffic:
entExtAuth:
authConfigRef:
name: oauth-authorization-code
namespace: agentgateway-system
backendRef:
name: ext-auth-service-enterprise-agentgateway
namespace: agentgateway-system
port: 8083
Why two resources — and what each field is for
AuthConfigvsEnterpriseAgentgatewayPolicy- Split-by-design.
AuthConfigdefines what the auth flow does (PKCE, Keycloak, Redis session).EnterpriseAgentgatewayPolicyattaches it where (this Gateway listener). OneAuthConfigcan be reused by many policies/listeners. appUrl+callbackPath- The redirect URI Keycloak sees:
${appUrl}${callbackPath}. Must match the client's registered redirect URI in Keycloak exactly — most "redirect mismatch" 400s trace back to a trailing-slash difference. session.failOnFetchFailure: true- If the gateway can't reach Redis on a request, the request fails closed — no anonymous fallthrough. Flip to
falseonly if you've decided reduced security during a Redis outage is acceptable. headers.idTokenHeader: jwt- After login, the gateway sets header
jwt: <id_token>on the upstream request. This is what lets a downstreamRequestAuthentication(or another extauth step) re-validate the user identity — without it, the upstream just sees a session cookie and has no token to verify. backendRef· port 8083- Points at the
ext-auth-service-enterprise-agentgatewayDeployment shipped with the enterprise agentgateway chart. The Gateway delegates auth decisions to it over the ext-authz protocol.
Cookie collisions are silent. If you reuse session.redis.cookieName across two AuthConfigs (different IdPs / realms) on the same domain and cookie path, sessions overwrite each other and users get logged out of one when they log into the other. Give each AuthConfig a distinct cookieName.
EnterpriseAgentgatewayPolicy · RFC 8693 token exchange (impersonation OBO)swap user JWT for service JWT before the downstream call
# Two things wire the exchange:
# 1. A Secret carrying the downstream IdP's client credentials and token URL
# 2. A policy that references that Secret via tokenExchange.elicitation.secretName
#
# The gateway's STS reads the Secret, then on each request it swaps the
# inbound user token for a downstream-scoped token before forwarding.
apiVersion: v1
kind: Secret
metadata:
name: mcp-backend-token-exchange
namespace: agentgateway-system
type: Opaque
stringData:
app_id: mcp-backend
client_id: "<downstream-oauth-client-id>"
client_secret: "<downstream-oauth-client-secret>"
authorize_url: "https://idp.example.com/oauth/authorize"
access_token_url: "https://idp.example.com/oauth/token"
scopes: "mcp.read mcp.write offline_access"
---
apiVersion: enterpriseagentgateway.solo.io/v1alpha1
kind: EnterpriseAgentgatewayPolicy
metadata:
name: impersonation
namespace: agentgateway-system
spec:
targetRefs:
- kind: AgentgatewayBackend
group: agentgateway.dev
name: mcp-backend
backend:
tokenExchange:
elicitation:
secretName: mcp-backend-token-exchange # carries the IdP client + token URL
---
# Microsoft Entra ID variant — same idea, but Entra needs the
# tenant/client wired explicitly instead of via the elicitation Secret.
apiVersion: enterpriseagentgateway.solo.io/v1alpha1
kind: EnterpriseAgentgatewayPolicy
metadata:
name: impersonation-entra
namespace: agentgateway-system
spec:
targetRefs:
- kind: AgentgatewayBackend
group: agentgateway.dev
name: graph-mcp-backend
backend:
tokenExchange:
entra:
tenantId: "<entra-tenant-uuid>"
clientId: "<gateway-app-client-id>"
clientSecretRef:
name: entra-client-secret
key: client_secret
scope: "https://graph.microsoft.com/.default"
---
# The exchange request the STS makes on the wire (you don't write this
# yourself — it's what the gateway sends to access_token_url):
#
# POST /oauth/token
# grant_type = urn:ietf:params:oauth:grant-type:token-exchange # RFC 8693
# (or urn:...:grant-type:jwt-bearer for Entra OBO)
# subject_token = <user JWT from corporate IdP>
# subject_token_type= urn:ietf:params:oauth:token-type:jwt
# requested_token_type = urn:ietf:params:oauth:token-type:access_token
# audience = mcp-backend
Per-backend exchange — two flavours, both backend-scoped
targetRefs→AgentgatewayBackend- Backend-scoped, not gateway-scoped. The same agentgateway can exchange tokens for one backend and pass the original token to another, so only the downstreams that need service-scoped audiences pay the round-trip.
tokenExchange.elicitation.secretName- The generic OAuth elicitation flow. The Secret carries the downstream IdP's
client_id,client_secret,authorize_url,access_token_url, and requestedscopes. The gateway exchanges the inbound user JWT for a downstream-scoped token using those credentials and forwards only the new token. The original Authorization header is replaced. tokenExchange.entra- The Microsoft Entra OBO shape. Entra needs the tenant and client wired explicitly rather than via an opaque Secret.
clientSecretRefstill points at a Secret, but the rest of the wiring is on the CRD itself. CEL validation rejects setting bothelicitationandentraon the same policy. - runtime dependency
- This policy is the attachment point only. The exchange webserver must be enabled at install time with
tokenExchange.enabled: true(next example). Without that, the policy parses but has nothing to call.
Impersonation vs delegation is a property of the minted token, not the policy. The CRD's mode field accepts ElicitationOnly or ExchangeOnly and controls whether the gateway elicits consent, performs the exchange, or both. Whether the returned token carries an act claim (delegation) or omits it (impersonation) is decided by the downstream IdP based on the grant type and audience, not by anything we set here.
tokenExchange Helm values · install-time toggleturns on the exchange webserver the policy talks to
# The chart only exposes two knobs. Everything else (which IdP, which
# scopes, which audience) is per-policy via the CRD above. The default
# secretName here is the cluster-wide default used when a policy's
# elicitation block omits its own secretName.
tokenExchange:
enabled: true # required to start the exchange webserver
elicitation:
secretName: mcp-backend-token-exchange # default IdP credential Secret
What the chart does, and what it doesn't
enabled: true- Starts the token-exchange webserver inside the enterprise-agentgateway pod. With this off, every
EnterpriseAgentgatewayPolicy.spec.backend.tokenExchangeattaches but has nothing to call — requests fall through to the downstream with the original Authorization header intact. elicitation.secretName- Cluster-wide default credential Secret. A policy that sets its own
elicitation.secretNameoverrides this; a policy with no Secret at all falls back to this default. Useful when most backends share an IdP and only one or two need a different one. - not configurable via Helm
- The chart does not expose issuer URLs, token lifetimes, subject/actor validators, or signing keys. Those are either fixed by the implementation or driven entirely from per-policy CRDs and the IdP's own response. If a customer asks for a tunable that isn't shown above, it isn't one today.
Per-backend IdPs are fine. Different backends can target different downstream IdPs by setting their own elicitation.secretName (or by using the entra block instead). One agentgateway, many downstream identity systems — no need to stand up extra instances per IdP.
What's in the swapped token — and what NOT to do
Both OBO modes produce a per-user, per-request
token — sub is always the user. The difference is
whether the agent's identity is also carried via act.
The third card is the shape OBO is designed to replace:
one shared service-account token for every call, with user
identity lost (or relegated to a header). Don't ship that.
{
"iss": "solo-sts",
"sub": "alice@corp.com",
"aud": "snowflake-mcp",
"exp": 1700003600
}
{
"iss": "solo-sts",
"sub": "alice@corp.com",
"act": {
"sub": "spiffe://.../mcp-agent"
},
"aud": "snowflake-mcp",
"exp": 1700003600
}
{
"iss": "solo-sts",
"sub": "agentgateway-sa",
"aud": "snowflake-mcp",
"exp": 1700003600
}
// + header: x-on-behalf-of: alice@corp.com
// (or worse — no user context at all)
Note on naming: which CRD you reach for depends on the gateway.
On agentgateway, JWT validation and ext-auth attach via
EnterpriseAgentgatewayPolicy.spec.traffic.jwtAuthentication
and .traffic.extAuth. On kgateway, the same building
blocks attach via EnterpriseKgatewayTrafficPolicy.spec.entJWT
and .spec.entExtAuth, with the OAuth recipe in a
separate AuthConfig. The upstream Istio
RequestAuthentication still works on a waypoint; the
Solo CRDs add policy attachment, multi-tenant routing and the STS
integration.
Where token exchange lives in agentgateway today
Why a swap is needed at all
A user's JWT was issued for the user's own app to call. It's not valid for a downstream backend like a Snowflake MCP server — different audience, different issuer, often different scopes too. The gateway cannot just forward it.
{
"iss": "https://corp.okta.com",
"sub": "alice@corp.com",
"aud": "my-spa-app",
"scope": "openid email",
"exp": 1700000000
}
iss wrong
cannot forward
{
"iss": "https://solo-sts.internal",
"sub": "alice@corp.com",
"aud": "snowflake-mcp",
"scope": "snowflake.query.read",
"exp": 1700000000
}
How the runtime is wired
The OBO YAML above is the protocol surface. The runtime is worth pinning down separately. The agentgateway data plane does not call an external IdP directly. It calls Solo's control-plane Security Token Service, which either signs the new token itself or delegates the exchange to an external IdP.
The path the data plane takes
[ User token from corporate IdP (Okta / Entra / Keycloak) ]
│
▼
┌───────────────┐
│ agentgateway │ ◄── data plane, takes the request
│ (data plane) │
└───────┬───────┘
│ RFC 8693 token-exchange call
▼
┌───────────────┐
│ Solo Control │ ◄── signs the new token itself
│ Plane (STS) │ OR delegates to an external IdP
└───────┬───────┘
│ delegation (tested today with Entra OBO)
▼
┌──────────┐
│ IdP │
└──────────┘
AuthConfig attached per route).
Why this matters — agentic credential brokering
The use case driving this is what makes agentgateway worth picking over a generic API gateway. An LLM agent calling an MCP server that fronts something sensitive — a Snowflake query, a customer database, a payment API — should call that backend as the originating user, not as a shared service principal.
- Eliminates the anti-pattern of static service-account credentials baked into MCP servers.
- Every backend call carries the user's identity (impersonation
mode) or the agent's identity plus the user's
subinside anactclaim (delegation mode), so audit trails attribute each request to the originating principal. - Aligns with DORA Article 28 third-party access logging and GDPR Article 30 records-of-processing — without building a custom STS shim or sharing a service principal across agents.
- Gives a clean SPIFFE-ready foundation for agent identity going forward — the workload-identity story from the trust & identity page meets the user-identity story here.
OSS Istio vs Solo Enterprise — what's in the box
The split between "you can do this with upstream Istio" and "you'd reach for Solo Enterprise" is where most architectural conversations end up. This is what I've been able to pin down from the docs.
| Capability | Upstream OSS Istio | Solo Enterprise |
|---|---|---|
| JWT validation at waypoint / gateway | Yes · RequestAuthentication |
Yes · upstream CR + EnterpriseAgentgatewayPolicy.spec.traffic.jwtAuthentication (agentgateway) or EnterpriseKgatewayTrafficPolicy.spec.entJWT (kgateway) |
| JWKS auto-refresh | Yes · istiod fetches + pushes inline via xDS by default (refresh PILOT_JWT_PUB_KEY_REFRESH_INTERVAL, default 20m). Flip to Envoy-fetch with PILOT_JWT_ENABLE_REMOTE_JWKS=envoy |
Yes · cacheDuration field on remote JWKS |
| Multiple issuers per endpoint | Yes · list of jwtRules |
Yes · map of providers on the Solo JWT policy |
Claims as AuthorizationPolicy predicates |
Yes · when.request.auth.claims[…] at L7 |
Yes · same upstream surface, plus a claims matcher on the Solo JWT policy |
| OIDC login flow at the gateway (PKCE redirect, session cookie) | No — bolt on ext-authz / oauth2-proxy | Yes · AuthConfig.oauth2.oidcAuthorizationCode |
| External auth integration (ext-authz) | Yes · raw Envoy ext-authz API | Yes · EnterpriseAgentgatewayPolicy.spec.traffic.extAuth or EnterpriseKgatewayTrafficPolicy.spec.entExtAuth, referencing an AuthConfig |
| RFC 8693 OAuth token exchange / OBO | No | Yes · built-in token-exchange webserver on agentgateway, configured per-backend via EnterpriseAgentgatewayPolicy.spec.backend.tokenExchange |
| External IdP support for OBO | No | Yes · tested with Entra OBO; the gateway-to-STS exchange is RFC 8693 compliant, so any RFC 8693 IdP should plug in |
| Per-route token-exchange endpoint | No | Yes — each policy points at its own credential Secret via elicitation.secretName, so one agentgateway can exchange against multiple downstream IdPs |
| Multi-tenant per-route OIDC login | Partial — one ext-authz cluster per route is doable but manual | Yes · AuthConfig attached via per-route policy |
| Session storage (Redis / cookie) | No | Yes · session.redis and signed-cookie modes |
| JWT claim manipulation (copy / output to header) | Yes · outputClaimToHeaders, outputPayloadToHeader |
Yes · claimsToHeaders on the Solo JWT policy |
| Per-route audience enforcement | Yes · audiences on each jwtRule |
Yes · same upstream field + per-policy attachment |
OBO with full audit chain (sub + act claim) |
No | Yes · agentgateway exchange returns whatever the downstream IdP mints — request a delegation grant and the token carries act alongside sub |
Sources fetched: upstream Istio request_authentication and authorization-policy references; Solo EnterpriseAgentgatewayPolicy JWT and ext-auth surfaces; agentgateway token-exchange, OBO impersonation, delegation, OIDC authorization-code and JWT setup. Rows marked Partial are where the docs index doesn't speak directly to multi-tenant per-route OIDC in OSS — verify with the Solo team if you're sizing a real deployment against it.
CLI — inspect tokens at runtime
🔬 What to run when authz misbehaves debug
Most JWT bugs are unsurprising once you've actually read the token,
diffed its iss against what RequestAuthentication
expects, and checked the JWKS Envoy is using. These are the
commands worth in shell history.
kubectl get · what auth resources exist wherefirst thing to run
# Every namespace, both kinds
kubectl get requestauthentication,authorizationpolicy -A
# And the Solo CRDs that attach the JWT and ext-auth recipes
kubectl get enterpriseagentgatewaypolicy,enterprisekgatewaytrafficpolicy,authconfig -A
kubectl describe requestauthentication · see the JWKS URI Envoy will hitspot stale issuer URLs
kubectl -n payments-prod describe requestauthentication payments-jwt
# Or just the issuer/jwksUri pair, no scrolling:
kubectl -n payments-prod get requestauthentication payments-jwt \
-o jsonpath='{range .spec.jwtRules[*]}{.issuer}{"\t"}{.jwksUri}{"\n"}{end}'
istioctl proxy-config · confirm the JwtAuthn filter is configured on the waypointgrep is your friend
# Find the waypoint pod
kubectl -n payments-prod get pods -l istio.io/gateway-name=waypoint
# Dump all listener / route / cluster config for that pod
istioctl proxy-config all payments-prod/waypoint-7b6c... | less
# The JWT filter shows up under the HTTP filter chain — grep for it
istioctl proxy-config listener payments-prod/waypoint-7b6c... -o json \
| jq '.. | .name? // empty' | grep -i jwt
Decode a JWT with jq + base64 · no tooling requiredone-liner
# Paste the token into $TOKEN. Header is segment 1, payload is segment 2.
TOKEN='eyJhbGciOi...'
# Header (alg, kid)
echo "$TOKEN" | cut -d. -f1 | base64 -d 2>/dev/null | jq .
# Payload (iss, sub, aud, exp, claims)
echo "$TOKEN" | cut -d. -f2 | base64 -d 2>/dev/null | jq .
# Just the bits that decide whether validation will pass:
echo "$TOKEN" | cut -d. -f2 | base64 -d 2>/dev/null \
| jq '{iss, aud, exp, scope, groups: .groups}'
curl · verify the 401 → 200 transition after fresh loginthe smoke test
GW=https://api.example.com
# 1. No token — expect 401 (if AuthorizationPolicy requires a JWT)
curl -i $GW/api/payments
# 2. Expired token — expect 401, with a www-authenticate header
curl -i -H "Authorization: Bearer $OLD_TOKEN" $GW/api/payments
# 3. Fresh token — expect 200
TOKEN=$(curl -s -X POST "$IDP/oauth/token" \
-d grant_type=client_credentials \
-d client_id=$CID -d client_secret=$CSEC \
-d audience=api://payments | jq -r .access_token)
curl -i -H "Authorization: Bearer $TOKEN" $GW/api/payments
# 4. Same token, wrong audience claim → 403 from your AuthorizationPolicy
Full reference table
The resources you'll touch on this surface, with API group, what they do, and where in the data path they're actually evaluated.
| Resource | Source | API | What it does | Where it runs |
|---|---|---|---|---|
RequestAuthentication |
OSS | security.istio.io/v1 |
Validates a JWT's signature, issuer and audience; parses claims into request.auth.*. Doesn't deny — pair with AuthorizationPolicy. |
waypoint · ingress gateway |
AuthorizationPolicy |
OSS | security.istio.io/v1 |
ALLOW/DENY rules. from.source.principals is L4 (mTLS); when.request.auth.claims[…] is L7 (JWT). |
ztunnel (L4) · waypoint (L7) |
EnterpriseAgentgatewayPolicy (spec.traffic.jwtAuthentication) |
Solo | enterpriseagentgateway.solo.io/v1alpha1 |
Per-route JWT enforcement on agentgateway: providers map, remote JWKS caching, CEL access to jwt.<claim>. |
agentgateway |
EnterpriseKgatewayTrafficPolicy (spec.entJWT / spec.entExtAuth) |
Solo | enterprisekgateway.solo.io/v1alpha1 |
Per-route JWT and ext-auth attachment on kgateway. entExtAuth references an AuthConfig for the actual OIDC/API-key/OPA/LDAP/basic recipe. |
kgateway · ext-auth-service |
AuthConfig |
Solo | extauth.solo.io/v1 |
The recipe — OIDC fields (issuerUrl, clientId, callbackPath, scopes, session), API-key store, OPA module. |
ext-auth-service |
EnterpriseAgentgatewayPolicy |
Solo | enterpriseagentgateway.solo.io/v1alpha1 |
Attaches JWT, OIDC and token-exchange (RFC 8693) config to an agentgateway route or backend. | agentgateway proxy |
| Token-exchange STS | Solo | /oauth/token · /.well-known/jwks.json |
RFC 8693 Security Token Service. Accepts subject_token (user JWT) and optionally actor_token; returns a service-scoped JWT. |
agentgateway (built-in) |
| The JWT itself | N/A | RFC 7519 | Not a CRD — but it's the thing every CR above is reasoning about. Header (alg, kid) + payload (iss, sub, aud, exp, custom claims) + signature. |
Authorization header · cookie · query param |
Negative tests
The CRDs above are only worth anything if you can see them reject
the things they're supposed to reject. Every case below references
a shell variable ($VALID_READONLY_JWT,
$EXPIRED_JWT, etc.) — those come from the shared mint
script. Each case names which CR is doing the work and what the
access log should show.
6.0 Prereq — mint the test tokens
Run the shared
Mint your own test JWTs
helper once, then source tokens.env. The helper
generates a fresh RS256 keypair, writes the matching
jwks.json the waypoint will fetch, and exports
every variant the cases below reference.
Point RequestAuthentication.jwtRules[].jwksUri at
http://<host>:8080/jwks.json, set
issuer: https://test-idp.example.com and
audiences: ["api.example.com"] to match what the
helper signs.
6.1 · RequestAuthentication Tampered or unsigned JWT
Use $TAMPERED_JWT. The signature segment was
altered byte-by-byte after signing, so the JWKS-published key
no longer verifies it.
$ curl -is -H "Authorization: Bearer $TAMPERED_JWT" \
http://api.example.com/orders | head -4
HTTP/1.1 401 Unauthorized
www-authenticate: Bearer realm="http", error="invalid_token"
content-length: 22
Jwt verification fails
Logs: request.auth.* is empty in the waypoint
access log because no claims were ever populated.
6.2 · RequestAuthentication Expired token
$EXPIRED_JWT has exp one hour in the
past. Signature verifies, time check rejects.
$ curl -is -H "Authorization: Bearer $EXPIRED_JWT" \
http://api.example.com/orders | head -4
HTTP/1.1 401 Unauthorized
www-authenticate: Bearer realm="http"
content-length: 14
Jwt is expired
6.3 · RequestAuthentication Wrong audience
$WRONG_AUD_JWT carries aud=api.other.com.
Not in the jwtRules[].audiences list, rejected.
$ curl -is -H "Authorization: Bearer $WRONG_AUD_JWT" \
http://api.example.com/orders | head -4
HTTP/1.1 403 Forbidden
www-authenticate: Bearer realm="http", error="invalid_token"
content-length: 32
Audiences in Jwt are not allowed
6.4 · RequestAuthentication Wrong issuer
$WRONG_ISS_JWT claims a different
iss. No matching jwtRules[].issuer
entry, so no JWKS to verify against.
$ curl -is -H "Authorization: Bearer $WRONG_ISS_JWT" \
http://api.example.com/orders | head -4
HTTP/1.1 401 Unauthorized
www-authenticate: Bearer realm="http"
content-length: 28
Jwt issuer is not configured
6.5 · AuthorizationPolicy Claim-based deny
$VALID_READONLY_JWT has groups:
["orders-readonly"]. The policy requires
request.auth.claims[groups] to contain
orders-admin for the
/orders/cancel path. Token verifies (signature, aud,
iss, exp all valid), claims are populated, but the L7 RBAC
check denies.
$ curl -is -H "Authorization: Bearer $VALID_READONLY_JWT" \
http://api.example.com/orders/cancel | head -4
HTTP/1.1 403 Forbidden
content-length: 19
RBAC: access denied
Now try the same URL with $VALID_ADMIN_JWT (whose
groups includes orders-admin) and
confirm 200. Waypoint access log on the denial:
response_code=403,
response_code_details=rbac_access_denied_matched_policy[ns/policy-name].
The user identity (request.auth.principal) is still
logged so you can see who was denied.
6.6 · STS Wrong subject_token type
The STS only accepts the
urn:ietf:params:oauth:token-type:jwt or
access_token token types. An unrecognised value is
rejected at the token endpoint before any claim is examined.
$ curl -is -X POST https://gw.example.com/oauth/token \
-d grant_type=urn:ietf:params:oauth:grant-type:token-exchange \
-d subject_token=$VALID_READONLY_JWT \
-d subject_token_type=urn:wrong:type \
-d audience=api.example.com | head -8
HTTP/1.1 400 Bad Request
content-type: application/json
{"error":"invalid_request",
"error_description":"unsupported subject_token_type"}
6.7 · STS Audience the client cannot request
The subject token is valid, but the request asks the STS to mint a token for an audience the calling client is not registered to obtain tokens for. The STS refuses to widen.
$ curl -is -X POST https://gw.example.com/oauth/token \
-d grant_type=urn:ietf:params:oauth:grant-type:token-exchange \
-d subject_token=$VALID_READONLY_JWT \
-d subject_token_type=urn:ietf:params:oauth:token-type:jwt \
-d audience=api.different-tenant.com | head -8
HTTP/1.1 403 Forbidden
content-type: application/json
{"error":"invalid_target",
"error_description":"audience not permitted for this client"}
The STS is the audience gatekeeper, not the upstream. A
successful exchange would have returned a JWT whose
aud claim was attacker-chosen.
6.8 · STS Downscoping is one-way
The inbound subject token carries
scope=invoke.orchestrator. The exchange request
asks for the superset
scope="invoke.planner admin.planner". The STS
returns either narrower-or-equivalent scope or rejects.
$ curl -s -X POST https://gw.example.com/oauth/token \
-d grant_type=urn:ietf:params:oauth:grant-type:token-exchange \
-d subject_token=$VALID_READONLY_JWT \
-d subject_token_type=urn:ietf:params:oauth:token-type:jwt \
-d scope="invoke.planner admin.planner" \
-d audience=planner | jq -r '.scope // .error_description'
invoke.planner
Confirms the chain of actors cannot escalate. The exchanged
token's act claim records who performed the
exchange, so audit logs preserve the original subject and the
actor that called the STS.
6.9 · Upstream Blind pass-through is rejected
Skip the exchange. Forward $VALID_READONLY_JWT
(whose aud=api.example.com) directly to a
downstream MCP server whose
RequestAuthentication.audiences=["mcp.upstream"].
The upstream rejects on aud mismatch.
$ curl -is -H "Authorization: Bearer $VALID_READONLY_JWT" \
http://mcp.upstream.example.com/tools | head -4
HTTP/1.1 403 Forbidden
www-authenticate: Bearer realm="http", error="invalid_token"
content-length: 32
Audiences in Jwt are not allowed
Evidence that the exchange is not optional: the only way a
request reaches the upstream with valid auth is by going
through the STS so the gateway swaps the token for one whose
aud matches the upstream.
6.10 · Upstream No act claim where one is required
The upstream's CEL policy requires
request.auth.claims["act"]["sub"] == "planner".
Present an STS-minted token that has no act claim
(i.e. it was issued by a flow that skipped a hop). The CEL
expression evaluates falsy and the policy denies.
# Exchange directly without a prior hop, omitting an actor_token
$ HOP_TOKEN=$(curl -s -X POST https://gw.example.com/oauth/token \
-d grant_type=urn:ietf:params:oauth:grant-type:token-exchange \
-d subject_token=$VALID_READONLY_JWT \
-d subject_token_type=urn:ietf:params:oauth:token-type:jwt \
-d audience=tool-mcp | jq -r .access_token)
$ curl -is -H "Authorization: Bearer $HOP_TOKEN" \
http://tool-mcp.example.com/mcp | head -3
HTTP/1.1 403 Forbidden
content-length: 19
RBAC: access denied
response_code_details=jwt_authn_access_denied).
Case 6.5 leaves an RBAC entry
(response_code_details=rbac_access_denied_matched_policy[ns/name]).
STS-side rejections (6.6–6.8) should leave a record on the STS
audit stream with the requested audience and
scope, the calling client identity and the
error code, but never the token contents.
Where to go from here
This page covered user identity — JWTs minted by an OIDC provider, validated and reasoned about at the edge. The other half of identity in a mesh is workload identity — SPIFFE IDs in SVIDs, used for mTLS between ztunnels. Two different stories, one runtime. See the trust & identity walk-through for that side. For where these CRs attach in the data path, Gateway API in Ambient covers the routing surface.
Upstream references: Istio JWT routing task, RequestAuthentication reference, AuthorizationPolicy reference. Solo references: agentgateway JWT setup, agentgateway token exchange, agentgateway OBO.