kagent quickstart: your first POC, by Tom O'Rourke

The goal. Give a customer a kagent POC they can build on: install it, run a tool through MCP, deploy an agent two different ways, lock down what it can call, and see every step in the UI. Helm only, YAML plus a copy-paste command for each step, and apply and validate kept in separate cells.

Step by step

Install: kagent-enterprise CRDs, controller and the Solo Enterprise UI, by Helm, behind Keycloak. kagent docs
MCP: run an MCP tool server and register it as a kagent MCPServer. MCP docs
Declarative agent: a kagent Agent wired to the MCP tools. Agent docs
BYO / ADK: bring your own Google ADK agent and host it on kagent. BYO docs
Tool governance (optional add-on, documented): per-tool MCP authz at agentgateway, by identity, no mesh. MCP authz docs
Observability: agent traces in the Solo Enterprise UI. Tracing docs

0. Prerequisites

Download the labs and change into this one's folder, then point at your cluster and load the licence + model key. On kind, create a throwaway one-node cluster; on an existing cluster, skip that and just set your context.

Run every cell from this lab's directory (kagent-quickstart-kind/). The yaml/ paths are relative, so running them from another lab's folder deploys that lab's manifests instead (its Keycloak is on a different port) and later steps fail with Service keycloak does not have a service port 18080.

bash get the lab

git clone https://github.com/tjorourke/solo-labs.git
cd solo-labs/kagent-quickstart-kind

bash cluster + licence + model key

# kind (skip on an existing cluster)
kind create cluster --name kagent-poc     # kind makes it your current context

# Solo Enterprise licence (kagent) and the model provider key used here
export SOLO_LICENSE_KEY=<your-licence-key>
export ANTHROPIC_API_KEY=sk-ant-...
# only needed for the OPTIONAL step 5 (tool governance at agentgateway) -- often the same Solo licence
export AGENTGATEWAY_LICENSE_KEY=<your-agentgateway-licence-key>

1. Install

kagent install OIDC / IdP OBO / token exchange

Three pieces: Keycloak (the OIDC issuer the controller validates against), the kagent-enterprise CRDs + controller, and the Solo Enterprise UI (management chart, which also brings ClickHouse + the OpenTelemetry collector for tracing later). Keycloak comes first because the controller does OIDC discovery at startup.

The controller and the UI validate tokens against the browser-reachable issuer host keycloak.localtest.me, so it must resolve in-cluster too. One CoreDNS rewrite (below) points that host at the Keycloak Service for the whole cluster and survives Helm upgrades, so the charts install cleanly with --wait.

Identity provider

kind (Keycloak) real cluster (your IdP)

For a POC, run the bundled Keycloak (dev mode, realm imported from a ConfigMap). It is the OIDC issuer the controller and UI validate against.

YAML Copy-paste

Upstream Keycloak, dev mode, realm solo imported from a ConfigMap (users alice/bob/carol, groups field-fte/field-trial/field-admin, public client kagent). Issuer is pinned to keycloak.localtest.me:18080 so the UI SSO redirect is browser-reachable. Ships as yaml/keycloak/.

yaml yaml/keycloak/keycloak.yaml (excerpt)

apiVersion: v1
kind: Service
metadata: { name: keycloak, namespace: keycloak }
spec:
  selector: { app: keycloak }
  ports: [{ name: http, port: 18080, targetPort: 8080 }]
---
# StatefulSet: quay.io/keycloak/keycloak:26.3, args [start-dev, --import-realm]
#   KC_HOSTNAME = http://keycloak.localtest.me:18080   # browser-reachable issuer
#   realm imported from the keycloak-realm-import ConfigMap

bash deploy Keycloak + import the realm

kubectl create namespace keycloak
kubectl -n keycloak create configmap keycloak-realm-import \
  --from-file=realm.json=yaml/keycloak/realm.json --dry-run=client -o yaml | kubectl apply -f -
kubectl -n keycloak apply -f yaml/keycloak/keycloak.yaml
kubectl -n keycloak rollout status statefulset/keycloak --timeout=300s

bash validate: OIDC discovery

kubectl -n keycloak port-forward svc/keycloak 18080:18080 &
for _ in $(seq 1 30); do curl -sf -o /dev/null localhost:18080/realms/solo 2>/dev/null && break; sleep 1; done  # wait for the tunnel

curl -s localhost:18080/realms/solo/.well-known/openid-configuration | jq -r .issuer
# -> http://keycloak.localtest.me:18080/realms/solo

On a real cluster, use your own OIDC provider instead of the bundled Keycloak. kagent needs three things from it: an issuer (OIDC discovery URL), a client (the audience tokens are minted for), and a groups claim it can map to kagent roles. Then follow the install steps below with three changes:

Skip the kind-only Wire the issuer host in-cluster (CoreDNS) step — a real issuer already resolves in-cluster.
On the controller and UI installs, set oidc.issuer to your issuer (see the table) and oidc.clientId to your client, and map your IdP's groups in rbac.roleMapping.
Reach the UI over its Ingress / LoadBalancer URL, not a port-forward.

Issuer per provider (confirm from the issuer's /.well-known/openid-configuration):

Provider	Issuer
Okta	`https://<tenant>.okta.com/oauth2/<server>`
Auth0	`https://<tenant>.auth0.com/`
Entra ID (Azure AD)	`https://login.microsoftonline.com/<tenant-id>/v2.0`
Google	`https://accounts.google.com`
Keycloak (prod)	`https://<host>/realms/<realm>`

Two things kagent needs whatever the provider: the token must carry a lowercase groups claim (map your IdP's group/role claim to it) and the role-mapper reads claims.groups (the chart default claims.Groups returns 401); and the controller must reach the issuer's JWKS — for a public HTTPS IdP that is automatic, so no CoreDNS rewrite is needed.

Wire the issuer host in-cluster kind only

The controller and UI validate tokens against keycloak.localtest.me:18080 (the browser-reachable issuer). Make that host resolve in-cluster to the Keycloak Service with one CoreDNS rewrite — cluster-wide, survives Helm upgrades, no per-pod hostAlias.

This step is a kind-only workaround. kind has no LoadBalancer, Ingress or real DNS, so we reach Keycloak over a port-forward on localtest.me:18080 and rewrite that host in-cluster. On a real cluster you skip this entirely: use an external IdP (the BYO IdP tab) or Keycloak behind a real Ingress + TLS, so the issuer has one DNS name that resolves the same for the browser and in-cluster. Then just set oidc.issuer to that URL — no rewrite, no :18080, no port-forward.

bash add a CoreDNS rewrite for the issuer host

kubectl -n kube-system get cm coredns -o jsonpath='{.data.Corefile}' \
  | awk '/^    ready$/{print; print "    rewrite name keycloak.localtest.me keycloak.keycloak.svc.cluster.local"; next} 1' > /tmp/Corefile
kubectl -n kube-system create cm coredns --from-file=Corefile=/tmp/Corefile --dry-run=client -o yaml | kubectl -n kube-system apply -f -
kubectl -n kube-system rollout restart deploy/coredns
kubectl -n kube-system rollout status deploy/coredns --timeout=90s

kagent-enterprise (CRDs + controller)

bash secrets: OBO signing key + OIDC client secrets

kubectl create namespace kagent

# RSA signing key the controller uses to mint OBO tokens; the Secret MUST be named 'jwt'
openssl genpkey -algorithm RSA -pkeyopt rsa_keygen_bits:2048 -out /tmp/kagent-obo.pem
kubectl -n kagent create secret generic jwt --from-file=jwt=/tmp/kagent-obo.pem

# the controller validates the API token (issuer + JWKS + aud) with the public 'kagent'
# client, so its secret is a placeholder. The UI is a browser SPA ('kagent-ui', PKCE) whose
# backend exchanges the code with a confidential client ('kagent-backend') -- this secret
# MUST match that client's secret in the realm (yaml/keycloak/realm.json).
kubectl -n kagent create secret generic kagent-enterprise-oidc-secret --from-literal=clientSecret=public-client-no-secret
kubectl -n kagent create secret generic ui-backend-oidc-secret        --from-literal=clientSecret=kagent-backend-secret

bash install the CRDs + controller (Helm)

helm upgrade -i kagent-crds \
  oci://us-docker.pkg.dev/solo-public/kagent-enterprise-helm/charts/kagent-enterprise-crds \
  -n kagent --version 0.5.2 --wait

# the CoreDNS rewrite lets the controller reach the issuer in-cluster, so --wait is safe
helm upgrade -i kagent \
  oci://us-docker.pkg.dev/solo-public/kagent-enterprise-helm/charts/kagent-enterprise \
  -n kagent --version 0.5.2 \
  --set global.licensing.licenseKey="$SOLO_LICENSE_KEY" \
  --set providers.default=anthropic \
  --set providers.anthropic.apiKey="$ANTHROPIC_API_KEY" \
  --set oidc.issuer="http://keycloak.localtest.me:18080/realms/solo" \
  --set oidc.clientId=kagent \
  --set oidc.skipOBO=false \
  --set kagent-tools.enabled=true \
  --set otel.tracing.enabled=true \
  --set otel.tracing.exporter.otlp.endpoint="http://solo-enterprise-telemetry-collector.kagent:4317" \
  --set-json 'controller.envFrom=[{"configMapRef":{"name":"kagent-enterprise-config"}}]' \
  --set-json 'rbac.roleMapping={"roleMapper":"claims.groups.transformList(i, v, v in rolesMap, rolesMap[v])","roleMappings":{"field-fte":"global.Admin","field-trial":"global.Reader","field-admin":"global.Admin"}}' \
  --wait --timeout 12m

Three flags are load-bearing and fail silently if wrong: oidc.skipOBO=false (mint on-behalf-of tokens), the controller.envFrom override (without it the controller runs in auto-auth mode and 401s forwarded tokens), and the role-mapper on lowercase claims.groups (the chart default claims.Groups returns 401 against Keycloak).

Solo Enterprise UI (tracing + AccessPolicies)

bash install the management chart (Helm)

helm upgrade -i management \
  oci://us-docker.pkg.dev/solo-public/solo-enterprise-helm/charts/management \
  -n kagent --version 0.5.2 \
  --set cluster=kagent-poc \
  --set products.kagent.enabled=true \
  --set products.kagent.namespace=kagent \
  --set oidc.issuer="http://keycloak.localtest.me:18080/realms/solo" \
  --set-json 'rbac.roleMapping={"roleMapper":"has(claims.groups) ? claims.groups.transformList(i, v, v in rolesMap, rolesMap[v]) : []","roleMappings":{"field-fte":"global.Admin","field-admin":"global.Admin","field-trial":"global.Reader"}}' \
  --set-string licensing.licenseKey="$SOLO_LICENSE_KEY"

# the UI resolves the issuer via the same CoreDNS rewrite; just wait for it (4 containers, first pull is slow)
kubectl -n kagent rollout status deploy/solo-enterprise-ui --timeout=300s

The UI role mapping must match your IdP groups. The management chart's default roleMapper reads claims.Groups (capital G) and maps groups named admins/readers — neither matches Keycloak here, which emits groups (lowercase) with values field-fte / field-admin / field-trial. Without the rbac.roleMapping override above, login succeeds but the UI returns PERMISSION_DENIED on UserService/GetCurrentUser (no role). The override maps those groups to global.Admin / global.Reader, matching the kagent controller's mapping from step 1. On a real cluster, set the keys to your own IdP group names.

bash validate: the controller accepts a Keycloak token

kubectl -n keycloak port-forward svc/keycloak 18080:18080 &
for _ in $(seq 1 30); do curl -sf -o /dev/null localhost:18080/realms/solo 2>/dev/null && break; sleep 1; done  # wait for the tunnel
TOKEN=$(curl -s -X POST localhost:18080/realms/solo/protocol/openid-connect/token \
  -d grant_type=password -d client_id=kagent -d username=alice -d password=alice | jq -r .access_token)

kubectl -n kagent port-forward svc/kagent-controller 8083:8083 &
sleep 3
curl -s -o /dev/null -w '%{http_code}\n' localhost:8083/api/agents                               # -> 400 (no token)
curl -s -o /dev/null -w '%{http_code}\n' -H "Authorization: Bearer $TOKEN" localhost:8083/api/agents  # -> 200

2. MCP: run a tool server

MCP servers

kagent runs MCP tool servers in-cluster as MCPServer resources. The controller deploys the server, discovers its tools, and makes them available to agents and the UI. Here we run the official everything reference server (echo, get-sum, get-tiny-image, and more) over Streamable HTTP.

YAML Copy-paste

The controller runs this server and provisions its Service. Ships as yaml/mcp/mcpserver.yaml.

yaml kagent MCPServer

apiVersion: kagent.dev/v1alpha1
kind: MCPServer
metadata: { name: everything, namespace: kagent }
spec:
  transportType: http          # required discriminator (stdio | http)
  deployment:
    image: node:22-alpine
    cmd: npx
    args: ["-y", "@modelcontextprotocol/server-everything", "streamableHttp"]
    port: 3001
  httpTransport:
    path: /mcp
    targetPort: 3001
  timeout: 30s

bash deploy the MCP server

kubectl apply -f - <<'EOF'
apiVersion: kagent.dev/v1alpha1
kind: MCPServer
metadata: { name: everything, namespace: kagent }
spec:
  transportType: http          # required discriminator (stdio | http)
  deployment:
    image: node:22-alpine
    cmd: npx
    args: ["-y", "@modelcontextprotocol/server-everything", "streamableHttp"]
    port: 3001
  httpTransport:
    path: /mcp
    targetPort: 3001
  timeout: 30s
EOF

kubectl -n kagent wait mcpserver/everything --for=condition=Ready --timeout=180s

transportType is a required discriminator — without it the controller rejects the server with unsupported transport type. Tool discovery runs once the server's Service exists, so the tools appear a few seconds after it reports Ready.

Confirm the tools were discovered (they also show on the UI Tools page):

bash validate: list the discovered tools

kubectl -n keycloak port-forward svc/keycloak 18080:18080 &
for _ in $(seq 1 30); do curl -sf -o /dev/null localhost:18080/realms/solo 2>/dev/null && break; sleep 1; done  # wait for the tunnel
TOKEN=$(curl -s -X POST localhost:18080/realms/solo/protocol/openid-connect/token   -d grant_type=password -d client_id=kagent -d username=alice -d password=alice | jq -r .access_token)

kubectl -n kagent port-forward svc/kagent-controller 8083:8083 &
sleep 3
curl -s -H "Authorization: Bearer $TOKEN" localhost:8083/api/tools   | jq -r '.data[] | select(.server_name=="kagent/everything") | .id'
# -> echo
#    get-sum
#    get-tiny-image
#    ... (14 tools)

3. Declarative agent

Agents

A declarative Agent is model + instruction + tools, no code. It references the tools from the MCP server in step 2. Note apiVersion: kagent.dev/v1alpha2 — the storage version that carries spec.type and spec.declarative (the older v1alpha1 has a flat spec and rejects these fields).

YAML Copy-paste

Ships as yaml/agent/declarative-agent.yaml.

yaml declarative Agent

apiVersion: kagent.dev/v1alpha2
kind: Agent
metadata: { name: mathbot, namespace: kagent }
spec:
  type: Declarative
  description: A concise helper that adds numbers using the everything MCP tools.
  declarative:
    modelConfig: default-model-config          # from the install (Anthropic claude-haiku-4-5)
    systemMessage: |
      You are a concise assistant. When asked to add numbers, call the get-sum tool
      and reply with just the result.
    tools:
      - type: McpServer
        mcpServer:
          kind: MCPServer
          apiGroup: kagent.dev
          name: everything
          toolNames: ["get-sum", "echo"]

bash deploy the agent

kubectl apply -f - <<'EOF'
apiVersion: kagent.dev/v1alpha2
kind: Agent
metadata: { name: mathbot, namespace: kagent }
spec:
  type: Declarative
  description: A concise helper that adds numbers using the everything MCP tools.
  declarative:
    modelConfig: default-model-config          # from the install (Anthropic claude-haiku-4-5)
    systemMessage: |
      You are a concise assistant. When asked to add numbers, call the get-sum tool
      and reply with just the result.
    tools:
      - type: McpServer
        mcpServer:
          kind: MCPServer
          apiGroup: kagent.dev
          name: everything
          toolNames: ["get-sum", "echo"]
EOF

kubectl -n kagent wait agent/mathbot --for=condition=Ready --timeout=180s

Invoke it through the controller's OIDC-protected A2A endpoint, as alice:

bash validate: ask the agent to add two numbers

kubectl -n keycloak port-forward svc/keycloak 18080:18080 &
for _ in $(seq 1 30); do curl -sf -o /dev/null localhost:18080/realms/solo 2>/dev/null && break; sleep 1; done  # wait for the tunnel
TOKEN=$(curl -s -X POST localhost:18080/realms/solo/protocol/openid-connect/token   -d grant_type=password -d client_id=kagent -d username=alice -d password=alice | jq -r .access_token)

kubectl -n kagent port-forward svc/kagent-controller 8083:8083 &
sleep 3
# A2A JSON-RPC message/send — note the trailing slash on the path
curl -s -X POST "localhost:8083/api/a2a/kagent/mathbot/"   -H "Authorization: Bearer $TOKEN" -H 'content-type: application/json'   -d '{"jsonrpc":"2.0","id":"1","method":"message/send","params":{"message":{"role":"user","messageId":"m1","parts":[{"kind":"text","text":"What is 21 plus 21? Use the tool."}]}}}'   | jq -r '.result.artifacts[].parts[].text'
# -> 42

4. BYO / ADK: bring your own agent

BYO agents

A declarative agent is convenient, but sometimes you have your own agent code. A type: BYO agent hosts your image. Here a minimal Google ADK greeter: a few lines of Python that serve the kagent A2A protocol. It ships in src/greeter/ (agent + a stock main.py that wraps the ADK root agent in kagent's A2A server).

python src/greeter/agent.py

import os
from google.adk.agents import LlmAgent
from google.adk.models.lite_llm import LiteLlm

MODEL = os.environ.get("MODEL", "claude-haiku-4-5")

root_agent = LlmAgent(
    name="greeter",
    model=LiteLlm(model=f"anthropic/{MODEL}"),   # reads ANTHROPIC_API_KEY from the env
    description="A friendly greeter agent, brought your own with Google ADK.",
    instruction=(
        "You are a warm, concise greeter. Greet the user in one short, friendly "
        "sentence. If they give a name, use it."
    ),
)

bash build the image and load it into kind

docker build -t greeter-adk:dev src/greeter
kind load docker-image --name kagent-poc greeter-adk:dev
# on a real cluster: push to a registry the cluster can pull, and set that image below

YAML Copy-paste

The model key goes in the pod env from a Secret. Ships as yaml/agent/byo-agent.yaml.

yaml BYO Agent

apiVersion: kagent.dev/v1alpha2
kind: Agent
metadata: { name: greeter, namespace: kagent }
spec:
  type: BYO
  description: A minimal BYO Google ADK greeter agent.
  byo:
    deployment:
      image: greeter-adk:dev
      imagePullPolicy: IfNotPresent
      env:
        - name: MODEL
          value: claude-haiku-4-5
        - name: ANTHROPIC_API_KEY
          valueFrom: { secretKeyRef: { name: anthropic-key, key: ANTHROPIC_API_KEY } }

bash deploy the BYO agent

kubectl -n kagent create secret generic anthropic-key --from-literal=ANTHROPIC_API_KEY="$ANTHROPIC_API_KEY"

kubectl apply -f - <<'EOF'
apiVersion: kagent.dev/v1alpha2
kind: Agent
metadata: { name: greeter, namespace: kagent }
spec:
  type: BYO
  description: A minimal BYO Google ADK greeter agent.
  byo:
    deployment:
      image: greeter-adk:dev
      imagePullPolicy: IfNotPresent
      env:
        - name: MODEL
          value: claude-haiku-4-5
        - name: ANTHROPIC_API_KEY
          valueFrom: { secretKeyRef: { name: anthropic-key, key: ANTHROPIC_API_KEY } }
EOF

kubectl -n kagent wait agent/greeter --for=condition=Ready --timeout=180s

bash validate: greet the agent

kubectl -n keycloak port-forward svc/keycloak 18080:18080 &
for _ in $(seq 1 30); do curl -sf -o /dev/null localhost:18080/realms/solo 2>/dev/null && break; sleep 1; done  # wait for the tunnel
TOKEN=$(curl -s -X POST localhost:18080/realms/solo/protocol/openid-connect/token   -d grant_type=password -d client_id=kagent -d username=alice -d password=alice | jq -r .access_token)

kubectl -n kagent port-forward svc/kagent-controller 8083:8083 &
sleep 3
curl -s -X POST "localhost:8083/api/a2a/kagent/greeter/"   -H "Authorization: Bearer $TOKEN" -H 'content-type: application/json'   -d '{"jsonrpc":"2.0","id":"1","method":"message/send","params":{"message":{"role":"user","messageId":"g1","parts":[{"kind":"text","text":"Hi, my name is Tom."}]}}}'   | jq -r '.result.artifacts[].parts[].text'
# -> Hey Tom, great to meet you! 👋

5. Tool governance: per-tool MCP authz at agentgateway optional add-on

agentgateway MCP authorization

Documented, not installed by this POC. The quickstart (steps 1–4, 6) needs no agentgateway. This section shows how you'd add mesh-free tool governance when you want it: it stands up a standalone agentgateway alongside kagent (its own install + AGENTGATEWAY_LICENSE_KEY). Treat the steps below as a reference you opt into, not part of the base flow. The manifest is schema-validated; the flow was verified live.

The idea: put agentgateway in front of the MCP server. After it validates the Keycloak JWT, it authorizes each tool call by identity: we lock alice to the single get-sum tool, and everyone else keeps full access. This is the same mechanism the agentgateway POC uses, and it needs no service mesh — just agentgateway sitting on the MCP path.

Why not the kagent AccessPolicy CRD? kagent's own AccessPolicy (subject → action → targetRef) is a real resource, but in kagent-enterprise v0.5.2 the controller enforces it by rendering an enterprise-agentgateway-waypoint — an Istio ambient waypoint (HBONE listener, istio.io/use-waypoint on the MCP Service) — plus an EnterpriseAgentgatewayPolicy on it. A waypoint only receives traffic through ztunnel, which istiod programs, so the CRD needs the ambient mesh. (rbac.istioAuthzTranslation.enabled=true doesn't gate that — it only adds an extra Istio AuthorizationPolicy on top.) To keep this POC mesh-free we skip the CRD and enforce with the same policy engine on a standalone agentgateway instead. The CRD path and its prerequisites (kagent + OIDC → ambient mesh → agentgateway → STS token exchange) are documented under L7 access policies (waypoint) and L4 access policies (optional ztunnel/SPIFFE defence-in-depth).

Install agentgateway (CRDs + control plane)

bash Gateway API CRDs + agentgateway (needs AGENTGATEWAY_LICENSE_KEY)

kubectl apply -f https://github.com/kubernetes-sigs/gateway-api/releases/download/v1.5.0/standard-install.yaml

helm upgrade -i --create-namespace --namespace agentgateway-system --version v2026.7.0 \
  enterprise-agentgateway-crds \
  oci://us-docker.pkg.dev/solo-public/enterprise-agentgateway/charts/enterprise-agentgateway-crds

helm upgrade -i -n agentgateway-system enterprise-agentgateway \
  oci://us-docker.pkg.dev/solo-public/enterprise-agentgateway/charts/enterprise-agentgateway \
  --version v2026.7.0 \
  --set-string licensing.licenseKey=${AGENTGATEWAY_LICENSE_KEY}

kubectl -n agentgateway-system rollout status deploy/enterprise-agentgateway --timeout=120s
until kubectl get gatewayclass enterprise-agentgateway >/dev/null 2>&1; do sleep 2; done
kubectl wait --for=condition=Accepted gatewayclass/enterprise-agentgateway --timeout=60s

Front the MCP server + require a JWT + authorize per tool

YAML Copy-paste

A Gateway, an MCP backend over the existing everything Service, a route at /mcp, a JWT-auth policy and a per-tool authz policy. Ships as yaml/agw-authz/agw-mcp-authz.yaml.

yaml agw-mcp-authz.yaml

apiVersion: gateway.networking.k8s.io/v1
kind: Gateway
metadata: { name: agw, namespace: kagent }
spec:
  gatewayClassName: enterprise-agentgateway
  listeners:
    - name: http
      protocol: HTTP
      port: 80
      allowedRoutes: { namespaces: { from: Same } }
---
apiVersion: enterpriseagentgateway.solo.io/v1alpha1
kind: EnterpriseAgentgatewayBackend
metadata: { name: mcp-backend, namespace: kagent }
spec:
  mcp:
    targets:
      - name: everything
        static:
          backendRef: { name: everything }   # the Service the MCPServer created
          port: 3001
          protocol: StreamableHTTP
---
apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata: { name: mcp, namespace: kagent }
spec:
  parentRefs: [{ name: agw }]
  rules:
    - matches: [{ path: { type: PathPrefix, value: /mcp } }]
      backendRefs:
        - name: mcp-backend
          group: enterpriseagentgateway.solo.io
          kind: EnterpriseAgentgatewayBackend
---
# Require a valid Keycloak JWT on /mcp (no token -> 401). The issuer string MUST match the
# token's 'iss' exactly; JWKS is fetched from the in-cluster Keycloak Service.
apiVersion: enterpriseagentgateway.solo.io/v1alpha1
kind: EnterpriseAgentgatewayPolicy
metadata: { name: mcp-jwt, namespace: kagent }
spec:
  targetRefs:
    - { group: gateway.networking.k8s.io, kind: HTTPRoute, name: mcp }
  traffic:
    jwtAuthentication:
      mode: Strict
      providers:
        - issuer: "http://keycloak.localtest.me:18080/realms/solo"
          jwks:
            remote:
              jwksPath: "/realms/solo/protocol/openid-connect/certs"
              backendRef: { kind: Service, name: keycloak, namespace: keycloak, port: 18080 }
---
# Per-tool authz: rules are OR-ed, so alice sees + can call only get-sum; everyone else keeps all.
# CEL context: jwt.<claim> (from the token) and mcp.tool.name (the MCP tool).
apiVersion: enterpriseagentgateway.solo.io/v1alpha1
kind: EnterpriseAgentgatewayPolicy
metadata: { name: mcp-authz, namespace: kagent }
spec:
  targetRefs:
    - { group: enterpriseagentgateway.solo.io, kind: EnterpriseAgentgatewayBackend, name: mcp-backend }
  backend:
    mcp:
      authorization:
        action: Allow
        policy:
          matchExpressions:
            - 'jwt.email == "alice@example.com" && mcp.tool.name == "get-sum"'
            - 'jwt.email != "alice@example.com"'

bash apply, then wait for the proxy

kubectl apply -f yaml/agw-authz/agw-mcp-authz.yaml

until kubectl -n kagent get gateway agw \
  -o jsonpath='{.status.conditions[?(@.type=="Programmed")].status}' 2>/dev/null | grep -q True; do sleep 2; done
kubectl -n kagent rollout status deploy/agw --timeout=120s

Validate the enforcement

bash alice sees only get-sum; echo is denied; no token is 401

kubectl -n keycloak port-forward svc/keycloak 18080:18080 &
kubectl -n kagent    port-forward svc/agw       8080:80    &
for _ in $(seq 1 30); do curl -sf -o /dev/null localhost:18080/realms/solo 2>/dev/null && break; sleep 1; done

ALICE=$(curl -s -X POST localhost:18080/realms/solo/protocol/openid-connect/token \
  -d grant_type=password -d client_id=kagent -d username=alice -d password=alice | jq -r .access_token)

# alice lists tools through agentgateway -> ONLY get-sum
npx -y @modelcontextprotocol/inspector --cli "http://localhost:8080/mcp" --transport http \
  --header "Authorization: Bearer $ALICE" --method tools/list | jq -r '.tools[].name'

# alice calls echo -> denied ("Unknown tool: echo")
npx -y @modelcontextprotocol/inspector --cli "http://localhost:8080/mcp" --transport http \
  --header "Authorization: Bearer $ALICE" --method tools/call --tool-name echo --tool-arg message=hi

# no token -> 401
curl -s -o /dev/null -w '%{http_code}\n' -X POST localhost:8080/mcp \
  -H 'content-type: application/json' -d '{"jsonrpc":"2.0","id":"1","method":"tools/list"}'

What this proves, and the one caveat. agentgateway enforces per-tool MCP access by the caller's JWT identity, with no mesh — any client (or agent) that presents a token on the /mcp path is governed. To route a kagent agent's tool calls through this policy you point its tool at http://agw.kagent.svc.cluster.local/mcp (a RemoteMCPServer) and forward the caller's token to it: kagent does not forward the end-user identity to a RemoteMCPServer automatically, so supply it with headersFrom (a Secret) for a service identity, or use the OBO token exchange for per-user identity. On a real cluster set the policy issuer and the JWKS backendRef to your own IdP (the BYO IdP tab in step 1).

6. Observability

Tracing

The management chart from step 1 runs the OpenTelemetry collector and ClickHouse that back the UI, and the controller was installed with otel.tracing.enabled=true pointing at that collector. So every agent invocation emits a span tree (invocation → call_llm → execute_tool). Send some traffic, then open the Tracing tab.

Tracing is off by default on the controller — the two otel.tracing flags in step 1 are what turn it on. Without them agents emit zero spans and the Tracing tab stays empty.

bash generate some agent traffic

kubectl -n keycloak port-forward svc/keycloak 18080:18080 &
for _ in $(seq 1 30); do curl -sf -o /dev/null localhost:18080/realms/solo 2>/dev/null && break; sleep 1; done  # wait for the tunnel
TOKEN=$(curl -s -X POST localhost:18080/realms/solo/protocol/openid-connect/token   -d grant_type=password -d client_id=kagent -d username=alice -d password=alice | jq -r .access_token)

kubectl -n kagent port-forward svc/kagent-controller 8083:8083 &
sleep 3
for q in "What is 7 plus 5?" "Add 40 and 2"; do
  curl -s -X POST "localhost:8083/api/a2a/kagent/mathbot/" -H "Authorization: Bearer $TOKEN"     -H 'content-type: application/json'     -d "{\"jsonrpc\":\"2.0\",\"id\":\"1\",\"method\":\"message/send\",\"params\":{\"message\":{\"role\":\"user\",\"messageId\":\"o1\",\"parts\":[{\"kind\":\"text\",\"text\":\"$q\"}]}}}" -o /dev/null
done

bash view traces in the UI

# The UI login redirects the browser to the issuer, so BOTH port-forwards must run:
# the UI itself, AND Keycloak on 18080 (keycloak.localtest.me resolves to 127.0.0.1, so the
# browser reaches the issuer through this forward -- without it the login page is "connection refused").
kubectl -n keycloak port-forward svc/keycloak            18080:18080 &
kubectl -n kagent   port-forward svc/solo-enterprise-ui  4000:80
# -> http://localhost:4000/   (the Tracing tab shows the span tree per invocation)

Log in as alice / alice (Admin). The realm seeds three users, each with password = username:

User	Password	Group	UI role
`alice`	`alice`	`field-fte`	global.Admin
`carol`	`carol`	`field-admin`	global.Admin
`bob`	`bob`	`field-trial`	global.Reader

The UI is a browser SPA that logs in over OIDC: it redirects to Keycloak with the public kagent-ui client (PKCE), and its backend exchanges the code with the confidential kagent-backend client. Both clients live in yaml/keycloak/realm.json alongside kagent (the API client). The kagent-ui redirect URIs allow http://localhost:4000/* for this port-forward; on a real cluster set them to your UI's actual URL (BYO IdP tab), and set ui-backend-oidc-secret to the kagent-backend client secret.

Done. What to explore next

You now have Solo Enterprise for kagent installed by Helm behind Keycloak, an MCP tool server, agents deployed two ways (declarative and BYO / ADK), and traces in the UI. Where to go next:

Route the agent through the gateway. Step 5 enforces per-tool MCP authz at agentgateway by identity (no mesh). To govern a kagent agent's own tool calls the same way, point its tool at the gateway with a RemoteMCPServer and forward identity via OBO token exchange: Per-user MCP tool RBAC walks the full per-user path.
Give the agent more tools. Point the declarative agent at more of the everything server's tools, or add a second MCPServer and reference it in spec.declarative.tools.

Tear down (kind): kind delete cluster --name kagent-poc

Versions

Built and verified on:

Enterprisevalidated 2026-07-27

Gateway APIv1.5.0

Keycloak26.3

Solo Enterprise for agentgatewayv2026.7.0

Solo Enterprise for kagent0.5.2

kagent-adk (BYO ADK)0.9.1

management (UI) chart0.5.2