MastertheMesh
Istio · ambient · sidecar · waypoint · ztunnel · OSS · kind
Live · Runs on kind

Sidecar to Ambient on OSS Istio

TO
Tom O'Rourke
EMEA Field CTO · Solo.io

Move a running petstore app from Istio sidecar mode to ambient on upstream community Istio, with nothing you have to buy or license. The dataplane switch is a helm upgrade; each namespace migrates with one label flip and a rolling restart, and flips back the same way. The L4-only namespace goes over with no waypoint; the two L7 namespaces share a single cluster-wide waypoint. The subset canary moves to an HTTPRoute, a load generator proves zero downtime at every cut, and the mixed-fleet policy gap that community images leave open is demonstrated live rather than talked about. One kind cluster, run step by step from the terminal.

Istio 1.30.3 (upstream) one cluster-wide waypoint label-flip migration + rollback HTTPRoute kind

The story: the estate runs Istio in sidecar mode and wants ambient, for the density win of one ztunnel per node instead of a proxy in every pod. And the whole migration is upstream: community images, community Helm charts, community labels. This lab does it end to end on OSS Istio, on one kind cluster, with a load generator running across every cut so you can see the zero in zero downtime. It also shows you, live, the one thing you must plan around on community images: while the fleet is mixed, a caller that still has a sidecar does not route through the destination's waypoint, so the waypoint's L7 policy does not yet apply to that caller.

Two design choices give this lab its shape. First, the switch is a label: after one helm upgrade the mesh speaks both dataplanes, and from then on each namespace chooses sidecar or ambient with a single label, in either direction. Second, one waypoint for the whole cluster: instead of a waypoint per namespace, a single Gateway in a platform namespace serves every L7 namespace that attaches to it, so the L7 fleet you operate is one deployment, not one per team.

The model to hold: in ambient there is no sidecar. ztunnel handles L4 for every enrolled pod (mTLS, L4 authorization, TCP telemetry). Everything the sidecar did at L7 moves to a waypoint. So a namespace with only L4 concerns needs no waypoint; a namespace with any L7 policy needs one before it is enrolled. That single decision drives the whole migration.

How to use this guide. The sections up to Run the lab are background: the model, the architecture and the YAML explained. When you are ready to do it, go to Run the lab: the numbered green STEP blocks are the exact commands to run, top to bottom. Everything else, including the amber notes and the collapsed YAML, is there to read, not run.
STEP — run this amber note — read only (FYI)

The end state

Where the migration lands. Every namespace is on ambient. The two L7 namespaces, petstore and petstore-orders, sit behind the same waypoint in mesh-infra, which does the HTTPRoute canary and enforces both GET-only authorization policies. The L4 namespace has only ztunnel. The calling estate is ambient too, which is what makes the waypoint's policy apply to its calls.

The OSS ambient end state: one waypoint for the whole cluster petstore-clients — ambient checkout · fortio — no sidecars HBONE (mTLS via ztunnel) cluster-waypoint — the ONE waypoint (mesh-infra) HTTPRoute canary · GET-only AuthorizationPolicy for both namespaces allowedRoutes: namespaces.from: All catalog v1 · v2 — petstore ambient · istio.io/use-waypoint orders — petstore-orders same waypoint, own namespace TCP :6379 (catalog identity) ztunnel — L4 (per node) mTLS · L4 AuthorizationPolicy · no waypoint redis — petstore-data (L4 only, no waypoint)

The switch is a label

There are two switches in this migration, and both are small on purpose.

The mesh-level switch makes the mesh bilingual. The cluster starts with plain base + istiod in sidecar mode. Three helm commands later (istiod upgraded to profile=ambient, plus the cni and ztunnel charts) the mesh can serve both dataplanes at once. Nothing migrates at this point: every app pod keeps its sidecar, the load generator keeps scoring 100%, and ztunnel sits idle because nothing is enrolled.

The namespace-level switch is one label swap. A namespace on sidecars carries istio-injection=enabled; to migrate it you swap that for istio.io/dataplane-mode=ambient and roll the deployments so the pods come back without sidecars:

bashthe whole per-namespace migration
kubectl label namespace petstore-data istio.io/dataplane-mode=ambient istio-injection- --overwrite
kubectl -n petstore-data rollout restart deploy

And the rollback is the same command run the other way: put istio-injection=enabled back, remove istio.io/dataplane-mode, roll the deployments, and the namespace is a sidecar namespace again. The lab does exactly that to petstore-orders near the end, under load, and then flips it forward a second time. Both directions are routine, which is the property you want before you migrate anything that matters.

One waypoint for the whole cluster

The default waypoint pattern is one per namespace: istioctl waypoint apply -n petstore creates a Gateway in petstore and that namespace uses it. Fine at small scale, but it recreates a shape you just left behind: an Envoy deployment per team, each with replicas, requests and limits, upgrades and monitoring. If the reason you wanted ambient was fewer proxies to run, a proxy per namespace is not the end state you had in mind.

Waypoints do not have to be per-namespace. A waypoint is just a Gateway, and a Gateway can admit routes from anywhere you allow. This lab deploys one waypoint in a platform namespace (mesh-infra) with allowedRoutes: namespaces.from: All, and every L7 namespace attaches to it with two labels:

bashattach a namespace to the shared waypoint
kubectl label namespace petstore \
  istio.io/use-waypoint=cluster-waypoint \
  istio.io/use-waypoint-namespace=mesh-infra

istio.io/use-waypoint names the waypoint; istio.io/use-waypoint-namespace says where it lives (without it, Istio looks for the waypoint in the workload's own namespace). Policy and routing stay exactly where they belong. The AuthorizationPolicy and HTTPRoute live in the workload's namespace and target its Service, so teams keep ownership of their rules while the platform team owns the one proxy that enforces them. In this lab, petstore and petstore-orders both attach to it, and you can watch a single Envoy pod enforce two namespaces' policies.

When would you not share one waypoint? The trade is the usual shared-infrastructure one: a shared waypoint is a shared failure domain and a shared noisy neighbour, and its pod count scales for the cluster's whole L7 load rather than one team's. Nothing stops you mixing patterns: a shared waypoint for the long tail of small namespaces and dedicated waypoints for the two services that dominate traffic. The attachment labels make that a per-namespace decision you can change later, not an architecture you commit to on day one.

L4 or L7: the one decision per namespace

Audit each namespace's policies and ask one thing: does anything here operate on the content of a request? ztunnel is an L4 proxy, and it does a lot with no waypoint at all: mutual TLS, authorization on identity, namespace and port, TCP telemetry, and L4 load balancing. A namespace whose only concerns are those migrates with ztunnel alone. In this lab that is petstore-data, whose STRICT mTLS and identity-based AuthorizationPolicy carry over without editing a single line of YAML.

You only need the waypoint when something looks inside HTTP: an AuthorizationPolicy on methods, paths or headers, VirtualService or HTTPRoute behaviour (routing, retries, timeouts), JWT rules, per-request telemetry. Deploy it (or attach to it) before you enrol the namespace, because if you enrol a namespace that has an L7 selector policy and no waypoint, ztunnel fails safe and denies all traffic to that workload. And the L7 policy needs its shape changed as it moves: a sidecar policy selects pods (selector.matchLabels), a waypoint policy targets the Service (targetRefs). Same rule, different attachment:

Sidecar (before)Waypoint (after)
L7 authz selector.matchLabels: {app: catalog} — enforced in catalog's own sidecar targetRefs: [{kind: Service, name: catalog}] — enforced on the shared waypoint
Canary DestinationRule subsets + VirtualService weights, applied by the caller's sidecar per-version Services + HTTPRoute weights, enforced on the waypoint (waypoints do not do subset routing)
L4 authz / mTLS selector-based AuthorizationPolicy, PeerAuthentication unchanged — ztunnel enforces the same objects

Who enforces your L7 policy today?

This is the question that decides your migration order, and the lab makes you watch the answer change. In sidecar mode the GET-only rule on catalog is enforced by catalog's own sidecar: any caller, from anywhere, gets a 403 on a DELETE. Once petstore migrates behind the waypoint, the rule is enforced by the waypoint. And on community Istio, traffic only routes through a waypoint when the caller is on the ambient dataplane. A caller still running a sidecar sends straight to the pods, so for that caller the DELETE that used to be denied comes back 200. The lab demonstrates this live in STEP 10 and then closes it in STEP 11 by migrating the calling namespace.

The practical rule for OSS ambient: migrate a service's callers in the same window as the service whenever the service carries L7 security policy, and treat the mixed period as a period where L7 rules are advisory for unmigrated callers. If you need the mixed period itself to be safe (sidecar and ingress callers routed through the waypoint so its policy holds for everyone during a long, team-by-team migration) that interop is what the Solo distribution adds on top of upstream (ENABLE_WAYPOINT_INTEROP), and the Enterprise edition of this lab runs the same migration with that behaviour on.

Notes from the field

  1. Your sidecars are initContainers now. On a current cluster (Kubernetes 1.29+, this lab runs 1.35) Istio injects istio-proxy as a native sidecar container: it lives in spec.initContainers with restartPolicy: Always, and kubectl get pod -o jsonpath='{.spec.containers[*].name}' will not show it. Scripts that audit "does this pod have a sidecar" must check both lists. The READY column still counts it (2/2).
  2. Roll every sidecar namespace once, right after ambient arrives. Sidecars injected before istiod ran with profile=ambient lack ISTIO_META_ENABLE_HBONE=true, so ztunnel can only reach them in plaintext, which STRICT mTLS rejects. One rolling restart per namespace, before any neighbour enrols, re-injects them HBONE-capable. Skip this and the first namespace you migrate loses its sidecar callers.
  3. ztunnel fails safe on L7 selector policies. Enrol a namespace that still has a selector-based L7 AuthorizationPolicy and ztunnel denies all traffic to those pods rather than let an HTTP rule go unenforced. Convert the policy to targetRefs (waypoint-enforced) before the label flip, then delete the selector version.
  4. Waypoints do not do DestinationRule subset routing. Move any subset-based VirtualService split to per-version Services + an HTTPRoute before its callers go ambient. Do it while everything is still on sidecars: sidecars apply Service-attached HTTPRoutes too, so the cutover is a no-op you can verify safely.
  5. Audit for mode: DISABLE before enrolling. STRICT and PERMISSIVE PeerAuthentication carry over to ambient; DISABLE has no ambient equivalent because HBONE is always mTLS. Find any DISABLE carve-outs and retire them first.
  6. An L4 deny does not reset the client's connection. With a client-side proxy in the path the client's connect() succeeds locally, so probing a deny with exit codes lies to you. Probe with a payload instead: send PING to redis and check whether +PONG comes back. Allowed identity: PONG. Denied identity: silence.
  7. Rollback order matters. Going back to sidecars: flip the labels, roll the deployments, and only then restore the selector policy and retire the targetRefs one. Apply the selector policy while the pods are still ambient and ztunnel fail-safes into denying the workload (note 3, in reverse).
  8. Zero downtime still needs graceful shutdown. Every migration step here is a rolling restart, and a server that dies instantly on SIGTERM will reset whatever keep-alive connections it is holding, mesh or no mesh. The app deployments carry a preStop: sleep: {seconds: 5} (the native sleep action, so it works in scratch images with no shell) so the endpoint deregisters and drains before the process stops. Without it, expect an occasional blip of a request or two per cut; the mesh gets blamed, but the pod lifecycle is the culprit.

Run the lab, step by step

This is what you run, top to bottom, from a clone of the lab repo (in the istio-ambient-migration-oss-kind directory). Each green STEP has the change to run (open view the YAML to see exactly what goes in) and the kubectl or curl to prove it, with expected output in comments. Nothing needs a licence, a registry login or a cloud account; the only tools are Docker, kind, kubectl, helm and istioctl. A fortio load generator runs in petstore-clients; a bounded fortio run at each cut should read 100% 200 once the mesh reconverges (allow a few seconds after a restart or a route change before you measure). If you would rather watch it than type it, ./scripts/e2e.sh runs this whole sequence with assertions.

STEP 1

Create the kind cluster and install the Gateway API CRDs

bashrun
kind create cluster --config kind/cluster.yaml

kubectl apply --server-side -f https://github.com/kubernetes-sigs/gateway-api/releases/download/v1.5.1/standard-install.yaml
bashverify
kubectl get nodes
#   ambient-oss-control-plane   Ready   control-plane   (3 nodes total, all Ready)
kubectl get crd gateways.gateway.networking.k8s.io httproutes.gateway.networking.k8s.io
#   both CRDs listed
yamlview the YAML · kind/cluster.yaml
kind: Cluster
apiVersion: kind.x-k8s.io/v1alpha4
name: ambient-oss
nodes:
  - role: control-plane
  - role: worker
  - role: worker

a 3-node kind cluster (1 control-plane, 2 workers) and the Gateway API CRDs installed.

STEP 2

Load the upstream Istio images into kind

Straight from Docker Hub, no registry auth. Pre-loading keeps the later helm installs fast and deterministic.

bashrun
ARCH=$(uname -m | sed 's/aarch64/arm64/;s/x86_64/amd64/')
for img in pilot proxyv2 install-cni ztunnel; do
  ref="docker.io/istio/$img:1.30.3"
  docker pull -q "$ref"
  docker save --platform "linux/$ARCH" "$ref" -o "/tmp/$img.tar"
  kind load image-archive "/tmp/$img.tar" --name ambient-oss
done
bashverify
docker exec ambient-oss-control-plane crictl images | grep -c docker.io/istio
#   4   (pilot, proxyv2, install-cni, ztunnel on the node)

the four upstream Istio images present on every kind node.

STEP 3

Install Istio in plain sidecar mode

This is the estate you are migrating from: base (CRDs) + istiod, no ambient components at all.

bashrun
helm upgrade --install istio-base base \
  --repo https://istio-release.storage.googleapis.com/charts \
  -n istio-system --create-namespace --version 1.30.3 \
  --set defaultRevision=default --wait

helm upgrade --install istiod istiod \
  --repo https://istio-release.storage.googleapis.com/charts \
  -n istio-system --version 1.30.3 --wait \
  --set global.hub=docker.io/istio --set global.tag=1.30.3 \
  --set meshConfig.accessLogFile=/dev/stdout
bashverify
kubectl -n istio-system get deploy istiod
#   istiod   1/1
kubectl -n istio-system get ds
#   No resources found   (no CNI node agent, no ztunnel — sidecar mode only)

istiod Running, and no DaemonSets: the ambient dataplane does not exist yet.

STEP 4

Deploy the petstore app and its sidecar-mode policies

Four namespaces, all labelled istio-injection=enabled: petstore (catalog v1/v2 + DR/VS canary + GET-only authz), petstore-orders (a second L7 service with its own GET-only rule), petstore-data (redis + identity-based L4 authz), petstore-clients (checkout + fortio). Plus mesh-wide STRICT mTLS.

bashrun
kubectl apply -f yaml/00-namespaces.yaml
kubectl apply -f yaml/10-apps/
kubectl apply -f yaml/20-policies-sidecar/
kubectl -n petstore rollout status deploy/catalog-v1 deploy/catalog-v2 deploy/data-client --timeout=300s
kubectl -n petstore-orders rollout status deploy/orders --timeout=180s
kubectl -n petstore-data rollout status deploy/redis --timeout=180s
kubectl -n petstore-clients rollout status deploy/checkout deploy/fortio --timeout=180s
bashverify — the sidecar estate works and enforces
# native sidecars: istio-proxy is an INIT container on k8s 1.29+ (READY still says 2/2)
kubectl -n petstore get pod -l app=catalog -o jsonpath='{.items[0].spec.initContainers[*].name}'; echo
#   istio-init istio-proxy

# the GET-only rule, enforced by catalog's own sidecar
kubectl -n petstore-clients exec deploy/checkout -c checkout -- curl -s -o /dev/null -w '%{http_code}\n' http://catalog.petstore/
#   200
kubectl -n petstore-clients exec deploy/checkout -c checkout -- curl -s -o /dev/null -w '%{http_code}\n' -X DELETE http://catalog.petstore/
#   403

# the canary is 100% v1 (the caller's sidecar applies the VirtualService)
kubectl -n petstore-clients exec deploy/checkout -c checkout -- sh -c \
  'i=0; while [ $i -lt 30 ]; do curl -s http://catalog.petstore/; i=$((i+1)); done' | grep -c '"version":"v2"'
#   0

# redis: the allowed identity gets PONG, the checkout identity gets silence (L4 deny)
kubectl -n petstore logs deploy/data-client --tail=1
#   PONG
kubectl -n petstore-clients exec deploy/checkout -c checkout -- sh -c 'printf "PING\r\n" | nc -w 3 redis.petstore-data 6379'
#   (no output — denied)

# the zero-downtime scoreboard
kubectl -n petstore-clients exec deploy/fortio -c fortio -- fortio load -c 4 -qps 25 -t 10s -quiet http://catalog.petstore/ | grep 'Code '
#   Code 200 : 250 (100.0 %)

a fully working sidecar estate: canary pinned to v1, DELETE denied 403, redis answering only the catalog identity, fortio at 100%.

STEP 5

The mesh-level switch: the ambient dataplane arrives, under load

Start a fortio run first, then make the change while it flies: istiod to profile=ambient, plus the cni and ztunnel charts. Sidecars keep serving throughout. Nothing is enrolled yet.

bashrun
# 150s of load in the background — it flies across the whole standup
kubectl -n petstore-clients exec deploy/fortio -c fortio -- \
  fortio load -c 4 -qps 25 -t 150s -quiet http://catalog.petstore/ > /tmp/fortio-ambient.txt &

helm upgrade istiod istiod \
  --repo https://istio-release.storage.googleapis.com/charts \
  -n istio-system --version 1.30.3 --wait --reuse-values \
  --set profile=ambient --set istio_cni.enabled=true

helm upgrade --install istio-cni cni \
  --repo https://istio-release.storage.googleapis.com/charts \
  -n istio-system --version 1.30.3 --wait \
  --set profile=ambient --set global.hub=docker.io/istio --set global.tag=1.30.3 \
  --set 'excludeNamespaces[0]=istio-system' --set 'excludeNamespaces[1]=kube-system'

helm upgrade --install ztunnel ztunnel \
  --repo https://istio-release.storage.googleapis.com/charts \
  -n istio-system --version 1.30.3 --wait \
  --set profile=ambient --set hub=docker.io/istio --set tag=1.30.3

wait   # for the fortio run to finish
bashverify
kubectl -n istio-system get ds
#   istio-cni-node   3/3
#   ztunnel          3/3     (one per node)
grep 'Code ' /tmp/fortio-ambient.txt
#   Code 200 : 3750 (100.0 %)   — zero dropped requests across the standup

the mesh is bilingual: sidecars still everywhere and still serving, ztunnel on every node with nothing enrolled, fortio at 100%.

STEP 6

Roll every app namespace once (HBONE re-injection)

Sidecars injected before the ambient profile lack ISTIO_META_ENABLE_HBONE, so ztunnel could only reach them in plaintext, which STRICT mTLS rejects. One roll, before anything enrols, and every re-injected sidecar can accept HBONE from ambient callers.

bashrun
for ns in petstore petstore-orders petstore-data petstore-clients; do
  kubectl -n $ns rollout restart deploy
  kubectl -n $ns get deploy -o name | xargs -I{} kubectl -n $ns rollout status {} --timeout=300s
done
bashverify
kubectl -n petstore get pod -l app=catalog -o jsonpath='{.items[0].spec.initContainers[?(@.name=="istio-proxy")].env[?(@.name=="ISTIO_META_ENABLE_HBONE")].value}'; echo
#   true

every sidecar advertises HBONE. This is the step people skip and then blame ambient for.

STEP 7

Migrate the L4-only namespace: one label, no waypoint

petstore-data has STRICT mTLS and an identity-based L4 policy, nothing L7. ztunnel enforces both, so the namespace migrates with the label flip alone and zero YAML changes.

bashrun
kubectl label namespace petstore-data istio.io/dataplane-mode=ambient istio-injection- --overwrite
kubectl -n petstore-data rollout restart deploy/redis
kubectl -n petstore-data rollout status deploy/redis --timeout=180s
bashverify
# no sidecar left (no istio-proxy init container either)
kubectl -n petstore-data get pod -l app=redis -o jsonpath='{.items[0].spec.containers[*].name} {.items[0].spec.initContainers[*].name}'; echo
#   redis

# the allowed identity still gets PONG — same policy, now enforced by ztunnel
kubectl -n petstore logs deploy/data-client --tail=1
#   PONG
# the denied identity still gets silence
kubectl -n petstore-clients exec deploy/checkout -c checkout -- sh -c 'printf "PING\r\n" | nc -w 3 redis.petstore-data 6379'
#   (no output — denied)

istioctl ztunnel-config workload | grep redis
#   petstore-data  redis-...   ...   ambient-oss-worker...   None   HBONE

redis on ambient with the identical selector-based L4 AuthorizationPolicy still enforcing: the PONG stream never blinks.

STEP 8

Modernise the canary: DR/VS subsets to HTTPRoute

Waypoints do not do subset routing, so the version split must become per-version Services + an HTTPRoute before catalog's callers go ambient. Do it now, while everything relevant is still on sidecars: sidecars apply Service-attached HTTPRoutes too, so this cutover is a traffic no-op you can verify.

bashrun
kubectl apply -f yaml/50-httproute/10-versioned-services.yaml
kubectl apply -f yaml/50-httproute/20-catalog-httproute.yaml
kubectl apply -f yaml/50-httproute/30-catalog-dr-nosubset.yaml
kubectl -n petstore delete virtualservice catalog
bashverify
# still 100% v1 — the HTTPRoute starts with the same weights the VS had
kubectl -n petstore-clients exec deploy/checkout -c checkout -- sh -c \
  'i=0; while [ $i -lt 30 ]; do curl -s http://catalog.petstore/; i=$((i+1)); done' | grep -c '"version":"v2"'
#   0
kubectl -n petstore-clients exec deploy/fortio -c fortio -- fortio load -c 4 -qps 25 -t 10s -quiet http://catalog.petstore/ | grep 'Code '
#   Code 200 : 250 (100.0 %)
yamlview the YAML · yaml/50-httproute/20-catalog-httproute.yaml
apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
  name: catalog
  namespace: petstore
spec:
  parentRefs:
    - group: ""
      kind: Service
      name: catalog
      port: 80
  rules:
    - backendRefs:
        - name: catalog-v1
          port: 80
          weight: 100
        - name: catalog-v2
          port: 80
          weight: 0

the split now lives in a Gateway API HTTPRoute attached to the catalog Service; traffic unchanged.

STEP 9

Deploy THE waypoint: one for the whole cluster

bashrun
kubectl apply -f yaml/30-waypoint/cluster-waypoint.yaml
kubectl -n mesh-infra wait --for=condition=Programmed gateway/cluster-waypoint --timeout=120s
kubectl -n mesh-infra rollout status deploy/cluster-waypoint --timeout=180s
bashverify
kubectl -n mesh-infra get gateway,pods
#   gateway.../cluster-waypoint   istio-waypoint   10.96.x.x   True (Programmed)
#   pod/cluster-waypoint-...      1/1   Running     ← ONE pod, the whole cluster's L7
yamlview the YAML · yaml/30-waypoint/cluster-waypoint.yaml
apiVersion: v1
kind: Namespace
metadata:
  name: mesh-infra
---
apiVersion: gateway.networking.k8s.io/v1
kind: Gateway
metadata:
  name: cluster-waypoint
  namespace: mesh-infra
  labels:
    istio.io/waypoint-for: service
spec:
  gatewayClassName: istio-waypoint
  listeners:
    - name: mesh
      port: 15008
      protocol: HBONE
      allowedRoutes:
        namespaces:
          from: All          # ← the line that makes it cluster-wide

a Programmed istio-waypoint Gateway and a single Envoy pod in mesh-infra. Nothing routes through it yet.

STEP 10

Migrate both L7 namespaces behind the shared waypoint, under load

Policy shape first (targetRefs in, selector out; ztunnel fail-safes if you skip this), then the label flips: use-waypoint + use-waypoint-namespace + dataplane-mode=ambient on both namespaces, and a roll.

bashrun
kubectl apply -f yaml/40-policies-waypoint/
kubectl -n petstore delete authorizationpolicy catalog-get-only
kubectl -n petstore-orders delete authorizationpolicy orders-get-only

kubectl -n petstore-clients exec deploy/fortio -c fortio -- \
  fortio load -c 4 -qps 25 -t 120s -quiet http://catalog.petstore/ > /tmp/fortio-waypoint.txt &

for ns in petstore petstore-orders; do
  kubectl label namespace $ns \
    istio.io/use-waypoint=cluster-waypoint \
    istio.io/use-waypoint-namespace=mesh-infra \
    istio.io/dataplane-mode=ambient istio-injection- --overwrite
  kubectl -n $ns rollout restart deploy
done
kubectl -n petstore get deploy -o name | xargs -I{} kubectl -n petstore rollout status {} --timeout=300s
kubectl -n petstore-orders rollout status deploy/orders --timeout=180s
wait
bashverify — one proxy, two namespaces
grep 'Code ' /tmp/fortio-waypoint.txt
#   Code 200 : 3000 (100.0 %)   — zero dropped requests across the enrolment

# ztunnel routes both namespaces' services via the ONE waypoint
istioctl ztunnel-config service | grep -E 'catalog|orders'
#   petstore         catalog      10.96.x.x   cluster-waypoint   4/4
#   petstore         catalog-v1   10.96.x.x   cluster-waypoint   2/2
#   petstore         catalog-v2   10.96.x.x   cluster-waypoint   2/2
#   petstore-orders  orders       10.96.x.x   cluster-waypoint   2/2

kubectl -n petstore-clients exec deploy/checkout -c checkout -- curl -s -o /dev/null -w '%{http_code}\n' http://catalog.petstore/
#   200
kubectl -n petstore-clients exec deploy/checkout -c checkout -- curl -s -o /dev/null -w '%{http_code}\n' http://orders.petstore-orders/
#   200
bashverify — the community mixed-fleet gap, live
# checkout still has a sidecar, and community sidecars do NOT route through waypoints.
# The GET-only rule now lives on the waypoint — so for THIS caller it no longer applies:
kubectl -n petstore-clients exec deploy/checkout -c checkout -- curl -s -o /dev/null -w '%{http_code}\n' -X DELETE http://catalog.petstore/
#   200   ← was 403 in STEP 4. The policy did not break; this caller bypasses its enforcement point.

both L7 namespaces served by the single waypoint at 100%, and the honest OSS caveat on display: an unmigrated caller is outside the waypoint's law until its own namespace flips (next step). On the Solo distribution, sidecar and ingress callers are routed through the waypoint instead, which is what the Enterprise edition shows.

STEP 11

Migrate the calling estate: the waypoint's policy now applies to it

bashrun
kubectl label namespace petstore-clients istio.io/dataplane-mode=ambient istio-injection- --overwrite
kubectl -n petstore-clients rollout restart deploy
kubectl -n petstore-clients rollout status deploy/checkout deploy/fortio --timeout=180s
bashverify
kubectl -n petstore-clients exec deploy/checkout -c checkout -- curl -s -o /dev/null -w '%{http_code}\n' -X DELETE http://catalog.petstore/
#   403   ← the gap is closed: ambient callers route through the waypoint
kubectl -n petstore-clients exec deploy/checkout -c checkout -- curl -s -o /dev/null -w '%{http_code}\n' -X DELETE http://orders.petstore-orders/
#   403   ← same single waypoint enforcing the second namespace's rule
kubectl -n petstore-clients exec deploy/fortio -c fortio -- fortio load -c 4 -qps 25 -t 10s -quiet http://catalog.petstore/ | grep 'Code '
#   Code 200 : 250 (100.0 %)

every namespace ambient, every L7 rule enforced again, by one proxy.

STEP 12

Shift the canary live at the waypoint

The HTTPRoute weights are now enforced by the waypoint. Shift them with a patch, watch v2 take half the traffic, then pin back.

bashrun
kubectl -n petstore patch httproute catalog --type=json -p='[
  {"op":"replace","path":"/spec/rules/0/backendRefs/0/weight","value":50},
  {"op":"replace","path":"/spec/rules/0/backendRefs/1/weight","value":50}
]'
bashverify
kubectl -n petstore-clients exec deploy/checkout -c checkout -- sh -c \
  'i=0; while [ $i -lt 30 ]; do curl -s http://catalog.petstore/; i=$((i+1)); done' | grep -c '"version":"v2"'
#   ~15   (half of 30 — the live 50/50 split)

# pin back to v1 before the rollback step
kubectl -n petstore patch httproute catalog --type=json -p='[
  {"op":"replace","path":"/spec/rules/0/backendRefs/0/weight","value":100},
  {"op":"replace","path":"/spec/rules/0/backendRefs/1/weight","value":0}
]'

a zero-downtime weight shift, done by the same knob Argo Rollouts would drive.

STEP 13

Roll back a namespace, and forward again

The switch has to work in both directions before you trust it. petstore-orders goes back to sidecars: labels flipped back, one roll, selector policy restored after the pods are sidecars again (never before; field note 7). Then forward a second time.

bashrun — back to sidecars
kubectl label namespace petstore-orders istio-injection=enabled \
  istio.io/dataplane-mode- istio.io/use-waypoint- istio.io/use-waypoint-namespace- --overwrite
kubectl -n petstore-orders rollout restart deploy/orders
kubectl -n petstore-orders rollout status deploy/orders --timeout=180s

# pods are sidecars again → restore the selector policy, retire the targetRefs one
kubectl apply -f yaml/20-policies-sidecar/40-l7-authz-orders.yaml
kubectl -n petstore-orders delete authorizationpolicy orders-get-only-waypoint
bashverify
kubectl -n petstore-orders get pod -l app=orders -o jsonpath='{.items[0].spec.initContainers[*].name}'; echo
#   istio-init istio-proxy      ← the sidecar is back
kubectl -n petstore-clients exec deploy/checkout -c checkout -- curl -s -o /dev/null -w '%{http_code}\n' -X DELETE http://orders.petstore-orders/
#   403   ← enforced by the sidecar again
bashrun — and forward again
kubectl apply -f yaml/40-policies-waypoint/20-orders-authz-targetref.yaml
kubectl -n petstore-orders delete authorizationpolicy orders-get-only
kubectl label namespace petstore-orders \
  istio.io/use-waypoint=cluster-waypoint \
  istio.io/use-waypoint-namespace=mesh-infra \
  istio.io/dataplane-mode=ambient istio-injection- --overwrite
kubectl -n petstore-orders rollout restart deploy/orders
kubectl -n petstore-orders rollout status deploy/orders --timeout=180s

rollback and re-enrolment are the same routine label flip. fortio against catalog reads 100% throughout: the other namespaces never notice.

STEP 14

The end state, then teardown

bashverify
# zero sidecars anywhere in the app namespaces
for ns in petstore petstore-orders petstore-data petstore-clients; do
  kubectl -n $ns get pods -o jsonpath='{range .items[*]}{.spec.initContainers[*].name}{" "}{end}'
done | tr ' ' '\n' | grep -c istio-proxy
#   0

# exactly ONE waypoint pod for the whole cluster; ztunnel on every node
kubectl get pods -A -l gateway.networking.k8s.io/gateway-name
#   mesh-infra   cluster-waypoint-...   1/1   Running      (just the one)
kubectl -n istio-system get ds ztunnel
#   ztunnel   3/3
bashteardown
kind delete cluster --name ambient-oss

the diagram at the top of this page, live: one waypoint, three ztunnels, no sidecars.

Migration checklist

The lab, compressed into the checks you would run against a real cluster.

  1. Before anything migrates
  2. Classify every namespace L4 or L7. Anything that reads HTTP content (methods, paths, headers, VS routing, JWT) is L7 and needs a waypoint attachment before enrolment; identity/port-only policy is L4 and needs nothing.
  3. Audit PeerAuthentication for mode: DISABLE: no ambient equivalent; retire the carve-outs first.
  4. If the mesh CA is external (cert-manager istio-csr, Vault), check it will sign ztunnel. ztunnel only generates ECDSA P-256 keys, so a signing role locked to RSA rejects every ambient certificate, and istio-csr needs app.server.caTrustedNodeAccounts for the ambient trust model. Sidecars to Ambient When Your PKI Is RSA rehearses exactly that failure and the fix.
  5. Find DR subset routing and plan its move to per-version Services + HTTPRoute; do the cutover while still on sidecars, where it is a no-op.
  6. Decide the waypoint topology: shared cluster-wide, per-namespace, or a mix. Shared needs allowedRoutes: namespaces.from: All (or a Selector) on the Gateway.
  7. The switch
  8. Enable ambient with helm (istiod profile=ambient, cni, ztunnel) under load; sidecars are untouched.
  9. Roll every sidecar namespace once so re-injected sidecars advertise HBONE, before any neighbour enrols.
  10. Per namespace, in order
  11. L4 namespaces first: label flip + roll; verify the same policies enforce (allowed identity still served, denied identity still silent).
  12. L7 namespaces: convert selector policies to targetRefs, attach the namespace to its waypoint, then flip the label and roll.
  13. Migrate callers in the same window as L7-protected services: on community images a sidecar caller bypasses the waypoint, so its policy view lapses until its namespace flips.
  14. Keep a load generator running and read it after every change. It should say 100%; if it does not, stop and find out why before you touch the next namespace.
  15. Safety net
  16. Rehearse the rollback: labels back, roll, selector policy restored after the pods are sidecars again. If you have not rehearsed it, you do not have it.

How Enterprise Istio helps with this migration

Everything above runs on community Istio, and for a small estate that can migrate all of a service's callers in one window, it is enough. The places a real fleet feels the difference are the mixed period, the audit, the lifecycle and the visibility while you cut. This is what Solo Enterprise for Istio adds to each stage of the exact migration you just ran:

See also

Versions

Built and verified on:

OSS
Gateway APIv1.5.1
Istio (upstream)1.30.3
Kubernetes (kind)1.35