The short version. For pure north-south traffic — TLS termination, JWT/OIDC, rate limiting, transformation — kgateway is the right landing spot when you're moving off NGINX Ingress or feel like Istio-Ingress-Gateway-as-edge is more mesh than you want to run. It's pure Envoy under a lightweight dedicated control plane, native Gateway API, and the same enterprise auth / rate-limit servers Solo customers already run with Istio plug in unchanged. When east-west service-to-service authz, mTLS and observability matter, layer Istio Ambient in underneath — same Gateway API resources keep working, no sidecars get injected.
Three options at a glance
The decision usually framed as "kgateway vs Istio GW" is really three options once you include where most teams are starting from.
| Criterion | NGINX Ingress current |
Istio Ingress Gateway mesh entry |
kgateway target |
|---|---|---|---|
| Primary role | Legacy ingress controller | Service-mesh entry point | Autonomous API gateway & next-gen ingress |
| Gateway API support | Partial / experimental | Yes, but mesh-oriented | Native & complete (core feature) |
| Data plane | NGINX | Envoy | Envoy |
| Control plane footprint | Low (but high technical debt) | Higher — istiod + sidecars or ztunnels |
Moderate — dedicated lightweight CP, no sidecars |
| Config update model | NGINX reload — slow under CI churn | xDS — sub-second, no restart | xDS — sub-second, no restart |
| Routing dimensions | Annotations (per-vendor) | Gateway API + Istio CRDs | Gateway API + delegated routes, transformations, ext-proc |
| JWT / OIDC at the edge | External (ingress-annotation hacks) | RequestAuthentication + AuthorizationPolicy |
OSS GatewayExtension (type: JWT) for validation, plus enterprise AuthConfig (extauth.solo.io) for OIDC code-flow, claims-to-headers, OPA — attached via EnterpriseKgatewayTrafficPolicy |
| Rate limiting | Per-IP per-instance only | EnvoyFilter + external rate-limit service (OSS + Solo enterprise) |
First-class RateLimitConfig + EnterpriseKgatewayTrafficPolicy.entRateLimit |
| AI / LLM traffic | None | Via Wasm (complex) | Not native — agentgateway is Solo's AI gateway product (prompt guards, token-based rate limits, multi-provider routing, model failover). Deployable alongside kgateway and shares the enterprise rate-limit server. |
| Non-K8s upstreams (Lambda, S3, VMs) | Not supported | Possible via ServiceEntry | Native backend types |
| Service mesh integration | None | Native (Istio) | Transparent integration — ambient waypoint on the gateway namespace |
| Governance | Devs can overwrite TLS / LB in one Ingress | Mesh-wide policy CRDs | Role-split via Gateway API: platform owns Gateway, dev owns HTTPRoute |
| Vendor governance | NGINX commercial cadence | CNCF Graduated (Istio) | CNCF Sandbox · originally Solo.io · kgateway-dev/kgateway |
Why move off NGINX Ingress
The case for leaving the Ingress API isn't ideological — it's operational. Same list I've heard from every customer serious about platform engineering.
Concrete pain points from production
- Annotation hell. Every controller (NGINX, HAProxy, Traefik) invented its own annotations to compensate for missing features — traffic splitting, rewrites, header manipulation. Manifests stop being portable; switching controllers means rewriting everything.
- Broken governance. An
Ingressbundles routes, TLS and LB config into a single resource. Devs can — and do — overwrite production TLS by accident. Splitting responsibility cleanly across teams is impossible. - Protocol limitations. HTTP/HTTPS only as first-class. TCP / UDP exposure is a global ConfigMap hack — one tenant's mistake can break the whole controller.
- Large blast radius. A bad Lua annotation crashes the controller cluster-wide.
- Slow propagation. NGINX reloads cope badly with CI/CD churn. Modern deploy cadences expose this on the daily.
The Gateway API fixes this structurally: GatewayClass
picked by the platform team, Gateway owned by infra
(ports, TLS, the network contract), and HTTPRoute /
TLSRoute / TCPRoute owned by dev teams,
attached explicitly via parentRefs with
allowedRoutes filtering by namespace or label. Bad
config in one route can't affect another. Non-HTTP protocols are
first-class. Propagation is xDS, not reload.
kgateway vs Istio Ingress Gateway
Both are Envoy under the hood. Both implement the Gateway API. The difference is what's around the proxy and what each is optimised for.
Istio Ingress Gateway is one component of the Istio
control plane. It's tuned for getting traffic into the mesh
and inherits the same istiod dependency, the same
CRDs, the same mental model. If you already run Istio end-to-end
for east-west, an Istio Ingress Gateway gives you one config
surface and one set of authz / observability primitives top to
bottom — the trade-off is you're running and maintaining
istiod + the mesh data plane (ambient or sidecars)
even if north-south is all you need today.
kgateway is a standalone API gateway with its own
lightweight Envoy control plane. What that buys you operationally:
sub-second config propagation via xDS (no NGINX
reload), config changes don't restart the proxy,
and the gateway doesn't depend on istiod being
healthy.
Pros / cons of "no istiod on the path":
Pro — fewer moving parts, a gateway outage doesn't drag the mesh
control plane and vice-versa, easier blast-radius story.
Con — if you do eventually adopt the mesh, you've got two control
planes (kgateway controller + istiod) to operate, and
policy at the edge and inside the mesh is configured in different
CRD surfaces. The kgateway + ambient model below makes that
coexistence cheaper by removing sidecars, but it doesn't
consolidate the two control planes.
Honest framing before the table. Most of the
capabilities people associate with kgateway (transformations,
ext-proc, JWT validation, rate limiting) are Envoy HTTP
filters. Istio Ingress Gateway is Envoy too, so it can do all
of them. The real kgateway-vs-Istio-GW difference on those rows is
how you configure them: kgateway exposes them as
fields on a TrafficPolicy CRD; Istio expresses the
same filters via EnvoyFilter (raw Envoy config that
Istio's control plane merges in). Same data plane, same filter
binaries, different YAML surface. A few items below are
more genuinely kgateway-shaped — route delegation, the first-class
Backend CRD for non-K8s targets.
Transformations parity, diff ergonomics
Modify request / response bodies and headers in Envoy without
writing code (Lua, or Solo's "Rustformation" engine).
kgateway: first-class
TrafficPolicy.transformation /
rustformations.
Istio Ingress GW: the same Envoy transformation
filter, wired in via EnvoyFilter.
ext-proc parity, diff ergonomics
What it actually is. Envoy receives a
request. Before forwarding it, Envoy makes a gRPC call to
your service: "here are the headers, here's the body —
what should I do?". Your service can rewrite headers, replace the
body, return a 4xx directly, or just pass through. Envoy then
continues. Same hooks exist on the response path. Concrete
example: a custom Go service that scans request bodies for card
numbers and masks the PAN field before the request leaves the
cluster.
Both gateways can do it. It's the Envoy filter
envoy.filters.http.ext_proc.
kgateway: CRD field —
TrafficPolicy.extProc +
GatewayExtension pointing at your service.
Istio Ingress GW: an EnvoyFilter
resource injecting the same Envoy filter and pointing at the
same service. Same capability, different YAML surface.
Route delegation kgateway-shaped
Split a large routing config across team-owned
HTTPRoutes that delegate from a parent. Built into
kgateway's controller. Istio doesn't have a direct equivalent —
the closest is stacking multiple HTTPRoutes with
parentRefs, without the explicit delegation
semantics.
Non-K8s backends kgateway-shaped
kgateway ships a first-class Backend CRD
(gateway.kgateway.dev) with types for AWS Lambda,
static IPs, AI providers, MCP and A2A. Istio routes to non-K8s
via ServiceEntry — possible, but the Lambda /
cloud-native-target path needs more glue.
For AI / LLM traffic, kgateway is not the AI gateway.
That's agentgateway — a separate Solo product
with its own gatewayClassName: enterprise-agentgateway
and its own CRDs (AgentgatewayBackend,
EnterpriseAgentgatewayPolicy). It deploys as a
dedicated AI-traffic Gateway, as an egress gateway for in-mesh
workloads calling external LLM providers, or as a Gateway in
Ambient that services target via the
istio.io/use-waypoint label (full agentgateway-as-
waypoint, replacing the default Envoy waypoint for HBONE
traffic, is in development with Solo + Microsoft). Capabilities:
prompt guards, token-based rate limits, multi-provider routing,
model failover.
Benchmark notes — howardjohn/gateway-api-bench (independent)
Methodology: single-node kind cluster on a 16-core AMD 9950x with 96 GB RAM, Gateway API v1.3.0. Versions tested: Kgateway v2.0.1, Istio v1.26.0, Envoy Gateway v1.4.0, Cilium v1.17.2, Kong v3.9, Traefik v35.3.0, NGINX v1.6.2. Bias disclosed by the author — he and his employer have contributed to Istio, Envoy, Kgateway and Cilium. Treat numbers as relative signal on this hardware, not absolute production figures.
- Data-plane throughput. At 512 connections, unlimited QPS, kgateway = 400,687 qps vs Istio = 328,169 qps — the author summarises this as kgateway achieving roughly 25% more throughput than Istio and Envoy Gateway at high connection counts. NGINX trails at 91,580 qps.
- P50 latency, 16 connections @ 100k QPS. kgateway 0.046 ms, Istio 0.097 ms, NGINX 0.186 ms.
- Control-plane efficiency under route churn (50 ns × 100 routes = 5,000 routes). Istio is the leader. Relative-to-best ratios from the report: Istio 1.0× CPU / 1.5× memory, kgateway 6.6× CPU / 6.2× memory. Read kgateway-vs-Istio: ~6.6× the CP CPU and ~4.1× the CP memory at the same churn.
- Functional reliability. Both pass all functional tests — no traffic loss during route changes, no scaling failures.
Cite responsibly — single-node test rig, the author is an active contributor to several of the projects, and "v2" of the report tests newer versions. Use as one input, not the deciding factor.
kgateway policy surface — OSS vs Enterprise TrafficPolicy
Worth being precise about what's in OSS kgateway and what the enterprise distribution adds — it's a clean superset, not a fork. Same CRD shape, two extra fields.
TrafficPolicy
gateway.kgateway.dev
Standard policy CRD in OSS kgateway. Attach to a
Gateway, HTTPRoute, or
Backend via targetRefs. Covers the
common Envoy filter set:
- cors
- csrf
- retry
- timeouts
- transformation
- rustformations
- headerModifiers
- buffer
- autoHostRewrite
- hashPolicies
- extAuth
- extProc
- rbac
- rateLimit (basic)
EnterpriseKgatewayTrafficPolicy
enterprisekgateway.solo.ioTrafficPolicy spec verbatimentExtAuth
→ AuthConfig · extauth.solo.io
OAuth2 / OIDC / JWT path — claimsToHeaders,
OIDC code-flow login, OPA, API keys, basic auth.
entRateLimit
→ RateLimitConfig · ratelimit.solo.io
Global, CEL-based, multi-descriptor rate limits backed by the
Solo enterprise rate-limit server. OSS rateLimit
covers local / single-descriptor; for per-tenant + per-endpoint
at fleet scale you want entRateLimit.
AuthConfig (extauth.solo.io)
and RateLimitConfig (ratelimit.solo.io).
All ship in the enterprise-kgateway-crds Helm chart.
Worked example — per-tenant + per-endpoint with entRateLimit
Two CRDs: a RateLimitConfig with the descriptor tree,
and an EnterpriseKgatewayTrafficPolicy that attaches it
to the Gateway and tells the rate-limit filter which request headers
to read for each descriptor key. The outer descriptor caps the whole
tenant at 1,000 req/min; the inner one caps a specific expensive
endpoint within that tenant at 10 req/min.
RateLimitConfig · descriptor tree
what's being limited and the limits per leaf
apiVersion: ratelimit.solo.io/v1alpha1
kind: RateLimitConfig
metadata:
name: per-tenant-per-path
namespace: kgateway-system
spec:
raw:
descriptors:
- key: tenant-id # outer: per-tenant ceiling
rateLimit:
unit: MINUTE
requestsPerUnit: 1000
descriptors:
- key: path # inner: per-endpoint within tenant
value: "/api/v1/expensive"
rateLimit:
unit: MINUTE
requestsPerUnit: 10
# how request fields map into descriptor keys
rateLimits:
- actions:
- requestHeaders: { descriptorKey: tenant-id, headerName: x-tenant-id }
- actions:
- requestHeaders: { descriptorKey: tenant-id, headerName: x-tenant-id }
- requestHeaders: { descriptorKey: path, headerName: ":path" }
EnterpriseKgatewayTrafficPolicy · attaches it to the Gateway
the enterprise field on the TrafficPolicy wrapper
apiVersion: enterprisekgateway.solo.io/v1alpha1
kind: EnterpriseKgatewayTrafficPolicy
metadata:
name: per-tenant
namespace: kgateway-system
spec:
targetRefs:
- { group: gateway.networking.k8s.io, kind: Gateway, name: http }
entRateLimit:
global:
rateLimitConfigRefs:
- { name: per-tenant-per-path }
How the tenant gets identified.
x-tenant-id doesn't come from the client — it's written
by the gateway from a verified JWT claim via
claimsToHeaders on the AuthConfig attached
through entExtAuth. So the order on every request is:
validate JWT → write x-tenant-id from the
tenantId claim → rate-limit filter reads
x-tenant-id from the request and keys descriptors off
it. See
the claims-to-headers KB
for the full flow.
An optional second layer — kgateway edge with Ambient east-west
One of the architectural options worth knowing about: kgateway at the cluster edge can be paired with Istio Ambient inside the cluster, when east-west requirements (service-to-service mTLS, fine-grained workload authorization, per-call traces between microservices) eventually come up. This isn't a recommendation — plenty of customers run kgateway alone and don't need the mesh. It's a note that the option exists, what it looks like, and the fact that adopting it later doesn't force a rewrite of the Gateway API resources you already have.
How it works: kgateway terminates external TLS, applies edge L7
policies (JWT validation, rate limits, transformations), and
forwards to backends. Enrol the gateway's namespace in ambient
(kubectl label ns <ns> istio.io/dataplane-mode=ambient)
and that forwarding hop becomes HBONE / mTLS via
each destination node's ztunnel. East-west calls
between workloads pick up the same mTLS + L4 authz from
ztunnel, and any L7 policy that needs to live in the
mesh (RequestAuthentication, fine-grained AuthorizationPolicy,
tracing) lives on a waypoint. No sidecar
injection. Same Gateway API resources keep working.
What you trade for it. You're now running two
control planes — kgateway's, and istiod for the
mesh — and edge policy vs in-mesh policy use different CRDs.
Worth doing when you actually need east-west capabilities, not
worth doing pre-emptively.
How to read it. External clients hit kgateway at
the edge with HTTPS + a JWT. kgateway terminates TLS, validates
the token against the IdP's JWKS, extracts the tenant claim into
a header, enforces the rate limit, and forwards to the in-cluster
backend. The moment the gateway's namespace is enrolled in
ambient (istio.io/dataplane-mode=ambient), that
forwarding hop becomes HBONE / mTLS via the
destination node's ztunnel, and any L7 policy you
want enforced inside the mesh sits on a waypoint. East-west
service-to-service calls between backend-api and
orders-svc get the same mTLS + authz without any
sidecar injection. You can adopt kgateway today and ambient
next quarter — same Gateway API resources work in both worlds.
Recommendation
If you're moving off NGINX Ingress today
TrafficPolicy.jwt + RateLimitConfig covers external OIDC IdPs and per-tenant/per-endpoint limits out of the box.Pricing & sizing
Per-cluster pricing is the right unit for this profile. Your workload CPU varies too fast to be a stable input, and per-cluster aligns with the fleet-management model you'll be running anyway (gloo-operator, Gloo Platform workspaces, per-cluster RBAC).
A specific figure belongs in a signed quote, not a public page. Your Solo AM will scope it from concurrent-live-cluster count, support tier, and the enterprise feature set you actually need (rate-limit server, ext-auth-service, the management plane, the UI, etc.). Solo Professional Services can be scoped alongside.
Talk to your account team for a quote — I'll wire you up with the right contact.
References
Background reading cited above
- howardjohn/gateway-api-bench — independent Gateway API benchmark across kgateway, Istio, Envoy Gateway, Cilium, Kong, Traefik, NGINX. Throughput, latency, control-plane scale.
- Solo blog: kgateway as ingress + Ambient as mesh — Solo's architectural framing of the two-layer model. Includes the namespace-enrolment walkthrough.
Adjacent demos on this site
- JWT, OIDC and on-behalf-of — the deep dive on auth mechanics that complements the JWT section above.
- Trust & identity — RootTrustPolicy to mTLS — the SPIFFE / mTLS foundation that ambient relies on.
- Gloo Operator — Istio Ambient across N clusters — the fleet-management piece for "500 concurrent clusters".
- Gateway API on Istio Ambient — what attaches to what — once you've decided to adopt ambient, this maps every Gateway API kind to where it lives.