01Recipes
Twenty tested copy-paste patterns, grouped by intent. Click a card to expand. Filter by category, or hit Expand all to scan everything at once.
1.1 · New HTTPRoute on an existing hostname3 fields, longest-prefix match, mind the sectionName.
sectionName.apiVersion: gateway.networking.k8s.io/v1 kind: HTTPRoute metadata: { name: my-app, namespace: demo } spec: parentRefs: - name: eg namespace: default sectionName: https-gateway # pin to one listener! hostnames: [ "gateway.wolfslight.cc" ] rules: - matches: - path: { type: PathPrefix, value: /my-app } backendRefs: - name: my-app port: 80
kubectl -n demo get httproute my-app -o jsonpath='{.status.parents[0].conditions[?(@.type=="Accepted")].status}' → TruesectionName the route attaches to EVERY compatible listener — including http :80 — and races the HTTP→HTTPS redirect by longest-prefix-match.
1.2 · Canary / weighted traffic splitTwo backends behind one rule. Weights are integers, summed any number.
rules: - matches: - path: { type: PathPrefix, value: /api } backendRefs: - { name: api-v1, port: 80, weight: 90 } - { name: api-v2, port: 80, weight: 10 } # 10% canary
for i in $(seq 1 100); do curl -s https://gateway.wolfslight.cc/api/ -H "X-Hello: 1" | jq -r .pod; done | sort | uniq -cx-canary: true for sticky canary.
1.3 · URLRewrite — strip a path prefixFrontend says /api/users/42, backend sees /users/42.
/api/users/42, backend sees /users/42.rules: - matches: - path: { type: PathPrefix, value: /api } filters: - type: URLRewrite urlRewrite: path: type: ReplacePrefixMatch replacePrefixMatch: / # strip /api backendRefs: - { name: api-svc, port: 80 }
curl https://gateway.wolfslight.cc/api/users/42 → backend's /users/42 handler fires.ReplaceFullPath rewrites to a fixed path (no captures). For Envoy's regex-rewrite use EnvoyExtensionPolicy with a Lua filter.
1.4 · Request redirect (301 / 302)Move a path, force HTTPS, or send a deprecated endpoint to its new home.
rules: - matches: - path: { type: PathPrefix, value: /old } filters: - type: RequestRedirect requestRedirect: scheme: https hostname: new.wolfslight.cc path: { type: ReplacePrefixMatch, replacePrefixMatch: /v2 } statusCode: 301
curl -I https://gateway.wolfslight.cc/old/x → 301 with Location: https://new.wolfslight.cc/v2/xbackendRefs — they're mutually exclusive per Gateway-API spec.
1.5 · Request mirror (fire-and-forget shadow traffic)Send a copy of every request to a staging backend without affecting the response.
rules: - matches: - path: { type: PathPrefix, value: /api } filters: - type: RequestMirror requestMirror: backendRef: { name: api-staging, port: 80 } backendRefs: - { name: api-prod, port: 80 }
1.6 · Cross-namespace backend (ReferenceGrant)Your Route in demo points to a Service in lb-demo. Spec says no without a grant.
demo points to a Service in lb-demo. Spec says no without a grant.# Lives in the TARGET namespace (the backend's namespace) apiVersion: gateway.networking.k8s.io/v1beta1 kind: ReferenceGrant metadata: { name: allow-from-demo, namespace: lb-demo } spec: from: - group: gateway.networking.k8s.io kind: HTTPRoute namespace: demo to: - group: "" kind: Service
ResolvedRefs=False, RefNotPermitted flips to True within seconds.
2.1 · Add a brand-new hostname (TLS + DNS + listener)Four moving parts; external-dns + cert-manager do most of it automatically.
eg (manifests/20-gateway.yaml):- name: https-my-app protocol: HTTPS port: 443 hostname: my-app.wolfslight.cc tls: mode: Terminate certificateRefs: - name: my-app-wolfslight-tls allowedRoutes: { namespaces: { from: All } }
apiVersion: cert-manager.io/v1 kind: Certificate metadata: { name: my-app-wolfslight-tls, namespace: default } spec: secretName: my-app-wolfslight-tls issuerRef: { name: letsencrypt-prod, kind: ClusterIssuer } dnsNames: [ my-app.wolfslight.cc ]
sectionName: https-my-app + hostname.kubectl get certificate my-app-wolfslight-tls -n default → READY=True (DNS-01 takes 3-5 min)_acme-challenge records, but only safely on distinct names.
2.2 · Enforce TLS 1.2+ and AEAD ciphersClientTrafficPolicy attaches to the Gateway, applies to every HTTPS listener.
apiVersion: gateway.envoyproxy.io/v1alpha1 kind: ClientTrafficPolicy metadata: { name: tls-min-1-2, namespace: default } spec: targetRefs: - group: gateway.networking.k8s.io kind: Gateway name: eg tls: minVersion: "1.2" ciphers: - ECDHE-ECDSA-AES256-GCM-SHA384 - ECDHE-RSA-AES256-GCM-SHA384 - ECDHE-ECDSA-CHACHA20-POLY1305 - ECDHE-RSA-CHACHA20-POLY1305
nmap --script ssl-enum-ciphers -p 443 gateway.wolfslight.cc → only TLSv1.2 + TLSv1.3 visible, all AEAD.
3.1 · CORS allowlistSame CRD for HTTPRoute and GRPCRoute — kind-agnostic.
apiVersion: gateway.envoyproxy.io/v1alpha1 kind: SecurityPolicy metadata: { name: cors-my-app, namespace: demo } spec: targetRefs: - group: gateway.networking.k8s.io kind: HTTPRoute # or GRPCRoute name: my-app cors: allowOrigins: [ "https://gateway.wolfslight.cc" ] allowMethods: [ GET, POST, OPTIONS ] allowHeaders: [ "Content-Type", "Authorization" ] exposeHeaders: [ "x-request-id" ] maxAge: "1h"
curl -i -X OPTIONS https://my-app.wolfslight.cc/ -H 'Origin: https://gateway.wolfslight.cc' -H 'Access-Control-Request-Method: POST' → 200 + access-control-allow-origin header.
3.2 · Basic Auth on a pathSecret with htpasswd file; SecurityPolicy points to it.
# Create secret htpasswd -nbB admin "PleaseChangeMe" | \ kubectl -n demo create secret generic my-app-basic-auth \ --from-file=.htpasswd=/dev/stdin --dry-run=client -o yaml | \ kubectl apply -f - # Attach policy spec: targetRefs: - { group: gateway.networking.k8s.io, kind: HTTPRoute, name: my-app } basicAuth: users: name: my-app-basic-auth # Secret name
curl -i https://my-app.wolfslight.cc/ → 401 · curl -u admin:PleaseChangeMe ... → 200.
3.3 · JWT verification (OIDC providers, Auth0, Keycloak)Envoy fetches the JWKS, verifies the signature, exposes claims as headers.
spec: targetRefs: - { group: gateway.networking.k8s.io, kind: HTTPRoute, name: my-api } jwt: providers: - name: auth0 issuer: "https://my-tenant.auth0.com/" audiences: [ "https://api.wolfslight.cc" ] remoteJWKS: uri: "https://my-tenant.auth0.com/.well-known/jwks.json" claimToHeaders: - { claim: sub, header: x-user-id } - { claim: email, header: x-user-email }
curl -i https://my-api.wolfslight.cc/private → 401 · with Authorization: Bearer <valid JWT> → 200, backend sees x-user-id header.jwks.cacheDuration overrides.
3.4 · IP allowlist / denylistSecurityPolicy with authorization.rules. CIDR-based.
authorization.rules. CIDR-based.spec: targetRefs: - { group: gateway.networking.k8s.io, kind: HTTPRoute, name: admin-panel } authorization: defaultAction: Deny # everything else → 403 rules: - name: office-cidr action: Allow principal: clientCIDRs: [ "203.0.113.0/24", "198.51.100.42/32" ]
externalTrafficPolicy: Local (EG default). With Cluster mode you see the node's IP — useless for filtering.
3.5 · Inject security headers (HSTS, CSP, X-Frame-Options)ResponseHeaderModifier filter on the HTTPRoute — no policy needed.
rules: - filters: - type: ResponseHeaderModifier responseHeaderModifier: add: - { name: Strict-Transport-Security, value: "max-age=31536000; includeSubDomains; preload" } - { name: X-Frame-Options, value: DENY } - { name: X-Content-Type-Options, value: nosniff } - { name: Referrer-Policy, value: "strict-origin-when-cross-origin" } - { name: Content-Security-Policy, value: "default-src 'self'" } backendRefs: - { name: my-app, port: 80 }
·, →, em-dashes etc. — Envoy rejects with cryptic error.
4.1 · Per-IP rate limit (no Redis needed)BackendTrafficPolicy with type: Local — in-process per Envoy replica.
type: Local — in-process per Envoy replica.spec: targetRefs: - { group: gateway.networking.k8s.io, kind: HTTPRoute, name: my-app } rateLimit: type: Local local: rules: - clientSelectors: - sourceCIDR: { value: "0.0.0.0/0", type: Distinct } limit: { requests: 10, unit: Second }
for i in $(seq 1 20); do curl -s -o /dev/null -w "%{http_code}\n" https://gateway.wolfslight.cc/my-app/; done | sort | uniq -c → 10× 200, 10× 429.Local is per Envoy replica. 2 pods × 10rps = 20rps total budget. For cluster-wide use Global + Redis.
4.2 · Timeouts and retriesPer-route. Defaults are too generous for most APIs (15s + 1 retry = 30s worst-case).
spec: targetRefs: - { group: gateway.networking.k8s.io, kind: HTTPRoute, name: my-api } timeout: http: requestTimeout: 5s connectionIdleTimeout: 30s retry: numRetries: 2 perRetry: timeout: 2s backOff: { baseInterval: 100ms, maxInterval: 1s } retryOn: triggers: [ 5xx, reset, connect-failure ]
numRetries: 0 or restrict to retriable-status-codes like 503-only.
4.3 · Circuit breaker (cap pending requests + connections)Saves a slow backend from being asphyxiated by retries.
spec: targetRefs: - { group: "", kind: Service, name: flaky-backend } circuitBreaker: maxConnections: 100 maxPendingRequests: 50 maxParallelRequests: 200 maxParallelRetries: 5
circuit_breakers.default.cx_open ticks up — Envoy rejects with 503 LO instead of overloading backend.
5.1 · gRPC service (HTTP/2 upstream)Service announces h2c via appProtocol, GRPCRoute attaches to https-grpc.
appProtocol, GRPCRoute attaches to https-grpc.apiVersion: v1 kind: Service metadata: { name: my-grpc, namespace: demo } spec: selector: { app: my-grpc } ports: - name: grpc port: 9000 appProtocol: kubernetes.io/h2c # the magic — HTTP/2 upstream --- apiVersion: gateway.networking.k8s.io/v1 kind: GRPCRoute metadata: { name: my-grpc, namespace: demo } spec: parentRefs: - { name: eg, namespace: default, sectionName: https-grpc } hostnames: [ "grpc.wolfslight.cc" ] rules: - backendRefs: - { name: my-grpc, port: 9000 }
grpcurl grpc.wolfslight.cc:443 list — your services appear.appProtocol: kubernetes.io/h2c Envoy falls back to HTTP/1.1 upstream → gRPC returns UNKNOWN. Browser calls? Use application/grpc-web-text — fetch can't read HTTP/2 trailers.
5.2 · Raw TCP service (Redis, Postgres, MQTT)Non-HTTP listener + TCPRoute. Port-based, no hostname.
eg:- name: tcp-pg protocol: TCP port: 5432 allowedRoutes: kinds: [ { kind: TCPRoute } ] namespaces: { from: All }
apiVersion: gateway.networking.k8s.io/v1alpha2 kind: TCPRoute metadata: { name: pg, namespace: demo } spec: parentRefs: - { name: eg, namespace: default, sectionName: tcp-pg } rules: - backendRefs: - { name: postgres, port: 5432 }
hostnames — port-based only. The ELB needs the external port open via the Envoy data-plane Service. Experimental CRDs installed by scripts/10-install-crds.sh.
6.1 · Provision a standalone ELB (outside Gateway API)Plain Service: LoadBalancer with CCE annotations. New ELB or attach to existing one.
Service: LoadBalancer with CCE annotations. New ELB or attach to existing one.# A) Autocreate new ELB + EIP metadata: annotations: kubernetes.io/elb.class: union kubernetes.io/elb.autocreate: | { "type": "public", "name": "k8s-my-lb", "bandwidth_name": "k8s-my-lb-bw", "bandwidth_chargemode": "traffic", "bandwidth_size": 5, "bandwidth_sharetype": "PER", "eip_type": "5_bgp" } spec: type: LoadBalancer # B) Reuse an existing ELB (saves cost + EIP) metadata: annotations: kubernetes.io/elb.class: union kubernetes.io/elb.id: "<existing-elb-id>"
autocreate JSON without bandwidth_name — webhook error is cryptic. Same field is mandatory for the Envoy Gateway data-plane Service.
6.2 · Inject custom request headers (Trace IDs, internal markers)RequestHeaderModifier filter — set/add/remove headers Envoy passes upstream.
rules: - filters: - type: RequestHeaderModifier requestHeaderModifier: add: - { name: x-injected-by-gateway, value: "true" } - { name: x-trace-id, value: "%REQ(x-request-id)%" } set: - { name: user-agent, value: "gateway-injected" } remove: [ "cookie" ] # strip cookies upstream backendRefs: - { name: my-app, port: 80 }
%REQ(...)% / %DOWNSTREAM_REMOTE_ADDRESS% / etc. when EG translates the filter to its config. Plain Gateway-API doesn't define this, but EG accepts it.02kubectl cheatsheet
The exact one-liners we use when something doesn't behave. Group by intent, not by resource — most debugging starts with "what state is this Route in?".
| Intent | Command |
|---|---|
| Switch to the right context (insecure-skip-tls-verify is NEVER what you want) | kubectl config use-context externalTLSVerify |
| What Routes exist, any kind, any namespace | kubectl get httproute,grpcroute,tcproute -A -o wide |
| Route accepted? backends resolved? | kubectl -n demo get httproute features -o jsonpath='{.status.parents[*].conditions[*].type}{"="}{.status.parents[*].conditions[*].status}{"\n"}' |
| Live Gateway status (Programmed=True is the gate) | kubectl get gateway -A -o wide |
| What EG-extension policies attach to a Route? | kubectl get securitypolicy,backendtrafficpolicy,clienttrafficpolicy,envoyextensionpolicy -A |
| Envoy data-plane logs (where Coraza WAF / rate-limit denials show up) | kubectl -n envoy-gateway-system logs -l app.kubernetes.io/name=envoy --tail=200 -f |
| Envoy Gateway controller logs (CRD reconcile errors) | kubectl -n envoy-gateway-system logs deploy/envoy-gateway --tail=200 -f |
| Restart the Envoy data plane after listener changes | kubectl -n envoy-gateway-system rollout restart deploy |
| Show all Certificates and whether they're ready | kubectl get certificate -A |
| Force a cert reissue (debug Let's Encrypt issues) | kubectl -n default annotate certificate <name> cert-manager.io/issue-temporary-certificate- |
| external-dns: what does it see + what's it doing? | kubectl -n external-dns logs -l app.kubernetes.io/name=external-dns --tail=100 |
| The ELB IP of the Gateway (without scraping the UI) | kubectl get svc -A -l gateway.envoyproxy.io/owning-gateway-name=eg -o jsonpath='{.items[0].status.loadBalancer.ingress[0].ip}' |
| Which pod backs my Service right now? | kubectl -n demo get endpoints my-svc -o yaml |
| Hit a Service directly (skip the ELB, sanity-check the pod) | kubectl -n demo run curl --rm -i --image=curlimages/curl -- curl -sv http://my-svc/ |
| What's on the cluster — full inventory | kubectl get gateway,httproute,grpcroute,tcproute,securitypolicy,backendtrafficpolicy,clienttrafficpolicy,envoyextensionpolicy -A |
Apply UI changes after editing ui-src/*.html |
./scripts/build-ui-manifest.sh --deploy |
03Debug decision tree
Three of the most common failure modes, in the order we actually check them. Each step is a question — answer it before moving on.
"My Route doesn't work — empty reply / 404 / wrong backend"
- Is the Gateway
Programmed=True?kubectl get gateway eg -o yaml | yq '.status.conditions'. If false, the EnvoyProxy CR or listener config is rejected — check controller logs. - Is the Route
Accepted=TrueANDResolvedRefs=True?
Accepted=False usually means a hostname conflict or missingsectionName. ResolvedRefs=False means a backend Service or cross-namespace ReferenceGrant is missing. - Does the path match what you think it matches?
Envoy uses longest-prefix-match on path./foo/barwins over/foo. Header/method matchers are ANDed within a rule. - Does the backend Service have
endpoints?kubectl get endpoints <svc>— if empty, the Service selector doesn't match any pod labels. - Still broken? Curl directly:
(a) from inside the clusterkubectl run curl --rm -i --image=curlimages/curl -- curl -sv http://svc.namespace.svc.cluster.local/(skips Envoy) — if this fails, it's a pod/service problem, not Gateway. (b) from your laptop with--resolveto bypass DNS.
"TLS handshake fails / browser shows cert warning"
- Is the Certificate
Ready=True?kubectl get certificate -A. If pending, Let's Encrypt is either rate-limited (use staging) or DNS-01 challenge is failing. - Does the Listener reference the right Secret name?
Cert-manager creates a Secret with the name fromspec.secretName. Listener'stls.certificateRefs[0].namemust match. - Is the cert in the SAME namespace as the Gateway?
Default behavior: certificateRefs only resolve in-namespace. Cross-namespace requires a ReferenceGrant. - Cert looks right, browser still warns?
Checkopenssl s_client -servername <hostname> -connect <hostname>:443— issuer should be R10/R11/R12/R13 (Let's Encrypt). If you see(STAGING), your ClusterIssuer is pointing at the staging URL.
"Browser fetch fails with CORS / Load failed"
- Is there a
SecurityPolicy.corsattached to the Route?kubectl get securitypolicy -A | grep cors. Without it, every cross-origin POST with a non-simple Content-Type triggers a preflight that gets 404. - Does
allowOriginsinclude the calling origin EXACTLY?
Origins are matched verbatim —https://example.comdoesn't matchhttps://example.com:443. Wildcards via"*"only work without credentials. - Does
allowHeadersinclude every custom header you send?
The browser preflight lists the actual headers — Envoy will reject if any is missing from the policy. - Browser still cached the old preflight?
maxAge: 1hmeans the browser keeps the old (failed) preflight for up to an hour. Disable cache in DevTools, or open a private tab.
04Glossary
The terms that trip people up — especially folks coming from Ingress.
parentRef
The thing a Route attaches to. Almost always a Gateway. Pinning sectionName further targets one specific listener on that Gateway.
sectionName
Names a specific listener on the Gateway. Without it, a Route attaches to every compatible listener — including http :80, which often isn't what you want.
allowedRoutes
Per-listener gate: which namespaces and Route kinds may attach? Use kinds: [GRPCRoute] to reserve a listener for gRPC, etc.
appProtocol
Tells Envoy what protocol to speak upstream. kubernetes.io/h2c = HTTP/2 cleartext (essential for gRPC backends). Default is HTTP/1.1.
ReferenceGrant
Cross-namespace permission. A Route in namespace A pointing to a Service in namespace B needs a ReferenceGrant in B that permits it. Required by spec, not just policy.
EnvoyProxy CR
Envoy-Gateway-specific. Controls how the data plane is provisioned — replicas, resources, and the ELB annotations baked into the Service.
SecurityPolicy
CORS, BasicAuth, JWT, OIDC, ext-auth, API-key. Single CRD covers all of them. Attaches via targetRefs to HTTPRoute or GRPCRoute — kind-agnostic.
BackendTrafficPolicy
Per-backend behaviour: rate limit, retries, timeouts, fault injection, circuit breaking. Attaches to a Route or Service.
ClientTrafficPolicy
Per-listener client-side knobs: TLS minimum, allowed ciphers, HTTP/3 enablement, proxy-protocol, max-concurrent-streams.
EnvoyExtensionPolicy
Inject WASM or Lua filters into Envoy. Backs the Coraza WAF on this cluster. Use when no native CRD exists for what you need.
autocreate vs elb.id
CCE-specific. autocreate = a new ELB+EIP per Service. elb.id = attach to an existing ELB (different port). Use the latter to share infrastructure.
grpc-web vs gRPC
Native gRPC uses HTTP/2 trailers — browsers can't read them via fetch(). grpc-web puts status inline in the body. Envoy Gateway translates between them transparently on the same GRPCRoute.
05Repo layout
Where things live in k8s-api-gw/.
| manifests/00-namespaces.yaml | Namespaces (demo, lb-demo, whoami) |
| manifests/10-gatewayclass-envoy.yaml | GatewayClass + EnvoyProxy (the CCE ELB annotations live here) |
| manifests/20-gateway.yaml | The Gateway with all 6 listeners (HTTP, 4× HTTPS, TCP) |
| manifests/30-demo-app.yaml | Generated by build-ui-manifest.sh — don't edit by hand. UI Kong + echo backend. |
| manifests/40-httproute-demo.yaml | Echo + UI HTTPRoutes |
| manifests/41-http-to-https.yaml | HTTP→HTTPS 301 redirect (pinned to http listener) |
| manifests/50-features-demo.yaml | The 14-rule features HTTPRoute + supporting backends (echo-v2, slow, hello) |
| manifests/15-hardening-tier1.yaml | TLS min, rate limit, CORS, security headers |
| manifests/16-hardening-waf.yaml | Coraza WAF (EnvoyExtensionPolicy + WASM) |
| manifests/60-secret-demo.yaml | Basic-Auth-protected secret page |
| manifests/70-grpc-demo.yaml | GRPCRoute + fortio + cors-grpc |
| manifests/80-tcp-demo.yaml | TCPRoute + Redis |
| manifests/whoami-demo.yaml | Direct LB demo (the colored pods) |
| scripts/10-install-crds.sh | Gateway API CRDs — standard + experimental channels |
| scripts/20-install-controller.sh | Helm install of Envoy Gateway |
| scripts/30-deploy.sh | Apply all manifests in order |
| scripts/50-verify.sh | End-to-end smoke test |
| scripts/91-external-dns.sh | Install external-dns with Cloudflare provider |
| scripts/92-cert-manager.sh | Install cert-manager + ClusterIssuer + per-hostname Certificates |
| scripts/build-ui-manifest.sh | Bundle ui-src/*.html into the Kong ConfigMap. Run with --deploy to apply. |
| ui-src/*.html | All UI pages. Edit here, never edit 30-demo-app.yaml directly. |
| charts/envoy-gateway/values.yaml | Helm values for Envoy Gateway (data-plane replicas, resources) |