52b87afa9e
1.16.x gateway-api hostNetwork mode is buggy on Sovereigns: cilium-envoy
NACKs listeners with "cannot bind '0.0.0.0:80': Permission denied" and
the loaded RDS for the Sovereign vhost only carries the default `/` route
to catalyst-ui — `/auth/*` and `/api/*` HTTPRoute matches defined in CEC
never reach envoy's live config. Result: console.<sov>/auth/handover?token=…
serves the React shell instead of the catalyst-api Go handler, defeating
the Phase-8b seamless handover. Caught live on otech46.
1.18+ ships the Gateway API implementation graduated from beta with the
hostNetwork bind path fixed; 1.19 is the current stable line (1.19.3).
Values shape verified backward-compatible across the keys we set:
gatewayAPI.hostNetwork.enabled, envoy.enabled, envoyConfig.enabled,
encryption.type=wireguard, encryption.nodeEncryption — all unchanged
between 1.16 and 1.19.
Bumps:
- bp-cilium chart 1.1.5 → 1.2.0 (minor — major upstream version jump)
- upstream cilium subchart 1.16.5 → 1.19.3
- blueprint.yaml spec.version 1.1.3 → 1.2.0 (was already drifted from
Chart.yaml; brings them back in sync per manifest-validation gate)
- clusters/_template/bootstrap-kit/01-cilium.yaml HelmRelease pin
1.1.5 → 1.2.0
Per-cluster overlays under clusters/<sovereign>/bootstrap-kit/ keep
their pinned versions until the operator opts in — fresh otechN
provisions render from _template/ and pick up 1.2.0 on first boot.
Will be verified live on the next fresh Sovereign provision (otech47+).
Co-authored-by: hatiyildiz <hatice@openova.io>
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
113 lines
4.2 KiB
YAML
113 lines
4.2 KiB
YAML
# bp-cilium — Catalyst bootstrap-kit Blueprint. CNI must come first; k3s started with --flannel-backend=none precisely so Cilium can take over.
|
|
#
|
|
# Wrapper chart: platform/cilium/chart/
|
|
# Catalyst-curated values: platform/cilium/chart/values.yaml
|
|
# Reconciled by: Flux on the new Sovereign's k3s control plane.
|
|
|
|
---
|
|
# kube-system is built into every Kubernetes cluster — never re-declare it.
|
|
# Earlier revisions of 01-cilium.yaml AND 05-sealed-secrets.yaml both
|
|
# declared it, which collided when kustomize tried to merge the two:
|
|
# "may not add resource with an already registered id:
|
|
# Namespace.v1.[noGrp]/kube-system.[noNs]"
|
|
# This Blueprint installs Cilium INTO kube-system; the HelmRelease's
|
|
# targetNamespace field below is sufficient.
|
|
apiVersion: source.toolkit.fluxcd.io/v1beta2
|
|
kind: HelmRepository
|
|
metadata:
|
|
name: bp-cilium
|
|
namespace: flux-system
|
|
spec:
|
|
type: oci
|
|
interval: 15m
|
|
url: oci://ghcr.io/openova-io
|
|
secretRef:
|
|
name: ghcr-pull
|
|
---
|
|
apiVersion: helm.toolkit.fluxcd.io/v2
|
|
kind: HelmRelease
|
|
metadata:
|
|
name: bp-cilium
|
|
namespace: flux-system
|
|
spec:
|
|
interval: 15m
|
|
releaseName: cilium
|
|
targetNamespace: kube-system
|
|
chart:
|
|
spec:
|
|
chart: bp-cilium
|
|
version: 1.2.0
|
|
sourceRef:
|
|
kind: HelmRepository
|
|
name: bp-cilium
|
|
namespace: flux-system
|
|
# Event-driven install: Helm completes when manifests apply, not when
|
|
# cilium-agent reaches Ready (agent waits for envoyconfig CRDs that the
|
|
# SAME chart installs — legitimate slow-Ready). Replaces blanket
|
|
# spec.timeout: 15m band-aid from PR #221.
|
|
install:
|
|
disableWait: true
|
|
remediation:
|
|
retries: 3
|
|
upgrade:
|
|
disableWait: true
|
|
remediation:
|
|
retries: 3
|
|
values:
|
|
cilium:
|
|
# Phase-8a bug #15 (otech8 deployment 1bfc46347564467b 2026-05-01):
|
|
# cilium-agent waits forever for the operator to register
|
|
# ciliumenvoyconfigs + ciliumclusterwideenvoyconfigs CRDs.
|
|
# Setting `envoy.enabled: true` (chart-level) runs Envoy as a separate
|
|
# daemonset but does NOT register those CRDs — that requires
|
|
# `envoyConfig.enabled: true`, a separate upstream chart toggle.
|
|
# Without it, the agent's node taint `node.cilium.io/agent-not-ready`
|
|
# never lifts and every other HelmRelease (37 of them) blocks on its
|
|
# dependsOn chain.
|
|
envoyConfig:
|
|
enabled: true
|
|
l7Proxy: true
|
|
prometheus:
|
|
enabled: false
|
|
serviceMonitor:
|
|
enabled: false
|
|
hubble:
|
|
metrics:
|
|
enabled: null
|
|
serviceMonitor:
|
|
enabled: false
|
|
relay:
|
|
enabled: false
|
|
ui:
|
|
enabled: false
|
|
---
|
|
# ─── Per-Sovereign Gateway API resources (issue #387) ────────────────────
|
|
#
|
|
# Cilium owns the GatewayClass (`cilium`) installed by the chart above
|
|
# (gatewayAPI.enabled=true, envoy.enabled=true in platform/cilium/chart/
|
|
# values.yaml). The single per-Sovereign Gateway listening on
|
|
# *.${SOVEREIGN_FQDN}:443 lives here so it boots alongside the CNI
|
|
# without needing a new bootstrap-kit slot — every Sovereign HTTP
|
|
# blueprint (catalyst-platform, gitea, keycloak, harbor, grafana,
|
|
# openbao, powerdns) attaches its HTTPRoute to this Gateway via
|
|
# parentRefs.
|
|
#
|
|
# TLS material: a wildcard Certificate is requested from
|
|
# letsencrypt-dns01-prod-powerdns (cert-manager + bp-cert-manager-
|
|
# powerdns-webhook from #373; webhook calls contabo's central PowerDNS
|
|
# at https://pdns.openova.io). The resulting Secret
|
|
# `sovereign-wildcard-tls` is referenced by the Gateway listener.
|
|
#
|
|
# Cross-namespace HTTPRoute attachment: allowedRoutes.namespaces.from=All
|
|
# permits every blueprint namespace (catalyst-system, gitea, keycloak,
|
|
# harbor, grafana-system, openbao, powerdns-system) to bind without a
|
|
# ReferenceGrant. This matches the Catalyst single-tenant Sovereign
|
|
# model — cross-tenant isolation is enforced by per-tenant vClusters
|
|
# (bp-vcluster), not by Gateway-level RBAC.
|
|
#
|
|
# Per ADR-0001 §9.4 and docs/INVIOLABLE-PRINCIPLES.md #4: this resource
|
|
# only renders when ${SOVEREIGN_FQDN} is set by Flux envsubst at the
|
|
# Sovereign apply time — contabo's bootstrap path does NOT include this
|
|
# template, so Traefik continues to serve console.openova.io/nova
|
|
# unchanged.
|