## Root cause (live on otech116 2026-05-05 14:38)
After the #968 fix shipped (0.1.19), the cutover engine reached Step-7
(87%) successfully — Step-01..07 all completed. Then Step-08 (egress-
block-test) caught 38/38 HelmRepositories had reverted to upstream:
```
external HelmRepositories still pointing at ghcr.io/openova-io: 38
OFFENDER flux-system/bp-cilium=oci://ghcr.io/openova-io
... (37 more)
FAIL — at least one HelmRepository did not pivot
```
But Step-06's job logs say:
```
[helmrepository-patches] OK bp-cilium -> oci://harbor.otech116.omani.works/openova-io
... (37 more OK)
ok=38 skip=0 fail=0
```
So Step-06 thought it succeeded — and it had, momentarily. But then
the bootstrap-kit Kustomization (which had successfully pivoted to
local Gitea via Step-05) reconciled its YAML from local Gitea, where
the YAML still declared `url: oci://ghcr.io/openova-io`. Within ~30s
every kubectl patch was undone. The cutover engine then aborted at
Step-8 verification.
## Fix
Step-06 now runs in two phases:
1. **Live K8s patches** (existing behaviour) — flips spec.url on every
HelmRepository immediately. Useful for the cluster between cutover
and the next reconcile.
2. **NEW — Push YAML edit to local Gitea** — clones `openova/openova`
from the local Gitea over basic-auth, sed-rewrites every
`clusters/_template/bootstrap-kit/*.yaml` declaration of `url:
oci://ghcr.io/openova-io` → `oci://harbor.<sov-fqdn>/openova-io`,
commits with a clear message, pushes back. Subsequent reconciles
see local Harbor as the steady-state.
After the push, the script annotates `flux-system/openova` GitRepository
to trigger immediate reconciliation so the new YAML lands without
waiting for the polling interval.
## Image change
Step-06 image bumped from `bitnami/kubectl:1.31.4` to `alpine/k8s:1.31.4`
because the new phase needs both `kubectl` and `git` in one image
(verified live on otech116 — both binaries present).
## Acceptance gate
Test case 16 added to cutover-contract.sh — guards against future
regressions that remove the `git clone`, the `git push origin main`,
or the `clusters/_template/bootstrap-kit` target dir reference.
## Live verification
Will fire on otech117 (next provision). Expected:
- Step-06 logs `cloning gitea-http.gitea.../openova/openova.git` then `pushed to ...`
- Step-08 verify PASSES (38/38 HelmRepositories pivoted in K8s + Gitea)
- self-sovereign-cutover-status `cutoverComplete: "true"`
- Egress block to ghcr.io safely activates
Co-authored-by: e3mrah <ebaysal@gmail.com>
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
608db53a25