openova/platform/kyverno
e3mrah 3a57e287e5
feat(platform): security umbrellas (falco/kyverno/trivy/sigstore/syft-grype/reloader/coraza/litmus) (#216)
* feat(bp-falco): umbrella chart for security layer

Catalyst Blueprint umbrella chart for falco — security/policy layer.

Pinned upstream + appVersion verified against the helm index on
2026-04-30. ServiceMonitor disabled per BLUEPRINT-AUTHORING.md §11.2.
Solo-Sovereign defaults; per-Sovereign overlays bump to HA later.

Part of security-stack umbrellas batch 3.

* feat(bp-kyverno): umbrella chart for security layer

Catalyst Blueprint umbrella chart for kyverno — security/policy layer.

Pinned upstream + appVersion verified against the helm index on
2026-04-30. ServiceMonitor disabled per BLUEPRINT-AUTHORING.md §11.2.
Solo-Sovereign defaults; per-Sovereign overlays bump to HA later.

Part of security-stack umbrellas batch 3.

* feat(bp-trivy): umbrella chart for security layer

Catalyst Blueprint umbrella chart for trivy — security/policy layer.

Pinned upstream + appVersion verified against the helm index on
2026-04-30. ServiceMonitor disabled per BLUEPRINT-AUTHORING.md §11.2.
Solo-Sovereign defaults; per-Sovereign overlays bump to HA later.

Part of security-stack umbrellas batch 3.

* feat(bp-sigstore): umbrella chart for security layer

Catalyst Blueprint umbrella chart for sigstore — security/policy layer.

Pinned upstream + appVersion verified against the helm index on
2026-04-30. ServiceMonitor disabled per BLUEPRINT-AUTHORING.md §11.2.
Solo-Sovereign defaults; per-Sovereign overlays bump to HA later.

Part of security-stack umbrellas batch 3.

* feat(bp-syft-grype): umbrella chart for security layer

Catalyst Blueprint umbrella chart for syft-grype — security/policy layer.

Pinned upstream + appVersion verified against the helm index on
2026-04-30. ServiceMonitor disabled per BLUEPRINT-AUTHORING.md §11.2.
Solo-Sovereign defaults; per-Sovereign overlays bump to HA later.

Part of security-stack umbrellas batch 3.

* feat(bp-reloader): umbrella chart for security layer

Catalyst Blueprint umbrella chart for reloader — security/policy layer.

Pinned upstream + appVersion verified against the helm index on
2026-04-30. ServiceMonitor disabled per BLUEPRINT-AUTHORING.md §11.2.
Solo-Sovereign defaults; per-Sovereign overlays bump to HA later.

Part of security-stack umbrellas batch 3.

* feat(bp-coraza): umbrella chart for security layer

Catalyst Blueprint umbrella chart for coraza — security/policy layer.

Pinned upstream + appVersion verified against the helm index on
2026-04-30. ServiceMonitor disabled per BLUEPRINT-AUTHORING.md §11.2.
Solo-Sovereign defaults; per-Sovereign overlays bump to HA later.

Part of security-stack umbrellas batch 3.

* feat(bp-litmus): umbrella chart for security layer

Catalyst Blueprint umbrella chart for litmus — security/policy layer.

Pinned upstream + appVersion verified against the helm index on
2026-04-30. ServiceMonitor disabled per BLUEPRINT-AUTHORING.md §11.2.
Solo-Sovereign defaults; per-Sovereign overlays bump to HA later.

Part of security-stack umbrellas batch 3.

---------

Co-authored-by: hatiyildiz <hatice.yildiz@openova.io>
2026-04-30 06:07:38 +02:00
..
chart feat(platform): security umbrellas (falco/kyverno/trivy/sigstore/syft-grype/reloader/coraza/litmus) (#216) 2026-04-30 06:07:38 +02:00
blueprint.yaml feat(platform): security umbrellas (falco/kyverno/trivy/sigstore/syft-grype/reloader/coraza/litmus) (#216) 2026-04-30 06:07:38 +02:00
README.md docs(pass-36): flux deep-scrutiny + sweep gap-fill (Pass 35 head -10 cutoff) 2026-04-27 22:49:05 +02:00

Kyverno

Policy engine for admission control, mutation, and policy generation. Per-host-cluster infrastructure (see docs/PLATFORM-TECH-STACK.md §3.3) — runs on every host cluster Catalyst manages. Enforces Catalyst's policy contracts (cosign-required-on-Blueprints, default-deny NetworkPolicies on Organization namespaces, priority-class injection, etc.).

Status: Accepted | Updated: 2026-04-27


Overview

Kyverno is the single enforcement point for all platform-wide guarantees. Every Deployment, StatefulSet, and Namespace that enters the cluster passes through Kyverno admission control. Policies are grouped into three operational domains:

Domain Goal
Resilience Workloads survive node failures, zone outages, and rolling updates without downtime
Security Zero-trust posture: least privilege, no implicit trust, defense in depth
Operational Consistent labeling, image provenance, resource hygiene

Policy types and when each is used:

Type Timing Mechanism Use Case
Validate Admission (sync) Reject non-compliant resources Enforce hard constraints that must never be violated
Mutate Admission (sync) Patch resources before persistence Inject sensible defaults so teams don't have to remember every field
Generate Background (async) Create companion resources Produce derived objects that always accompany a parent resource

Architecture

flowchart TB
    subgraph Admission["Admission Pipeline (synchronous)"]
        direction LR
        Mutate[Mutate Policies<br/>Inject defaults]
        Validate[Validate Policies<br/>Enforce constraints]
        Mutate --> Validate
    end

    subgraph Background["Background Controller (async)"]
        Generate[Generate Policies<br/>Create companions]
    end

    subgraph Generated["Auto-Generated Resources"]
        VPA[VPA]
        PDB[PodDisruptionBudget]
        NP[NetworkPolicy]
        CNP[CiliumNetworkPolicy]
        RQ[ResourceQuota]
        LR[LimitRange]
    end

    Resource[Incoming Resource] --> Admission
    Admission -->|Accepted| etcd[(etcd)]
    etcd --> Background
    Background --> Generated

Policy Matrix — Resilience

The following table is the authoritative reference for every resilience policy enforced on the platform. Each row describes what is enforced, how (mutate / generate / validate), why, and the opt-out mechanism.

Generate — Companion Resources

These policies create companion resources whenever a qualifying parent resource is persisted.

# Policy Name Trigger Resource Generated Resource Match Condition Default Behavior Opt-Out Annotation
G1 generate-vpa Deployment VerticalPodAutoscaler All Deployments updateMode: Auto, min 50m CPU / 64Mi, max 2 CPU / 4Gi vpa.openova.io/skip: "true"
G2 generate-vpa-statefulset StatefulSet VerticalPodAutoscaler All StatefulSets updateMode: Initial (avoids disruptive eviction of stateful pods) vpa.openova.io/skip: "true"
G3 generate-pdb Deployment PodDisruptionBudget spec.replicas >= 2 minAvailable: 50% (round up) pdb.openova.io/skip: "true"
G4 generate-pdb-statefulset StatefulSet PodDisruptionBudget spec.replicas >= 2 maxUnavailable: 1 (strict for stateful workloads) pdb.openova.io/skip: "true"
G5 generate-default-deny Namespace NetworkPolicy (default-deny ingress+egress) Label openova.io/organization exists Deny all ingress and egress; teams must explicitly allow networkpolicy.openova.io/skip: "true"
G6 generate-cilium-default-deny Namespace CiliumNetworkPolicy (L3/L4/L7 deny) Label openova.io/organization exists L7-aware default deny (complements K8s NetworkPolicy) networkpolicy.openova.io/skip: "true"
G7 generate-resource-quota Namespace ResourceQuota Label openova.io/tier exists Quotas derived from tier label (see Resource Tiers below) quota.openova.io/skip: "true"
G8 generate-limit-range Namespace LimitRange Label openova.io/tier exists Default container requests: 50m/64Mi, limits: 500m/512Mi limitrange.openova.io/skip: "true"

Mutate — Injected Defaults

These policies patch resources at admission time so that every workload gets resilience primitives without requiring teams to configure them manually.

# Policy Name Target Resource What Is Injected Match Condition Behavior Opt-Out Annotation
M1 inject-topology-spread Deployment, StatefulSet topologySpreadConstraints for zone + node spread spec.replicas >= 2 Zone: maxSkew: 1, whenUnsatisfiable: ScheduleAnyway; Node: maxSkew: 1, whenUnsatisfiable: ScheduleAnyway topology.openova.io/skip: "true"
M2 inject-pod-anti-affinity Deployment affinity.podAntiAffinity (preferred) spec.replicas >= 2 and no existing anti-affinity Prefer spreading across nodes using app.kubernetes.io/name label affinity.openova.io/skip: "true"
M3 inject-security-context Pod Default securityContext on all containers All Pods without explicit securityContext runAsNonRoot: true, readOnlyRootFilesystem: true, allowPrivilegeEscalation: false, drop ALL capabilities securitycontext.openova.io/skip: "true"
M4 inject-seccomp-profile Pod seccompProfile on pod spec All Pods without explicit seccomp type: RuntimeDefault seccomp.openova.io/skip: "true"
M5 rewrite-image-registry Pod Rewrite container image to Harbor mirror All Pods with images not already from Harbor docker.io/library/nginx:1.25harbor.<cluster>/dockerhub-proxy/library/nginx:1.25 harbor.openova.io/skip: "true"
M6 inject-priority-class Deployment, StatefulSet priorityClassName No existing priorityClassName set Derived from namespace tier label: platform-critical, tenant-high, tenant-default priority.openova.io/skip: "true"
M7 disable-service-account-token Pod automountServiceAccountToken: false All Pods without explicit setting Prevents unnecessary API server access sa-token.openova.io/skip: "true"
M8 inject-gpu-tolerations Pod GPU node tolerations + nodeSelector Pods requesting nvidia.com/gpu Adds nvidia.com/gpu: NoSchedule toleration and GPU node selector N/A (conditional on GPU request)

Validate — Hard Constraints

These policies reject resources that violate resilience requirements. Enforce means the resource is blocked; Audit means it is logged but allowed (used during migration periods).

# Policy Name Target Resource What Is Checked Action Rationale
V1 require-resource-requests Pod All containers must set resources.requests.cpu and resources.requests.memory Enforce VPA and scheduler require requests to function; without them, bin-packing and right-sizing are impossible
V2 require-resource-limits-memory Pod All containers must set resources.limits.memory Enforce Prevents OOM-killed neighbors; CPU limits deliberately not enforced (VPA + CFS throttling handles it)
V3 require-health-probes Deployment, StatefulSet Must define readinessProbe and livenessProbe on all containers Enforce Without probes, failed pods stay in rotation and PDB disruption calculations are meaningless
V4 require-startup-probe-slow-start Deployment, StatefulSet Must define startupProbe if initialDelaySeconds > 30 on liveness probe Audit Prevents premature kills of slow-starting apps (JVM, ML model loading)
V5 minimum-replicas-production Deployment spec.replicas >= 2 in namespaces labeled openova.io/env-type: prod Enforce Single-replica deployments have zero fault tolerance; PDB and topology spread are useless with 1 replica
V6 minimum-replicas-statefulset StatefulSet spec.replicas >= 2 in production namespaces (except where singleton.openova.io/allowed: "true") Audit StatefulSets often have leader-election singletons; audit mode catches unintentional single replicas
V7 restrict-pdb-max-unavailable PodDisruptionBudget maxUnavailable must not be set to 100% or total replica count Enforce A PDB that allows all pods to be disrupted simultaneously provides no protection
V8 require-emptydir-sizelimit Pod emptyDir volumes must set sizeLimit Enforce Unbounded emptyDir can exhaust node disk, causing eviction storms across all pods on the node
V9 prevent-hpa-keda-conflict HorizontalPodAutoscaler Reject HPA if a KEDA ScaledObject already targets the same Deployment Enforce HPA and KEDA both manage replica count; dual ownership causes flapping and unpredictable scaling
V10 require-pdb-for-replicated Deployment Deployments with replicas >= 3 in production must have a corresponding PDB Audit Catches cases where PDB generation was skipped but manual PDB was not created

Policy Matrix — Security & Zero Trust

Validate — Pod Security

Based on the Kubernetes Pod Security Standards (Restricted profile) with OpenOva-specific additions.

# Policy Name Target What Is Checked Action Rationale
S1 disallow-privileged Pod securityContext.privileged must be false or unset Enforce Privileged containers have full host access; equivalent to root on the node
S2 disallow-privilege-escalation Pod allowPrivilegeEscalation must be false Enforce Prevents child processes from gaining more privileges than parent via setuid/setgid
S3 require-non-root Pod runAsNonRoot: true on pod or all containers Enforce Running as root inside the container is unnecessary for all platform workloads
S4 require-non-root-uid Pod runAsUser must be >= 1000 (if set) Audit UIDs below 1000 are conventionally reserved for system accounts; catches hardcoded runAsUser: 0
S5 restrict-capabilities Pod Only NET_BIND_SERVICE allowed; all others must be dropped Enforce Least-privilege capability set; Cilium handles networking, so NET_RAW, SYS_ADMIN etc. are unnecessary
S6 require-readonly-rootfs Pod readOnlyRootFilesystem: true Enforce Prevents runtime binary modification; apps write to mounted volumes only
S7 disallow-host-namespaces Pod hostNetwork, hostPID, hostIPC must all be false Enforce Host namespace access breaks pod isolation boundaries completely
S8 disallow-host-ports Pod containerPort.hostPort must be unset or 0 Enforce Host port bindings conflict across pods and bypass service abstraction
S9 restrict-volume-types Pod Only allow: configMap, csi, downwardAPI, emptyDir, ephemeral, persistentVolumeClaim, projected, secret Enforce Blocks hostPath, nfs, gitRepo and other dangerous volume types
S10 disallow-hostpath Pod No hostPath volume mounts Enforce HostPath provides arbitrary read/write to node filesystem; most common container escape vector
S11 require-seccomp-profile Pod seccompProfile.type must be RuntimeDefault or Localhost Enforce Seccomp restricts system calls; Unconfined leaves full syscall surface exposed
S12 restrict-sysctls Pod Only safe sysctls allowed: net.ipv4.ip_local_port_range, net.ipv4.tcp_syncookies, kernel.shm_rmid_forced Enforce Unsafe sysctls affect the entire node; only whitelisted safe sysctls permitted
S13 restrict-proc-mount Pod procMount must be Default (not Unmasked) Enforce Unmasked /proc exposes sensitive kernel parameters to container processes
S14 restrict-apparmor Pod AppArmor annotation must be runtime/default or localhost/* if set Enforce Prevents explicitly setting unconfined to bypass AppArmor profiles

Validate — Image Supply Chain

# Policy Name Target What Is Checked Action Rationale
S15 disallow-latest-tag Pod No container may use :latest tag or omit the tag Enforce Mutable tags make rollbacks impossible and break GitOps reproducibility
S16 require-harbor-registry Pod All images must be pulled from harbor.<cluster-domain>/ Enforce Ensures all images pass Harbor's Trivy vulnerability scanning before deployment
S17 verify-image-signature Pod Images must have a valid cosign signature (Sigstore) Audit Ensures image provenance; audit mode until all teams sign images

Validate — Network & Zero Trust

# Policy Name Target What Is Checked Action Rationale
S18 disallow-default-namespace Deployment, StatefulSet, DaemonSet, Pod Must not be deployed to default namespace Enforce Default namespace has no NetworkPolicy isolation; all workloads must live in explicit namespaces
S19 disallow-nodeport-services Service spec.type must not be NodePort Enforce NodePort exposes services directly on node IPs, bypassing Gateway API and Cilium L7 policies
S20 disallow-external-lb-annotation Service Must not carry cloud-provider LB annotations unless in openova-system namespace Enforce Load balancers are provisioned via Gateway API; ad-hoc LB annotations cause cost leaks and security gaps
S21 require-network-policy-exists Namespace Namespaces labeled openova.io/organization must contain at least one NetworkPolicy Audit Catches deleted or missing network policies; complements the generate policy G5
S22 restrict-external-names Service type: ExternalName only allowed when annotated externalname.openova.io/approved: "true" Enforce ExternalName services can redirect to arbitrary external endpoints, bypassing egress policies

Validate — Operational Hygiene

# Policy Name Target What Is Checked Action Rationale
S23 require-labels Deployment, StatefulSet Must have app.kubernetes.io/name, app.kubernetes.io/version, app.kubernetes.io/part-of Enforce Required for PDB label selectors, topology spread, monitoring dashboards, and cost attribution
S24 require-owner-label Namespace Must have openova.io/owner label Audit Enables cost attribution and orphaned-namespace detection
S25 disallow-wildcard-rbac ClusterRole, Role No rule may use * for resources or verbs Enforce Wildcard RBAC grants full cluster admin; roles must be scoped to specific resources and verbs
S26 restrict-automount-sa-token ServiceAccount automountServiceAccountToken must be false unless annotated sa-token.openova.io/required: "true" Audit Most workloads don't need API server access; leaked tokens enable lateral movement

Resource Tiers

Namespace-level tier labels drive ResourceQuota and PriorityClass injection.

Tier Label Value CPU Quota Memory Quota PriorityClass Use Case
platform 16 cores 32Gi platform-critical (1000000) OpenOva system components
high 8 cores 16Gi tenant-high (100000) Production Organization workloads
default 4 cores 8Gi tenant-default (10000) Development / staging Organization workloads
batch 2 cores 4Gi tenant-batch (1000) Background jobs, scale-to-zero

The tenant-* priority class names are legacy deployment artifacts. They map to Organization workloads in current terminology. Renaming the priority class names themselves is tracked as a separate migration item — until then, the names remain as-is in deployed clusters.


Interaction with Scaling Components

Kyverno's policies are designed to complement VPA, KEDA, and the platform's scaling architecture. The following table clarifies ownership boundaries.

Concern Owner Kyverno's Role
Vertical sizing (CPU/memory per pod) VPA Generate VPA for every Deployment (G1) and StatefulSet (G2); Validate that resource requests exist (V1) so VPA has a baseline to optimize
Horizontal scaling (replica count) KEDA Validate that HPA and KEDA don't both target the same workload (V9); Enforce minimum replicas in production (V5)
HPA KEDA replaces HPA Validate no HPA when ScaledObject exists (V9); mutate policies do not inject HPA — KEDA is the sole horizontal scaler
PDB (disruption budget) Kyverno (generated) Generate PDB for replicated workloads (G3, G4); Validate PDB is not set to allow 100% unavailability (V7)
Topology spread Kyverno (mutated) Inject topologySpreadConstraints at admission (M1) so pods spread across zones and nodes
Anti-affinity Kyverno (mutated) Inject preferred pod anti-affinity (M2) for zone-level resilience
Priority / preemption Kyverno (mutated) Inject priorityClassName based on namespace tier (M6) so critical workloads preempt batch workloads under pressure
Resource quotas Kyverno (generated) Generate ResourceQuota per namespace (G7) to prevent noisy-neighbor resource exhaustion
LimitRange Kyverno (generated) Generate LimitRange per namespace (G8) so pods without explicit requests get safe defaults

VPA + PDB Coordination

VPA in Auto mode evicts pods to apply new resource recommendations. The generated PDB ensures that VPA evictions respect availability:

VPA Updater wants to evict Pod-A
  → Checks PDB: minAvailable 50%
    → 3/4 pods running → eviction allowed (3 > 50% of 4)
    → 2/4 pods running → eviction blocked (2 = 50% of 4, not > 50%)

For StatefulSets, VPA is generated with updateMode: Initial (G2), meaning resource changes apply only on pod restart — no VPA-initiated evictions. This avoids disruption to stateful workloads where pod identity matters.

KEDA Scale-to-Zero and PDB

When KEDA scales a workload to zero replicas, the PDB is effectively dormant (no pods to protect). On scale-up, the PDB re-activates. This is safe because:

  • Scale-to-zero is only allowed for batch tier namespaces
  • KEDA's minReplicaCount: 0 is only valid when the workload has no SLO requiring availability

Network Policy Strategy

Network isolation follows a default-deny, explicit-allow model at two layers:

flowchart TB
    subgraph L3L4["Layer 3/4 (Kubernetes NetworkPolicy)"]
        KNP[Default Deny<br/>Generated by G5]
    end

    subgraph L7["Layer 3/4/7 (CiliumNetworkPolicy)"]
        CNP[Default Deny + L7 Rules<br/>Generated by G6]
    end

    subgraph Workload["Organization Workload"]
        Pod[Pod]
    end

    KNP -->|"IP + Port filtering"| Pod
    CNP -->|"HTTP method/path filtering"| Pod
Layer Resource Managed By Scope
L3/L4 NetworkPolicy Kyverno generates default-deny (G5); teams add allow rules IP addresses, port numbers, namespace selectors
L3-L7 CiliumNetworkPolicy Kyverno generates default-deny (G6); platform team adds L7 rules HTTP methods, paths, headers, DNS names, gRPC services
Egress CiliumNetworkPolicy Platform team Restrict external egress to known endpoints (DNS-based FQDN policies)
mTLS Cilium WireGuard Cilium agent (automatic) All pod-to-pod traffic encrypted transparently

Zero-Trust Enforcement Checklist

Principle How Enforced Policy ID
No implicit network access Default-deny NetworkPolicy + CiliumNetworkPolicy per namespace G5, G6
No host-level access Disallow hostNetwork, hostPID, hostIPC, hostPath, hostPort S7, S8, S10
Least-privilege containers Drop all capabilities, non-root, read-only rootfs, no privilege escalation S1-S6
Encrypted pod-to-pod Cilium WireGuard mTLS (transparent, no sidecar) Cilium config (automatic)
Image provenance Harbor-only registry, signature verification, no latest tags S15, S16, S17
Service account lockdown automountServiceAccountToken: false by default, no wildcard RBAC M7, S25, S26
Namespace isolation No default namespace usage, mandatory Organization labels S18, S23, S24
Ingress control No NodePort, no ad-hoc LBs, Gateway API only S19, S20
Egress control Default deny egress, FQDN-based allow policies via Cilium G5, G6
Syscall restriction Seccomp RuntimeDefault on all pods M4, S11

Example Policy Manifests

G1 — Generate VPA for Deployments

apiVersion: kyverno.io/v1
kind: ClusterPolicy
metadata:
  name: generate-vpa
  annotations:
    policies.kyverno.io/category: Resilience
    policies.kyverno.io/description: >-
      Generate a VPA for every Deployment to enable vertical right-sizing.
      VPA Updater respects PDB during evictions.      
spec:
  rules:
    - name: generate-vpa-for-deployment
      match:
        any:
          - resources:
              kinds:
                - Deployment
      exclude:
        any:
          - resources:
              annotations:
                vpa.openova.io/skip: "true"
          - resources:
              namespaces:
                - kube-system
                - kyverno
      generate:
        synchronize: true
        apiVersion: autoscaling.k8s.io/v1
        kind: VerticalPodAutoscaler
        name: "{{request.object.metadata.name}}-vpa"
        namespace: "{{request.object.metadata.namespace}}"
        data:
          spec:
            targetRef:
              apiVersion: apps/v1
              kind: Deployment
              name: "{{request.object.metadata.name}}"
            updatePolicy:
              updateMode: Auto
            resourcePolicy:
              containerPolicies:
                - containerName: "*"
                  minAllowed:
                    cpu: 50m
                    memory: 64Mi
                  maxAllowed:
                    cpu: "2"
                    memory: 4Gi
                  controlledResources:
                    - cpu
                    - memory
                  controlledValues: RequestsAndLimits

G3 — Generate PDB for Replicated Deployments

apiVersion: kyverno.io/v1
kind: ClusterPolicy
metadata:
  name: generate-pdb
  annotations:
    policies.kyverno.io/category: Resilience
    policies.kyverno.io/description: >-
      Generate a PDB for Deployments with 2+ replicas to protect availability
      during voluntary disruptions (node drain, cluster upgrade, VPA eviction).      
spec:
  rules:
    - name: generate-pdb-for-deployment
      match:
        any:
          - resources:
              kinds:
                - Deployment
      exclude:
        any:
          - resources:
              annotations:
                pdb.openova.io/skip: "true"
      preconditions:
        all:
          - key: "{{request.object.spec.replicas}}"
            operator: GreaterThanOrEquals
            value: 2
      generate:
        synchronize: true
        apiVersion: policy/v1
        kind: PodDisruptionBudget
        name: "{{request.object.metadata.name}}-pdb"
        namespace: "{{request.object.metadata.namespace}}"
        data:
          spec:
            minAvailable: "50%"
            selector:
              matchLabels:
                app.kubernetes.io/name: "{{request.object.metadata.labels.\"app.kubernetes.io/name\"}}"

M1 — Inject TopologySpreadConstraints

apiVersion: kyverno.io/v1
kind: ClusterPolicy
metadata:
  name: inject-topology-spread
  annotations:
    policies.kyverno.io/category: Resilience
    policies.kyverno.io/description: >-
      Inject zone and node topology spread constraints for multi-replica
      workloads to survive zone failures and balance node utilization.      
spec:
  rules:
    - name: spread-across-zones
      match:
        any:
          - resources:
              kinds:
                - Deployment
                - StatefulSet
      exclude:
        any:
          - resources:
              annotations:
                topology.openova.io/skip: "true"
      preconditions:
        all:
          - key: "{{request.object.spec.replicas}}"
            operator: GreaterThanOrEquals
            value: 2
          - key: "{{ request.object.spec.template.spec.topologySpreadConstraints[] || `[]` | length(@) }}"
            operator: Equals
            value: 0
      mutate:
        patchStrategicMerge:
          spec:
            template:
              spec:
                topologySpreadConstraints:
                  - maxSkew: 1
                    topologyKey: topology.kubernetes.io/zone
                    whenUnsatisfiable: ScheduleAnyway
                    labelSelector:
                      matchLabels:
                        app.kubernetes.io/name: "{{request.object.metadata.labels.\"app.kubernetes.io/name\"}}"
                  - maxSkew: 1
                    topologyKey: kubernetes.io/hostname
                    whenUnsatisfiable: ScheduleAnyway
                    labelSelector:
                      matchLabels:
                        app.kubernetes.io/name: "{{request.object.metadata.labels.\"app.kubernetes.io/name\"}}"

V5 — Minimum Replicas in Production

apiVersion: kyverno.io/v1
kind: ClusterPolicy
metadata:
  name: minimum-replicas-production
  annotations:
    policies.kyverno.io/category: Resilience
    policies.kyverno.io/description: >-
      Production Deployments must have at least 2 replicas. Single-replica
      workloads have zero fault tolerance and render PDB, topology spread,
      and anti-affinity policies meaningless.      
spec:
  validationFailureAction: Enforce
  rules:
    - name: min-replicas
      match:
        any:
          - resources:
              kinds:
                - Deployment
              namespaceSelector:
                matchLabels:
                  openova.io/env: production
      validate:
        message: >-
          Production Deployments must have at least 2 replicas.
          Found {{request.object.spec.replicas}} replica(s) in
          Deployment {{request.object.metadata.name}}.          
        pattern:
          spec:
            replicas: ">=2"

S1 + S2 + S3 — Pod Security Baseline

apiVersion: kyverno.io/v1
kind: ClusterPolicy
metadata:
  name: pod-security-restricted
  annotations:
    policies.kyverno.io/category: Security
    policies.kyverno.io/description: >-
      Enforces Kubernetes Pod Security Standards (Restricted) profile.
      Covers privileged mode, privilege escalation, and non-root enforcement.      
spec:
  validationFailureAction: Enforce
  rules:
    - name: disallow-privileged
      match:
        any:
          - resources:
              kinds:
                - Pod
      validate:
        message: "Privileged containers are not allowed."
        pattern:
          spec:
            =(initContainers):
              - =(securityContext):
                  =(privileged): false
            containers:
              - =(securityContext):
                  =(privileged): false

    - name: disallow-privilege-escalation
      match:
        any:
          - resources:
              kinds:
                - Pod
      validate:
        message: "allowPrivilegeEscalation must be set to false."
        pattern:
          spec:
            =(initContainers):
              - securityContext:
                  allowPrivilegeEscalation: false
            containers:
              - securityContext:
                  allowPrivilegeEscalation: false

    - name: require-run-as-non-root
      match:
        any:
          - resources:
              kinds:
                - Pod
      validate:
        message: "Pods must run as non-root. Set runAsNonRoot to true."
        anyPattern:
          - spec:
              securityContext:
                runAsNonRoot: true
          - spec:
              containers:
                - securityContext:
                    runAsNonRoot: true

G5 — Generate Default-Deny NetworkPolicy

apiVersion: kyverno.io/v1
kind: ClusterPolicy
metadata:
  name: generate-default-deny
  annotations:
    policies.kyverno.io/category: Network Security
    policies.kyverno.io/description: >-
      Generate a default-deny NetworkPolicy for every Organization-labeled namespace.
      All ingress and egress must be explicitly allowed via additional policies.      
spec:
  rules:
    - name: default-deny-all
      match:
        any:
          - resources:
              kinds:
                - Namespace
              selector:
                matchExpressions:
                  - key: openova.io/organization
                    operator: Exists
      exclude:
        any:
          - resources:
              annotations:
                networkpolicy.openova.io/skip: "true"
      generate:
        synchronize: true
        apiVersion: networking.k8s.io/v1
        kind: NetworkPolicy
        name: default-deny
        namespace: "{{request.object.metadata.name}}"
        data:
          spec:
            podSelector: {}
            policyTypes:
              - Ingress
              - Egress

Skip Annotations Reference

All skip annotations follow the pattern <resource>.openova.io/skip: "true".

Annotation Skips Policy When to Use
vpa.openova.io/skip: "true" G1, G2 Workloads with custom VPA configuration or external VPA controllers
pdb.openova.io/skip: "true" G3, G4 Workloads with custom PDB requirements (e.g., maxUnavailable: 0 for databases)
networkpolicy.openova.io/skip: "true" G5, G6 Namespaces with fully custom network policies managed by Flux
quota.openova.io/skip: "true" G7 Namespaces with externally managed quotas
limitrange.openova.io/skip: "true" G8 Namespaces with custom LimitRange
topology.openova.io/skip: "true" M1 Single-zone clusters, DaemonSet-like workloads, or custom spread constraints
affinity.openova.io/skip: "true" M2 Workloads with hand-crafted affinity rules
securitycontext.openova.io/skip: "true" M3 System components that require specific security contexts
seccomp.openova.io/skip: "true" M4 Workloads requiring custom seccomp profiles
harbor.openova.io/skip: "true" M5 Development namespaces with local image builds
priority.openova.io/skip: "true" M6 Workloads with explicit priority class requirements
sa-token.openova.io/skip: "true" M7 Workloads that legitimately need API server access

Policy Enforcement Modes

Policies are deployed in phases to avoid disrupting existing workloads.

Phase Mode Behavior Duration
1. Observe Audit Log violations, no blocking 2 weeks
2. Warn Audit + webhook warning Admission succeeds but user sees warning 1 week
3. Enforce Enforce Reject non-compliant resources Permanent

New policies always start in Phase 1. The rollout timeline per policy is tracked in the platform's Catalyst console.


Monitoring

Metric Description
kyverno_policy_results_total{rule_result="pass|fail|warn|error|skip"} Policy evaluation results by outcome
kyverno_admission_review_duration_seconds Admission webhook latency (p99 target: <500ms)
kyverno_policy_execution_duration_seconds Individual policy execution time
kyverno_controller_reconcile_total Background controller reconciliation count

Alerting Rules

Alert Condition Severity
KyvernoAdmissionLatencyHigh p99 > 1s for 5m Warning
KyvernoAdmissionLatencyCritical p99 > 3s for 2m Critical
KyvernoPolicyFailuresHigh > 50 failures/min for 5m Warning
KyvernoWebhookDown Webhook endpoint unreachable for 2m Critical
KyvernoBackgroundScanBehind Background scan queue > 1000 for 10m Warning

Kyverno High Availability

Kyverno itself must be resilient. The following configuration is applied to the Kyverno deployment:

Setting Value Reason
Replicas 3 Webhook availability during node failures
PDB minAvailable: 2 At least 2 replicas must survive during drains
TopologySpreadConstraints Zone + Node Spread admission webhooks across failure domains
PriorityClass system-cluster-critical Kyverno must not be preempted
Resource requests 500m CPU, 512Mi memory Sized for ~50 policies with moderate admission volume
Resource limits 2 CPU, 2Gi memory Prevent runaway memory during large generate batches
Anti-affinity Required (not preferred) Hard requirement: no two Kyverno pods on same node

Part of OpenOva