talent-mesh 435f49738d feat: restructure platform to 52 components and 9 products

Technology forecast and strategic review restructure:
- Remove 13 components (backstage, mongodb, activemq, vitess, airflow, camel, dapr, superset, searxng, langserve, trino, lago, rabbitmq)
- Add 10 components (sigstore, syft-grype, nemo-guardrails, langfuse, reloader, matrix, ferretdb, litmus, livekit, coraza)
- Rename product: Synapse → Axon (SaaS LLM Gateway)
- Merge products: Titan + Fuse → Fabric (Data & Integration)
- New product: Relay (Communication)
- Replace Backstage with Catalyst IDP
- Replace MongoDB with FerretDB (MongoDB wire protocol on CNPG)
- Add supply chain security (Sigstore/Cosign, Syft+Grype)
- Add AI safety and observability (NeMo Guardrails, LangFuse)
- Add technology forecast 2027-2030 document
- Full verification pass: zero stale references across all docs

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

2026-02-26 21:00:19 +00:00

3.3 KiB

Raw Blame History

KEDA

Event-driven horizontal autoscaling for OpenOva platform.

Status: Accepted | Updated: 2026-01-16

Overview

KEDA (Kubernetes Event-driven Autoscaling) provides horizontal pod autoscaling based on external metrics and events:

Queue-based scaling (Kafka via Strimzi)
Metric-based scaling (Prometheus, custom metrics)
Cron-based scaling
Scale-to-zero capability

Architecture

flowchart TB
    subgraph KEDA["KEDA"]
        Operator[KEDA Operator]
        Metrics[Metrics Adapter]
    end

    subgraph Sources["Event Sources"]
        Kafka[Kafka]
        Prometheus[Prometheus/Mimir]
        Cron[Cron]
    end

    subgraph Workloads["Workloads"]
        Deploy[Deployments]
        Pods[Pods]
    end

    Sources --> Operator
    Operator --> Deploy
    Deploy --> Pods
    Metrics --> Operator

Scalers

Scaler	Use Case
kafka	Kafka consumer lag
prometheus	Custom metrics
cron	Time-based scaling
cpu/memory	Resource utilization

Configuration

ScaledObject

apiVersion: keda.sh/v1alpha1
kind: ScaledObject
metadata:
  name: <tenant>-worker
  namespace: <tenant>
spec:
  scaleTargetRef:
    name: <tenant>-worker
  minReplicaCount: 1
  maxReplicaCount: 10
  cooldownPeriod: 300
  triggers:
    - type: kafka
      metadata:
        bootstrapServers: kafka-kafka-bootstrap.databases.svc:9092
        consumerGroup: <tenant>-workers
        topic: <tenant>-jobs
        lagThreshold: "100"

Prometheus Scaler

apiVersion: keda.sh/v1alpha1
kind: ScaledObject
metadata:
  name: <tenant>-api
  namespace: <tenant>
spec:
  scaleTargetRef:
    name: <tenant>-api
  minReplicaCount: 2
  maxReplicaCount: 20
  triggers:
    - type: prometheus
      metadata:
        serverAddress: http://mimir.monitoring.svc:8080/prometheus
        metricName: http_requests_per_second
        query: |
          sum(rate(http_requests_total{namespace="<tenant>"}[1m]))          
        threshold: "100"

Cron Scaler

apiVersion: keda.sh/v1alpha1
kind: ScaledObject
metadata:
  name: <tenant>-batch
  namespace: <tenant>
spec:
  scaleTargetRef:
    name: <tenant>-batch
  minReplicaCount: 0
  maxReplicaCount: 5
  triggers:
    - type: cron
      metadata:
        timezone: UTC
        start: "0 8 * * 1-5"
        end: "0 18 * * 1-5"
        desiredReplicas: "3"

VPA + KEDA Coordination

flowchart LR
    subgraph Scaling["Scaling"]
        VPA[VPA<br/>Vertical]
        KEDA[KEDA<br/>Horizontal]
    end

    subgraph Workload["Workload"]
        Deploy[Deployment]
        Pods[Pods]
    end

    VPA -->|"Right-size resources"| Pods
    KEDA -->|"Scale replicas"| Deploy
    Deploy --> Pods

VPA: Optimizes CPU/memory per pod
KEDA: Scales replica count based on events
Combined: Optimal resource utilization with event-driven elasticity

Scale-to-Zero

KEDA supports scaling to zero for batch workloads:

spec:
  minReplicaCount: 0  # Allow scale-to-zero
  idleReplicaCount: 0  # Scale to zero when idle

Monitoring

Metric	Description
`keda_scaler_active`	Whether scaler is active
`keda_scaler_metrics_value`	Current metric value
`keda_scaled_object_errors`	Scaling errors

Part of OpenOva

3.3 KiB Raw Blame History