openova/platform/stalwart-tenant
e3mrah a1ca1872aa
feat(bp-stalwart-tenant): wire Keycloak OIDC SSO end-to-end (#915) (#920)
Closes the C2 sub-task of EPIC #915 — alice's Stalwart authenticates
SMTP/IMAP/JMAP/webmail logins against her per-tenant Keycloak realm,
not a shared otech-level IdP.

Three layered changes (matching the three things broken on otech103):

1. Orchestrator (`smeTenantBPStalwart` in sme_tenant_gitops.go)
   now emits per-tenant OIDC values matching the bp-wordpress-tenant
   + bp-openclaw shape:
     keycloak.realmURL = https://keycloak.<sub>.<parent>/realms/sme-<sub>
     keycloak.clientID = stalwart
     keycloak.clientSecretName = stalwart-oidc-client-secret
     keycloak.oidcExternalSecret.remoteRef.key
       = sovereign/<otech-fqdn>/stalwart/<tenant>/oidc
   plus admin externalSecret + dependsOn bp-keycloak so the SME's
   three apps (wordpress, openclaw, stalwart) SSO against ONE realm
   with distinct client IDs (#915 C1 registers all three in the realm
   bootstrap).

2. Chart bootstrap config.toml drops the pre-0.16 kebab-case
   `[directory.keycloak] type = "oidc"` block (silently ignored by
   the upstream registry parser — verified against
   crates/registry/src/schema/structs.rs in stalwartlabs/stalwart;
   OidcDirectory serdes camelCase: `@type = "Oidc"`, `issuerUrl`,
   `claimUsername`, `claimName`, `claimGroups`, `requireScopes`).
   The `internal` directory stays as the bootstrap fallback so the
   admin can log in before the post-install Job seeds OIDC.

3. setupJob defaults to enabled (was off in 0.1.1) and POSTs the
   canonical OIDC directory entry to `/api/settings`:
     directory.keycloak.@type            = "Oidc"
     directory.keycloak.issuerUrl        = <realm URL>
     directory.keycloak.claimUsername    = preferred_username
     directory.keycloak.claimName        = name
     directory.keycloak.claimGroups      = groups
     directory.keycloak.requireScopes    = [openid email profile groups]
     directory.keycloak.usernameDomain   = <tenant domain>
     storage.directory                   = keycloak
   The setting POSTs are idempotent (`assert_empty: false`) so Helm
   upgrades re-run without breaking existing logins. Re-uses the
   upstream Stalwart container (ships curl + stalwart-cli) — no new
   image needed.

Tests:
  - `chart/tests/oidc-render.sh` (NEW): asserts every settings key
    is rendered, the [oauth] env block propagates the per-tenant
    realm URL, and the bootstrap config.toml parses as valid TOML.
  - `chart/tests/expression-syntax.sh`: re-passes (Stalwart
    expression `==` audit per stalwart_expression_syntax.md).
  - `TestRenderSMETenantOverlay_StalwartEmitsKeycloakOIDC` (NEW):
    Go test verifies the orchestrator emits the per-tenant realm
    URL, client metadata, and ExternalSecret-store remoteRef paths.
  - All existing TestRenderSMETenantOverlay_* tests pass.
  - `helm template` clean with default values AND with a per-tenant
    overlay (--api-versions external-secrets.io/v1beta1).

Chart bumps 0.1.1 → 0.1.2; blueprint.yaml spec.version mirrors per
issue #817 (chart/blueprint version invariant).

Co-authored-by: hatiyildiz <hatice@openova.io>
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-05 13:37:46 +04:00
..
chart feat(bp-stalwart-tenant): wire Keycloak OIDC SSO end-to-end (#915) (#920) 2026-05-05 13:37:46 +04:00
blueprint.yaml feat(bp-stalwart-tenant): wire Keycloak OIDC SSO end-to-end (#915) (#920) 2026-05-05 13:37:46 +04:00
README.md feat(bp-stalwart-tenant): per-SME dedicated mail server v0.1.0 (#801) (#815) 2026-05-04 22:22:46 +04:00

bp-stalwart-tenant

Per-SME (per-vcluster) dedicated Stalwart mail server. Implements locked decision [Q3] of EPIC #795 — every SME on a Sovereign gets its own Stalwart in its tenant namespace, with its own domain, own MTA reputation, and own queue.

Status: v0.1.0 (Application Blueprint, scratch chart) | Updated: 2026-05-04 (#801)

NOT the same as the otech-shared openova.io Stalwart in openova-private/clusters/contabo-mkt/apps/stalwart/ — that is the OpenOva-corp mail server. This Blueprint is the per-SME-tenant mail server that ships inside each SME vcluster.


Why per-tenant (and the trade-off)

Locked in #795: founder explicitly chose this over a shared otech-level multi-domain Stalwart. The trade buys:

  • Stronger isolation — one SME's deliverability problem doesn't affect another SME's MTA reputation.
  • Per-customer DKIM — each SME signs with their own key on their own domain.
  • Per-customer queue — bounce-floods, blocklist hits, rate-limit pushes from one SME stay in their queue.

Cost: mail-server resources multiply by N tenants. Each install = 1 small StatefulSet (100m / 256Mi requests) + 1 PVC (default 20Gi). #795 trade-off table tracks this.


What ships

Resource Purpose
StatefulSet Stalwart pod, single replica, RocksDB on PVC
Service (×3) LoadBalancer for SMTP/submission/submissions, LoadBalancer for IMAP/IMAPS, ClusterIP for webmail/JMAP
HTTPRoute or Ingress webmail UI at mail.<domain> (Cilium Gateway by default; Traefik fallback)
ConfigMap (config) Stalwart bootstrap config.toml — applied when RocksDB is empty
ConfigMap (dns-records-required) MX/SPF/DKIM/DMARC the SME admin must publish — surfaced by unified-rbac UI
ExternalSecret (admin) Pulls Stalwart admin password from OpenBao
ExternalSecret (oidc) Pulls Keycloak client secret from OpenBao
Job (post-install) Bootstraps admin principal + send-allow row (idempotent)
NetworkPolicy Default-deny + explicit allows for SMTP/IMAP/webmail/Keycloak/PowerDNS/DNS/outbound SMTP
ServiceAccount Identity for the Stalwart pod and the setup Job

SSO via SME-vcluster Keycloak

The Stalwart webmail authenticates users against the SME's per-vcluster Keycloak realm — NOT the otech-level Keycloak.

The OIDC client stalwart is registered in the SME realm at vcluster provisioning time (handled by #804 — tenant provisioning pipeline). The client secret is written to OpenBao at the canonical path:

sovereign/<sovereign-fqdn>/stalwart/<tenant>/oidc → property OIDC_CLIENT_SECRET

The chart's oidc-externalsecret.yaml pulls it down into the SME tenant namespace.

Per-user mailbox provisioning is event-driven (per ADR-0003 §3): when the SME admin creates a user via the unified-rbac console, the unified-rbac service POSTs Stalwart's /api/principal admin API to create the mailbox. This chart ships only the bootstrap admin principal in the post-install Job — it does not loop on the NATS subject by default. Per-tenant overlays may flip mailboxProvisioner.natsSubscriber.enabled=true once the SME vcluster's NATS subject is wired.


Domain modes

Free-subdomain mode (default)

Operator overlay sets domain.primary: <slug>.<otech-fqdn> (e.g. acme.omantel.omani.works). The chart records the required DNS records in the *-dns-records-required ConfigMap and a follow-up controller (in unified-rbac) posts them to the otech PowerDNS API.

BYO domain mode

Operator overlay sets domain.primary: acme.com and domain.mode: byo. The records ConfigMap is still emitted; the unified-rbac console UI surfaces them to the SME admin to paste into their public DNS provider. Smoke test in #804 asserts the records are reachable post-creation.


Required DNS records (rendered into the ConfigMap)

Kind Name Value template
MX <domain> priority 10 → mail.<domain>
TXT <domain> v=spf1 mx <policy> (default -all = hard fail)
TXT <selector>._domainkey.<domain> v=DKIM1; k=ed25519; p=<DKIM-PUBLIC-KEY> (the public-key blob is stamped in by the unified-rbac controller after first-boot DKIM mint)
TXT _dmarc.<domain> v=DMARC1; p=reject; rua=mailto:dmarc@<domain> (operator-tunable)

Stalwart config.toml gotchas

The bootstrap config.toml follows the pattern committed by the openova-private contabo-mkt Stalwart, with two memory-recorded gotchas:

  1. == not = in expression matchers (queue routing, sieve conditions, send-allow expressions). A single = is assignment and silently never matches (incident 2026-04-14, huawei.com TLS rule). Every comparison in templates/config-configmap.yaml uses ==. Per-tenant overlays adding queue-routing rules MUST follow the same convention. See stalwart_expression_syntax.md memory.

  2. Group principals need explicit email-receive — Stalwart group principals do NOT inherit email-receive from the default user role. Without it, every inbound email to the group bounces with 550 5.5.0 This account is not authorized to receive email. (incident 2026-04-20). The post-install Job's PATCH on the admin principal is the canonical fix; future shared-mailbox additions in tenant overlays MUST PATCH the same field. See stalwart_send_as.md memory.

The bootstrap config.toml is applied only once — when RocksDB is empty (first install). Subsequent runtime config edits via webadmin or stalwart-cli persist in RocksDB and do not sync back to the ConfigMap. For disaster recovery, snapshot the running configuration via stalwart-cli server list-config and re-render this ConfigMap.


Inbound spam filtering

Disabled by default per the founder directive on the corp Stalwart (feedback_no_spam_filtering.md memory) — accept everything, filter at the client. Per-SME deployments inherit the same posture; individual SMEs may opt in via webadmin runtime config.


Required values (per-tenant overlay)

# clusters/<sovereign>/sme-overlays/<tenant>/stalwart.yaml
domain:
  primary: "acme.omantel.omani.works"   # or "acme.com" for BYO
  mode: "free-subdomain"                 # or "byo"

keycloak:
  realmURL: "https://auth.acme.omantel.omani.works/realms/sme"
  clientID: "stalwart"
  clientSecretName: "stalwart-oidc"
  oidcExternalSecret:
    remoteRef:
      key: "sovereign/omantel.omani.works/stalwart/acme/oidc"

admin:
  externalSecret:
    remoteRef:
      key: "sovereign/omantel.omani.works/stalwart/acme/admin"

dns:
  powerdns:
    enabled: true
    apiURL: "https://pdns.omantel.omani.works/api"
    apiKeySecretName: "powerdns-api-key"
    zone: "omantel.omani.works"
  dmarc:
    rua: "dmarc@acme.omantel.omani.works"

Capacity

Default per-tenant: 100m / 256Mi requests, 1 CPU / 1Gi limits, 20Gi PVC. Roughly 50 mailboxes / 5 GB mail spool comfortably; bump stalwart.resources and persistence.spool.size per-tenant for larger SMEs. Single replica per tenant — Stalwart RocksDB is single-writer by design at this tier.


  • EPIC #795 — SME-tenant turnkey experience
  • #796 — Hook contract (ADR-0003)
  • #802 — Unified RBAC SME-tier (consumes the dns-records ConfigMap)
  • #804 — Tenant provisioning pipeline (registers OIDC client + writes secrets)
  • #805 — End-to-end demo

Part of OpenOva