Polycrate Operator

The Polycrate Operator is a Kubernetes operator that synchronizes Ingresses, HTTPRoutes, nodes, backups, TLS certificates, volumes and deployed apps with the Polycrate API. Reference for CRDs, OperatorConfig, discovery controllers, S3Bucket provisioning, cloud controller and DNS webhooks.

Polycrate Operator

The Polycrate Operator is a Kubernetes operator that automatically synchronizes cluster resources with the Polycrate API: Ingresses, HTTPRoutes (Gateway API), Nodes, backups (Velero and CloudNativePG), TLS certificates, volumes and deployed apps are continuously detected and reported to the API.

Overview

The operator watches Kubernetes resources and synchronizes them with the Polycrate API:

┌─────────────────────────────────────────────────────────────────┐
│                    Kubernetes Cluster                           │
│  ┌───────────────────────────────────────────────────────────┐  │
│  │                 polycrate Namespace                        │  │
│  │  ┌─────────────────────────────────────────────────────┐  │  │
│  │  │            Polycrate Operator                        │  │  │
│  │  └─────────────────────────────────────────────────────┘  │  │
│  │           │                                               │  │
│  │           ▼                                               │  │
│  │  ┌─────────────────────────────────────────────────────┐  │  │
│  │  │  Custom Resources (CRs)                              │  │  │
│  │  │  - OperatorConfig                                    │  │  │
│  │  │  - K8sCluster                                        │  │  │
│  │  │  - Endpoint / Host / K8sApp / K8sVolume              │  │  │
│  │  │  - Backup / BackupSchedule / Certificate             │  │  │
│  │  │  - S3Bucket                                          │  │  │
│  │  └─────────────────────────────────────────────────────┘  │  │
│  └───────────────────────────────────────────────────────────┘  │
└─────────────────────────────────────────────────────────────────┘
                              │
                              ▼
                   ┌─────────────────────┐
                   │   Polycrate API     │
                   └─────────────────────┘

Custom Resource Definitions (CRDs)

The operator manages 10 custom resource types:

CRD	Short name	Description
`OperatorConfig`	`oc`, `opcfg`	Operator-wide configuration (discovery, API sync, local cluster)
`K8sCluster`	`k8sc`	Registration of external Kubernetes clusters
`Endpoint`	`ep`, `pep`	Endpoint for HTTP monitoring (created automatically from Ingress or HTTPRoute)
`Host`	—	Node in the cluster (created automatically from Kubernetes nodes)
`K8sApp`	`app`, `pca`	Deployed app (detected from meta secrets or pod annotations)
`K8sVolume`	`vol`, `pvol`	PersistentVolume in the cluster (created automatically from PersistentVolumes)
`Backup`	`backup`	Backup status (Velero and CloudNativePG)
`BackupSchedule`	`bs`	Backup schedule (Velero and CloudNativePG)
`Certificate`	`cert`, `pcert`	TLS certificate (created automatically from cert-manager)
`S3Bucket`	`s3`	S3 bucket (provisioned declaratively via CR, credentials as a Secret)

Installation

1. Install CRDs

Since Polycrate CLI 0.30.0 the CRDs are included in the polycrate-operator block (in the crds/ directory) and are applied automatically by the install playbook. The recommended way is therefore:

# Recommended: CRDs are applied automatically during installation
polycrate run polycrate-operator install

For manual CRD installation (e.g. GitOps):

# Install CRDs directly from the CLI (legacy)
polycrate operator install-crds

# Alternatively: print CRDs and install manually
polycrate operator crds | kubectl apply -f -

# Print only specific CRDs
polycrate operator crds --kind Organization
polycrate operator crds --kind OperatorConfig

# List all available CRDs
polycrate operator crds --list

For GitOps setups where CRDs are managed separately, the automatic installation can be disabled:

# workspace.poly
blocks:
  - name: polycrate-operator
    from: cargo.ayedo.cloud/ayedo/k8s/polycrate-operator
    config:
      crds:
        install: false  # CRDs are deployed separately via GitOps

2. Start the operator

# Run locally (development)
polycrate operator run --kubeconfig ~/.kube/config

# With a specific kubeconfig
polycrate operator run --kubeconfig /path/to/kubeconfig.yml

# With leader election (for HA deployments)
polycrate operator run --leader-elect

Creating resources

K8sCluster

Registers an external Kubernetes cluster:

apiVersion: polycrate.io/v1alpha1
kind: K8sCluster
metadata:
  name: production-cluster
  namespace: polycrate
spec:
  displayName: "Production Cluster"
  apiServer: "https://api.prod.example.com:6443"

  authentication:
    serviceAccountToken:
      secretRef:
        name: prod-cluster-token
        key: token

  # Or via kubeconfig
  # authentication:
  #   kubeconfig:
  #     secretRef:
  #       name: prod-cluster-kubeconfig
  #       key: kubeconfig

  caCertificate: |
    -----BEGIN CERTIFICATE-----
    ...
    -----END CERTIFICATE-----

→ Loopback integration — user perspective

CLI commands

`polycrate operator crds`

Prints CRD definitions:

# Print all CRDs
polycrate operator crds

# Print a specific CRD type
polycrate operator crds --kind Organization

# Save to files
polycrate operator crds --output-dir ./crds/

# List CRDs
polycrate operator crds --list

`polycrate operator install-crds`

Installs CRDs directly into the cluster:

# With the default kubeconfig
polycrate operator install-crds

# With a specific kubeconfig
polycrate operator install-crds --kubeconfig ~/.kube/admin.yaml

# Dry run (shows what would be installed)
polycrate operator install-crds --dry-run

`polycrate operator run`

Starts the operator:

# Locally with kubeconfig
polycrate operator run --kubeconfig ~/.kube/config

# With leader election
polycrate operator run --leader-elect

# With metrics enabled
polycrate operator run --metrics-bind-address :8080

Options:

Flag	Default	Description
`--kubeconfig`	`$KUBECONFIG` or `~/.kube/config`	Path to the kubeconfig
`--leader-elect`	`false`	Enable leader election for HA
`--metrics-bind-address`	`:8080`	Metrics endpoint
`--health-probe-bind-address`	`:8081`	Health probe endpoint

`polycrate operator health`

Checks the health status of the operator:

# Check the local operator
polycrate operator health

# Check a remote operator
polycrate operator health --url http://polycrate-operator.polycrate:8081

`polycrate operator status`

Shows the status of all operator resources:

polycrate operator status

# Output:
# Polycrate Operator Status
# =========================
# Cluster: https://kubernetes.default.svc
#
# K8sClusters
# -----------
#   NAME                NAMESPACE    PHASE
#   production-cluster  polycrate    Ready

Rate limiting & error handling

The operator implements rate limiting via the controller manager:

Base delay: 5 seconds
Max delay: 60 seconds (exponential backoff)
Max concurrent reconciles: configurable via --max-concurrent-reconciles (default: 4 for endpoint monitoring and API agent)

On errors (e.g. API not reachable) the reconciliation is retried with an exponentially growing delay.

OperatorConfig: central discovery configuration

Discovery is configured via an OperatorConfig CR. It is a singleton (name: default, namespace: polycrate).

apiVersion: polycrate.io/v1alpha1
kind: OperatorConfig
metadata:
  name: default
  namespace: polycrate
spec:
  # API sync configuration
  api_sync:
    enabled: true
    api_url: "https://api.polycrate.io"
    credentials_ref:
      secret_name: polycrate-api-creds
      token_key: token
    # workspace_id and organization_id are NO LONGER configured in the spec!
    # These values are derived automatically from the agent token and stored in the status.
    # See "Auto workspace resolution" below.

  # Register the local cluster with the API
  # Ignored if the API already has a cluster for this workspace
  local_cluster:
    enabled: true
    cluster_name: "my-workspace"      # Optional: ignored if the cluster already exists

  # Endpoint discovery (Ingress + HTTPRoute → API)
  endpoint_discovery:
    enabled: true
    # "annotation" (default since 0.38.0): only Ingress/HTTPRoute with polycrate_endpoint_monitor=true
    # "auto": all, except polycrate_endpoint_monitor=false
    # HTTPRoute: on as soon as Gateway API CRDs exist (no extra flag)
    mode: annotation
    watch_namespaces: []  # Empty = all namespaces
    ignore_namespaces:    # These namespaces are ignored
      - kube-system
      - kube-public
      - polycrate
    ingress_classes: []   # Empty = all ingress classes
    gateway_classes: []   # Empty = all GatewayClasses

  # App discovery (meta secrets + pod labels → API)
  app_discovery:
    enabled: true
    watch_namespaces: []
    ignore_namespaces:
      - kube-system
      - kube-public
    label_discovery_enabled: true  # Label-based K8sApp discovery (since 0.29.17)
    stale_cleanup_minutes: 10     # Interval for the stale sweep (since 0.29.17)

  # Node discovery (nodes → API)
  node_discovery:
    enabled: true
    label_selector: ""

  # Backup discovery (Velero → API)
  backup_discovery:
    enabled: true
    velero_namespace: velero
    ignore_namespaces: []
    cnpg_enabled: true

Namespace filtering

Namespace filtering works the same way for all discovery types:

Configuration	Behavior
`watch_namespaces: []`	Watch all namespaces
`watch_namespaces: [prod, staging]`	Only these namespaces
`ignore_namespaces: [kube-system]`	Always ignore these namespaces

Important: ignore_namespaces is subtracted from watch_namespaces. If a namespace is in both lists, it is ignored.

Local cluster registration

The operator can link the cluster it runs in with the Polycrate API:

Idempotency logic:

Checks whether a cluster for this workspace already exists in the API
If yes: the API cluster takes precedence, cluster_name is ignored
If no and cluster_name is set: a new cluster is created
If no and cluster_name is not set: no cluster is created

The resolved cluster ID is stored in OperatorConfig.status.local_cluster for idempotent operations.

Important: the kubeconfig is not synchronized automatically – this is handled by the Git sync of the workspaces.

Auto workspace resolution (since 0.29.0)

Since version 0.29.0 workspace_id and organization_id are derived automatically from the agent token.

How it works:

The operator sends a health report to the API with the configured agent token
The API identifies the associated workspace and organization based on the token
The API returns workspace_id, workspace_name, organization_id and organization_name in the health response
The operator stores these values in OperatorConfig.status.api_sync

Advantages:

No more manual UUID configuration required
Token-based authentication = automatic workspace assignment
Fewer sources of error in the configuration

Status fields:

status:
  api_sync:
    workspace_id: "550e8400-e29b-41d4-a716-446655440000"
    workspace_name: "production"
    organization_id: "6ba7b810-9dad-11d1-80b4-00c04fd430c8"
    organization_name: "acme-corp"
    resolved_at: "2026-01-06T14:30:00Z"

Migration from older versions:

If workspace_id and organization_id were configured in the spec in older versions, these fields can be removed after updating to 0.29.0. The values are derived automatically from the token.

Discovery: operator-driven API synchronization

The operator automatically detects Kubernetes resources and synchronizes them directly with the Polycrate API. Unlike server-side import, the synchronization happens in real time through the operator.

┌──────────────────────┐     ┌─────────────────┐     ┌─────────────────┐
│  K8s Resource        │     │    Operator     │     │  Polycrate API  │
│  Ingress / HTTPRoute │     │   Controller    │     │   (Django)      │
└──────────────────────┘     └─────────────────┘     └─────────────────┘
        │                       │                       │
        │  1. Watch             │                       │
        │◄──────────────────────│                       │
        │                       │  2. Create/Update     │
        │                       │──────────────────────►│
        │  3. Annotate          │  4. Response: id      │
        │  (id, synced-at)      │◄──────────────────────│
        │◄──────────────────────│                       │
        │                       │                       │
        │  [Deletion]           │  5. Delete            │
        │  Finalizer Cleanup    │──────────────────────►│
        │◄──────────────────────│                       │

Detected resources

K8s resource	API target	Discovery controller
Ingress	Endpoint	`EndpointDiscoveryReconciler`
HTTPRoute	Endpoint	`HTTPRouteDiscoveryReconciler` (only if Gateway API CRDs exist; Gateway only queues)
Polycrate meta secret	K8sApp	`K8sAppDiscoveryReconciler`
Pod with `polycrate_k8sapp_name` annotation	K8sApp	`K8sAppLabelDiscoveryReconciler`
Node	Host	`HostDiscoveryReconciler`
Velero Backup	Backup	`BackupDiscoveryReconciler`
Velero Schedule	BackupSchedule	`BackupDiscoveryReconciler`
cert-manager Certificate	Certificate	`CertificateDiscoveryReconciler`
PersistentVolume	K8sVolume	`K8sVolumeDiscoveryReconciler`
S3Bucket CR	S3Bucket	`S3BucketReconciler`

The change was necessary because Kubernetes validates finalizer names as DNS subdomains and no longer accepted values such as `polycrate.io/s3bucket-finalizer`. Finalizers now follow the format `polycrate.io/<resource>-<purpose>` (a correct FQDN), labels the format `polycrate_<key>` (also valid).

→ [Labels & conventions](/docs/polycrate/polycrate-api/labels-conventions/)

API sync and self-healing (since 0.30.4)

The operator automatically keeps discovered resources in line with the API – including self-healing in case of deviations:

Resource	Behavior
Endpoint	API existence check: before every sync the operator checks whether the endpoint still exists in the API. If the API has deleted the endpoint (e.g. by reconciliation or manually), it is automatically recreated.
K8sApp	Adopt existing: if a K8sApp already exists in the API (e.g. through workspace sync and API reconciliation from block instances), the operator adopts its API ID instead of creating a duplicate entry.

Practical effect:

Endpoints do not stay permanently "failed" after API brown-outs or manual deletion – the operator repairs them on the next reconcile.
K8sApp duplicates no longer arise when workspace and API were updated before the operator sync.

Label-based K8sApp discovery (since 0.29.17)

In addition to the existing secret-based detection (Polycrate CLI meta secrets), the operator can also detect pods that carry the annotation polycrate_k8sapp_name. This allows apps that were not deployed via the Polycrate CLI to be registered in the Polycrate API.

Usage

Add the following annotation to pods (or Deployments/StatefulSets):

apiVersion: apps/v1
kind: Deployment
metadata:
  name: my-app
  namespace: my-namespace
spec:
  template:
    metadata:
      annotations:
        polycrate_k8sapp_name: my-app
    spec:
      containers:
        - name: app
          image: my-registry/my-app:latest

Rules

The label value must point uniquely within a namespace to one app
Several pods with the same label value in the same namespace = 1 K8sApp (several replicas)
If the same label value occurs in several namespaces, the app is ignored (cross-namespace conflict)
For apps in several namespaces: use disjoint names (my-app-ns-a, my-app-ns-b)
If an app was also deployed via the Polycrate CLI (meta secret exists), the secret discovery takes precedence

Configuration

app_discovery:
  enabled: true
  label_discovery_enabled: true  # Default: true
  watch_namespaces: []
  ignore_namespaces:
    - kube-system
    - kube-public

Differences to secret-based discovery

Property	Secret-based	Annotation-based
Source	Polycrate CLI meta secret	Pod with `polycrate_k8sapp_name` annotation
Block kind	`polycrate`, `helm`, etc.	`external`
Block version	from the secret	not available
Pod status	all pods in the namespace	only pods with the matching annotation
Cleanup	on secret deletion	when no matching pods exist anymore

Pod status: Job and CronJob pods excluded (since 0.29.16)

The K8sApp pod status does not take pods from Jobs or CronJobs into account – e.g. Velero backup or maintain jobs. These transient pods would distort the metrics (status "Completed"/"Failed"). Only pods from Deployments, StatefulSets and DaemonSets are included in the K8sApp status.

Endpoint–K8sApp association (since 0.29.17, annotation since 0.38.0)

Endpoints from Ingress and HTTPRoute discovery can be assigned to a K8sApp. Since version 0.38.0 an annotation is used for this instead of a label:

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: my-app-ingress
  annotations:
    polycrate_endpoint_monitor: "true"
    polycrate_k8sapp_name: my-app  # Link to K8sApp (annotation, since 0.38.0)
    # Deprecated (label, removed automatically on reconcile):
    # k8sapps.polycrate.io/name: my-app
spec:
  rules:
    - host: my-app.example.com
      http:
        paths:
          - path: /
            pathType: Prefix
            backend:
              service:
                name: my-app
                port:
                  number: 80

The association is stored in the Endpoint CR status and transmitted to the API. This later enables more precise downtime detection per K8sApp.

Bring-your-own-App

Three annotations together — the BYOA block sets them automatically on the Ingress and the HTTPRoute:

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: acme-shop
  annotations:
    polycrate_endpoint_monitor: "true"
    polycrate_k8sapp_name: acme-shop
    polycrate_k8sapp_byoa: "true"
---
apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
  name: acme-shop
  annotations:
    polycrate_endpoint_monitor: "true"
    polycrate_k8sapp_name: acme-shop
    polycrate_k8sapp_byoa: "true"

Annotation	Required for	Effect
`polycrate_endpoint_monitor=true`	Endpoint (mode `annotation`)	One Endpoint CR per host → API checks
`polycrate_k8sapp_name=<name>`	K8sApp	Create a CR `kind=external` or adopt a secret/block app of the same name
`polycrate_k8sapp_byoa=true`	BYOA flag	API `K8sApp.byoa=true` (only on CRs with `discovery-source=ingress` or `httproute`)

Without polycrate_k8sapp_name the endpoint remains unclaimed — monitoring yes, no app downtime. HTTPRoute discovery is on as soon as the Gateway API CRDs are present in the cluster. Same host in the same namespace: HTTPRoute wins the Endpoint CR.

Companion: API Spec 763 (K8sApp.byoa), CLI Spec 292 / 300, Endpoint monitoring.

Stale sweep and resource cleanup (since 0.29.17)

All discovery controllers have a periodic stale sweep that cleans up orphaned custom resources. This is a safety net for scenarios in which source objects are deleted without the operator directly noticing the deletion.

How it works

Active deletion: when a source object (Ingress, HTTPRoute, Secret, etc.) is deleted while the operator is running, the associated CRs are actively deleted immediately
Stale sweep: periodically, every discovery controller checks whether the source objects of its CRs still exist. Orphaned CRs are deleted automatically
Finalizer: the sync controller of a deleted CR ensures the removal of the API object via a finalizer

Configuration

The sweep interval is configurable per discovery controller:

endpoint_discovery:
  stale_cleanup_minutes: 10  # Default: 10

app_discovery:
  stale_cleanup_minutes: 10  # Default: 10

backup_discovery:
  stale_cleanup_minutes: 10  # Default: 10

certificate_discovery:
  stale_cleanup_minutes: 10  # Default: 10

Scope

The stale sweep handles exclusively discovered objects (label polycrate_discovered=true). API-assigned objects (e.g. endpoints configured via the API) have their own lifecycle and are not affected by the stale sweep.

Endpoint discovery: Ingress and HTTPRoute → API endpoint

The operator always watches Ingress, and HTTPRoute as soon as the Gateway API CRDs (gateway.networking.k8s.io/v1, kind HTTPRoute) are present in the cluster. No extra flag: the gate is CRD + endpoint_discovery.enabled. One endpoint per host is created in the Polycrate API for HTTP monitoring (Endpoint monitoring).

The Gateway object is not a source. It re-queues dependent HTTPRoutes (listener TLS/port). GRPCRoute and TLSRoute are not watched.

Same hostname in the same namespace: HTTPRoute wins the Endpoint CR (SourceHTTPRoute); Ingress does not take it back.

HTTPRoute (Gateway API)

Hosts come from spec.hostnames (empty = no endpoint). Path: annotation polycrate_endpoint_path, otherwise the first PathPrefix, otherwise /. TLS and port from the parent gateway listener (parentRefs.sectionName, otherwise the first HTTPS, otherwise HTTP listener). gateway_classes filters via the GatewayClass of the parent (empty = all).

apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
  name: acme-shop
  namespace: acme-shop
  annotations:
    polycrate_endpoint_monitor: "true"
    polycrate_endpoint_path: "/readyz"
    polycrate_k8sapp_name: acme-shop
    polycrate_k8sapp_byoa: "true"
spec:
  parentRefs:
    - name: acme-shop-https
      namespace: envoy-gateway-system
      sectionName: https
  hostnames:
    - shop.acme.example
  rules:
    - matches:
        - path:
            type: PathPrefix
            value: /
      backendRefs:
        - name: acme-shop
          port: 8080

The Ingress examples below apply analogously: the same polycrate_* annotations on the HTTPRoute.

Discovery modes (since 0.38.0)

Since version 0.38.0 the operator supports two discovery modes:

Mode	Behavior
`annotation` (default)	Only Ingress/HTTPRoute with `polycrate_endpoint_monitor: "true"` are processed (opt-in)
`auto`	All Ingress/HTTPRoute are processed unless `polycrate_endpoint_monitor: "false"` is set (opt-out, legacy behavior)

Configuration in OperatorConfig:

endpoint_discovery:
  enabled: true
  mode: annotation  # "annotation" (default) or "auto"
  gateway_classes: []  # Empty = all; filters HTTPRoute via the parent GatewayClass

All available annotations (since 0.38.0)

Annotation (new, canonical)	Deprecated	Type	Default	Description
`polycrate_endpoint_monitor`	(replaces `endpoints.polycrate.io/enabled` + `ignore`)	String	—	`"true"`: enable the monitor (required in `annotation` mode); `"false"`: ignore the Ingress/HTTPRoute
`polycrate_endpoint_path`	`endpoints.polycrate.io/path`	String	`/`	HTTP path for the health check (fallback for all hosts)
`polycrate_endpoint_timeout`	`endpoints.polycrate.io/timeout`	String	`"5"`	Timeout in seconds (fallback for all hosts)
`polycrate_endpoint_interval`	`endpoints.polycrate.io/interval`	String	`"60"`	Check interval in seconds (fallback for all hosts)
`polycrate_endpoint_expected_status`	`endpoints.polycrate.io/expected-status`	String	`"200"`	Expected HTTP status (fallback for all hosts)
`polycrate_endpoint_tls`	`endpoints.polycrate.io/tls`	String	—	TLS configuration (fallback for all hosts)
`polycrate_endpoint_port`	`endpoints.polycrate.io/port`	String	`443`	Port for the health check (fallback for all hosts)
`polycrate_endpoint_host_config`	—	JSON	—	Per-host overrides for all the fields above (see below)
`polycrate_k8sapp_name`	(replaces label `k8sapps.polycrate.io/name`)	String	—	Create a K8sApp or assign to an existing one (CLI ≥ 0.53.0, Spec 292)
`polycrate_k8sapp_byoa`	—	String	—	`"true"`: ingress- or httproute-sourced K8sApp as Bring-your-own-App, API `K8sApp.byoa=true`

Per-host configuration: `polycrate_endpoint_host_config`

Ingresses with multiple hostnames may need different health check parameters per host. The annotation polycrate_endpoint_host_config accepts a JSON object that maps hostnames to individual configurations.

Supported fields per host (all optional — unset fields inherit the ingress-wide fallback value):

Field	Type	Description
`path`	String	HTTP path for the health check
`expected_status`	Array[int]	Expected HTTP status codes
`timeout`	int	Timeout in seconds
`interval`	int	Check interval in seconds
`tls`	bool	Enable/disable TLS
`port`	int	Port for the health check

Behavior:

Hosts that are not listed in host_config receive the ingress-wide values unchanged (polycrate_endpoint_path, polycrate_endpoint_expected_status, etc.)
Fields missing in the host entry also fall back to the ingress-wide fallback
Malformed JSON is logged as a warning; discovery continues and ignores the per-host overrides

Example 1: minimal configuration (annotation mode)

In the default mode (annotation) the Ingress must explicitly opt in:

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: webapp
  namespace: production
  annotations:
    # Opt-in for endpoint monitoring (required in annotation mode)
    polycrate_endpoint_monitor: "true"
spec:
  ingressClassName: nginx
  rules:
    - host: app.example.com
      http:
        paths:
          - path: /
            pathType: Prefix
            backend:
              service:
                name: webapp
                port:
                  number: 80

Result: endpoint with GET / on https://app.example.com:443, timeout 30s, interval 60s.

Example 2: custom health check path

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: api-service
  namespace: production
  annotations:
    polycrate_endpoint_monitor: "true"
    # Health check on /api/health instead of root
    polycrate_endpoint_path: "/api/health"
spec:
  ingressClassName: nginx
  tls:
    - hosts:
        - api.example.com
      secretName: api-tls
  rules:
    - host: api.example.com
      http:
        paths:
          - path: /
            pathType: Prefix
            backend:
              service:
                name: api
                port:
                  number: 8080

Result: endpoint with GET /api/health on https://api.example.com:443.

Example 3: critical service with a short interval

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: payment-gateway
  namespace: production
  annotations:
    polycrate_endpoint_monitor: "true"
    # Critical service: more frequent checks
    polycrate_endpoint_path: "/health/ready"
    polycrate_endpoint_interval: "30"
    polycrate_endpoint_timeout: "10"
    polycrate_endpoint_expected_status: "200"
spec:
  ingressClassName: nginx
  tls:
    - hosts:
        - pay.example.com
      secretName: payment-tls
  rules:
    - host: pay.example.com
      http:
        paths:
          - path: /
            pathType: Prefix
            backend:
              service:
                name: payment-api
                port:
                  number: 443

Result: endpoint with a 30s interval (instead of 60s), 10s timeout, expects HTTP 200.

Example 4: body validation for a deep health check

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: database-api
  namespace: production
  annotations:
    polycrate_endpoint_monitor: "true"
    polycrate_endpoint_path: "/health/deep"
    polycrate_endpoint_expected_status: "200"
    # Checks whether "database":"connected" is contained in the response body
    endpoints.polycrate.io/expected-body: '"database":"connected"'
spec:
  ingressClassName: nginx
  tls:
    - hosts:
        - db-api.example.com
      secretName: db-api-tls
  rules:
    - host: db-api.example.com
      http:
        paths:
          - path: /
            pathType: Prefix
            backend:
              service:
                name: db-api
                port:
                  number: 8080

Result: the check only succeeds if HTTP 200 AND the body contains "database":"connected".

Example 5: exclude an Ingress from discovery

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: internal-admin
  namespace: kube-system
  annotations:
    # Do NOT monitor this Ingress (effective in both modes)
    polycrate_endpoint_monitor: "false"
spec:
  ingressClassName: nginx
  rules:
    - host: admin.internal.local
      http:
        paths:
          - path: /
            pathType: Prefix
            backend:
              service:
                name: admin-dashboard
                port:
                  number: 80

Result: no endpoint is created. Useful for internal services or test environments.

Example 6: slow backend service

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: report-generator
  namespace: production
  annotations:
    # Report generation takes longer
    polycrate_endpoint_monitor: "true"
    polycrate_endpoint_path: "/api/health"
    polycrate_endpoint_timeout: "120"
    polycrate_endpoint_interval: "300"
spec:
  ingressClassName: nginx
  tls:
    - hosts:
        - reports.example.com
      secretName: reports-tls
  rules:
    - host: reports.example.com
      http:
        paths:
          - path: /
            pathType: Prefix
            backend:
              service:
                name: report-service
                port:
                  number: 8080

Result: 120s timeout, 5-minute interval for slow services.

Example 7: redirect-capable endpoint (allow 3xx)

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: legacy-app
  namespace: production
  annotations:
    polycrate_endpoint_monitor: "true"
    polycrate_endpoint_path: "/"
    # Redirect is OK (e.g. HTTP→HTTPS or /→/login)
    polycrate_endpoint_expected_status: "302"
spec:
  ingressClassName: nginx
  rules:
    - host: legacy.example.com
      http:
        paths:
          - path: /
            pathType: Prefix
            backend:
              service:
                name: legacy-app
                port:
                  number: 80

Example 8: complete production setup

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: main-api
  namespace: production
  labels:
    app: main-api
    environment: production
  annotations:
    # === Polycrate endpoint discovery ===
    polycrate_endpoint_monitor: "true"
    polycrate_endpoint_path: "/api/v1/health"
    polycrate_endpoint_timeout: "15"
    polycrate_endpoint_interval: "30"
    polycrate_endpoint_expected_status: "200"

    # === Nginx Ingress Controller ===
    nginx.ingress.kubernetes.io/proxy-body-size: "50m"
    nginx.ingress.kubernetes.io/proxy-read-timeout: "60"

    # === cert-manager ===
    cert-manager.io/cluster-issuer: "letsencrypt-prod"
spec:
  ingressClassName: nginx
  tls:
    - hosts:
        - api.mycompany.com
      secretName: api-tls-cert
  rules:
    - host: api.mycompany.com
      http:
        paths:
          - path: /api
            pathType: Prefix
            backend:
              service:
                name: api-service
                port:
                  number: 8080
          - path: /docs
            pathType: Prefix
            backend:
              service:
                name: docs-service
                port:
                  number: 80

Example 9: different health check paths per hostname

An Ingress with multiple hostnames whose backend services have different health endpoints:

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: multi-service-platform
  namespace: production
  annotations:
    polycrate_endpoint_monitor: "true"
    # Fallback: applies to all hosts without an entry in host_config
    polycrate_endpoint_path: "/health"
    polycrate_endpoint_expected_status: "200"
    # Per-host overrides
    polycrate_endpoint_host_config: |
      {
        "api.example.com": {
          "path": "/api/health",
          "expected_status": [200]
        },
        "events.example.com": {
          "path": "/health",
          "expected_status": [202]
        },
        "data.example.com": {
          "path": "/readyz",
          "expected_status": [200],
          "timeout": 10
        }
      }
spec:
  ingressClassName: nginx
  tls:
    - hosts:
        - api.example.com
        - events.example.com
        - data.example.com
        - booking.example.com
      secretName: platform-tls
  rules:
    - host: api.example.com
      http:
        paths:
          - path: /
            pathType: Prefix
            backend:
              service:
                name: api-service
                port:
                  number: 8080
    - host: events.example.com
      http:
        paths:
          - path: /
            pathType: Prefix
            backend:
              service:
                name: events-service
                port:
                  number: 80
    - host: data.example.com
      http:
        paths:
          - path: /
            pathType: Prefix
            backend:
              service:
                name: data-service
                port:
                  number: 8080
    - host: booking.example.com
      http:
        paths:
          - path: /
            pathType: Prefix
            backend:
              service:
                name: booking-service
                port:
                  number: 80

Result: the operator creates four Endpoint CRs with different configurations:

Hostname	Path	Expected status	Timeout
`api.example.com`	`/api/health`	200	5s (fallback)
`events.example.com`	`/health`	202	5s (fallback)
`data.example.com`	`/readyz`	200	10s (override)
`booking.example.com`	`/health`	200	5s (fallback)

booking.example.com has no entry in host_config and therefore automatically inherits path: /health and expected_status: 200 from the ingress-wide annotations.

Endpoint custom resource

For every hostname in an Ingress or HTTPRoute the operator creates a separate Endpoint custom resource:

apiVersion: polycrate.io/v1alpha1
kind: Endpoint
metadata:
  name: app-example-com
  namespace: production
  labels:
    endpoints.polycrate.io/source-ingress: webapp
    polycrate_discovered: "true"
spec:
  displayName: "app.example.com"
  remoteAddress: "app.example.com"
  remotePort: 443
  kind: "http"
  tls: true
  path: "/api/health"
  timeout: 30
  interval: 60
  expectedStatusCodes: [200]
status:
  phase: "synced"
  apiID: "550e8400-e29b-41d4-a716-446655440000"
  configHash: "abc123..."

Advantages of the CRD model:

Clear 1:1 mapping: every hostname = one CR = one API ID
Garbage collection: OwnerReferences enable automatic cleanup
Easy debugging: kubectl get endpoints.polycrate.io shows all endpoints

# Show all Endpoint CRs
kubectl get endpoints.polycrate.io -A

# Details of an Endpoint CR
kubectl describe endpoint app-example-com -n production

Endpoint monitoring: operator → API → agent

The operator is the first step in the monitoring pipeline. After synchronization with the API, monitoring agents take over the actual HTTP checks.

Architecture overview

┌─────────────────────────────────────────────────────────────────┐
│                    Kubernetes Cluster                           │
│  ┌─────────────────────────────────────────────────────────────┐│
│  │  Ingress / HTTPRoute      Polycrate Operator                ││
│  │  + Annotations    ───►    Endpoint Discovery                ││
│  │                           + API Sync                        ││
│  └─────────────────────────────────────────────────────────────┘│
└─────────────────────────────────────────────────────────────────┘
        │                              │
        │                              ▼
        │                     ┌─────────────────┐
        │                     │  Polycrate API  │
        │                     │  - Endpoints    │
        │                     │  - Agent assign.│
        │                     └─────────────────┘
        │                              │
        │                              ▼
        │                     ┌─────────────────┐
        │◄────────────────────│ Monitoring Agent│
        │   HTTP Check        │ - Fetch Config  │
        │                     │ - Execute Check │
        │                     │ - Prometheus    │
        └─────────────────────┴─────────────────┘

Data flow in detail

Step	Component	Action
1	Ingress / HTTPRoute	Annotation `polycrate_endpoint_path=/health` set
2	Operator	Detects the source, creates an Endpoint CRD with `spec.path`
3	Operator	Synchronizes the CRD with the API (`spec.http.path`)
4	API	Stores the endpoint, assigns agents
5	Agent	Queries assigned endpoints (incl. `spec`)
6	Agent	Builds the check URL: `https://host:443/health`
7	Agent	Performs the HTTP check, exposes Prometheus metrics

Propagating changes

When you change an annotation, the change is propagated automatically:

# Change the annotation
kubectl annotate ingress my-app \
  polycrate_endpoint_path="/api/v2/health" \
  --overwrite
# analogous: kubectl annotate httproute my-app polycrate_endpoint_path="/api/v2/health" --overwrite

Propagation chain:

Immediately: Kubernetes triggers operator reconciliation
Immediately: the operator detects the changed path, updates the CRD
Immediately: the CRD change triggers the API sync
~60s: the agent fetches the new configuration on its next refresh
Next check: the agent checks against the new URL

Prometheus metrics

The monitoring agents expose check results as Prometheus metrics:

# Endpoint availability
probe_success{endpoint="api-example-com", organization="acme-corp"}

# Response time in seconds
probe_duration_seconds{endpoint="api-example-com"}

# HTTP status code
probe_http_status_code{endpoint="api-example-com"}

These metrics are scraped by VictoriaMetrics/Prometheus and are available for alerting and dashboards.

→ Detailed agent documentation: Polycrate API - Endpoint monitoring

Finalizer: automatic cleanup

The operator sets a finalizer on all discovered resources:

metadata:
  finalizers:
    - endpoints.polycrate.io/discovery-cleanup            # Ingress
    - endpoints.polycrate.io/httproute-discovery-cleanup  # HTTPRoute

Workflow on deletion:

The user deletes the Ingress or HTTPRoute
Kubernetes sets deletionTimestamp on the source
The operator detects the deletion and performs the cleanup: - Deletes the endpoint from the Polycrate API via DELETE /api/v1/endpoints/{id}/
The operator removes the finalizer
Kubernetes deletes the source object

This ensures that API objects always stay in sync with their K8s sources.

K8sApp discovery: meta secrets → CR → API

After an action, the CLI often writes a meta secret. The operator creates a K8sApp CR from it and syncs it to the API (PatchK8sApp). The API model K8sAppInstance has been removed (API 0.32.0) — not to be confused with the block kind k8sappinstance.

Details (CRD example, label discovery, adopt): K8sApp discovery: deployed blocks → API and Label-based discovery.

Host discovery: nodes → API

Kubernetes nodes are automatically captured as hosts in the API.

Mapped fields

Node field	Host field	Source
`metadata.name`	`name`, `hostname`	Node name
`status.addresses[ExternalIP]`	`default_ipv4`	Node IP
`status.capacity.cpu`	`resource_cpu_cores`	CPU cores
`status.capacity.memory`	`resource_memory`	Memory (MB)
`status.nodeInfo.architecture`	`resource_cpu_architecture`	Architecture
`spec.providerID`	`provider_id`	Cloud provider ID

Backup discovery

The operator synchronizes backups and schedules from Velero and (since CLI/operator 0.49.0) CloudNativePG (CNPG) into the Polycrate API. Each entry includes, among others, kind, provider and source_namespace.

Sources

Source	Detected resources
Velero	`Backup`, `Schedule`
CloudNativePG	`Backup`, `ScheduledBackup`

Backup mapping (Velero)

Velero Backup	API Backup	Mapping
`metadata.name`	`name`	Direct
`status.phase`	`status`	Completed→completed, Failed→failed
`spec.ttl`	`retention_policy`	TTL as a string
`status.startTimestamp`	`started_at`	ISO 8601
`status.completionTimestamp`	`completed_at`	ISO 8601
`status.progress.itemsBackedUp`	`items_backed_up`	Integer

Schedule mapping (Velero)

Velero Schedule	API BackupSchedule	Mapping
`metadata.name`	`name`	Direct
`spec.schedule`	`schedule_cron`	Cron expression
`spec.paused`	`paused`	Boolean
`status.lastBackup.startTime`	`last_backup_at`	Timestamp
`status.lastBackup.name`	`last_backup_name`	String

CNPG objects are registered analogously as backups or schedules with provider cloudnative-pg (or the detected provider).

Certificate discovery: cert-manager → API

The operator watches cert-manager Certificate CRs and synchronizes them with the Polycrate API for central TLS certificate management.

Prerequisites

cert-manager must be installed in the cluster
The operator automatically detects whether cert-manager CRDs are present
If cert-manager is not installed, the CertificateDiscoveryController is skipped

Architecture

┌─────────────────────────────────────────────────────────────────┐
│                    Kubernetes Cluster                            │
│                                                                  │
│  ┌──────────────────┐     ┌─────────────────────────────────┐   │
│  │ cert-manager     │     │      Polycrate Operator          │   │
│  │ Certificate CR   │────►│  CertificateDiscoveryController  │   │
│  │ (cert-manager.io)│     │           ▼                      │   │
│  └──────────────────┘     │  polycrate.io/Certificate CR     │   │
│                           │           ▼                      │   │
│                           │  CertificateController           │   │
│                           │           │                      │   │
│                           └───────────┼──────────────────────┘   │
└───────────────────────────────────────┼──────────────────────────┘
                                        │
                                        ▼
                               ┌─────────────────┐
                               │  Polycrate API  │
                               │  /certificates/ │
                               └─────────────────┘

Two-controller architecture

Controller	Watches	Creates/updates
`CertificateDiscoveryController`	cert-manager `Certificate`	polycrate.io `Certificate` CR
`CertificateController`	polycrate.io `Certificate`	API object via REST

Advantages:

CRD-based: certificates are visible as Kubernetes resources (kubectl get certificates.polycrate.io)
Finalizer cleanup: automatic API deletion on K8s deletion
Change detection: hash-based change detection prevents unnecessary API updates
Inventory check: API objects are checked for existence on every reconcile

Certificate CRD

For every cert-manager Certificate the operator creates a polycrate.io/Certificate CR:

apiVersion: polycrate.io/v1alpha1
kind: Certificate
metadata:
  name: certmanager-production-api-example-com-tls
  namespace: production
  labels:
    certificates.polycrate.io/issuer: letsencrypt-production
    certificates.polycrate.io/source: api-example-com-tls
    certificates.polycrate.io/ready: "true"
    operator.polycrate.io/discovered: "true"
spec:
  name: api-example-com-tls
  display_name: "api.example.com-tls"
  namespace: production
  secret_name: api-example-com-tls
  dns_names:
    - api.example.com
  issuer_name: letsencrypt-production
  issuer_kind: clusterissuer
  issuer_group: cert-manager.io
  certificate_status: ready
  is_ready: true
  not_before: "2024-01-01T00:00:00Z"
  not_after: "2024-04-01T00:00:00Z"
  renewal_time: "2024-03-01T00:00:00Z"
  source_cert_manager:
    name: api-example-com-tls
    namespace: production
    uid: "abc-123-..."
    revision: 1
status:
  phase: Synced
  sync_status: synced
  api_id: "550e8400-e29b-41d4-a716-446655440000"
  config_hash: "abc123..."
  last_synced_at: "2024-01-15T10:30:00Z"

API mapping

cert-manager Certificate	Polycrate Certificate CR	API Certificate
`metadata.name`	`spec.source_cert_manager.name`	- (reference)
`spec.secretName`	`spec.secret_name`	`secret_name`
`spec.dnsNames`	`spec.dns_names`	`dns_names`
`spec.ipAddresses`	`spec.ip_addresses`	`ip_addresses`
`spec.issuerRef.name`	`spec.issuer_name`	`issuer_name`
`spec.issuerRef.kind`	`spec.issuer_kind`	`issuer_kind`
`status.notBefore`	`spec.not_before`	`not_before`
`status.notAfter`	`spec.not_after`	`not_after`
`status.renewalTime`	`spec.renewal_time`	`renewal_time`
Ready condition	`spec.is_ready`, `spec.certificate_status`	`is_ready`, `certificate_status`

Status mapping

cert-manager status	`certificate_status`
Ready=True	`ready`
Ready=False, Reason=Issuing	`pending`
Ready=False, Reason=Failed	`failed`
Ready=True, NotAfter < now	`expired`
Ready=True, NotAfter < now+30d	`expiring_soon`

OperatorConfig: certificate discovery

apiVersion: polycrate.io/v1alpha1
kind: OperatorConfig
metadata:
  name: default
  namespace: polycrate
spec:
  # ... other configuration ...

  # Certificate discovery (cert-manager → API)
  certificate_discovery:
    enabled: true
    watch_namespaces: []      # Empty = all namespaces
    ignore_namespaces:        # Ignore these namespaces
      - kube-system
      - cert-manager
    issuer_filter:            # Optional: only specific issuers
      include_issuers: []     # Empty = all issuers
      exclude_issuers:        # Ignore these issuers
        - selfsigned-issuer

CLI commands

# Show all Certificate CRs
kubectl get certificates.polycrate.io -A

# Show certificate details
kubectl describe certificate.polycrate.io certmanager-production-api-example-com-tls -n production

# Check the API sync status
kubectl get certificates.polycrate.io -A -o custom-columns=\
  NAME:.metadata.name,\
  NAMESPACE:.metadata.namespace,\
  ISSUER:.spec.issuer_name,\
  READY:.spec.is_ready,\
  STATUS:.spec.certificate_status,\
  SYNC:.status.sync_status,\
  API_ID:.status.api_id

Usage in the Polycrate API

The synchronized certificates are available in the Polycrate API under /api/v1/certificates/:

Certificates from the cluster are visible in the API
Warnings on status expiring_soon
Assignment to endpoints and K8sClusters

K8sApp discovery: deployed blocks → API

The operator automatically detects deployed Polycrate blocks based on the meta secrets (*.poly) and creates K8sApp custom resources.

Architecture

┌─────────────────────────────────────────────────────────────────┐
│                    Kubernetes Cluster                            │
│                                                                  │
│  ┌──────────────────┐     ┌─────────────────────────────────┐   │
│  │ Meta-Secret      │     │      Polycrate Operator          │   │
│  │ (*.poly)         │────►│  K8sAppDiscoveryController       │   │
│  │                  │     │           ▼                      │   │
│  └──────────────────┘     │  polycrate.io/K8sApp CR          │   │
│                           │           ▼                      │   │
│                           │  K8sAppController                │   │
│                           │           │                      │   │
│                           └───────────┼──────────────────────┘   │
└───────────────────────────────────────┼──────────────────────────┘
                                        │
                                        ▼
                               ┌─────────────────┐
                               │  Polycrate API  │
                               │  /k8sapps/      │
                               └─────────────────┘

K8sApp CRD

For every meta secret the operator creates a K8sApp CR:

apiVersion: polycrate.io/v1alpha1
kind: K8sApp
metadata:
  name: nginx-ingress
  namespace: production
  labels:
    k8sapps.polycrate.io/block: nginx-ingress
    k8sapps.polycrate.io/kind: helm
    k8sapps.polycrate.io/type: ingress-controller
    operator.polycrate.io/discovered: "true"
  ownerReferences:
    - apiVersion: v1
      kind: Secret
      name: nginx-ingress.poly
      uid: abc-123-...
spec:
  block:
    name: "nginx-ingress"
    display_name: "NGINX Ingress Controller"
    version: "1.2.0"
    app_version: "1.9.6"
    from: "cargo.ayedo.cloud/ayedo/k8s/nginx-ingress"
    type: "ingress-controller"
    flavor: "nginx"
    chart:
      name: "ingress-nginx"
      version: "4.9.0"
      repo_url: "https://kubernetes.github.io/ingress-nginx"
  workspace:
    name: "production"
    organization: "acme-corp"
  action:
    name: "install"
    status: "success"
    last_executed_at: "2025-12-20T10:00:00Z"
status:
  phase: Synced
  installed: true
  api_id: "550e8400-e29b-41d4-a716-446655440000"
  sync_status: synced
  artifacts:
    - name: "cargo-ayedo-cloud-library-nginx-1-25-3"
      image: "cargo.ayedo.cloud/library/nginx:1.25.3"
      pod_count: 3
  artifact_count: 2

CLI commands

# Show all K8sApps
kubectl get k8sapps -A

# Output:
# NAMESPACE   NAME             BLOCK            VERSION   ARTIFACTS   PHASE    AGE
# prod        nginx-ingress    nginx-ingress    1.2.0     3           Synced   7d
# prod        redis            redis            7.2.0     1           Synced   14d

# Show details
kubectl describe k8sapp nginx-ingress -n prod

Installation status derivation

The operator derives the installation status from the last action run:

action_name	action_status	Result
`install`	`success`	`installed=true`
`install`	`running`	`installation_running=true`
`install`	`failed`	`installation_failed=true`
`uninstall`	`success`	`uninstalled=true`

Block ID linking

During API synchronization the block ID is resolved automatically:

Find the workspace from spec.workspace.name + spec.workspace.organization
Find the block from spec.block.name in the workspace
Send the K8sApp with the block ID to the API

This enables full correlation between K8sApp ↔ block ↔ workspace in the API.

S3Bucket provisioning: CR → API → Secret (since 0.35.0)

The operator can provision S3 buckets declaratively via custom resources. A Kubernetes user creates an S3Bucket CR — the operator creates the bucket in the Polycrate API and stores the access credentials as a Secret in the namespace.

Usage

apiVersion: polycrate.io/v1alpha1
kind: S3Bucket
metadata:
  name: my-app-data
  namespace: production
spec:
  name: my-app-data
  region: fsn1
  reclaim_policy: retain

Field	Required	Default	Description
`spec.name`	Yes	-	Name of the S3 bucket
`spec.region`	No	`default_s3_bucket_cluster` from SystemConfig	Region string (must correspond to an S3Cluster in the API)
`spec.reclaim_policy`	No	`retain`	`retain` = the bucket remains on CR deletion, `delete` = the bucket is deleted

Credential Secret

The operator automatically creates a Secret s3-<bucket-name>-credentials in the namespace of the CR:

apiVersion: v1
kind: Secret
metadata:
  name: s3-my-app-data-credentials
  namespace: production
  labels:
    polycrate_managed_by: operator
    polycrate_object_type: s3-credentials
    polycrate_s3_bucket_name: my-app-data
    polycrate_s3_bucket_region: fsn1
  ownerReferences:
    - apiVersion: polycrate.io/v1alpha1
      kind: S3Bucket
      name: my-app-data
type: Opaque
stringData:
  AWS_ACCESS_KEY_ID: "<access-key>"
  AWS_SECRET_ACCESS_KEY: "<secret-key>"
  S3_ENDPOINT: "https://s3.fsn1.example.com"
  S3_REGION: "fsn1"
  BUCKET_NAME: "my-app-data"

The Secret is bound to the CR via ownerReferences — it is removed automatically when the CR is deleted.

Status phases

Phase	Meaning
`Pending`	CR created, processing pending
`Provisioning`	Bucket is being created in the API
`WaitingForCredentials`	Bucket exists, credentials are being fetched
`Ready`	Bucket and Secret are available
`Error`	Error during provisioning or credential retrieval

Adopt pattern

If a bucket with the given name and region already exists in the API, it is adopted. The operator only creates the credential Secret — the bucket is not created twice.

Configuration

apiVersion: polycrate.io/v1alpha1
kind: OperatorConfig
metadata:
  name: default
  namespace: polycrate
spec:
  s3_bucket_provisioning:
    enabled: true  # Default: true (since 0.35.0)

Default region

If no region is specified in the CR, the operator uses the default_s3_bucket_cluster setting from the Polycrate API SystemConfig. It can be configured in the dashboard under System Settings.

Cloud controller (CCM-lite)

The operator can act as the only cloud controller in the cluster (no parallel provider CCM such as hcloud-ccm). Opt-in via OperatorConfig.spec.cloud_controller.

Semantics

Capability	Behavior
Node init	After a successful host API sync: `providerID=polycrate://<clusterID>/<node>`, label `topology.kubernetes.io/region` = `status.agent.pop_name`, remove taint `node.cloudprovider.kubernetes.io/uninitialized`. Without `pop_name`: requeue, taint remains. No automatic deletion of nodes.
LoadBalancer	`Service` of type `LoadBalancer` → Polycrate API `LoadbalancerInstance` (NodeIP:NodePort). The public IP is polled and written to `Service.status.loadBalancer.ingress`.

Cluster prerequisites

kubelet with --cloud-provider=external
No other CCM in parallel for the same nodes/load balancers

Configuration

apiVersion: polycrate.io/v1alpha1
kind: OperatorConfig
metadata:
  name: default
  namespace: polycrate
spec:
  cloud_controller:
    enabled: true
    load_balancer:
      enabled: true
      # all (default): all Services type=LoadBalancer except polycrate.io/loadbalancer=ignore
      # opt-in: only Services with polycrate.io/loadbalancer=managed
      service_scope: all

Annotation	Effect
`polycrate.io/loadbalancer=ignore`	Service is not managed (`service_scope: all`)
`polycrate.io/loadbalancer=managed`	Service is managed (`service_scope: opt-in`)

DNS01 webhook (cert-manager)

The operator can embed the cert-manager DNS01 challenge webhook (no separate binary). Challenges write TXT records via the Polycrate DNS API (ResolveDNSZone + DNSRecord CRUD).

Scope

DNS01 webhook = Present/CleanUp for ACME challenges
Certificate discovery (see above) = inventory of cert-manager Certificate CRs → API — independent of the webhook

Configuration

apiVersion: polycrate.io/v1alpha1
kind: OperatorConfig
metadata:
  name: default
  namespace: polycrate
spec:
  dns01_webhook:
    enabled: true
    group_name: webhook.dns.polycrate.io
    solver_name: polycrate
    port: 10250

Default: enabled: false (opt-in). When enabled, the operator block deploys the TLS Secret, APIService and RBAC (see the block templates).

ClusterIssuer example

apiVersion: cert-manager.io/v1
kind: ClusterIssuer
metadata:
  name: letsencrypt-dns
spec:
  acme:
    server: https://acme-v02.api.letsencrypt.org/directory
    privateKeySecretRef:
      name: letsencrypt-dns-account
    solvers:
      - dns01:
          webhook:
            groupName: webhook.dns.polycrate.io
            solverName: polycrate

Zone resolution: the API prefers kind=internal, otherwise an external zone with primary_zone=true. The agent token needs DNS permissions (read zone, CRUD records).

external-dns webhook

The operator provides an external-dns webhook provider (HTTP, default port 8888). external-dns remains a separate deployment; sources, policy and ownership are the responsibility of external-dns.

Operator configuration

apiVersion: polycrate.io/v1alpha1
kind: OperatorConfig
metadata:
  name: default
  namespace: polycrate
spec:
  external_dns_webhook:
    enabled: true   # Default: true
    port: 8888

DomainFilter and record CRUD use the Polycrate DNS API in the scope of the agent token (all visible zones of the organization). The webhook is reachable via ClusterIP; the NetworkPolicy allows ingress on port 8888 from the entire cluster (no label/namespace coupling to the external-dns installation).

external-dns deployment example

apiVersion: apps/v1
kind: Deployment
metadata:
  name: external-dns
  namespace: external-dns
spec:
  selector:
    matchLabels:
      app: external-dns
  template:
    metadata:
      labels:
        app: external-dns
    spec:
      serviceAccountName: external-dns
      containers:
        - name: external-dns
          image: registry.k8s.io/external-dns/external-dns:v0.18.0
          args:
            - --source=ingress
            - --provider=webhook
            - --webhook-provider-url=http://polycrate-operator.polycrate.svc:8888
            - --txt-owner-id=my-cluster
            - --policy=upsert-only

Flag	Note
`--provider=webhook`	Required — the provider is the operator webhook
`--webhook-provider-url`	ClusterIP service of the operator (port from `external_dns_webhook.port`)
`--txt-owner-id`	Core logic of external-dns — set uniquely per cluster so that records of other writers (e.g. DNS01 `_acme-challenge`) are not deleted
`--source` / `--policy`	external-dns only; not part of the OperatorConfig

Polycrate Operator

Polycrate Operator

Overview

Custom Resource Definitions (CRDs)

Installation

1. Install CRDs

2. Start the operator

Creating resources

K8sCluster

CLI commands

polycrate operator crds

polycrate operator install-crds

polycrate operator run

polycrate operator health

polycrate operator status

Rate limiting & error handling

OperatorConfig: central discovery configuration

Namespace filtering

Local cluster registration

Auto workspace resolution (since 0.29.0)

Discovery: operator-driven API synchronization

Detected resources

API sync and self-healing (since 0.30.4)

Label-based K8sApp discovery (since 0.29.17)

Usage

Rules

Configuration

Differences to secret-based discovery

Pod status: Job and CronJob pods excluded (since 0.29.16)

Endpoint–K8sApp association (since 0.29.17, annotation since 0.38.0)

Bring-your-own-App

Stale sweep and resource cleanup (since 0.29.17)

How it works

Configuration

Scope

Endpoint discovery: Ingress and HTTPRoute → API endpoint

HTTPRoute (Gateway API)

Discovery modes (since 0.38.0)

All available annotations (since 0.38.0)

Per-host configuration: polycrate_endpoint_host_config

Example 1: minimal configuration (annotation mode)

Example 2: custom health check path

Example 3: critical service with a short interval

Example 4: body validation for a deep health check

Example 5: exclude an Ingress from discovery

Example 6: slow backend service

Example 7: redirect-capable endpoint (allow 3xx)

Example 8: complete production setup

Example 9: different health check paths per hostname

Endpoint custom resource

Endpoint monitoring: operator → API → agent

Architecture overview

Data flow in detail

Propagating changes

Prometheus metrics

Finalizer: automatic cleanup

K8sApp discovery: meta secrets → CR → API

Host discovery: nodes → API

Mapped fields

Backup discovery

Sources

Backup mapping (Velero)

Schedule mapping (Velero)

Certificate discovery: cert-manager → API

Prerequisites

Architecture

Two-controller architecture

Certificate CRD

API mapping

Status mapping

OperatorConfig: certificate discovery

CLI commands

Usage in the Polycrate API

K8sApp discovery: deployed blocks → API

Architecture

K8sApp CRD

CLI commands

Installation status derivation

Block ID linking

S3Bucket provisioning: CR → API → Secret (since 0.35.0)

`polycrate operator crds`

`polycrate operator install-crds`

`polycrate operator run`

`polycrate operator health`

`polycrate operator status`

Per-host configuration: `polycrate_endpoint_host_config`