Helm Charts¶

The Kamerplanter Helm chart is based on the bjw-s common library and defines all Kubernetes resources in a single chart. Container images and the chart itself are published as OCI artifacts on the GitHub Container Registry.

Registry overview¶

Chart information¶

name: kamerplanter
type: application
version: 0.2.1-dev      # Chart version in the develop tree — pre-release channel
appVersion: "1.0.0"     # Application version

Dependencies¶

Chart structure¶

helm/kamerplanter/
├── Chart.yaml            # Chart metadata and dependencies
├── Chart.lock            # Pinned dependency versions
├── values.yaml           # Default values (production)
├── values-dev.yaml       # Override for development
├── templates/
│   └── common.yaml       # bjw-s library loader
└── charts/
    ├── common-5.2.1.tgz  # bjw-s Common Library
    ├── valkey-0.12.0.tgz # Valkey sub-chart
    └── ollama-1.87.0.tgz # Ollama sub-chart (off by default)

The chart uses the bjw-s common.loader.all approach: all Kubernetes resources (Deployments, StatefulSets, Services, ConfigMaps, Ingress) are defined declaratively via values.yaml — there are no custom templates.

Configuration reference¶

Controllers (Deployments & StatefulSets)¶

Backend¶

controllers:
  backend:
    type: deployment
    replicas: 1                    # Chart default; typically raised to 2 for production (see Kubernetes Deployment)
    strategy: RollingUpdate
    containers:
      main:
        image:
          repository: ghcr.io/nolte/kamerplanter-backend
          tag: 0.2.1@sha256:af9bec…    # immutable digest, written by the release job — see "Pin a specific image version"
        envFrom:
          - secret: kamerplanter-secrets    # ARANGODB_PASSWORD, JWT_SECRET_KEY, FERNET_KEY, ERASURE_TOMBSTONE_SALT, LOG_PSEUDONYM_SALT
        env:
          ARANGODB_HOST: "..."
          ARANGODB_PORT: "8529"
          ARANGODB_DATABASE: "kamerplanter"
          ARANGODB_USERNAME: '{{ .Values.database.arangodb.appUsername }}'    # kamerplanter
          REDIS_URL: "redis://kamerplanter-valkey:6379/0"
          CORS_ORIGINS: '["..."]'
          DEBUG: "false"
          KAMERPLANTER_MODE: "light"    # or "full" (chart default)
        resources:
          requests:
            cpu: 250m
            memory: 512Mi
          limits:
            cpu: "1"
            memory: 2Gi                 # Image uploads (REQ-034) decode in memory — 512Mi OOMKills on upload

Celery worker¶

The worker processes three queues (issue #2128):

The chart starts the worker with -Q critical,celery,bulk. A queue no worker reads keeps its tasks in Valkey forever — without any error. If you override the worker args in your values, carry the queue list over completely or drop -Q altogether (the worker then reads every queue the application declares).

One worker per queue¶

One switch gives every queue a worker of its own:

celeryWorkers:
  dedicatedQueues: true

Without the switch (the default) the one celery-worker reads all three queues. A long run in bulk then occupies one of its two processes; the second keeps working. With the switch, a dataset or image acquisition can no longer hold up a reminder, an erasure or a frost warning at all.

The two additional workers are copies of celery-worker: the chart makes them at render time from the merged values. Whatever you set on controllers.celery-worker in an overlay, via --set or in an ArgoCD valuesObject (image, environment, securityContext, probes, resources, pod options) and on networkpolicies.celery-worker (egress) therefore applies to the copies too. Every policy that admits celery-worker as a peer (ArangoDB, Valkey, inference-service) admits the copies as well, and every volume of the worker is mounted into the copies. Only the queue differs. To tune one copy, set controllers.<copy>.<key>, e.g. controllers.celery-worker-bulk.containers.main.resources; the key is merged over that copy. Without the switch there are no copies and such keys have no effect.

If a profile switches celery-worker off (the ki and recognition overlays), no copies are created either. If you override the worker args yourself, the rule still holds: every queue needs at least one worker that reads it.

Acknowledged after the run. Clean-up runs whose second pass finds nothing left (expired tokens, consents, audit entries, erasure records, IP anonymisation, expired overrides) acknowledge their message only once they are done. If a worker pod disappears mid-run, Valkey hands the message out again after the visibility timeout (just over four hours) instead of losing the run until the next day. Notifications, the actuator control loop and erasure runs still acknowledge on start — a second run there would notify or switch twice.

Readiness (issue #2154). The worker counts as ready only once it takes tasks: with WORKER_READY_FILE it checks its ArangoDB login at start (for up to 60 seconds), writes the file as soon as its consumer runs, and the startup and readiness probes test it (test -f, which costs next to nothing). If a new worker cannot log in after an upgrade — wrong password, application account not provisioned — it exits, never becomes ready, and the rollout keeps the old worker. The liveness probe asks its own worker with inspect ping -d "celery@${HOSTNAME}"; an inspect ping without a destination succeeds as long as any worker answers.

Every task has a time limit as a backstop: 30 minutes (soft, the task can clean up), after 35 minutes the process is ended. Runs that are resumable or started on purpose (erasure runs, data export, storage migration, dataset acquisition) get three hours. Each worker process now reserves one message ahead instead of four.

Frontend¶

controllers:
  frontend:
    type: deployment
    replicas: 1                    # Chart default; typically raised to 2 for production
    containers:
      main:
        image:
          repository: ghcr.io/nolte/kamerplanter-frontend
          tag: latest@sha256:fea5a3…
        resources:
          requests:
            cpu: 100m
            memory: 128Mi
          limits:
            cpu: 500m
            memory: 256Mi

The frontend is served behind nginx. The nginx configuration is automatically mounted as a ConfigMap and proxies /api/ requests to the backend.

ArangoDB¶

controllers:
  arangodb:
    type: statefulset
    replicas: 1                    # Single node (no cluster)
    containers:
      main:
        image:
          repository: arangodb
          tag: "3.12.9"
        env:
          ARANGO_ROOT_PASSWORD:             # only ARANGO_ROOT_PASSWORD, not the whole Secret
            valueFrom:
              secretKeyRef:
                name: '{{ .Values.database.arangodb.rootPasswordSecret }}'
                key: ARANGO_ROOT_PASSWORD
                optional: true
        resources:
          requests:
            cpu: 250m
            memory: 512Mi
          limits:
            cpu: "1"
            memory: 1Gi
    statefulset:
      volumeClaimTemplates:
        - name: data
          accessMode: ReadWriteOnce
          size: 5Gi                 # Adjustable
          advancedMounts:
            main:
              - path: /var/lib/arangodb3

Services¶

service:
  backend:
    controller: backend
    ports:
      http:
        port: 8000
  frontend:
    controller: frontend
    ports:
      http:
        port: 80          # Service port stays 80 (Ingress/DNS)
        targetPort: 8080  # nginx-unprivileged container listens on 8080, not 80
  arangodb:
    controller: arangodb
    ports:
      http:
        port: 8529

Ingress¶

ingress:
  main:
    enabled: true                   # without hosts no Ingress is created
    hosts:
      - host: plants.example.com
        paths:
          - path: /              # /api too: the frontend nginx proxies it to the backend
            pathType: Prefix
            service:
              identifier: frontend

Do not route /api straight to the backend Service: the backend NetworkPolicy admits only the frontend on port 8000, and the frontend nginx applies the upload limit and the X-Forwarded-For entry that TRUSTED_PROXY_HOPS: "1" is set for.

ingress:
  main:
    enabled: true
    annotations:
      cert-manager.io/cluster-issuer: letsencrypt-prod
    hosts:
      - host: plants.example.com
        paths: [...]
    tls:
      - secretName: kamerplanter-tls
        hosts:
          - plants.example.com

Upload limit (request body)¶

The frontend nginx accepts request bodies up to storage.maxFileSizeMb + 1 MiB (default: 26m). The extra MiB covers the multipart envelope of an upload: an attachment of the full allowed size reaches the backend, a slightly larger one gets a clear 413 answer from there, and anything above the limit is refused by nginx itself with 413. Without this setting the limit was nginx's default of 1 MiB, and every larger upload failed at the proxy.

The chart sets the same limit on the ingress main as the annotation nginx.ingress.kubernetes.io/proxy-body-size, because ingress-nginx would otherwise cut at 1 MiB too. Both follow storage.maxFileSizeMb: when you raise the upload limit, nginx and the ingress grow with it. Traefik and Contour/Envoy do not read the annotation and set no body limit of their own; there the frontend nginx's limit applies, as long as / and /api go through the frontend. If you use an ingress under another key than main or your own HTTPProxy, set the limit there yourself to the same value.

The backend enforces the same limit again before it reads an upload. That matters when a request bypasses the frontend nginx — via kubectl port-forward, from inside the cluster or through an ingress without a body limit. An upload with a larger Content-Length gets 413 (REQUEST_BODY_TOO_LARGE) without the backend reading a byte; an upload without a declared length is cut off at the limit. Every other request (JSON, forms) may be at most 1 MiB.

While reading, the backend keeps uploads above 1 MiB in /tmp, a memory volume of 64 MiB in the chart. So that concurrent uploads cannot fill it, they share a budget of 52 MiB (UPLOAD_SPOOL_BUDGET_MB). It counts the bytes that already arrived: at the default limit two full uploads run at the same time, a further one waits up to 30 seconds (UPLOAD_SPOOL_WAIT_SECONDS) and then gets 503 with Retry-After. If you raise storage.maxFileSizeMb above 51, the chart sets the budget to one full upload, so only one large upload runs at a time. The size of /tmp does not follow automatically: from an upload limit of about 60 MiB you must raise persistence.backend-tmp.sizeLimit yourself.

Valkey (Redis-compatible cache)¶

Valkey is both cache and Celery broker: the tasks wait in it for the worker. The chart ships these defaults for it:

valkey:
  deploymentStrategy: Recreate
  dataStorage:
    enabled: true
    requestedSize: 1Gi
    keepPvc: true
  valkeyConfig: |
    maxmemory 256mb
    maxmemory-policy noeviction
    appendonly yes
    appendfsync everysec
  resources:
    requests:
      cpu: 50m
      memory: 128Mi
    limits:
      memory: 640Mi    # at least twice maxmemory

• Own volume: Valkey keeps its data on a PersistentVolumeClaim (kamerplanter-valkey, 1 GiB, ReadWriteOnce). It survives helm uninstall. The Valkey chart calls the size requestedSize — it does not read a size key.
• Recreate: the old Valkey pod ends before the new one starts. A ReadWriteOnce volume attaches to one node; a new pod next to the old one could not reach the volume from another node.
• noeviction rather than an eviction policy: when memory is full, Valkey refuses new writes instead of silently dropping waiting tasks. A full memory thus shows up as an error.
• appendonly yes records every change (written to disk once per second); after a restart the tasks of the last seconds are back too.
• Memory limit at least twice maxmemory (default 640 MiB for 256 MB): to rewrite the append-only file and to take snapshots, Valkey forks a child process that copies the data set; every page that changes meanwhile occupies memory twice. On top come the rewrite buffer and the client buffers, which do not count against maxmemory. With a tight limit, Kubernetes kills Valkey before Valkey itself can refuse. If you raise maxmemory, raise resources.limits.memory with it — the chart's render check requires at least twice.
• Watch the memory: with noeviction a full Valkey reports errors (OOM command not allowed) instead of losing tasks — backend and worker can then no longer queue tasks. Set up an alert on used_memory relative to maxmemory (for example from 80 %) so you can step in first.

Environment variables¶

Full list of every mandatory secret per enabled feature: Configuration Matrix — Mandatory secrets.

Development overrides (values-dev.yaml)¶

A separate values file exists for local development, used automatically by Skaffold:

Common customizations¶

Reduce resources (small cluster / Raspberry Pi)¶

controllers:
  backend:
    replicas: 1
    containers:
      main:
        resources:
          requests:
            cpu: 100m
            memory: 128Mi
          limits:
            cpu: 500m
            memory: 256Mi
  frontend:
    replicas: 1
    containers:
      main:
        resources:
          requests:
            cpu: 50m
            memory: 64Mi
          limits:
            cpu: 250m
            memory: 128Mi
  arangodb:
    containers:
      main:
        resources:
          requests:
            cpu: 100m
            memory: 256Mi
          limits:
            cpu: 500m
            memory: 512Mi

Pin a specific image version¶

A published chart already ships pinned digests (the release job writes them at packaging time) — override them only if you want a version other than the one the chart carries. In that case, override them completely, digest included:

controllers:
  backend:
    containers:
      main:
        image:
          tag: "1.2.3@sha256:c6689b…"    # (1)!
  frontend:
    containers:
      main:
        image:
          tag: "1.2.3@sha256:6727d2…"

1. Resolve the digest with: docker buildx imagetools inspect ghcr.io/nolte/kamerplanter-backend:1.2.3

Storage Configuration (NFR-013)¶

Kamerplanter stores all binary data (photos, imports, exports) through an interchangeable storage adapter. You choose the backend and the associated Kubernetes persistence with the storage block in your values: the chart derives the STORAGE_* variables of the backend and the Celery worker, the PVC and its mounts from it.

Local Filesystem (Default)¶

In the default setup, the chart automatically creates the PVC backend-attachments and mounts it in the backend and Celery worker pods at /data/attachments. The PVC carries helm.sh/resource-policy: keep and the ArgoCD sync option Prune=false,Delete=false: neither helm uninstall nor an ArgoCD prune (for example after switching to S3) nor deleting the ArgoCD Application deletes the attachments.

storage:
  backend: local-fs                  # Default; no external storage needed
  maxFileSizeMb: 25
  presignTtlSeconds: 900
  virusScan:
    enabled: false
    endpoint: ""

  localFs:
    root: /data/attachments           # Container-internal mount path
    pvc:
      size: 20Gi                      # Chart default; increase as needed
      accessMode: ReadWriteOnce       # For single-replica (default)
      storageClass: ""                # Empty = cluster default
    singleNode: false                 # true only on a cluster with exactly one node

Multi-replica operation (backend or worker replicas > 1) without S3:

storage:
  localFs:
    pvc:
      accessMode: ReadWriteMany       # Requires an RWX-capable StorageClass
      storageClass: longhorn          # Or: nfs, cephfs, etc.

kubectl create secret generic kamerplanter-storage-signing \
  --from-literal=STORAGE_LOCALFS_SIGNING_SECRET="$(openssl rand -hex 32)" \
  --namespace kamerplanter

controllers:
  backend:
    containers:
      main:
        envFrom:
          - secretRef:
              name: kamerplanter-storage-signing

S3-compatible (Production)¶

Non-secret S3 parameters are set directly in values.yaml. Credentials come exclusively from a Secret — ideally produced by the External Secrets Operator (ESO), never as plain text in Git. The chart reads exactly the two keys named in credentialsRef (secretKeyRef), not the whole Secret. With storage.backend: s3 the chart renders no PVC and no mount.

storage:
  backend: s3
  maxFileSizeMb: 25
  presignTtlSeconds: 900

  s3:
    endpointUrl: https://s3.eu-central-1.amazonaws.com
    region: eu-central-1
    bucket: kamerplanter-prod
    usePathStyle: false               # true for MinIO and non-AWS providers
    forceTls: true
    kmsKeyId: ""                      # Optional: customer-managed key (SSE-KMS)

    # S3 credentials via External Secrets Operator (NEVER plain text)
    credentialsRef:
      secretName: storage-s3-credentials
      accessKeyIdKey: STORAGE_S3_ACCESS_KEY_ID
      secretAccessKeyKey: STORAGE_S3_SECRET_ACCESS_KEY

External Secrets Operator — ESO Secret:

apiVersion: external-secrets.io/v1beta1
kind: ExternalSecret
metadata:
  name: storage-s3-credentials
  namespace: kamerplanter
spec:
  refreshInterval: 1h
  secretStoreRef:
    name: vault-backend              # Or AWS Secrets Manager, etc.
    kind: ClusterSecretStore
  target:
    name: storage-s3-credentials
    creationPolicy: Owner
  data:
    - secretKey: STORAGE_S3_ACCESS_KEY_ID
      remoteRef:
        key: kamerplanter/storage
        property: access_key_id
    - secretKey: STORAGE_S3_SECRET_ACCESS_KEY
      remoteRef:
        key: kamerplanter/storage
        property: secret_access_key

kubectl create secret generic storage-s3-credentials \
  --from-literal=STORAGE_S3_ACCESS_KEY_ID="your-access-key" \
  --from-literal=STORAGE_S3_SECRET_ACCESS_KEY="your-secret-key" \
  --namespace kamerplanter

NetworkPolicy for S3 Endpoints¶

There is no separate storage NetworkPolicy. The backend and the Celery worker reach a public S3 endpoint through their general egress rule (networkpolicies.backend and networkpolicies.celery-worker: ports 80/443 to 0.0.0.0/0 without the RFC1918 ranges and without 169.254.0.0/16 — the cloud metadata address stays blocked). An endpoint inside the cluster or on a private network (MinIO, Ceph RGW) is therefore not reachable: add an egress rule for exactly that endpoint to both policies in your values.

MinIO In-Cluster¶

storage:
  backend: s3
  s3:
    endpointUrl: http://minio.kamerplanter.svc:9000
    region: us-east-1
    bucket: kamerplanter
    usePathStyle: true
    forceTls: false
    allowPrivateEndpoint: true       # Permits a non-publicly-reachable endpoint
    credentialsRef:
      secretName: storage-s3-credentials
      accessKeyIdKey: STORAGE_S3_ACCESS_KEY_ID
      secretAccessKeyKey: STORAGE_S3_SECRET_ACCESS_KEY

Egress for notification channels (Apprise)¶

Kamerplanter checks Apprise targets when saving and again when sending (scheme allow-list, literal hosts, resolved addresses of Gotify, Matrix and ntfy servers). Two things lie outside the process and are up to the operator's network policy:

• Apprise follows HTTP redirects and performs Matrix .well-known discovery itself; neither can be controlled in the backend.
• A small window remains between the DNS check and the actual connection (DNS rebinding).

The DNS resolution behind this check is bounded so that a user with host names that never answer cannot slow others down: saving and sending each have their own pool of 8 threads, one user holds at most 3 of them at a time, and an answer must arrive within 3 s. Anything beyond that is refused at once (fail closed), not queued. When saving, answers are cached briefly per host name (60 s, a failure 10 s); when sending, Kamerplanter resolves fresh every time and takes only a refusal from the cache. The values are built in; there is no setting for them.

Targets in private networks (10.0.0.0/8, 172.16.0.0/12, 192.168.0.0/16, 100.64.0.0/10 and fc00::/7) are refused by default in full mode, when saving and when sending. Inside a cluster these are not home-network devices but the API server, ArangoDB and Valkey on pod IPs. APPRISE_ALLOW_PRIVATE_TARGETS=true allows them; in light mode they are allowed without any setting, and false refuses them there too. Loopback, link-local and cloud metadata stay refused in every case.

The chart therefore already restricts egress for backend and celery-worker (networkpolicies.backend and networkpolicies.celery-worker in values.yaml): destination 0.0.0.0/0 excluding 10.0.0.0/8, 172.16.0.0/12, 192.168.0.0/16 and 169.254.0.0/16, on ports 80, 443, 465 and 587 only. This keeps cloud metadata and in-cluster targets out of reach even if a redirect leads there. 100.64.0.0/10 is not in that exception list; if your cluster's pods live there, only the backend check protects them.

Outbound connections to Home Assistant and InvenTree¶

Kamerplanter does not only check the Home Assistant and InvenTree URL when it is saved: every connection checks the addresses it opens to and connects only to one of them. A DNS answer that changes between the check and the connection (DNS rebinding) therefore does not lead to an internal address. Cloud metadata and link-local stay refused in every case, private addresses only with HA_ALLOW_PRIVATE_ENDPOINT or INVENTREE_ALLOW_PRIVATE_ENDPOINT. None of these connections follows a redirect, and neither does Web Push.

Virus Scanning (optional)¶

storage:
  virusScan:
    enabled: true
    endpoint: http://clamav-rest.kamerplanter.svc:9000

ClamAV must run as a separate deployment in the cluster. The backend blocks an upload when the scanner reports a finding.

Common Provider Configurations¶

storage:
  backend: s3
  s3:
    endpointUrl: https://fsn1.your-objectstorage.com
    region: eu-central
    bucket: my-kamerplanter-bucket
    usePathStyle: false
    forceTls: true
    credentialsRef:
      secretName: storage-s3-credentials
      accessKeyIdKey: STORAGE_S3_ACCESS_KEY_ID
      secretAccessKeyKey: STORAGE_S3_SECRET_ACCESS_KEY

storage:
  backend: s3
  s3:
    endpointUrl: https://<account-id>.r2.cloudflarestorage.com
    region: auto
    bucket: kamerplanter
    usePathStyle: false
    forceTls: true
    credentialsRef:
      secretName: storage-s3-credentials
      accessKeyIdKey: STORAGE_S3_ACCESS_KEY_ID
      secretAccessKeyKey: STORAGE_S3_SECRET_ACCESS_KEY

storage:
  backend: s3
  s3:
    endpointUrl: https://s3.eu-central-003.backblazeb2.com
    region: eu-central-003
    bucket: kamerplanter
    usePathStyle: false
    forceTls: true
    credentialsRef:
      secretName: storage-s3-credentials
      accessKeyIdKey: STORAGE_S3_ACCESS_KEY_ID
      secretAccessKeyKey: STORAGE_S3_SECRET_ACCESS_KEY

Metrics (Prometheus)¶

One switch: monitoring.enabled: true. The chart then sets METRICS_PORT=9464 on the backend, adds the Service port metrics, renders a ServiceMonitor and the NetworkPolicy backend-metrics, which admits exactly the Prometheus pods to that port.

monitoring:
  enabled: true
  scrapeInterval: 30s
  prometheus:
    namespace: monitoring          # namespace Prometheus runs in
    podName: prometheus            # value of the Prometheus pods' app.kubernetes.io/name label
    serviceMonitorLabels:
      release: kube-prometheus-stack   # what the operator's serviceMonitorSelector expects

The metrics are not public: they run on a port of their own, never as a route of the API. The ingress sends / and /api to the frontend nginx, which forwards to port 8000 only. The backend policy still admits only the frontend on 8000, so Prometheus cannot use the scrape path to reach the API around nginx.

Served are http_request_duration_seconds (histogram) and http_requests_total with the labels method, handler (route pattern, never the path; unmatched for a request no route matched) and status, http_requests_in_progress, and process metrics (memory, CPU, file descriptors). A tenant is deliberately not a label — it would multiply every series by the tenant count and disclose the tenant list to anyone allowed to scrape. For the per-tenant view, log lines carry tenant=ten_….

Also served are the Celery queue lengths: celery_queue_length with the label queue (critical, celery, bulk) counts the tasks waiting in Valkey for a worker, and celery_broker_up is 1 when the last read of Valkey succeeded. The backend reads the lengths at scrape time, not on a request, and at most every 5 seconds. When Valkey is unreachable, the queue values are absent from that scrape (no stale value) and celery_broker_up is 0; the scrape itself does not fail. Every backend replica reports the same numbers — aggregate them with max, not sum.

Not included yet: metrics from inside the Celery worker (run times, failures per task), and the recording and alerting rules (PrometheusRule) of NFR-007.

Scaling (HPA and PDB)¶

Without further settings every component runs one pod. You switch on an autoscaler (HorizontalPodAutoscaler) and a disruption budget (PodDisruptionBudget) per controller; the chart renders them only when you set them.

storage:
  backend: s3                      # prerequisite for more than one backend pod
  # ...

controllers:
  backend:
    horizontalPodAutoscaler:
      minReplicas: 2               # replaces replicas
      maxReplicas: 6
      metrics:
        - type: Resource
          resource:
            name: cpu
            target:
              type: Utilization
              averageUtilization: 70
    podDisruptionBudget:
      minAvailable: 1
  frontend:
    replicas: 2
    podDisruptionBudget:
      minAvailable: 1

• Autoscaler: with horizontalPodAutoscaler the Deployment carries no replicas; minReplicas takes its role. So a helm upgrade or an ArgoCD sync does not reset the pod count to the fixed value. The autoscaler needs the metrics-server in the cluster.
• A new backend pod only helps after its start — up to a few minutes, see Backend start-up time. The autoscaler therefore does not absorb short load peaks.
• Celery worker: CPU is the wrong signal for the worker — it mostly waits for external services. Scale it on the length of its queue, see Worker autoscaling by queue length.

The chart refuses to render (helm template, helm upgrade, ArgoCD) with a message when a setting would go wrong silently in the cluster:

Celery beat always stays a single pod with the Recreate strategy: the old pod ends before the new one starts.

Worker autoscaling by queue length¶

The chart can scale the Celery workers on the number of waiting tasks. That takes three things, two of which the chart brings:

1. The metric celery_queue_length{queue} from the backend — switched on with monitoring.enabled: true (see Metrics).
2. The Prometheus Adapter in the cluster, serving that series as an external metric (external.metrics.k8s.io). You install and configure it yourself; the chart does not install it.
3. The worker autoscalers — switched on with celeryWorkers.queueAutoscaling.enabled: true.

storage:
  backend: s3                      # prerequisite for more than one worker pod
  # ...

monitoring:
  enabled: true                    # required: serves celery_queue_length

celeryWorkers:
  dedicatedQueues: true            # optional: one worker per queue, see above
  queueAutoscaling:
    enabled: true
    metricName: celery_queue_length   # name the adapter serves the series under
    workers:
      celery-worker:               # reads critical (all three without dedicatedQueues)
        minReplicas: 1
        maxReplicas: 3
        targetPerReplica: 20       # waiting tasks per worker pod
      celery-worker-default:       # reads celery
        minReplicas: 1
        maxReplicas: 3
        targetPerReplica: 20
      celery-worker-bulk:          # reads bulk
        minReplicas: 1
        maxReplicas: 2
        targetPerReplica: 5

Each worker gets a HorizontalPodAutoscaler (autoscaling/v2) with exactly one metric of type External. Which queues it selects, the chart reads from the worker's -Q: without dedicatedQueues the one worker scales on the sum of all three queues, with dedicatedQueues each worker on its own. The target is AverageValue: waiting tasks divided by the worker's pods. An entry for celery-worker-default or celery-worker-bulk only takes effect with dedicatedQueues: true; enabled: false in an entry leaves that one worker out.

Rule for the Prometheus Adapter (values of the prometheus-community/prometheus-adapter chart):

rules:
  external:
    - seriesQuery: 'celery_queue_length{namespace!="",queue!=""}'
      resources:
        overrides:
          namespace: {resource: namespace}
      name:
        matches: "^celery_queue_length$"
        as: "celery_queue_length"     # = celeryWorkers.queueAutoscaling.metricName
      # Every backend replica reports the same numbers: max, not sum.
      metricsQuery: 'max by (queue) (<<.Series>>{<<.LabelMatchers>>})'

To check that the adapter serves the series (adjust the namespace):

kubectl get --raw "/apis/external.metrics.k8s.io/v1beta1/namespaces/kamerplanter/celery_queue_length?labelSelector=queue%20in%20(critical)"
kubectl describe hpa kamerplanter-celery-worker

The chart refuses to render when queueAutoscaling.enabled is set without monitoring.enabled or without metricName, an entry has no maxReplicas or no targetPerReplica above 0, minReplicas is below 1 or not below maxReplicas, a worker also carries its own controllers.<worker>.horizontalPodAutoscaler, or the name is not a worker of the chart. More than one worker pod on the default ReadWriteOnce volume is refused like for every autoscaler.

Secrets from a secret store¶

Instead of creating kamerplanter-secrets by hand, the External Secrets Operator can write it from your secret store (Vault, AWS Secrets Manager, …). The chart ships an ExternalSecret for this, off by default:

externalSecrets:
  app:
    enabled: true
    secretStoreRef:
      kind: ClusterSecretStore     # or a SecretStore in the same namespace
      name: my-secret-store
    dataFrom:
      - extract:
          key: kamerplanter/app    # store entry with the mandatory keys, without ARANGO_ROOT_PASSWORD

• Every key of the store entry becomes an entry of kamerplanter-secrets, and backend, worker and beat read that Secret in full through envFrom. So put only the mandatory keys into the entry (mandatory secrets), not ARANGO_ROOT_PASSWORD: the root password belongs in a Secret of its own (its own store entry with a second ExternalSecret, or created by hand) that you name in database.arangodb.rootPasswordSecret.
• A ClusterSecretStore without restriction serves every namespace in the cluster. Restrict it to the Kamerplanter namespace through spec.conditions (namespaces or namespaceSelector), or use a SecretStore in the same namespace (secretStoreRef.kind: SecretStore).
• Without secretStoreRef.name the render stops with a message.
• The operator and its CRDs (external-secrets.io/v1) must be installed in the cluster.
• If the entry later disappears from the store, the last Secret is kept (deletionPolicy: Retain) — pods keep starting.
• Retain only covers a deleted store entry. On helm uninstall the ExternalSecret goes away, and because with creationPolicy: Owner it owns kamerplanter-secrets, Kubernetes garbage collection deletes the Secret with it. If you still need the values afterward, they are in the store.

Further Secrets — the S3 credentials (storage.s3.credentialsRef), the backup (backup.s3.credentialsRef) and the database root password (database.arangodb.rootPasswordSecret) — are added the same way as further entries under externalSecrets.

TimescaleDB (sensor data)¶

TimescaleDB stores sensor time series. It is optional and off by default; without it Kamerplanter runs without a time-series store. You switch it on with a single value:

controllers:
  timescaledb:
    enabled: true

The chart then renders the database (StatefulSet, one pod, 10 GiB volume), its Service and its NetworkPolicy, and backend, worker and beat receive TIMESCALEDB_ENABLED=true together with the address. Only backend and worker may reach the database on port 5432.

Before that, add the key TIMESCALEDB_PASSWORD to kamerplanter-secrets:

kubectl patch secret kamerplanter-secrets -n kamerplanter --type merge \
  -p "{\"stringData\":{\"TIMESCALEDB_PASSWORD\":\"$(openssl rand -hex 24)\"}}"

• The database reads only this one key from the Secret; backend and worker receive it like every other key through envFrom.
• Without the key the database pod does not start (CreateContainerConfigError). The backend refuses to start on the built-in default password as soon as DEBUG=false.
• With externalSecrets.app you put the key into the same store entry as the other mandatory keys.
• The database runs without root privileges (user postgres, uid 70) on a read-only file system; only the data volume, the socket directory and /tmp are writable.
• More than one pod is not possible: a second pod would be a second, separate database. The chart refuses replicas above 1 and an autoscaler.
• The volume survives helm uninstall. The chart does not back up the time series (see Backup and restore).

External databases¶

Instead of the bundled ArangoDB and the Valkey subchart you can connect databases outside the chart, such as a managed service or an instance in another namespace. Both switches are off by default; without them the chart renders as before.

ArangoDB¶

database.arangodb.external.enabled: true is the only switch. The ArangoDB StatefulSet, its service, its NetworkPolicy and its volumes are not rendered. Backend, Celery worker, Celery beat and the backup connect to scheme://host:port:

database:
  arangodb:
    appUsername: kamerplanter
    external:
      enabled: true
      host: arango.example.net    # host name only, no scheme
      port: 8529
      scheme: https               # http | https
      cidr: 203.0.113.10/32       # address range of the database (NetworkPolicy)

• The chart creates nothing in an external database. The application database (kamerplanter) and the account from appUsername with read/write access to it must exist; ARANGODB_PASSWORD in kamerplanter-secrets is its password.
• scheme: https sets ARANGODB_SCHEME=https. The backend verifies the server certificate against the certificates in the image. For a private certificate authority, put its certificate into the container and name the path in REQUESTS_CA_BUNDLE. This covers the application only: the backup job's arangodump does not read that variable, so with a private certificate authority the backup is not covered by this chart.
• The backup (backup.enabled) then dumps the external database and connects over TLS with https.

Valkey¶

valkey.enabled: false switches the Valkey subchart off. Backend, worker and beat then get database.valkey.external.url as REDIS_URL:

valkey:
  enabled: false
database:
  valkey:
    external:
      url: "rediss://:$(VALKEY_PASSWORD)@valkey.example.net:6380/0"
      cidr: 203.0.113.20/32

• redis:// and rediss:// (TLS) are allowed.
• A password never goes into the values. Store it as a key in kamerplanter-secrets (here VALKEY_PASSWORD) and reference it as $(KEY); Kubernetes inserts the value when the container starts. Percent-encode the characters / # ? @ in the password, otherwise the backend refuses to start.

NetworkPolicies¶

Under the default-deny baseline the pods reach an external database only through an egress policy of its own. The chart renders it with the matching switch:

• cidr is therefore required. If you reach the database through a namespace or pod selector, replace networkpolicies.<policy>.rules.egress with your rule, or switch the policy off with enabled: false and bring your own.
• If the database listens on another port, replace rules.egress as well. Ports in a NetworkPolicy must be numbers; a template would produce a string, which Kubernetes reads as a port name.

Render checks¶

The chart refuses to render a combination that would fail silently in the cluster:

• an external ArangoDB without host, with a scheme inside host, with a scheme other than http/https, or together with controllers.arangodb.enabled: true
• valkey.enabled: false without url or with a URL that does not start with redis:// or rediss://, and a url set while the Valkey subchart runs
• an egress policy without cidr or without the port you configured

See also¶

• Kubernetes Deployment — Step-by-step guide
• Environment Variables — Full reference of all environment variables
• Configure Storage (Object Storage) — Admin UI and migration