Data Retention & Anonymization¶

Kamerplanter stores data only as long as technically or legally required. This document describes the retention matrix, the automated enforcement mechanism (Celery task), TimescaleDB aggregation for sensor data, and configuration via environment variables.

Basis: GDPR Art. 5(1)(e).

Retention Matrix — Personal Data¶

Every period except R-11 (TTL index), R-06a and R-38 (fixed) is read from exactly one setting (see Environment Variables for the exact names). For R-01, R-05 and R-07 there were already documented, older variable names before this change, but they had no effect — the code used fixed values. These older names now actually work and remain valid as aliases in addition. Every period comparison compares instants rather than strings, so a record is not captured too early or too late by a differently spelled timestamp.

Unconfirmed Accounts (R-02)¶

A never-confirmed account is removed automatically by the daily app.tasks.auth_tasks.cleanup_unverified_accounts task — RETENTION_UNVERIFIED_ACCOUNT_DAYS days after registration (default 7 days, minimum 1 day). The task runs the same full erasure as a self-filed erasure request, with the same erasure request as proof and the same automatic retry on a failed step — more on this further below under "All deletion paths do the same".

The task selects only accounts that were never confirmed, never demoted by an administrator and never signed in, and that have no provider row. Someone who registered locally has a LOCAL provider row and is therefore not yet reached. The task does not delete those accounts; it counts them on every run (local_registrations_pending in the result and the unverified_local_registrations_pending warning). Only once you have seen the number do you release the deletion with RETENTION_UNVERIFIED_LOCAL_REAP_ENABLED=true; after that only federated provider rows (Google, GitHub, Apple, OIDC) count as "linked", and service accounts stay exempt. If the count fails, the task keeps the narrow selector.

IP Anonymization (R-03)¶

IP addresses are automatically anonymized RETENTION_IP_ANONYMIZATION_DAYS days after the session was issued (default 7 days, minimum 1 day) — not deleted, as they may still be needed for detecting compromised sessions:

• IPv4: Last octet set to 0 — 192.168.1.42 → 192.168.1.0
• IPv6: Truncated to /48 prefix — 2001:db8:85a3::8a2e:370:7334 → 2001:db8:85a3::

The ip_anonymized_at field is set to the time of anonymization. This covers session IP addresses only — the IP address held by a consent record (see R-04 above) is not touched by it.

Export files: the file first, then the status (R-05)¶

The period is configurable via RETENTION_EXPORT_FILE_RETENTION_HOURS (default 72 hours, minimum 1 hour; the older name PRIVACY_EXPORT_RETENTION_HOURS remains valid as an alias). Cleanup of an expired data export runs in a fixed order: the export file is deleted from object storage first, and only then does the export switch to the expired status. If deleting the file fails, the export is left unchanged — the next hourly run retries it. Downloads are already refused once the 72-hour window has passed, regardless of the stored status. If no object storage is configured on the instance, the file cannot be deleted at all: the export is left open, and the run logs an error instead of flipping the status anyway.

Erasure Record Purge (R-06)¶

The daily retention.purge_expired_erasure_records task (04:30 UTC, after the erasure task at 04:00 UTC) hard-deletes completed erasure requests (erasure_requests, status=completed) whose completed_at is more than RETENTION_ERASURE_AUDIT_RETENTION_YEARS years in the past (default 3 years, minimum 1 year, maximum 3 years, counted in calendar years). This applies regardless of who triggered the erasure (origin: self_service, platform_admin, or unverified_cleanup).

A request that is still open or only partially completed (scheduled, in_progress, partially_completed) is never purged — it is still owed a run. A completed request without a completed_at is kept too. The run logs only the number of purged requests, never an account or request key.

Email Change Requests (R-07)¶

The hourly retention.expire_email_change_requests task (minute 15) sets a still unconfirmed email-change request to expired RETENTION_EMAIL_CHANGE_RETENTION_HOURS hours after the request (default 24 hours, minimum 1 hour; the older name PRIVACY_EMAIL_CHANGE_TTL_HOURS remains valid as an alias) and hard-deletes it in the same run, so the new address is not kept.

Revert Window of a Confirmed Email Change (R-07a)¶

Once an email change is confirmed, the previous address receives a one-time-use revert link (see Privacy — Changing Your Email Address). For that, the record keeps three extra fields: previous_email, the hash of the revert token (revert_token_hash), and its expiry (revert_expires_at), set to the confirmation time plus RETENTION_EMAIL_CHANGE_REVERT_DAYS (default 7 days, minimum 1 day).

The same hourly task as R-07 (retention.expire_email_change_requests, minute 15) closes the window in the same run: for every record whose revert_token_hash is set and whose revert_expires_at has passed or is unreadable, it clears all three fields — conservatively, so a record with an unreadable expiry is treated as expired. The revert link no longer works afterwards; the same run hard-deletes the rest of the record (new_email, timestamps) as R-07b once the stored revert_expires_at has passed.

Retention Matrix — Sensor Data¶

Sensor data can indirectly allow inferences about the presence and behavior of persons (CO2 curves, motion sensors, manual overrides). They are therefore subject to a tiered retention policy in TimescaleDB:

flowchart LR
    A["Raw data<br/>(full resolution)"]
    B["Hourly averages<br/>(avg, min, max)"]
    C["Daily averages<br/>(avg, min, max)"]
    D["Deleted"]

    A -- "after 90 days" --> B
    B -- "after 2 years" --> C
    C -- "after 5 years" --> D

TimescaleDB Continuous Aggregates¶

Automatic downsampling is handled by TimescaleDB Continuous Aggregates and Retention Policies:

-- Stage 2: Hourly averages (automatically after 90 days)
CREATE MATERIALIZED VIEW sensor_hourly
  WITH (timescaledb.continuous) AS
  SELECT
    time_bucket('1 hour', timestamp) AS bucket,
    location_key,
    sensor_type,
    AVG(value)   AS avg_value,
    MIN(value)   AS min_value,
    MAX(value)   AS max_value,
    COUNT(*)     AS sample_count
  FROM sensor_readings
  GROUP BY bucket, location_key, sensor_type;

-- Stage 3: Daily averages (automatically after 2 years)
CREATE MATERIALIZED VIEW sensor_daily
  WITH (timescaledb.continuous) AS
  SELECT
    time_bucket('1 day', bucket) AS bucket,
    location_key,
    sensor_type,
    AVG(avg_value) AS avg_value,
    MIN(min_value) AS min_value,
    MAX(max_value) AS max_value,
    SUM(sample_count) AS sample_count
  FROM sensor_hourly
  GROUP BY time_bucket('1 day', bucket), location_key, sensor_type;

-- Retention policies (automatic deletion)
SELECT add_retention_policy('sensor_readings', INTERVAL '90 days');
SELECT add_retention_policy('sensor_hourly',   INTERVAL '2 years');
SELECT add_retention_policy('sensor_daily',    INTERVAL '5 years');

Statutory Minimum Retention Periods¶

These records must not be deleted before the statutory period has elapsed. For a GDPR erasure request (Art. 17), they are anonymized (user reference removed) but not deleted (Art. 17(3)(b)):

When the period ends after a tenant deletion¶

The period counts from the date the record itself carries — harvest_date (harvest), applied_at (treatment), inspected_at (inspection) — and keeps running unchanged if the tenant or your account is deleted in the meantime. The deletion neither restarts nor shortens it.

Once it has ended, the daily task retention.purge_expired_legal_retention_rows (04:45 UTC) deletes the record for good: together with its quality assessments and yield metrics (for a harvest) and every link that still touches it — for treatments, the inherited pre-harvest-interval links to plants and runs as well.

Only records whose tenant no longer exists are deleted. The harvests, treatments and inspections of an existing garden are that garden's own records; this rule sets no maximum for them, and they stay.

A record without a readable date is never deleted — its age is not established. Exception: a harvest without a harvest date counts from its creation, because that is the date the system assumed as the harvest date when it was created.

What an anonymized record looks like¶

The account reference is not set to null; it is replaced. Its new form depends on whether the erased account's records must stay linkable to each other for an audit:

• The tombstone hash is anon_ followed by 16 hex characters. It is derived from the account key and the secret salt ERASURE_TOMBSTONE_SALT, cannot be reversed, and is always the same for the same account. The harvests, treatments and inspections of an erased account therefore stay linkable to each other for an audit without naming anyone.
• The name fields (harvester, assessed_by, applied_by, inspector) are free text someone typed in when recording. A name is personal data on its own, so they are emptied in the same step.
• Quality assessments are part of the harvest documentation and are now anonymized exactly like the harvest itself. They also appear in your data access report (GDPR Art. 15).
• Yield metrics (YieldMetric) carry no account reference, so there is nothing to anonymize.

What happens to your personal garden¶

When you register, Kamerplanter creates a personal garden for you. On account deletion it is always deleted with the account — even when other members are in it. It is fully deleted through the tenant-erasure inventory: sites, plants, planting runs, diary, tasks, tanks and everything else in it is gone afterward. Only harvest, quality, treatment and inspection records remain — as with any account deletion — under your tombstone hash, because the CanG (5 years) and PflSchG (3 years) require it; their free-text name fields are emptied. This garden deletion gets its own deletion record, and your erasure request states in the end what happened to the garden.

A personal garden can, like any garden, take further members (see Tenants & Gardens). For them:

• Warned beforehand: as soon as you request the deletion, every other active member gets an email. It states the date of the final deletion so they can secure their data until then. The email names neither you nor your address nor the garden, because a personal garden is named after its owner.
• What is guaranteed when the email does not arrive: Kamerplanter records on the deletion request whether the members were notified and retries an unsent message every night before the garden is deleted. A deletion request from before this rule, which nobody has been told about yet, gets the message in the next nightly run; the garden is then deleted no earlier than RETENTION_ERASURE_MEMBER_NOTICE_DAYS days after it (default 7). If the message cannot be delivered during that whole wait (for example because the operator has not set up mail), the deletion goes ahead anyway: your right to erasure does not depend on somebody else's mailbox. The operator sees the failure in the log and on the deletion request (members_notice_failures), with no address in either.
• Deletion by an administrator: when a platform administrator deletes an account, it happens at once. The other members of the personal garden get an email saying the deletion is happening now; there is no grace period to secure anything. The administrator sees before confirming which gardens are affected and how many other members this hits.
• Their content goes with it: everything the other members created in this garden is deleted too. Their own accounts and their memberships in other gardens are unaffected.
• Preview before you confirm: the delete-account dialog shows, for each personal garden, that it is deleted and how many other members this affects.
• Only exception — a late joiner: if someone newly joins between your request and the deletion run (invitations into the garden are revoked when you request the deletion, so this only happens by detours), the garden is kept for that person. Then only your owner reference is removed:
  • The owner reference is replaced with _anonymized.
  • The name and the short name become anonymized- followed by a string that cannot be traced back to the key. The short name stays unique, and no new garden can take it first: Kamerplanter never hands out short names that start with anonymized.
  • Who takes over the garden afterward is currently not defined by Kamerplanter.

If the deployment cannot delete your personal garden — for example because a sensor-reading store, the tombstone salt, or the reference-index/pest-image store is missing or misconfigured — it refuses the entire account deletion before anything changes, and retries automatically once the configuration is fixed (see below, "All deletion paths do the same").

A community garden you founded keeps its name, because the name belongs to the group. Only the owner reference is replaced. The owner reference grants no rights; rights come from the role in a membership. Only one thing is handed over: if you are the last person with the management right in an organization, the longest-serving remaining lead takes over management, and the members are notified. If no lead is left or you are the only member, the organization becomes orphaned: it is closed and deleted with all its data after the tenant-deletion grace period (RETENTION_TENANT_ERASURE_GRACE_DAYS, default 90 days); the remaining members and the operators are notified. The preview shows you both before you confirm the deletion. Both happen as soon as you request the deletion — not only at the final deletion up to 90 days later: your account is closed from the request on, and the organization must not stay without management for that long. The grace period of an orphaned organization starts with your request.

An AI tip you dismissed stays dismissed for the other members of your garden. Only the note that you dismissed it is replaced with _anonymized.

Accounts that were never confirmed, and that Kamerplanter removes once the period runs out, go through the same full deletion as any other account deletion — with the same erasure request as proof and the same automatic retry if a step fails.

What else is deleted¶

Besides the familiar categories, account deletion also deletes these records, which belong only to you and fall under no retention period:

• your AI assistant conversations
• your notifications and your notification settings
• your calendar feeds: the feed link stops working after the deletion
• your plant-disease diagnosis requests
• your location assignments in gardens you were a member of
• invitations you accepted (they name your email address)
• the file entries of your own pest photos — the original file and its preview images (WebP, 128/512/1280 px) are already deleted at this point
• your contributed reference-image vectors for plant recognition (curated reference images contributed by other users are unaffected)
• your contributed pest-recognition vectors — regardless of whether your contribution was still promoted or had already been demoted at the time of deletion

The daily cleanup deactivates accounts that were never confirmed right away and deletes them the same way. It now also removes their membership and location assignments. If a step fails, the erasure request is left open and — as described further below — automatically retried with backoff instead of being silently dropped. The personal garden of such an account is handled the same way as for any other account deletion (see above, What happens to your personal garden).

Logs of an erasure¶

The log lines of an erasure do not name your account key. They carry a salted reference instead, so the lines of one erasure can be linked to each other without naming anyone. They cannot be linked to the pseudonymised erasure audit, though.

All deletion paths do the same¶

It makes no difference whether a platform admin deletes your account through user management, the daily cleanup removes a never-confirmed account, or you file an erasure request yourself: all three paths run the same erasure, with the same erasure request as proof. It cleans object storage (originals along with their preview images) and the recognition base (only your own reference-image and pest-image contributions — curated references stay), deletes your other records, anonymizes the ones under a retention obligation as described above, and removes your account last. The database part runs as one unit: it is either done completely or not at all.

The difference is timing: your own request is carried out by the daily run only once the 90 days have passed. A deletion by a platform admin or by the cleanup for never-confirmed accounts instead runs at once — your account is deactivated and all sessions ended first, then the erasure runs without waiting. In all three cases: afterward the request reads completed and carries the tombstone hash instead of your account key. If a run fails, the request stays open as partially_completed and is retried automatically with backoff until it succeeds (see below). While a request is open, you cannot file a second one; a second deletion attempt by a platform admin instead resumes the open request at once.

Attachments are deduplicated per tenant by the file's SHA-256 hash (issue #1770): if a second member uploads exactly the same file (or you do, in another category), a record of its own is created, but the bytes are stored once and referenced by both records. Storage cleanup (Phase 0) therefore only deletes the file once no other record still points to it; if a record belonging to another member still points to it, the object is kept. The erasure request (erasure_requests) counts both: storage_objects_removed (objects actually deleted) and storage_objects_retained_shared (objects kept because another member's record still holds them). The same numbers are logged by the retention.erasure.storage_hard_delete line as deleted and retained_shared, per scope and tenant. If the other member deletes their record while your erasure is running, no one holds the object once the ArangoDB step has run, so the erasure asks again afterwards and removes it. That is counted as storage_objects_released, stored with the completed status; a failure there is logged as retention.erasure.shared_object_release_failed.

Tenant Deletion¶

When a platform admin or a member with the Management extra permission deletes a tenant, the system follows its own process, similarly shaped to account erasure — the difference is that the entire domain data of a garden is affected, not just the records of a single account.

Order¶

1. Upfront checks: The special, technical platform tenant cannot be deleted (403). If the instance is not correctly configured for tenant deletion (missing ERASURE_TOMBSTONE_SALT, a misconfigured reference index or pest-image store), the system refuses the deletion with 503 and changes nothing.
2. Proof, lock and grace period: A deletion record is created (it names neither the tenant's name, slug nor owner). The tenant enters the state "Deletion scheduled" (pending_deletion) and is closed at once for every member, API key and MCP client. Every member receives an email with the deletion date: until then they can save their own data through the personal data export. The actual deletion starts only after RETENTION_TENANT_ERASURE_GRACE_DAYS days (default 90). Until then a member with the Lead role and Management — or a platform admin in the admin area — can cancel the deletion with their own password; the tenant is back in the state it had before the deletion afterwards (a suspended one stays suspended), with every membership unchanged. Once the grace period has ended, the memberships are deactivated and the state changes to "Being deleted" — from then on nothing can be taken back.
3. External cleanup: Contributed recognition vectors and pest prototypes are removed, followed by the tenant's sensor readings in the time-series database and all binary data under the object-storage prefix t/{tenant_key}/.
4. One database transaction: Every domain record the tenant holds is deleted — sites, plants, planting runs, diary entries, tasks, tanks, sensors, feeding and watering logs, notifications, calendar feeds, its own master-data entries, memberships, invitations, location assignments and API keys restricted to the tenant — together with every link to a deleted record, and finally the tenant record itself.

What is retained¶

As with account erasure, the statutory minimum retention period for harvest and treatment documentation applies (R-16 through R-18, see above): harvest batches, quality assessments, treatments and inspections remain but are pseudonymized — each affected member's account reference is replaced by their tombstone hash, and name fields are emptied. The AI call log and the MCP call log are likewise retained until their own retention window expires, as is the deletion record itself — it is the proof of the deletion and drives its retry.

The retained harvest, treatment and inspection data is deleted once its period has ended (see When the period ends after a tenant deletion). The deletion record is kept as long as the tenant's longest-retained record, but at most 5 years after the deletion completed (R-06a); the daily task retention.purge_expired_tenant_erasure_records (04:50 UTC) deletes it afterwards. If the deletion retained no harvest, treatment or inspection data, the record is kept for the full 5 years.

Completeness is measured, not assumed¶

After the transaction, the system counts whether any record — even in a collection that is not part of the declared inventory — still references the tenant. If so, or if a step failed (e.g. the inference service was unreachable, 502), the deletion record stays partially_completed. The daily retry run picks it up at 04:30 UTC — after the account erasures — and retries every open tenant deletion with the same backoff as account erasure: 1, 2, 4 and then at most 7 days. A second deletion attempt for the same tenant while one is already running responds with 409 instead of starting another attempt.

What is not affected¶

The members' own accounts are unaffected — they keep their account and their memberships in other tenants. A member's personal tenant is not touched by the deletion of an other tenant. Deleting your own account, however, runs your personal tenant through this exact tenant-erasure inventory — fully, even when other members use it; only a late joiner keeps it alive (see above, What happens to your personal garden).

Celery Enforcement: Automated Execution¶

Each rule runs as its own Celery beat task on a schedule that matches its period. There is no central task that orchestrates every rule in one shared run, and none is planned: a single daily schedule would even be wrong for the hour-granular periods (R-05, R-07) — a run at 02:00 UTC would overshoot a 24-hour period by up to a day, keeping the record longer than declared.

Each task logs its run in structured form (structlog) under its own event name with counters, for example:

• retention.execute_scheduled_erasures.completed (processed)
• cleanup_unverified_accounts (removed, failed, deferred, skipped, blocked)
• anonymize_old_ips (anonymized)
• retention.expire_data_exports.completed (expired)
• retention.purge_expired_erasure_records.completed (purged, held_without_tombstone)
• retention.expire_email_change_requests.completed (expired, revert_windows_closed)
• cleanup_expired_tokens (removed)
• expired_invitations_cleaned (count)
• retention.purge_expired_legal_retention_rows.completed (purged per rule: rows, children, edges)
• retention.purge_expired_tenant_erasure_records.completed (purged)

All retention tasks run on the clock (a cron expression), not as a fixed interval from the start of the beat process: restarting the beat pod does not move a run, so the "at the latest" statements of the privacy policy (period plus one beat interval) also hold across restarts.

Where an age filter cannot judge a record because its timestamp is missing or unreadable, it does not delete it but counts it and reports the counter as held_undated in the run event: in the account cleanup task (R-02), the call logs, the orphaned task photos, the consent records (R-04, purge_expired_consent_records), the confirmed e-mail changes (R-07b, expire_email_change_requests) and the glossary cache (glossary_cleanup_cache).

There is no single event line that summarizes every rule.

Metrics¶

There are currently no Prometheus metrics for the retention runs (internal reference: issue #1800). The only observability source is the structured log lines above, for example via kubectl logs -l app=celery-beat.

Configuration via Environment Variables¶

Every period is read from exactly one setting (RetentionService); its constructor checks the same floor again:

If both names of a row are set, the RETENTION_* name wins. The older names were documented before this change but had no effect — the code used fixed values; they now actually apply.

Interaction with GDPR Erasure Requests¶

When a data subject submits an erasure request under GDPR Art. 17, the following procedure applies:

flowchart TD
    A["Erasure request Art. 17"] --> B{"Statutory retention<br/>obligation?"}
    B -- "No" --> C["Immediately: Hard-delete<br/>or Soft-delete + 90d"]
    B -- "Yes (CanG, PflSchG)" --> D["Anonymization:<br/>account key → anon_… hash<br/>name fields emptied<br/>Record remains"]
    D --> E["After period elapses:<br/>Automatic hard-delete<br/>via Celery task"]

Sweep: Orphaned Pest-Recognition Prototypes¶

Before issue #1766, deleting a pest-photo contribution left its recognition prototype behind in the inference service's reference index (pest_embeddings, source = user_contributed). Such rows name no user — the Art. 17 account erasure finds prototypes only through a user's contribution documents and therefore cannot reach them; only a tenant deletion removed them along the way.

The Celery task pest_image.sweep_orphaned_prototypes (beat entry retention-sweep-orphaned-pest-prototypes-daily, daily at 04:30 UTC, after the scheduled erasures at 04:00 UTC) closes this gap:

1. It pages through the contribution keys the index holds (POST /pest/reference/contributions/keys on the inference service).
2. It checks in ArangoDB which of those keys still have a pest_image_contributions document.
3. It deletes the prototypes of the others — active and deactivated rows alike — in batches of 500 through the same POST /pest/reference/contributions/erase endpoint the Art. 17 erasure uses.

Idempotent: a second run finds no orphan left and removes nothing. Fails loud: if the inference service is unreachable, the task fails (log event pest_prototype_orphan_sweep_failed), nothing is recorded, and the next beat run retries. A celery-worker missing both PEST_DETECTION_ENABLED/INFERENCE_SERVICE_ENABLED, once a pest-photo contribution has ever been promoted into the recognition base, refuses the run just like the scheduled erasure does (issue #1759). Without those flags and without that promotion marker the task reports skipped (log event pest_prototype_orphan_sweep_skipped) and records no run — it never saw the index.

Each run's counts are recorded in the system_settings singleton document under pest_prototype_orphan_sweep (first_run_at, last_run_at, last_examined, last_orphaned, last_removed, total_removed, binding) and as the log event pest_prototype_orphan_sweep_completed (examined/orphaned/removed/binding) — both name counts only, never a key.

Deactivated (demoted) prototypes whose contribution document still exists are left untouched by the sweep (curation).

kubectl exec deploy/<release>-celery-worker -- \
  celery -A app.tasks call pest_image.sweep_orphaned_prototypes

Migration v0062: Splitting Shared Attachments Into Owned Records¶

Before issue #1770, a second upload of the same bytes within a tenant got back the first uploader's record — even across categories, for example between a pest-image contribution and a documentary photo (diary, task, inspection, harvest/storage observation, plant gallery). If the first uploader deleted their account, the second member's record was hard-deleted or anonymized along with it; if the second uploader deleted their account, the erasure never reached their contribution because no record belonged to them.

The migration v0062_split_shared_attachment_ownership brings existing data as close to the new shape as the data allows to reconstruct:

1. The unique index on attachments.storage_key is dropped. Several uploaders now share one stored file; a non-unique index replaces it.
2. Every pest-image contribution gets its own pest_reference record, unless the record it pointed to was already its own: a new record with the deterministic key pic-<contribution key> is created over the same stored file, named after the contributor, and the contribution is repointed to it. The original uploader's filename is not carried over.
3. A pest_reference record that a documentation carrier references (diary entry, task, inspection, harvest/storage observation, plant gallery) is recategorized into a record of that category — otherwise it would be hard-deleted when the pest-image owner is erased, instead of being anonymized and retained like every other documentary photo.

Run it like any migration via python -m app.migrations upgrade; --dry-run computes every change and logs it (split_shared_attachment_ownership_dry_run, with the same counters) without writing anything. An interrupted run leaves no inconsistent state: the split key is deterministic and written with an UPSERT, so a re-run picks up exactly where it left off.

Migration v0066: Resetting Legacy Stamps on Seed Rows¶

When tenants were introduced, the old migration v0004 assigned every row without an owner to a "default tenant" — including the bundled seed data. Since tenant deletion (#1769) removes the rows of its tenant, deleting that tenant would take seed data with it that is meant for everyone. On a legacy volume this concerns fertilizers, nutrient plans with their phase entries, workflow templates and task templates.

The migration v0066_reset_legacy_seed_tenant_stamps resets those rows to global (tenant_key == ""), but only when two things hold: the bundled seed file names the row (same key as the seed loader: product and brand, plan or workflow name, workflow name and task name), and the owner key is proven to be the v0004 stamp (a phase entry or task template carries it under a global parent, or it is on more than half of the expected seed fertilizers and workflows). A plan a tenant created itself and named like a seed stays that tenant's: if a name has a second global or identically stamped row, or is a clone, nothing is reset. Rows the seed file does not name are left alone.

RETURN {
  entries_under_global_plan: LENGTH(
    FOR d IN nutrient_plan_phase_entries
      FILTER d.tenant_key != null AND d.tenant_key != ""
      LET p = DOCUMENT(CONCAT("nutrient_plans/", d.plan_key))
      FILTER p != null AND (p.tenant_key == null OR p.tenant_key == "")
      RETURN 1),
  nutrient_plans_stamped_by_tenant: (FOR d IN nutrient_plans
    FILTER d.tenant_key != null AND d.tenant_key != ""
    COLLECT t = d.tenant_key WITH COUNT INTO n RETURN {tenant_key: t, rows: n})
}

Run it like any migration via python -m app.migrations upgrade; --dry-run counts the changes (reset_legacy_seed_tenant_stamps_dry_run) without writing anything. A second run changes nothing.

Migration v0067: Cleaning Up Legacy References From the #1871 Holes¶

Before #1871/#1876 some routes let a tenant store a reference to another tenant's data. The fixes stop new cases; migration v0067_clean_legacy_foreign_references cleans the existing data:

1. Shared workflow templates of a private species (generated, tenant_key == "") are assigned to the tenant that owns the species. Nothing is deleted.
2. Slots whose location field and has_slot edge disagree: the edge follows the field, but only when both locations belong to the same tenant.
3. Foreign edges (equipment_at, assigned_to_location, log_slot, feeds_from, entry_for_species) are deleted when both tenants are known and differ. Global targets and species granted to the tenant stay.

Also left unchanged, on purpose: templates of species granted to a tenant or when the owner already holds a generated plan, log_slot edges on slots whose field and edge name different tenants, and entry_for_species edges younger than the merge of the #1871 fixes (2026-09-26), where a grant that was later withdrawn can be the cause.

Anything that cannot be classified with certainty is left unchanged and shows up as …__left_unclassified in the counts. The run logs counts only and is idempotent. Count first with python -m app.migrations upgrade --dry-run.

Legacy data: cleaning up Home Assistant readings from before the fix¶

Until the Home Assistant poll was fixed, Kamerplanter stored every polled reading with an empty tenant key. Deleting a sensor or a tenant does not find such rows, so they stay until the 90-day, 2-year and 5-year tiers expire them — even when their sensor is long gone. New rows of this kind are no longer created. The legacy rows are not touched automatically, because a delete cannot be undone: you run the cleanup yourself, in two steps.

The command sorts every series with an empty tenant key into a class — in the raw data and in the hourly and daily averages:

1. Dry run (the default, writes nothing):

kubectl exec -n kamerplanter deploy/<release>-backend -- \
  python -m app.migrations.purge_orphan_ha_readings

The output lists, per class, the series and the rows per table (raw, hourly, daily); for how many live series the tenant could be derived from the parent site, tank or location (a number only — those rows are not rewritten); and last the total number of orphan rows. The same number is in the prepared confirmation command.

2. Delete, confirmed with exactly that number:

kubectl exec -n kamerplanter deploy/<release>-backend -- \
  python -m app.migrations.purge_orphan_ha_readings --confirm-delete-orphans <number>

• If the number differs from what the command measures now (rows arrived, an earlier run deleted part of them), it refuses (exit code 4) and deletes nothing. Repeat the dry run and confirm the new number.
• Before each series the command checks again that its sensor is really missing, deletes in seven-day windows with one commit each, and measures again afterwards.
• Stopping in the middle is harmless: windows already deleted stay deleted, the dry run shows the rest.
• It never touches rows with a non-empty tenant key (not even for the same sensor); those belong to the tenant or sensor deletion.
• The log carries counts only, no sensor or tenant keys.
• The output also says how many sensor documents the command found in ArangoDB. If it shows 0 while orphan rows are reported, the wrong database is usually configured (ARANGODB_DATABASE): do not confirm. The command refuses a database that was never initialised.

Without TimescaleDB (light mode) the command reports "not applicable" and exits 0; when TimescaleDB is configured but unreachable it exits 1 and changes nothing.

Frequently Asked Questions¶

See also¶

• Environment Variables
• Database Schema
• Kubernetes Deployment