SuperAPI AI Directory
Partner with us中文AI newsFavorites
Skillsautomating-storage-limitation-controls

automating-storage-limitation-controls

Agents & models

View repository

Updated 10-16 01:30

About this skill

Automating Storage Limitation Controls

Overview

Article 5(1)(e) of the GDPR requires that personal data be "kept in a form which permits identification of data subjects for no longer than is necessary for the purposes for which the personal data are processed." This is the storage limitation principle. Controllers must define retention periods for each category of personal data based on the purpose of processing (Article 13(2)(a) requires disclosure of retention periods or criteria).

Automation is critical because manual retention management at scale inevitably leads to compliance gaps. The EDPB Guidelines 4/2019 on Article 25 specifically identify automated deletion mechanisms as an example of data protection by design.

Retention Policy Framework

Retention Period Determination

Each retention period must be justified by reference to the processing purpose, legal requirements, or legitimate business need:

Data Category Purpose Lawful Basis Retention Period Retention Trigger Legal Reference
Customer account data Service delivery Art. 6(1)(b) Duration of contract + 90 days Account closure Art. 5(1)(e), Art. 17(1)(a)
Transaction records Tax compliance Art. 6(1)(c) 10 years Transaction date Swiss CO Art. 958f, EU VAT Directive 2006/112/EC Art. 242
Support tickets Customer service Art. 6(1)(b) 3 years Ticket closure Art. 5(1)(e), limitation period
Server access logs Security monitoring Art. 6(1)(f) 12 months Log creation date Art. 5(1)(e), ENISA recommendation
Marketing consent records Accountability Art. 6(1)(c) Duration of consent + 5 years Consent withdrawal Art. 7(1), Art. 5(2)
Employment records Legal obligation Art. 6(1)(c) Duration of employment + 10 years Employment end Swiss OR Art. 127-128
Cookie analytics Product improvement Art. 6(1)(a) 13 months Data collection ePrivacy Directive Art. 5(3), CNIL guidance
DPIA records Accountability Art. 6(1)(c) Duration of processing + 3 years Processing cessation Art. 35, Art. 5(2)

TTL-Based Deletion Architecture

┌─────────────────────────────────────────────────────────────┐
│                   Retention Policy Engine                     │
│  ┌───────────────┐  ┌────────────────┐  ┌───────────────┐  │
│  │ Policy Store  │  │ TTL Calculator │  │ Legal Hold    │  │
│  │ (per category)│  │ (trigger+period)│  │ Registry      │  │
│  └───────┬───────┘  └───────┬────────┘  └──────┬────────┘  │
│          └──────────────────┼───────────────────┘           │
└─────────────────────────────┼───────────────────────────────┘
                              │
┌─────────────────────────────▼───────────────────────────────┐
│                   Lifecycle Scheduler                         │
│  ┌──────────────────┐  ┌──────────────┐  ┌──────────────┐  │
│  │ Expiry Scanner   │  │ Archive Mover│  │ Deletion     │  │
│  │ (daily cron)     │  │ (pre-delete) │  │ Executor     │  │
│  └──────────────────┘  └──────────────┘  └──────────────┘  │
└─────────────────────────────┬───────────────────────────────┘
                              │
┌─────────────────────────────▼───────────────────────────────┐
│                   Audit and Verification                     │
│  ┌──────────────────┐  ┌──────────────┐  ┌──────────────┐  │
│  │ Deletion Log     │  │ Verification │  │ Exception    │  │
│  │ (immutable)      │  │ Scanner      │  │ Reporter     │  │
│  └──────────────────┘  └──────────────┘  └──────────────┘  │
└─────────────────────────────────────────────────────────────┘

Deletion Patterns for Distributed Systems

Pattern Description Use Case Complexity
TTL columns Database column storing expiry timestamp; background job deletes expired rows Single-database systems, Redis, DynamoDB Low
Event-driven deletion Lifecycle event (account closure) publishes deletion command to message queue Microservice architectures Medium
Tombstone + compaction Mark records as deleted; background compaction physically removes data Append-only stores, event sourcing Medium
Cascade deletion Delete propagates across foreign key relationships Relational databases with referential integrity Medium
Distributed saga Coordinated deletion across multiple services with rollback on failure Polyglot persistence, cross-service data High
Crypto-shredding Destroy the encryption key rendering the data unreadable without physical deletion Systems where physical deletion is impractical (backups, immutable logs) Medium

Legal Hold Management

Legal holds suspend automated deletion when data is subject to litigation, regulatory investigation, or audit proceedings.

Legal Hold Lifecycle:

  1. Hold initiation — Legal counsel issues a hold notice specifying the scope (data categories, date range, affected subjects) and the matter reference
  2. Hold registration — The Retention Policy Engine registers the hold, creating an exemption that overrides TTL-based deletion for matching records
  3. Hold monitoring — Monthly review by legal counsel to confirm the hold remains necessary
  4. Hold release — Legal counsel issues a release notice; the Retention Policy Engine removes the exemption and resumes normal retention processing
  5. Catch-up deletion — Records that would have been deleted during the hold period are now evaluated against current retention policies and deleted if expired

Prism Data Systems AG Implementation:

Hold Parameter Value
Hold scope format JSON document specifying data_categories, date_range_start, date_range_end, subject_identifiers, matter_reference
Hold registry PostgreSQL table with immutable audit trail
Review cadence Monthly by legal counsel, quarterly by DPO
Maximum hold duration without renewal 12 months
Hold release approval Dual approval: legal counsel + DPO

Archival Policies

Three-Tier Storage Lifecycle

Tier Storage Type Access Pattern Encryption Retention Phase
Hot Primary database (PostgreSQL) Real-time read/write AES-256-GCM field-level Active processing
Warm Object storage (S3-compatible, Glacier Instant Retrieval class) Occasional read, no write AES-256-GCM object-level Post-purpose, pre-deletion (legal/audit)
Cold Offline archive (tape/vault) Restore on request (24-48h) AES-256-GCM + envelope encryption Statutory retention only

Transition Rules for Prism Data Systems AG:

Transition Trigger Automation
Hot → Warm Purpose fulfilled (e.g., account closed) + 90 days Daily lifecycle job
Warm → Cold Warm retention period expires, statutory retention still applies Monthly lifecycle job
Cold → Deletion Statutory retention period expires, no legal hold active Quarterly lifecycle job with DPO approval gate
Any → Legal Hold Legal hold registered for matching scope Immediate suspension

Implementation Workflow

  1. Inventory data stores — Catalog every system that stores personal data, including primary databases, caches, message queues, logs, backups, and third-party integrations.

  2. Map retention requirements — For each data category and processing purpose, determine the retention period based on purpose necessity, legal obligations, and limitation periods.

  3. Implement TTL metadata — Add retention-related columns or metadata to each data store: created_at, retention_trigger_event, retention_trigger_date, ttl_expiry_date, legal_hold_id.

  4. Deploy lifecycle scheduler — Implement a background service that scans for expired records daily, checks for active legal holds, and executes deletion or archival transitions.

  5. Implement deletion verification — After each deletion cycle, run a verification scan to confirm that all targeted records have been removed from primary stores, replicas, and caches.

  6. Configure backup rotation — Ensure that backup retention aligns with the longest applicable retention period. Implement crypto-shredding for backup media where granular deletion is impractical.

  7. Deploy monitoring and alerting — Monitor deletion job success rates, flag records that exceed their TTL by more than 7 days, and alert on legal hold expirations.

Key Regulatory References

  • GDPR Article 5(1)(e) — Storage limitation principle
  • GDPR Article 13(2)(a) — Retention period disclosure requirement
  • GDPR Article 17(1)(a) — Right to erasure when data no longer necessary
  • GDPR Article 25(1) — Data protection by design
  • GDPR Recital 39 — Storage limitation guidance
  • EDPB Guidelines 4/2019 on Article 25 Data Protection by Design and by Default
  • Swiss Code of Obligations Art. 958f — 10-year retention for business records
  • EU VAT Directive 2006/112/EC Art. 242 — Invoice retention requirements