ai-agent-memory-erasure-proof

AI Agent Memory Erasure Proof

Acknowledgement: Shared by Peter Bamuhigire, techguypeter.com, +256 784 464178.

Use When

• A data subject (or controller on their behalf) submits an erasure request under GDPR Art. 17, CCPA / CPRA, POPIA s.24, KE DPA s.40 that includes agent-derived memory (working / episodic / semantic / vectors / fine-tune corpora).
• Proving end-to-end that the 9-step cascade ran to completion and that no residue remains in any tier, any replica, any backup index, any vector store, any subprocessor cache.
• Producing a signed proof-of-erasure artefact the data subject's controller (or regulator) can verify offline.
• Closing the loop between ai-agent-memory (which performs the cascade) and the compliance evidence pipeline.

Do Not Use When

• Designing the memory tiers themselves — ai-agent-memory.
• Designing the tenant-level erasure orchestrator (whole tenant erasure) — saas-tenant-data-portability-and-erasure.
• Drafting the policy / DPIA — SRS engine.
• Erasing action audit log entries — those have a separate redaction flow (audit log is retained per regulatory minima with PII redaction, not deleted).

Required Inputs

• Erasure request envelope: request_id, subject_id, tenant_id, data_classes ⊆ {working,episodic,semantic,vectors,fine_tune,uploads,derivatives}, legal_basis (regulation citation), received_at, requester_identity_proof_ref.
• Memory tier catalogue with storage locator for each tier (DB tables, object stores, vector indexes, LLM-provider knowledge-base IDs, fine-tune model IDs).
• Subprocessor inventory with erasure-API surface (which subprocessors hold subject data and how to delete from each).
• Hash-chained action audit log (so the cascade itself is recorded as event_class=erasure_step).

Workflow

1. Read this SKILL.md.
2. Receive the request, validate identity, record subject_id → erasure_request_id on the action audit log (§1).
3. Run the 9-step cascade (§2) — each step writes a step record, calls the underlying tier API, and emits a verification probe.
4. After the cascade, run the verification job (§3) — probes every tier (and subprocessors) for residue using the deterministic subject fingerprint.
5. Produce the proof-of-erasure pack (§4) — signed, hash-chained, includes step records + verification results.
6. Notify the requester with the pack reference and the verified_at timestamp; honour the regulatory clock (GDPR: 1 month, CCPA: 45 days, POPIA / KE DPA: "without undue delay").
7. Retain the proof for the retention class (minimum 6 years; longer in HIPAA-adjacent contexts) — the request and the proof are evidence, the underlying data is gone.
8. Apply anti-patterns (§5).

Quality Standards

• The 9 steps are idempotent — re-running the cascade is safe and emits identical results.
• Each step record carries before / after fingerprints so the auditor can verify the deletion observably reduced the search space.
• Verification probes are independent of the deletion path (different code, different storage handles).
• Subprocessor receipts are stored alongside the proof (LLM provider deletion ticket IDs, vector-store deletion confirmations).
• No silent failures — a step that cannot complete (e.g. fine-tune model still in use) opens a regulator-grade exception within SLO.
• The proof is portable — the requester / their controller / a regulator can verify the signature without our platform.

Anti-Patterns

• Soft-delete with a deleted_at flag in the memory table. Not erasure; data still recoverable.
• Vector store deletion via "remove from index" without confirming the underlying record was purged. ANN indexes often retain shards.
• Fine-tune corpora deleted from source but the fine-tuned model retained. The model itself contains the data; either retrain or document why retention is lawful (Art. 17(3) exception with documented justification).
• Subprocessor "we deleted it" e-mail. Need an API confirmation with a ticket ID stored in the proof.
• Erasing the action audit log entries about the subject. The audit log is retained with PII redacted; deleting it destroys evidence that the erasure happened.
• Single-tier erasure (e.g. only working memory) without checking the others. Episodic and semantic often contain reconstructions of the same subject.
• Pack signed by the engineer who ran the cascade. Segregation-of-duties failure; sign by a separate compliance owner.

Outputs

• erasure_requests table + step-by-step erasure_steps table.
• 9-step cascade orchestrator (Python).
• Independent verification job (Python).
• Signed proof-of-erasure pack format.
• Subprocessor receipt collator.
• Auditor portal endpoint.

Evidence Produced

Category	Artifact	Format	Example
Compliance	Erasure request record	DB row + JSON snapshot	evidence/erasure/{request_id}/request.json
Compliance	Step records (×9)	JSONL	.../steps.jsonl
Compliance	Verification probe results	JSON	.../verification.json
Compliance	Subprocessor receipts	JSONL	.../subprocessor-receipts.jsonl
Compliance	Proof-of-erasure pack	tar.gz + signature	.../proof.tar.gz
Compliance	Audit-log redaction record	JSON	.../audit-log-redaction.json

References

• references/erasure-verification-job.md — Full Python implementation of the cascade orchestrator + independent verification job + proof pack writer.
• Companions: ai-agent-memory (the cascade implementer), saas-tenant-data-portability-and-erasure (tenant-level), ai-agent-audit-log-integrity (redaction not deletion), ai-agent-evidence-automation (pack pipeline), ai-agent-hipaa-security-controls (PHI agent erasure constraints), uganda-dppa-compliance (KE / UG specifics), ai-agent-soc2-controls (C1.2, P5).

§1 Intake

# privacy/erasure/intake.py
from dataclasses import dataclass
from datetime import datetime, timedelta

REGULATORY_CLOCKS = {
    "GDPR":  timedelta(days=30),
    "CCPA":  timedelta(days=45),
    "POPIA": timedelta(days=30),    # "reasonable time" — internally bound to 30
    "KE_DPA": timedelta(days=30),
}

@dataclass
class ErasureRequest:
    request_id: str
    tenant_id: str
    subject_id: str
    data_classes: list[str]
    legal_basis: str
    received_at: datetime
    requester_identity_proof_ref: str
    sla_deadline: datetime

def accept(request: dict) -> ErasureRequest:
    req = ErasureRequest(
        request_id=request["request_id"],
        tenant_id=request["tenant_id"],
        subject_id=request["subject_id"],
        data_classes=request.get("data_classes", ["working","episodic","semantic","vectors","fine_tune","uploads","derivatives"]),
        legal_basis=request["legal_basis"],
        received_at=datetime.fromisoformat(request["received_at"]),
        requester_identity_proof_ref=request["requester_identity_proof_ref"],
        sla_deadline=datetime.fromisoformat(request["received_at"]) + REGULATORY_CLOCKS[request["legal_basis"]],
    )
    audit.emit(event_class="erasure_request_accepted",
               subject_id=req.subject_id, request_id=req.request_id,
               legal_basis=req.legal_basis, sla_deadline=req.sla_deadline.isoformat())
    return req

§2 The 9-Step Cascade

#	Step	Tier / Action
1	Working memory	Wipe in-flight session state for subject.
2	Episodic memory	Delete subject-tagged rows in episodic store; verify replication lag drained.
3	Semantic memory	Delete subject-attributed nodes; rebuild affected entity graph.
4	Vector store	Delete vectors by subject metadata; rebuild affected ANN shards.
5	Fine-tune corpora	Delete subject-attributed examples from training datasets; record affected model lineage.
6	Uploads / artefacts	Delete object-store blobs (with replicas across regions).
7	Derivatives	Delete embeddings, summaries, indexes derived from subject data.
8	Subprocessors	Issue delete API calls to LLM provider knowledge bases, vector-store SaaS, fine-tune providers; collect receipts.
9	Audit-log redaction	Redact (not delete) PII fields in action audit log entries; preserve hashes and chain.

# privacy/erasure/cascade.py
from datetime import datetime
import hashlib, json

CASCADE = [
    ("working",     wipe_working_memory),
    ("episodic",    delete_episodic),
    ("semantic",    delete_semantic),
    ("vectors",     delete_vectors),
    ("fine_tune",   purge_finetune_corpora),
    ("uploads",     delete_uploads),
    ("derivatives", delete_derivatives),
    ("subprocessor", delete_at_subprocessors),
    ("audit_log",   redact_audit_log),
]

def run_cascade(req: ErasureRequest) -> list[dict]:
    fp_before = subject_fingerprint(req)
    steps: list[dict] = []
    for name, fn in CASCADE:
        if name not in req.data_classes and name != "audit_log":  # audit_log always redacted
            continue
        rec = {"step": name, "started_at": datetime.utcnow().isoformat()}
        before = fp_before[name]
        try:
            out = fn(req)
            after = probe_residue(req, tier=name)
            rec.update({"status": "ok", "before_count": before, "after_count": after, "output": out})
        except Exception as e:
            rec.update({"status": "fail", "error": str(e)})
        rec["ended_at"] = datetime.utcnow().isoformat()
        steps.append(rec)
        audit.emit(event_class="erasure_step",
                   subject_id=req.subject_id, request_id=req.request_id, step=name,
                   status=rec["status"], after_count=rec.get("after_count"))
    return steps

subject_fingerprint(req) returns per-tier residue counts before the cascade, so the proof shows before / after.

Full per-step implementations and the residue probes are in references/erasure-verification-job.md.

§3 Independent Verification Job

The verification job is a separate code path that re-queries every tier (and subprocessors) using a freshly-resolved subject fingerprint and asserts residue is zero.

# privacy/erasure/verify.py
def verify(req: ErasureRequest) -> dict:
    probes = {
        "working":     probe_working_memory(req),
        "episodic":    probe_episodic(req),
        "semantic":    probe_semantic(req),
        "vectors":     probe_vectors(req),
        "fine_tune":   probe_finetune(req),
        "uploads":     probe_uploads(req),
        "derivatives": probe_derivatives(req),
        "subprocessor": probe_subprocessors(req),
        "audit_log_redacted": probe_audit_log_redaction(req),
    }
    all_clear = all(v["residue"] == 0 for k, v in probes.items() if k != "audit_log_redacted") and probes["audit_log_redacted"]["unredacted_pii_count"] == 0
    return {"verified_at": datetime.utcnow().isoformat(), "all_clear": all_clear, "probes": probes}

If all_clear == False, the erasure is not complete; an exception opens and the requester is not notified yet.

§4 Proof-of-Erasure Pack

evidence/erasure/{request_id}/
├── manifest.json
├── request.json                 # the validated ErasureRequest
├── steps.jsonl                  # one row per cascade step
├── verification.json            # the verification probe results
├── subprocessor-receipts.jsonl  # API ticket IDs per subprocessor
├── audit-log-redaction.json     # the redaction range + chain witness
├── attestation.txt              # signed by DPO (not the cascade runner)
└── signature.sig

Sample manifest.json:

{
  "pack_id": "erasure-req-001928",
  "request_id": "req-001928",
  "subject_id": "sub_5e3a... (hashed)",
  "tenant_id": "ten_0440",
  "legal_basis": "GDPR",
  "received_at": "2026-04-20T09:11:00Z",
  "completed_at": "2026-04-22T15:44:00Z",
  "sla_deadline": "2026-05-20T09:11:00Z",
  "all_clear": true,
  "signer": "dpo@example.com",
  "signature_key_id": "compliance-dpo-2026",
  "files": [...]
}

§5 Anti-Patterns

• Soft-delete (deleted_at flag) — not erasure under GDPR/CCPA/POPIA/KE DPA.
• Vector "remove from index" without underlying record purge.
• Deleting fine-tune corpora but keeping the fine-tuned model. Either retrain or document a lawful retention basis (Art. 17(3)).
• "Deletion confirmed via email" from a subprocessor. Require API confirmation with ticket ID.
• Deleting audit log rows. Redact PII; keep the chain.
• Verification done by the same code path as the cascade. Independent probes are required.
• Pack signed by the engineer who ran the cascade. Sign by an independent DPO.
• Re-running the cascade is unsafe (non-idempotent). Each step must tolerate replay.
• Not capturing the subprocessor receipts. The auditor will ask for them; verbal assurance does not pass.

§6 Cross-Links

• ai-agent-memory — implements the actual tier deletes; this skill consumes those.
• saas-tenant-data-portability-and-erasure — whole-tenant erasure orchestrator; references this skill for the agent-memory leg.
• ai-agent-audit-log-integrity — defines the PII redaction (not deletion) flow for the audit log; redaction record is part of the proof.
• ai-agent-soc2-controls — C1.2 (confidential information disposal), P5 (retention and disposal).
• ai-agent-hipaa-security-controls — additional constraints when subject is a patient (BAA / 164.310(d)(2)(i) media disposal).
• uganda-dppa-compliance — KE / UG specifics for legal_basis ∈ {KE_DPA, UG_DPPA}.