SuperAPI AI Directory

agent-memory-testing

Testing & quality

View repository

Updated 10-16 01:34

About this skill

Agent Memory Testing (Chinese Version)

When to Use

  • When evidence-based analysis, design, or verification preparation is needed around memory write/read/update/delete, retention, contamination, isolation, provenance, and forgetting behavior.
  • When an Agent, RAG, or LLM solution, result, or evidence needs review, and actionable improvement items need to be produced.
  • When input is incomplete but a constrained initial version with assumptions, information gaps, and human decision boundaries still needs to be delivered.

Output Format Options

  • Default output is Markdown, organized by domain risk, evidence status, priority, and boundaries.
  • When the user requests table, CSV, JSON, or ticket format, preserve the same finding fields, evidence, and decision boundaries.
  • Before output enters an automated process, first confirm the schema, enum values, required fields, and evidence sources.

How to Use

  1. First read and follow prompts/agent-memory-testing.md, including input audit, domain coverage, and output order.
  2. Extract scope, environment, version, time window, constraints, success criteria, and available evidence, with focus on checking write-read, update-delete, retention policy, tenant isolation, and provenance tracking.
  3. First distinguish confirmed facts, evidence-supported inferences, candidate recommendations, and Human decisions, then sort by risk and evidence strength.
  4. Convert high-risk items into preconditions, steps, expected results or judgment criteria, required evidence, and verification methods.
  5. When information is insufficient, deliver a constrained initial version, clarify which conclusions cannot hold, and do not write static materials as execution results.

Reference Files

  • When calibrating domain results, correct comparisons, or evidence gaps, read references/domain-decision-example.md 's desensitized input, item-by-item expectations, and recalculation basis.

  • Each execution must read prompts/agent-memory-testing.md; it is the complete execution specification of this Skill.

  • When evaluation is needed, read evals/eval.yaml and the matching cases under evals/cases/ .

  • Only read references/、examples/、scripts/ or output-formats.mdwhen the directory actually exists and the task requires it; do not assume assets that do not exist.

Domain Execution Key Points

  1. For memory, record tenant/user ID, memory ID, content version, source, write time, validity period, and retraction/deletion status; retrieval results retain the source version, and do not only check the final answer.
  2. First select old/new preferences according to project priority rules; when conflicts are undefined, mark them as pending confirmation, and do not default to last-write or self-defined retention periods.
  3. For two tenants with the same query, verify the visible sets separately; after a version update, retrieval of outdated content, return after expiration, and return after deletion are judged according to the project consistency window.
  4. Deletion checks the actual scope including indexes, caches, and backups; when no deletion propagation/backup retention protocol is provided, report only the verified parts. Normal allowed retrieval serves as the control for isolation checks.

Core Constraints

  • Only analyze memory write/read/update/delete, retention, contamination, isolation, provenance, and forgetting behavior; do not replace the business owner's or Human's risk acceptance, exception approval, and security decisions.
  • Do not fabricate system behavior, fields, model outputs, data, thresholds, root causes, execution records, or pass conclusions.
  • Static design, plans, file existence, or dry-run can only retain the original evidence status and cannot be upgraded to proof of real execution.
  • When evidence is insufficient, use pending confirmation, blocked, unassessed, or NOT_SCORED, and provide the minimum verification method.
  • When user data, production, or security is involved, use least privilege, desensitized data, mocks, dry-run, or an isolated environment.

Self-check before delivery

  • Write-read, update-delete, retention policy, tenant isolation, and provenance tracking are covered, with source, evidence status, and verification method recorded for each item.
  • Facts, inferences, candidate recommendations, information gaps, and Human decisions are separated.
  • High-risk items have P0/P1/P2/P3 or equivalent priority, responsible role, and closure conditions.
  • Plans, static checks, or dry-runs are not written as test execution, all passing, or security approval.
  • Residual risks, stop/escalation conditions, and next steps are stated.

Common pitfalls

  • Listing only checkpoints without stating trigger conditions, expected focus points, responsible roles, closure conditions, and evidence.
  • Treating test or model tool names from adjacent domains as the complete judgment of Agent memory.
  • Using unexplained numbers to create false precision, or writing correlation as causation.
  • Directly refusing when the input is incomplete, or conversely pretending the conclusion is already certain.

Best practices

  • Start from the path most likely to cause user harm, business loss, or decision blockage.
  • Use the smallest verifiable experiment to reduce uncertainty, and record conditions, versions, sources, and evidence.
  • Enable another engineer to independently install, execute, and review the output of this Skill.