The LLM is not the source of truth. It retrieves excerpts, generates with citation, and updates the index when the fact changes. Model client: claude-api / chatgpt-api / gemini-api / ollama-api. Classic search if the repo already has it: elasticsearch.
Preserve the project's store (pgvector, Chroma, Pinecone, OpenSearch, …). Postgres: pgvector. Do not swap the embedding model without reindexing the entire corpus.
Agent's duty — update the RAG
Whenever the relevant information changes (doc, FAQ, policy, price, schema, runbook, user correction, code that feeds the corpus), change the RAG. Do not "teach" only in the system prompt nor leave the index stale.
- Find the canonical document/chunk (
source_id, path, URL, slug). - Edit the source (markdown, CMS, table
documents) — the index is derived. - Upsert chunks + embeddings of the doc; delete orphan vectors of the same
source_id. - Update
updated_at/ version; invalidate retrieval cache. - If the embedding model or the tokenizer changed: reindex full, not patch.
- Confirm in retrieve (smoke query) that the new fact appears and the old one does not.
Prohibited: answering with a new fact only in the current conversation and not persisting; duplicating the same doc with new IDs on every edit.
Memory (three layers)
| Layer | What it stores | Where |
|---|---|---|
| Working | Turn, plan, tool results | Request context (short) |
| Episodic | Preferences, decisions, "user X prefers Y" | Memory store with TTL, user_id, consent |
| Semantic | Product/domain facts | RAG corpus + vectors + metadata |
- Do not mix session PII into the public/tenant-wide index.
- Episodic memory is explicit write (validated summary), not a dump of the conversation.
- On conflict: versioned semantic fact wins over old preference; ask if it is preference vs. policy.
- Forget: delete/TTL on episodic; tombstone on semantic.
Intelligence (retrieve better, not a bigger prompt)
- Rewrite the query (typos, turn context, HyDE if the project already uses it).
- Hybrid: vector + lexical (BM25/ES). Vector alone fails on IDs, codes, proper nouns.
- Filters: tenant, language,
doc_type, current (valid_to). ACL before generating. - Rerank our top-k if the project has a rerank model; otherwise small k and minimum score.
- Grounding: generate only from excerpts; cite
source. Empty retrieval or low score → “I don't know” / ask for clarification — do not make things up. - Agent: retrieve → gap → new query (max N cycles). No infinite loop.
- Routes: distinct collections (docs vs tickets vs code). Not a single “vector dumpster”.
Learning
- User correction or “this is wrong” → patch in the corpus (homework steps) + log
query,chunk_ids,gold. - Feedback (👍/👎, clicked citation) becomes an eval signal, not an invisible weight in the prompt.
- Continuous eval: recall@k in retrieval; groundedness/faithfulness in the response. Versioned golden set.
- Do not fine-tune the LLM “to remember facts” — that is RAG. Fine-tune only if the repo already has that pipeline.
Pipeline
fonte → chunk (overlap, ids estáveis) → embed → upsert
query → rewrite → retrieve (+ filter) → rerank → generate + cites
- Chunk by structure (title, section), not just blind N tokens. Small and consistent overlap.
- Stable ID:
hash(source_id + chunk_index + content_version)or the project's equivalent. - Same embedding model at ingestion and at query time.
- Minimal metadata:
source_id,source_uri,title,updated_at,acl,locale.
Non-negotiable rules
- Secrets, keys, and third-party PII outside the corpus.
- Idempotent upsert per document; cascading delete of chunks.
- Generate context: retrieved excerpts + question — not the database dump.
- Tenancy: query embedding does not break through
WHERE tenant_id =. - Observability: query, k, scores,
source_idsused (without logging PII).
When to reindex
| Event | Action |
|---|---|
| Edit of a doc | Upsert of that source_id |
| Delete from the source | Delete vectors + linked memory |
| New doc type | Collection/metadata; backfill |
| Embedding model change | Full reindex |
| Drift (eval dropped) | Audit duplicate/stale chunks |
Ingestion in CI or a worker, not in the synchronous request path unless the product is already “upload → question”.
Anti-patterns
- System prompt with the entire manual “because RAG is hard”
- Index that never updates
- Re-embed on every chat message
- One vector per 80-page PDF
- Mixing user A's memory into user B's retrieve
- Changing the embedding model and keeping old vectors
- Answering without citation in a factual domain (legal, pricing, medical-adjacent)
Integration with other skills
- LLMs / embeddings HTTP:
claude-api,chatgpt-api,gemini-api,ollama-api - SQL persistence + vectors:
pgvector(the repo's Prisma/Drizzle/sqlc) - Lexical index:
elasticsearch - Go-live:
production-readiness(secrets, PII, timeouts)
Completion criteria
- Editable canonical source; derived and updatable index
- Agent (and the app) persist de facto changes in the RAG
- Episodic memory separate from semantic memory, with ACL
- Hybrid or justified retrieval; refusal if there is no excerpt
- Eval or smoke query after reindex