omnigraph-best-practices

Operating Omnigraph Locally

This skill captures the operational rules for working with a locally deployed Omnigraph (RustFS-backed or remote S3). Follow them when authoring schema, writing queries, loading data, evolving schema, or automating graph operations.

The Seven Rules

1. Lint before commit — omnigraph lint --schema schema.pg --query queries/foo.gq validates both sides against each other. No running repo required. (omnigraph query lint still works as a deprecated alias.)
2. Plan before apply — never run schema apply without a successful schema plan first. Apply is destructive; plan is free. (Cluster mode has the same rule with different verbs: cluster plan before cluster apply — the plan embeds the engine's real migration steps.)
3. Branches are for data; apply is for schema — review data ingests on a feature branch then merge. Schema changes go straight to main (single-graph: omnigraph schema apply; cluster mode: edit the .pg and run cluster apply — there is no direct schema apply in cluster deployments).
4. Pick the right write command — mutate for edits (typechecked, parameterized), load --mode merge for bulk upsert on local repos, ingest for remote, load --mode overwrite only for clean slates.
5. Parameterize everything — never string-interpolate values into .gq bodies or --params. Declare $var: Type and pass via --params.
6. Expose agent operations as aliases — not raw CLI invocations. Aliases decouple the operation name from the query implementation.
7. Verify after every remote write — compare commit list --branch main head before and after. The CLI's exit code is not authoritative on remote graphs; proxies can drop the response while the write commits server-side. See references/remote-ops.md for the verification ritual and how to recover from 504s.

Five Ontology Design Criteria (Gruber 1993)

Omnigraph schemas are ontologies. The canonical design criteria from Gruber's Toward Principles for the Design of Ontologies Used for Knowledge Sharing (Int. J. Human-Computer Studies 43:907–928) apply directly when authoring .pg files.

1. Clarity — definitions should communicate intended meaning unambiguously and be independent of social or computational context. In Omnigraph: precise type names, narrow enums over String, @check/@range for stated invariants. A reviewer should understand the domain from the schema alone.
2. Coherence — inferences sanctioned by the schema must be consistent with the domain modeled. Gruber's trap: defining quantity as a (magnitude, unit) pair makes 6 feet ≠ 2 yards even though they describe the same length. In Omnigraph: watch for @card, @unique, and edge directionality that let the schema distinguish things the domain treats as equal.
3. Extendibility — the schema should support specialization without revising existing definitions. In Omnigraph: prefer interfaces for shared shape, leave enums open where the domain genuinely admits more, model identifiers via mapping functions rather than baking units/formats into the entity.
4. Minimal encoding bias — representation choices made for notation or implementation convenience leak into the model. In Omnigraph: don't type dates as String because the source API returns strings; separate conceptual entities (a publication date, a person) from their surface encoding (a year integer, a name string) when both matter.
5. Minimal ontological commitment — make as few claims about the world as the use case requires. In Omnigraph: don't add required properties, closed enums, or @card(1..1) "in case"; tighten later via schema plan/apply when a real constraint emerges. Weaker schemas leave consumers room to specialize.

The criteria trade off against each other — Clarity wants tight definitions while Minimal Commitment wants weak ones. Gruber's resolution: having decided a distinction is worth making, give it the tightest possible definition. Decide what to model conservatively; once modeled, constrain precisely.

Schema Authoring Principles

Twelve practical rules for .pg authoring — full text and examples in docs/omni-schema.md. In short: schema-is-the-contract · explicit identity via @key · model meaning not tables · strong intentional types · deliberate optionality · shared shape in interfaces · schema-level constraints (@unique/@index/@range/@check/@card) · search as a schema decision · edge semantics matter · reviewable schemas · intentional migrations (@rename_from) · domain clarity over ORM habits.

Design flow: entities → stable keys → relationships worth their own edge → enum candidates → uniqueness/bounds/cardinality → search needs → shared shape into interfaces → evolution plan.

Provenance Is Structural (Multi-Agent Source of Truth)

When Omnigraph serves as canonical truth across multiple agents, every assertion must answer who said it, when, based on what evidence. This is the runtime guarantee Gruber's criteria don't cover — his agents shared vocabulary; ours additionally must share attribution. Provenance belongs in the schema, not in logs.

Without structural provenance, agents cannot reconcile contradictory assertions, retract facts when a source is discredited, replay graph state at a past timestamp, or distinguish high-evidence facts from speculation.

In Omnigraph: model provenance as a Claim-style interface (or a separate Claim node linked to each sourced fact) with required fields — asserted_by: Actor, asserted_at: DateTime, evidence_source: Source, optionally confidence: F64. Don't stash provenance into a free-text source: String or a metadata: JSON dump — structured provenance is queryable, indexable, and migratable; free-form is none of these.

Local Setup

Bootstrap a local RustFS + Omnigraph in one command

Requires Docker. RustFS runs in a container — install Docker and verify before running the bootstrap:

docker version >/dev/null 2>&1 || { echo "Install Docker first: https://docs.docker.com/get-docker/"; exit 1; }
curl -fsSL https://raw.githubusercontent.com/ModernRelay/omnigraph/main/scripts/local-rustfs-bootstrap.sh | bash

Defaults: RustFS S3 on 127.0.0.1:9000, console on :9001, omnigraph-server on :8080, bucket omnigraph-local. Override with BUCKET=foo PREFIX=repos/bar BIND=127.0.0.1:8080 curl ....

Heads up — port :8080 is in use after bootstrap. The bootstrap auto-starts an omnigraph-server against its own demo repo. If you start a second server (e.g. for a different cookbook) without stopping the first, you'll get a silent port collision. Either stop the bootstrap server or start yours with --bind 127.0.0.1:8090.

Bootstrap also installs omnigraph and omnigraph-server binaries under <workdir>/.omnigraph-rustfs-demo/bin/ — not on PATH by default. Add it or invoke binaries by absolute path.

AWS env vars (for init, load, and the server)

init and load write S3-backed storage directly, and omnigraph-server reads from it. Both need AWS credentials pointed at RustFS. Keep them in .env.omni (git-ignored):

AWS_ACCESS_KEY_ID=rustfsadmin
AWS_SECRET_ACCESS_KEY=rustfsadmin
AWS_REGION=us-east-1
AWS_ENDPOINT_URL=http://127.0.0.1:9000
AWS_ENDPOINT_URL_S3=http://127.0.0.1:9000
AWS_ALLOW_HTTP=true
AWS_S3_FORCE_PATH_STYLE=true

Source before running CLI commands:

set -a && source .env.omni && set +a

Or reference auth.env_file: .env.omni in omnigraph.yaml for automatic loading.

Validate the setup

curl http://127.0.0.1:8080/healthz
omnigraph snapshot s3://omnigraph-local/repos/<your-repo> --json

Project Layout

Two deployment models — pick one per project

• Cluster mode (omnigraph >= 0.7.0, recommended for new projects): a cluster.yaml declares the deployment (graphs, schemas, stored queries, policies); omnigraph cluster apply converges it and omnigraph-server --cluster . serves it. omnigraph.yaml shrinks to per-operator ergonomics (aliases, CLI defaults, cli.actor). See references/cluster.md.
• Single-graph mode (classic): omnigraph.yaml is the operational backbone below. Fully supported; required for S3-hosted graphs today.

A server boots from one source or the other — never a merge of both.

omnigraph.yaml is the operational backbone (single-graph mode)

Run CLI commands from the folder containing omnigraph.yaml — relative paths for queries/, schema.pg, and .env.omni resolve from there.

graphs:
  local_s3:
    uri: s3://omnigraph-local/repos/spike-intel
  local_server:
    uri: http://127.0.0.1:8080
    bearer_token_env: OMNIGRAPH_BEARER_TOKEN

server:
  graph: local_s3
  bind: 127.0.0.1:8080

cli:
  graph: local_s3
  branch: main
  output_format: jsonl      # use `table` for human reading; `jsonl` for automation

auth:
  env_file: .env.omni

query:
  roots: [queries, .]

aliases:
  signal:
    command: query        # `read` still accepted (deprecated)
    query: signals.gq
    name: get_signal
    args: [slug]
    format: kv

Key naming: the config field is graphs: (not targets: — that's the old schema). cli.graph and server.graph replaced cli.target / server.target. The CLI flag is still --target <name> though.

What to commit

Commit: schema.pg, queries/*.gq, omnigraph.yaml, seed.md, seed.jsonl, per-cookbook README.md and CLAUDE.md.

Ignore: .env.omni (credentials), .claude/ (local agent state), *.omni/ (local repo artifacts), .omnigraph-rustfs-demo/ (bootstrap state).

Give agents a CLAUDE.md

A per-cookbook CLAUDE.md tells coding agents where files live and what conventions matter. Without it, agents re-discover the same things every session.

Common Gotchas

These are the traps most likely to bite. Scan this table before debugging any parse or runtime error.

Trap	Symptom	Fix
# comments in .pg	parse error: expected schema_file	Use //
Standalone enum Foo { ... } block	parse error: expected EOI or schema_decl	Inline: kind: enum(a, b)
[Category] (list of enum)	compile error	Use [String]; lists must contain scalars
@embed(text) without quotes	unexpected constraint_name	@embed("text")
@unique(src) on edge without body block	parse error	@card(1..1) { @unique(src) }
load --mode merge after @embed source change	stale embeddings	omnigraph embed --reembed_all or load --mode overwrite
schema apply with feature branches open	rejected	Merge or delete branches first
nearest(...) / bm25(...) / rrf(...) without limit	compile error	Add limit N
Adding non-nullable property without backfill	unsupported migration	Make optional → backfill → tighten in follow-up apply
Config uses targets: / target:	graph 'X' not found in omnigraph.yaml	Rename to graphs: / graph:
omnigraph init --json	unexpected argument --json	init doesn't support --json; drop the flag
omnigraph init on an already-initialized URI	AlreadyInitialized error (v0.6.0+)	--force to re-init (skips the schema preflight; does not purge data)
schema apply dropping a property/type	soft-dropped or rejected (no data loss)	add --allow-data-loss to actually drop the column
Committing .env.omni	credential leak	Add .env* to .gitignore
Non-parameterized query values	typecheck surprise, injection risk	Declare $param: Type and pass via --params
Missing required field in insert	T12: insert for 'X' must provide non-nullable property 'Y'	Accept the param in the mutation signature
Long-lived feature branches	merge conflicts, schema apply blocked	Merge promptly; delete when done
mutation { ... } wrapper in .gq	parse error: expected query_file at line 1	Use query <name>(...) { insert T { ... } }; there is no top-level mutation keyword
--config placed before subcommand	unexpected argument --config	Put --config after the subcommand (e.g. omnigraph schema show --config X)
Reading a large schema via stdout-capped tool	Truncated, garbled, or duplicated output	omnigraph schema show > /tmp/schema.pg first; then read the file with offset/limit
omnigraph load against a remote URI	load is only supported against local repo URIs in this milestone	Use ingest for remote graphs
Blind retry after 504	Duplicate Signal/Decision/Claim (append-only types lack @key dedup)	commit list --branch main --json first; head advanced means it landed; only retry if unchanged
sync_branch() mentioned in version-drift error	Searching for nonexistent CLI command	Server-internal directive in error text; retry once, fall back to ingest if persistent
Stale ingest/<name> branches at main's head	504-orphaned empty branches; eventually block writes	List branches, find ones at main's graph_commit_id, omnigraph branch delete --config X <name>
Top-level policy:/queries: with a named graph (server.graph/--target)	server refuses to boot with migration guidance (v0.6.1)	Nest under graphs.<name>.policy / graphs.<name>.queries. Top-level is valid only for an anonymous bare-URI single-graph server
omnigraph optimize against a table with a Blob property	table is skipped, not failed (Lance blob-v2 compaction bug)	Expected — --json reports it under skipped; non-blob tables still compact

Edge Traversal Casing

Schema declares edges in PascalCase (FormsPattern), but queries traverse them in lowerCamelCase:

match {
    $s: Signal
    $s formsPattern $p       // edge FormsPattern: Signal -> Pattern
}

Deep Dives

• references/cluster.md — cluster-mode declarative deployments: cluster.yaml, the validate/import/plan/apply loop, approval-gated deletes, --cluster serving, the two-file contract, recovery

For anything beyond the basics, load the relevant reference file. Each is self-contained — load only what you need.

Reference	When to load
references/schema.md	Editing .pg files, running schema plan/apply, renaming types, backfilling required fields
references/queries.md	Writing or linting .gq files, search functions, aggregations, multi-hop patterns
references/data.md	Choosing between mutate, load, and ingest; branch review workflow; destructive ops
references/remote-ops.md	Operating against a remote/CloudFront-fronted graph: 504 verification ritual, version drift, ingest fingerprints, append-only retry safety
references/search.md	Embeddings, @embed, vector/text ranking, scope-then-rank pattern
references/aliases.md	Defining aliases for agents, structured output, JSON args
references/stored-queries.md	Server-side stored-query registry (v0.6.1): queries: config, omnigraph queries validate/list, GET /queries + POST /queries/{name}, invoke_query Cedar gating, MCP exposure
references/server-policy.md	Starting the HTTP server, routes, bearer auth, Cedar policy gating, multi-graph mode
references/commands.md	snapshot, export, commit list/show, config resolution order