SuperAPI AI Directory
Partner with us中文AI newsFavorites
Skillspackage-langfuse-red

package-langfuse-red

Agents & models

View repository

Updated 10-16 02:38

About this skill

Langfuse Package Skill (Red)

Provisions six AWS or Vultr machines in one VPC and converges Langfuse v4.27.0 across them: a Neon storage tier (storage broker, pageserver, one safekeeper, Postgres 17 under compute_ctl, layers and WAL in R2), Redis 7.2, three ClickHouse replicas each with a Keeper voter, and the app host running langfuse-web, langfuse-worker and Caddy behind Cloudflare.

The Neon tier is not reimplemented here: this package SHA-pins getcolors/neon and renders its templates out of the installed package. The ClickHouse templates are this package's own, derived from getcolors/clickhouse.

Install the launcher

npx skills add getcolors/langfuse
cp .agents/skills/package-langfuse-red/red ./red
chmod +x red

The root red is a copy of the payload, not a symlink. npx skills update -p rewrites the payload and leaves the copy alone, so copy it again after every update or the project keeps running the old pin.

Verbs

./red build              # render .colors/<profile>/ — no provider calls, no credentials
./red create --dry-run   # walk the workflow, skip every side effect
./red create             # converge for real; the gates run inside it
./red rehearse           # restore both stores from backup, boot the pinned image, drill
./red describe           # every host's last monitor result, over SSH
./red delete             # guarded by compute-prevent-destroy; removes nothing in R2

build and --dry-run work on a fresh checkout with an empty environment. Exit code 2 means validation failure and lists every problem at once.

Credentials

Non-secret desired state lives in colors.yml. Every credential is a COLORS_PAR_* environment variable, conventionally in a gitignored .envrc.private:

Variable For
COLORS_PAR_VULTR_API_KEY VPC, firewall groups, instances, SSH key resource
COLORS_PAR_CLOUDFLARE_API_TOKEN the DNS record; Zone:Read + DNS:Edit
COLORS_PAR_R2_ACCESS_KEY_ID / _SECRET_ACCESS_KEY OpenTofu state only; reaches no host
COLORS_PAR_LANGFUSE_STORAGE_R2_* events, media, exports (Object Read & Write on the storage bucket)
COLORS_PAR_NEON_R2_* Neon layers and WAL — the deployment's .envrc maps the storage pair onto these
COLORS_PAR_LANGFUSE_BACKUP_R2_* backups (Object Read & Write on the backup bucket)
COLORS_PAR_LANGFUSE_ENCRYPTION_KEY 64 hex chars, operator-held
COLORS_PAR_LANGFUSE_SALT ≥ 32 chars, operator-held
COLORS_PAR_LANGFUSE_INIT_USER_PASSWORD ≥ 12 chars, operator-held

Never export COLORS_PAR_PROFILE: it selects the deployment's remote state.

The package refuses a create when the storage or backup pair equals the state pair, or the backup pair equals the storage pair, unless colors.yml records r2-credential-sharing: shared-accepted as a deliberate, committed line.

Three secrets outlive the app host

ENCRYPTION_KEY encrypts every stored secret and SALT hashes every API key. A Postgres backup restored onto a fresh app host is readable only with the same two values, so they are the operator's to hold, never generated on a disk a rebuild would wipe. The init password logs you in after that rebuild, when the generated project keys are gone. Keep all three outside these machines.

Everything else — Neon, Redis and ClickHouse passwords, NEXTAUTH_SECRET, the project API keys — is generated on the host that owns it. sudo langfuse-credential on the app host prints the project keys.

What convergence guarantees

Gates that run on every converge and fail it if they fail:

  • raw TCP from the app host to Neon, Redis and the ClickHouse client ports, and a refusal on Keeper — the network says what desired state says
  • UTC on both databases; three replicas visible through clusterAllReplicas; an authenticated cross-node query; Keeper quorum; system.query_log present
  • a trace, a generation and a score ingested through the app-host API, read back, found on node 0 and the last replica, and a new raw-event object in R2
  • a media file up through a presigned URL and back with the same sha256
  • wrong API key, anonymous request, unauthenticated Redis PING, wrong ClickHouse and Postgres passwords all refused
  • 200 traces queryable within the timeout, host memory under 85 %
  • the backup credential cannot read live data; no backup credential in ClickHouse's query_log
  • the ready marker <prefix>.colors-ready, written last and read back

./red rehearse then proves recovery: fresh sets, restore both stores, boot the pinned image in a second Compose project on loopback with the operator-held keys, read the trace, the project and an encrypted LLM connection back through the API, stop a replica under ingestion, restart Redis with a job queued, and write .colors-recovery-verified.

Recovery, stated honestly

Failure Recovers from RPO
a ClickHouse replica the other two, automatically 0
a Redis restart the AOF 0
the Redis host nothing automated: queued jobs are lost, their raw events stay in R2 the queue
Neon local state R2 layers + safekeeper replay ~0
the Neon host the Postgres dump 6 h
the whole ClickHouse cluster the nightly native backup, paired with the next Postgres dump 24 h
the app host a converge with the operator-held keys; new project keys 0

Backups pair at restore time: a ClickHouse set with the oldest Postgres dump completed after it, so Postgres is the newer snapshot and every project a restored trace references exists.

Managed AWS deployment

Use provider-compute: aws, provider-backend: s3, s3-bucket-mode: managed, langfuse-storage-provider: s3, and langfuse-storage-managed: true. The deployment creates the state bucket before Terraform initialization, and creates separate Neon, Langfuse and backup buckets with scoped IAM credentials. Use distinct globally unique bucket names and native S3 endpoints/regions in the existing *-r2-* configuration keys. The three operator-held application secrets are still required.

Authorized delete removes the managed application buckets and their contents as well as the machines; the state bucket is finalized last. Existing R2 storage retains its prior preservation behavior.

The application smoke test uses loopback on the app host. Run an external client as well to prove public DNS, Cloudflare proxying and HTTPS.

Media in the browser

The API media path needs nothing more. UI rendering needs a CORS rule on the storage bucket for https://<langfuse-host>, which only an Admin R2 token can set; add it in the R2 dashboard or with wrangler r2 bucket cors put. The smoke gate reports the preflight as WARN until it is there. Managed AWS storage creates the CORS rule during bucket provisioning.

Configuration reference

See references/configuration.md.

Compute uses the pinned colors-compute library. The package declares roles and security policy; the library owns shared and per-node OpenTofu state, managed key lifecycle and cloud credentials. Use R2 or S3 state. Builds render compute/shared and compute/nodes/<node_id>. Existing monolithic <profile>/langfuse-infrastructure.tfstate requires explicit migration; do not create over it or remove the refusal guard.