SuperAPI AI Directory
Partner中文AI newsFavorites
Skillsmetro-ai-apps-recipe

metro-ai-apps-recipe

Deployment & operations

View repository

Updated 10-16 02:39

About this skill

Metro AI Apps Recipe — DLSPS + WebRTC + Mosquitto + Node-RED + Grafana + Nginx

Build an end-to-end {{OBJECT}}-analytics stack on Intel hardware in ./{{STACK_DIR}}/ with Docker Compose. Vertical-agnostic: the same seven-container topology (Nginx, DLSPS, Mosquitto, Node-RED, Grafana, MediaMTX, Coturn) serves any DL Streamer / OpenVINO CV pipeline — only the model, class filter, alert rule, dashboard, and topic names differ. Follows the open-edge-platform Metro Vision AI App Recipe MediaMTX + Coturn + WebRTC path, streamlined (no Prometheus/OTel). SceneScape is off by default — opt-in multi-camera spatial analysis (references/SCENESCAPE.md). Detection metadata flows DLSPS→MQTT→Node-RED→Grafana; video is decoupled (DLSPS overlays via gvawatermark, pushes each source to MediaMTX over WHIP, ENABLE_WEBRTC=true, per-source peer-id) — see architecture below.

When to use this skill

Use when building or deploying an object-detection, classification, object-counting, or zone/line-crossing alerting pipeline for any vertical — smart city/ITS, retail, industrial, logistics, healthcare, or a custom OpenVINO/ONNX model. Optionally adds a SceneScape multi-camera spatial-analysis path, or a lightweight demo/PoC single app (one DL Streamer pipeline or a minimal OpenVINO inference script) when no full stack is needed.

Not for: non-Intel or cloud-only deployments, Prometheus/OpenTelemetry metrics stacks, or training/exporting models.

Supported verticals & use-cases

Vertical Example use-cases (each = one invoking prompt)
Smart city / ITS person/vehicle detection, ANPR, smart-parking, wrong-way
Retail customer counting, queue-length, shelf out-of-stock, dwell-time
Industrial / logistics surface-defect, PPE compliance, zone intrusion, pallet/forklift tracking
Healthcare / facilities fall detection, hand-hygiene, occupancy, perimeter intrusion
Custom any OpenVINO IR / ONNX detector + optional classifier

The invoking prompt maps its vertical to concrete {{OBJECT}}, {{PIPELINE_NAME}}, {{DEFAULT_MODEL}}, {{DEFAULT_RULE}}, and {{DASHBOARD_SLUG}} — nothing else changes.

How to use this skill

  1. Read this file end-to-end.
  2. Ask Question 0 (mode) first. If Demo/PoC, branch to Demo/PoC mode + load references/DEMO_POC.md — skip questions 1–7. Else (Full-stack production, default) continue.
  3. Ask the 7 questions in ONE batched message (defaults in brackets); accept go/defaults/empty. Question 7 selects the SceneScape path.
  4. Run parameter validation (below); refuse to proceed on any failure.
  5. Load reference file(s) on demand per component — not all up front. Load references/SCENESCAPE.md only when {{SCENESCAPE}}=yes.
  6. Verify against completion criteria before declaring success; record throughput/latency against the benchmark.md baselines.

Reference files (load on demand)

File Load when authoring
references/PIPELINE.md DLSPS config.json, GPU/NPU variants, REST launcher, watchdog
references/PROXY_UI.md nginx.conf proxy (WHEP/WHIP + WebRTC-TCP), Grafana iframe panels, dashboard provisioning, Mosquitto
references/NODE_RED.md flows.json, MQTT wildcard, gva_meta probe, alert flow
references/INSTALL.md file layout, .env, validate_env.sh + validation rules, install.sh, docker-compose.yml (MediaMTX + Coturn) volumes
references/TESTS.md conftest.py, test_webrtc_stream.py, assertion contracts for other tests
references/SCENESCAPE.md Only when {{SCENESCAPE}}=yes — opt-in multi-camera scene-fusion path, delegating to the scenescape-setup skill
references/DEMO_POC.md Only when {{MODE}}=demo — lightweight single-app path (DL Streamer or OpenVINO); no full stack

Parameters (from invoking prompt)

Param Purpose
{{MODE}} demo | production (default production). demo = lightweight single-app path (references/DEMO_POC.md); params below apply to production only
{{OBJECT}} class label in dashboard/alerts (e.g. person, vehicle, hardhat, defect, fall) — any MQTT/Grafana-safe string
{{STACK_DIR}} e.g. person-detect-stack, ppe-compliance-stack, anpr-stack
{{DEFAULT_MODEL}}, {{OTHER_MODELS}} allowed model options
{{PIPELINE_NAME}} canonical DLSPS pipeline name (e.g. yolov11s); variants <name>/<name>_gpu/<name>_npu; topic {{DETECTIONS_TOPIC_PREFIX}}_X/<name>
{{CLASSIFIER}} secondary model or none; if set, also {{CLASSIFIER_URL}} + {{CLASSIFIER_XML}}
{{CLASS_FILTER_IDS}} JSON array of class IDs to keep ([]=all). Filtered in Node-RED
{{DEFAULT_RULE}} e.g. count>2 in 10s
{{RULE_SCOPE}} per-source | aggregate (default per-source)
{{ALERT_TOPIC}} e.g. alerts/{{OBJECT}}
{{DETECTIONS_TOPIC_PREFIX}} e.g. object_detection (per-source _1, _2, …)
{{COUNT_TOPIC}} e.g. stats/{{OBJECT}}_count
{{LABEL_RULE_NOTE}} model-specific classification note for Node-RED
{{DASHBOARD_SLUG}} e.g. smart-parking
{{NUM_SOURCES}} default 4
{{SCENESCAPE}} yes | no (default no). yes = opt-in multi-camera spatial-analysis path (references/SCENESCAPE.md)
{{SCENE_NAME}} (SceneScape only) human-readable scene name, e.g. intersection-1
{{CAMERA_IDS}} (SceneScape only) unique IDs (no /), one per input stream, in input order
{{TURN_USER}}, {{TURN_PASS}} Coturn / MediaMTX ICE credentials (default turnuser / a generated secret)

Questions (single batched prompt)

Question 0 — Mode [production]: demo (lightweight single-app PoC) or production (full end-to-end stack). If demo, STOP here and follow Demo/PoC mode; do not ask questions 1–7. Questions 1–7 apply only to production.

  1. Model [{{DEFAULT_MODEL}}] (also: {{OTHER_MODELS}})
  2. Classifier [{{CLASSIFIER}}] (or none)
  3. Device [CPU] (GPU, NPU, AUTO)
  4. Inputs [{{NUM_SOURCES}}× sample-video] (or RTSP URLs / /dev/videoN / local paths)
  5. Node-RED rule [{{DEFAULT_RULE}}, {{RULE_SCOPE}}]
  6. Alert channel [MQTT {{ALERT_TOPIC}}]
  7. SceneScape multi-camera spatial analysis? [{{SCENESCAPE}}, default no] (if yes, also collect {{SCENE_NAME}} + one unique {{CAMERA_IDS}} per input stream → references/SCENESCAPE.md)

Parameter validation (enforce BEFORE install.sh runs)

Ship validate_env.sh and call it as step 0 of install.sh; reject on any failure. Both the script body and the full validation rules table (MODE, HOST_IP, NUM_SOURCES, DEVICE, PIPELINE_NAME, topics, TURN creds, inputs, SceneScape params, …) are in references/INSTALL.md.

Reference architecture

Single Compose network app_network. Nginx publishes 80/443; Coturn also publishes 3478/udp (WebRTC TURN). Nginx routes: /api/→DLSPS REST, /grafana/→Grafana, /nodered/→Node-RED, /mediamtx/<pid>/→WHEP iframe, /<pid>/whep|whip→WHEP/WHIP signalling, /webrtc/→MediaMTX local TCP (ICE 8189). Data: DLSPS→MQTT→Mosquitto→Node-RED→Grafana (mqtt datasource); DLSPS→WHIP→MediaMTX (peer-id {{DETECTIONS_TOPIC_PREFIX}}_N, ICE/TURN via Coturn); Grafana embeds <iframe src="/mediamtx/{{DETECTIONS_TOPIC_PREFIX}}_N/">.

Demo/PoC mode

When Question 0 selects demo, do not build the full stack (no Compose topology, no MediaMTX/Coturn/Node-RED/Grafana/Nginx, no SceneScape). Produce a single lightweight app proving a model runs on Intel hardware. Two sub-paths (ask which):

  • DL Streamer app — a simple DL Streamer / GStreamer pipeline; delegate to the dlstreamer-coding-agent skill.
  • OpenVINO app — a minimal Python script (load → compile_model → infer → post-process); no dedicated skill, follow the OpenVINO 2026 docs.

Full guidance and lightweight completion criteria are in references/DEMO_POC.md — load only on this branch; production criteria (1–11) do not apply in demo mode.

SceneScape spatial-analysis path (optional, {{SCENESCAPE}}=yes)

When Question 7 selects SceneScape, branch: keep the DLSPS detection pipeline but replace the MediaMTX/WebRTC + Node-RED + Grafana-MQTT tail with an Intel® SceneScape multi-camera scene-fusion stack. Do not re-implement by hand — delegate to the scenescape-setup skill, passing {{SCENE_NAME}} and {{CAMERA_IDS}}. Architecture, images, validation, and completion criteria are in references/SCENESCAPE.md; load only on this branch. The default ({{SCENESCAPE}}=no) path is unchanged.

Images — pin to the latest available tag (never :latest)

Resolve each image to the newest published stable tag on Docker Hub (query the repo's tags API with ordering=last_updated), then pin that concrete version — never floating :latest, and ignore *-weekly pre-releases. Grafana is the one exception: keep it pinned to 11.5.4 (the MQTT datasource plugin only works on that tag).

  • intel/dlstreamer-pipeline-server:2026.1.0-ubuntu24
  • eclipse-mosquitto:2.1.2-alpine
  • nodered/node-red:5.0.4
  • nginx:1.31.3-alpine
  • bluenviron/mediamtx:1.20.0 (WebRTC: WHIP in, WHEP out)
  • coturn/coturn:4.17.0 (ICE/TURN)
  • grafana/grafana:11.5.4 (pinned — do not upgrade) with GF_INSTALL_PLUGINS="grafana-mqtt-datasource 1.3.3,yesoreyeram-infinity-datasource 3.11.1" — a bad plugin version (plugin.versionNotFound) kills the container → Nginx 502
  • intel/dlstreamer:2026.1.0-ubuntu24 (one-shot in install.sh: model download + INT8 quantize + TLS cert)

Layout (flat)

Generate a flat {{STACK_DIR}}/ — top-level README.md, docker-compose.yml, .env, validate_env.sh, install.sh, the sample_*.sh/update_dashboard.sh scripts, a src/ tree (dlstreamer-pipeline-server/, mosquitto/, node-red/, grafana/, nginx/), and tests/. The full annotated file tree is in references/INSTALL.md.

README.md (required content)

The generated README.md MUST document, at minimum:

  • Architecture — data path DLSPS → MQTT (Mosquitto) → Node-RED → Grafana plus decoupled video DLSPS WHIP → MediaMTX → browser WHEP (Grafana <iframe>), Coturn for ICE/TURN, all behind the Nginx TLS proxy. Include the reference ASCII diagram and the seven containers (nginx, dlstreamer-pipeline-server, broker, node-red, grafana, mediamtx, coturn).
  • Quick start — ./install.sh → docker compose up -d → ./sample_start.sh <cpu|gpu|npu>, plus stop/status scripts.
  • Access URLs and credentials — dashboard https://<HOST_IP>/grafana/, default Grafana login admin / admin (state explicitly; prompts to change on first login), DLSPS REST base https://<HOST_IP>/api/pipelines, WHEP player https://<HOST_IP>/mediamtx/{{DETECTIONS_TOPIC_PREFIX}}_N/.
  • Configuration — key .env values (HOST_IP, GIDs, TURN creds) and how to swap the video source for RTSP.

Template variable substitution

Every {{VAR}} in code blocks MUST be substituted with its concrete value BEFORE writing the file — a literal {{...}} left in nginx.conf, config.json, flows.json, the dashboard JSON, or a test file is a syntax error.

Execution guardrails

  • Hard timeouts: model dl+INT8 300 s; video dl 120 s/file; compose pull 300 s; compose up -d 120 s + 180 s healthy; each pytest 60 s.
  • Max 2 retries per step, then STOP and print last 30 log lines from the failing container. Never loop.
  • Before compose up: ss -ltn must show :80 and :443 free, and ss -lun must show :3478 free (Coturn TURN).
  • Bypass host proxy for all localhost/LAN curl — corporate http_proxy/https_proxy routes https://localhost/... through an unreachable proxy (→ Could not resolve host / 502). Every curl in sample_*.sh MUST use --noproxy '*' and --cacert src/nginx/ssl/server.crt (self-signed cert); tests set NO_PROXY=* in conftest.py.
  • Test WebRTC signalling: curl --cacert src/nginx/ssl/server.crt --noproxy '*' -sf -o /dev/null -w '%{http_code}' https://<HOST>/mediamtx/{{DETECTIONS_TOPIC_PREFIX}}_1/ (expect 200; stream exists only after sample_start.sh).
  • Test MQTT: docker run --rm --network <project>_app_network eclipse-mosquitto:2.1.2-alpine mosquitto_sub -h broker -t '#' -v.
  • pytest venv at ./.venv inside stack dir (python -m venv .venv) — system pip is PEP-668 blocked; /tmp may be noexec.

Optional external skills

If available in the session, invoke; otherwise write files from the reference templates.

  • dlstreamer-coding-agent — pipeline JSON authoring (and the demo/PoC DL Streamer app when {{MODE}}=demo)
  • dlsps-user (oep/skills) — DLSPS deploy/config/REST; default {{MODE}}=production path (references/PIPELINE.md)
  • model-download (oep/edge-ai-libraries) — OMZ model IR
  • scenescape-setup (oep/skills) — only when {{SCENESCAPE}}=yes (references/SCENESCAPE.md)

Reference implementation

The upstream smart-parking/src/ recipe uses the same MediaMTX + Coturn + WebRTC path — consult it for the config.json, mosquitto.conf, nginx.conf, datasources.yml, dashboards.yml, flows.json shapes; drop prometheus/otel-collector/metrics-manager.

Completion criteria (all must pass)

When {{SCENESCAPE}}=yes, criteria 3–11 are superseded by the SceneScape criteria in references/SCENESCAPE.md; 1–2 still apply.

  1. ./install.sh succeeds: .env populated; INT8 model + optional classifier IR under src/dlstreamer-pipeline-server/models/…; videos downloaded; TLS cert with SAN generated.
  2. ./validate_env.sh cpu exits 0 with a valid .env; HOST_IP=127.0.0.1 ./validate_env.sh cpu exits non-zero.
  3. docker compose up -d → all containers running / healthy (including mediamtx-server and coturn).
  4. curl --cacert src/nginx/ssl/server.crt https://localhost/api/pipelines/status returns 3 variants.
  5. ./sample_start.sh <cpu|gpu|npu> launches {{NUM_SOURCES}} pipelines; none QUEUED, all RUNNING.
  6. Detections arrive on {{DETECTIONS_TOPIC_PREFIX}}_1..{{NUM_SOURCES}}/{{PIPELINE_NAME}} (or _gpu/_npu) within 30 s.
  7. curl --cacert src/nginx/ssl/server.crt https://localhost/mediamtx/{{DETECTIONS_TOPIC_PREFIX}}_1/ returns 200 (WHEP player HTML) once pipelines run; MediaMTX logs show the WHIP publisher connected for each peer-id.
  8. Node-RED publishes JSON {{ALERT_TOPIC}} and scalar {{COUNT_TOPIC}} / stats/alert_active / stats/alert_total per {{RULE_SCOPE}}. mosquitto_sub -t '{{COUNT_TOPIC}}/#' -C 1 MUST parse as int() — JSON here silently breaks Grafana plotting.
  9. Grafana at https://localhost/grafana (admin/admin) shows live {{OBJECT}} counts + alert data and {{NUM_SOURCES}} <iframe> WebRTC panels (needs GF_SECURITY_ALLOW_EMBEDDING=true). The MQTT datasource health endpoint (/grafana/api/datasources/uid/mqtt_ds/health) MUST return "MQTT Connected" — broker address goes in jsonData.uri (tcp://broker:1883), NOT url:/jsonData.host/port (→ blank panels). Root redirect-loop → drop the trailing slash from /grafana/ proxy_pass.
  10. pytest -q tests/ passes; pytest --collect-only -q tests/ | tail -1 reports ≥ 9 tests collected (no empty stubs).
  11. Watchdog continuity (file:// sources): after video-length + 30 s, /api/pipelines/status shows {{NUM_SOURCES}} RUNNING (COMPLETED history is fine), WebRTC re-establishes, MQTT still flowing.

    {{NUM_SOURCES}} RUNNING means the watchdog dedup guard is missing.