guaardvark

Guaardvark is a self-hosted AI studio whose full capabilities run on the user's own hardware, implemented through Ollama, ComfyUI, and Guaardvark's own GPU service. It does not require cloud API keys, has no per-token or per-minute fees, and does not send data externally; all outbound paths such as network access and posting are disabled by default until the user actively enables them, and these paths are uniformly declared in the egress.json file.

Main capabilities

1. Local RAG file chat Local RAG based on pgvector: each query fuses keyword and vector search, uses a cross-encoder for reranking, provides source citations, and displays the text chunks and scores behind the answer. PDF, DOCX, and PPTX are parsed in reading order and by section headings, and PDF paragraphs carry page numbers; code uses AST-aware chunking to keep functions intact. Retrieval can display the text chunks and scores actually used, and can return the query trace. Autoresearch can automatically adjust retrieval settings overnight.

2. Desktop and browser agents The agent controls a real Ubuntu desktop (Xvfb and XFCE, display :99), sees the screen through a vision model, moves the mouse, clicks, types, browses, and inspects its own work. The desktop is isolated from the user session, with separate XDG_DESKTOP_DIR and XDG_CONFIG_HOME. The vision model returns click coordinates in a single inference, uses closed-loop aiming, and after moving checks where the cursor actually lands and corrects it. Every step of reasoning, action, and goal flows into the chat in real time and retains history. Common browser operations are executed immediately by a deterministic recipe library. The screen agent is supported only on Linux; on macOS it will clearly state that the tool is unavailable, without affecting other features.

3. Parallel coding agents (Swarm) Multiple coding agents are started at the same time, each working on a separate git worktree and branch. Results are merged in dependency order, with conflict detection, optional test validation, and a real-time dashboard. The backend can use Claude Code, or run Cline/OpenClaw locally through Ollama. Flight Mode stays offline and uses only the local backend. By default, 5 agents run simultaneously, and the dashboard supports up to 20.

4. MCP server, agent skills, and Claude Code plugin Bidirectional support for Model Context Protocol: It can both expose tools to any MCP client and call tools on other MCP servers. python -m backend.mcp install can automatically discover and configure clients such as Claude Code, Codex, Cursor, Grok, Antigravity, opencode, Claude Desktop, Zed, and Gemini CLI; python -m backend.mcp doctor can diagnose configuration issues. The Claude Code plugin can be installed via /plugin marketplace add guaardvark/guaardvark and /plugin install guaardvark@guaardvark. By default, more than 50 tools are exposed (RAG, files, generation, memory, code search, Web); desktop control, shell, browser automation, and others are hidden by default and need to be added to the allowlist.

5. Local video, image, music, and speech generation Video: 13 local video models, covering five families: Wan 2.2, LTX, HunyuanVideo, CogVideoX, and MiniMax H3. Supports 512 to 1920px resolutions, Fast/Standard/High/Maximum quality tiers, RIFE frame interpolation, prompt enhancement styles, low-VRAM mode, and batch queues. MiniMax H3 can generate visuals and sound at once. Video generation usually requires a 16GB-class GPU, while Wan 2.2 5B can start from 11GB. Image: Z-Image, FLUX.1, Krea 2, SDXL, and Sana Sprint generation, instruction-based photo editing, inpainting and outpainting, and 4K/8K upscaling. Music and speech: full songs with vocals, sound effects, three neural TTS engines, consented voice cloning, and hands-free voice chat.

6. LoRA training and consistent characters You can train character, environment, and prop LoRAs on your own GPU using a small number of reference photos, or import LoRAs trained elsewhere, and keep the same character across images, videos, music videos, and Film Crew productions.

7. Directing features Beat-synced music video directing, a five-role Film Crew (turning a one-sentence pitch into a complete video), and a built-in video editor.

8. Self-improvement and commandcenter Guaardvark can test itself, draft fixes, and wait for user review after a codebase lock; the system map draws the entire codebase based on real import relationships. GPU orchestrator, plugin manager with real-time VRAM budgeting, tasks and scheduling, schema-aware backups, and multi-machine Interconnector.

9. Terminal and API Provides the guaardvark CLI interactive REPL, as well as REST, GraphQL, and Socket.IO interfaces.

Installation and usage Requires Linux and an NVIDIA GPU (16GB for video, 12GB for Wan 2.2 5B; lower requirements for chat, RAG, and agents). A fresh Ubuntu desktop requires running sudo apt install -y curl git first. The installation script is install.sh in the repository, which clones to ~/guaardvark and runs ./start.sh, installing all components on first run. After that, open http://localhost:5173。也可先下载脚本阅读后再运行，或手动安装。其他方式包括 Docker, macOS on Apple Silicon, the CLI on PyPI, and the full installation guide.

Use cases and limitations Suitable for users who want to run RAG, agents, coding collaboration, and media generation in a unified way on a local GPU, especially scenarios that value keeping data local and avoiding cloud costs. Limitations include: requires Linux and an NVIDIA GPU; the screen agent is only available on Linux; some models have VRAM requirements and license requirements (for example, the MiniMax H3 community license involves region-specific applications); OCR for scanned documents is not installed by default. For detailed hardware tiers and the complete capability list, see the documentation CAPABILITIES.md and docs/HARDWARE.md.