OpenJarvis - Stanford University's open-source local AI agent framework

What is OpenJarvis

OpenJarvis is an open-source local AI agent framework launched by Stanford University's Scaling Intelligence Lab. Its core philosophy is to make AI execution fully local, with cloud calls as an optional choice. The framework provides five major modules: a unified model catalog layer, a hardware-aware inference engine, an agent orchestration system, tools and memory, and learning optimization. It supports one-click installation via pip install openjarvis and offers four interaction methods: browser, desktop application, Python SDK, and CLI. Research has found that local models can already handle 88.7% of single-turn conversational queries, and intelligence efficiency has improved 5.3 times within two years. OpenJarvis evaluates energy consumption, latency, cost, and accuracy with equal weight, and includes built-in standardized evaluation tools. It is suitable for privacy-sensitive individual users and provides a viable path for enterprises to reduce their reliance on cloud APIs.

Main features of OpenJarvis

• Local-first architecture: AI execution runs entirely on personal devices, with cloud calls as an optional choice, and data remains local by default to protect privacy.
• Unified model management: The Intelligence module provides a local AI model catalog layer, simplifying model selection, compatibility, and deployment processes.
• Hardware-aware inference: The Engine module automatically detects hardware configuration and supports multiple inference backends such as Ollama, llama.cpp, vLLM, and SGLang.
• Agent orchestration system: The Agents module transforms model capabilities into executable behaviors, supporting the division of roles between Orchestrator and Operative.
• Tool and memory connection: Connects external tools through the MCP protocol and Google A2A, and supports semantic indexing and persistent storage of local content.
• Local learning optimization: The Learning module performs closed-loop optimization based on local interaction data, supporting fine-tuning methods such as SFT, GRPO, and DPO as well as automatic prompt optimization.
• Standardized efficiency evaluation: Built-in jarvis bench tools evaluate energy consumption, latency, FLOPs, dollar cost, and accuracy with equal weight.
• Multi-platform interaction: Offers four ways to use it: browser application, desktop application (macOS/Windows/Linux), Python SDK, and CLI.
• OpenAI-compatible API：jarvis serve launches the service and provides a local API endpoint compatible with the OpenAI format.

Technical principles of OpenJarvis

• Local-first execution model: It adopts a "cloud as optional" architecture, running inference on personal devices (laptops/workstations) by default. 88.7% of single-turn conversation and reasoning queries require no cloud involvement, achieving efficient local computation through hardware accelerators (Apple Silicon, NVIDIA GPU, etc.).
• Modular layered architecture: Five major modules with a loosely coupled design—Intelligence (model abstraction layer), Engine (inference runtime), Agents (behavior orchestration), Tools & Memory (external connection), and Learning (closed-loop optimization). Each layer can be independently upgraded or replaced.
• Hardware-aware scheduling mechanism: The Engine module automatically detects available hardware resources (CPU/GPU/NPU), dynamically selects the optimal inference backend (Ollama/llama.cpp/vLLM/SGLang), and completes environment diagnostics and configuration recommendations through jarvis init and jarvis doctor .
• Agent role separation: Agents adopt an Orchestrator-Operative division-of-labor model, where the Orchestrator is responsible for task planning and tool selection, and the Operative focuses on executing specific actions, supporting both single-agent and multi-agent collaboration modes.
• MCP/A2A Protocol Integration: The Tools & Memory layer connects external data sources and tools through the Model Context Protocol and the Google A2A standard protocol, enabling semantic indexing of local content (vector database) and persistent memory storage.
• Local Continuous Learning Framework: The Learning module performs local model fine-tuning based on user interaction data, supporting three algorithms: SFT (Supervised Fine-Tuning), GRPO (Group Relative Policy Optimization), and DPO (Direct Preference Optimization), as well as automatic prompt optimization (APE).
• Multi-Dimensional Efficiency Evaluation System: Breaking through the traditional single metric of accuracy, it establishes a four-dimensional evaluation matrix of energy consumption (Watts), latency (ms), computation (FLOPs), and cost ($), with built-in jarvis bench standardized evaluation tools to achieve reproducible comparisons.

OpenJarvis project address

• Github repository：https://github.com/open-jarvis/OpenJarvis

OpenJarvis application scenarios

• Privacy-sensitive personal assistant: Suitable for professionals handling confidential information such as lawyers, doctors, and psychological counselors. All data is stored locally and not uploaded to the cloud, avoiding the risk of sensitive content leakage.
• Intelligent office work in offline environments: For business professionals who frequently travel or are in environments with unstable networks, they can continue to use AI to complete document processing, email drafting, and schedule planning in no-network scenarios such as on airplanes and high-speed trains.
• Local knowledge base question answering: Enterprise employees import internal documents, code repositories, and product manuals into a local vector database to build a privatized intelligent search and question-answering system, replacing traditional knowledge management tools.
• Edge device AI development: IoT developers use OpenJarvis to deploy lightweight agents on edge devices such as Raspberry Pi and Jetson, enabling real-time inference scenarios such as smart home control and industrial quality inspection.
• Low-cost AI prototype validation: Startup teams and individual developers replace cloud API calls with local models, significantly reducing token consumption costs during the development phase and supporting a smooth migration from prototype to production.
• Education and scientific research experiment platform: University researchers use the framework's built-in jarvis bench evaluation tools and various fine-tuning methods to conduct academic experiments such as model efficiency optimization and agent behavior research.