KoboldCpp is free, open-source software for running GGUF large language models (LLMs) on the user's own computer. It is inspired by KoboldAI and built on llama.cpp, providing a single executable with no installation required, with official ready-to-run downloads available for Windows, Linux, and macOS.
Purpose
KoboldCpp's core purpose is local text generation. Users can run any GGUF language model to generate text in chat, adventure, instruct, or story writing modes. Compatible vision models also support image understanding. In addition, it supports image generation and editing, video generation, speech recognition, speech generation, music generation, and has built-in agent and various tool capabilities.
Main capabilities
- Local text generation: Run GGUF language models, supporting chat, adventure, instruct, story writing, and other modes; compatible vision models can perform image understanding.
- Image generation and editing: Supports Stable Diffusion 1.5, SDXL, SD3, Flux, Qwen Image, Ideogram, Z-Image, Klein, Krea2, and more.
- Video generation: Supports WAN 2.2, LTX2.3, MiniMax H3, and more.
- Speech recognition: Uses Whisper for speech-to-text, and supports multimodal audio for Gemma4 E2B and E4B.
- Speech generation: Uses Qwen3TTS, Kokoro, OuteTTS, Parler, Dia for text-to-speech.
- Music generation: Supports ACE Step 1.5 and ACE Step XL.
- Tools and agents: Supports MCP servers, tool calling, web search, retrieval-augmented generation (RAG) via TextDB, and the built-in KoboldCpp Agent, which can be used to write and edit code, run programs, and schedule tasks. To launch the agent, select Launch KoboldCpp Agent in the Admin tab of the launcher, or add --agent to the launch command.
- Writing and roleplay tools: Built-in KoboldAI Lite WebUI, including multiple interface themes, editing tools, memory, world info, author's notes, characters, scenarios, and persistent story saving. Tavern character cards and other supported formats can be imported from files or external sites. Also includes the classic llama.cpp WebUI.
- Application integration: compatible with endpoints such as KoboldAI, OpenAI, Ollama, A1111/Forge, ComfyUI, Whisper transcription, XTTS, and OpenAI voice clients.
- Fully portable: a single executable for Windows, Linux, or macOS, no installation required, no external dependencies. Can run on CPU or GPU, with full or partial offloading. Can also run on Colab, Docker, and, after self-compilation, other platforms as well (such as running on Android via Termux, and Raspberry Pi).
How to Use
- Download the version for your operating system from the latest KoboldCpp release page. Windows users can download koboldcpp.exe and double-click to open the launcher; if NVIDIA CUDA support is not needed, the smaller koboldcpp-nocuda.exe can be used; older CPUs or GPUs can try koboldcpp-oldpc.exe. Linux users download koboldcpp-linux-x64, use chmod +x to grant executable permission, then run it from the terminal; macOS (Apple Silicon) users download koboldcpp-mac-arm64, likewise grant executable permission, then run it.
- Download a GGUF text model. Models are provided separately from the software; KoboldCpp does not include model files. GGUF models can be obtained at huggingface.co, or you can select Newbie Templates in the launcher's Get Help for easy setup.
- Open KoboldCpp, select the model in the GGUF Text Model field, and choose the hardware settings suitable for your computer. The default settings usually work.
- Click Launch and wait for the model to finish loading, keeping KoboldCpp running.
- In your browser, visit http://localhost:5001 to connect to the Web UI.
Command-line usage example: koboldcpp.exe --model "C:\Models\model.gguf"; for more information, see koboldcpp.exe --help.
If you do not have a local model, you can also connect to external AI providers, such as AI Horde, OpenAI Compatible, Anthropic, OpenRouter, Gemini, Grok, Mistral, and others. Start with --nomodel, or select Allow Launch Without Models in the launcher's Loaded Files tab. You can also directly use the online KoboldAI Lite WebUI.
Other ways to run include: Google Colab (the official KoboldCpp Colab GPU Notebook), RunPod (the KoboldCpp RunPod image or SimplePod), and the HuggingFace Space public demo. The official Docker image is mainly aimed at experts and cloud GPU rental; internally it uses an x86-64 Ubuntu environment and requires an NVIDIA or AMD GPU.
Use Cases
- Local chat, story creation, role-playing, and writing assistance.
- Multimodal local AI workflows requiring image, video, speech, and music generation.
- Connecting local models to other applications via compatible APIs.
- Use on devices without installation permissions or where portable operation is needed.
- Using the built-in agent for coding, running programs, and task scheduling.
Limitations and Notes
- KoboldCpp does not include model files; they must be obtained separately.
- Image, speech, embedding, and music generation require loading the corresponding models.
- The official Docker image may perform poorly in some Windows or macOS environments, and ARM systems may not be able to run it.
- If system memory is insufficient, try a smaller model or a shorter context; if GPU VRAM is insufficient, reduce the number of GPU layers, use a smaller model, or use a shorter context.
- If generation is slow, check whether the correct GPU backend is selected and whether layers are offloaded.
- If an old computer crashes on startup, try the oldpc version or --noavx2.
- If the browser cannot connect, wait for the model to finish loading and check the address printed in the terminal; the default is localhost:5001.
- If the model fails to load, check the terminal errors, available memory, and whether the KoboldCpp version supports the model.
- If the model output is incoherent, an incorrect chat template may be in use; try using --jinja or enabling the Jinja toggle in the GUI.
- Official notice: koboldcpp.com is a malicious counterfeit website and is unrelated to this project; only the official GitHub release page should be trusted https://github.com/LostRuins/koboldcpp/releases/latest for downloads.
