LLMHub is a lightweight management platform designed to simplify the operation and interaction of various large language models (LLMs). It provides an intuitive command-line interface (CLI) and RESTful API for managing, starting, stopping, and interacting with LLMs. The platform supports running multiple models with different configurations and context sizes.
Purpose: LLMHub CLI is a command-line interface tool for managing and interacting with various LLM (large language model) servers. It allows you to start, stop, update, and manage LLM processes easily and efficiently.
Key capabilities:
- Manage LLM servers
- Start, stop, and update LLM processes
- List available models and their status
- Provide OpenAI-compatible API endpoints for completions and model listing
- Easy configuration via YAML files
- Support different engines and quantization formats
Usage: LLMHub CLI can be installed directly from PyPI. After installation, you can use the llmhub command to interact with the tool. Example commands include: start processes, update all processes, list all models. Configuration is handled through YAML files. You can place the config.yaml file in the ~/.llmhub/ directory, or specify a custom path when initializing ConfigManager. LLMHub CLI also provides OpenAI-compatible API endpoints:
- /v1/completions: handles completion requests.
- /v1/chat/completions: handles chat completion requests.
- /v1/models: lists available models.
Use cases: Suitable for developers and operations personnel who need to manage multiple large language models locally or self-hosted, and interact with them via the command line or an OpenAI-compatible API.
Limitations and notes: This repository is deprecated, and the current version is built with Node. Contributions are welcome; you can submit a Pull Request. The project uses the MIT license. If you have questions, please submit an issue in the GitHub repository.
