FreeLLMAPI - Open-source AI model aggregation platform with unified OpenAI-compatible format

What is FreeLLMAPI

FreeLLMAPI is an open-source OpenAI-compatible proxy service that aggregates the free quotas of about 14 AI providers into a single API endpoint, totaling approximately 1.3 billion tokens of free inference quota per month. The project is built on TypeScript/Express and supports automatic failover, intelligent routing, key encryption, and usage tracking. It is designed for local experimentation and prototype validation by individual developers.

Main Features of FreeLLMAPI

• OpenAI-compatible API proxy: Exposes standard /v1/chat/completions and /v1/models endpoints externally, compatible with the official OpenAI SDK, LangChain, LlamaIndex, and other clients. Integration only requires modifying base_url .
• Multi-provider free quota aggregation: Simultaneously connects to about 14 platforms including Google Gemini, Groq, Cerebras, SambaNova, NVIDIA NIM, Mistral, OpenRouter, GitHub Models, HuggingFace, Cohere, Cloudflare, Zhipu Z.ai, Moonshot Kimi, and MiniMax, totaling approximately 1.3 billion tokens of free inference quota per month.
• Automatic failover: When the preferred provider returns 429/5xx or the request times out, it is automatically placed in a cooldown state, and the next provider is tried according to a preset priority chain, with up to 20 retries.
• Intelligent rate and quota tracking: Tracks RPM, RPD, TPM, and TPD usage in real time for each (平台, 模型, 密钥) combination, ensuring that only healthy keys within limits are always used.
• Sticky session locking: Multi-turn conversations are locked to the same model for 30 minutes, preventing mid-conversation model switches that could cause output quality jumps or context breaks.
• AES-256-GCM key encryption: Upstream provider API keys are encrypted at rest in the SQLite database and are only temporarily decrypted in memory before making a request, reducing the risk of key leakage.
• Unified authentication token management: Clients use a single freellmapi-… Bearer Token to access the proxy, with no need to expose multiple upstream real keys in the code.
• SSE streaming output: Fully supports stream: true Server-Sent Events streaming responses, compatible with word-by-word generation scenarios for long text.
• Tool calling (Function Calling): Supports OpenAI-style tools / tool_choice multi-turn tool calling workflows, meeting Agent development needs.
• Visual dashboard and Playground: Includes a built-in React/Vite admin backend that supports drag-and-drop adjustment of fallback chain priority, quick model testing in the Playground, and viewing statistics such as latency, success rate, and token usage over 24h/7d/30d.
• Lightweight local deployment: Built on TypeScript/Express, using only about 40MB RSS when idle, and can run persistently on ARM devices such as Raspberry Pi via PM2/systemd.

How to use FreeLLMAPI

• Clone and install：git clone https://github.com/tashfeenahmed/freellmapi.git && cd freellmapi && npm install。
• Configure keys：cp .env.example .env and generate a 32-byte ENCRYPTION_KEY for AES-256-GCM encryption.
• Start the service：npm run dev to start both the backend API (:3001) and the dashboard (:5173）。
• Add upstream Keys: On the http://localhost:5173 page of Keys , paste the free API Key for each platform.
• Configure the fallback chain: On the Fallback Chain page, drag to set provider priority.
• Obtain a unified Token: Generate a freellmapi-… single Bearer Token in the panel.
• SDK invocation: Modify base_url="http://localhost:3001/v1"，model and set it to "auto" or a specific model name.
• cURL invocation: Send a standard OpenAI-format request to http://localhost:3001/v1/chat/completions .
• Playground verification: Enter prompts directly in the dashboard Playground to test connectivity and routing.
• Background daemon: Hosted via PM2 or systemd, it can run long-term on ARM devices such as Raspberry Pi.

Core advantages of FreeLLMAPI

• OpenAI standard compatibility: Exposes /v1/chat/completions and /v1/models endpoints externally, supporting any compatible client such as the official OpenAI SDK, LangChain, LlamaIndex, etc. Access only requires modifying base_url .
• Aggregated free quotas from 14 platforms: Simultaneously integrates Google Gemini, Groq, Cerebras, SambaNova, NVIDIA NIM, Mistral, OpenRouter, GitHub Models, HuggingFace, Cohere, Cloudflare, Zhipu Z.ai, Moonshot Kimi, MiniMax, etc., totaling approximately 1.3 billion tokens of free inference quota per month.
• Automatic failover: When the preferred provider returns 429/5xx or times out, it is automatically put into cooldown and the next one is tried according to the priority chain, with up to 20 retries, ensuring service continuity.
• Intelligent rate and quota tracking: Tracks RPM, RPD, TPM, and TPD usage in real time for each (平台, 模型, 密钥) combination, always selecting a healthy key that has not exceeded limits and avoiding manual switching.
• Sticky session mechanism: Multi-turn conversations lock onto the same model for 30 minutes, preventing output quality jumps or context breaks caused by switching models mid-conversation.
• AES-256-GCM key encryption: Upstream provider API keys are encrypted at rest in SQLite and are only temporarily decrypted in memory before initiating a request, then destroyed after use, reducing the risk of key leakage.
• Unified authentication token: Clients only need to use a single freellmapi-… Bearer Token, without exposing multiple upstream real keys in code, simplifying management.
• Full streaming and tool calling: Supports SSE streaming output (stream: true) and OpenAI-style tools / tool_choice multi-turn tool calling workflows.

FreeLLMAPI project address

• GitHub repository：https://github.com/tashfeenahmed/freellmapi

Comparison of FreeLLMAPI with similar competitors

FreeLLMAPI use cases

• Local experimentation and prototype validation for individual developers: The project is explicitly aimed at personal learning and prototype development, stacking free quotas from multiple providers into a single local OpenAI endpoint, allowing rapid validation of AI application ideas without paying.
• Aggregated invocation of free quotas across multiple platforms: Simultaneously connects to the free tiers of about 14 platforms such as Gemini, Groq, Cerebras, GitHub Models, and Zhipu, totaling about 1.3 billion tokens per month, suitable for personal projects that need large usage but have a limited budget.
• Always-on proxy for low-power devices: Idle RSS is only about 40MB, and it can run long-term via PM2/systemd on ARM devices such as Raspberry Pi, serving as a lightweight AI gateway for home intranets or edge nodes.
• Learning OpenAI-compatible APIs: Exposes standard /v1/chat/completions endpoints externally, suitable for learning or debugging ecosystem tools such as the OpenAI SDK, LangChain, and LlamaIndex; switching only requires changing base_url .
• Quick multi-model comparison testing: Through a unified interface and the Playground dashboard, you can quickly compare the response quality, latency, and output style of different free providers in the same environment.