IntelliServer is an open-source microservice designed to provide a unified access point for multiple AI models, allowing developers to easily integrate cutting-edge AI capabilities into their own projects. It adopts a microservice architecture, running AI middleware as an independent service with dedicated resources, thereby improving scalability.
Main capabilities include:
- Chatbot: Implement chat functionality using popular models such as ChatGPT, Llama, AWS Sagemaker, etc.
- LLM evaluation: Evaluate and compare different AI models through a unified service, helping to choose the most suitable model for a specific use case based on data.
- Semantic search: Leverage context-aware semantic search capabilities in text documents.
- Image generation: Generate high-quality images from descriptions using diffusion image models.
- Chat context: Retrieve relevant messages from chatbot conversations.
- Parser: Convert documents such as PDF and Word into text.
- OCR: Extract text from images using AWS or Google Vision.
How to use: IntelliServer can be run by pulling the published image from Docker Hub. For Mac M-series processors, the arm64 version must be pulled. When starting the container, two environment variables need to be set: API_KEY is used to protect the server endpoints, and client requests must include this key; ADMIN_KEY is used for administrative operations, such as changing the API key. These keys can also be defined in the .env file in the repository root directory. To test IntelliServer, you can find the endpoint collection in the postman repository, or visit the Swagger interactive documentation (localhost/api-docs/). To customize the default keys and settings, you can create a .env file in the cloned intelliserver repository and fill in the corresponding values.
Applicable scenarios:
- Development projects that need a unified interface to access and switch between multiple AI models.
- Scenarios where LLMs are compared and evaluated to choose the optimal solution.
- Building end-to-end AI features such as chatbots, semantic search, and image generation.
- Applications that require document parsing or OCR text extraction.
Limitations: IntelliServer is released under the MIT License. More details regarding the specific list of supported models, performance metrics, pricing model, platform compatibility, etc. are not yet available in the official materials.
