The Large Model Systems Organization (LMSYS Org) is positioned to develop open, accessible, and scalable large models and systems. Its official website showcases multiple projects and technical updates of the organization in the direction of large models and systems.
In terms of purpose: LMSYS Org mainly targets areas such as inference services, training and fine-tuning, evaluation and comparison, and routing scheduling for large language models and multimodal models, providing open-source tools and platforms for researchers and developers to use.
Main capabilities and projects:
- SGLang: A high-performance serving engine for large language models and multimodal models, used for inference deployment.
- Chatbot Arena: An LLM benchmark evaluation platform that uses crowdsourcing for anonymous, random model battles.
- FastChat: An open and scalable platform for training, fine-tuning, deploying, and evaluating LLM-based chatbots.
- Vicuna: An open-source chatbot that the official website says impresses GPT-4 with 90%* of ChatGPT quality.
- RouteLLM: A framework for LLM routing services and evaluation.
Technical updates: The official website also lists several technical articles, including running SGLang (inference) and Miles (reinforcement learning training) on NVIDIA Vera Rubin, using SGLang to scale JEV-like decision models, and using NVFP4 KV cache to accelerate long-context and agent inference, among others.
Support and partners: LMSYS Org receives support from multiple leading enterprises and institutions. The sponsors and partners listed on the official website include Voltage Park, NVIDIA, Nebius, Google Cloud, AtlasCloud, a16z, AMD, InnoMatrix, Laude Institute, Hyperbolic, NovitaAI, Verda Cloud, Sky9, Kaggle, MBZUAI, Together, RunPod, Anyscale, HuggingFace.
Applicable scenarios: Suitable for researchers and developers who need open-source large model inference engines, model battle evaluation, chatbot training and deployment evaluation, and LLM routing research and evaluation.
Limitations and notes: The official website does not provide specific fees, prices, platform support details, or detailed usage steps; the above projects are all open-source in nature, and specific installationand usage methods should refer to each project's own documentation, and are not currently provided in the official website materials.
