KwaiAgents is a collection of a series of Agent-related works open-sourced by the KwaiKEG team at Kuaishou Technology, positioned as a general-purpose information retrieval agent system based on large language models (LLMs).
Purpose
This project is aimed at agent research and development, providing complete open-source content ranging from models, data, and evaluation to a runnable system, used to build and evaluate LLM agents with information retrieval capabilities.
Main capabilities and open-source content
KwaiAgents includes the following four parts:
-
KAgentSys-Lite: A lightweight version of KAgentSys in the paper. While retaining some functions of the original system, it has certain differences and limitations compared with the full version, for example: a more limited toolset; lack of a memory mechanism; slightly reduced performance; and a different codebase, as it evolved from open-source projects such as BabyAGI and Auto-GPT. Nevertheless, KAgentSys-Lite can still provide comparable performance among many open-source Agent systems.
-
KAgentLMs: A series of large language models with agent capabilities such as planning, reflection, and tool use, obtained through the Meta-agent tuning proposed in the paper.
-
KAgentInstruct: Over 200,000 Agent-related instruction fine-tuning data proposed in the paper (partially manually edited).
-
KAgentBench: Over 3,000 manually edited automated evaluation data entries, used to test agent capabilities, with evaluation dimensions including planning, tool use, reflection, summarization, and profiling.
Usage
Environment preparation
You need to first install miniconda to set up the environment, then create the build environment. It is recommended to use vLLM and FastChat to deploy the model inference service, and you need to first install the corresponding packages (refer to the documentation of these two projects for specific usage). There are corresponding installation commands for Qwen-7B-MAT and Baichuan-13B-MAT respectively.
Deploying KAgentLMs
When deploying, you need to first start the controller in one terminal; then in another terminal deploy the single-GPU inference service, where $model_path is the local path of the downloaded model. If the GPU does not support Bfloat16, you can add --dtype half to the command line; then start the REST API server in a third terminal; finally you canUse the curl command to call the model in the OpenAI invocation format, replacing kagentlms_qwen_7b_mat with the deployed model.
llama-cpp-python provides a web server that can serve as a replacement for the OpenAI API, enabling llama.cpp-compatible models to be used with any OpenAI-compatible client. The converted model can be found at kwaikeg/kagentlms_qwen_7b_mat_gguf.
Using KAgentSys-Lite
Download and install KwaiAgents; Python>=3.10 is recommended.
ChatGPT usage: Declare some environment variables. WEATHER_API_KEY is not mandatory, but it needs to be configured when asking weather-related questions; the API key can be obtained from the specified website (the same applies to local model usage).
Local model usage: The corresponding model service needs to be deployed as described above.
For the complete command parameters, see the project documentation. If you need to use the browse_website tool, chromedriver must be configured on the server. If searches fail multiple times, it may be because the network cannot access duckduckgo_search, which can be resolved by setting http_proxy. For custom tool usage, refer to examples/custom_tool_example.py.
Evaluating with KAgentBench
Only two lines are needed to evaluate agent capabilities. The command will output the results. For more details, refer to the benchmark/ directory.
Applicable Scenarios
Suitable for agent research, information retrieval tasks, LLM agent capability evaluation, and building agent applications with planning, reflection, and tool-use capabilities based on open-source models.
Limitations
Compared with the full version, KAgentSys-Lite has limitations such as a limited toolset, no memory mechanism, slightly lower performance, and a different codebase. Using the browse_website tool requires configuring chromedriver; the search function depends on network access to duckduckgo_search. Weather-related questions require configuring WEATHER_API_KEY.
Update Record
2024.4.19 Released the Qwen1.5-14B-MAT model; 2024.4.9 Refreshed benchmark results; 2024.1.29 Released the Qwen-14B-MAT model; 2023.1.5 Released training data; 2023.12.27 KwaiAgents was reported by multiple websites; 2023.12.13 Released the benchmark and evaluation code; 2023.12.08 Released the technical report; 2023.11.17 Initial release.
