Atla MCP Server is an MCP server implementation designed to provide LLMs with a standardized interface to interact with the Atla API for state-of-the-art LLM evaluation. This repository was archived on July 21, 2025, and the Atla API is no longer active.
Purpose: This server is primarily used for LLM evaluation scenarios, allowing AI agents to call Atla's evaluation models through a standardized protocol to score and critique LLM-generated responses.
Main capabilities: It provides two available tools:
- evaluate_llm_response: Evaluates an LLM's response to a prompt according to given evaluation criteria. This function uses the Atla evaluation model under the hood and returns a dictionary containing the score of the model's response and a textual critique containing feedback on the model's response.
- evaluate_llm_response_on_multiple_criteria: Evaluates an LLM's response to a prompt across multiple evaluation criteria. This function uses the Atla evaluation model under the hood and returns a list of dictionaries, each containing the evaluation score and critique for a given criterion.
Usage: To use this MCP server, you need an Atla API key. You can look for an existing API key in the specified location or create a new one. Using uv to manage the Python environment is recommended. After installing uv and obtaining an Atla API key, you can manually run the MCP server using uvx (provided by uv).
Connection methods: Multiple client connections are supported. For the OpenAI Agents SDK, you need to install the SDK and use it to connect to the server; refer to the official documentation for more details. For Claude Desktop, you need to add configuration in the claude_desktop_config.json file, and after restarting, you will see the atla-mcp-server option in the list of available MCP tools. For Cursor, you need to add configuration in the .cursor/mcp.json file, after which you will see atla-mcp-server in the list of available MCP servers.
Applicable scenarios: Suitable for developers and researchers who need to perform automated evaluation of LLM responses, especially when using MCP-supporting clients (such as Claude Desktop, Cursor) or the OpenAI Agents SDK .
Limitations: This repository has been archived, and the Atla API is no longer active, so it may no longer be usable. Usage requires an Atla API key and depends on the uv environment. The project uses the MIT license.
