llm-advisor-mcp

llm-advisor-mcp is an open-source Model Context Protocol (MCP) server designed to provide AI assistants with real-time knowledge of LLMs and VLMs (vision-language models), including pricing, benchmarks, and recommendations. The tool updates its data hourly, ensuring that AI assistants can access the latest model information without being limited by training data knowledge cutoffs.

Purpose

Large language models (LLMs) usually have a knowledge cutoff and cannot answer questions about the performance of the latest models. For example, when a user asks Claude "What is the current best coding model?", Claude may be unable to give an answer based on the latest data. llm-advisor-mcp solves this problem by injecting real-time model intelligence directly into the AI assistant's context window. It is suitable for scenarios that require the latest model comparisons, prices, and benchmark information.

Main Capabilities

• Zero configuration: No API key required, no registration required, installable with one command.
• Low token consumption: Outputs compact Markdown tables (about 300 tokens) instead of raw JSON (about 3,000 tokens), helping to save context window space.
• Five benchmark sources: Integrates SWE-bench, LM Arena Elo, OpenCompass VLM, Aider Polyglot, and OpenRouter pricing to provide a unified view.
• Four core tools:
  • list_top_models: Lists top models, filterable by category (such as coding).
  • compare_models: Compares multiple models and generates a side-by-side table.
  • recommend_model: Recommends models based on constraints such as budget.
  • get_model_info: Gets detailed information about a specific model, including pricing, benchmarks, percentile rankings, capabilities, and API code examples.

How to Use

Installation and configuration are simple. For Claude Code (Windows) or clients such as Claude Desktop / Cursor / Windsurf, simply add the server to the MCP configuration file. No API key or .env file is required. The server communicates with MCP clients via stdio (JSON-RPC).

Example Queries

• "What is the current best coding model?" — Use list_top_modelswith the category set to coding.
• "Compare Claude, GPT, and Gemini" — Use compare_modelsto generate a side-by-side table.
• "Find a cheap model with 1M context" — Use recommend_modelwith a budget constraint.
• "What benchmarks does model X have?" — Use get_model_infoto get percentile rankings.

Model Information Example

get_model_info provides detailed information including: pricing (input/output price per million tokens), context length, release date, benchmark scores (such as SWE-bench, Aider, Arena Elo), strengths (such as reasoning, tools, vision), and a ready-to-use API code example.

Model Comparison Example

compare_models Can compare three models simultaneously, outputting recommendation scores and key metrics. For example:

• anthropic/claude-sonnet-4: input $3.00/1M, output $15.00/1M, context 200K, release date 2025-06-25, SWE-bench 76.8%, Aider 72.1%, Arena 1467, strengths: reasoning, tools, vision, score 78.
• google/gemini-2.5-flash: input $0.15/1M, output $0.60/1M, context 1M, release date 2025-05-20, SWE-bench 62.9%, Arena 1445, strengths: tools, vision, 1M+ context, score 74.
• openai/o4-mini: input $1.10/1M, output $4.40/1M, context 200K, release date 2025-04-16, SWE-bench 73.6%, Arena 1430, strengths: reasoning, tools, score 71.

Architecture

The server communicates with MCP clients via stdio (JSON-RPC). Internally it includes a tool layer (4 tools), a unified registry, an in-memory cache, a normalizer (slug mapping), percentile calculation (5 categories), and a data fetcher (fetching data from 5 sources such as OpenRouter, SWE-bench, Arena/VLM/Aider).

Use Cases

• AI assistants need real-time model recommendations and comparisons.
• Developers need to choose models based on budget, context length, or specific capabilities (such as coding, vision).
• Research or evaluate the performance and pricing of different LLM/VLMs.
• Integrate into MCP-supporting clients such as Claude Code, Claude Desktop, Cursor, Windsurf, etc.

Limitations

• This tool relies on external data sources; data is updated hourly, but there may still be delays.
• The information currently provided is based on public data; actual model capabilities may vary depending on usage.
• The client must support the MCP protocol.
• Open-source project, licensed under the MIT License, built by Cognisant LLC.

Contributing

Contributions are welcome: fork the repository, create a feature branch, add tests for new features, run npm test to ensure all 51 tests pass, then submit a pull request.