ai-vision-mcp is a Model Context Protocol (MCP) server that provides AI-powered image and video analysis capabilities for MCP-compatible clients. It is based on Google Gemini and Vertex AI models, supports multimodal content analysis, and is suitable for integration into tools such as Claude Desktop, Claude Code, Cursor, Cline, and others.
Purpose
The server is designed to extend visual understanding capabilities for AI clients and can be used in scenarios such as UI/UX evaluation, visual regression testing, interface understanding, and design compliance auditing.
Main capabilities
- Dual provider support: choose between Google Gemini API or Vertex AI.
- Multimodal analysis: supports both image and video content analysis.
- Flexible file handling: can upload via URL, local file, or base64.
- Storage integration: built-in Google Cloud Storage support.
- Comprehensive validation: uses Zod for data validation.
- Error handling: includes retry logic and circuit breaker mechanisms.
- TypeScript: full TypeScript support with strict type checking.
The server provides four main MCP tools:
- analyze_image: analyzes an image and returns a detailed description. Supports multiple modes: general (general image analysis), palette (extract design tokens: colors, spacing, fonts), hierarchy (analyze visual hierarchy and visual flow), components (catalog UI components and design system maturity). Accepts imageSource, prompt, mode, and options parameters.
- compare_images: compares multiple images and returns a detailed comparative analysis. imageSources is an array, with a minimum of 2 and a maximum of 4, supporting URL, base64, or file paths.
- detect_objects_in_image: detects objects in an image and generates an annotated image with bounding boxes, returns the coordinates of detected objects, and can save the annotated image to a specified path or temporary directory. Supports custom detection prompts and allows AI parameters to be adjusted via environment variables.
- audit_design: audits UI/UX design compliance, performing pixel-level analysis and AI commentary. Extracts primary colors (K-means clustering takes 5 main colors), edge complexity (Sobel operator), WCAG contrast (W3C relative luminance formula, AA/AAA), brightness statistics (mean brightness and standard deviation), automatically detects contrast, complexity, and brightness issues, and provides Gemini-based improvement suggestions.
- analyze_video: analyzes a video and returns a detailed description. Supports YouTube URLs, GCS URIs, or local file paths. Note: public video URLs only support YouTube URLs.
Usage
Environment variable configuration: you can choose the Google AI Studio provider (recommended, simple) or the Vertex AI provider (production environment). The corresponding API key or credentials need to be set. It is recommended to set the MCP client timeout to more than 5 minutes.
Installation to different MCP clients: configuration guides are provided for Claude Desktop, Cursor, Cline, etc. The server uses stdio transport, follows the standard MCP protocol, can be run via commands, and can be integrated into any compatible MCP client.
Configuration priority: runtime parameters > feature-specific variables > task-specific variables > general variables > system defaults.
Development commands: npm run build builds the TypeScript project; npm run dev starts the development server (watch mode); npm run lint runs ESLint; npm run format formats with Prettier; npm start starts the built server.
Applicable scenarios
- UI/UX evaluation and design compliance auditing
- Visual regression testing
- Interface understanding and component cataloging
- General analysis, comparison, and object detection of image content
- Video content analysis
Limitations
- Public video URLs currently only support YouTube.
- The object detection tool uses optimized default parameters and does not accept runtime options parameters; customization must be done through environment variables.
- When using stdio transport, stdout must contain only MCP JSON-RPC messages, and logs must be output to stderr; otherwise, the client may disconnect.
- You need to configure Google AI Studio or Vertex AI credentials yourself.
- The project uses the MIT license.
