SuperAPI AI Directory
Partner with us中文AI newsFavorites
DirectoryNews and tutorials

GLM-4.5V - Zhipu's latest generation visual reasoning model

Author: AI小集 · Original source: AI工具集

What is GLM-4.5V

GLM-4.5V is the latest generation visual reasoning model released by Zhipu. The model is built on a 106B parameter scale and has 12B activation capability, making it a currently leading vision-language model (VLM). The model is upgraded fromGLM-4.1V-Thinkingand inherits its excellent architecture, combining it with the new-generation text foundation modelGLM-4.5-Airfor training. The model performs exceptionally well in visual understanding and reasoning capabilities, and is suitable for scenarios such as web front-end replication, grounding, geo-guessing games, and video understanding, and is expected to promote the further development of multimodal applications. To help developers intuitively experience the powerful capabilities of GLM-4.5V and build exclusive multimodal applications, the team has open-sourced a desktop assistant application that can capture screenshots and record screens in real time, and with the help of the GLM-4.5V model handles various visual tasks such as code assistance, video analysis, game solving, and document interpretation.

GLM-4.5V

Main features of GLM-4.5V

  • Visual understanding and reasoning: It can understand and analyze visual content such as images and videos, and perform complex visual reasoning tasks, such as identifying objects, scenes, and relationships between people.
  • Multimodal interaction: It supports the integrated processing of text and visual content, such as generating images based on text descriptions, or generating text descriptions based on images.
  • Web front-end replication: It generates front-end code based on web page design mockups, enabling rapid web development.
  • Geo-guessing games: It supports image-based search and matching tasks, such as finding a specific target in a complex scene.
  • Video understanding: It supports analyzing video content, extracting key information, and performing tasks such as video summarization and event detection.
  • Cross-modal generation: It supports generating text from visual content, or generating visual content from text, achieving seamless conversion of multimodal content.

Technical principles of GLM-4.5V

  • Large-scale pretraining: The model is based on a 106B parameter pretraining architecture and is trained with massive amounts of text and visual data to learn joint representations of language and vision.
  • Vision-language fusion: It adopts the Transformer architecture to fuse text and visual features, and realizes interaction between text and visual information based on cross-attention mechanisms.
  • Activation mechanism: The model is designed with 12B activation parameters, used to dynamically activate relevant parameter subsets during inference, improving computational efficiency and inference performance.
  • Architecture inheritance and optimization: It inherits the excellent structure of GLM-4.1V-Thinking, combines it with the new-generation text foundation model GLM-4.5-Air for training, and further improves performance.
  • Multimodal task adaptation: Based on fine-tuning and optimization, the model can adapt to a variety of multimodal tasks, such as visual question answering, image caption generation, and video understanding.

Performance of GLM-4.5V

  • General VQA: GLM-4.5V performs best in general visual question answering tasks, especially scoring as high as 88.2 on the MMBench v1.1 benchmark.
  • STEM: GLM-4.5V is also leading in science, technology, engineering, and mathematics-related tasks, for example scoring 84.6 on the MathVista test.
  • Long Document OCR & Chart: In the OCRBench test for processing long documents and charts, GLM-4.5V demonstrates excellent performance with a score of 86.5.
  • Visual Grounding: GLM-4.5V performs outstandingly in visual grounding tasks, with a score of 91.3 on the RefCOCO+loc (val) test.
  • Spatial Reasoning: In terms of spatial reasoning ability, GLM-4.5V achieves an excellent score of 87.3 on the CV-Bench test.
  • Coding: In programming tasks, GLM-4.5V scored 82.2 on the Design2Code benchmark, demonstrating its capabilities in code generation and understanding.
  • Video Understanding: GLM-4.5V also performs excellently in video understanding, scoring 74.6 on the VideoMME (w/o sub) test.
GLM-4.5V

GLM-4.5V Project Address

  • GitHub Repository：https://github.com/zai-org/GLM-V/
  • HuggingFace Model Hub：https://huggingface.co/collections/zai-org/glm-45v-68999032ddf8ecf7dcdbc102
  • Technical Paper：https://github.com/zai-org/GLM-V/tree/main/resources/GLM-4.5V_technical_report.pdf
  • Desktop Assistant Application：https://huggingface.co/spaces/zai-org/GLM-4.5V-Demo-App

How to Use GLM-4.5V

  • Register and Log In: Visit the Z.ai official website and register an account with your email. After completing registration, log in to your account.
  • Select Model: After logging in, select GLM-4.5V from the model selection dropdown.
  • Experience Features：
    • Web Front-End Replication: Upload a web design image, and the model will automatically generate front-end code.
    • Visual Reasoning: Upload images or videos, and the model will perform tasks such as visual understanding, object recognition, and scene analysis.
    • Image Seeking Game: Upload a target image, and the model will find the matching image in a complex scene.
    • Video Understanding: Upload a video file, and the model will extract key information, generate a video summary, or produce event detection results.

GLM-4.5V API Call Pricing

  • Input: 2 yuan/M tokens
  • Output: 6 yuan/M tokens
  • Response Speed: Reaches 60-80 tokens/s

GLM-4.5V Application Scenarios

  • Web Front-End Replication: Upload a web design image, and the model quickly generates front-end code, helping developers implement web development efficiently.
  • Visual Question Answering: Users upload images and ask questions, and the model generates accurate answers based on the image content, usable in fields such as education and intelligent customer service.
  • Image Seeking Game: Quickly find the target image in complex scenes, suitable for security monitoring, smart retail, and entertainment game development.
  • Video Understanding: Analyze video content, extract key information to generate summaries or detect events, optimizing video recommendation, editing, and monitoring.
  • Image Description Generation: Generate accurate descriptive text for uploaded images, assisting visually impaired people in understanding images and improving the social media sharing experience.