What is Ovis1.6
Ovis1.6 is a multimodal large model launched by Alibaba International AI Team. It has achieved excellent results on the authoritative multimodal comprehensive evaluation benchmark OpenCompass, especially ranking first in overall score among models with under 3 billion parameters, surpassing other mainstream models. The Ovis1.6 model performs excellently in multiple tasks such as mathematical reasoning and visual understanding, even surpassing the closed-source GPT-4o-mini model. Ovis1.6 can process multiple data inputs including text and images, and possesses strong multimodal task processing capabilities such as visual perception reasoning, mathematical and scientific problem solving, and life scenario understanding.
Main features of Ovis1.6
- Mathematical reasoning Q&A: Accurately answer various mathematical questions, including complex mathematical formulas and logical reasoning.
- Object recognition: Recognize different objects, such as flower varieties, demonstrating its capability in image recognition.
- Text extraction: Supports text extraction in multiple languages. Ovis1.6 can recognize and extract text information from various documents.
- Complex task decision-making: Process and understand multiple types of data inputs, and perform complex decision-making tasks, such as comprehensive analysis of images and text.
- Image understanding: Reaches SOTA (State of the Art) level in image understanding tasks, and can process images with high resolution and extreme aspect ratios.
Technical principles of Ovis1.6
- Innovative architecture design: Ovis1.6 is based on the architecture of a visual tokenizer plus a visual embedding table and a large language model. The design introduces a learnable visual embedding table, converting continuous visual features into probabilistic visual tokens, and then obtains structured visual embeddings through multiple index weightings of the visual embedding table, improving performance on multimodal tasks.
- High-score image processing: Ovis1.6 supports processing images with extreme aspect ratios and is compatible with high-resolution images, enabling the model to demonstrate excellent capabilities in image understanding tasks.
- Comprehensive data optimization: Ovis1.6 uses multiple types of datasets in training, including Caption, VQA, OCR, Table, Chart, etc. Comprehensive data coverage significantly improves the model's performance on tasks such as multimodal question answering and instruction following.
- Outstanding model performance: On the authoritative multimodal comprehensive evaluation OpenCompass, Ovis1.6-Gemma2-9B achieved the first overall ranking among models with under 30B parameters, demonstrating excellent performance.
Project address of Ovis1.6
- GitHub repository：https://github.com/AIDC-AI/Ovis
- HuggingFace model library：https://huggingface.co/AIDC-AI/Ovis1.6-Gemma2-9B
- arXiv technical paper:https://arxiv.org/pdf/2405.20797
Application scenarios of Ovis1.6
- Education and learning assistance: Ovis1.6 can accurately answer mathematical questions, recognize and explain mathematical formulas. As an educational tool, it can help students learn and understand complex concepts.
- Agriculture and plant recognition: Through object recognition capabilities, Ovis1.6 helps identify different varieties of plants, playing an important role in fields such as agricultural research and plant protection.
- Language translation and text processing: Supports text extraction and translation in multiple languages, suitable for cross-language communication, international business, and multilingual content creation.
- Image recognition and analysis: Recognize handwriting and complex images, suitable for image content moderation, security monitoring, and artwork analysis.
- Autonomous driving: Integrate visual data to improve the environmental perception and decision-making capabilities of autonomous vehicles, and enhance driving safety.
- Medical Diagnosis: Assist doctors in medical image analysis to improve the accuracy and efficiency of disease diagnosis.