What is the SenseNova Fusion Large Model
"SenseNova" Fusion Large Model (SenseNova) is a multimodal large model officially launched by SenseTime on January 10, 2025. The model achieves native fusion of modalities, with both deep reasoning capabilities and multimodal information processing capabilities greatly improved. It can process various types of information such as text, images, and video, breaking through the limitations between modalities and claiming first place on both authoritative evaluation leaderboards, SuperCLUE and OpenCompass, becoming the "double champion."
Main functions of the SenseNova Fusion Large Model
- Image recognition and analysis: Can accurately recognize and analyze the content in images, including blurred text, complex scenes, and more.
- Video processing: Can process video content, extract key information, perform video editing and generation operations, and enhance the video interaction experience.
- Speech recognition and synthesis: Combines speech and natural language processing capabilities to enhance the interaction experience, such as in applications including voice customer service and online education.
- Text processing: Has powerful text understanding and generation capabilities, and can process complex rich-modal documents, such as documents in fused forms including tables, text, images, and video.
- Mathematical calculation and logical reasoning: Can solve complex mathematical problems, such as calculating whether 2 to the power of 31 or 3 to the power of 21 is larger, and answering using the method of logarithmic functions.
- Data analysis and decision support: Can analyze information in data charts, extract key elements, provide conclusions, and offer decision support for users.
Technical principles of the SenseNova Fusion Large Model
- Native fusion of modalities: The model can simultaneously process various types of information such as text, images, and video, breaking through the limitation of traditional large language models that only support single text input.
- Fused multimodal data synthesis：
- Inverse rendering technology: Through inverse rendering technology, image and text data are fused to generate large amounts of synthetic data. These synthetic data establish numerous interaction bridges between image and text modalities, enabling the model to more solidly grasp the rich relationships between modalities.
- Hybrid-semantics-based image generation: Uses hybrid semantic generation technology to further enrich fused multimodal data and enhance the model's ability to understand multimodal information.
- Fusion task-enhanced training: Builds rich cross-modal tasks, providing a solid foundation for model training. These tasks include not only traditional text processing tasks, but also multimodal tasks such as image recognition and video analysis, enabling the model to effectively respond to user needs in various business scenarios.
- Deep reasoning capability：
- Proficient in both arts and sciences: In the SuperCLUE annual evaluation, the liberal arts score ranked first globally with 81.8 points, and the science score won a gold medal, with the calculation dimension ranking first domestically with 78.2 points.
- Complex problem solving: Can process complex rich-modal documents, such as documents in fused forms including tables, text, images, and video, and provide in-depth reasoning support.
Project address of the SenseNova Fusion Large Model
- Project address：https://platform.sensenova.cn/home
Application scenarios of the SenseNova Fusion Large Model
- Autonomous driving: Processes complex multimodal information and enhances decision-making capability.
- Video Interaction: Improve the efficiency of video content generation, editing, and analysis.
- Office and Education: Efficiently process complex rich-modality documents to improve office and education efficiency.
- Finance: Analyze and process multi-source heterogeneous data to provide accurate risk assessment and investment advice.
- Park Management: Improve the management efficiency and security of parks.
- Industrial Manufacturing: Optimize production processes and quality control.