MiniCPM-V 4.6 - OpenBMB's Open-Source On-Device Multimodal Large Model

What is MiniCPM-V 4.6

MiniCPM-V 4.6 is an on-device multimodal large model released by OpenBMB, with an LLM parameter count of only 1.3B, specifically optimized for local deployment on mobile devices. The model is implemented based on the llama.cpp framework, supports fully offline operation on three major platforms: iOS, Android, and HarmonyOS NEXT. The model size is approximately 1.6GB (Q4_K_M quantization), and it can run smoothly with a minimum of 6GB of memory. MiniCPM-V 4.6 possesses multimodal capabilities such as image understanding, OCR, and video understanding, achieving visual perception performance close to larger models with an extremely small parameter scale.

Main Features of MiniCPM-V 4.6

• Pure Local On-Device Operation: Implemented based on llama.cpp , it can complete image Q&A, OCR recognition, visual understanding, and other tasks on a phone without an internet connection.
• Ultra-Lightweight Model Size: The LLM is only 0.5GB + the vision tower is 1.1GB, with a total download size of approximately 1.6GB, greatly lowering the barrier to on-device deployment.
• Native Support for Three Platforms: Provides complete project source code for iOS (Xcode / Swift), Android (Gradle / Kotlin), and HarmonyOS (DevEco / ArkTS).
• Multimodal Understanding: Supports tasks such as image description, visual question answering, document OCR, and video frame understanding.
• Low Memory Threshold: Recommended 6GB RAM is sufficient to run, compatible with mid-range and low-end phones and tablet devices.

Technical Principles of MiniCPM-V 4.6

• Architecture: The model adopts the MiniCPM-V series architecture, 1.3B parameter LLM + vision encoder (ViT) + projection layer (mmproj).
• Quantization Strategy: The vision tower maintains f16 precision to avoid degradation in perception quality, while the LLM uses Q4_K_M GGUF quantization to compress size.
• Inference Engine: Based on llama.cpp（Support-iOS-Demo branch), deeply optimized for the ARM architecture (arm64-v8a).
• Context Management: Default 4K tokens context window, KV Cache and model weights share device memory.

How to Use MiniCPM-V 4.6

• Build from Source：
  • Run git clone https://github.com/OpenBMB/MiniCPM-V-Apps.git to clone the repository and enter the project directory.
  • Run git submodule update --init --recursive to pull the llama.cpp submodule and related dependencies.
  • iOS developers use Xcode to open MiniCPM-V-demo/MiniCPM-V-demo.xcodeproj, select the target device, and click the Run button to build and run.
  • Android developers enter the MiniCPM-V-demo-Android directory and execute the ./gradlew assembleDebug command to generate a Debug installation package.
  • HarmonyOS developers use DevEco Studio to open the MiniCPM-V-demo-HarmonyOS project, configure automatic signing, connect the device, and click Run.
• Model File Deployment：
  • Download the approximately 0.5GB language model GGUF file from the HuggingFace openbmb/MiniCPM-V-4.6-gguf repository.
  • Download the approximately 1.1GB mmproj-model-f16.gguf vision model file from the same repository to maintain perception accuracy.
  • When launching the app for the first time, open the built-in Model Manager and click the Download button to automatically complete the model download.
  • Android users can use the adb push command to manually push the model files to the app-specified external storage directory.
  • HarmonyOS users can use the hdc file send command to manually import the model to the specified directory.

Core Advantages of MiniCPM-V 4.6

• Extremely small parameter scale: 1.3B parameters achieve usable multimodal capabilities, with inference speed far faster than 7B+ models.
• On-device privacy protection: Data is processed entirely locally and is not uploaded to the cloud, making it suitable for sensitive documents and privacy scenarios.
• Complete open-source Demo: Provides prebuilt installation packages (TestFlight / APK / HAP) and complete source code, supporting secondary development.

MiniCPM-V 4.6 project address

• GitHub repository：https://github.com/OpenBMB/MiniCPM-V-edge-demo
• HuggingFace model library：https://huggingface.co/openbmb/MiniCPM-V-4.6

Comparison of MiniCPM-V 4.6 with similar competitors

Application scenarios of MiniCPM-V 4.6

• Mobile offline AI assistant: Users can perform image question answering, object recognition, and visual content analysis by taking photos with their phones in no-network environments, meeting intelligent interaction needs in outdoor or weak-network scenarios.
• On-device document scanning and OCR recognition: Supports local offline recognition of document content such as contracts, invoices, business cards, and handwritten notes, ensuring sensitive business information is not uploaded to the cloud and safeguarding data privacy and security.
• Visual content analysis for privacy-sensitive scenarios: Suitable for scenarios such as local pre-screening of medical images and extraction of personal ID information. All image data is processed on-device, eliminating the risk of third-party cloud leaks.
• Offline image annotation in low-bandwidth environments: Provides local image description, tag generation, and content moderation capabilities for content creators and e-commerce operators in regions with unstable networks, without relying on online APIs.
• Embedded visual interaction for smart hardware: Can be integrated into edge hardware such as smart homes, in-vehicle terminals, and industrial inspection equipment to achieve low-power, low-latency real-time visual understanding and voice feedback.