What is MingQi
MingQi is the first medical multimodal large model for precise diagnosis of rare diseases, launched by the LoCCS Lab of the School of Computer Science at Shanghai Jiao Tong University. Based on a dual-engine driven architecture of "large model capability matrix + expert routing collaboration", it integrates data such as medical imaging, medical record text, and laboratory indicators to provide doctors with high-precision, interpretable diagnostic support. In the diagnosis of gastrointestinal rare diseases such as Crohn's disease, the accuracy exceeds 92%, surpassing the level of senior specialist doctors. Based on model distillation and quantization technology, MingQi greatly reduces computing power requirements, enables low-cost, localized deployment, helps sink medical resources, and brings new possibilities for solving the problem of imbalanced medical resources in China.
Main functions of MingQi
- Precise diagnosis: Integrates multimodal data (imaging, medical records, laboratory indicators) to achieve high-precision diagnosis of rare diseases (such as Crohn's disease), with an accuracy exceeding 92%.
- Interpretability: Visualizes the diagnostic process, provides reasoning basis and comparison with similar cases, and enhances doctor trust.
- Multi-expert collaboration: Simulates the diagnostic thinking of multiple experts, synthesizes opinions from multiple parties, and improves the comprehensiveness and accuracy of diagnosis.
- Localized deployment: Reduces computing power requirements, enables low-cost, secure localized operation, and meets data privacy requirements.
Technical principles of MingQi
- Dual-engine driven architecture：
- Large model capability matrix: Based on large-scale pretrained models, learns features and patterns from massive medical data, and provides strong foundational capabilities for rare disease diagnosis.
- Expert routing collaboration: Simulates the diagnostic thinking of multiple experts, integrates the experience and knowledge of different experts into the model, and enables multi-expert collaborative diagnosis.
- Multimodal deep learning: Integrates multimodal data such as medical imaging (e.g., CT, MRI, endoscopic images, etc.), medical record text (e.g., medical history, symptom descriptions, etc.), and laboratory indicators (e.g., blood, biochemical indicators, etc.). Based on deep learning algorithms, extracts key features from multimodal data to achieve comprehensive diagnosis of rare diseases.
- Transparent diagnostic cabin mechanism: Visually presents each step and reasoning logic in the diagnostic process, allowing doctors to clearly understand the basis of AI diagnosis. Provides comparison with a similar case library to help doctors quickly locate suspected lesion areas and enhance the interpretability of diagnosis.
- Model distillation and quantization: Based on model distillation technology, compresses complex pretrained models into smaller, more efficient models, reducing computing power requirements. Based on quantization technology, further optimizes the model's operating efficiency and enables efficient operation on low-cost hardware devices.
Project address of MingQi
- Project official website：https://news.sjtu.edu.cn/jdyw/
Application scenarios of MingQi
- Precise diagnosis of rare diseases: Assists doctors in quickly and accurately diagnosing rare diseases and shortens diagnosis time.
- Empowerment of primary healthcare: Provides low-cost, efficient diagnostic services for primary hospitals and alleviates the shortage of expert resources.
- Teaching and research support: Supports medical teaching and research, improves diagnostic capabilities, and promotes technological development.
- Telemedicine services: Combined with telemedicine, provides remote diagnosis for patients in remote areas and improves medical accessibility.
- Medical data management: Integrate multimodal data to facilitate storage, querying, and analysis, and to uncover disease risk factors.