XrayGLM is a Chinese multimodal medical large model focused on chest X-ray report generation, aiming to fill the gap in multimodal large models in the medical field, especially for the Chinese community.
Main capabilities:
- XrayGLM can generate diagnostic reports for chest X-ray images and supports multi-turn interactive dialogue.
- The model is fine-tuned based on VisualGLM-6B, using a constructed Chinese chest X-ray diagnostic dataset.
- This dataset was constructed from ChatGPT and public datasets (MIMIC-CXR and OpenI), and includes English reports and their Chinese translations.
Usage:
- Users can run XrayGLM through Google Colab Pro, or deploy it locally and access http://127.0.0.1:7860 for interaction.
- The model training reproduction process was carried out on A100 (4X, 80GB), and Chinese diagnostic and treatment reports and X-ray image data need to be prepared.
- The project provides an official video tutorial and a reference to Fudan University's course 'Theory and Practice of Large Language Models'.
Applicable scenarios:
- Suitable for medical imaging diagnosis research, development of multi-turn medical dialogue systems, and academic exploration of Chinese medical multimodal large models.
Limitations:
- The project-related resources are for academic research only and are strictly prohibited for commercial use.
- The content generated by the model is affected by factors such as computation, randomness, and quantization precision loss, so accuracy cannot be guaranteed, and it cannot be used as a basis for actual medical diagnosis.
- The model's capabilities depend on data support. Currently, it is trained on low-quality data and weights, so its performance is limited.
- When using third-party code, the corresponding open-source licenses must be followed.
