external-mindstudio-msmodelslim-model-adapt

msModelSlim Basic Model Adaptation Skill

This Skill guides how to create a basic adapter for a new model so that it can run through the W8A8/W4A16 quantization process.

Scope

• Supported: Decoder-only LLM, understanding-type VLM (only processes the text/LLM backbone)
• Not supported: multimodal generation (such as Stable Diffusion/Flux/Wan), Encoder-only, non-Transformers

Core Workflow

1. Preparation

• Download the model: It is recommended to use modelscope download to download non-weight files.
  • Example:modelscope download --model <org>/<model> --local_dir ./models/<name> --exclude '*.safetensors'
• Analyze the model: Read config.json and modeling_*.py, and confirm the structure and implementation.
  • See:Model Structure Analysis Guide

2. Create the adapter

• Use the template：
  • LLM: assets/model_adapter_template.py
  • VLM: assets/vlm_model_adapter_template.py
• Implement the interfaces: Implement handle_dataset, init_model, generate_model_visit, generate_model_forward, enable_kv_cache。
• Key principles：
  • visit and forward must be strictly consistent.
  • For MoE models, it is recommended to unpack into pure linear layers.
  • See:Adapter Implementation Guide

3. Register and install

• In config/config.ini register the model and entry point, and execute bash install.sh。
• See:Adapter Registration Guide

4. Verify the adapter (required)

• The four-step verification must be performed: generate a test model -> full-fallback quantization -> verify that the full-fallback model is strictly consistent with the floating-point weights and can be fully loaded/saved -> verify that the actual quantization process works properly (including validation of description file rules).
• See:Adapter Verification Guide

Common Scripts

The scripts are located in the scripts/ directory:

• scripts/step1_generate_test_model.py
• scripts/step2_run_quantization.py
• scripts/step3_verify_weights.py
• scripts/step4_verify_quant_description.py

References

• Model Structure Analysis Guide
• Adapter Implementation Guide
• Adapter Registration Guide
• Adapter Verification Guide
• Interface Checklist
• Core Workflow