Cortex is an open-source LLM project dedicated to making training costs affordable for individual developers. It implements the entire process of building a large model from scratch, with fully open-source and decoupled code.
Main Capabilities
- Open-source full-pipeline training code: Covers the full lifecycle training flow of Pretrain, Midtrain, SFT, DPO, and PPO.
- Domestic chip adaptation: Completes the full-process training on domestic chips (MLU370 × 4) for the first time.
- Extremely lightweight MoE architecture: Adopts a Mixture of Experts (MoE) design, with a total parameter count of only 0.1B and inference activated parameters as low as approximately 67M, enabling ultra-high inference throughput on extremely low-compute devices.
- LLM as Judge-driven PPO training: Introduces a large language model as a judge to provide feedback signals during the reinforcement learning stage, improving the quality and efficiency of alignment training.
- Frontier architecture feature extensions: Adds support for the Attention Residuals mechanism, disabled by default, which can be enabled by modifying ENABLE_ATTN_RES in utils.py.
- Thinking control: Supports choosing to enable or disable thinking mode by adding /think and /no think tags.
- Highly decoupled: The model definition (qibin0506/llm_model) and training framework (qibin0506/llm_trainer) are separated into independent repositories.
Usage
Environment Preparation
Ensure Python >= 3.10.
Get the Code
git clone https://github.com/qibin0506/Cortex.git
cd Cortex
Install Dependencies
pip3 install -r requirements.txt
Start the Service
python3 app.py
The first run will automatically download the model files. After startup, visit http://0.0.0.0:8080/ to try it out.
Training Process
The training process is divided into four main stages, please execute them in order.
- Monitoring: Logs are located in the ./log directory. View metrics:
vis_log ./log/log.txt; view learning rate:vis_lr ./log/lr.txt. - Checkpoint conversion: After each stage ends, DeepSpeed's Checkpoint needs to be converted to a standard bin file so that it can be loaded in the next stage.
- PPO stage: Includes joint training of the Policy Model and Value Model. It uses LLM as Judge as the reward model. By setting JUDGE_API_KEY (required), JUDGE_BASE_URL (optional, default is siliconflow.cn), and JUDGE_MODEL (optional, default is DeepSeek-V3.2), you can choose a suitable LLM as the reward model. Note: If JUDGE_BASE_URL and JUDGE_MODEL use the defaults, JUDGE_API_KEY needs to be applied for from siliconflow.
Model experience
The Cortex 3.1 MoE model has been released and supports controlling whether thinking is enabled by adding the /think and /no think tags. You can visit the ModelScope Studio to experience the model's performance directly.
Applicable scenarios
Suitable for individual developers, researchers, and learners to practice full-process training of large language models, and especially suitable for exploring MoE architecture, reinforcement learning alignment (PPO), and domestic chip training under limited computing power conditions.
Limitations
Specific hardware requirements, training data scale, model performance metrics, license information, etc. for the project are not yet provided in the materials.
