What is MARS
MARS (Make vAriance Reduction Shine) is an innovative optimization framework launched by ByteDance that improves the efficiency of large model training. MARS combines preconditioned gradient methods with variance reduction techniques, optimizing gradient estimation based on the scaled stochastic recursive momentum technique. The MARS framework is flexible and supports full-matrix or diagonal Hessian approximations, giving rise to three optimization algorithm instances based on AdamW, Lion, and Shampoo. Experimental results show that MARS demonstrates outstanding performance when training GPT-2 models compared with the traditional AdamW optimizer.
Main features of MARS
- Improved training efficiency: MARS combines preconditioned gradient methods with variance reduction techniques to improve the efficiency of large model training, especially when training deep neural networks and large language models.
- Unified optimization framework: Provides a unified framework adaptable to different Hessian approximation methods, including full-matrix and diagonal-matrix approximations.
- Algorithm instantiation: Under the MARS framework, three specific optimization algorithm instances are implemented: MARS-AdamW, MARS-Lion, and MARS-Shampoo, based respectively on different preconditioned gradient update strategies.
- Variance reduction: Introduces the scaled stochastic recursive momentum technique, effectively reducing gradient variance during training and accelerating model convergence.
Technical principles of MARS
- Preconditioned gradient methods: Adjusts the learning rate based on preconditioned gradient methods, giving each parameter or parameter group a customized learning rate adapted to its local curvature.
- Variance reduction techniques: Introduces variance reduction techniques, such as STORM (Stochastic Recursive Momentum), to reduce the variance of stochastic gradients and accelerate the optimization process.
- Scaled stochastic recursive momentum: Introduces a scaling parameter on the basis of STORM to adjust the strength of variance reduction and define a new gradient estimator.
- Gradient clipping and exponential moving average: To optimize training stability, MARS applies gradient clipping in the gradient estimator and uses exponential moving average (EMA) to compute the recursive momentum.
MARS project address
- arXiv technical paper：https://arxiv.org/pdf/2411.10438
Application scenarios of MARS
- Deep learning model training: Training deep neural networks, especially complex models with many parameters.
- Large-scale language models: Optimizing the training process of large language models, such as GPT-series models, improving training efficiency and model performance.
- Computer vision tasks: In computer vision fields such as image classification and object detection, accelerating model training and improving model generalization ability.
- Reinforcement learning algorithms: In reinforcement learning, optimizing the parameters of policy networks or value functions, especially when facing high-variance gradients.
- Recommender system models: When building recommender systems, optimizing model parameters to better handle large-scale user and item features.