Purpose
This project is an AI Workbench sample project provided by NVIDIA, used to deploy LLaMA-Factory and demonstrate an end-to-end model development workflow. Its core goal is to complete fine-tuning of large language models and related processing workflows within the AI Workbench environment.
Main Capabilities
According to the project materials, the workflow includes the following stages:
- Use LLaMA-Factory to fine-tune large language models.
- Fuse the generated LoRA adapter with the base model.
- Perform Post Training Quantization on the model.
- Export the model as a TensorRT-LLM checkpoint.
- Support unsloth.
Usage
This project is in the form of an AI Workbench project and requires deploying LLaMA-Factory in AI Workbench. The materials recommend following the RTX AI Toolkit LLM fine-tuning tutorial for operation.
Environment requirements:
- Minimum recommended AI Workbench version: 0.50.16
- Docker Desktop version: 4.31+
Applicable Scenarios
Suitable for developers and researchers who need to experience or practice large language model fine-tuning, LoRA fusion, post-training quantization, and TensorRT-LLM export workflows in the AI Workbench environment.
Limitations
The materials do not provide a list of supported models, hardware configuration requirements, specific commands, parameter settings, pricing, or licensing information. These contents are not currently provided.
