h2o-llmstudio

H2O LLM Studio is a framework and no-code GUI for fine-tuning large language models (LLMs).

Purpose

The tool is designed to allow users to easily and effectively fine-tune state-of-the-art LLMs without any coding experience. It provides a graphical user interface (GUI) specifically designed for large language models, and also supports a command-line interface (CLI) for fine-tuning.

Key capabilities

• Use a graphical user interface (GUI) specifically designed for large language models.
• Fine-tune any LLM with a large number of hyperparameters.
• Use recent fine-tuning techniques such as low-rank adaptation (LoRA) and 8-bit model training with a low memory footprint.
• Fine-tune models using reinforcement learning (RL) (experimental).
• Evaluate model-generated answers using advanced evaluation metrics.
• Visually track and compare model performance, with W&B integration available.
• Chat with the model to get instant feedback on model performance.
• Easily export models to the Hugging Face Hub and share them with the community.

Usage

Installation

• It is recommended to install H2O LLM Studio using uv with Python 3.10.
• A virtual environment setup method is provided.
• The GUI can be run via Docker.
• The H2O LLM Studio GUI can be run.
• The H2O LLM Studio GUI can be run (using Docker).
• H2O LLM Studio can be run using the command-line interface (CLI).

Running the GUI

Use the corresponding command to start the H2O Wave server and app, then navigate to http://localhost:10101/（推荐使用 Chrome) to access H2O LLM Studio and start fine-tuning models.

Fine-tuning with the CLI

You can fine-tune using the CLI and specifying a configuration file (.yaml) containing all experiment parameters. Multi-GPU runs in DDP mode are supported. By default, the framework runs on the first k GPUs; if you need to specify particular GPUs, you can use the CUDA_VISIBLE_DEVICES environment variable before the command.

Interactive chat

Use the command to start an interactive chat with a trained model, where experiment_name is the output folder of the experiment you want to chat with. Interactive chat also works for models fine-tuned using the UI.

Publish to Hugging Face

Use the command to publish the model to Hugging Face. path_to_experiment is the output folder of the experiment; device is the target device on which to run the model, either 'cpu' or 'cuda:0', with 'cuda:0' as the default; api_key is the Hugging Face API Key, which can be omitted if the user is already logged in; user_id is the Hugging Face user ID, which can be omitted if the user is already logged in; model_name is the name of the model to publish to Hugging Face and can be omitted; safe_serialization is a flag indicating whether to use safe serialization, with True as the default.

Cloud and remote setup

If running on a cloud-based machine such as runpod, you may need to set specific environment variables to allow the H2O Wave server to accept connections from the proxy. If you encounter timeouts when running the H2O Wave server remotely, you can increase the timeout by setting specific environment variables; the default is 5 seconds for both, so increase them if you encounter timeouts, and using -1 disables the timeout.

Training a model

First upload a dataset, then start training the model. First create an experiment, after which you can monitor and manage experiments, compare experiments, or push the model to Hugging Face to share it with the community. You can refer to the example guide for running an experiment via the CLI on OASST data.

Data format

For details on the data format required to import data or sample data that can be used to try out H2O LLM Studio, please refer to the Data format section in the H2O LLM Studio documentation.

Use casesand limitations

System requirements

H2O LLM Studio requires a machine with Ubuntu 16.04+ and at least one relatively recent NVIDIA GPU, NVIDIA driver version >= 580 (PyTorch runs on CUDA 13). For larger models, at least 24GB of GPU memory is recommended.

Development status

Due to the current rapid development, full backward compatibility of new features cannot be guaranteed. It is recommended to pin the framework version to the version used in the experiment. To reset, delete/back up the data and output folders.

Feature changes

• The new problem type Causal Regression Modeling allows training single-target regression data with LLMs.
• RLHF has been completely removed, replaced by DPO/IPO/KTO optimization.
• The separate maximum length settings for prompt and answer have been removed, replaced by a single max_length setting, to better resemble the chat_template functionality of transformers.
• Added KTOPairLoss for DPO modeling, allowing models to be trained with simple preference data; the data currently needs to be prepared manually, randomly matching positive and negative examples into pairs.
• Began deprecating RLHF, switching to DPO/IPO optimization. Training has been disabled, but old experiments can still be viewed. RLHF will be completely removed in a future version.
• Introduced a new problem type for DPO/IPO optimization, which can serve as an alternative to RLHF.
• Introduced DeepSpeed for sharded training, allowing larger models to be trained on multi-GPU machines, requiring NVLink. This feature replaces FSDP and provides more flexibility. DeepSpeed requires a system-installed CUDA Toolkit, and version 12.1 is recommended.
• The new problem type Causal Classification Modeling allows training binary and multiclass classification models with LLMs.
• User keys are now handled more securely and flexibly. Support has been added for handling keys with the 'keyring' library. User settings will attempt to migrate automatically.

Other information

All open-source datasets and models are published on H2O.ai's Hugging Face page and the H2OGPT repository. Detailed documentation and frequently asked questions (FAQ) can be found in https://docs.h2o.ai/h2o-llmstudio/ . To contribute to the documentation, please navigate to the /documentation folder of this repository and refer to README.md. Contributions to the H2O LLM Studio project are welcome; please refer to the CONTRIBUTING.md file. H2O LLM Studio is released under the Apache 2.0 license; please refer to the LICENSE file.