SuperAPI AI Directory
Partner with us中文AI newsFavorites
DirectoryNews and tutorials

Kiln AI - Open-source AI prototyping and dataset collaborative development tool, fine-tune custom models

Author: AI小集 · Original source: AI工具集

What is Kiln AI

Kiln AI is an open-source AI development tool that simplifies fine-tuning of large language models (LLMs), synthetic data generation, and dataset collaboration. Kiln AI provides an intuitive desktop application supporting Windows, MacOS, and Linux, allowing users to fine-tune multiple models (such as Llama, GPT4o, and Mixtral) in a zero-code manner, enabling automatic deployment. Kiln AI provides interactive tools to generate training data and supports Git-based version control, facilitating team collaboration. Kiln AI supports automatic prompt generation, integration of multiple models, and prioritizes privacy to ensure user data security. Kiln AI's Python library is open-source, making it easy for developers to integrate into existing workflows.

Kiln AI

Main features of Kiln AI

  • Intuitive desktop application: Supports Windows, MacOS, and Linux systems, provides one-click installation and use, with a simple and intuitive design.
  • Zero-code fine-tuning: Supports multiple language models, such as Llama, GPT4o, and Mixtral, with automatic serverless deployment of models.
  • Synthetic data generation: Provides interactive visualization tools for generating training data.
  • Team collaboration: Git-based version control, supports multi-person collaboration, suitable for QA, PM, and domain experts to jointly participate in dataset construction.
  • Automatic prompt generation: Supports automatically generating prompts from data, including chain-of-thought, few-shot, and multi-shot prompts, etc.
  • Broad support for models and providers:Supports models based on Ollama, OpenAI, OpenRouter, Fireworks, Groq, AWS, or any model compatible with the OpenAI API.

Technical principles of Kiln AI

  • Git-based version control: Uses Git as the underlying version control system, supporting multi-person collaboration and version management of datasets. Dataset files are stored in JSON format, supporting parallel collaboration and conflict resolution.
  • Serverless Deployment: Fine-tuned models support automatic deployment to the cloud or locally, without the need for manual server configuration. Supports multiple cloud platforms and local environments.
  • Interactive data generation tools: Provides an interactive interface to help users generate high-quality synthetic data based on visualization tools. Supports multiple data generation strategies, such as few-shot learning, multi-shot learning, etc.
  • Python library integration: Provides an open-source Python library, making it easy for developers to integrate datasets into existing workflows. Supports use in Jupyter Notebook, facilitating in-depth analysis by data scientists.
  • Multi-model support: Supports multiple language models and platforms based on the adapter pattern, providing a unified API interface, making it easy for users to switch between different models and providers.

Kiln AI project address

Kiln AI quick start

  • Download and install：
    • Desktop application: Download and install the free desktop application on MacOS, Windows, and Linux based on the provided download link.
    • Python library: Use the pip install kiln-ai command to install the Python library, integrate datasets into your own workflow, build fine-tuned models, use Kiln AI in Notebooks, build custom tools, etc.
  • Launch the application：
    • After completing the Android setup, launch the app and follow the guide to create projects and tasks, and connect to AI providers (such as Ollama, OpenAI, OpenRouter, etc.).
    • Use example tasks for a quick experience, or define custom tasks according to your needs.

Kiln AI's models and AI providers

  • AI models and providers supported by Kiln:
    • Supported providers: Kiln AI is compatible with a variety of cloud service providers and AI models, such as OpenAI, Groq, OpenRouter, AWS, Fireworks, etc. Users need to provide their own API keys, and the system will not access users' datasets.
    • Compatible servers: Users can connect to any OpenAI-compatible server, such as LiteLLM, for vLLM, etc.
  • Set up AI providers：
    • Initial setup: When Kiln is run for the first time, the app prompts the user to set up one or more AI providers. At least one provider is required to use the core features of Kiln AI.
    • Add or remove providers: Users can add a new provider in "Settings > AI Providers and Models," or edit the ~/.kiln_ai/settings.yaml file to remove an existing provider.
  • Using and adding models：
    • Recommended built-in models: The models have been tested and are compatible with various features of Kiln AI, making them very easy to use without errors. Users only need to connect any AI provider on the settings page and select a model on the run screen.
    • Custom models: If users want to use a model that is not in the list, but the model is from an AI provider supported by Kiln AI. Users need to use the model in "Add Model" under "Settings > AI Providers and Models," and the model will appear in the "Untested" section of the model dropdown menu.
    • Custom OpenAI-compatible servers: If users have an OpenAI-compatible server (such as LiteLLM, vLLM, etc.), it can be used in Kiln AI. Users need to add a "Custom API" in "Settings > AI Providers and Models." All models supported by that API will appear in the "Untested" section of the model dropdown menu.

Kiln AI's synthetic data generation

  • How it works：
    • Synthetic data methods
      • Zero-shot data generation: Generate data directly based on the task definition, with no additional configuration required.
      • Topic tree data generation: Quickly generate a broad range of data based on a generated topic tree (including nested topics). Users can choose automatic topic generation or manually add topics.
      • Structured data generation: If the task requires structured input or output, the generated data will follow the user-defined JSON schema. All generated values will be validated, and data that does not conform to the schema will not be saved.
    • Generation options：
      • Model selection: Supports a variety of models (such as OpenAI, Anthropic, Llama, Google, Mistral, etc.) and a variety of hosts (such as Ollama). Users can choose the appropriate model according to their needs.
      • Prompt options: After rating some examples, unlock more powerful prompt options, such as few-shot, multi-shot, chain-of-thought prompting, etc., to improve the quality of generated data.
    • Human guidance：
      • Generate content covering global topics, not limited to the United States.
      • Generate examples in Spanish.
      • Generate specific data for problems that are difficult for the model to classify (such as sarcastic information).
  • Iterate：
    • Open the synthetic dataset tab.
    • Choose high-quality models (including production models that are not fast enough and are costly).
    • Start generating data that demonstrates the problem, using the human guidance feature and better models to ensure output quality.
    • Manually delete examples with incorrect style.
    • The synthetic data tool can generate correct data (by combining models and guidance), scaling generation to hundreds of samples.
    • Save the new synthetic dataset.

Kiln AI fine-tuning guide

  • Step 1: Define the task and objectives
    • Task definition: Create a new task in the Kiln UI, including the initial prompt, requirements, and input/output schema.
    • Example task: Generate news headlines; given a summary of a news topic, generate news headlines in different styles.
  • Step 2: Generate training data using synthetic data
    • Synthetic data generation: Provides an interactive interface to quickly generate high-quality synthetic datasets.
    • Generation process: Generate 920 training samples in 9 minutes, supporting multiple models and multiple prompting strategies.
    • Prompting techniques: Use high-quality models and detailed prompts (such as multi-sample prompting and chain-of-thought) to improve data quality.
  • Step 3: Choose the model to fine-tune
    • Supported models：
      • OpenAI: GPT-4o and 4o-Mini
      • Mistral：Mixtral 8x7b MoE
      • Others: Llama 3.2 (1b/3b/11b), Llama 3.1 (8b/70b), etc.
  • Step 4: Start the fine-tuning job
    • Fine-tuning operations: In the "Fine-tuning" tab of the Kiln UI, select the model, dataset, and training parameters.
    • Dataset splitting: It is recommended to create a test set and a validation set to evaluate model performance after fine-tuning is complete.
  • Step 5: Deploy and run the model
    • Automatic deployment: After fine-tuning is complete, the system automatically deploys the model with no additional configuration required.
    • How to use: Select and use the model via the "Run" tab in the Kiln UI.
  • Step 6 (optional): Train on your own infrastructure
    • Export the dataset: Export the dataset to a common format so it can be fine-tuned on your own infrastructure.
    • Recommended platforms: Unsloth and Axolotl, which support a variety of open-source models.
    • Example: Use the Unsloth notebook to load a dataset exported from Kiln AI and fine-tune it locally or in Google Colab.
  • Next steps
    • Evaluate the model: Use the test set and validation set to evaluate the performance of the fine-tuned model.
    • Iterative improvement: Based on the evaluation results, adjust hyperparameters, optimize prompts, fix errors, and fine-tune again.
    • Data strategy: Use a "stepwise" data strategy, gradually expanding from a small number of high-quality samples to a large amount of synthetic data.
  • Cost: Fine-tuned models from Fireworks and OpenAI use "serverless" deployment, charging by usage with no fixed costs.

Kiln AI Guide to Training Reasoning Models

  • Key steps for training reasoning models：
    • Ensure training data contains "reasoning": Use reasoning models or chain-of-thought prompting to generate training data, ensuring the dataset contains reasoning content.
    • Create a training dataset containing reasoning: When creating the dataset, filter for samples that contain reasoning/thinking.
    • Choose the correct training strategy: Choose the "final response and intermediate reasoning" training strategy to include reasoning data.
    • Use appropriate prompts when calling the fine-tuned model: It is recommended to use the same prompt used during training for the best results.
  • Choosing between reasoning and chain-of-thought：
    • Reasoning models: Suitable for scenarios requiring cross-domain reasoning capabilities, creating smaller, faster models based on fine-tuning large reasoning models (such as Deepseek R1).
    • Chain-of-thought: Improve output quality based on a simple "think step by step" prompt, or customize thinking prompts to generate training sets for specific tasks.

Kiln AI application scenarios

  • Intelligent customer service: Intelligent customer service systems generate customer service conversation datasets and fine-tune language models to improve answer accuracy and relevance.
  • Healthcare: In AI projects in the healthcare field, doctors (domain experts) generate medical datasets, data scientists perform model fine-tuning, and the QA team is responsible for verifying data quality.
  • Rapid prototyping and experimentation:When developing text generation tools, use few-shot prompting and multi-model support to quickly experiment with the generation effects of different models.
  • Education:EdTech companies build educational datasets, including student questions and answers, for use in fine-tuning educational AI models.
  • Financial industry: The financial industry fine-tunes risk assessment models, with all data processed locally to ensure customer data is not leaked.