SuperAPI AI Directory
Partner with us中文AI newsFavorites
All toolsAI developer platformsC2C

C2C

C2C (Cache-to-Cache) is the official code implementation of the paper "Cache-to-Cache: Direct Semantic Communication Between Large Language Models," accepted at ICLR'26. It enables large language models to communicate directly through KV-Cache, bypassing the text generation step. The project projects and fuses KV-Cache between different models, allowing each model to share semantic representations in a more direct way. Official data shows that this method improves accuracy by 8.5–10.5% compared to a single model, improves by 3.0–5.0% compared to text-based communication, and achieves a 2.0x speedup in latency. The project provides the Python package rosetta (named after the Rosetta Stone), offering a general-purpose RosettaModel wrapper, training and evaluation scripts, configuration files, and pretrained C2C Fuser weights, supports multi-sharer model fusion, and provides an online Demo for hands-on experience.

Open website

About this tool

C2C (Cache-to-Cache) is the official code implementation of the paper "Cache-to-Cache: Direct Semantic Communication Between Large Language Models," accepted at the ICLR'26 conference. Its core goal is to enable large language models to communicate directly through KV-Cache, rather than first converting internal representations into text and then passing them to another model.

Purpose and Principle

C2C projects and fuses KV-Cache between models, enabling originally independent LLMs to share semantic representations in a richer and more direct way. According to the official description, this method improves accuracy by 8.5–10.5% compared to a single model, improves by 3.0–5.0% compared to text-based communication, and brings a 2.0x speedup in latency.

The Python package is named rosetta, taken from the Rosetta Stone—the same passage presented in multiple scripts, which unlocked the translation of ancient Egyptian hieroglyphs. C2C plays a similar role: translating between the KV-cache representations of multiple independent LLMs, letting them speak the same language.

Main Capabilities

  • Direct cache communication: Transmits semantics between different models through KV-Cache projection and fusion, bypassing text generation.
  • General-purpose wrapper: Provides the RosettaModel wrapper, which can apply C2C to your own LLM with a small amount of code.
  • Multi-model fusion: Supports fusing the KV-cache of multiple sharer models into a single receiver (available since 2025/12, officially marked as preliminary).
  • Pretrained Fuser: Provides released C2C Fuser weights on Hugging Face and a complete list of usable fuser pairings.
  • Arbitrary model pairs: Automatically handles cases such as different hidden dimensions, different numbers of layers, different attention head configurations, and different tokenizers.
  • Training and evaluation workflow: Only updates the C2C projector parameters, while the source model and target model remain frozen; provides configuration examples and a unified evaluation script.
  • Interactive experience: Provides a live chat example and an online Demo, allowing side-by-side model comparison.

Usage

Environment setup: Create a new environment and install the package; training and evaluation require additional dependencies.

Using an existing Fuser: Load the released C2C from the Hugging Face collection Fuser weights and run the inference script.

Run the chat example: script/playground/live_chat_example.py provides an interactive chat example demonstrating cache-to-cache communication using a pretrained projector; multi-sharer support can also be found in this file.

Apply to any LLM: Through the RosettaModel general wrapper, you can integrate your own model with just a few lines of code.

Train the C2C Fuser: Prepare the training configuration in recipe/train_recipe/, specifying the base model, teacher model, projector type and parameters, training hyperparameters, dataset, and output directory; refer to C2C_0.6+0.5.json. During training, only the C2C projector parameters are updated.

Evaluation: Prepare the evaluation configuration in recipe/eval_recipe/, specifying the base model, teacher model, checkpoint directory, generation configuration, evaluation dataset, and output directory; refer to unified_eval.yaml, then run the evaluation.

Code Structure

  • rosetta/: The main package. model/ contains the core model components; train/ contains training utilities; baseline/ contains baseline implementations; utils/ contains utility functions for evaluation and model registration.
  • script/: Experiment scripts. train/ contains SFT_train.py; evaluation/ contains unified_evaluator.py; dataset/ contains dataset preparation scripts; examples/ contains usage examples.
  • recipe/: Configuration files. train_recipe/ contains training configurations; eval_recipe/ contains evaluation configurations.

How to Extend

  • Add a Projector: Add a new projector architecture in rosetta/model/projector.py, responsible for transforming the source model's KV-Cache into the target model's semantic space; key components include the projection network (MLP/concatenation-based), the layer-wise selection gating mechanism, and temperature annealing training. Register it in the configuration via {"projector": {"type": "MyProjector", "params": {...}}}.
  • Add a Dataset: In rosetta/train/dataset_adapters.py, add a new dataset, and use {"data": {"type": "MyDataset"}} in the configuration.
  • Add a Benchmark: Following the pattern of unified_evaluator.py, add evaluation logic in script/evaluation/.

Applicable Scenarios

  • Research on non-text communication between LLMs and KV-Cache semantic fusion.
  • Experimental comparison of the efficiency and accuracy of multi-model collaboration and model ensemble inference.
  • Train and evaluate the C2C projector between your own model pairs.

Limitations and Notes

Multi-sharer support is still at a preliminary stage, and the official team states that it is still under active development. The project's latest updates mention that agent-managed KV-Cache and the corresponding service system will be released in the future. Other details (such as the specific license, pricing, platform requirements, etc.) are not yet provided in the materials.

Related tools

Browse category