recurrentgemma
AI developer platformsRecurrentGemma is a series of open-weight language models released by Google DeepMind, based on the novel Griffin architecture. This architecture replaces global attention with a hybrid mechanism of local attention and linear recurrence, thereby achieving fast inference when generating long sequences. This repository contains the model implementation as well as example code for sampling and fine-tuning. The official recommendation is for most users to use the highly optimized Flax implementation, while an unoptimized PyTorch reference implementation is also provided. Model checkpoints can be downloaded via Kaggle and include weights and tokenizer. The code supports running on CPU, GPU, or TPU; the Flax implementation is optimized for TPU and includes low-level Pallas kernels for linear scans of recurrent layers. The project uses Poetry to manage dependencies and provides Colab tutorials covering sampling and fine-tuning.
