What is Gemma 2
Gemma 2 is a new generation open-source AI model released by Google DeepMind, including 9 billion and 27 billion parameter versions. The model is characterized by excellent performance, efficient inference speed, and broad hardware compatibility, and can compete with models with larger parameter counts (such as Llama 3, Claude 3 Sonnet). Gemma 2 is designed to be easily integrated into developers' workflows, supports multiple AI frameworks, and is provided free of charge through platforms such as Google Cloud, Kaggle, and Hugging Face.
Features of Gemma 2
- Parameter scale: Gemma 2 currently provides two parameter scale models, namely 9 billion (9B) and 27 billion (27B) parameter versions, to adapt to different application needs and resource constraints. A 2.6 billion parameter (2.6B) model will be released later.
- Performance optimization: The 27B version of Gemma 2 can match the performance of models with more than twice its parameter count, showing an extremely high performance-to-efficiency ratio. In the LMSYS Chatbot Arena, the 27 billion parameter Gemma 2 instruction-tuned model defeated the 70 billion parameter Llama 3, and surpassed models such as Nemotron 4 340B, Claude 3 Sonnet, Command R+, and Qwen 72B, ranking first among all open-weight models.
- Inference efficiency: Gemma 2 has specially optimized the inference process and can run at full precision on a single high-end GPU or TPU without additional hardware resources, thereby greatly reducing usage costs.
- Hardware compatibility: Gemma 2 can run quickly on a variety of hardware platforms, including personal computers, workstations, gaming laptops, and cloud servers.
- Open license: Gemma 2 uses a commercially friendly license agreement, allowing developers and researchers to freely share, use, and commercialize their applications.
- Framework support: Gemma 2 is compatible with multiple mainstream AI frameworks, including Hugging Face Transformers, JAX, PyTorch, and TensorFlow, enabling developers to choose suitable tools according to their preferences.
- Deployment tools: Google provides Gemma Cookbook, a resource library containing practical examples and guides to help users build applications and fine-tune Gemma 2 models.
- Responsible AI: Google provides a series of tools and resources, such as the Responsible Generative AI Toolkit and LLM Comparator, to support developers and researchers in building and deploying AI responsibly.
How to use Gemma 2
Gemma 2 can easily work with the tools and workflows users commonly use, and is compatible withHugging Face mainstream AI frameworks such as Transformers, JAX, PyTorch, and TensorFlow, and can be implemented through native Keras 3.0, vLLM,Gemma.cpp、Llama.cppandOllamaIn addition, Gemma has also been optimized throughNVIDIA TensorRT-LLMto run on NVIDIA accelerated infrastructure or as anNVIDIA NIMinference microservice, and will be optimized forNVIDIA's NeMo.
Gemma 2 is now available inGoogle AI Studio, and users can test its full performance at 27B speed without hardware requirements. Developers can also download the model weights of Gemma 2 fromKaggleandHugging Face Models,Vertex AI Model Gardencoming soon.
To facilitate research and development, Gemma 2 is also available for free throughKaggleor Colab notebooks. First-time Google Cloud customers are eligible to receive$300 in credit. Academic researchers can apply forthe Gemma 2 Academic Research Programto receive Google Cloud credits to accelerate their research using Gemma 2.Applicationsare open from now until August 9.