SuperAPI AI Directory
Partner with us中文AI newsFavorites
DirectoryNews and tutorials

GameGen-O - Tencent's game video generation model that automatically generates characters, scenes, actions, and events

Author: AI小集 · Original source: AI工具集

What is GameGen-O

GameGen-O is a game video generation model based on the Transformer architecture launched by Tencent, specifically designed to generate open-world video games. The model can simulate multiple functions of a game engine, including generating game characters, dynamic environments, complex actions, etc., supports interactive control, and allows users to control game content through text, operation signals, and video prompts. The development of GameGen-O involves large-scale data collection and processing, creating the first open-world video game dataset (OGameData), and going through a two-stage training process, including foundation model pre-training and instruction tuning. The launch of the model will have a significant impact on the game development industry, as it can reduce development costs and time while providing players with more creative freedom.

Main Features of GameGen-O

  • Character Generation: Can generate various characters based on user text instructions, such as cowboys, astronauts, mages, etc.
  • Environment Generation: Can create dynamic game environments that adapt to different game styles and scenes.
  • Action Generation: Supports the generation of complex character actions, such as driving, flying, shooting, etc.
  • Event Generation: Can generate various events in the game, such as weather changes, natural disasters, etc.
  • Interactive Control: Users can control game content through text, operation signals, and video prompts, achieving an interactive gaming experience.

Technical Principles of GameGen-O

  • Open-Domain Generation: GameGen-O can generate various types of game elements, such as characters, environments, actions, and events, expanding the possibilities of games.
  • Interactive Controllability: The model can generate game content and supports interactive control by users through the InstructNet branch, such as changing character behavior, environment layout, event occurrence, etc.
  • OGameData Dataset: To train GameGen-O, the research team built a large-scale open-world video game dataset OGameData, containing over 4,000 hours of video clips from more than 150 games, covering various game types and styles.
  • Two-Stage Training: The model adopts a two-stage training strategy. The first stage is foundation model pre-training, learning to generate high-quality game videos; the second stage is instruction fine-tuning, giving the model the ability to generate and control content according to user instructions.
  • Technical Innovation: GameGen-O adopts multiple advanced technologies, such as 2+1D VAE video compression, hybrid training strategy, masked attention mechanism, etc., ensuring the stability of the model and generation quality.
  • Dataset Construction and Training Process: 32,000 raw videos were collected from the internet, and after screening by human experts and annotation by GPT-4o, high-quality training data was formed. In the foundation training stage, the model used a variational autoencoder to compress video clips and adopted a hybrid training strategy with different frame rates and resolutions.
  • InstructNet: In the model fine-tuning stage, a trainable InstructNet was used to accept multimodal inputs, including text, operation signals, and video prompts, achieving interactive control over the generated content.

GameGen-O Project Address

GameGen-O Application Scenarios

  • Game Prototyping: Developers can use GameGen-O to quickly create game prototypes, test different game elements, and save the time and effort of building a game from scratch.
  • Environment and Scene Generation: GameGen-O can create dynamic game environments and complex scenes, adding rich visual effects to the game world.
  • Action and Event Generation: The model supports the generation of complex character actions and various events in games, such as tsunamis, tornadoes, fires, etc., increasing the interactivity and challenge of the game.
  • Assisted Game Development: GameGen-O can assist in the game development process, replacing part of the development work through AI models and improving development efficiency.
  • Research and Education: For researchers and educators, GameGen-O can serve as a research tool to help explore fields such as video game AI development, interactive control, and immersive virtual environments.