SuperAPI AI Directory
Partner with us中文AI newsFavorites
DirectoryNews and tutorials

Game-TARS - A general-purpose game agent launched by ByteDance

Author: AI小集 · Original source: AI工具集

What is Game-TARS

Game-TARS is a general-purpose game agent developed by ByteDance's Seed team. It is trained on a unified keyboard-mouse action space and can undergo large-scale pretraining in operating systems, web pages, and simulated environments. Leveraging multimodal training data on the scale of over 500 billion annotations, combined with sparse reasoning and decayed continuous loss, it greatly improves the agent's scalability and generalization. The core innovation of Game-TARS lies in enabling the agent to operate with a keyboard and mouse like a human, completing every action in a game by simulating human operations, thereby achieving direct alignment with the physical interaction methods of human users. In tasks such as FPS, open-world, and WEB games, its performance surpassesGPT-5、Gemini-2.5-Pro、Claude-4-Sonnet。

Game-TARS

Main features of Game-TARS

  • Cross-platform game operation: Game-TARS can operate games on different platforms (such as PC, web pages, and simulated environments) through a unified keyboard-mouse action space, without the need to write specific scripts for each platform, enabling cross-platform automated testing and game interaction.
  • Multimodal data pretraining: Pretraining is based on multimodal data on the scale of over 500 billion annotations, covering various data types such as game trajectories and graphical user interface interactions, enabling the agent to have strong generalization ability and adaptability and to handle various complex game tasks.
  • Efficient reasoning and sparse thinking: It adopts a sparse reasoning strategy, performing deep reasoning only at key decision points, and combines rejection fine-tuning technology to optimize the reasoning process, improve reasoning efficiency and the operability of actions, and enable the agent to make more precise decisions in complex environments.
  • Combination of long-term and short-term memory: It introduces a dual-layer memory mechanism. Short-term memory preserves the latest image information, while long-term memory retains refined sparse thinking text, helping the agent maintain memory of key information in long-term tasks and improving task completion rates and interaction intelligence.
  • Zero-shot transfer capability: After large-scale pretraining, Game-TARS can achieve zero-shot transfer in unseen 3D web games, quickly adapting to new environments and completing tasks without additional training, demonstrating strong generalization ability.
  • Instruction following and action semantic understanding: By randomly replacing key bindings and other methods, it strengthens the agent's ability to follow instructions, enabling it to understand action semantics based on system prompts, accurately execute task instructions, and improve adaptability and operational accuracy in different game environments.

Technical principles of Game-TARS

  • Unified action space: It adopts native keyboard-mouse input actions aligned with humans, such asmouseMove、mouseClickandkeyPressetc., decoupling the agent's action instruction set from specific applications or operating systems and achieving cross-platform universality.
  • Multimodal data pretraining: Pretraining is based on multimodal data with over 500 billion tokens, including various data types such as game trajectories, graphical user interface interactions, code generation, and scientific research tasks, improving the agent's generalization ability and adaptability.
  • Sparse reasoning strategy: Through Sparse-Thinking technology, deep reasoning is performed only at key decision points, and combined with Rejection Fine-Tuning, the reasoning process is optimized, improving reasoning efficiency and the operability of actions.
  • Integration of vision-language models: Integrates visual perception, strategic reasoning, action execution, and long-term memory into a single vision-language model (VLM), eliminating the need to write game-specific code or rules, enabling the agent to autonomously learn operations and complete tasks.
  • Continuous Pretraining Framework: Adopts a unified single-stage continuous pretraining approach that fuses all data sources together to pretrain the model at scale, while further strengthening the agent's execution capabilities and interactive intelligence in specific tasks during the post-training stage.
  • Long-Term and Short-Term Memory Mechanism: Introduces a dual-layer memory mechanism, where short-term memory preserves the latest image information and long-term memory retains refined sparse thought text, helping the agent maintain memory of key information in long-horizon tasks and improving task completion rates and interactive intelligence.

Game-TARS Project Address

  • Project Website：https://seed-tars.com/game-tars/
  • arXiv Technical Paper：https://arxiv.org/pdf/2510.23691

Game-TARS Application Scenarios

  • Game Automation Testing: Game-TARS can be used for automated testing of various games, including FPS (first-person shooter) games, open-world games, web games, etc., helping developers quickly discover issues in games and improve testing efficiency and quality.
  • Cross-Platform Compatibility Testing: Due to its unified action space design, Game-TARS can perform compatibility testing on different platforms (such as PC, mobile, and web), ensuring game compatibility and consistency across multiple platforms.
  • Complex Task Execution: Game-TARS can handle complex game tasks, such as completing construction and exploration tasks in Minecraft, and even achieve zero-shot transfer in unseen 3D web games, demonstrating strong generalization ability.
  • Anomalous Scenario Handling: Game-TARS has an error recovery mechanism and can handle anomalous scenarios in games through self-supervised learning from failure cases, improving the agent's robustness.
  • Game Development Assistance: In addition to testing, Game-TARS can also assist game development, for example by providing interactive prototype testing during the game design stage, helping developers optimize game design and user experience.
  • Research and Education: The open-source nature of Game-TARS makes it a valuable resource in the fields of research and education. Researchers can use its framework for research in related fields such as artificial intelligence and machine learning, and educational institutions can also use it as a teaching case to cultivate students' practical abilities.