What is NavFoM
NavFoM (Navigation Foundation Model) is the world's first cross-embodiment omnidirectional navigation foundation model released by Galaxy General in collaboration with teams from Peking University, the University of Adelaide, Zhejiang University, and others. It has full-scenario support capabilities, can be applied to both indoor and outdoor environments, and can achieve zero-shot operation in unseen scenarios. NavFoM supports multiple navigation tasks, such as natural language instruction-driven target following and autonomous navigation, and can quickly adapt to different embodiments such as robot dogs, wheeled humanoid robots, drones, and cars. Its core technologies include TVI Tokens and the BATS strategy, establishing a new general paradigm: 'video stream + text instruction -> action trajectory,' completing the entire navigation process end-to-end.
Main features of NavFoM
-
Full-scenario support: NavFoM can support both indoor and outdoor scenarios and can achieve zero-shot operation in unseen environments without additional mapping or data collection, demonstrating strong environmental adaptability.
-
Multi-task support: The model supports different subdivided navigation tasks such as natural language instruction-driven target following and autonomous navigation, and can complete corresponding navigation actions according to different instructions.
-
Cross-embodiment adaptation: NavFoM can quickly and cost-effectively adapt to heterogeneous embodiments of different sizes, such as robot dogs, wheeled humanoids, legged humanoids, drones, and cars, giving it broad applicability.
-
Technical innovation: NavFoM uses TVI Tokens (Temporal-Viewpoint-Indexed Tokens) to enable the model to understand time and direction, as well as the BATS strategy (Budget-Aware Token Sampling) to keep the model smart even under limited computing power. Through these technical innovations, the model's performance is improved.
-
Unified paradigm: NavFoM establishes a new general paradigm: 'video stream + text instruction -> action trajectory.' It no longer relies on modular assembly, but instead completes the entire process of 'see - understand - act' end-to-end, simplifying the navigation workflow.
-
Dataset construction: NavFoM has built a massive cross-task dataset, containing about eight million cross-task, cross-embodiment navigation data entries, as well as four million open question-answering data entries, providing rich data support for model training.
Technical principles of NavFoM
-
TVI Tokens（Temporal-Viewpoint-Indexed Tokens）: Through tokens indexed by time and viewpoint, the model can understand time and direction, thereby better handling navigation tasks in dynamic environments.
-
BATS strategy (Budget-Aware Token Sampling): Under limited computing power, a budget-aware token sampling strategy ensures that the model can still operate efficiently, improving its feasibility in practical applications.
-
End-to-end general paradigm: It adopts the paradigm of 'video stream + text instruction -> action trajectory,' integrating visual input, language instructions, and action output into a unified framework to achieve direct mapping from perception to action.
-
Cross-task dataset: A massive cross-task dataset containing about eight million navigation data entries and four million open question-answering data entries has been constructed, providing rich multi-scenario, multi-task data support for model training and improving the model's generalization ability.
NavFoM project address
No related address has been announced yet
Application scenarios of NavFoM
-
Robot navigation: In complex environments such as shopping malls, airports, and other public places, robots can autonomously navigate and follow targets based on natural language instructions, achieving efficient service and guidance functions.
-
Autonomous Driving: Applied to automotive autonomous driving systems, it enhances the vehicle's autonomous decision-making and navigation capabilities under complex road conditions, and improves the safety and reliability of autonomous driving.
-
Drone Navigation: It provides drones with autonomous navigation capabilities, enabling them to perform autonomous flight and task execution in complex terrain and environments, such as logistics delivery and environmental monitoring.
-
Humanoid Robots: It supports humanoid robots of different forms, such as wheeled humanoids and legged humanoids, enabling them to better adapt to various environments and complete complex navigation and interaction tasks.
-
Develop Application Models: Developers can use NavFoM as a foundation and further develop application models that meet specific navigation requirements through post-training, expanding its range of applications in different fields.