LingBot-Map - Ant Lingbo's Open-Source Streaming 3D Reconstruction Model

LingBotWhat is -Map

LingBot-Map is an open-source streaming 3D reconstruction model from Ant Lingbo Technology. An ordinary RGB camera can perform real-time camera pose estimation and 3D scene structure reconstruction during video capture. The model is centered on pure autoregressive modeling and is based on a Geometric Context Transformer architecture. On authoritative benchmarks such as Oxford Spires, it achieves trajectory accuracy about 2.8 times higher than the previous state-of-the-art streaming methods, filling a key technical gap in the field of real-time spatial perception.

LingBotMain Features of -Map

• Real-time streaming reconstruction: Understand while viewing, process the current and historical frames frame by frame, and continuously output camera pose and depth information.
• Stable operation over long sequences: Supports continuous inference on long videos of 10,000+ frames, with almost no degradation in accuracy during long-sequence operation.
• Pure visual spatial perception: No complex hardware is required; a single ordinary camera can achieve real-time 3D mapping.
• Camera trajectory estimation: Supports accurate estimation of the camera's motion trajectory in 3D space (pose estimation).

How to use LingBot-Map

• Environment preparation: Ensure the local environment is equipped with an NVIDIA GPU (recommended VRAM ≥ 12GB), and install Python 3.8+, PyTorch 2.0+, and the CUDA toolchain. The system needs to support Linux or Windows WSL2 environments to be compatible with dependency library compilation.
• Installation and deployment: Visit the GitHub repository https://github.com/Robbyant/lingbot-map to clone the code, then enter the project directory and install dependencies via pip:pip install -r requirements.txt. The environment will automatically install the DINO backbone, Transformer architecture, and libraries such as Open3D required for 3D visualization.
• Model acquisition: Download the pretrained weights from HuggingFace (robbyant/lingbot-map) or ModelScope, and place them in the project checkpoints/ directory. The model includes the Geometric Context Attention (GCA) core network and the Camera/Depth prediction heads.
• Run inference:
  • Offline video mode: Input a monocular RGB video file. The model extracts DINO features frame by frame, processes them through Frame Attention and GCA layers, outputs the camera pose trajectory (Trajectory) and depth maps (Depth), and finally fuses them into a 3D point cloud map.
  • Real-time camera mode: Connect an ordinary USB camera, set the input stream resolution to 640×480 or 1280×720, and the model performs real-time inference at ~20 FPS, continuously outputting the current camera pose and incrementally updating the 3D scene structure.
• Result output: The reconstruction results include a camera trajectory file (standard coordinate format) and a dense point cloud with scale. The 3D reconstruction effect can be viewed through visualization scripts. During long-sequence inference, the GCA mechanism automatically manages memory, supporting continuous processing of 10,000+ frames without restarting.
• Advanced configuration: Adjust the config.yaml in anchor_context and trajectory_memory parameters to balance accuracy and computational cost; for large scenes, enable local window optimization (Local Pose-Reference Window) to improve long-trajectory stability. For specific API call examples and parameter descriptions, refer to the GitHub repository's README.md and demo.py。

LingBotKey information and usage requirements of -Map

• Development team: Ant Lingbo Technology (Robbyant)
• Open-source license: Model weights and code have been open-sourced (GitHub, HuggingFace, ModelScope)
• Hardware Requirements: Requires GPU support (an environment with sufficient VRAM is recommended)
• Inference Speed: Approximately 20 FPS real-time inference

LingBot-Map's Core Advantages

• Leading Accuracy: On the Oxford Spires dataset, the Absolute Trajectory Error (ATE) is only 6.42 meters, significantly outperforming the offline methods DA3 (12.87 meters) and VIPE (10.52 meters); on the ETH3D benchmark, the reconstruction F1 score reaches 85.70, an improvement of over 8% compared with the second place.
• Real-Time and Efficient: 20 FPS stably supports robotic operations, and the computational and storage overhead remains almost constant over long sequences.
• Low Hardware Barrier: No depth camera or LiDAR is required; it can run with an ordinary RGB camera.
• End-to-End Learning: It breaks through the limitations of traditional SLAM, which relies on handcrafted design and complex optimization, with the core logic uniformly learned by the model.

LingBot-Map's Project Address

• Project Website：https://technology.robbyant.com/lingbot-map
• GitHub Repository：https://github.com/Robbyant/lingbot-map
• HuggingFace Model Library：https://huggingface.co/robbyant/lingbot-map
• arXiv Technical Paper：https://arxiv.org/pdf/2604.14141

LingBot-Map's Comparison with Similar Competitors

LingBot-Map's Application Scenarios

• Robot Navigation and Obstacle Avoidance: Provides mobile robots with real-time spatial perception capabilities and supports continuous online processing.
• Autonomous Driving: Builds a three-dimensional map of the environment in real time, helping vehicles understand the physical spatial structure.
• Embodied Intelligence: Serves as the spatial perception foundation for the LingBot series (including models such as Depth, VLA, World, and VA), supporting robotic operation and interaction.
• AR/VR Spatial Computing: Reconstructs the physical environment in real time, enabling precise alignment between the virtual and real worlds.