KITTI, whose full name is the KITTI Vision Benchmark Suite, is a computer vision algorithm evaluation dataset and benchmark platform for autonomous driving scenarios, jointly developed by the Karlsruhe Institute of Technology (KIT) and the Toyota Technological Institute at Chicago (TTI-C). The data collection platform is based on the autonomous driving vehicle Annieway, equipped with two high-resolution color and grayscale cameras, and precise ground truth is provided by a Velodyne laser scanner and GPS positioning system. The collection scenes cover the urban area of Karlsruhe, rural roads, and highways, with up to about 15 vehicles and 30 pedestrians visible in each image, and include varying degrees of occlusion and truncation.
The main tasks and benchmarks include: stereo vision (Stereo 2012/2015), optical flow (Flow 2012/2015), depth completion and depth prediction, 2D/3D object detection and bird's-eye-view detection, multi-object tracking, multi-object tracking and segmentation (MOTS), pixel-wise segmentation and tracking, semantic segmentation (pixel-level and instance-level), visual odometry, and road detection, among others. The platform provides evaluation metrics and an online evaluation website for each benchmark, and allows results to be submitted for ranking.
How to use: users can download the raw data, benchmark data for each task, and the development kit (devkit) from the official website, train and test algorithms locally, and then submit results through the evaluation website. There are frequency limits on submissions, currently at most 3 times per month, counted separately for different benchmarks.
License and restrictions: all datasets and benchmarks use the Creative Commons Attribution-NonCommercial-ShareAlike 3.0 license and are for academic use only, and may not be used for commercial purposes; when using them, attribution and citation are required as requested by the authors, and if you modify or build works based on this data, you must distribute them under the same license. The official website also provides a privacy policy; if an individual or object appears in the data and feels uncomfortable about it, they can contact the site for removal.
Citation requirements: when using the stereo 2012, flow 2012, odometry, object detection, or tracking benchmarks, the Geiger et al. 2012 CVPR paper must be cited; when using the raw dataset, the Geiger et al. 2013 IJRR paper must be cited; for the road benchmark, the Fritsch et al. 2013 ITSC paper must be cited; for the stereo 2015, flow 2015, and scene flow 2015 benchmarks, the Menze and Geiger 2015 CVPR paper must be cited.
The platform is continuously updated, for example in 2In February 2021, tracking and MOTS evaluation switched to the HOTA metric and the TrackEval codebase; in December 2019, the MOTS benchmark was added; in March 2018, semantic segmentation and instance segmentation benchmarks were added; in December 2017, depth completion and single-image depth prediction benchmarks were added; and in July 2017, 3D object detection and bird's-eye-view evaluation were added.
