github.com/TIERS/tiers-lidars-dataset-enhanced @main

A Benchmark for Multi-Modal Lidar SLAM with Ground Truth in GNSS-Denied Environments

Welcome to TIERS Enhanced lidars dataset! You can check our preprint paper here for more detatil.

(Left)Ground truth map for one of the indoor sequences generated based on the proposed approach (SLAM-assisted ICP-based prior map). This enables benchmarking of lidar odometry and mapping algorithms in larger environments where a motion capture system or similar is not available, with significantly higher accuracy than GNSS/RTK solutions.

OS0	OS1	Horizon	Avia	Velo

Our dataset was captured by a rich suite of sensors. Subsets of the data from the Indoor10 sequence are visualized here. From left to right are the lidar data of Ouster OS0, OusterOS1, Livox Horizon, Livox Avia, and Velodyne Lidar.

The above images show the reflection map (top) and the range map (bottom) of Ouster OS0, respectively. The left one in the middle is the image data of L515, and the one on the right is the fisheye image data of T265.

Indoor data(Calibrate Sequence)	OpenRoad SLAM example(Road03 Sequence)
Corridor SLAM example(Indoor10 Sequence)	Forest SLAM example(Forest01 Sequence)

The dataset is available at the University of Turku servers. Specific links for each sequence and for the ground truth data are available in Section 3.1 of this file.

ABSTRACT:

Lidar-based simultaneous localization and mapping (SLAM) approaches have obtained considerable success in autonomous robotic systems. This is in part owing to the high-accuracy of robust SLAM algorithms and the emergence of new and lower-cost lidar products. This study benchmarks current state-of-the-art lidar SLAM algorithms with a multi-modal lidar sensor setup showcasing diverse scanning modalities (spinning and solid-state) and sensing technologies, and lidar cameras, mounted on a mobile sensing and computing platform. We extend our previous multi-modal multi-lidar dataset with additional sequences and new sources of ground truth data. Specifically, we propose a new multi-modal multi-lidar SLAM-assisted and ICP-based sensor fusion method for generating ground truth maps. With these maps, we then match real-time pointcloud data using a natural distribution transform (NDT) method to obtain the ground truth with full 6 DOF pose estimation. This novel ground truth data leverages high-resolution spinning and solid-state lidars. We also include new open road sequences with GNSS-RTK data and additional indoor sequences with motion capture (MOCAP) ground truth, complementing the previous forest sequences with MOCAP data. We perform an analysis of the positioning accuracy achieved with ten different SLAM algorithm and lidar combinations. We also report the resource utilization in four different computational platforms and a total of five settings (Intel and Jetson ARM CPUs). Our experimental results show that current state-of-the-art lidar SLAM algorithms perform very differently for different types of sensors.

Keywords: Lidar, Dataset, Multi-modal, Multi-scenario, SLAM, Solid-state lidarsAutonomous driving, LiDAR SLAM benchmark solid-state LiDAR, SLAM

MAIN CONTRIBUTIONS:

• a ground truth trajectory generation method for environments where MOCAP or GNSS/RTK are unavailable that leverages the multi-modality of the data acquisition platform and high-resolution sensors;% by solid-state lidar with a Non-repetitive scanning pattern and high-resolution spinning lidar.

• a new dataset with data from 5 different lidar sensors, one lidar camera, and one stereo fisheye cameras in a variety of environments as illustrated in here . Ground truth data is provided for all sequences;

• the benchmarking of ten state-of-the-art filter-based and optimization-based SLAM methods on our proposed dataset in terms of the accuracy of odometry, memory and computing resource consumption. The results indicate the limitations of current SLAM algorithms and potential future research directions.

Updates

2022.09.20 Initial dataset upload \ 2022.09.20 Update rosbag links

1. LICENSE

This work is licensed under the MIT license and is provided for academic purpose. Please contact us at sierha@utu.fi or qingqli@utu.fi for further information.

2. SENSOR SETUP

2.1 Data acquisition platform

Physical drawings and schematics of the sensor suite is given below. The unit of the figures is centimeter.

Our data collecting platform, front view RGB (left).

2.2 Sensor parameters

Sensor specification for the presented dataset. Angular resolution is configurable in the OS1-64 (varying the vertical FoV). Livox lidars have a non-repetitive scan pattern that delivers higher angular resolution with longer integration times. Range is based on manufacturerinformation, with values corresponding to 80% Lambertian reflectivity and 100 klx sunlight, except for the L515 and Realsense T265 lidar camera.

2.3 ROS topics

2.3.1 Dataset based on NDT method

The rostopics of our rosbag sequences are listed as follows:

• VLP-16 LIDAR : \ /velodyne_points sensor_msgs/PointCloud2

• OS0 LIDAR : \ /os0_cloud_node/imu : sensor_msgs/Imu
/os0_cloud_node/points : sensor_msgs/PointCloud2 /os0_img_node/nearir_image : sensor_msgs/Image
/os0_img_node/range_image : sensor_msgs/Image
/os0_img_node/reflec_image : sensor_msgs/Image

• OS1 LIDAR : \

/os1_cloud_node/imu : sensor_msgs/Imu
/os1_cloud_node/points : sensor_msgs/PointCloud2

• Horizon LIDAR : \ /horizon/livox/imu : sensor_msgs/Imu /horizon/livox/lidar : livox_ros_driver/CustomMsg

• AVIA LIDAR : \ /avia/livox/imu : sensor_msgs/Imu
/avia/livox/lidar : livox_ros_driver/CustomMsg

• L515 LIDAR CAMERA: \ /l515/accel/sample : sensor_msgs/Imu
/l515/color/image_raw : sensor_msgs/Image
/l515/depth/color/points : sensor_msgs/PointCloud2
/l515/depth/image_rect_raw : sensor_msgs/Image
/l515/gyro/sample : sensor_msgs/Imu

• T265 LIDAR CAMERA: \ /t265/accel/sample : sensor_msgs/Imu
/t265/fisheye1/image_raw : sensor_msgs/Image
/t265/fisheye2/image_raw : sensor_msgs/Image
/t265/gyro/sample : sensor_msgs/Imu
/t265/odom/sample : nav_msgs/Odometry

2.3.2 Dataset based on MOCAP or GNSS/RTK

The rostopics of our rosbag sequences are listed as follows:

• VLP-16 LIDAR : \ /velodyne_points sensor_msgs/PointCloud2

• OS0 LIDAR : \ /os0_cloud_node/imu : sensor_msgs/Imu
/os0_cloud_node/points : sensor_msgs/PointCloud2 /os0_img_node/nearir_image : sensor_msgs/Image
/os0_img_node/range_image : sensor_msgs/Image
/os0_img_node/reflec_image : sensor_msgs/Image

• OS1 LIDAR : \

/os1_cloud_node/imu : sensor_msgs/Imu
/os1_cloud_node/points : sensor_msgs/PointCloud2

• Horizon LIDAR : \ /horizon/livox/imu : sensor_msgs/Imu /horizon/livox/lidar : livox_ros_driver/CustomMsg

• AVIA LIDAR : \ /avia/livox/imu : sensor_msgs/Imu
/avia/livox/lidar : livox_ros_driver/CustomMsg

• L515 LIDAR CAMERA: \ /l515/accel/sample : sensor_msgs/Imu
/l515/color/image_raw : sensor_msgs/Image
/l515/depth/color/points : sensor_msgs/PointCloud2
/l515/depth/image_rect_raw : sensor_msgs/Image
/l515/gyro/sample : sensor_msgs/Imu

• T265 LIDAR CAMERA: \ /t265/accel/sample : sensor_msgs/Imu
/t265/fisheye1/image_raw : sensor_msgs/Image
/t265/fisheye2/image_raw : sensor_msgs/Image
/t265/gyro/sample : sensor_msgs/Imu
/t265/odom/sample : nav_msgs/Odometry

• MOCAP SYSTEM: /vrpn_client_node/optitest/pose : geometry_msgs/PoseStamped

• GNSS/RTK: \

  /mavros/global_position/global : sensor_msgs/NavSatFix /mavros/global_position/local : nav_msgs/Odometry /mavros/global_position/raw/fix : sensor_msgs/NavSatFix /mavros/global_position/raw/gps_vel : geometry_msgs/TwistStamped /mavros/global_position/raw/satellites : std_msgs/UInt32 /mavros/imu/data : sensor_msgs/Imu /mavros/imu/mag : sensor_msgs/MagneticField /mavros/local_position/odom : nav_msgs/Odometry

3. DATASET SEQUENCES

List of data sequences in our dataset

3.1 Main dataset

Sequence Name	Collection Date	Total Size	Duration	Features	Rosbag	GroundTruth
Indoor06	2022-08-10	19.7g	64s	day,indoor,office	Rosbag	SLAM+ICP link
Indoor07	2022-07-16	22.4g	73s	day,indoor,office	Rosbag	SLAM+ICP link
Indoor08	2022-07-05	33.1g	89s	day,indoor,office	Rosbag	SLAM+ICP link
Indoor09	2022-07-11	48.7g	168s	day,indoor,corridor	Rosbag	SLAM+ICP link
Indoor10	2022-06-07	43.0g	121s	day,indoor,corridor	Rosbag	SLAM+ICP link
Indoor11	2021-06-06	79.5g	237s	day,indoor,hall	Rosbag	SLAM+ICP link
Road3	2022-06-17	44.0g	146s	summer,outdoor,road	Rosbag	GNSS/RTK link
Forest01	2022-02-08	21.9g	62s	Winter,night,Square	Rosbag	MOCAP link
Forest02	2022-02-08	22.4g	73s	Winter,night,Straight	Rosbag	MOCAP link

• There are also previous datasets, please check here

3.2 Ground Truth:

The meaning of each column in ground truth files is as follows:

timestamp, pose.position.x,  pose.position.y,  pose.position.z, pose.orientation.x, pose.orientation.y, pose.orientation.z, pose.orientation.w. 

In the MOCAP system available environment, the ground truth data from the MOCAP system are recorded in rosbag. The user can generate the ground truth file by himself. A Script named 'result_sub_ros.py' is provided in the scripts folder to record the result and save it into a CSV file.

python2 result_sub_ros.py

3.3 Other data

Sequence Name	Collection Date	Total Size	Duration	Features	Rosbag
LidarsCali	2022-02-11	21.9g	19.1s	room	Rosbag

3.4 Time-offset of dataset

The graph shows the change in the yaw value of the IMU of each lidar in the dataset. It can be seen from the picture that the average time offset of the dataset does not exceed 5ms.

4. SLAM RESULTS

We teseted some well-known lidar SLAM methods, which are listed below:

• LeGO-LOAM

• FAST_LIO

• LIO_LIVOX

• [LILI-OM](https://github.com/KI