Scene understanding is a pivotal task for autonomous vehicles to safely navigate in the environment. Recent advances in deep learning enable accurate semantic reconstruction of the surroundings from LiDAR data. However, these models encounter a large domain gap while deploying them on vehicles equipped with different LiDAR setups which drastically decreases their performance. Fine-tuning the model for every new setup is infeasible due to the expensive and cumbersome process of recording and manually labeling new data. Unsupervised Domain Adaptation (UDA) techniques are thus essential to fill this domain gap and retain the performance of models on new sensor setups without the need for additional data labeling. In this paper, we propose AdaptLPS, a novel UDA approach for LiDAR panoptic segmentation that leverages task-specific knowledge and accounts for variation in the number of scan lines, mounting position, intensity distribution, and environmental conditions. We tackle the UDA task by employing two complementary domain adaptation strategies, data-based and model-based. While data-based adaptations reduce the domain gap by processing the raw LiDAR scans to resemble the scans in the target domain, model-based techniques guide the network in extracting features that are representative for both domains. Extensive evaluations on three pairs of real-world autonomous driving datasets demonstrate that AdaptLPS outperforms existing UDA approaches by up to 6.41 pp in terms of the PQ score.
Our approach comprises two domain adaptation strategies, namely, data-based and model-based. The data-based domain adaptation minimizes the domain gap between the raw LiDAR scans from the source and target domains using three techniques, namely, (i) pose correction which accounts for the different mounting positions of LiDARs, (ii) virtual scan generation which simulates the point cloud from the target domain using the source domain, and (iii) intensity mapping which learns a residual to map the LiDAR intensities between the source and target domains. In contrast, model-based domain adaptation minimizes the domain gap between the panoptic segmentation models by aligning the source and target distributions using our novel multi-scale feature-space optimal transport augmented with instance-aware sampling. During inference, we further reduce the domain gap using our efficient PDC-Lite technique which re-calibrates the first batch normalization layer to remove any bias and variance accumulated from the source domain.
Using extensive evaluations on three different dataset pairs, we demonstrate that our AdaptLPS strategy consistently outperforms existing domain adaptation techniques on both panoptic and semantic segmentation tasks. Lastly, we present a detailed ablation study along with a comprehensive qualitative analysis that highlights the improvement brought about by the different constituent elements of our domain adaption strategy.
This work was funded by the Eva Mayr-Stihl Stiftung and the Federal Ministry of Education and Research (BMBF) of Germany under ISA 4.0.