Yang, Chenhao (University of Tübingen), Liu, Yuyi (Kyoto University), Zell, Andreas (University of Tübingen)

Learning-Based Camera Relocalization with Domain Adaptation Via Image-To-Image Translation

Scheduled for presentation during the Regular Session "UAS Applications IV" (FrA4), Friday, June 18, 2021, 09:00−09:20, Naoussa

2021 International Conference on Unmanned Aircraft Systems (ICUAS), June 15-18, 2021, Athens, Greece

This information is tentative and subject to change. Compiled on April 24, 2024

Abstract

Camera relocalization is promising for robot navigation; however, the lack of sufficient reference images and challenging shifts of conditions limit its further application. This paper proposes a learning-based three-step pipeline that applies spatial information features for camera relocalization problems. We first introduce a frustum intersection over union (IoU) to represent the image pair's spatial similarity. This representation is used to train an image retrieval model to find nearest neighbor candidates for query images. A spatial sample consensus (SPASAC) operation is then deployed to filter the outliers in the candidates. Afterward, a relative camera pose regressor is used with the valid candidates to predict every query image's absolute pose. Besides, we introduce two implementations of image-to-image translation networks for camera relocalization to increase the number of synthetic reference images and challenging night-to-day localization performance. Experiments show that our method can estimate camera poses across different domains and outperforms related methods in four benchmarks. Specifically, the experiments on the Tuebingen Buildings dataset demonstrate the robustness of our approach when localizing UAV-captured images with high-speed movement and large viewpoint variation.