Deep learning for simultaneous localization and mapping
Abstract
In this project we are using machine learning for multi-modal localization problems. Simultaneous localization and mapping is important in several industrial applications, for example autonomous robot navigation, consumer electronics and augmented reality. In particular we will investigate how deep neural networks can be used for end-to-end robot localization using sound recordings and possibly other data modalities. Given sound recordings from several microphones and their ground truth position, we train a model to estimate the position of the sound source. Doing this requires training large transformer models on large datasets. Here we will first consider the LuViRA dataset (https://arxiv.org/abs/2302.05309) which consists of ground truth robot trajectories recorded in a motion capture lab. The task consists of localizing a robot given audio recordings from microphones in the room. As a continuation of the project, we will also consider incorporation the other data modalities in the dataset, which are MIMO radio signals and depth camera images. Our initial investigation on a small part of the dataset has showed that it is possible to obtain good localization performance using only a small fraction of the audio data. Using the entire dataset, we expect that we can train a state-of-the-art acoustic localization model, but this will require more computational resources.
