Abstract:

Accurate location information obtained by mobile agents is the key to building a stable mixed reality (MR) system. However, foreground objects in an MR scene have a significant impact on the accuracy of traditional location algorithms. At present, location algorithms based on deep learning show relatively improved accuracy by identifying foreground objects, but the time consumption of a deep learning model is too high, resulting in a decline in the real-time performance of the algorithms. To solve this problem, this paper proposes a foreground object-aware location algorithm based on an MR semantic feature propagation model. The algorithm builds a semantic feature propagation model based on a semantic segmentation network and the oriented FAST and rotated BRIEF feature extraction algorithm to realize high-speed semantic feature extraction. The model and a geometric feature detection method are fused to realize the foreground object perception layer in the algorithm, which eliminates the feature points on the foreground objects in MR, and to construct a background feature point set to realize high precision and high real-time location. Experimental results show that the proposed algorithm reduces the relative pose error by 60.5% and improves the real-time location performance by 39.5% compared to the dynamic scenes simultaneous localization and mapping location algorithm in the high-dynamic foreground object scene of the Technical University of Munich public dataset. Therefore, this algorithm has high application value in MR scenes.

Key words: mixed reality (MR), foreground object, location, semantic feature