Fusing Computer Vision and Wireless Signal for Accurate Sensor Localization in AR View

Recent years have seen increasing traction to enable new applications that can localize sensors on the screen of an Augmented Reality (AR) device (e.g. smartphone, tablet) so that sensors can be controlled more intuitively. Despite recent advances in this area, both wireless signal dependent and computer vision based localization solutions have seen a slow acceptance due to signal noise, multipath effect, and limited AR device-sensor interactivity. In this paper, we propose a novel solution to combine the complementary advantages of wireless signal based localization solution with the computer vision based solution to track IoT devices and sensors more accurately. Experimental result shows that our system can accurately track IoT devices with an average pixel error of 34 pixels in a 1024 × 768 pixels image, which is a 75.8% improvement from the state-of-the-art model.