Abstract
Landmark detection technology has a wide range of applications in people's lives, including map correcting, localization and navigation, etc. Besides, landmarks are also utilized to label different areas for automatic floor plan construction. Currently, vision-based landmark detection methods have some limitations, such as light, camera shaking, and privacy-invasive. In addition, deep learning-based methods increase the time consumption of marking labels due to the huge requirement for data. Targeting the above challenges, our work first proposes a landmark detection approach based on Human Activity Recognition (HAR) for automatic floor plan construction, which introduces a self-attention model to recognize various landmarks by walker's daily activities due to their strong correlation. First, the accelerometer and gyroscope sensor data are extracted and eliminated by a Gaussian filter and are divided into the same length segments by slide window. Next, it is input into the self-attention network to train a human activity recognition model. Finally, the corresponding relationship between human activities and landmarks is created to detect landmarks through the trained HAR model. Empirical results on two publicly available USC-HAD and OPPORTUNITY datasets show our proposed approach can recognize landmarks effectively.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Yanying, G., Lo, A., Niemegeers, I.: A survey of indoor positioning systems for wireless personal networks. IEEE Commun. Surv. Tutorials 11, 13–32 (2009)
Forman, G.H., Zahorjan, J.: The challenges of mobile computing. Computer 27, 38–47 (1994)
Barry, B., et al.: Educating for mobile computing: addressing the new challenges. In: Proceedings of the Final Reports on Innovation and Technology in Computer Science Education 2012 Working Groups Haifa, Israel: ACM, pp. 51–63 (2012)
Kakousis, K., Paspallis, N., Papadopoulos, G.A.: A survey of software adaptation in mobile and ubiquitous computing. Enterp. Inf. Syst. 4, 355–389 (2010)
Ladd, D., Alan, D., Avimanyu, S., et al.: Trends in mobile computing with in the is discipline: a ten-year retrospective. Commun. Assoc. Inf. Syst. 27, 285–316 (2010)
Gay, G.: Context-aware mobile computing: affordances of space, social awareness, and social influence. Synthesis Lectures on Human-Centered Informatics. Morgan and Claypool Publishers, San Rafael. vol. 2, pp. 1–62 (2009)
Sana.: A survey of indoor localization techniques. IOSR J. Electr. Electron. Eng. (IOSR-JEEE). 6, 69–76 (2013)
Alzantot, M.: Youssef, M.: Crowdinside: automatic construction of indoor floorplans. In: Proceedings of the 20th International Conference on Advances in Geographic Information Systems, New York, United States, pp. 99–108 (2012)
X. Zhang, Y. Jin, et al. CIMLoc: A crowdsourcing indoor digital map construction system for localization. In 2014 IEEE Ninth International Conference on Intelligent Sensors, Sensor Networks and Information Processing (ISSNIP), Singapore, pp. 1–6, IEEE (2014)
Elhamshary, M., Alzantot, M., Youssef, M.: JustWalk: a crowdsourcing approach for the automatic construction of indoor floorplans. IEEE Trans. Mob. Comput. 18(10), 2358–2371 (2018)
Zhou, B., Li, Q., Mao, Q., Tu, W., et al.: ALIMC: activity landmark-based indoor mapping via crowdsourcing. IEEE Trans. Intell. Transp. Syst. 16(5), 2774–2785 (2015)
Amarasinghe, D., Mann, G.K., Gosine, R.G.: Landmark detection and localization for mobile robot applications: a multisensor approach. Robotica 28(5), 663–673 (2010)
Alansary, A., Oktay, O., et al.: Evaluating reinforcement learning agents for anatomical landmark detection. Med. Image Anal. 53, 156–164 (2019)
Nilwong, S., Hossain, D., et al.: Deep learning-based landmark detection for mobile robot outdoor localization. Machines 7(2), 25 (2019)
Wang, Z., Vandersteen, C., Raffaelli, C., Guevara, N., Patou, F., Delingette, H.: One-shot learning for landmarks detection. In: Engelhardt, S., et al. (eds.) Deep Generative Models, and Data Augmentation, Labelling, and Imperfections. Lecture Notes in Computer Science, vol. 13003, pp. 163–172. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-88210-5_15
Weishaupt, F., Will, P.S., et al.: Robust point-shaped landmark detection using polarimetric radar. In: 2021 IEEE Intelligent Vehicles Symposium (IV), pp. 859–865, IEEE (2021)Â
Narayana, K., Goulette, F., Steux, B.: Planar landmark detection using a specific arrangement of LIDAR scanners. In: IEEE/ION Position, Location and Navigation Symposium, pp. 1057–1069, IEEE, May 2010
Ravankar, A., Hoshino, Y., Kobayashi, Y.: Robust landmark detection in vineyards using laser range sensor. In: The Proceedings of JSME annual Conference on Robotics and Mechatronics (Robomec), pp. 1A1-E03 (2019)
Sun, S., Yin, Y., et al.: D. Robust landmark detection and position measurement based on monocular vision for autonomous aerial refueling of UAVs. IEEE Trans. Cybern. 49(12), 4167–4179 (2018)
Rous, M., Lupschen, H., et al.: Vision-based indoor scene analysis for natural landmark detection. In: Proceedings of the 2005 IEEE International conference on Robotics and Automation, Barcelona, Spain, pp. 4642–4647 (2005)
Sun, S., Yin, Y., Wang, X., Xu, D.: Robust landmark detection and position measurement based on monocular vision for autonomous aerial refueling of UAVs. IEEE Trans. Cybern. 49(12), 4167–4179 (2018)
Zheng, Y., Liu, D., Georgescu, B., Nguyen, H., Comaniciu, D.: 3D deep learning for efficient and robust landmark detection in volumetric data. In: Navab, N., Hornegger, J., Wells, W.M., Frangi, A.F. (eds.) Medical Image Computing and Computer-Assisted Intervention – MICCAI 2015. Lecture Notes in Computer Science, vol. 9349, pp. 565–572. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-24553-9_69
Schwendicke, F., et al.: Deep learning for cephalometric landmark detection: systematic review and meta-analysis. Clin. Oral Invest. 25(7), 4299–4309 (2021). https://doi.org/10.1007/s00784-021-03990-w
Han, D., Gao, Y., Wu, G., Yap, P.-T., Shen, D.: Robust anatomical landmark detection for MR brain image registration. In: Golland, P., Hata, N., Barillot, C., Hornegger, J., Howe, R. (eds.) Medical Image Computing and Computer-Assisted Intervention – MICCAI 2014. Lecture Notes in Computer Science, vol. 8673, pp. 186–193. Springer, Cham (2014). https://doi.org/10.1007/978-3-319-10404-1_24
Zhang, J., Liu, M., Shen, D.: Detecting anatomical landmarks from limited medical imaging data using two-stage task-oriented deep neural networks. IEEE Trans. Image Process. 26(10), 4753–4764 (2017)
Jheng, Y.-C., et al.: A novel machine learning-based algorithm to identify and classify lesions and anatomical landmarks in colonoscopy images. Surg. Endosc. 36(1), 640–650 (2021). https://doi.org/10.1007/s00464-021-08331-2
Zhang, Z., Luo, P., et al.: Facial landmark detection by deep multi-task learning. In: European Conference on Computer Vision, Part II, Zurich, Switzerland, pp. 94–108, 6–12 Sep 2014
Liu, Z., et al.: Robust target recognition and tracking of self-driving cars with radar and camera information fusion under severe weather conditions. IEEE Trans. Intell. Transp. Syst. 23(7) 6640–653 (2021)
Wang, C., Liu, J., Chen, Y., et al.: Towards in-baggage suspicious object detection using commodity wifi. In: 2018 IEEE Conference on Communications and Network Security (CNS), pp. 1–9. IEEE, May 2018
Beltrán, J., Guindel, C., Moreno, F.M., et al.: BirdNet: a 3D object detection framework from lidar information. In: 2018 21st International Conference on Intelligent Transportation Systems (ITSC), pp. 3517–3523. IEEE, November 2018
Zhou, B., Elbadry, M., Gao, R., Ye, F.: Towards scalable indoor map construction and refinement using acoustics on smartphones. IEEE Trans. Mob. Comput. 19(1), 217–230 (2019)
Dubois, A., François, C.: Human activities recognition with RGB-Depth camera using HMM. In: 2013 35th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC). IEEE, Osaka, Japan, 3–7 Jul 2013
Wang, K., He, J., Zhang, L.: Attention-based convolutional neural network for weakly labeled human activities’ recognition with wearable sensors. IEEE Sens. J. 19(17), 7598–7604 (2019)
Varshney, N., Bakariya, B.: Deep convolutional neural model for human activities recognition in a sequence of video by combining multiple CNN streams. Multimedia Tools Appl. 81, 1–13 (2021). https://doi.org/10.1007/s11042-021-11220-4
Liu, Z., Han, Y., Chen, Z., Fang, Y., Qian, H., Zhou, J.: Human activities recognition from videos based on compound deep neural network. In: Liu, Qi., Liu, X., Shen, T., Qiu, X. (eds.) The 10th International Conference on Computer Engineering and Networks. Advances in Intelligent Systems and Computing, vol. 1274, pp. 314–326. Springer, Singapore (2021). https://doi.org/10.1007/978-981-15-8462-6_37
Gnouma, M., Ladjailia, A., Ejbali, R., Zaied, M.: Stacked sparse autoencoder and history of binary motion image for human activity recognition. Multimedia Tools Appl. 78(2), 2157–2179 (2018). https://doi.org/10.1007/s11042-018-6273-1
Snoun, A., Jlidi, N., Bouchrika, T., Jemai, O., Zaied, M.: Towards a deep human activity recognition approach based on video to image transformation with skeleton data. Multimedia Tools Appl. 80(19), 29675–29698 (2021). https://doi.org/10.1007/s11042-021-11188-1
Murad, A., Pyun, J.Y.: Deep recurrent neural networks for human activity recognition. Sensors 17(11), 2556 (2017)
Xu, C., et al.: InnoHAR: a deep neural network for complex human activity recognition. IEEE Access 7, 9893–9902 (2019)
Zhang, F., et al.: Towards a diffraction-based sensing approach on human activity recognition. In: Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 3(1), 1–25 (2019)
Yan, H., et al.: WiAct: a passive WiFi-based human activity recognition system. IEEE Sens. J. 20(1), 296–305 (2019)
Bashar, S.K., Abdullah, A.F., Ki, H.C.: Smartphone based human activity recognition with feature selection and dense neural network. In: 42nd Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), Montreal, Canada, pp. 20–24 (2020)
Mahmud, S., Tonmoy, M.: et al.: Human activity recognition from wearable sensor data using self-attention. arXiv preprint arXiv:2003.09018. (2020)
Zhang, M., Sawchuk, A.A.: USC-HAD: a daily activity dataset for ubiquitous activity recognition using wearable sensors. In: Proceedings of the 2012 ACM Conference on Ubiquitous Computing, Pittsburgh, USA, pp. 1036–1043 (2012)
Roggen. D., Calatroni, A., et al:. Collecting complex activity datasets in highly rich networked sensor environments. In: 2010 Seventh International Conference on Networked Sensing Systems (INSS), Kassel, Germany, pp. 233–240, IEEE (2010)
Thakur, D., Biswas, S., Ho., et al.: ConvAE-LSTM: convolutional Autoencoder Long Short-Term Memory Network for Smartphone-Based Human Activity Recognition. IEEE Access 10, 4137–4156 (2022)
Lim, X.Y., Gan, K.B., et al.: Deep ConvLSTM network with dataset resampling for upper body activity recognition using minimal number of IMU sensors. Appl. Sci. 11(8), 3543 (2021)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 ICST Institute for Computer Sciences, Social Informatics and Telecommunications Engineering
About this paper
Cite this paper
Huang, Z., Poslad, S., Li, Q., Li, J., Chen, C. (2022). Landmark Detection Based on Human Activity Recognition for Automatic Floor Plan Construction. In: Gao, H., Wang, X., Wei, W., Dagiuklas, T. (eds) Collaborative Computing: Networking, Applications and Worksharing. CollaborateCom 2022. Lecture Notes of the Institute for Computer Sciences, Social Informatics and Telecommunications Engineering, vol 461. Springer, Cham. https://doi.org/10.1007/978-3-031-24386-8_25
Download citation
DOI: https://doi.org/10.1007/978-3-031-24386-8_25
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-24385-1
Online ISBN: 978-3-031-24386-8
eBook Packages: Computer ScienceComputer Science (R0)