Abstract
In virtual reality surgery training, magnetic levitation instruments have gained popularity due to the advantages of non-mechanical friction and low inertia. However, it is difficult to obtain high accuracy, frequency, and robust navigation stability, and this will not capture the subtle changes in the user’s actions resulting in a much weaker sense of immersion. To tackle this issue, previous works have used inconvenient motion tracking sensors for navigation. Nevertheless, these techniques did not consider the navigation effects caused by the environmental limitations of the sensors. In this work, we propose a Transformer-based high-frequency prediction model (HPformer) to predict the direction and position data by designing an incremental module to learn the increment of navigation information in an accumulative manner. Also, to reduce the position prediction value error, we propose an initialization module related to uniform acceleration. By building a testbed, experimental results show that our method can obtain accurate navigation (the mean absolute error is less than 0.026) and increase the navigation frequency 200 Hz.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Bahdanau, D., Cho, K., Bengio, Y.: Neural machine translation by jointly learning to align and translate. In: ICLR (2015)
Bai, S., Kolter, J.Z., Koltun, V.: An empirical evaluation of generic convolutional and recurrent networks for sequence modeling. arXiv preprint arXiv:1803.01271 (2018)
Berkelman, P., Bozlee, S., Miyasaka, M.: Interactive rigid-body dynamics and deformable surface simulations with co-located maglev haptic and 3d graphic display (2013)
Berkelman, P., Miyasaka, M., Anderson, J.: Co-located 3d graphic and haptic display using electromagnetic levitation. In: 2012 IEEE Haptics Symposium (HAPTICS), pp. 77–81. IEEE (2012)
Boulanger, P., Wu, G., Bischof, W., Yang, X.: Hapto-audio-visual environments for collaborative training of ophthalmic surgery over optical network. In: 2006 IEEE International Workshop on Haptic Audio Visual Environments and their Applications (HAVE 2006), pp. 21–26. IEEE (2006)
Clark, R., Wang, S., Wen, H., Markham, A., Trigoni, N.: Vinet: visual-inertial odometry as a sequence-to-sequence learning problem. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 31 (2017)
Hamza-Lup, F.G., Bogdan, C.M., Popovici, D.M., Costea, O.D.: A survey of visuo-haptic simulation in surgical training. arXiv preprint arXiv:1903.03272 (2019)
Kitaev, N., Kaiser, L., Levskaya, A.: Reformer: the efficient transformer. In: ICLR (2020). https://openreview.net/forum?id=rkgNKkHtvB
Lai, G., Chang, W.C., Yang, Y., Liu, H.: Modeling long-and short-term temporal patterns with deep neural networks. In: SIGIR (2018)
Miyasaka, M., Berkelman, P.: Magnetic levitation with unlimited omnidirectional rotation range. Mechatronics 24(3), 252–264 (2014)
Moody, L., Baber, C., Arvanitis, T.N.: The role of haptic feedback in the training and assessment of surgeons using a virtual environment. In: proceedings of Eurohaptics, pp. 170–173 (2001)
Mourikis, A.I., Roumeliotis, S.I.: A multi-state constraint kalman filter for vision-aided inertial navigation. In: Proceedings 2007 IEEE International Conference on Robotics and Automation, pp. 3565–3572. IEEE (2007)
Pedram, S.A., Klatzky, R.L., Berkelman, P.: Torque contribution to haptic rendering of virtual textures. IEEE Trans. Haptics 10(4), 567–579 (2017)
Qin, T., Li, P., Shen, S.: Vins-mono: a robust and versatile monocular visual-inertial state estimator. IEEE Trans. Rob. 34(4), 1004–1020 (2018)
Satava, R.M., Jones, S.B.: Current and future applications of virtual reality for medicine. In: Proceedings of the IEEE, vol. 86, no. 3, pp. 484–489 (1998)
Teixeira, B., Silva, H., Matos, A., Silva, E.: Deep learning approaches assessment for underwater scene understanding and egomotion estimation. In: OCEANS 2019 MTS/IEEE SEATTLE, pp. 1–9. IEEE (2019)
Tong, Q., Yuan, Z., Liao, X., Zheng, M., Yuan, T., Zhao, J.: Magnetic levitation haptic augmentation for virtual tissue stiffness perception. IEEE Trans. Visual Comput. Graph. 24(12), 3123–3136 (2017)
Tong, Q., Yuan, Z., Zheng, M., Zhu, W., Zhang, G., Liao, X.: A novel magnetic levitation haptic device for augmentation of tissue stiffness perception. In: Proceedings of the 22nd ACM Conference on Virtual Reality Software and Technology, pp. 143–152 (2016)
Vaswani, A., et al.: Attention is all you need. In: Advances in Neural Information Processing Systems, pp. 5998–6008 (2017)
Viitanen, M., Vanne, J., Hämäläinen, T.D., Kulmala, A.: Low latency edge rendering scheme for interactive 360 degree virtual reality gaming. In: 2018 IEEE 38th International Conference on Distributed Computing Systems (ICDCS), pp. 1557–1560. IEEE (2018)
Wu, C., Wu, F., Qi, T., Huang, Y., Xie, X.: Fastformer: additive attention can be all you need. arXiv preprint arXiv:2108.09084 (2021)
Zhou, H., et al.: Informer: beyond efficient transformer for long sequence time-series forecasting. In: Proceedings of AAAI (2021)
Acknowledgment
The work was supported by National Natural Science Foundation of China (62073248) and Translational Medicine and Interdisciplinary Research Joint Fund of Zhongnan Hospital of Wuhan University (ZNJC201926).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Huang, J., Zhao, J., Qiu, Z., Yuan, Z. (2023). Transformer Based High-Frequency Predictive Model for Visual-Haptic Feedback of Virtual Surgery Navigation. In: Tanveer, M., Agarwal, S., Ozawa, S., Ekbal, A., Jatowt, A. (eds) Neural Information Processing. ICONIP 2022. Lecture Notes in Computer Science, vol 13625. Springer, Cham. https://doi.org/10.1007/978-3-031-30111-7_13
Download citation
DOI: https://doi.org/10.1007/978-3-031-30111-7_13
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-30110-0
Online ISBN: 978-3-031-30111-7
eBook Packages: Computer ScienceComputer Science (R0)