Cited By
View all- Siwilai WMalaithong CRaksanawes TSookhom ARasrichai KKerdvibulvech C(2024)PALLA: Palpation Suit for Breast Cancer ExaminationSN Computer Science10.1007/s42979-024-03002-05:6Online publication date: 18-Jun-2024
A Method of Touchable 3d Model Reconstruction based on Mixed Reality –A Case Study of Medical Training Applications
Pages 83 - 89
Abstract
The application of trauma and hemorrhage medical treatment training has generated higher and higher requirements for the authenticity of the medical model. With the increasing integration of mixed reality technology and the medical field, multi-view-based 3D reconstruction technology can generate trauma medical object models not limited to specific targets, with low cost and high precision. However, the current reconstruction methods have weak points such as the inability to build the object surface beyond the direct view of humans, the uneven distribution of reconstructed point clouds, many holes, and reconstructed triangles. And it is time-consuming and laborious to fill and modify manually and requires highly skilled modeling expertise. Therefore, the models generated by traditional 3D reconstruction methods cannot meet the needs of interaction and touch in mixed reality applications. In order to solve these problems, we propose a new model generation method based on multi-view and point cloud processing, which enables the model to reduce the computational cost under the premise of meeting the need for direct interaction and touch in mixed reality applications. Finally, based on the reconstruction method proposed in this paper, a new medical trauma hemorrhage control and rescue training system is designed and implemented, which proves the effectiveness of the method proposed in this paper.
References
[1]
Tepper OM,Rudy HL,Lefkowitz A,et al. Mixed Reality with HoloLens: Where Virtual Reality Meets Augmented Reality in the Operating Room[J]. Plast Reconstr Surg,2017,140(5):1066-1070.
[2]
Chen L,Day TW,Tang W,et al. Recent Developments and Future Challenges in Medical Mixed Reality[J]. IEEE International Symposium,2017 :123-135.
[3]
Lee SC,Fuerst B,Tateno K,et al. Multi-modal imaging, modelbased tracking, and mixed reality visualisation for orthopaedic surgery[J]. Healthc Technol Lett,2017,4(5):168-173.
[4]
Application of mixed reality technology in orthopedic field CHEN Lin, LIU Bailian, PU Qi, WANG Daxing, YANG Huiqin Department of Orthopedics, Yan'an Hospital Affiliated to Kunming Medical University, Kunming, Yunnan 650051, P. R. China Corresponding author: YANG Huiqin, Email: [email protected].
[5]
Levoy M, Pulli K, Curless B, The Digital michelangelo project: 3D scanning of large statues//Proceedings of the Siggraph. New Orleans, USA, 2000: 1-14.
[6]
Seitz S, Curless B, Diebel J, Scharstein D, Szeliski R. A comparison and evaluation of multi-view stereo reconstruction algorithms//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. New York, USA, 2006: 519-526.
[7]
Strecha C, Hansen W. von, Van Gool L, Fua P, Thoennessen U. On benchmarking camera calibration and multiview stereo for high resolution imagery//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Anchorage, USA, 2008: 1-8.
[8]
Schöps T, L. Schönberger J, Galliani S, Sattler T, Schindler K, Pollefeys M, Geiger A. A multi-view stereo benchmark with high-resolution images and multi-camera videos// Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Honolulu, USA, 2017: 1-8.
[9]
Schönberger J. L, Zheng E, Pollefeys M, Frahm J. M. Pixelwise view selection for unstructured multi-view stereo//Proceedings of the European Conference on Computer Vision. Amsterdam, The Netherlands, 2016: 1-17.
[10]
Galliani S, Lasinger K, Schindler K. Massively Parallel Multiview Stereopsis by Surface Normal Diffusion//Proceedings of the IEEE International Conference on Computer Vision. Santiago, Chile, 2015: 873-881.
[11]
Bailer C, Finckh M, Lensch H P A. Scale robust multiview stereo//Proceedings of the European Conference on Computer Vision. Florence, Italy, 2012: 1-14.
[12]
Li, Z, Zuo W, Wang Z, Zhang L. Confidence-based large-scale dense multi-view stereo. IEEE Transactions on Image Processing, 2020, 29: 7176-7191.
[13]
Kostrikov I, Horbert E, Leibe B. Probabilistic labeling cost for high-accuracy multi-view reconstruction//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Columbus, USA, 2014: 1-8
[14]
Hane C, Zach C, Cohen A, Pollefeys M. Dense semantic 3D reconstruction. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 39(9): 1730-1743.
[15]
Johannes Lutz S, Jan-Michael F. Structure-from-motion revisited// Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas, USA, 2016: 4104- 4113.
[16]
Zitian Huang, Yikuan Yu1, Jiawen Xu, Feng Ni2, Xinyi Le1.PF-Net: Point Fractal Network for 3D Point Cloud Completion.CVPR, 2020.
[17]
Erkan Bostanci, Nadia Kanwal, Shoaib Ehsan, and Adrian F. Clark, "User Tracking Methods for Augmented Reality," International Journal of Computer Theory and Engineering vol. 5, no. 1, pp. 93-98, 2013.
[18]
Mohamed-Amine Abidi, Barbara Lyonnet, Pierre Chevaillier, and Rosario Toscano, "Contribution of Virtual Reality for Lines Production's Simulation in a Lean Manufacturing Environment," International Journal of Computer Theory and Engineering vol. 8, no. 3, pp. 182-189, 2016.
[19]
Wei-Yu Chen, Makoto Sato, Yen-Ming Chu, and Teruki Honma, "SPIDAR-if: A Joint User Interface Approach for Hapticised Web Content and Augment Reality Enhancement," International Journal of Computer Theory and Engineering vol. 8, no. 6, pp. 444-449, 2016.
[20]
Enhancing User Experience in Points of Interest with Augmented Reality, "Fernando Vera, J. Alfredo Sánchez, and Ofelia Cervantes," International Journal of Computer Theory and Engineering vol. 8, no. 6, pp. 450-457, 2016.
[21]
D. S. Pamungkas and K. Ward, "Electro-Tactile Feedback System to Enhance Virtual Reality Experience," International Journal of Computer Theory and Engineering vol. 8, no. 6, pp. 465-470, 2016.
[22]
Jinsuk Kang, "Testing and Evaluation of a Augmented Reality System for Battlefield Solders," International Journal of Computer Theory and Engineering vol. 9, no. 3, pp. 192-196, 2017.
[23]
Georgi P. Dimitrov, Galina Panayotova, Rosen P. Toshev, and Iva Kostadinova, "Analysis of the Growth Perspectives of the Early Stages of Technological Innovation in the Case of the Augmented and Virtual Reality Application," International Journal of Computer Theory and Engineering vol. 9, no. 6, pp. 443-446, 2017.
[24]
Jonathan Rodriguez and Ching-Yu Huang, "An Emerging Study in Augmented Reality & Geographical Information System," International Journal of Computer Theory and Engineering vol. 9, no. 6, pp. 447-450, 2017.
[25]
G. Ko, S. Ryu, S. Nam, J. Lee, and K. Suh, "Design of Virtual Reality Prototyping System and Hand-Held Haptic Controller," International Journal of Computer Theory and Engineering vol. 11, no. 4, pp. 72-75, 2019.
[26]
Konstantinos P. Ferentinos, Yannis Stavrakas, Harry Nakos, Kostis Pristouris, and Myrto S. Barda, "Initial Design and Features of an Augmented Reality System for Urban Park Touring and Management," International Journal of Computer Theory and Engineering vol. 12, no. 5, pp. 106-112, 2020.
Index Terms
- A Method of Touchable 3d Model Reconstruction based on Mixed Reality –A Case Study of Medical Training Applications
Recommendations
Real-time mixed-reality telepresence via 3D reconstruction with HoloLens and commodity depth sensors
ICMI '17: Proceedings of the 19th ACM International Conference on Multimodal InteractionWe present a demo of low-cost mixed reality telepresence system that performs real-time 3D reconstruction of a person or an object and wirelessly transmits the reconstructions to Microsoft's HoloLens head mounted display at frame rates perceived as ...
Medical 3D Images in Multimodal Virtual Reality
IUI '18 Companion: Companion Proceedings of the 23rd International Conference on Intelligent User InterfacesWe present a multimodal medical 3D image system for radiologists in an virtual reality (VR) environment. Users can walk freely inside the virtual room and interact with the system using speech, going through patient records, and manipulate 3D image data ...
Estimating the Pose of a Medical Manikin for Haptic Augmentation of a Virtual Patient in Mixed Reality Training
SVR '21: Proceedings of the 23rd Symposium on Virtual and Augmented RealityVirtual medical emergency training provides complex while safe interactions with virtual patients. Classical non-virtual medical training includes the use of medical manikins. Haptically integrating such a manikin into virtual training has the potential ...
Comments
Information & Contributors
Information
Published In
January 2023
246 pages
ISBN:9781450398572
DOI:10.1145/3582649
Copyright © 2023 Owner/Author.
This work is licensed under a Creative Commons Attribution International 4.0 License.
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 07 April 2023
Check for updates
Qualifiers
- Research-article
- Research
- Refereed limited
Conference
ICIGP 2023
ICIGP 2023: 2023 The 6th International Conference on Image and Graphics Processing
January 6 - 8, 2023
Chongqing, China
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- View Citations1Total Citations
- 229Total Downloads
- Downloads (Last 12 months)185
- Downloads (Last 6 weeks)15
Reflects downloads up to 26 Jul 2024
Other Metrics
Citations
Cited By
View all- Siwilai WMalaithong CRaksanawes TSookhom ARasrichai KKerdvibulvech C(2024)PALLA: Palpation Suit for Breast Cancer ExaminationSN Computer Science10.1007/s42979-024-03002-05:6Online publication date: 18-Jun-2024
View Options
View options
View or Download as a PDF file.
PDFeReader
View online with eReader.
eReaderHTML Format
View this article in HTML Format.
HTML FormatGet Access
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in