Revisiting Surgical Instrument Segmentation Without Human Intervention: A Graph Partitioning View
Pages 16 - 25
Abstract
Surgical instrument segmentation (SIS) on endoscopic images stands as a long-standing and essential task in the context of computer-assisted interventions for boosting minimally invasive surgery. Given the recent surge of deep learning methodologies and their data-hungry nature, training a neural predictive model based on massive expert-curated annotations has been dominating and served as an off-the-shelf approach in the field, which could, however, impose prohibitive burden to clinicians for preparing fine-grained pixel-wise labels corresponding to the collected surgical video frames. In this work, we propose an unsupervised method by reframing the video frame segmentation as a graph partitioning problem and regarding image pixels as graph nodes, which is significantly different from the previous efforts. A self-supervised pre-trained model is firstly leveraged as a feature extractor to capture high-level semantic features. Then, Laplacian matrixs are computed from the features and are eigendecomposed for graph partitioning. On the "deep" eigenvectors, a surgical video frame is meaningfully segmented into different modules such as tools and tissues, providing distinguishable semantic information like locations, classes, and relations. The segmentation problem can then be naturally tackled by applying clustering or threshold on the eigenvectors. Extensive experiments are conducted on various datasets (e.g., EndoVis2017, EndoVis2018, UCL, etc.) for different clinical endpoints. Across all the challenging scenarios, our method demonstrates outstanding performance and robustness higher than unsupervised state-of-the-art (SOTA) methods. The code is released at https://github.com/MingyuShengSMY/GraphClusteringSIS.git.
References
[1]
Rameen Abdal, Peihao Zhu, Niloy J. Mitra, and Peter Wonka. 2021. Labels4Free: Unsupervised Segmentation using StyleGAN. In 2021 IEEE/CVF International Conference on Computer Vision (ICCV). 13950--13959. https://doi.org/10.1109/ICCV48922.2021.01371
[2]
Max Allan, Satoshi Kondo, Sebastian Bodenstedt, Stefan Leger, Rahim Kadkhodamohammadi, Imanol Luengo, Felix Fuentes, Evangello Flouty, Ahmed Mohammed, Marius Pedersen, Avinash Kori, Varghese Alex, Ganapathy Krishnamurthi, David Rauber, Robert Mendel, Christoph Palm, Sophia Bano, Guinther Saibro, Chi-Sheng Shih, Hsun-An Chiang, Juntang Zhuang, Junlin Yang, Vladimir Iglovikov, Anton Dobrenkii, Madhu Reddiboina, Anubhav Reddy, Xingtong Liu, Cong Gao, Mathias Unberath, Myeonghyeon Kim, Chanho Kim, Chaewon Kim, Hyejin Kim, Gyeongmin Lee, Ihsan Ullah, Miguel Luna, Sang Hyun Park, Mahdi Azizian, Danail Stoyanov, Lena Maier-Hein, and Stefanie Speidel. 2020. 2018 Robotic Scene Segmentation Challenge. https://doi.org/10.48550/arXiv.2001.11190 arxiv: 2001.11190 [cs.CV]
[3]
Max Allan, Alex Shvets, Thomas Kurmann, Zichen Zhang, Rahul Duggal, Yun-Hsuan Su, Nicola Rieke, Iro Laina, Niveditha Kalavakonda, Sebastian Bodenstedt, Luis Herrera, Wenqi Li, Vladimir Iglovikov, Huoling Luo, Jian Yang, Danail Stoyanov, Lena Maier-Hein, Stefanie Speidel, and Mahdi Azizian. 2019. 2017 Robotic Instrument Segmentation Challenge. https://doi.org/10.48550/arXiv.1902.06426 arxiv: 1902.06426 [cs.CV]
[4]
Mohamed Attia, Mohammed Hossny, Saeid Nahavandi, and Hamed Asadi. 2017. Surgical tool segmentation using a hybrid deep CNN-RNN auto encoder-decoder. In 2017 IEEE International Conference on Systems, Man, and Cybernetics (SMC). 3373--3378. https://doi.org/10.1109/SMC.2017.8123151
[5]
Yaniv Benny and Lior Wolf. 2020. OneGAN: Simultaneous Unsupervised Learning of Conditional Image Generation, Foreground Segmentation, and Fine-Grained Clustering. In Computer Vision -- ECCV 2020, Andrea Vedaldi, Horst Bischof, Thomas Brox, and Jan-Michael Frahm (Eds.). Springer International Publishing, Cham, 514--530. https://doi.org/10.1007/978--3-030--58574--7_31
[6]
Adam Bielski and Paolo Favaro. 2019. Emergence of Object Segmentation in Perturbed Generative Models. Advances in Neural Information Processing Systems, Vol. 32 (2019). https://proceedings.neurips.cc/paper_files/paper/2019/file/af8d9c4e238c63fb074b44eb6aed80ae-Paper.pdf
[7]
Loubna Bouarfa, Oytun Akman, Armin Schneider, Pieter P. Jonker, and Jenny Dankelman. 2012. In-vivo real-time tracking of surgical instruments in endoscopic video. Minimally Invasive Therapy & Allied Technologies, Vol. 21, 3 (2012), 129--134. https://doi.org/10.3109/13645706.2011.580764 21574828.
[8]
David Bouget, Rodrigo Benenson, Mohamed Omran, Laurent Riffaud, Bernt Schiele, and Pierre Jannin. 2015. Detecting Surgical Tools by Modelling Local Appearance and Global Shape. IEEE Transactions on Medical Imaging, Vol. 34, 12 (2015), 2603--2617. https://doi.org/10.1109/TMI.2015.2450831
[9]
Mathilde Caron, Hugo Touvron, Ishan Misra, Hervé Jegou, Julien Mairal, Piotr Bojanowski, and Armand Joulin. 2021. Emerging Properties in Self-Supervised Vision Transformers. In 2021 IEEE/CVF International Conference on Computer Vision (ICCV). 9630--9640. https://doi.org/10.1109/ICCV48922.2021.00951
[10]
Kai-Yueh Chang, Tyng-Luh Liu, and Shang-Hong Lai. 2011. From co-saliency to co-segmentation: An efficient and fully unsupervised energy minimization model. In Computer Vision and Pattern Recognition (CVPR). 2129--2136. https://doi.org/10.1109/CVPR.2011.5995415
[11]
Jiachen Chen, Mengyang Li, Hu Han, Zhiming Zhao, and Xilin Chen. 2024. SurgNet: Self-Supervised Pretraining With Semantic Consistency for Vessel and Instrument Segmentation in Surgical Images. IEEE Transactions on Medical Imaging, Vol. 43, 4 (2024), 1513--1525. https://doi.org/10.1109/TMI.2023.3341948
[12]
Mickaël Chen, Thierry Artières, and Ludovic Denoyer. 2019. Unsupervised Object Segmentation by Redrawing. Advances in Neural Information Processing Systems, Vol. 32 (2019). https://proceedings.neurips.cc/paper_files/paper/2019/file/32bbf7b2bc4ed14eb1e9c2580056a989-Paper.pdf
[13]
Emanuele Colleoni, Philip Edwards, and Danail Stoyanov. 2020. Synthetic and Real Inputs for Tool Segmentation in Robotic Surgery. In Medical Image Computing and Computer Assisted Intervention -- MICCAI 2020, Anne L. Martel, Purang Abolmaesumi, Danail Stoyanov, Diana Mateus, Maria A. Zuluaga, S. Kevin Zhou, Daniel Racoceanu, and Leo Joskowicz (Eds.). Springer International Publishing, Cham, 700--710. https://doi.org/10.1007/978--3-030--59716-0_67
[14]
Jianan Fan, Dongnan Liu, Hang Chang, Heng Huang, Mei Chen, and Weidong Cai. 2023. Taxonomy Adaptive Cross-Domain Adaptation in Medical Imaging via Optimization Trajectory Distillation. In 2023 IEEE/CVF International Conference on Computer Vision (ICCV). 21117--21127. https://doi.org/10.1109/ICCV51070.2023.01936
[15]
Jianan Fan, Dongnan Liu, Hang Chang, Heng Huang, Mei Chen, and Weidong Cai. 2024. Seeing Unseen: Discover Novel Biomedical Concepts via Geometry-Constrained Probabilistic Modeling. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 11524--11534.
[16]
Miroslav Fiedler. 1973. Algebraic connectivity of graphs. Czechoslovak mathematical journal, Vol. 23, 2 (1973), 298--305. https://dml.cz/handle/10338.dmlcz/101168
[17]
Luis C. García-Peraza-Herrera, Wenqi Li, Caspar Gruijthuijsen, Alain Devreker, George Attilakos, Jan Deprest, Emmanuel Vander Poorten, Danail Stoyanov, Tom Vercauteren, and Sébastien Ourselin. 2017. Real-Time Segmentation of Non-rigid Surgical Tools Based on Deep Learning and Tracking. In Computer Assisted and Robotic Endoscopy, Terry Peters, Guang-Zhong Yang, Nassir Navab, Kensaku Mori, Xiongbiao Luo, Tobias Reichl, and Jonathan McLeod (Eds.). Springer International Publishing, Cham, 84--95. https://doi.org/10.1007/978--3--319--54057--3_8
[18]
Luis C. García-Peraza-Herrera, Wenqi Li, Lucas Fidon, Caspar Gruijthuijsen, Alain Devreker, George Attilakos, Jan Deprest, Emmanuel Vander Poorten, Danail Stoyanov, Tom Vercauteren, and Sébastien Ourselin. 2017. ToolNet: Holistically-nested real-time segmentation of robotic surgical tools. In 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). 5717--5722. https://doi.org/10.1109/IROS.2017.8206462
[19]
Cristina González, Laura Bravo-Sánchez, and Pablo Arbelaez. 2020. ISINet: An Instance-Based Approach for Surgical Instrument Segmentation. In Medical Image Computing and Computer Assisted Intervention - MICCAI 2020, Anne L. Martel, Purang Abolmaesumi, Danail Stoyanov, Diana Mateus, Maria A. Zuluaga, S. Kevin Zhou, Daniel Racoceanu, and Leo Joskowicz (Eds.). Springer International Publishing, Cham, 595--605. https://doi.org/10.1007/978--3-030--59716-0_57
[20]
Maria Grammatikopoulou, Ricardo Sanchez-Matilla, Felix Bragman, David Owen, Lucy Culshaw, Karen Kerr, Danail Stoyanov, and Imanol Luengo. 2024. A spatio-temporal network for video semantic segmentation in surgical videos. International Journal of Computer Assisted Radiology and Surgery, Vol. 19, 2 (2024), 375--382. https://doi.org/10.1007/s11548-023-02971--6
[21]
Tamás Haidegger, Stefanie Speidel, Danail Stoyanov, and Richard M. Satava. 2022. Robot-Assisted Minimally Invasive Surgery?Surgical Robotics in the Data Age. Proc. IEEE, Vol. 110, 7 (2022), 835--846. https://doi.org/10.1109/JPROC.2022.3180350
[22]
Emanuela Haller and Marius Leordeanu. 2017. Unsupervised Object Segmentation in Video by Efficient Selection of Highly Probable Positive Features. In 2017 IEEE International Conference on Computer Vision (ICCV). 5095--5103. https://doi.org/10.1109/ICCV.2017.544
[23]
Mark Hamilton, Zhoutong Zhang, Bharath Hariharan, Noah Snavely, and William T. Freeman. 2022. Unsupervised Semantic Segmentation by Distilling Feature Correspondences. https://doi.org/10.48550/arXiv.2203.08414 arxiv: 2203.08414 [cs.CV]
[24]
Md. Kamrul Hasan, Lilian Calvet, Navid Rabbani, and Adrien Bartoli. 2021. Detection, segmentation, and 3D pose estimation of surgical tools using convolutional neural networks and algebraic geometry. Medical Image Analysis, Vol. 70 (2021), 101994. https://doi.org/10.1016/j.media.2021.101994
[25]
Kaiming He, Xinlei Chen, Saining Xie, Yanghao Li, Piotr Dollár, and Ross Girshick. 2022. Masked Autoencoders Are Scalable Vision Learners. In 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 15979--15988. https://doi.org/10.1109/CVPR52688.2022.01553
[26]
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. 2016. Deep Residual Learning for Image Recognition. In 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 770--778. https://doi.org/10.1109/CVPR.2016.90
[27]
W. Y. Hong, C. L. Kao, Y. H. Kuo, J. R. Wang, W. L. Chang, and C. S. Shih. 2020. CholecSeg8k: A Semantic Segmentation Dataset for Laparoscopic Cholecystectomy Based on Cholec80. https://doi.org/10.48550/arXiv.2012.12453 arxiv: 2012.12453 [cs.CV]
[28]
Wei-Chih Hung, Varun Jampani, Sifei Liu, Pavlo Molchanov, Ming-Hsuan Yang, and Jan Kautz. 2019. SCOPS: Self-Supervised Co-Part Segmentation. In 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 869--878. https://doi.org/10.1109/CVPR.2019.00096
[29]
Jang Hyun Cho, Utkarsh Mall, Kavita Bala, and Bharath Hariharan. 2021. PiCIE: Unsupervised Semantic Segmentation using Invariance and Equivariance in Clustering. In 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 16789--16799. https://doi.org/10.1109/CVPR46437.2021.01652
[30]
Mobarakol Islam, Daniel Anojan Atputharuban, Ravikiran Ramesh, and Hongliang Ren. 2019. Real-Time Instrument Segmentation in Robotic Surgery Using Auxiliary Supervised Deep Adversarial Learning. IEEE Robotics and Automation Letters, Vol. 4, 2 (2019), 2188--2195. https://doi.org/10.1109/LRA.2019.2900854
[31]
Xu Ji, Andrea Vedaldi, and Joao Henriques. 2019. Invariant Information Clustering for Unsupervised Image Classification and Segmentation. In 2019 IEEE/CVF International Conference on Computer Vision (ICCV). 9864--9873. https://doi.org/10.1109/ICCV.2019.00996
[32]
Yueming Jin, Keyun Cheng, Qi Dou, and Pheng-Ann Heng. 2019. Incorporating Temporal Prior from Motion Flow for Instrument Segmentation in Minimally Invasive Surgery Video. In Medical Image Computing and Computer Assisted Intervention - MICCAI 2019, Dinggang Shen, Tianming Liu, Terry M. Peters, Lawrence H. Staib, Caroline Essert, Sean Zhou, Pew-Thian Yap, and Ali Khan (Eds.). Springer International Publishing, Cham, 440--448. https://doi.org/10.1007/978--3-030--32254-0_49
[33]
Tsung-Wei Ke, Jyh-Jing Hwang, Yunhui Guo, Xudong Wang, and Stella X. Yu. 2022. Unsupervised Hierarchical Semantic Segmentation with Multiview Cosegmentation and Clustering Transformers. In 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 2561--2571. https://doi.org/10.1109/CVPR52688.2022.00260
[34]
Iro Laina, Nicola Rieke, Christian Rupprecht, Josué Page Vizcaíno, Abouzar Eslami, Federico Tombari, and Nassir Navab. 2017. Concurrent Segmentation and Localization for Tracking of Surgical Instruments. In Medical Image Computing and Computer Assisted Intervention - MICCAI 2017, Maxime Descoteaux, Lena Maier-Hein, Alfred Franz, Pierre Jannin, D. Louis Collins, and Simon Duchesne (Eds.). Springer International Publishing, Cham, 664--672. https://doi.org/10.1007/978--3--319--66185--8_75
[35]
Måns Larsson, Erik Stenborg, Carl Toft, Lars Hammarstrand, Torsten Sattler, and Fredrik Kahl. 2019. Fine-Grained Segmentation Networks: Self-Supervised Segmentation for Improved Long-Term Visual Localization. In 2019 IEEE/CVF International Conference on Computer Vision (ICCV). 31--41. https://doi.org/10.1109/ICCV.2019.00012
[36]
Daochang Liu, Yuhui Wei, Tingting Jiang, Yizhou Wang, Rulin Miao, Fei Shan, and Ziyu Li. 2020. Unsupervised Surgical Instrument Segmentation via Anchor Generation and Semantic Diffusion. In Medical Image Computing and Computer Assisted Intervention - MICCAI 2020, Anne L. Martel, Purang Abolmaesumi, Danail Stoyanov, Diana Mateus, Maria A. Zuluaga, S. Kevin Zhou, Daniel Racoceanu, and Leo Joskowicz (Eds.). Springer International Publishing, Cham, 657--667. https://doi.org/10.1007/978--3-030--59716-0_63
[37]
Dongnan Liu, Donghao Zhang, Yang Song, Fan Zhang, Lauren O'Donnell, Heng Huang, Mei Chen, and Weidong Cai. 2020. Unsupervised Instance Segmentation in Microscopy Images via Panoptic Domain Adaptation and Task Re-Weighting. In 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 4242--4251. https://doi.org/10.1109/CVPR42600.2020.00430
[38]
Jie Liu, Xiaoqing Guo, and Yixuan Yuan. 2022. Graph-Based Surgical Instrument Adaptive Segmentation via Domain-Common Knowledge. IEEE Transactions on Medical Imaging, Vol. 41, 3 (2022), 715--726. https://doi.org/10.1109/TMI.2021.3121138
[39]
Jonathan Long, Evan Shelhamer, and Trevor Darrell. 2015. Fully convolutional networks for semantic segmentation. In 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 3431--3440. https://doi.org/10.1109/CVPR.2015.7298965
[40]
Yuxin Ma, Yang Hua, Hanming Deng, Tao Song, Hao Wang, Zhengui Xue, Heng Cao, Ruhui Ma, and Haibing Guan. 2021. Self-Supervised Vessel Segmentation via Adversarial Learning. In 2021 IEEE/CVF International Conference on Computer Vision (ICCV). 7516--7525. https://doi.org/10.1109/ICCV48922.2021.00744
[41]
Lena Maier-Hein, Swaroop S Vedula, Stefanie Speidel, Nassir Navab, Ron Kikinis, Adrian Park, Matthias Eisenmann, Hubertus Feussner, Germain Forestier, Stamatia Giannarou, et al. 2017. Surgical data science for next-generation interventions. Nature Biomedical Engineering, Vol. 1, 9 (2017), 691--696. https://doi.org/10.1038/s41551-017-0132--7
[42]
Luke Melas-Kyriazi, Christian Rupprecht, Iro Laina, and Andrea Vedaldi. 2022. Deep Spectral Methods: A Surprisingly Strong Baseline for Unsupervised Semantic Segmentation and Localization. In 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 8354--8365. https://doi.org/10.1109/CVPR52688.2022.00818
[43]
Zhen-Liang Ni, Gui-Bin Bian, Zeng-Guang Hou, Xiao-Hu Zhou, Xiao-Liang Xie, and Zhen Li. 2020. Attention-Guided Lightweight Network for Real-Time Segmentation of Robotic Surgical Instruments. In 2020 IEEE International Conference on Robotics and Automation (ICRA). 9939--9945. https://doi.org/10.1109/ICRA40945.2020.9197425
[44]
Zhen-Liang Ni, Gui-Bin Bian, Xiao-Liang Xie, Zeng-Guang Hou, Xiao-Hu Zhou, and Yan-Jie Zhou. 2019. RASNet: Segmentation for Tracking Surgical Instruments in Surgical Videos Using Refined Attention Segmentation Network. In 2019 41st Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC). 5735--5738. https://doi.org/10.1109/EMBC.2019.8856495
[45]
Zhen-Liang Ni, Xiao-Hu Zhou, Guan-An Wang, Wen-Qian Yue, Zhen Li, Gui-Bin Bian, and Zeng-Guang Hou. 2022. SurgiNet: Pyramid Attention Aggregation and Class-wise Self-Distillation for Surgical Instrument Segmentation. Medical Image Analysis, Vol. 76 (2022), 102310. https://doi.org/10.1016/j.media.2021.102310
[46]
Daniil Pakhomov, Vittal Premachandran, Max Allan, Mahdi Azizian, and Nassir Navab. 2019. Deep Residual Learning for Instrument Segmentation in Robotic Surgery. In Machine Learning in Medical Imaging, Heung-Il Suk, Mingxia Liu, Pingkun Yan, and Chunfeng Lian (Eds.). Springer International Publishing, Cham, 566--573. https://doi.org/10.1007/978--3-030--32692-0_65
[47]
Liang Qiu, Changsheng Li, and Hongliang Ren. 2019. Real-time surgical instrument tracking in robot-assisted surgery using multi-domain convolutional neural network. Healthcare Technology Letters, Vol. 6, 6 (2019), 159--164. https://doi.org/10.1049%2Fhtl.2019.0068
[48]
Cristian da Costa Rocha, Nicolas Padoy, and Benoit Rosa. 2019. Self-Supervised Surgical Tool Segmentation using Kinematic Information. In 2019 International Conference on Robotics and Automation (ICRA). 8720--8726. https://doi.org/10.1109/ICRA.2019.8794334
[49]
Olaf Ronneberger, Philipp Fischer, and Thomas Brox. 2015. U-Net: Convolutional Networks for Biomedical Image Segmentation. In Medical Image Computing and Computer Assisted Intervention - MICCAI 2015, Nassir Navab, Joachim Hornegger, William M. Wells, and Alejandro F. Frangi (Eds.). Springer International Publishing, Cham, 234--241. https://doi.org/10.1007/978--3--319--24574--4_28
[50]
Tobias Rueckert, Daniel Rueckert, and Christoph Palm. 2024. Corrigendum to 'Methods and datasets for segmentation of minimally invasive surgical instruments in endoscopic images and videos: A review of the state of the art' [Comput. Biol. Med. 169 (2024) 107929]. Computers in Biology and Medicine, Vol. 170 (2024), 108027. https://doi.org/10.1016/j.compbiomed.2024.108027
[51]
Luca Sestini, Benoit Rosa, Elena De Momi, Giancarlo Ferrigno, and Nicolas Padoy. 2023. FUN-SIS: A Fully Unsupervised approach for Surgical Instrument Segmentation. Medical Image Analysis, Vol. 85 (2023), 102751. https://doi.org/10.1016/j.media.2023.102751
[52]
Wenting Shen, Yaonan Wang, Min Liu, Jiazheng Wang, Renjie Ding, Zhe Zhang, and Erik Meijering. 2023. Branch Aggregation Attention Network for Robotic Surgical Instrument Segmentation. IEEE Transactions on Medical Imaging, Vol. 42, 11 (2023), 3408--3419. https://doi.org/10.1109/TMI.2023.3288127
[53]
Jianbo Shi and J. Malik. 2000. Normalized cuts and image segmentation. IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 22, 8 (2000), 888--905. https://doi.org/10.1109/34.868688
[54]
Oriane Siméoni, Gilles Puy, Huy V. Vo, Simon Roburin, Spyros Gidaris, Andrei Bursuc, Patrick Pérez, Renaud Marlet, and Jean Ponce. 2021. Localizing Objects with Self-Supervised Transformers and no Labels. Proceedings of the British Machine Vision Conference (BMVC) (2021). https://doi.org/10.48550/arXiv.2109.14279 arxiv: 2109.14279 [cs.CV]
[55]
Andru P. Twinanda, Sherif Shehata, Didier Mutter, Jacques Marescaux, Michel de Mathelin, and Nicolas Padoy. 2017. EndoNet: A Deep Architecture for Recognition Tasks on Laparoscopic Videos. IEEE Transactions on Medical Imaging, Vol. 36, 1 (2017), 86--97. https://doi.org/10.1109/TMI.2016.2593957
[56]
Wouter Van Gansbeke, Simon Vandenhende, Stamatios Georgoulis, and Luc Van Gool. 2021. Unsupervised Semantic Segmentation by Contrasting Object Mask Proposals. In 2021 IEEE/CVF International Conference on Computer Vision (ICCV). 10032--10042. https://doi.org/10.1109/ICCV48922.2021.00990
[57]
Bowen Wang, Liangzhi Li, Yuta Nakashima, Ryo Kawasaki, Hajime Nagahara, and Yasushi Yagi. 2021. Noisy-LSTM: Improving Temporal Awareness for Video Semantic Segmentation. IEEE Access, Vol. 9 (2021), 46810--46820. https://doi.org/10.1109/ACCESS.2021.3067928
[58]
Jiacheng Wang, Yueming Jin, Liansheng Wang, Shuntian Cai, Pheng-Ann Heng, and Jing Qin. 2021. Efficient Global-Local Memory for Real-Time Instrument Segmentation of Robotic Surgical Video. In Medical Image Computing and Computer Assisted Intervention -- MICCAI 2021, Marleen de Bruijne, Philippe C. Cattin, Stéphane Cotin, Nicolas Padoy, Stefanie Speidel, Yefeng Zheng, and Caroline Essert (Eds.). Springer International Publishing, Cham, 341--351. https://doi.org/10.1007/978--3-030--87202--1_33
[59]
Xudong Wang, Rohit Girdhar, Stella X. Yu, and Ishan Misra. 2023. Cut and Learn for Unsupervised Object Detection and Instance Segmentation. In 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 3124--3134. https://doi.org/10.1109/CVPR52729.2023.00305
[60]
Yan Wang, Qiyuan Sun, Zhenzhong Liu, and Lin Gu. 2022. Visual detection and tracking algorithms for minimally invasive surgical instruments: A comprehensive review of the state-of-the-art. Robotics and Autonomous Systems, Vol. 149 (2022), 103945. https://doi.org/10.1016/j.robot.2021.103945
[61]
Wenxi Yue, Hongen Liao, Yong Xia, Vincent Lam, Jiebo Luo, and Zhiyong Wang. 2023. Cascade Multi-Level Transformer Network for Surgical Workflow Analysis. IEEE Transactions on Medical Imaging, Vol. 42, 10 (Oct 2023), 2817--2831. https://doi.org/10.1109/TMI.2023.3265354
[62]
Wenxi Yue, Jing Zhang, Kun Hu, Yong Xia, Jiebo Luo, and Zhiyong Wang. 2024. SurgicalSAM: Efficient Class Promptable Surgical Instrument Segmentation. Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 38, 7 (Mar. 2024), 6890--6898. https://doi.org/10.1609/aaai.v38i7.28514
[63]
Zixu Zhao, Yueming Jin, Junming Chen, Bo Lu, Chi-Fai Ng, Yun-Hui Liu, Qi Dou, and Pheng-Ann Heng. 2021. Anchor-guided online meta adaptation for fast one-Shot instrument segmentation from robotic surgical videos. Medical Image Analysis, Vol. 74 (2021), 102240. https://doi.org/10.1016/j.media.2021.102240
[64]
Zixu Zhao, Yueming Jin, Xiaojie Gao, Qi Dou, and Pheng-Ann Heng. 2020. Learning Motion Flows for Semi-supervised Instrument Segmentation from Robotic Surgical Video. In Medical Image Computing and Computer Assisted Intervention -- MICCAI 2020, Anne L. Martel, Purang Abolmaesumi, Danail Stoyanov, Diana Mateus, Maria A. Zuluaga, S. Kevin Zhou, Daniel Racoceanu, and Leo Joskowicz (Eds.). Springer International Publishing, Cham, 679--689. https://doi.org/10.1007/978--3-030--59716-0_65
[65]
Hongbo Zhou and Qiang Cheng. 2011. O(N) implicit subspace embedding for unsupervised multi-scale image segmentation. In Computer Vision and Pattern Recognition (CVPR). 2209--2215. https://doi.org/10.1109/CVPR.2011.5995606
[66]
Juan Carlos Ángeles Cerón, Gilberto Ochoa Ruiz, Leonardo Chang, and Sharib Ali. 2022. Real-time instance segmentation of surgical instruments using attention and multi-scale feature fusion. Medical Image Analysis, Vol. 81 (2022), 102569. https://doi.org/10.1016/j.media.2022.102569
Index Terms
- Revisiting Surgical Instrument Segmentation Without Human Intervention: A Graph Partitioning View
Recommendations
Unsupervised Surgical Instrument Segmentation via Anchor Generation and Semantic Diffusion
Medical Image Computing and Computer Assisted Intervention – MICCAI 2020AbstractSurgical instrument segmentation is a key component in developing context-aware operating rooms. Existing works on this task heavily rely on the supervision of a large amount of labeled data, which involve laborious and expensive human efforts. In ...
Co-generation and Segmentation for Generalized Surgical Instrument Segmentation on Unlabelled Data
Medical Image Computing and Computer Assisted Intervention – MICCAI 2021AbstractSurgical instrument segmentation for robot-assisted surgery is needed for accurate instrument tracking and augmented reality overlays. Therefore, the topic has been the subject of a number of recent papers in the CAI community. Deep learning-based ...
Discriminative metric learning for multi-view graph partitioning
We propose discriminative metric learning for multi-view graph partitioning.We envision the multi-view graph as an adaptive dynamic system.The intra-view connections and the inter-view couplings are interplayed.Extensive experiments have been conducted ...
Comments
Information & Contributors
Information
Published In
October 2024
85 pages
ISBN:9798400711954
DOI:10.1145/3688868
- Program Chairs:
- Xuequan Lu,
- Wenxi Yue,
- Imran Razzak,
- Kun Hu,
- Jinglei Lv,
- Sen Zhang,
- Junhui Hou,
- Zhiyong Wang,
- Jiebo Luo,
- Wei Xiang
Copyright © 2024 Owner/Author.
This work is licensed under a Creative Commons Attribution International 4.0 License.
Sponsors
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 31 October 2024
Check for updates
Author Tags
Qualifiers
- Research-article
Conference
MM '24
Sponsor:
MM '24: The 32nd ACM International Conference on Multimedia
October 28 - November 1, 2024
Melbourne VIC, Australia
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 124Total Downloads
- Downloads (Last 12 months)124
- Downloads (Last 6 weeks)51
Reflects downloads up to 27 Jan 2025
Other Metrics
Citations
View Options
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in