research-article

GHOSM: Graph-based Hybrid Outline and Skeleton Modelling for Shape Recognition

Authors:

Basheer Alwaely,

Charith AbhayaratneAuthors Info & Claims

ACM Transactions on Multimedia Computing, Communications and Applications, Volume 19, Issue 2s

Article No.: 86, Pages 1 - 23

https://doi.org/10.1145/3554922

Published: 17 February 2023 Publication History

Abstract

An efficient and accurate shape detection model plays a major role in many research areas. With the emergence of more complex shapes in real-life applications, shape recognition models need to capture the structure with more effective features to achieve high accuracy rates for shape recognition. This article presents a new method for 2D/3D shape recognition based on graph spectral domain handcrafted features, which are formulated by exploiting both an outline and a skeleton shape through the global outline and internal details. A fully connected graph is generated over the shape outline to capture the global outline representation while a hierarchically clustered graph with adaptive connectivity is formed on the skeleton to capture the structural descriptions of the shape. We demonstrate the ability of the Fiedler vector to provide the graph partitioning of the skeleton graph. The performance evaluation demonstrates the efficiency of the proposed method compared to state-of-the-art studies with increments of 4.09%, 2.2%, and 14.02% for 2D static hand gestures, 2D shapes, and 3D shapes, respectively.

References

[1]

A. Aghasi and J. Romberg. 2018. Extracting the principal shape components via convex programming. IEEE Trans. Image Process. 27, 7 (2018), 3513–3528.

[2]

M. Akimaliev and M. Demirci. 2015. Improving skeletal shape abstraction using multiple optimal solutions. Pattern Recogn. 48, 11 (2015), 3504–3515.

Digital Library

[3]

B. Alwaely and C. Abhayaratne. 2019. Graph spectral domain feature learning with application to in-air hand-drawn number and shape recognition. IEEE Access 7 (2019), 159661–159673.

[4]

B. Alwaely and C. Abhayaratne. 2019. Graph spectral domain features for static hand gesture recognition. In Proceedings of the European Signal Processing Conference (EUSIPCO’19). 1–5.

[5]

B. Alwaely and C. Abhayaratne. 2020. AGSF: Adaptive graph formulation and handcrafted graph spectral features for shape representation. IEEE Access 8, 1 (2020), 1–13.

[6]

M. A. Aowal, A. S. Zaman, S. M. Rahman, and D. Hatzinakos. 2014. Static hand gesture recognition using discriminative 2D Zernike moments. In Proceedings of the Region 10 Conference (TENCON’14). IEEE, 1–5.

[7]

M. Asad and C. Abhayaratne. 2013. Kinect depth stream pre-processing for hand gesture recognition. In Proceedings of the International Conference on Image Processing. IEEE, 3735–3739.

[8]

C. Aslan, A. Erdem, E. Erdem, and S. Tari. 2008. Disconnected skeleton: Shape at its absolute scale. IEEE Trans. Pattern Anal. Mach. Intell. 30, 12 (2008), 2188–2203.

Digital Library

[9]

V. Ayzenberg and S. F. Lourenco. 2019. Skeletal descriptions of shape provide unique perceptual information for object recognition. Sci. Rep. 9 (2019), 9359.

[10]

X. Bai, L. Latecki, and W. Liu. 2007. Skeleton pruning by contour partitioning with discrete curve evolution. IEEE Trans. Pattern Anal. Mach. Intell. 29, 3 (2007).

Digital Library

[11]

X. Bai, W. Liu, and Z. Tu. 2009. Integrating contour and skeleton for shape classification. In Proceedings of the International Conference on Computer Vision Workshops (ICCV’09). IEEE, 360–367.

[12]

N. Baker and P. J. Kellman. 2018. Abstract shape representation in human visual perception. J. Exper. Psychol.: Gen. 147, 9 (2018), 1295.

[13]

A. Barczak, N. Reyes, M. Abastillas, A. Piccio, and T. Susnjak. 2011. A new 2D static hand gesture colour image dataset for ASL gestures. Res. Lett. Info. Math. Sci. 15 (2011), 12–20.

[14]

S. L. Brincat and C. E. Connor. 2004. Underlying principles of visual shape selectivity in posterior inferotemporal cortex. Nature Neurosci. 7, 8 (2004), 880.

[15]

M. Carcassoni and E. Hancock. 2002. Alignment using spectral clusters. In Proceedings of the British Machine Vision Conference (BMVC’02). 1–10.

[16]

A. Chaouch, and M. Verroust-Blondet. 2007. A new descriptor for 2D depth image indexing and 3D model retrieval. In Proceedings of the International Conference on Image Processing, Vol. 6. IEEE, VI–373.

[17]

X. Chen, A. Golovinskiy, and T. Funkhouser. 2009. A benchmark for 3D mesh segmentation. ACM Trans. Graph. 28 (2009), 73.

Digital Library

[18]

C. E. Connor, S. L. Brincat, and A. Pasupathy. 2007. Transformation of shape information in the ventral pathway. Curr. Opin. Neurobiol. 17, 2 (2007), 140–147.

[19]

T. Cour, P. N. Srinivasan, and J. Shi. 2007. Balanced graph matching. Adv. Neural Info. Process. Syst. 19 (2007), 313.

[20]

G. E. da Silva and A. R. Backes. 2018. Characterizing 3D shapes: A complex network-based approach. In Proceedings of the European Signal Processing Conference (EUSIPCO’18). 608–612.

[21]

M. Demirci, A. Shokoufandeh, and S. Dickinson. 2009. Skeletal shape abstraction from examples. IEEE Trans. Pattern Anal. Mach. Intell. 31, 5 (2009), 944–952.

Digital Library

[22]

E. Dibra, S. Melchior, A. Balkis, T. Wolf, C. Oztireli, and M. Gross. 2018. Monocular RGB hand pose inference from unsupervised refinable nets. In Proceedings of the the IEEE Conference on Computer Vision and Pattern Recognition Workshops. 1075–1085.

[23]

F. Emmert-Streib, M. Dehmer, and Y. Shi. 2016. Fifty years of graph matching, network alignment and network comparison. Info. Sci. 346 (2016), 180–197.

Digital Library

[24]

P. Felzenszwalb and D. Huttenlocher. 2004. Efficient graph-based image segmentation. J. Comput. Vision 59, 2 (2004), 167–181.

[25]

M. Fiedler. 1975. A property of eigenvectors of nonnegative symmetric matrices and its application to graph theory. Czech. Math. J. 25, 4 (1975), 619–633.

[26]

Y. Guo, M. Bennamoun, F. Sohel, M. Lu, J. Wan, and N. M. Kwok. 2016. A comprehensive performance evaluation of 3D local feature descriptors. Int. J. Comput. Vision 116, 1 (2016), 66–89.

Digital Library

[27]

Z. Han, Z. Liu, C. Vong, Y. Liu, S. Bu, J. Han, and C. Chen. 2017. BoSCC: Bag of spatial context correlations for spatially enhanced 3D shape representation. IEEE Trans. Image Process. 26, 8 (2017), 3707–3720.

Digital Library

[28]

R. Horaud and H. Sossa. 1995. Polyhedral object recognition by indexing. Pattern Recogn. 28, 12 (1995), 1855–1870.

Digital Library

[29]

H. Jomma and A. Hussein. 2016. Circle views signature: A novel shape representation for shape recognition and retrieval. Can. J. Electric. Comput. Eng. 39, 4 (2016), 274–282.

[30]

L. Latecki, R. Lakamper, and T. Eckhardt. 2000. Shape descriptors for non-rigid shapes with a single closed contour. In Proceedings of the the Computer Vision and Pattern Recognition (CVPR’00), Vol. 1. IEEE, 424–429.

[31]

Guillaume Lavoué. 2012. Combination of bag-of-words descriptors for robust partial shape retrieval. Visual Computer 28, 9 (2012), 931–942.

Digital Library

[32]

M. Leordeanu and M. Hebert. 2005. A spectral technique for correspondence problems using pairwise constraints. In Proceedings of the International Conference on Computer Vision (ICCV’05). 1482–1489.

Digital Library

[33]

S. Li, H. Lu, and X. Shao. 2014. Human body segmentation via data-driven graph cut. IEEE Trans. Cybernet. 44, 11 (2014), 2099–2108.

[34]

Y. Li, R. Bu, M. Sun, W. Wu, X. Di, and B. Chen. 2018. Pointcnn: Convolution on x-transformed points. In Advances in Neural Information Processing Systems. MIT Press, 820–830.

[35]

J. Liu, S. Song, C. Liu, Y. Li, and Y. Hu. 2020. A benchmark dataset and comparison study for multi-modal human action analytics. ACM Trans. Multimedia Comput. Commun. Appl. 16, 2 (2020), 1–24.

Digital Library

[36]

L. Luo, C. Shen, X. Liu, and C. Zhang. 2015. A computational model of the short-cut rule for 2D shape decomposition. IEEE Trans. Image Process. 24, 1 (2015), 273–283.

[37]

X. Luo, F. Wong, and H. Hu. 2020. FIN: Feature integrated network for object detection. ACM Trans. Multimedia Comput. Commun. Appl. 16, 2 (2020), 1–18.

Digital Library

[38]

T. Martinetz and K. Schulten. 1991. A neural-gas network learns topologies. In Proceedings of the International Conference on Artificial Neural Networks. 397–402.

[39]

M. Masoumi and A. Hamza. 2017. Spectral shape classification: A deep learning approach. J. Visual Commun. Image Represent. 43 (2017), 198–211.

Digital Library

[40]

D. Maturana and S. Scherer. 2015. Voxnet: A 3D convolutional neural network for real-time object recognition. In Proceedings of the international conference on Intelligent Robots and Systems (IROS’15). IEEE, 922–928.

Digital Library

[41]

A. Memo, L. Minto, and P. Zanuttigh. 2015. Exploiting silhouette descriptors and synthetic data for hand gesture recognition. In Proceedings of the Conference on Smart Tools & Apps for Graphics (STAG’15).

[42]

S. O. Murray, B. A. Olshausen, and D. L. Woods. 2003. Processing shape, motion and three-dimensional shape-from-motion in the human cortex. Cerebral Cortex 13, 5 (2003), 508–516.

[43]

L. Nanni, S. Brahnam, and A. Lumini. 2012. Local phase quantization descriptor for improving shape retrieval/classification. Pattern Recogn. Lett. 33, 16 (2012), 2254–2260.

Digital Library

[44]

C. R. Qi, L. Yi, H. Su, and L. J. Guibas. 2017. Pointnet++: Deep hierarchical feature learning on point sets in a metric space. In Advances in Neural Information Processing Systems. MIT Press, 5099–5108.

Digital Library

[45]

Z. Ren, J. Yuan, J. Meng, and Z. Zhang. 2013. Robust part-based hand gesture recognition using Kinect sensor. IEEE Trans. Multimedia 15, 5 (Aug.2013), 1110–1120.

Digital Library

[46]

I. Reppa and E. C. Leek. 2019. Surface diagnosticity predicts the high-level representation of regular and irregular object shape in human vision. Attent. Percept. Psychophys. 81 (2019), 1589–1608.

[47]

P. Roberto, F. Emanuele, Z. Primo, M. Adriano, L. Jelena, and P. Marina. 2019. Design, large-scale usage testing, and important metrics for augmented reality gaming applications. ACM Trans. Multimedia Comput. Commun. Appl. 15, 2 (2019), 1–18.

Digital Library

[48]

R. B. Rusu, N. Blodow, and M. Beetz. 2009. Fast point feature histograms (FPFH) for 3D registration. In Proceedings of the International Conference on Robotics and Automation. IEEE, 3212–3217.

[49]

R. B. Rusu, N. Blodow, Z. C. Marton, and M. Beetz. 2008. Aligning point cloud views using persistent feature histograms. In Proceedings of the the International Conference on Intelligent Robots and Systems. IEEE, 3384–3391.

[50]

T. Sebastian, P. Klein, and B. Kimia. 2004. Recognition of shapes by editing their shock graphs. IEEE Trans. Pattern Anal. Mach. Intell. 26, 5 (2004), 550–571.

Digital Library

[51]

A. Shamaie and A. Sutherland. 2001. Graph-based matching of occluded hand gestures. In Proceedings of the Applied Imagery Pattern Recognition Workshop. 67–73.

[52]

S. Shapiro and J. Brady. 1992. Feature-based correspondence: An eigenvector approach. Image Vision Comput. 10, 5 (1992), 283–288.

Digital Library

[53]

D. I. Shuman, S. K. Narang, P. Frossard, A. Ortega, and P. Vandergheynst. 2013. The emerging field of signal processing on graphs: Extending high-dimensional data analysis to networks and other irregular domains. IEEE Signal Process. Mag. 30, 3 (May2013), 83–98.

[54]

K. Siddiqi, J. Zhang, D. Macrini, A. Shokoufandeh, S. Bouix, and S. Dickinson. 2008. Retrieving articulated 3D models using medial surfaces. Mach. Vision Appl. 19, 4 (2008), 261–275.

Digital Library

[55]

A. Sinha, J. Bai, and K. Ramani. 2016. Deep learning 3D shape surfaces using geometry images. In Proceedings of the European Conference on Computer Vision. Springer, 223–240.

[56]

H. Su, S. Maji, E. Kalogerakis, and E. Miller. 2015. Multi-view convolutional neural networks for 3D shape recognition. In Proceedings of the International Conference on Computer Vision. 945–953.

Digital Library

[57]

B. Taati and Mi Greenspan. 2011. Local shape descriptor selection for object recognition in range data. Comput. Vision Image Understand. 115, 5 (2011), 681–694.

Digital Library

[58]

Y. S. Tan, K. M. Lim, C. Tee, C. P. Lee, and C. Y. Low. 2020. Convolutional neural network with spatial pyramid pooling for hand gesture recognition. Neural Comput. Appl. 33 (2021), 5339–5351.

[59]

S. Umeyama. 1988. An eigendecomposition approach to weighted graph matching problems. IEEE Trans. Pattern Anal. Mach. Intell. 10, 5 (Sept.1988), 695–703.

Digital Library

[60]

D. Vranic and D. Saupe. 2002. Description of 3D-shape using a complex function on the sphere. In Proceedings of the International Conference on Multimedia and Expo, Vol. 1. IEEE, 177–180.

[61]

B. Wang, W. Shen, W. Liu, and X. Bai. 2010. Shape classification using tree-unions. In Proceedings of the International Conference on of Pattern Recognition(ICPR’10). 983–986.

Digital Library

[62]

C. Wang, Z. Liu, and S. C. Chan. 2015. Superpixel-based hand gesture recognition with Kinect depth camera. IEEE Trans. Multimedia 17, 1 (2015), 29–39.

[63]

X. Wang, Y. Tian, X. Zhao, T. Yang, J. Gelernter, J. Wang, G. Cheng, and W. Hu. 2020. Improving multiperson pose estimation by mask-aware deep reinforcement learning. ACM Trans. Multimedia Comput. Commun. Appl. 16, 3 (2020), 1–18.

Digital Library

[64]

R. Watt and D. Andrews. 1982. Contour curvature analysis: Hyperacuities in the discrimination of detailed shape. Vision Res. 22, 4 (1982), 449–460.

[65]

Z. Wu, S. Song, A. Khosla, F. Yu, L. Zhang, X. Tang, and J. Xiao. 2015. 3D shapenets: A deep representation for volumetric shapes. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR’15). 1912–1920.

[66]

J. Xie, Y. Fang, F. Zhu, and E. Wong. 2015. Deepshape: Deep learned shape descriptor for 3D shape matching and retrieval. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR’15). 1275–1283.

[67]

Z. Yasseen, A. Verroust, and A. Nasri. 2016. Shape matching by part alignment using extended chordal axis transform. Pattern Recogn. 57 (2016), 115–135.

Digital Library

[68]

C. Zahn and R. Roskies. 1972. Fourier descriptors for plane closed curves. IEEE Trans. Comput. 100, 3 (1972), 269–281.

Digital Library

[69]

Y. Zheng, B. Guo, Y. Yan, and W. He. 2019. O2O method for fast 2D shape retrieval. IEEE Trans. Image Process. 28, 11 (2019), 5366–5378.

Digital Library

[70]

S. Zhi, Y. Liu, X. Li, and Y. Guo. 2018. Toward real-time 3D object recognition: A lightweight volumetric CNN framework using multitask learning. Comput. Graph. 71 (2018), 199–207.

[71]

F. Zhou and F. De La Torre. 2015. Factorized graph matching. IEEE Trans. Pattern Anal. Mach. Intell. 38, 9 (2015), 1774–1789.

Digital Library

[72]

Y. Zhou and F. Zeng. 2017. 2D compressive sensing and multi-feature fusion for effective 3D shape retrieval. Info. Sci. 409 (2017), 101–120.

Cited By

Lin PLi CChen SHuangfu JYuan W(2024)Intelligent Gesture Recognition Based on Screen Reflectance Multi-Band Spectral FeaturesSensors10.3390/s2417551924:17(5519)Online publication date: 26-Aug-2024
https://doi.org/10.3390/s24175519
Giveki DSoltanshahi MRastegar H(2024)Shape classification using a new shape descriptor and multi-view learningDisplays10.1016/j.displa.2023.10263682(102636)Online publication date: Apr-2024
https://doi.org/10.1016/j.displa.2023.102636
Flor AAanjaneya M(2024)Spectral reordering for faster elasticity simulationsThe Visual Computer: International Journal of Computer Graphics10.1007/s00371-024-03513-040:7(5067-5077)Online publication date: 1-Jul-2024
https://dl.acm.org/doi/10.1007/s00371-024-03513-0

Index Terms

GHOSM: Graph-based Hybrid Outline and Skeleton Modelling for Shape Recognition
1. Computing methodologies
  1. Artificial intelligence
    1. Computer vision
      1. Computer vision problems
        Object recognition
      2. Computer vision representations
        Hierarchical representations
        Shape representations
2. Mathematics of computing
  1. Discrete mathematics
    1. Graph theory
      1. Graphs and surfaces

Recommendations

Skeleton graph matching based on critical points using path similarity
ACCV'09: Proceedings of the 9th Asian conference on Computer Vision - Volume Part III

This paper proposes a novel graph matching algorithm based on skeletons and applies it to shape recognition based on object silhouettes The main idea is to match the critical points (junction points and end points) on skeleton graphs by comparing the ...
Shape recognition by bag of skeleton-associated contour parts

We associate a contour point with an object thickness value.We propose a new descriptor to combine both contour and skeleton information.Our method achieves the state-of-the-arts results on several shape dataset. Contour and skeleton are two ...
Error-tolerant geometric graph similarity and matching
Highlights
- An intuitive approach to graph similarity and matching using geometric graphs.
- ...
Abstract
Geometric graph matching is the process of evaluating the similarity between geometric graphs, where each vertex has an associated coordinate point in a plane. Exact matching computes a strict correspondence between two geometric ...

Comments

Information & Contributors

Information

Published In

cover image ACM Transactions on Multimedia Computing, Communications, and Applications

ACM Transactions on Multimedia Computing, Communications, and Applications Volume 19, Issue 2s

April 2023

545 pages

ISSN:1551-6857

EISSN:1551-6865

DOI:10.1145/3572861

Editor:
Abdulmotaleb El Saddik
Mohamed Bin Zayed University of Artificial Intelligence, UAE and University of Ottawa, Canada

Issue’s Table of Contents

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 17 February 2023

Online AM: 04 August 2022

Accepted: 27 July 2022

Revised: 25 April 2022

Received: 04 February 2021

Published in TOMM Volume 19, Issue 2s

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

2
Total Citations
View Citations
192
Total Downloads

Downloads (Last 12 months)81
Downloads (Last 6 weeks)2

Reflects downloads up to 11 Sep 2024

Other Metrics

View Author Metrics

Citations

Cited By

Lin PLi CChen SHuangfu JYuan W(2024)Intelligent Gesture Recognition Based on Screen Reflectance Multi-Band Spectral FeaturesSensors10.3390/s2417551924:17(5519)Online publication date: 26-Aug-2024
https://doi.org/10.3390/s24175519
Giveki DSoltanshahi MRastegar H(2024)Shape classification using a new shape descriptor and multi-view learningDisplays10.1016/j.displa.2023.10263682(102636)Online publication date: Apr-2024
https://doi.org/10.1016/j.displa.2023.102636
Flor AAanjaneya M(2024)Spectral reordering for faster elasticity simulationsThe Visual Computer: International Journal of Computer Graphics10.1007/s00371-024-03513-040:7(5067-5077)Online publication date: 1-Jul-2024
https://dl.acm.org/doi/10.1007/s00371-024-03513-0

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Full Text

View this article in Full Text.

HTML Format

View this article in HTML Format.

Media

Figures

Other

Tables

View full text|Download PDF

View Issue’s Table of Contents