Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
Skip to main content

Advertisement

Springer Nature Link
Log in

3D indirect shape retrieval based on hand interaction

  • Original Article
  • Published:
The Visual Computer Aims and scope Submit manuscript

Abstract

In this work, we present a novel 3D indirect shape analysis method which successfully retrieves 3D shapes based on hand–object interaction. To this end, the human hand information is first transferred to the virtual environment by the Leap Motion controller. Position-, angle- and intersection-based novel features of the hand and fingers are used for this part. In the guidance of these features that define the way humans grab objects, a support vector machine (SVM) classifier is trained. Experiments validate that SVM results are useful for retrieval of 3D shapes. We also compare the retrieval performance of our method with an interaction-based indirect method based on the Data Glove controller as well as a direct method based on 3D shape distribution histograms. These comparisons reveal different advantages of our method, which are (i) being lower-cost and more accurate compared to the Data Glove, and (ii) being more discriminative compared to a direct approach. We finally note that our algorithm is rigid-motion invariant and able to explore databases of arbitrarily represented 3D shapes.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Similar content being viewed by others

Explore related subjects

Discover the latest articles, news and stories from top researchers in related subjects.

References

  1. Funkhouser, T., Min, P., Kazhdan, M., Chen, J., Halderman, A., Dobkin, D., Jacobs, D.: A search engine for 3D models. ACM Trans. Gr. 22(1), 83–105 (2003)

    Article  Google Scholar 

  2. Passalis, G., Theoharis, T., Kakadiaris, I.: Ptk: a novel depth buffer-based shape descriptor for three-dimensional object retrieval. Vis. Comput. 23, 5–14 (2007)

    Article  Google Scholar 

  3. Tangelder, J., Veltkamp, R.: A survey of content based 3D shape retrieval methods. Multimed. Tools Appl. 39(3), 441–471 (2008)

    Article  Google Scholar 

  4. Osada, R., Funkhouser, T., Chazelle, B., Dobkin, D.: Shape distributions. ACM Trans. Gr. 21(4), 807–832 (2002)

    Article  MathSciNet  Google Scholar 

  5. Paquet, E., Rioux, M., Murching, A., Naveen, T.: Description of shape information for 2-D and 3-D objects. Signal Process. Image Commun. 16(1–2), 103–122 (2000)

    Article  Google Scholar 

  6. Zheng, Y., Neo, Y., Chua, T., Tian, Q.: Toward a higher-level visual representation for object-based image retrieval. Vis. Comput. 25, 13–23 (2009)

    Article  Google Scholar 

  7. Novotni, M., Klein, R.: Shape retrieval using 3D Zernike descriptors. Comput. Aided Des. 36(11), 1047–1062 (2004)

    Article  Google Scholar 

  8. Canterakis, N.: 3D Zernike moments and Zernike affine invariants for 3D image analysis and recognition. In: 11th Scandinavian Conference on Image Analysis, pp. 85–93 (1999)

  9. Chen, D., Tian, X., Shen, Y., Ouhyoung, M.: On visual similarity based 3D model retrieval. Comput. Graph. Forum 22(3), 223–232 (2003)

    Article  Google Scholar 

  10. Shilane, P., Funkhouser, T.: Selecting distinctive 3D shape descriptors for similarity retrieval. In: Proceedings of IEEE International Conference on Shape Modeling and Applications, SMI 2006, p. 18 (2006)

  11. Wang, P., Liu, Y., Guo, Y., Sun, C., Tong, X.: O-CNN: octree-based convolutional neural networks for 3D shape analysis. ACM Trans. Gr. 36(4), 1–11 (2017)

    Google Scholar 

  12. Gao, Y., Yang, Y., Dai, Q., Zhang, N.: Representative views re-ranking for 3d model retrieval with multi-bipartite graph reinforcement model. In: Proceedings of the 18th ACM International Conference on Multimedia, pp. 947–950 (2010)

  13. Gao, Y., Wang, M., Shen, J., Dai, Q., Zhang, N.: Intelligent query: open another door to 3d object retrieval. In: Proceedings of the 18th ACM International Conference on Multimedia, ACM, pp. 1711–1714 (2010)

  14. Bai, S., Bai, X., Zhou, Z., Zhang, Z., Tian, Q., Latecki, L.: GIFT: towards scalable 3D shape retrieval. IEEE Trans. Multimed. 19(6), 1257–1271 (2017)

    Article  Google Scholar 

  15. Gao, Y., Wang, M., Tao, D., Ji, R., Dai, Q.: 3-D object retrieval and recognition with hypergraph analysis. IEEE Trans. Image Process. 21(9), 4290–4303 (2012)

    Article  MathSciNet  Google Scholar 

  16. Gao, Y., Tang, J., Hong, R., Yan, S., Dai, Q., Zhang, N., Chua, T.: Camera constraint-free view-based 3-D object retrieval. IEEE Trans. Image Process. 21(4), 2269–2281 (2012)

    Article  MathSciNet  Google Scholar 

  17. Leifman, G., Meir, R., Tal, A.: Semantic-oriented 3d shape retrieval using relevance feedback. Vis. Comput. 21, 865–875 (2005)

    Article  Google Scholar 

  18. Lian, Z., Godil, A., Bustos, B., Daoudi, M., Hermans, J., Kawamura, S., Kurita, Y., Lavoué, G., Nguyen, H., Ohbuchi, R., Ohkita, Y., Ohishi, Y., Porikli, F., Reuter, M., Sipiran, I., Smeets, D., Suetens, P., Tabia, H., Vandermeulen, D.: A comparison of methods for non-rigid 3D shape retrieval. Pattern Recognit. 46(1), 449–461 (2013)

    Article  Google Scholar 

  19. Sipiran, I., Meruane, R., Bustos, B., Schreck, T., Li, B., Lu, Y., Johan, H.: A benchmark of simulated range images for partial shape retrieval. Vis. Comput. 30(11), 1293–1308 (2014)

    Article  Google Scholar 

  20. Hilaga, M., Shinagawa, Y.: Topology matching for fully automatic similarity estimation of 3D shapes. In: Proceedings of the 28th Annual Conference on Computer Graphics and Interactive Techniques, pp. 203–212 (2001)

  21. Sundar, H., Silver, D., Gagvani, N., Dickinson, S.: Skeleton based shape matching and retrieval. In: Proceedings of SMI 2003: Shape Modeling International, pp. 130–139 (2003)

  22. Jain, V., Zhang, H.: A spectral approach to shape-based retrieval of articulated 3D models. CAD Comput. Aided Des. 39(5), 398–407 (2007)

    Article  Google Scholar 

  23. Reuter, M., Wolter, F., Peinecke, N.: Laplace-spectra as fingerprints for shape matching. In: SPM ’05 Proceedings of the 2005 ACM Symposium on Solid and Physical Modeling, vol. 1, no. 212, pp. 101–106 (2005)

  24. Sahillioğlu, Y., Sezgin, M.: Sketch-based articulated 3d shape retrieval. IEEE Comput. Gr. Appl. 37(6), 88–101 (2017)

    Article  Google Scholar 

  25. Sahillioğlu, Y., Yemez, Y.: Minimum-distortion isometric shape correspondence using em algorithm. IEEE Trans. Pattern Anal. Mach. Intell. 34(11), 2203–2215 (2012)

    Article  Google Scholar 

  26. Memoli, F., Sapiro, G.: A theoretical and computational framework for isometry invariant recognition of point cloud data. Found. Comput. Math. 5(3), 313–347 (2005)

    Article  MathSciNet  Google Scholar 

  27. Bronstein, A.M., Bronstein, M.M., Guibas, L.J., Ovsjanikov, M.: Shape google: geometric words and expressions for invariant shape retrieval. ACM Trans. Gr. 30(1), 1 (2011)

    Article  Google Scholar 

  28. Sahillioğlu, Y.: A shape deformation algorithm for constrained multidimensional scaling. Comput. Gr. 53, 156–165 (2015)

    Article  Google Scholar 

  29. Sahillioğlu, Y., Kavan, L.: Detail-preserving mesh unfolding for non-rigid shape retrieval. ACM Trans. Gr. 35(3), 27 (2016)

    Article  Google Scholar 

  30. Pickup, D., Liu, J., Sun, X., Rosin, P., Martin, R., Cheng, Z., Lian, Z., Nie, S., Jin, L., Shami, G., Sahillioğlu, Y., Kavan, L.: An evaluation of canonical forms for non-rigid 3d shape retrieval. Gr. Models 97, 17–29 (2018)

    Article  MathSciNet  Google Scholar 

  31. Hu, J., Hua, J.: Salient spectral geometric features for shape matching and retrieval. Vis. Comput. 25, 667–675 (2009)

    Article  Google Scholar 

  32. Li, C., Hamza, A.: A multiresolution descriptor for deformable 3d shape retrieval. Vis. Comput. 29, 513–524 (2013)

    Article  Google Scholar 

  33. Toldo, R., Castellani, U., Fusiello, A.: The bag of words approach for retrieval and categorization of 3d objects. Vis. Comput. 26, 1257–1268 (2010)

    Article  Google Scholar 

  34. Lavoué, G.: Combination of bag-of-words descriptors for robust partial shape retrieval. Vis. Comput. 28, 931–942 (2012)

    Article  Google Scholar 

  35. Fan, Q., Shen, X., Hu, Y.: Detail-preserved real-time hand motion regression from depth. Vis. Comput. 34, 1145–1154 (2018)

    Article  Google Scholar 

  36. Abbasi, A., Kalkan, S., Sahillioğlu, Y.: Deep 3d semantic scene extrapolation. Vis. Comput. (2018). https://doi.org/10.1007/s00371-018-1586-7

    Article  Google Scholar 

  37. Xie, J., Fang, Y., Zhu, F., Wong, E.: Deepshape: deep learned shape descriptor for 3D shape matching and retrieval. In: Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, vol. 07, pp. 1275–1283 (2015)

  38. Liu, Z., Xie, C., Bu, S., Wang, X., Han, J., Lin, H., Zhang, H.: Indirect shape analysis for 3D shape retrieval. Comput. Gr. 46, 110–116 (2015)

    Article  Google Scholar 

  39. Kim, V.G., Chaudhuri, S., Guibas, L., Funkhouser, T.: Shape2pose: human-centric shape analysis. ACM Trans. Gr. 33(4), 120 (2014)

    Google Scholar 

  40. Gibson, J.: The theory of affordances. In: Perceiving Acting, and Knowing, pp. 127–142 (1977)

  41. Hu, R., Zhu, C., van Kaick, O., Liu, L., Shamir, A., Zhang, H.: Interaction context (icon): towards a geometric functionality descriptor. ACM Trans. Gr. 34(4), 83:1–83:12 (2015)

    Article  Google Scholar 

  42. Zhu, Y., Fathi, A., Fei-Fei, L.: Reasoning about object affordances in a knowledge base representation. In: Lecture Notes in Computer Science (Including Subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 8690 LNCS, pp. 408–424 (2014)

    Chapter  Google Scholar 

  43. Bar-Aviv, E., Rivlin, E.: Functional 3D object classification using simulation of embodied agent. In: Proceedings of the British Machine Vision Conference, pp. 32.1–32.10 (2006)

  44. 5DT: 5DT Data Glove Ultra Series User Manual

  45. Osada, R., Funkhouser, T., Chazelle, B., Dobkin, D.: Matching 3D models with shape distributions. In: International Conference on Shape Modeling and Applications, pp. 154–166 (2001)

Download references

Acknowledgements

This work has been supported by TUBITAK under the project EEEAG-115E471.

Author information

Authors and Affiliations

  1. Game Technologies Department, Middle East Technical University, Ankara, Turkey

    Erdem Can Irmak

  2. Computer Engineering Department, Middle East Technical University, Ankara, Turkey

    Yusuf Sahillioğlu

Authors
  1. Erdem Can Irmak
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yusuf Sahillioğlu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Erdem Can Irmak.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Irmak, E.C., Sahillioğlu, Y. 3D indirect shape retrieval based on hand interaction. Vis Comput 36, 5–17 (2020). https://doi.org/10.1007/s00371-018-1597-4

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00371-018-1597-4

Keywords

Search

Navigation