Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
skip to main content
research-article

Using grammars for pattern recognition in images: A systematic review

Published: 01 November 2013 Publication History

Abstract

Grammars are widely used to describe string languages such as programming and natural languages and, more recently, biosequences. Moreover, since the 1980s grammars have been used in computer vision and related areas. Some factors accountable for this increasing use regard its relatively simple understanding and its ability to represent some semantic pattern models found in images, both spatially and temporally. The objective of this article is to present an overview regarding the use of syntactic pattern recognition methods in image representations in several applications. To achieve this purpose, we used a systematic review process to investigate the main digital libraries in the area and to document the phases of the study in order to allow the auditing and further investigation. The results indicated that in some of the studies retrieved, manually created grammars were used to comply with a particular purpose. Other studies performed a learning process of the grammatical rules. In addition, this article also points out still unexplored research opportunities in the literature.

References

[1]
Angluin, D. 1992. Computational learning theory: Survey and selected bibliography. In Proceedings of the 24th Annual ACM Symposium on Theory of Computing (STOC'92). ACM, New York, 351--369.
[2]
Böhm, C., Berchtold, S., and Keim, D. A. 2001. Searching in high-dimensional spaces: Index structures for improving the performance of multimedia databases. ACM Comput. Surv. 33, 3, 322--373.
[3]
Chanda, G. and Dellaert, F. 2004. Grammatical Methods in Computer Vision: An Overview. Tech. rep. Georgia Tech Institute of Technology.
[4]
Chen, H., Xu, Z. J., Liu, Z. Q., and Zhu, S. C. 2006. Composite templates for cloth modeling and sketching. In Proceedings of the 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Vol. 1 (CVPR'06). IEEE Computer Society, Washington, DC, 943--950.
[5]
Christensen, H. I., Matas, J., and Kittler, J. 1996. Using grammars for scene interpretation. In Proceedings of the International Conference on Image Processing, vol. 1. 793--796
[6]
Doi, K. 2007. Computer-aided diagnosis in medical imaging: Historical review, current status and future potential. Comput. Med. Imaging Graphics 31, 4--5, 191--211.
[7]
Fernau, H. 2003. Parallel grammars: A phenomenology. Grammars 6, 1, 25--87.
[8]
Ferreira, M. J., Santos, C. P., and Monteiro, J. 2007a. Texture cue based tracking system using wavelet transform and a fuzzy grammar. In Proceedings of the 5th IEEE International Conference on Industrial Informatics, vol. 1, 393--398.
[9]
Ferreira, M. J., Santos, C. P., and Monteiro, J. 2007b. Texture segmentation based on fuzzy grammar for cork parquet quality control. In Proceedings of the IEEE International Symposium on Industrial Electronics (ISIE'07). 1832--1837.
[10]
Ferreira, M., Santos, C., and Monteiro, J. 2009. Cork parquet quality control vision system based on texture segmentation and fuzzy grammar. IEEE Trans. Ind. Electron. 56, 3, 756--765.
[11]
Gao, J., Ding, X., and Zheng, J. 2000. Image pattern recognition based on examples—A combined statistical and structural-syntactic approach. In Advances in Pattern Recognition, F. J. Ferri, J. M. Inesta, A. Amin, and P. Pudil, Eds., Lecture Notes in Computer Science, vol. 1876. Springer, Berlin, 57--66.
[12]
Gidas, B. and Zelic, A. 1997. Object recognition via hierarchical syntactic models. In Proceedingsof the 13th International Conference on Digital Signal Processing (DSP'97), vol. 1, 315--318.
[13]
Glomb, P. 2007. Image language terminal symbols from feature analysis. In Proceedings of the IEEE International Workshop on Imaging Systems and Techniques (IST'07). 1--6.
[14]
Hamdi, S., Abdallah, A. B., and Bedoui, M. H. 2012. Grammar-based image segmentation and automatic area estimation. In Proceedings of the 16th IEEE Mediterranean Electrotechnical Conference (MELECON'12). 356--359.
[15]
Han, F. and Zhu, S.-C. 2005. Bottom-up/top-down image parsing by attribute graph grammar. In Computer Vision, 2005. ICCV 2005. Tenth IEEE International Conference on, vol. 2. Beijing, China, 1778--1785, vol. 2. 1550--5499.
[16]
Han, F. and Zhu, S.-C. 2009. Bottom-up/top-down image parsing with attribute grammar. IEEE Trans. Pattern Anal. Mach. Intell. 31, 1, 59--73.
[17]
Hemberg, M. and OReilly, U.-M. 2004. Extending grammatical evolution to evolve digital surfaces with Genr8. In Genetic Programming, M. Keijzer, U.-M. OReilly, S. Lucas, E. Costa, and T. Soule, Eds., Lecture Notes in Computer Science, vol. 3003. Springer, Berlin, 299--308.
[18]
Hingway, S. P. and Bhurchandi, K. M. 2011. A simple graph theoretic approach for object recognition. In Proceedings of the 4th International Conference on Emerging Trends in Engineering and Technology (ICETET'11). 200--205.
[19]
Jin, Y. and Geman, S. 2006. Context and hierarchy in a probabilistic image model. In Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, vol. 2, 2145--2152.
[20]
Kanungo, T. and Mao, S. 2003. Stochastic language models for style-directed layout analysis of document images. IEEE Trans. Image Process. 12, 5, 583--596.
[21]
Kong, J., Barkol, O., Bergman, R., Pnueli, A., Schein, S., Zhang, K., and Zhao, C. 2012. Web interface interpretation using graph grammars. IEEE Trans. Syst. Man Cybern., Part C Appl. Rev. 42, 4, 590--602.
[22]
Lin, W.-C. and Fu, K.-S. 1986. A syntactic approach to three-dimensional object recognition. IEEE Trans. Syst. Man Cybern. 16, 405--422.
[23]
Luo, P., He, J., Lin, L., and Chao, H. 2009. Hierarchical 3D perception from a single image. In Proceedings of the 16th IEEE International Conference on Image processing (ICIP'09). IEEE, Los Alamitos, CA, 4209--4212.
[24]
Mäkinen, E. 1992. On the structural grammatical inference problem for some classes of context-free grammars. Inf. Process. Lett. 42, 1, 1--5.
[25]
Mao, S., Rosenfeld, A., and Kanungo, T. 2003. Stochastic attributed K-d tree modeling of technical paper title pages. In Proceedings of the 2003 International Conference on Image Processing (ICIP'03). 533--536.
[26]
Mas, J., Jorge, J. A., Sanchez, G., and Llados, J. 2008. Representing and parsing sketched symbols using adjacency grammars and a grid-directed parser. In Graphics Recognition. Recent Advances and New Opportunities, W. Liu, J. Llados, and J.-M. Ogier, Eds., Lecture Notes in Computer Science, vol. 5046. Springer, Berlin, 169--180.
[27]
Mas, J., Sanchez, G., and Llados, J. 2005. An adjacency grammar to recognize symbols and gestures in a digital pen framework. In Pattern Recognition and Image Analysis, J. S. Marques, N. P. de la Blanca, and P. Pina, Eds., Lecture Notes in Computer Science, vol. 3523. Springer, Berlin, 115--122.
[28]
Nunes, F. L. S., Schiabel, H., and Goes, C. E. 2007. Contrast enhancement in dense breast images to aid clustered microcalcifications detection. J. Digital Imaging 20, 1, 53--66.
[29]
Ogiela, L., Ogiela, M. R., and Tadeusiewicz, R. 2009. Mathematical linguistics in cognitive medical image interpretation systems. J. Math. Imaging Vision 34, 3, 328--340.
[30]
Ogiela, L., Tadeusiewicz, R., and Ogiela, M. R. 2008. Cognitive modeling in medical pattern semantic understanding. In Proceedings of the International Conference on Multimedia and Ubiquitous Engineering (MUE'08). 15--18.
[31]
Parag, T., Bahlmann, C., Shet, V., and Singh, M. 2012. A grammar for hierarchical object descriptions in logic programs. In Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops (CVPRW'12). 33--38.
[32]
Peng, S., Liu, L., Yang, X., and Sang, N. 2008. A database schema for large scale annotated image dataset. In Proceedings of the Congress on Image and Signal Processing (CISP'08). 57--62.
[33]
Petrakis, E. G. M., Faloutsos, C., and Lin, K.-I. 2002. ImageMap: An image indexing method based on spatial similarity. IEEE Trans. Knowl. Data Eng. 14, 5, 979--987.
[34]
Prusinkiewicz, P., Lindenmayer, A., and Hanan, J. 1988. Development models of herbaceous plants for computer imagery purposes. SIGGRAPH Comput. Graph. 22, 4, 141--150.
[35]
Qu, H., Zhu, Q., Zeng, L., Guo, M., and Lu, Z. 2008. Automata-based L-Grammar extraction from multiple images for virtual plants. In Proceedings of the 3rd International Conference on Bio-Inspired Computing: Theories and Applications (BICTA'08). 89--96.
[36]
Reddy, H. T., Karibasappa, K., and Damodaram, A. 2009. Probabilistic parser for face detection. In Proceeding of International Conference on Methods and Models in Computer Science (ICM2CS'09). 1--7.
[37]
Ron, D., Singer, Y., and Tishby, N. 1995. On the learnability and usage of acyclic probabilistic finite automata. In Proceedings of the 8th Annual Conference on Computational Learning Theory (COLT'95). ACM, New York, 31--40.
[38]
Rothrock, B. and Zhu, S.-C. 2011. Human parsing using stochastic and-or grammars and rich appearances. In Proceedings of the 2011 IEEE International Conference on Computer Vision Workshops (ICCV Workshops'11). 640--647.
[39]
Sainz, M. and Sanfeliu, A. 1996. Learning bidimensional context-dependent models using a context-sensitive language. In Proceedings of the 13th International Conference on Pattern Recognition, vol. 4, 565--569.
[40]
Sakakibara, Y. 1992. Efficient learning of context-free grammars from positive structural examples. Inf. Comput. 97, 1, 23--60.
[41]
Sakakibara, Y. 1995. Grammatical inference: An old and new paradigm. In Algorithmic Learning Theory, K. P. Jantke, T. Shinohara, and T. Zeugmann, Eds., Lecture Notes in Computer Science, vol. 997. Springer, Berlin, 1--24.
[42]
Sakakibara, Y. 2005. Learning context-free grammars using tabular representations. Pattern Recogn. 38, 9, 1372--1383.
[43]
Sakakibara, Y. and Muramatsu, H. 2000. Learning context-free grammars from partially structured examples. In Grammatical Inference: Algorithms and Applications, A. L. Oliveira, Ed., Lecture Notes in Computer Science, vol. 1891. Springer, Berlin, 229--240.
[44]
Schlecht, J., Barnard, K., Spriggs, E., and Pryor, B. 2007. Inferring grammar-based structure models from 3D microscopy data. In Proceedings of the IEEE International Conference on Computer Vision and Pattern Recognition (CVPR'07). 1--8.
[45]
Shet, V., Singh, M., Bahlmann, C., and Ramesh, V. 2009. Predicate logic based image grammars for complex pattern recognition. In Proceedings of the IEEE International Conference on Computer Vision and Pattern Recognition Workshops (CVPR Workshops'09). 7.
[46]
Shilman, M., Liang, P., and Viola, P. 2005. Learning nongenerative grammatical models for document analysis. In Proceedings of the IEEE International Conference on Computer Vision (ICCV'05), vol. 2, 962--969.
[47]
Si, Z. and Zhu, S.-C. 2011. Unsupervised learning of stochastic AND-OR templates for object modeling. In Proceedings of the 2011 IEEE International Conference on Computer Vision Workshops (ICCV Workshops'11). 648--655.
[48]
Siddiqi, K. and Kimia, B. B. 1996. A shock grammar for recognition. In Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'96). 507--513.
[49]
Siddiqi, K., Shokoufandeh, A., Dickenson, S. J., and Zucker, S. W. 1998. Shock graphs and shape matching. In Proceedings of the 6th International Conference on Computer Vision. 222--229.
[50]
Sipser, M. 2006. Introduction to the Theory of Computation (2nd ed.). Thomson Course Technology.
[51]
Siskind, J., Sherman Jr, J., Polak, T., Harper, M., and Bouman, C. 2007. Spatial random tree grammars for modeling hierarchical structure in images with regions of arbitrary shape. IEEE Trans. Pattern Anal. Mach. Intell. 29, 9, 1504--1518.
[52]
Soltanpour, S. and Ebrahimnezhad, H. 2010. Learning novel object parts model for object categorization. In Proceedings of the 5th International Symposium on Telecommunications (IST'10). 796--800.
[53]
Stuckelberg, M. V. and Doermann, D. 1999. On musical score recognition using probabilistic reasoning. In Proceedings of the 5th International Conference on Document Analysis and Recognition (ICDAR'99). 115--118.
[54]
Subramanian, K. G., Mary, A. R. S., and Dersanambika, K. S. 2005. Splicing array grammar systems. In Proceedings of the International Conference on Theoretical Aspects of Computing (ICTAC'05), D. Hung and M. Wirsing, Eds., Lecture Notes in Computer Science, vol. 3722. Springer, Berlin, 125--135.
[55]
Sun, R., Jia, J., Li, H., and Jaeger, M. 2009. Image-based lightweight tree modeling. In Proceedings of the 8th International Conference on Virtual Reality Continuum and its Applications in Industry (VRCAI'09). ACM, New York, 17--22.
[56]
Takada, Y. 1988. Grammatical inference for even linear languages based on control sets. Inform. Process. Lett. 28, 4, 193--199.
[57]
Toshev, A., Mordohai, P., and Taskar, B. 2010. Detecting and parsing architecture at city scale from range data. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR'10). 398--405.
[58]
Trzupek, M., Ogiela, M. R., and Tadeusiewicz, R. 2009. Image content analysis for cardiac 3D visualizations. In Knowledge-Based and Intelligent Information and Engineering Systems, J. D. Velásquez, S. A. Ríos, R. J. Howlett, and L. C. Jain, Eds., Lecture Notes in Computer Science, vol. 5711. Springer, Berlin 192--199.
[59]
Trzupek, M., Ogiela, M. R., and Tadeusiewicz, R. 2011. Intelligent image content description and analysis for 3D visualizations of coronary vessels. In Intelligent Information and Database Systems, N. T. Nguyen, C.-G. Kim, and A. Janiak, Eds., Lecture Notes in Computer Science, vol. 6592. Springer, Berlin, 193--202.
[60]
Tylecek, R. and Sara, R. 2011. Modeling symmetries for stochastic structural recognition. In Proceedings of the IEEE International Conference on Computer Vision Workshops (ICCV Workshops'11). 632--639.
[61]
Wang, Q. and Jiang, Z. 2009. A grammatical framework for building rooftop extraction. In Proceedings of the IEEE International Symposium on Geoscience and Remote Sensing (IGARSS'09). III--334--III--337.
[62]
Wang, W., Pollak, I., Bouman, C. A., and Harper, M. P. 2005b. Classification of images using spatial random trees. In Proceedings of the 2005 IEEE/SP 13th Workshop on Statistical Signal Processing. 449--452.
[63]
Wang, W., Pollak, I., Wong, T.-S., Bouman, C. A., Harper, M. P., and Siskind, J. M. 2006. Hierarchical stochastic image grammars for classification and segmentation. IEEE Trans. Image Process. 15, 10, 3033--3052.
[64]
Wang, Y., Bahrami, S., and Zhu, S.-C. 2005a. Perceptual scale space and its applications. In Proceedings of the 10th IEEE International Conference on Computer Vision (ICCV'05). Vol. 1, 58--65.
[65]
Wu, Y. and Bian, H. 2009. Image segmentation integrating generative and discriminative methods. In Proceedings of the International Conference on Web Information Systems and Mining (WISM'09). 769--774.
[66]
Yao, B., Yang, X., and Wu, T. 2009. Image parsing with stochastic grammar: The Lotus Hill dataset and inference scheme. In Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops (CVPR Workshops'09).
[67]
Zaboli, H. and Rahmati, M. 2007. An improved shock graph approach for shape recognition and retrieval. In Proceedings of the 1st Asia International Conference on Modelling Simulation (AMS'07). 438--443.
[68]
Zhu, L., Chen, Y., and Yuille, A. 2009. Unsupervised learning of probabilistic grammar-Markov models for object categories. IEEE Trans. Pattern Anal. Mach. Intell. 31, 1, 114--128.
[69]
Zhu, S.-C. and Mumford, D. 2006. A stochastic grammar of images. Found. Trends. Comput. Graph. Vis. 2, 4, 259--362.

Cited By

View all
  • (2023)Modelo de regressão para identificação de ilha de calorPARC Pesquisa em Arquitetura e Construção10.20396/parc.v14i00.866838614(e023026)Online publication date: 1-Dec-2023
  • (2023)A Stochastic Grammar Approach to Mass Classification in MammogramsIEEE/ACM Transactions on Computational Biology and Bioinformatics10.1109/TCBB.2023.324714420:3(2302-2313)Online publication date: 1-May-2023
  • (2022)A Spatial Lexical Analyzer and 3D Grammars that Recognize Voxel Based Structures Using Linear Positional Grammars in Minecraft2022 21st Brazilian Symposium on Computer Games and Digital Entertainment (SBGames)10.1109/SBGAMES56371.2022.9961122(1-6)Online publication date: 24-Oct-2022
  • Show More Cited By

Recommendations

Reviews

Markus Wolf

Grammars were used in computer science from the beginning for designing compilers for the first programming language. They were later used in image generation (for example, L-systems were used to generate beautiful pictures of plants). Therefore, using grammars for pattern recognition in images seems to be a natural research avenue. This paper represents an overview of papers and research directions concerning the use of grammars for pattern recognition in images over the last decade. After a short introduction of the field and a discussion of older reviews, the paper continues with an explanation of how the reviewed papers where sampled. The main body of the paper presents the review results. The results are stated quantitatively in the form of graphs and tables and are clustered according to similar techniques, objectives, or type of grammar used. A qualitative analysis section includes discussion of the different pattern recognition approaches in several short sections with references to the most important papers in the field. Following the results, the authors discuss the advantages and limits of the use of grammars for pattern recognition and point to future research directions. A short conclusion closes the paper. The paper is well written and gives a nice overview of the state of affairs with many useful references for in-depth study. This is a good starting point for anyone interested in techniques for pattern recognition in images. Due to the discussion of possible research directions, it is also good for anyone searching for research topics in this area. Online Computing Reviews Service

Access critical reviews of Computing literature here

Become a reviewer for Computing Reviews.

Comments

Information & Contributors

Information

Published In

cover image ACM Computing Surveys
ACM Computing Surveys  Volume 46, Issue 2
November 2013
483 pages
ISSN:0360-0300
EISSN:1557-7341
DOI:10.1145/2543581
Issue’s Table of Contents
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 01 November 2013
Accepted: 01 May 2013
Revised: 01 December 2012
Received: 01 March 2012
Published in CSUR Volume 46, Issue 2

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. Image grammars
  2. computer vision
  3. formal languages
  4. image representation
  5. pattern recognition
  6. syntactic methods

Qualifiers

  • Research-article
  • Research
  • Refereed

Funding Sources

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)19
  • Downloads (Last 6 weeks)1
Reflects downloads up to 03 Oct 2024

Other Metrics

Citations

Cited By

View all
  • (2023)Modelo de regressão para identificação de ilha de calorPARC Pesquisa em Arquitetura e Construção10.20396/parc.v14i00.866838614(e023026)Online publication date: 1-Dec-2023
  • (2023)A Stochastic Grammar Approach to Mass Classification in MammogramsIEEE/ACM Transactions on Computational Biology and Bioinformatics10.1109/TCBB.2023.324714420:3(2302-2313)Online publication date: 1-May-2023
  • (2022)A Spatial Lexical Analyzer and 3D Grammars that Recognize Voxel Based Structures Using Linear Positional Grammars in Minecraft2022 21st Brazilian Symposium on Computer Games and Digital Entertainment (SBGames)10.1109/SBGAMES56371.2022.9961122(1-6)Online publication date: 24-Oct-2022
  • (2021)Perceived healthiness of foods: A systematic review of qualitative studiesFuture Foods10.1016/j.fufo.2021.1000564(100056)Online publication date: Dec-2021
  • (2021)An application of metadata-based image retrieval system for facility managementAdvanced Engineering Informatics10.1016/j.aei.2021.10141750:COnline publication date: 1-Oct-2021
  • (2021)DeepCPCFG: Deep Learning and Context Free Grammars for End-to-End Information ExtractionDocument Analysis and Recognition – ICDAR 202110.1007/978-3-030-86331-9_54(838-853)Online publication date: 5-Sep-2021
  • (2020)preservação digital em longo prazo amparada por planos de açõesRDBCI Revista Digital de Biblioteconomia e Ciência da Informação10.20396/rdbci.v18i0.866118518Online publication date: 4-Oct-2020
  • (2020)Towards an approach using grammars for automatic classification of masses in mammogramsComputational Intelligence10.1111/coin.1232037:4(1515-1544)Online publication date: 2-Apr-2020
  • (2019)A New Syntactic Approach for Masses Classification in Digital Mammograms2019 IEEE 32nd International Symposium on Computer-Based Medical Systems (CBMS)10.1109/CBMS.2019.00083(385-390)Online publication date: Jun-2019
  • (2019)Is mass classification in mammograms a solved problem? - A critical review over the last 20 yearsExpert Systems with Applications10.1016/j.eswa.2018.10.032119(90-103)Online publication date: Apr-2019
  • Show More Cited By

View Options

Get Access

Login options

Full Access

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Media

Figures

Other

Tables

Share

Share

Share this Publication link

Share on social media