research-article

On Assisting and Automatizing the Semantic Segmentation of Masonry Walls

Authors:

Francesca Giuliani,

Massimiliano Corsini,

Federico Ponchio,

Marco Callieri,

Paolo CignoniAuthors Info & Claims

ACM Journal on Computing and Cultural Heritage (JOCCH), Volume 15, Issue 2

Article No.: 22, Pages 1 - 17

https://doi.org/10.1145/3477400

Published: 07 April 2022 Publication History

Abstract

In Architectural Heritage, the masonry’s interpretation is an essential instrument for analysing the construction phases, the assessment of structural properties, and the monitoring of its state of conservation. This work is generally carried out by specialists that, based on visual observation and their knowledge, manually annotate ortho-images of the masonry generated by photogrammetric surveys. This results in vector thematic maps segmented according to their construction technique (isolating areas of homogeneous materials/structure/texture or each individual constituting block of the masonry) or state of conservation, including degradation areas and damaged parts.

This time-consuming manual work, often done with tools that have not been designed for this purpose, represents a bottleneck in the documentation and management workflow and is a severely limiting factor in monitoring large-scale monuments (e.g., city walls). This article explores the potential of AI-based solutions to improve the efficiency of masonry annotation in Architectural Heritage. This experimentation aims at providing interactive tools that support and empower the current workflow, benefiting from specialists’ expertise.

References

[1]

Nabila Abraham and Naimul Mefraz Khan. 2019. A novel focal tversky loss function with improved attention u-net for lesion segmentation. In Proceedings of the 2019 IEEE 16th International Symposium on Biomedical Imaging. IEEE, 683–687.

[2]

Marco Giorgio Bevilacqua, Costantino Caciagli, and Cristina Salotti. 2011. Le Mura Di Pisa: Fortificazioni, Ammodernamenti E Modificazioni Dal XII al XIX Secolo. Edizioni ETS.

[3]

G. Bitelli, C. Balletti, R. Brumana, L. Barazzetti, M. G. D’Urso, F. Rinaudo, and G. Tucci. 2019. The GAMHer research project for metric documentation of cultural heritage: Current developments. ISPRS - International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences. XLII-2/W11 (2019), 239–246. DOI:

[4]

Anna Boato and Sergio Lagomarsino. 2010. Stratigrafia e statica. In Proceedings of the Archeologia Dell’architettura. Gian Pietro Brogiolo (Ed.), Vol. XV, All’Insegna del Giglio, 47–53.

[5]

Antonio Borri, Marco Corradi, Giulio Castori, and Alessandro De Maria. 2015. A method for the analysis and classification of historic masonry. Bulletin of Earthquake Engineering 13, 9 (2015), 2647–2665.

[6]

Antonio Borri, Marco Corradi, and Alessandro De Maria. 2020. The failure of masonry walls by disaggregation and the masonry quality index. Heritage 3, 4 (2020), 1162–1198.

[7]

Y. Y. Boykov and M.-P. Jolly. 2001. Interactive graph cuts for optimal boundary amp; region segmentation of objects in N-D images. In Proceedings of the 8th IEEE International Conference on Computer Vision. 105–112. DOI:

[8]

Gian Pietro Brogiolo and Paolo Faccio. 2010. Stratigrafia e prevenzione. In Proceedings of the Archeologia Dell’architettura. Gian Pietro Brogiolo (Ed.), Vol. XV, All’Insegna del Giglio, 55–63.

[9]

L. Castrejón, K. Kundu, R. Urtasun, and S. Fidler. 2017. Annotating object instances with a Polygon-RNN. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 4485–4493. DOI:

[10]

Liang-Chieh Chen, Yukun Zhu, George Papandreou, Florian Schroff, and Hartwig Adam. 2018. Encoder-decoder with atrous separable convolution for semantic image segmentation. In Computer Vision – ECCV 2018. Lecture Notes in Computer Science, vol 11211. Springer, Cham.

[11]

Ludovico Dipasquale, Luisa Rovero, and Fabio Fratini. 2020. Ancient stone masonry constructions. In Proceedings of the Nonconventional and Vernacular Construction Materials. Elsevier, 403–435.

[12]

Francesco Doglioni. 2010. Leggibilitá della costruzione, percorsi di ricerca stratigrafica e restauro. In Proceedings of the Archeologia Dell’architettura. Gian Pietro Brogiolo (Ed.), Vol. XV, All’Insegna del Giglio, 65–79.

[13]

Marco Forte, Brian Price, Scott Cohen, Ning Xu, and François Pitié. 2020. Getting to 99% accuracy in interactive segmentation. arXiv:2003.07932. Retrieved from https://arxiv.org/abs/2003.07932.

[14]

Kaiming He, Georgia Gkioxari, Piotr Dollár, and Ross Girshick. 2017. Mask R-CNN. In IEEE International Conference on Computer Vision (ICCV). 2980–2988, DOI:

[15]

Yahya Ibrahim, Balázs Nagy, and Csaba Benedek. 2019. CNN-based watershed marker extraction for brick segmentation in masonry walls. In Proceedings of the Image Analysis and Recognition. Fakhri Karray, Aurélio Campilho, and Alfred Yu (Eds.), Springer International Publishing, Cham, 332–344.

Digital Library

[16]

Anton Konushin Konstantin Sofiiuk and Ilia Petrov. 2021. Reviving iterative training with mask guidance for interactive segmentation. arXiv:2102.06583. Retrieved from https://arxiv.org/abs/2102.06583.

[17]

Jerry Ma and Denis Yarats. 2019. Quasi-hyperbolic momentum and Adam for deep learning. In Proceedings of the International Conference on Learning Representations.

[18]

K.-K. Maninis, S. Caelles, J. Pont-Tuset, and L. Van Gool. 2018. Deep extreme cut: From extreme points to object segmentation. In Proceedings of the Computer Vision and Pattern Recognition.

[19]

D. P. Papadopoulos, J. R. R. Uijlings, F. Keller, and V. Ferrari. 2017. Extreme clicking for efficient object annotation. In Proceedings of the International Conference on Computer Vision.4940–4949. DOI:

[20]

Gaia Pavoni, Massimiliano Corsini, Marco Callieri, Giuseppe Fiameni, Clinton Edwards, and Paolo Cignoni. 2020. On improving the training of models for the semantic segmentation of benthic communities from orthographic imagery. Remote Sensing 12, 18 (2020), 3106.

[21]

Nicole E. Pedersen, Clinton B. Edwards, Yoan Eynaud, Arthur C. R. Gleason, Jennifer E. Smith, and Stuart A. Sandin. 2019. The influence of habitat and adults on the spatial distribution of juvenile corals. Ecography 42, 10 (2019), 1703–1713.

[22]

Olaf Ronneberger, Philipp Fischer, and Thomas Brox. 2015. U-Net: convolutional networks for biomedical image segmentation. In Medical Image Computing and Computer-Assisted Intervention – MICCAI 2015. Lecture Notes in Computer Science, vol 9351. Springer, Cham.

[23]

M. P. Schuller, R. H. Atkinson, and J. L. Noland. 1995. Structural evaluation of historic masonry buildings. APT Bulletin: The Journal of Preservation Technology 26, 2/3 (1995), 51–61.

[24]

Chiara Stefani, Xavier Brunetaud, Sarah Janvier-Badosa, Kevin Beck, Livio De Luca, and Muzahim Al-Mukhtar. 2012. 3D information system for the digital documentation and the monitoring of stone alteration. In Proceedings of the Progress in Cultural Heritage Preservation. Marinos Ioannides, Dieter Fritsch, Johanna Leissner, Rob Davies, Fabio Remondino, and Rossella Caffo (Eds.), Springer, Berlin, 330–339.

Digital Library

[25]

Ivan Ustyuzhaninov, Claudio Michaelis, Wieland Brendel, and Matthias Bethge. 2018. One-shot texture segmentation. arXiv:1807.02654. Retrieved from http://arxiv.org/abs/1807.02654.

Cited By

Giuliani FGaglio FMartino MDe Falco A(2024)A HBIM pipeline for the conservation of large-scale architectural heritage: the city Walls of PisaHeritage Science10.1186/s40494-024-01141-412:1Online publication date: 1-Feb-2024
https://doi.org/10.1186/s40494-024-01141-4
Bartolini GGiuliani FDe Falco A(2023)Proof of Concept for Methodological Framework Including Point Clouds in the Non-Destructive Diagnosis of Historical Masonry StructuresInternational Journal of Architectural Heritage10.1080/15583058.2023.2260769(1-24)Online publication date: 27-Sep-2023
https://doi.org/10.1080/15583058.2023.2260769

Index Terms

On Assisting and Automatizing the Semantic Segmentation of Masonry Walls

Recommendations

Seismic Behaviour of Masonry Veneer Walls
Analyzing the effect of workmanship quality on performance of unreinforced masonry walls through numerical methods

The effect of workmanship quality (in terms of mortar cohesion) on wall's lateral strength is greater in in-plane direction than out-of-plane direction.The performance of a masonry wall under in-plane load is very sensitive to mortar joint cohesion. ...
Simple homogenization model for the non-linear analysis of in-plane loaded masonry walls

A simple homogenized model for the non-linear and limit analysis of masonry walls in-plane loaded is presented. A rectangular running bond elementary cell (RVE) is discretized by means of a few plane-stress three-noded triangular elements and ...

Comments

Information & Contributors

Information

Published In

cover image Journal on Computing and Cultural Heritage

Journal on Computing and Cultural Heritage Volume 15, Issue 2

June 2022

403 pages

ISSN:1556-4673

EISSN:1556-4711

DOI:10.1145/3514179

Editor:
Franco Niccolucci
VAST-LAB at PIN, University of Florence, Italy

Issue’s Table of Contents

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 07 April 2022

Accepted: 01 July 2021

Received: 01 May 2021

Published in JOCCH Volume 15, Issue 2

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Refereed

Funding Sources

Innovation for Data Elaboration in Heritage Areas - IDEHA project
National Research Program, MIUR

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

2
Total Citations
View Citations
499
Total Downloads

Downloads (Last 12 months)107
Downloads (Last 6 weeks)11

Reflects downloads up to 26 Jul 2024

Other Metrics

View Author Metrics

Citations

Cited By

Giuliani FGaglio FMartino MDe Falco A(2024)A HBIM pipeline for the conservation of large-scale architectural heritage: the city Walls of PisaHeritage Science10.1186/s40494-024-01141-412:1Online publication date: 1-Feb-2024
https://doi.org/10.1186/s40494-024-01141-4
Bartolini GGiuliani FDe Falco A(2023)Proof of Concept for Methodological Framework Including Point Clouds in the Non-Destructive Diagnosis of Historical Masonry StructuresInternational Journal of Architectural Heritage10.1080/15583058.2023.2260769(1-24)Online publication date: 27-Sep-2023
https://doi.org/10.1080/15583058.2023.2260769

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Full Text

View this article in Full Text.

HTML Format

View this article in HTML Format.

Media

Figures

Other

Tables

View full text|Download PDF

View Issue’s Table of Contents