research-article

Processing Historical Film Footage with Photogrammetry and Machine Learning for Cultural Heritage Documentation

Authors:

Francesca Condorelli and

Fulvio RinaudoAuthors Info & Claims

SUMAC '19: Proceedings of the 1st Workshop on Structuring and Understanding of Multimedia heritAge Contents

October 2019

Pages 39 - 46

https://doi.org/10.1145/3347317.3357248

Published: 15 October 2019 Publication History

Abstract

Historical film footages in many cases represent the only remaining traces of Cultural Heritage that has been lost or changed over time. Photogrammetry is a powerful technique to document the heritage transformations, but its implementation is technically challenging due to the difficulty in finding the historical data suitable to be process. This paper aims to examine the possibility to extract metric information of historic buildings from historical film footage for their 3D virtual reconstruction. In order to make automatic the research of a specific monument to document, in the first part of the study an algorithm for the detection of architectural heritage in historical film footage was developed using Machine Learning. This algorithm allowed the identification of the frames in which the monument appeared and their processing with photogrammetry. In the second part, with the implementation of open source Structure-from-Motion algorithms, the 3D virtual reconstruction of the monument and its metric information were obtained. The results were compared with a benchmark for evaluate the metric quality of the model, according to specific camera motion. This research, analysing the metric potentialities of historical film footage, provides fundamental support to documentation of Cultural Heritage, creating tools useful for both geomatics and historians.

References

[1]

F. Condorelli and F. Rinaudo (2019). Benchmark of metric quality assessment in photogrammetric reconstruction for historical film footage. Int. Arch. Photogramm. Remote Sens. Spatial Inf. Sci., XLII-2/W11, 443--448. 2019.

[2]

B. Markhoff, T. B. Nguyen, C. Niang, C (2017). When It Comes to Querying Semantic Cultural Heritage Data. In: Advances in Databases and Information Systems: New Trends in Databases and Information Systems, pp. 384--394.

[3]

F. Amato, V. Moscato, A. Picariello, F. Colace, M. De Santo, F. Schreiber, L. Tanca (2017). Big Data Meets Digital Cultural Heritage: Design and Implementation of SCRABS, A Smart Context-awaRe Browsing Assistant for Cultural EnvironmentS. ACM Journal on Computing and Cultural Heritage, Vol. 10, No. 1, Article 6, 10(1):6.1--6.23.

Digital Library

[4]

S. Caraceni, M. Carpenè, M. D'Antonio, G. Fiameni, A. Guidazzoli, S. Imboden, MC. Liguori, M. Montanari, G. Trotta, G. Scipione, D. Hanegreefs, (2017). I-media-cities, a searchable platform on moving images with automatic and manual annotations. 23rd International Conference on Virtual System & Multimedia (VSMM), Dublin, Ireland.

[5]

F. Condorelli and F. Rinaudo (2018). Cultural Heritage reconstruction from historical photographs and videos. Int. Arch. Photogramm. Remote Sens. Spatial Inf. Sci., XLII-2, 259--265. 2018.

[6]

R. Arandjelovic, P. Gronat, A. Torii, T. Pajdla, J. Sivic (2018). NetVLAD: CNN Architecture for Weakly Supervised Place Recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 40, Issue 6. Page(s): 1437 - 1451

[7]

J. Llamas, P. M. Lerones, R. Medina, E. Zalama, J. Gómez-García-Bermejo (2017). Classification of Architectural Heritage Images Using Deep Learning Techniques. Appl. Sci. 2017, 7, 992;

[8]

A. Saini, T. Gupta, G. Kumar, A. Gupta, M. Panwar, A. Mittal (2017). Image based Indian Monument Recognition using Convoluted Neural Networks. International Conference on Big Data, IoT and Data Science (BID).

[9]

A. Belhi (2018). Leveraging Known Data for Missing Label Prediction in Cultural Heritage Context. Applied Sciences.

[10]

A. Miech, I. Laptev, J. Sivic (2017). Learnable pooling with Context Gating for video classification. hal-01547378.

[11]

S. Ren, K. He, R. Girshick, J. Sun (2016). Faster R-CNN: Towards real-time object detection with region proposal networks. IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol 39, 6.

[12]

L. O'Connell (2001). Afterlives of the Tour Saint-Jacques: Plotting the Perceptual History of an Urban Fragment. Journal of the Society of Architectural Historians, Vol. 60, pp.450--473.

[13]

M. Abadi et al. (2015). TensorFlow: Large-scale machine learning on heterogeneous systems. http://tensorflow.org.

[14]

J. Rey, I. Tayler, A. Descoins, D. Rodriguez, A. Azzinnari, et al., Open source computer vision toolkit, https://github.com/tryolabs/luminoth.

[15]

P. Waldhäusl, CL. Ogleby, JL. Lerma, A. Georgopoulos (2013) 3x3 rules for simple photogrammetric documentation of architecture.

[16]

F. Remondino (2003). Recovering metric information from old monocular video sequences. In: Proceedings of 6th Conference on Optical 3D Measurement Techniques, Gruen/Kahmen editors, pp.214--222.

[17]

F. Remondino and N. Börlin (2004). Photogrammetric calibration of sequences acquired with a rotating camera. ISPRS Archives, Volume XXXIV-5/W16.

[18]

G. Fangi (2007). The multi-image spherical panoramas as a tool for architectural survey. XXI International CIPA Symposium, Athens, Greece.

[19]

G. Pavoni, M. Dellepiane, M. Callieri, R. Scopigno (2016). Automatic selection of video frames for path regularization and 3D Reconstruction. EUROGRAPHICS Workshop on Graphics and Cultural Heritage.

[20]

D. Zawieska, JS. Markiewicz, J. Kopiasz, J. Tazbir, A. Tobiasz, (2017). 3D modelling of the Lusatian Borough in Biskupin using archival data. In: Int. Arch. Photogramm. Remote Sens. Spatial Inf. Sci., XLII-2/W3, 665--669.

[21]

J. Cusicanqui, N. Kerle, F. Nex (2018). Usability of aerial video footage for 3-D scene reconstruction and structural damage assessment. Nat. Hazards Earth Syst. Sci., 18, 1583--1598.

[22]

J. L. Schönberger and J. M. Frahm (2016). Structure-from-motion revisited. IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2016, Vol. 2016, 4104--4113, IEEE Computer Society.

[23]

J. L. Schönberger, E. Zheng, JM. Frahm, M. Pollefeys (2016). Pixelwise View Selection for Unstructured Multi-View Stereo. ECCV 2016, Lecture Notes in Computer Science, vol 9907, Springer, Cham.

[24]

T. Schöps, J. L. Schönberger, S. Galliani, T. Sattler, K. Schindler, M. Pollefeys, A. Geiger (2016). A Multi-View Stereo Benchmark with High-Resolution Image and Multi-Camera Videos. 30th IEEE Conference on Computer Vision and Pattern Recognition (CVPR 2017), Honolulu, HI, USA.

Cited By

Silva COliveira L(2024)Artificial Intelligence at the Interface between Cultural Heritage and Photography: A Systematic Literature ReviewHeritage10.3390/heritage70701807:7(3799-3820)Online publication date: 17-Jul-2024
https://doi.org/10.3390/heritage7070180
Münster SMaiwald FBruschke JKröber CSun YDworak DKomorowicz DMunir IBeck CMünster D(2024)A Digital 4D Information System on the World Scale: Research Challenges, Approaches, and Preliminary ResultsApplied Sciences10.3390/app1405199214:5(1992)Online publication date: 28-Feb-2024
https://doi.org/10.3390/app14051992

Index Terms

Processing Historical Film Footage with Photogrammetry and Machine Learning for Cultural Heritage Documentation

Recommendations

UAS-based photogrammetry of cultural heritage sites: a case study addressing Chapel of Espírito Santo and photogrammetric software comparison
ICGDA '18: Proceedings of the International Conference on Geoinformatics and Data Analysis

The cost-effectiveness of unmanned aerial systems (UAS) makes them suitable platforms to survey cultural heritage sites. Developments in photogrammetry provide methods capable to generate accurate 3D models out of 2D aerial images. Considering the ...
Read More
3D Models in Cultural Heritage: Approaches for Their Creation and Use

Metric surveys are a key component in Cultural Heritage. Metric surveys are in fact a basic input for a wide range of activities, from documentation to study, from restoration to valorization, that rely on the current condition of the monument. In ...
Read More
Spherical photogrammetry for cultural heritage: san galgano abbey, siena, italy and roman theatre, sabratha, libya
eHeritage '10: Proceedings of the second workshop on eHeritage and digital art preservation

Here are exposed the results of the photogrammetric surveys of two noticeable monuments, the Roman Theatre in Sabratha, Libya and San Galgano Abbey, in Italy. They have been performed with a new photogrammetric technique, the Spherical Photogrammetry, ...
Read More

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

SUMAC '19: Proceedings of the 1st Workshop on Structuring and Understanding of Multimedia heritAge Contents

October 2019

87 pages

ISBN:9781450369107

DOI:10.1145/3347317

General Chairs:
Valerie Gouet-Brunet
IGN/LaSTIG, France
,
Margarita Khokhlova
IGN/LaSTIG, Centrale Lyon/LIRIS, France
,
Liming Chen
Centrale Lyon/LIRIS, France
,
Sander Münster
Technische Universität Dresden, Germany

Copyright © 2019 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGMM: ACM Special Interest Group on Multimedia

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 15 October 2019

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

MM '19

Sponsor:

SIGMM

MM '19: The 27th ACM International Conference on Multimedia

October 21, 2019

Nice, France

Acceptance Rates

Overall Acceptance Rate 5 of 6 submissions, 83%

Upcoming Conference

MM '24

Sponsor:
sigmm

The 32nd ACM International Conference on Multimedia

October 28 - November 1, 2024

Melbourne , VIC , Australia

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

2
Total Citations
View Citations
212
Total Downloads

Downloads (Last 12 months)47
Downloads (Last 6 weeks)4

Other Metrics

View Author Metrics

Citations

Cited By

Silva COliveira L(2024)Artificial Intelligence at the Interface between Cultural Heritage and Photography: A Systematic Literature ReviewHeritage10.3390/heritage70701807:7(3799-3820)Online publication date: 17-Jul-2024
https://doi.org/10.3390/heritage7070180
Münster SMaiwald FBruschke JKröber CSun YDworak DKomorowicz DMunir IBeck CMünster D(2024)A Digital 4D Information System on the World Scale: Research Challenges, Approaches, and Preliminary ResultsApplied Sciences10.3390/app1405199214:5(1992)Online publication date: 28-Feb-2024
https://doi.org/10.3390/app14051992

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents