research-article

Surfing on artistic documents with visually assisted tagging

Authors:

Daniele Borghesani,

Costantino Grana,

Rita CucchiaraAuthors Info & Claims

MM '10: Proceedings of the 18th ACM international conference on Multimedia

Pages 1343 - 1352

https://doi.org/10.1145/1873951.1874216

Published: 25 October 2010 Publication History

Abstract

This paper describes a complete architecture for the interactive exploration and annotation of artistic collections. In particular the focus is on Renaissance illuminated manuscripts, which typically contain thousands of pictures, used to comment or embellish the manuscript Gothic text. The final aim is to create a human centered multimedia application allowing the non practitioners to enjoy these masterpieces and expert users to share their knowledge. The system is composed by a modern user interface for browsing, surfing and querying, an automatic segmentation module, to ease the initial picture extraction task, and a similarity based retrieval engine, used to provide visually assisted tagging capabilities. A relevance feedback procedure is included to further refine the results. Experiments are reported regarding the adopted visual features based on covariance matrices and the Mean Shift Feature Space Warping relevance feedback. Finally some hints on the user interface for museum installations are discussed.

References

[1]

H. Bang and T. Chen. Feature space warping: an approach to relevance feedback. In IEEE International Conference on Image Processing, pages 968--971, 2002.

[2]

H. Bay, A. Ess, T. Tuytelaars, and L. Van Gool. Speeded-Up Robust Features (SURF). Computer Vision and Image Understanding, 110(3):346--359, 2008.

Digital Library

[3]

Y. Chang, K. Kamataki, and T. Chen. Mean shift feature space warping for relevance feedback. In IEEE International Conference on Image Processing, pages 1849--1852, 2009.

Digital Library

[4]

N. Chen and D. Blostein. A survey of document image classification: problem statement, classifier architecture and performance evaluation. International Journal of Document Analysis and Recognition, 10(1):1--16, 2007.

Digital Library

[5]

C. R. Dance, G. Csurka, L. Fan, J. Willamowski, and C. Bray. Visual categorization with bags of keypoints. In ECCV Workshop on Statistical Learning in Computer Vision, pages 1--22, 2004.

[6]

R. Datta, D. Joshi, J. Li, and J. Z. Wang. Image retrieval: Ideas, influences, and trends of the new age. ACM Computer Surveys, 40(2):1--60, 2008.

Digital Library

[7]

A. Elgammal. Human-centered multimedia: representations and challenges. In ACM international workshop on Human-centered multimedia, pages 11--18, 2006.

Digital Library

[8]

W. Förstner and B. Moonen. A metric for covariance matrices. Technical report, Stuttgart University, 1999.

[9]

C. Grana, D. Borghesani, and R. Cucchiara. Picture Extraction from Digitized Historical Manuscripts. In ACM International Conference on Image and Video Retrieval, July 2009.

Digital Library

[10]

C. Grana, D. Borghesani, and R. Cucchiara. Optimized Block-based Connected Components Labeling with Decision Trees. IEEE Transactions on Image Processing, 19(6), June 2010.

Digital Library

[11]

N. Journet, J. Ramel, R. Mullot, and V. Eglin. Document image characterization using a multiresolution analysis of the texture: application to old documents. International Journal of Document Analysis and Recognition, 11(1):9--18, 2008.

Digital Library

[12]

L. Kennedy, S. Chang, and A. Natsev. Query-Adaptive Fusion for Multimodal Search. Proceedings of the IEEE, 96(4):567--588, 2008.

[13]

A. Kitamoto, M. Onishi, T. Ikezaki, D. Deuff, E. Meyer, S. Sato, T. Muramatsu, R. Kamida, T. Yamamoto, and K. Ono. Digital Bleaching and Content Extraction for the Digital Archive of Rare Books. In International Conference on Document Image Analysis for Libraries, pages 133--144, 2006.

Digital Library

[14]

F. Le Bourgeois and H. Emptoz. DEBORA: Digital accEss to BOoks of the RenAissance. International Journal of Document Analysis and Recognition, 9(2):193--221, 2007.

Digital Library

[15]

X. Li, C. Snoek, and M. Worring. Learning tag relevance by neighbor voting for social image retrieval. In ACM International Conference on Multimedia Information Retrieval, pages 180--187, 2008.

Digital Library

[16]

D. G. Lowe. Distinctive Image Features from Scale-Invariant Keypoints. International Journal of Computer Vision, 60(2):91--110, 2004.

Digital Library

[17]

G. Meng, N. Zheng, Y. Song, and Y. Zhang. Document Images Retrieval Based on Multiple Features Combination. In International Conference on Document Analysis and Recognition, volume 1, pages 143--147, 2007.

Digital Library

[18]

G. Nagy. Twenty years of document image analysis in PAMI. IEEE Transactions on Pattern Analysis and Machine Intelligence, 22(1):38--62, 2000.

Digital Library

[19]

S. Nicolas, J. Dardenne, T. Paquet, and L. Heutte. Document Image Segmentation Using a 2D Conditional Random Field Model. In International Conference on Document Analysis and Recognition, volume 1, pages 407--411, 2007.

Digital Library

[20]

J. Ogier and K. Tombre. Madonne: Document Image Analysis Techniques for Cultural Heritage Documents. In EVA Conference on Digital Cultural Heritage, pages 107--114, 2006.

[21]

G. Park, Y. Baek, and H.-K. Lee. Re-ranking algorithm using post-retrieval clustering for content-based image retrieval. Information Processing & Management, 41(2):177--194, 2005.

Digital Library

[22]

J. Ramel, S. Busson, and M. Demonet. AGORA: the interactive document image analysis tool of the BVH project. In International Conference on Document Image Analysis for Libraries, pages 145--155, 2006.

Digital Library

[23]

Y. Rui, T. S. Huang, and S. Mehrotra. Content-Based Image Retrieval With Relevance Feedback In MARS. In IEEE International Conference on Image Processing, pages 815--818, 1997.

[24]

A. Turpin and F. Scholer. User performance versus precision measures for simple search tasks. In ACM SIGIR Conference on Research and Development in Information Retrieval, pages 11--18, 2006.

Digital Library

[25]

O. Tuzel, F. Porikli, and P. Meer. Pedestrian Detection via Classification on Riemannian Manifolds. IEEE Transactions on Pattern Analysis and Machine Intelligence, 30(10):1713--1727, 2008.

Digital Library

[26]

J. R. R. Uijlings, A. W. M. Smeulders, and R. J. H. Scha. Real-Time Bag of Words, Approximately. In ACM International Conference on Image and Video Retrieval, 2009.

Digital Library

[27]

M. Wang, K. Yang, X. Hua, and H. Zhang. Visual tag dictionary: interpreting tags with visual words. In 1st Workshop on Web-scale Multimedia Corpus, pages 1--8, 2009.

Digital Library

Cited By

Meinecke C(2022)Labeling of Cultural Heritage Collections on the Intersection of Visual Analytics and Digital Humanities2022 IEEE 7th Workshop on Visualization for the Digital Humanities (VIS4DH)10.1109/VIS4DH57440.2022.00009(19-24)Online publication date: Oct-2022
https://doi.org/10.1109/VIS4DH57440.2022.00009
Zou DZeng MXiong YFu ZZhang LSu Z(2019)Detecting floating-point errors via atomic conditionsProceedings of the ACM on Programming Languages10.1145/33711284:POPL(1-27)Online publication date: 20-Dec-2019
https://dl.acm.org/doi/10.1145/3371128
Pédrot PTabareau N(2019)The fire triangle: how to mix substitution, dependent elimination, and effectsProceedings of the ACM on Programming Languages10.1145/33711264:POPL(1-28)Online publication date: 20-Dec-2019
https://dl.acm.org/doi/10.1145/3371126
Show More Cited By

Index Terms

Surfing on artistic documents with visually assisted tagging
1. Information systems
  1. Information systems applications
    1. Multimedia information systems

Recommendations

Relevance feedback strategies for artistic image collections tagging
ICMR '11: Proceedings of the 1st ACM International Conference on Multimedia Retrieval

This paper provides an analysis on relevance feedback techniques in a multimedia system designed for the interactive exploration and annotation of artistic collections, in particular illuminated manuscripts. The relevance feedback is presented not only ...
Rerum novarum: interactive exploration of illuminated manuscripts
MM '10: Proceedings of the 18th ACM international conference on Multimedia

This paper describes an interactive application for the exploration and annotation of illuminated manuscripts, which typically contain thousands of pictures, used to comment or embellish the manuscript Gothic text. The system is composed by a modern ...
Automatic tag expansion using visual similarity for photo sharing websites

In this paper we present an automatic photo tag expansion method designed for photo sharing websites. The purpose of the method is to suggest tags that are relevant to the visual content of a given photo at upload time. Both textual and visual cues are ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

MM '10: Proceedings of the 18th ACM international conference on Multimedia

October 2010

1836 pages

ISBN:9781605589336

DOI:10.1145/1873951

General Chairs:
Alberto del Bimbo
University of Florence, Italy
,
Shih-Fu Chang
Columbia University, USA
,
Program Chair:
Arnold Smeulders
University of Amsterdam, NL

Copyright © 2010 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGMM: ACM Special Interest Group on Multimedia

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 25 October 2010

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

MM '10

Sponsor:

SIGMM

MM '10: ACM Multimedia Conference

October 25 - 29, 2010

Firenze, Italy

Acceptance Rates

Overall Acceptance Rate 2,145 of 8,556 submissions, 25%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

19
Total Citations
View Citations
194
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 24 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Meinecke C(2022)Labeling of Cultural Heritage Collections on the Intersection of Visual Analytics and Digital Humanities2022 IEEE 7th Workshop on Visualization for the Digital Humanities (VIS4DH)10.1109/VIS4DH57440.2022.00009(19-24)Online publication date: Oct-2022
https://doi.org/10.1109/VIS4DH57440.2022.00009
Zou DZeng MXiong YFu ZZhang LSu Z(2019)Detecting floating-point errors via atomic conditionsProceedings of the ACM on Programming Languages10.1145/33711284:POPL(1-27)Online publication date: 20-Dec-2019
https://dl.acm.org/doi/10.1145/3371128
Pédrot PTabareau N(2019)The fire triangle: how to mix substitution, dependent elimination, and effectsProceedings of the ACM on Programming Languages10.1145/33711264:POPL(1-28)Online publication date: 20-Dec-2019
https://dl.acm.org/doi/10.1145/3371126
An SSingh RMisailovic SSamanta R(2019)Augmented example-based synthesis using relational perturbation propertiesProceedings of the ACM on Programming Languages10.1145/33711244:POPL(1-24)Online publication date: 20-Dec-2019
https://dl.acm.org/doi/10.1145/3371124
Friedrichs KMünster SKröber CBruschke J(2018)Creating Suitable Tools for Art and Architectural Research with Historic Media RepositoriesDigital Research and Education in Architectural Heritage10.1007/978-3-319-76992-9_8(117-138)Online publication date: 13-Mar-2018
https://doi.org/10.1007/978-3-319-76992-9_8
Durand EMerienne FPere CCallet P(2014)Ray-on, an On-Site Photometric Augmented Reality DeviceJournal on Computing and Cultural Heritage 10.1145/26294857:2(1-13)Online publication date: 1-Jun-2014
https://dl.acm.org/doi/10.1145/2629485
Manitsaris SGlushkova ABevilacqua FMoutarde F(2014)Capture, Modeling, and Recognition of Expert Technical Gestures in Wheel-Throwing Art of PotteryJournal on Computing and Cultural Heritage 10.1145/26277297:2(1-15)Online publication date: 1-Jun-2014
https://dl.acm.org/doi/10.1145/2627729
Correia NRomão TRicardo AMota TMelo MCastro RCarvalho RMiranda A(2014)Design of an Interactive Experience with Medieval IlluminationsJournal on Computing and Cultural Heritage 10.1145/26262897:2(1-19)Online publication date: 1-Jun-2014
https://dl.acm.org/doi/10.1145/2626289
Kenderdine SChan LShaw J(2014)Pure LandJournal on Computing and Cultural Heritage 10.1145/26145677:2(1-15)Online publication date: 1-Jun-2014
https://dl.acm.org/doi/10.1145/2614567
Marton FRodriguez MBettio FAgus MVillanueva AGobbetti E(2014)IsoCamJournal on Computing and Cultural Heritage 10.1145/26115197:2(1-24)Online publication date: 1-Jun-2014
https://dl.acm.org/doi/10.1145/2611519
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten