research-article

A methodology for evaluating algorithms for table understanding in PDF documents

Authors:

Max Göbel,

Tamir Hassan,

Ermelinda Oro,

Giorgio OrsiAuthors Info & Claims

DocEng '12: Proceedings of the 2012 ACM symposium on Document engineering

Pages 45 - 48

https://doi.org/10.1145/2361354.2361365

Published: 04 September 2012 Publication History

Get Access

Abstract

This paper presents a methodology for the evaluation of table understanding algorithms for PDF documents. The evaluation takes into account three major tasks: table detection, table structure recognition and functional analysis. We provide a general and flexible output model for each task along with corresponding evaluation metrics and methods. We also present a methodology for collecting and ground-truthing PDF documents based on consensus-reaching principles and provide a publicly available ground-truthed dataset.

References

[1]

M. J. Cafarella, A. Halevy, and J. Madhavan. Structured data on the web. Commun. ACM, 54(2):72--79, 2011.

Digital Library

Google Scholar

[2]

A. C. e Silva. Metrics for evaluating performance in document analysis: application to tables. IJDAR, 14(1):101--109, 2011.

Digital Library

Google Scholar

[3]

T. Hassan. Towards a common evaluation strategy for table structure recognition algorithms. In Proc. of DocEng, 2010.

Digital Library

Google Scholar

[4]

J. Hu, R. Kashi, D. Lopresti, and G. Wilfong. Evaluating the performance of table processing algorithms. IJDAR, 4(3):140--153, 2002.

Crossref

Google Scholar

[5]

J. Hu, R. Kashi, D. Lopresti, G. Wilfong, and G. Nagy. Why table ground-truthing is hard. In Proc. of ICDAR, pages 129--133, 2001.

Digital Library

Google Scholar

[6]

M. Hurst. The Interpretation of Tables in Texts. PhD thesis, University of Edinburgh, 2000.

Google Scholar

[7]

M. Hurst. A constraint-based approach to table structure derivation. In Proc. of ICDAR, pages 911--915, 2003.

Digital Library

Google Scholar

[8]

T. Kieninger and A. Dengel. An approach towards benchmarking of table structure recognition results. In Proc. of ICDAR, pages 1232--1236, 2005.

Digital Library

Google Scholar

[9]

D. D. Lewis. Evaluating and optimizing autonomous text classification systems. In Proc. of SIGIR, pages 246--254, 1995.

Digital Library

Google Scholar

[10]

E. Oro and M. Ruffolo. PDF-TREX: An approach for recognizing and extracting tables from PDF documents. In Proc. of ICDAR, pages 906--910, 2009.

Digital Library

Google Scholar

[11]

I. T. Phillips. User's reference manual for the uw english/technical document image database III. Technical report, Seattle University, 1996.

Google Scholar

[12]

A. Shahab, F. Shafait, T. Kieninger, and A. Dengel. An open approach towards the benchmarking of table structure recognition systems. In Proc. of DAS, pages 113--120, 2010.

Digital Library

Google Scholar

[13]

X. Wang. Tabular Abstraction, Editing and Formatting. PhD thesis, University of Waterloo, 1996.

Digital Library

Google Scholar

Cited By

View all

Long RXing HYang ZZheng QYu ZHuang FYao C(2025)LORE++: Logical location regression network for table structure recognition with pre-trainingPattern Recognition10.1016/j.patcog.2024.110816157(110816)Online publication date: Jan-2025
https://doi.org/10.1016/j.patcog.2024.110816
Salaheldin Kasem MAbdallah ABerendeyev AElkady EMahmoud MAbdalla MHamada MVascon SNurseitov DTaj-Eddin I(2024)Deep Learning for Table Detection and Structure Recognition: A SurveyACM Computing Surveys10.1145/365728156:12(1-41)Online publication date: 10-Apr-2024
https://dl.acm.org/doi/10.1145/3657281
Zhang ZHu PMa JDu JZhang JYin BYin BLiu C(2024)SEMv2: Table separation line detection based on instance segmentationPattern Recognition10.1016/j.patcog.2024.110279149(110279)Online publication date: May-2024
https://doi.org/10.1016/j.patcog.2024.110279
Show More Cited By

Index Terms

A methodology for evaluating algorithms for table understanding in PDF documents
1. Applied computing
  1. Document management and text processing
    1. Document capture
      1. Document analysis
2. Information systems
  1. Information retrieval
    1. Evaluation of retrieval results

Recommendations

Document understanding of graphical content in natively digital PDF documents
DocEng '12: Proceedings of the 2012 ACM symposium on Document engineering

This paper presents an object-based method for analysing the content drawn by graphical operators in natively digital PDF documents. We propose that graphical content in a document can be classified either as structural or non-structural and present an ...
PDF-VQA: A New Dataset for Real-World VQA on PDF Documents
Machine Learning and Knowledge Discovery in Databases: Applied Data Science and Demo Track
Abstract
Document-based Visual Question Answering examines the document understanding of document images in conditions of natural language questions. We proposed a new document-based VQA dataset, PDF-VQA, to comprehensively examine the document ...
User-Guided Wrapping of PDF Documents Using Graph Matching Techniques
ICDAR '09: Proceedings of the 2009 10th International Conference on Document Analysis and Recognition

There are a number of established products on the market for wrapping - semi-automatic navigation and extraction of data - from web pages.These solutions make use of the inherent structure of HTML to locate instances of data to be wrapped.As PDF ...

Comments

Information & Contributors

Information

Published In

DocEng '12: Proceedings of the 2012 ACM symposium on Document engineering

September 2012

256 pages

ISBN:9781450311168

DOI:10.1145/2361354

General Chair:
Cyril Concolato
Telecom ParisTech, France
,
Program Chair:
Patrick Schmitz
University of California, Berkeley, USA

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

In-Cooperation

SIGDOC: ACM Special Interest Group for Design of Communications

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 04 September 2012

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

DocEng '12

Sponsor:

SIGWEB

DocEng '12: ACM Symposium on Document Engineering

September 4 - 7, 2012

Paris, France

Acceptance Rates

Overall Acceptance Rate 194 of 564 submissions, 34%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

52
Total Citations
View Citations
454
Total Downloads

Downloads (Last 12 months)33
Downloads (Last 6 weeks)3

Reflects downloads up to 05 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

View all

Long RXing HYang ZZheng QYu ZHuang FYao C(2025)LORE++: Logical location regression network for table structure recognition with pre-trainingPattern Recognition10.1016/j.patcog.2024.110816157(110816)Online publication date: Jan-2025
https://doi.org/10.1016/j.patcog.2024.110816
Salaheldin Kasem MAbdallah ABerendeyev AElkady EMahmoud MAbdalla MHamada MVascon SNurseitov DTaj-Eddin I(2024)Deep Learning for Table Detection and Structure Recognition: A SurveyACM Computing Surveys10.1145/365728156:12(1-41)Online publication date: 10-Apr-2024
https://dl.acm.org/doi/10.1145/3657281
Zhang ZHu PMa JDu JZhang JYin BYin BLiu C(2024)SEMv2: Table separation line detection based on instance segmentationPattern Recognition10.1016/j.patcog.2024.110279149(110279)Online publication date: May-2024
https://doi.org/10.1016/j.patcog.2024.110279
Choi YKim TKim NLee TJoe S(2024)End to End Table TransformerDocument Analysis and Recognition - ICDAR 202410.1007/978-3-031-70533-5_20(331-345)Online publication date: 8-Sep-2024
https://doi.org/10.1007/978-3-031-70533-5_20
Zhang WZhang CNing ZWang GBai YJiang ZZhang D(2023)M2SH: A Hybrid Approach to Table Structure Recognition using Two-Stage Multi-Modality Feature Fusion2023 IEEE International Conference on Systems, Man, and Cybernetics (SMC)10.1109/SMC53992.2023.10394093(791-798)Online publication date: 1-Oct-2023
https://doi.org/10.1109/SMC53992.2023.10394093
Ohta MAoyagi HUwano FKanazawa TTakasu A(2023)An End-to-End Table Structure Analysis Method Using Graph Attention NetworksLeveraging Generative Intelligence in Digital Libraries: Towards Human-Machine Collaboration10.1007/978-981-99-8088-8_20(230-239)Online publication date: 30-Nov-2023
https://doi.org/10.1007/978-981-99-8088-8_20
Smock BPesala RAbraham R(2023)GriTS: Grid Table Similarity Metric for Table Structure RecognitionDocument Analysis and Recognition - ICDAR 202310.1007/978-3-031-41734-4_33(535-549)Online publication date: 19-Aug-2023
https://doi.org/10.1007/978-3-031-41734-4_33
Li ZPeng FXue YHao NJin L(2023)Scene Table Structure Recognition with Segmentation and Key Point CollaborationDocument Analysis and Recognition - ICDAR 202310.1007/978-3-031-41679-8_17(295-310)Online publication date: 19-Aug-2023
https://doi.org/10.1007/978-3-031-41679-8_17
Smock BPesala RAbraham R(2022)PubTables-1M: Towards comprehensive table extraction from unstructured documents2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)10.1109/CVPR52688.2022.00459(4624-4632)Online publication date: Jun-2022
https://doi.org/10.1109/CVPR52688.2022.00459
Liu HLi XLiu BJiang DLiu YRen B(2022)Neural Collaborative Graph Machines for Table Structure Recognition2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)10.1109/CVPR52688.2022.00449(4523-4532)Online publication date: Jun-2022
https://doi.org/10.1109/CVPR52688.2022.00449
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Cited By

Index Terms

Recommendations

Document understanding of graphical content in natively digital PDF documents

PDF-VQA: A New Dataset for Real-World VQA on PDF Documents

User-Guided Wrapping of PDF Documents Using Graph Matching Techniques

Comments

Published In

Sponsors

In-Cooperation

Publisher

Publication History

Permissions

Check for updates

Author Tags

Qualifiers

Conference

Acceptance Rates

Other Metrics

Article Metrics

Other Metrics

Cited By

Login options

Full Access

PDF

eReader

Abstract

References

Cited By

Index Terms

Recommendations

Document understanding of graphical content in natively digital PDF documents

PDF-VQA: A New Dataset for Real-World VQA on PDF Documents

User-Guided Wrapping of PDF Documents Using Graph Matching Techniques

Comments

Information

Published In

Sponsors

In-Cooperation

Publisher

Publication History

Permissions

Check for updates

Author Tags

Qualifiers

Conference

Acceptance Rates

Contributors

Other Metrics

Bibliometrics

Article Metrics

Other Metrics

Citations

Cited By

Login options

Full Access

View options

PDF

eReader

Figures

Other

Share

Share this Publication link

Share on social media

Affiliations