Article

DuEE-Fin: A Large-Scale Dataset for Document-Level Event Extraction

Authors:

Jinchuan Zhang,

Zengfeng ZengAuthors Info & Claims

Natural Language Processing and Chinese Computing: 11th CCF International Conference, NLPCC 2022, Guilin, China, September 24–25, 2022, Proceedings, Part I

Pages 172 - 183

https://doi.org/10.1007/978-3-031-17120-8_14

Published: 24 September 2022 Publication History

Abstract

To tackle the data scarcity problem of document-level event extraction, we come up with a large-scale benchmark, DuEE-Fin, which consists of 15,000+ events categorized into 13 event types, and 81,000+ event arguments mapped in 92 argument roles. We constructed DuEE-Fin from real-world Chinese financial news, which allows one document to contain several events, multiple arguments to share the same argument role and one argument to play different roles in different events. Therefore, it presents some considerable challenges in document-level event extraction task such as multi-event recognition and multi-value argument identification, that are referred to as key issues for document-level event extraction task. Along with DuEE-Fin, we also hosted an open competition, which has attracted 1,690 teams and achieved exciting results. We performed experiments on DuEE-Fin with most popular document-level event extraction systems. However, results showed that even some SOTA models performed poorly with our data. Facing these challenges, we found it necessary to propose more effective methods.

References

[1]

Chen, M., et al.: Event-centric natural language processing. In: Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing: Tutorial Abstracts. pp. 6–14. Association for Computational Linguistics, Online (Aug 2021)., https://aclanthology.org/2021.acl-tutorials.2

[2]

Ebner, S., Xia, P., Culkin, R., Rawlins, K., Van Durme, B.: Multi-sentence argument linking. In: Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics. pp. 8057–8077. Association for Computational Linguistics, Online (2020)., https://www.aclweb.org/anthology/2020.acl-main.718

[3]

Fung, Y., et al.: InfoSurgeon: cross-media fine-grained information consistency checking for fake news detection. In: Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers). pp. 1683–1698. Association for Computational Linguistics, Online (2021)., https://aclanthology.org/2021.acl-long.133

[4]

Grishman, R., Sundheim, B.: Message understanding conference- 6: a brief history. In: COLING 1996 Volume 1: The 16th International Conference on Computational Linguistics (1996). https://aclanthology.org/C96-1079

[5]

Kipf, T.N., Welling, M.: Semi-supervised classification with graph convolutional networks. CoRR abs/1609.02907 (2016). http://arxiv.org/abs/1609.02907

[6]

Li, M., et al.: GAIA: A fine-grained multimedia knowledge extraction system. In: Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics: System Demonstrations. pp. 77–86. Association for Computational Linguistics, Online (2020)., https://www.aclweb.org/anthology/2020.acl-demos.11

[7]

Li, S., Ji, H., Han, J.: Document-level event argument extraction by conditional generation. In: Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. pp. 894–908. Association for Computational Linguistics, Online (2021)., https://www.aclweb.org/anthology/2021.naacl-main.69

[8]

Li X, Li F, Pan L, Chen Y, Peng W, Wang Q, Lyu Y, and Zhu Y Zhu X, Zhang M, Hong Y, and He R Duee: a large-scale dataset for Chinese event extraction in real-world scenarios Natural Language Processing and Chinese Computing 2020 Cham Springer International Publishing 534-545

[9]

Li, Z., Ding, X., Liu, T.: Constructing narrative event evolutionary graph for script event prediction. In: Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence. pp. 4201–4207. International Joint Conferences on Artificial Intelligence Organization, Stockholm, Sweden (Jul 2018)., https://www.ijcai.org/proceedings/2018/584

[10]

Lin, Y., Ji, H., Huang, F., Wu, L.: A joint neural model for information extraction with global features. In: Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics. pp. 7999–8009. Association for Computational Linguistics, Online (Jul 2020)., https://aclanthology.org/2020.acl-main.713

[11]

Liu, J., Chen, Y., Liu, K., Bi, W., Liu, X.: Event extraction as machine reading comprehension. In: Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP). pp. 1641–1651. Association for Computational Linguistics, Online (Nov 2020)., https://aclanthology.org/2020.emnlp-main.128

[12]

Vaswani, A., et al.: Attention is all you need. In: Proceedings of the 31st International Conference on Neural Information Processing Systems. vol. 30, pp. 5998–6008 (2017)

[13]

Wadden, D., Wennberg, U., Luan, Y., Hajishirzi, H.: Entity, relation, and event extraction with contextualized span representations. In: Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP). pp. 5784–5789. Association for Computational Linguistics, Hong Kong, China (Nov 2019)., https://aclanthology.org/D19-1585

[14]

Xu, R., Liu, T., Li, L., Chang, B.: Document-level event extraction via heterogeneous graph-based interaction model with a tracker. In: Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers). pp. 3533–3546. Association for Computational Linguistics, Online (2021)., https://aclanthology.org/2021.acl-long.274

[15]

Yang, H., Chen, Y., Liu, K., Xiao, Y., Zhao, J.: DCFEE: a document-level chinese financial event extraction system based on automatically labeled training data. In: Proceedings of ACL 2018, System Demonstrations. pp. 50–55. Association for Computational Linguistics, Melbourne, Australia (2018)., http://aclweb.org/anthology/P18-4009

[16]

Zheng, S., Cao, W., Xu, W., Bian, J.: Doc2EDAG: an end-to-end document-level framework for chinese financial event extraction. In: Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP). pp. 337–346. Association for Computational Linguistics, Hong Kong, China (2019)., https://www.aclweb.org/anthology/D19-1032

[17]

Zhu, T., et al.: Efficient document-level event extraction via pseudo-trigger-aware pruned complete graph (2021)

Cited By

Xu ZWang PKe WLi GLiu JJi KChen XWu CLarson K(2024)Incorporating schema-aware description into document-level event extractionProceedings of the Thirty-Third International Joint Conference on Artificial Intelligence10.24963/ijcai.2024/729(6597-6605)Online publication date: 3-Aug-2024
https://dl.acm.org/doi/10.24963/ijcai.2024/729
Tao ZWang CTian ZXu KGuo YLi SBai YXie D(2024)SIAT: Document-level Event Extraction via Spatiality-Augmented Interaction Model with Adaptive ThresholdingACM Transactions on Asian and Low-Resource Language Information Processing10.1145/369826123:10(1-21)Online publication date: 7-Oct-2024
https://dl.acm.org/doi/10.1145/3698261
Zhang GZhang HLi RTan H(2024)EADRE: Event-type Aware Dynamic Representation of Entities in Document-level Event ExtractionACM Transactions on Asian and Low-Resource Language Information Processing10.1145/369576723:12(1-17)Online publication date: 13-Sep-2024
https://dl.acm.org/doi/10.1145/3695767
Show More Cited By

Index Terms

DuEE-Fin: A Large-Scale Dataset for Document-Level Event Extraction

Index terms have been assigned to the content through auto-classification.

Recommendations

CFinDEE: A Chinese Fine-Grained Financial Dataset for Document-Level Event Extraction
WWW '24: Companion Proceedings of the ACM Web Conference 2024

Document-level event extraction faces numerous challenges in accurately modeling real-world financial scenarios, particularly due to the inadequacies in existing datasets regarding data scale and fine-grained annotations. The development of datasets is a ...
Multi-Turn and Multi-Granularity Reader for Document-Level Event Extraction
Most existing event extraction works mainly focus on extracting events from one sentence. However, in real-world applications, arguments of one event may scatter across sentences and multiple events may co-occur in one document. Thus, these scenarios ...
DuEE: A Large-Scale Dataset for Chinese Event Extraction in Real-World Scenarios
Natural Language Processing and Chinese Computing
Abstract
This paper introduces DuEE, a new dataset for Chinese event extraction (EE) in real-world scenarios. DuEE has several advantages over previous EE datasets. (1) Scale: DuEE consists of 19,640 events categorized into 65 event types, along with 41,...

Comments

Information & Contributors

Information

Published In

cover image Guide Proceedings

Natural Language Processing and Chinese Computing: 11th CCF International Conference, NLPCC 2022, Guilin, China, September 24–25, 2022, Proceedings, Part I

Sep 2022

877 pages

ISBN:978-3-031-17119-2

DOI:10.1007/978-3-031-17120-8

Editors:
Wei Lu
Singapore University of Technology and Design, Singapore, Singapore
,
Shujian Huang
Nanjing University, Nanjing, China
,
Yu Hong
Soochow University, Suzhou, China
,
Xiabing Zhou
Soochow University, Soochow, China

© The Author(s), under exclusive license to Springer Nature Switzerland AG 2022.

Publisher

Springer-Verlag

Berlin, Heidelberg

Publication History

Published: 24 September 2022

Author Tags

Qualifiers

Article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

6
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 08 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

Xu ZWang PKe WLi GLiu JJi KChen XWu CLarson K(2024)Incorporating schema-aware description into document-level event extractionProceedings of the Thirty-Third International Joint Conference on Artificial Intelligence10.24963/ijcai.2024/729(6597-6605)Online publication date: 3-Aug-2024
https://dl.acm.org/doi/10.24963/ijcai.2024/729
Tao ZWang CTian ZXu KGuo YLi SBai YXie D(2024)SIAT: Document-level Event Extraction via Spatiality-Augmented Interaction Model with Adaptive ThresholdingACM Transactions on Asian and Low-Resource Language Information Processing10.1145/369826123:10(1-21)Online publication date: 7-Oct-2024
https://dl.acm.org/doi/10.1145/3698261
Zhang GZhang HLi RTan H(2024)EADRE: Event-type Aware Dynamic Representation of Entities in Document-level Event ExtractionACM Transactions on Asian and Low-Resource Language Information Processing10.1145/369576723:12(1-17)Online publication date: 13-Sep-2024
https://dl.acm.org/doi/10.1145/3695767
Wan QWan CXiao KXiong HLiu DLiu XHu R(2024)Token-Event-Role Structure-Based Multi-Channel Document-Level Event ExtractionACM Transactions on Information Systems10.1145/364388542:4(1-27)Online publication date: 7-Feb-2024
https://dl.acm.org/doi/10.1145/3643885
Xian GDu STang XShi YJia BTang BLeng ZLi L(2024)DLEE: a dataset for Chinese document-level legal event extractionNeural Computing and Applications10.1007/s00521-024-09907-436:25(15581-15597)Online publication date: 1-Sep-2024
https://dl.acm.org/doi/10.1007/s00521-024-09907-4
Lian SZhao KLiu XLei XYang BZhang WWang KLiu Z(2024)What is the Best Model? Application-Driven Evaluation for Large Language ModelsNatural Language Processing and Chinese Computing10.1007/978-981-97-9437-9_6(67-79)Online publication date: 2-Nov-2024
https://dl.acm.org/doi/10.1007/978-981-97-9437-9_6

View Options

View options

Figures

Tables

Media

View Table of Conten