Abstract
Deep learning-based blockchain named entity recognition (NER) often necessitates large quantities of labeled samples, but there are few blockchain-labeled text data readily available and the cost of manual labeling is high. The limited availability of labeled data makes it necessary to train systems with limited data. For Chinese NER in particular, transformer models such as BERT have shown to be efficient in encoding character-level texts, but the performance is unstable and leaves room for improvement because of the challenges in adding more features and the lack of word-level features. Chinese text modeling has been rarely studied, and most models in this regard have poor generalization abilities. In this paper, we propose the Fused Lexical Features-Prototypical Networks (LT-PN network), a novel approach for a few Chinese NER, consisting of (1) a flexible word Lattice Transformer encoder with a network transformer that can encode texts at the word level and be flexibly combined with character-level functions, and (2) a gated mixed of experts (MoE) network which overcomes the feature overload problem by training the model to conditionally combine contextual and positional features rather than assigning fixed weights to them.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Bansal, T., Jha, R., Munkhdalai, T., McCallum, A.: Self-supervised meta-learning for few-shot natural language classification tasks. arXiv preprint arXiv:2009.08445 (2020)
Chai, Y., Du, L., Qiu, J., Yin, L., Tian, Z.: Dynamic prototype network based on sample adaptation for few-shot malware detection. IEEE Trans. Knowl. Data Eng. (2022)
Chai, Y., Qiu, J., Yin, L., Zhang, L., Gupta, B.B., Tian, Z.: From data and model levels: improve the performance of few-shot malware classification. IEEE Trans. Netw. Serv. Manag. (2022)
Cui, Y., Che, W., Liu, T., Qin, B., Yang, Z.: Pre-training with whole word masking for Chinese BERT (2021)
Devlin, J., Chang, M.-W., Lee, K., Toutanova, K.: BERT: pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805 (2018)
Ding, R., Xie, P., Zhang, X., Lu, W., Li, L., Si, L.: A neural multi-digraph model for Chinese NER with gazetteers. In: Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pp. 1462–1467 (2019)
Fritzler, A., Logacheva, V., Kretov, M.: Few-shot classification in named entity recognition task. In: Proceedings of the 34th ACM/SIGAPP Symposium on Applied Computing, pp. 993–1000 (2019)
Fu, R., Zhang, Z., Li, L.: Using LSTM and GRU neural network methods for traffic flow prediction. In: 2016 31st Youth Academic Annual Conference of Chinese Association of Automation (YAC), pp. 324–328. IEEE (2016)
Geng, R., Li, B., Li, Y., Zhu, X., Jian, P., Sun, J.: Induction networks for few-shot text classification. arXiv preprint arXiv:1902.10482 (2019)
Gu, J., Wang, Y., Chen, Y., Cho, K., Li, V.O.K.: Meta-learning for low-resource neural machine translation. arXiv preprint arXiv:1808.08437 (2018)
Gui, T., Ma, R., Zhang, Q., Zhao, L., Jiang, Y.-G., Huang, X.: CNN-based Chinese NER with lexicon rethinking. IJCAI, 4982–4988 (2019)
Gui, T., Zou, Y., Zhang, Q., Peng, M., Fu, J., Wei, Z., Huang, X.-J.: A lexicon-based graph neural network for Chinese NER. In: Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pp. 1040–1050 (2019)
Han, X., Zhu, H., Yu, P., Wang, Z., Yao, Y., Liu, Z., Sun, M.: Fewrel: a large-scale supervised few-shot relation classification dataset with state-of-the-art evaluation. arXiv preprint arXiv:1810.10147 (2018)
He, H., Sun, X.: A unified model for cross-domain and semi-supervised named entity recognition in Chinese social media. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 31 (2017)
Huang, J., Li, C., Subudhi, K., Jose, D., Balakrishnan, S., Chen, W., Peng, B., Gao, J., Han, J.: Few-shot named entity recognition: an empirical baseline study. In: Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pp. 10408–10423 (2021)
Li, Jing, Sun, Aixin, Han, Jianglei, Li, Chenliang: A survey on deep learning for named entity recognition. IEEE Trans. Knowl. Data Eng. 34(1), 50–70 (2020)
Li, X., Yan, H., Qiu, X., Huang, X.: Flat: Chinese NER using flat-lattice transformer. arXiv preprint arXiv:2004.11795 (2020)
Liu, W., Fu, X., Zhang, Y., Xiao, W.: Lexicon enhanced Chinese sequence labeling using BERT adapter. arXiv preprint arXiv:2105.07148 (2021)
Liu, Y., Ott, M., Goyal, N., Du, J., Joshi, M., Chen, D., Levy, O., Lewis, M., Zettlemoyer, L., Stoyanov, V.: Roberta: a robustly optimized BERT pretraining approach. arXiv preprint arXiv:1907.11692 (2019)
Ma, R., Peng, M., Zhang, Q., Huang, X.: Simplify the usage of lexicon in Chinese NER. arXiv preprint arXiv:1908.05969 (2019)
Meng, T., Fang, A., Rokhlenko, O., Malmasi, S.: GEMNET: effective gated gazetteer representations for recognizing complex entities in low-context input. In: Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pp. 1499–1512 (2021)
Mengge, X., Bowen, Y., Tingwen, L., Yue, Z., Erli, M., Bin, W.: Porous lattice-based transformer encoder for Chinese NER. arXiv preprint arXiv:1911.02733 (2019)
Snell, J., Swersky, K., Zemel, R.: Prototypical networks for few-shot learning. In: Advances in Neural Information Processing Systems, vol. 30 (2017)
Sperber, M., Neubig, G., Pham, N.-Q., Waibel, A.: Self-attentional models for lattice inputs. arXiv preprint arXiv:1906.01617 (2019)
Sui, D., Chen, Y., Liu, K., Zhao, J., Liu, S.: Leverage lexical knowledge for Chinese named entity recognition via collaborative graph network. In: Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pp. 3830–3840 (2019)
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, Ł., Polosukhin, I.: Attention is all you need. In: Advances in Neural Information Processing Systems, vol. 30 (2017)
Wang, G., Li, C., Wang, W., Zhang, Y., Shen, D., Zhang, X., Henao, R., Carin, L.: Joint embedding of words and labels for text classification. arXiv preprint arXiv:1805.04174 (2018)
Wiseman, S., Stratos, K.: Label-agnostic sequence labeling by copying nearest neighbors. arXiv preprint arXiv:1906.04225 (2019)
Wolf, T., Debut, L., Sanh, V., Chaumond, J., Delangue, C., Moi, A., Cistac, P., Rault, T., Louf, R., Funtowicz, M., et al.: Transformers: state-of-the-art natural language processing. In: Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: System Demonstrations, pp. 38–45 (2020)
Wu, F., Liu, J., Wu, C., Huang, Y., Xie, X.: Neural Chinese named entity recognition via CNN-LSTM-CRF and joint training with word segmentation. In: The World Wide Web Conference, pp. 3342–3348 (2019)
Xiao, F., Li, J., Zhao, H., Wang, R., Chen, K.: Lattice-based transformer encoder for neural machine translation. arXiv preprint arXiv:1906.01282 (2019)
Xu, L., Dong, Q., Liao, Y., Yu, C., Tian, Y., Liu, W., Li, L., Liu, C., Zhang, X., et al.: Cluener2020: fine-grained named entity recognition dataset and benchmark for Chinese. arXiv preprint arXiv:2001.04351 (2020)
Yang, B., Tu, Z., Wong, D.F., Meng, F., Chao, L.S., Zhang, T.: Modeling localness for self-attention networks. arXiv preprint arXiv:1810.10182 (2018)
Yang, B., Wang, L., Wong, D., Chao, L.S., Tu, Z.: Convolutional self-attention networks. arXiv preprint arXiv:1904.03107 (2019)
Yang, J., Zhang, Y., Dong, F.: Neural reranking for named entity recognition. arXiv preprint arXiv:1707.05127 (2017)
Zhang, Z., Sabuncu, M.: Generalized cross entropy loss for training deep neural networks with noisy labels. In: Advances in Neural Information Processing Systems, vol. 31 (2018)
Ziyadi, M., Sun, Y., Goswami, A., Huang, J., Chen, W.: Example-based named entity recognition. arXiv preprint arXiv:2008.10570 (2020)
Acknowledgements
This work was supported in part by the National Key R&D Program of China (2022ZD0119602), National Natural Science Foundation of China (62272114), Major Key Project of PCL (PCL2022A03, PCL2021A02, PCL2021A09), Joint Research Foundation of Guangzhou University (202201020380), Guangdong Higher Education Innovation Group (2020KCXTD007), and Pearl River Scholars Funding Program of Guangdong Universities (2019).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2024 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Chen, H. et al. (2024). Few-Shot Blockchain Domain Named Entity Recognition with Fused Lexical Features. In: Li, S. (eds) Computational and Experimental Simulations in Engineering. ICCES 2023. Mechanisms and Machine Science, vol 146. Springer, Cham. https://doi.org/10.1007/978-3-031-44947-5_48
Download citation
DOI: https://doi.org/10.1007/978-3-031-44947-5_48
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-44946-8
Online ISBN: 978-3-031-44947-5
eBook Packages: EngineeringEngineering (R0)