Abstract
Relation classification is a key natural language processing task that receives much attentions these years. The goal is to assign pre-defined relation labels to the nominal pairs marked in given sentences. It is obvious that different words in a sentence are differentially informative. Moreover, the importance of words is highly relation-dependent, i.e., the same word may be differentially important for different relations. To include sensitivity to this fact, we present a novel model, referred to as TCA-CNN, which takes the attention mechanism at the word level to pay different attention to individual words according to the semantic relation concentrated when constructing the representation of a sentence. Experimental results show that TCA-CNN achieves a comparable performance compared with the state-of-the-art models on the SemEval 2010 relation classification task.
J. Zhu–The work was conducted when Jizhao Zhu visited CAS Key Lab of Network Data Science and Technology.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Wu, F., Weld, D.S.: Open information extraction using Wikipedia. In: 48th Annual Meeting of the Association for Computational Linguistics, pp. 118–127. ACL Press, Stroudsburg (2010)
Golub, D., He, X.: Character-level question answering with attention. arXiv preprint arXiv:1604.00727 (2016)
Shin, J., Wu, S., Wang, F., De Sa, C., Zhang, C., Ré, C.: Incremental knowledge base construction using deepdive. Proc. VLDB Endowment 8, 1310–1321 (2015)
Jia, Y., Wang, Y., Lin, H., Jin, X., Cheng, X.: Locally adaptive translation for knowledge graph embedding. In: 30th AAAI Conference on Artificial Intelligence, pp. 992–998. AAAI Press, Menlo Park (2016)
LeCun, Y., Bottou, L., Bengio, Y., Haffner, P.: Gradient-based learning applied to document recognition. Proc. IEEE 86, 2278–2324 (1998)
Socher, R., Huval, B., Manning, C.D., Ng, A.Y.: Semantic compositionality through recursive matrix-vector spaces. In: 2012 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning, pp. 1201–1211. ACL Press, Stroudsburg (2012)
Xu, Y., Mou, L., Li, G., Chen, Y., Peng, H., Jin, Z.: Classifying relations via long short term memory networks along shortest dependency paths. In: 2015 Conference on Empirical Methods in Natural Language Processing, pp. 1785–1794. ACL Press, Stroudsburg (2015)
Wang, L., Cao, Z., de Melo, G., Liu, Z.: Relation classification via multi-level attention cnns. In: 54th Annual Meeting of the Association for Computational Linguistics, pp. 1398–1307. ACL Press, Stroudsburg (2016)
Hochreiter, S., Schmidhuber, J.: Long short-term memory. Neural Comput. 9, 1735–1780 (1997)
Cho, K., Van Merriënboer, B., Bahdanau, D., Bengio, Y.: On the properties of neural machine translation: Encoder-decoder approaches. arXiv preprint arXiv:1409.1259 (2014)
Zeng, D., Liu, K., Lai, S., Zhou, G., Zhao, J.: Relation classification via convolutional deep neural network. In: 25th International Conference on Computatinal Linguistics: Technical Papers, pp. 2335–2344. ACM, New York (2014)
Mikolov, T., Chen, K., Corrado, G., Dean, J.: Efficient estimation of word representations in vector space. arXiv preprint arXiv:1301.3781 (2013)
Duchi, J., Hazan, E., Singer, Y.: Adaptive subgradient methods for online learning and stochastic optimization. J. Mach. Learn. Res. 12, 2121–2159 (2011)
Hendrickx, I., Kim, S. N., Kozareva, Z., Nakov, P., Ó Séaghdha, D., Padó, S., Pennacchiotti, M., Romano, L., Szpakowicz, S.: Semeval-2010 task 8: multi-way classification of semantic relations between pairs of nominals. In: Proceedings of the Workshop on Semantic Evaluations: Recent Achievements and Future Directions, pp. 94–99. ACL Press, Stroudsburg (2009)
Santos, C.N.D., Xiang, B., Zhou, B.: Classifying relations by ranking with convolutional neural networks. arXiv preprint arXiv:1504.06580 (2015)
Liu, Y., Wei, F., Li, S., Ji, H., Zhou, M., Wang, H.: A dependency-based neural network for relation classification. In: 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing(Short Papers), pp. 285–290. ACL Press, Stroudsburg (2015)
Xu, K., Feng, Y., Huang, S., Zhao, D.: Semantic relation classification via convolutional neural networks with simple negative sampling. In: 2015 Conference on Empirical Methods in Natural Language Processing, pp. 536–540. ACL Press, Stroudsburg (2015)
Xu, Y., Jia, R., Mou, L., Li, G., Chen, Y., Lu, Y., Jin, Z.: Improved relation classification by deep recurrent neural networks with data augmentation. arXiv preprint arXiv:1601.03651 (2016)
Acknowledgments
This work is supported by the 973 Program of China under Grant Nos. 2013CB329606 and 2014CB340405, the National Key Research and Development Program of China under Grant No. 2016YFB1000902, the National Natural Science Foundation of China (NSFC) under Grant Nos. 61272177, 61402442, 61572469, 91646120 and 61572473.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2017 Springer International Publishing AG
About this paper
Cite this paper
Zhu, J., Qiao, J., Dai, X., Cheng, X. (2017). Relation Classification via Target-Concentrated Attention CNNs. In: Liu, D., Xie, S., Li, Y., Zhao, D., El-Alfy, ES. (eds) Neural Information Processing. ICONIP 2017. Lecture Notes in Computer Science(), vol 10635. Springer, Cham. https://doi.org/10.1007/978-3-319-70096-0_15
Download citation
DOI: https://doi.org/10.1007/978-3-319-70096-0_15
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-70095-3
Online ISBN: 978-3-319-70096-0
eBook Packages: Computer ScienceComputer Science (R0)