Abstract
Reasoning commonsense knowledge is essential for Artificial Intelligence, which requires high-quality commonsense knowledge. Recently, much progress has been made in automatic commonsense knowledge generation. However, most of the works focus on obtaining positive knowledge and lack negative information. Only a few works capture the importance of negative statements, but they struggle to produce high-quality knowledge. Although some efforts have been made to generate negative statements, they fail to consider the taxonomic hierarchy between entities and are not generally applicable, leading to the generation of low-quality negative samples. To resolve the issue, we put forward Negation, a framework for effectively generating hard negative knowledge. For each entity in the commonsense knowledge base, congeners are identified with hierarchical and semantic information. Then, negative candidates are produced by replacing the entity with congeners in each triple. In order to make negative knowledge more confusing and avoid false positive examples, we design two filtering steps to remove the amount of meaningless candidates. We empirically evaluate our proposed method Negation on the downstream task, and the results demonstrate that Negation and its components effectively help generate high-quality negative knowledge.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Speer, R., Chin, J., Havasi, C.: Conceptnet 5.5: An open multilingual graph of general knowledge. In: Proceedings of the AAAI Conference on Artificial Intelligence. vol. 31 (2017)
Sap, M., et al.: Atomic: an atlas of machine commonsense for if-then reasoning. In: Proceedings of the AAAI Conference on Artificial Intelligence. vol. 33, pp. 3027–3035 (2019)
Nguyen, T.P., Razniewski, S., Weikum, G.: Advanced semantics for commonsense knowledge extraction. In: Proceedings of the Web Conference 2021, pp. 2636–2647 (2021)
Nguyen, T.P., Razniewski, S., Romero, J., Weikum, G.: Refined commonsense knowledge from large-scale web contents. IEEE Trans. Knowl. Data Eng. (2022)
Chen, Z., Wang, Y., Zhao, B., Cheng, J., Zhao, X., Duan, Z.: Knowledge graph completion: a review. IEEE Access 8, 192435–192456 (2020)
Lin, Q., Mao, R., Liu, J., Xu, F., Cambria, E.: Fusing topology contexts and logical rules in language models for knowledge graph completion. Inf. Fusion 90, 253–264 (2023)
Safavi, T., Zhu, J., Koutra, D.: Negater: unsupervised discovery of negatives in commonsense knowledge bases. arXiv preprint arXiv:2011.07497 (2020)
Arnaout, H., Razniewski, S., Weikum, G., Pan, J.Z.: Uncommonsense: Informative negative knowledge about everyday concepts. In: Proceedings of the 31st ACM International Conference on Information Knowledge Management, pp. 37–46 (2022)
Arnaout, H., Razniewski, S.: Can large language models generate salient negative statements? arXiv preprint arXiv:2305.16755 (2023)
Chen, J., Shi, W., Fu, Z., Cheng, S., Li, L., Xiao, Y.: Say what you mean! large language models speak too positively about negative commonsense knowledge. arXiv preprint. arXiv:2305.05976 (2023)
Galárraga, L., Razniewski, S., Amarilli, A., Suchanek, F.M.: Predicting completeness in knowledge bases. In: Proceedings of the tenth ACM International Conference on Web Search and Data Mining, pp. 375–383 (2017)
Romero, J., Razniewski, S., Pal, K., Z. Pan, J., Sakhadeo, A., Weikum, G.: Commonsense properties from query logs and question answering forums. In: Proceedings of the 28th ACM International Conference on Information and Knowledge Management, pp. 1411–1420 (2019)
Onoe, Y., Boratko, M., McCallum, A., Durrett, G.: Modeling fine-grained entity types with box embeddings. arXiv preprint. arXiv:2101.00345 (2021)
Devlin, J., Chang, M.W., Lee, K., Toutanova, K.: Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint. arXiv:1810.04805 (2018)
Ponza, M., Ferragina, P., Chakrabarti, S.: A two-stage framework for computing entity relatedness in wikipedia. In: Proceedings of the 2017 ACM on Conference on Information and Knowledge Management, pp. 1867–1876 (2017)
Pennington, J., Socher, R., Manning, C.D.: Glove: Global vectors for word representation. In: Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), pp. 1532–1543 (2014)
Dasgupta, S., Boratko, M., Zhang, D., Vilnis, L., Li, X., McCallum, A.: Improving local identifiability in probabilistic box embeddings. Adv. Neural. Inf. Process. Syst. 33, 182–192 (2020)
Chheda, T., G.: Box embeddings: an open-source library for representation learning using geometric structures. arXiv preprint arXiv:2109.04997 (2021)
Li, X., Taheri, A., Tu, L., Gimpel, K.: Commonsense knowledge base completion. In: Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 1445–1455 (2016)
Cai, T.T., Frankle, J., Schwab, D.J., Morcos, A.S.: Are all negatives created equal in contrastive instance discrimination? arXiv preprint arXiv:2010.06682 (2020)
Zhao, W.X., et al.: A survey of large language models. arXiv preprint. arXiv:2303.18223 (2023)
OpenAI: Gpt-4 technical report (2023)
Anil, R., et al.: Palm 2 technical report. arXiv preprint. arXiv:2305.10403 (2023)
Touvron, H., et al.: Llama: open and efficient foundation language models. arXiv preprint arXiv:2302.13971 (2023)
Du, Z., et al.: GLM: General language model pretraining with autoregressive blank infilling. arXiv preprint. arXiv:2103.10360 (2021)
Zeng, A., et al.: GLM-130b: An open bilingual pre-trained model. arXiv preprint. arXiv:2210.02414 (2022)
Speer, R., et al.: Representing general relational knowledge in conceptnet 5. In: LREC. vol. 2012, pp. 3679–86 (2012)
Lin, T.Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017)
Saito, I., Nishida, K., Asano, H., Tomita, J.: Commonsense knowledge base completion and generation. In: Proceedings of the 22nd Conference on Computational Natural Language Learning, pp. 141–150 (2018)
Bosselut, A., Rashkin, H., Sap, M., Malaviya, C., Celikyilmaz, A., Choi, Y.: Comet: commonsense transformers for automatic knowledge graph construction. arXiv preprint. arXiv:1906.05317 (2019)
Sun, Z., Deng, Z.H., Nie, J.Y., Tang, J.: Rotate: Knowledge graph embedding by relational rotation in complex space. arXiv preprint arXiv:1902.10197 (2019)
Ahrabian, K., Feizi, A., Salehi, Y., Hamilton, W.L., Bose, A.J.: Structure aware negative sampling in knowledge graphs. arXiv preprint. arXiv:2009.11355 (2020)
Wei, J., et al.: Emergent abilities of large language models. arXiv preprint. arXiv:2206.07682 (2022)
Kaddour, J., Harris, J., Mozes, M., Bradley, H., Raileanu, R., McHardy, R.: Challenges and applications of large language models. arXiv preprint. arXiv:2307.10169 (2023)
Nguyen, T.P., Razniewski, S., Varde, A., Weikum, G.: Extracting cultural commonsense knowledge at scale. In: Proceedings of the ACM Web Conference 2023, pp. 1907–1917 (2023)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2024 The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Sheng, Y., Zeng, W., Tang, J. (2024). Negation: An Effective Method to Generate Hard Negatives. In: Song, X., Feng, R., Chen, Y., Li, J., Min, G. (eds) Web and Big Data. APWeb-WAIM 2023 International Workshops. APWeb-WAIM 2023. Communications in Computer and Information Science, vol 2094. Springer, Singapore. https://doi.org/10.1007/978-981-97-2991-3_3
Download citation
DOI: https://doi.org/10.1007/978-981-97-2991-3_3
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-97-2990-6
Online ISBN: 978-981-97-2991-3
eBook Packages: Computer ScienceComputer Science (R0)