Abstract
Large language models are beneficial for various natural language tasks. However, outdated knowledge in their parameters lead to erroneous outputs. To address it, researchers have proposed methods for editing the model to update the knowledge. Nevertheless, these approaches have not explored the use of instance-level information to guide desired outputs, nor have they effectively rectified commonsense errors in specific contextual scenarios. To tackle this, we establish a benchmark for evidence-based commonsense correction in question-answering. We propose assessment metrics and employ a self-retrieval strategy to extract relevant evidence. Using a hypernetwork, we dynamically inject evidence during correction, yielding improved results over baseline methods. The code is available at https://github.com/xinykou/edit_knwoledge.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Chen, J., Shi, W., Fu, Z., Cheng, S., et al: Say what you mean! large language models speak too positively about negative commonsense knowledge. In: Proceedings of ACL (2023)
Chen, Z., Weiss, G., Mitchell, E., et al: Reckoning: reasoning through dynamic knowledge encoding. Advances in Neural Information Processing Systems (2024)
Gupta, A., Mondal, D., et al: Editing commonsense knowledge in gpt. arXiv preprint arXiv:2305.14956 (2023)
Hartvigsen, T., et al: Aging with grace: Lifelong model editing with discrete key-value adaptors. Advances in Neural Information Processing Systems (2024)
Hu, E.J., Wallis, P., et al: Lora: Low-rank adaptation of large language models. In: International Conference on Learning Representations (2021)
Huang, Z., Shen, Y., Zhang, X., et al: Transformer-patcher: One mistake worth one neuron. In: The Eleventh International Conference on Learning Representations (2022)
Ivison, H., Peters, M.E.: Hyperdecoders: Instance-specific decoders for multi-task nlp. In: Proceedings of EMNLP (2022)
Kirkpatrick, J., Pascanu, R., Rabinowitz, N., et al: Overcoming catastrophic forgetting in neural networks. Proceedings of the National Academy of Sciences of the United States of America 114(13), 3521–3526 (2017)
Lee, K., Han, W., Hwang, S.w., Lee, H., Park, J., Lee, S.W.: Plug-and-play adaptation for continuously-updated qa. In: Proceedings of ACL (2022)
Li, D., Rawat, A.S., Zaheer, M., Wang, X., Lukasik, M., Veit, A., Yu, F., Kumar, S.: Large language models with controllable working memory. In: Proceedings of ACL (2023)
Mitchell, E., Lin, C., et al: Memory-based model editing at scale. In: International Conference on Machine Learning (2022)
Mitchell, E., Lin, C., Bosselut, A., et al: Fast model editing at scale. In: International Conference on Learning Representations (2021)
Neeman, E., Aharoni, R., et al: Disentqa: Disentangling parametric and contextual knowledge with counterfactual question answering. In: Proceedings of ACL (2023)
Rolnick, D., Ahuja, A., Schwarz, J., Lillicrap, T., Wayne, G.: Experience replay for continual learning. Advances in neural information processing systems 32 (2019)
Thorne, J., Vlachos, A., Christodoulopoulos, C., Mittal, A.: Fever: a large-scale dataset for fact extraction and verification. In: Proceedings of ACL (2018)
Wang, Z., Pan, X., Yu, D., Yu, D., Chen, J., Ji, H.: Zemi: Learning zero-shot semi-parametric language models from multiple tasks. In: Proceedings of ACL (2023)
Yao, Y., Wang, P., Tian, B., et al: Editing large language models: Problems, methods, and opportunities. In: Proceedings of EMNLP (2023)
Zhou, W., Zhang, S., Poon, H., Chen, M.: Context-faithful prompting for large language models. In: Proceedings of EMNLP (2023)
Acknowledgement
This work was supported by National Key R&D Program of China (No. 2021YFC3340700), NSFC grant (No. 62136002), Ministry of Education Research Joint Fund Project (8091B042239) and Shanghai Trusted Industry Internet Software Collaborative Innovation Center.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2024 The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Yi, X., Wang, L., Wang, X., He, L. (2024). Scenarios-aware Commonsense Correcton via Instance-level Knowledge Injection. In: Onizuka, M., et al. Database Systems for Advanced Applications. DASFAA 2024. Lecture Notes in Computer Science, vol 14854. Springer, Singapore. https://doi.org/10.1007/978-981-97-5569-1_23
Download citation
DOI: https://doi.org/10.1007/978-981-97-5569-1_23
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-97-5568-4
Online ISBN: 978-981-97-5569-1
eBook Packages: Computer ScienceComputer Science (R0)