Abstract
In recent years many multi-label classification methods have exploited label dependencies to improve performance of classification tasks in various domains, hence casting the tasks to structured prediction problems. We argue that multi-label predictions do not always satisfy domain constraint restrictions. For example when the dialogue state tracking task in task-oriented dialogue domains is solved with multi-label classification approaches, slot-value constraint rules should be enforced following real conversation scenarios.
To address these issues we propose an energy-based neural model to solve the dialogue state tracking task as a structured prediction problem. Furthermore we propose two improvements over previous methods with respect to dialogue slot-value constraint rules: (i) redefining the estimation conditions for the energy network; (ii) regularising label predictions following the dialogue slot-value constraint rules. In our results we find that our extended energy-based neural dialogue state tracker yields better overall performance in term of prediction accuracy, and also behaves more naturally with respect to the conversational rules.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
Notes
- 1.
From here we simplify the name of this dataset to WOZ as in common practice.
References
Belanger, D., McCallum, A.: Structured prediction energy networks. In: Proceedings of the 33rd International Conference on Machine Learning, vol. 48 (2016)
Gygli, M., Norouzi, M., Angelova, A.: Deep value networks learn to evaluate and iteratively refine structured outputs. In: Proceedings of the 34th International Conference on Machine Learning (2017)
Henderson, M., Thomson, B., Williams, J.D.: The second dialog state tracking challenge. In: Proceedings of the SIGDIAL 2014 Conference, pp. 263–272 (2014)
Henderson, M., Thomson, B., Williams, J.D.: The third dialog state tracking challenge. In: Proceedings of 2014 IEEE Workshop on Spoken Language Technology, pp. 324–329 (2014)
Henderson, M., Thomson, B., Young, S.: Robust dialog state tracking using delexicalised recurrent neural networks and unsupervised adaptation. In: Proceedings of 2014 IEEE Workshop on Spoken Language Technology, pp. 360–365 (2014)
Henderson, M., Thomson, B., Young, S.: Word-based dialog state tracking with recurrent neural networks. In: Proceedings of the SIGDIAL 2014 Conference, pp. 292–299 (2014)
Hochreiter, S., Schmidhuber, J.: Long short-term memory. Neural Comput. 9(8), 1735–1780 (1997). https://doi.org/10.1162/neco.1997.9.8.1735
LeCun, Y., Chopra, S., Hadsell, R., Ranzato, M.A., Huang, F.J.: A tutorial on energy-based learning. In: Predicting Structured Data (2006)
Mrksic, N., et al.: Multi-domain dialog state tracking using recurrent neural networks. In: Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics, pp. 794–799 (2015)
Mrksic, N., O’Seaghdha, D., Wen, T.H., Thomson, B., Young, S.: Neural belief tracker: data-driven dialogue state tracking. In: Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (2017). https://doi.org/10.18653/v1/P17-1163
Nouri, E., Hosseini-Asl, E.: Toward scalable neural dialogue state tracking model. In: 32nd Conference on Neural Information Processing Systems (NeurIPS 2018), 2nd Conversational AI Workshop (2018)
Smith, R.W.: Comparative error analysis of dialog state tracking. In: Proceedings of the SIGDIAL 2014 Conference, pp. 300–309 (2014)
Trinh, A.D., Ross, R.J., Kelleher, J.D.: A multi-task approach to incremental dialogue state tracking. In: Proceedings of the 22nd Workshop on the Semantics and Pragmatics of Dialogue, SEMDIAL, pp. 132–145 (2018)
Trinh, A.D., Ross, R.J., Kelleher, J.D.: Capturing dialogue state variable dependencies with an energy-based neural dialogue state tracker. In: Proceedings of the SIGDial 2019 Conference, pp. 75–84 (2019)
Trinh, A.D., Ross, R.J., Kelleher, J.D.: Energy-based modelling for dialogue state tracking. In: Proceedings of the 1st Workshop on NLP for Conversational AI, pp. 77–86 (2019)
Trinh, A.D., Ross, R.J., Kelleher, J.D.: Investigating variable dependencies in dialogue states. In: Proceedings of the 23rd Workshop on the Semantics and Pragmatics of Dialogue, pp. 195–197 (2019)
Vodolan, M., Kadlec, R., Kleindienst, J.: Hybrid dialog state tracker with ASR features. In: Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics, EACL, vol. 2, pp. 205–210 (2017)
Wang, B., Li, C., Pavlu, V., Aslam, J.: Regularizing model complexity and label structure for multi-label text classification. In: Proceedings of KDD 2017, Halifax, Nova Scotia, Canada (2017)
Wen, T.H., et al.: A network-based end-to-end trainable task-oriented dialogue system. In: Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics, EACL, pp. 438–449 (2017)
Williams, J.D., Raux, A., Henderson, M.: The dialog state tracking challenge series: a review. Dialogue Discourse 7(3), 4–33 (2016). https://doi.org/10.5087/dad.2016.301
Zhong, V., Xiong, C., Socher, R.: Global-locally self-attentive dialogue state tracker. In: Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics, pp. 1458–1467 (2018)
Acknowledgements
This research was conducted with the financial support of Science Foundation Ireland under Grant Agreement No. 13/RC/2106 at the ADAPT SFI Research Centre at Technological University Dublin.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Switzerland AG
About this paper
Cite this paper
Trinh, A.D., Ross, R.J., Kelleher, J.D. (2020). F-Measure Optimisation and Label Regularisation for Energy-Based Neural Dialogue State Tracking Models. In: Farkaš, I., Masulli, P., Wermter, S. (eds) Artificial Neural Networks and Machine Learning – ICANN 2020. ICANN 2020. Lecture Notes in Computer Science(), vol 12397. Springer, Cham. https://doi.org/10.1007/978-3-030-61616-8_64
Download citation
DOI: https://doi.org/10.1007/978-3-030-61616-8_64
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-61615-1
Online ISBN: 978-3-030-61616-8
eBook Packages: Computer ScienceComputer Science (R0)