Abstract
This paper presents a property-directed approach to verifying recurrent neural networks (RNNs). To this end, we learn a deterministic finite automaton as a surrogate model from a given RNN using active automata learning. This model may then be analyzed using model checking as a verification technique. The term property-directed reflects the idea that our procedure is guided and controlled by the given property rather than performing the two steps separately. We show that this not only allows us to discover small counterexamples fast, but also to generalize them by pumping towards faulty flows hinting at the underlying error in the RNN. We also show that our method can be efficiently used for adversarial robustness certification of RNNs.
The first four authors contributed equally, the remaining authors are ordered alphabetically. This work was partly supported by the PHC PROCOPE 2020 project LeaRNNify (number 44707TK), funded by DAAD and Campus France and the Deutsche Forschungsgemeinschaft (DFG, German Research Foundation) grant number 434592664.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
Notes
- 1.
In the index of the right congruence associated with L and in the size of the longest counterexample obtained as a reply to an EQ.
- 2.
References
Akintunde, M.E., Kevorchian, A., Lomuscio, A., Pirovano, E.: Verification of RNN-based neural agent-environment systems. In: Proceedings of AAAI 2019, pp. 6006–6013. AAAI Press (2019). https://doi.org/10.1609/aaai.v33i01.33016006
Angluin, D.: Learning regular sets from queries and counterexamples. Inf. Comput. 75(2), 87–106 (1987)
Ayache, S., Eyraud, R., Goudian, N.: Explaining black boxes on sequential data using weighted automata. In: Proceedings of ICGI 2018, Proceedings of Machine Learning Research, vol. 93, pp. 81–103. PMLR (2018)
Baier, C., Katoen, J.: Principles of Model Checking. MIT Press, Cambridge (2008)
Bernardi, O., Giménez, O.: A linear algorithm for the random sampling from regular languages. Algorithmica 62(1–2), 130–145 (2012)
Cho, K., et al.: Learning phrase representations using RNN encoder-decoder for statistical machine translation. In: Proceedings of the EMNLP, pp. 1724–1734. ACL (2014)
Clarke, E., Grumberg, O., Jha, S., Lu, Y., Veith, H.: Counterexample-guided abstraction refinement. In: Emerson, E.A., Sistla, A.P. (eds.) CAV 2000. LNCS, vol. 1855, pp. 154–169. Springer, Heidelberg (2000). https://doi.org/10.1007/10722167_15
Du, X., Li, Y., Xie, X., Ma, L., Liu, Y., Zhao, J.: Marble: model-based robustness analysis of stateful deep learning systems. In: ASE 2020, pp. 423–435. IEEE (2020)
Elboher, Y.Y., Gottschlich, J., Katz, G.: An abstraction-based framework for neural network verification. In: Lahiri, S.K., Wang, C. (eds.) CAV 2020. LNCS, vol. 12224, pp. 43–65. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-53288-8_3
Giacomo, G.D., Vardi, M.Y.: Synthesis for LTL and LDL on finite traces. In: Proceedings of IJCAI 2015, pp. 1558–1564. AAAI Press (2015)
Hochreiter, S., Schmidhuber, J.: Long short-term memory. Neural Comput. 9(8), 1735–1780 (1997)
Holme, P.: Temporal networks. In: Encyclopedia of Social Network Analysis and Mining, pp. 2119–2129. Springer, Heidelberg (2014)
Jacoby, Y., Barrett, C.W., Katz, G.: Verifying recurrent neural networks using invariant inference. CoRR abs/2004.02462 (2020)
Keck, C.: Principles of Public Health Practice. Cengage Learning (2002)
Kwiatkowska, M.Z.: Safety verification for deep neural networks with provable guarantees (Invited Paper). In: Proceedings of CONCUR 2019. Leibniz International Proceedings in Informatics (LIPIcs), vol. 140, pp. 1:1–1:5. Schloss Dagstuhl-Leibniz-Zentrum fuer Informatik (2019)
Mayr, F., Visca, R., Yovine, S.: On-the-fly black-box probably approximately correct checking of recurrent neural networks. In: Holzinger, A., Kieseberg, P., Tjoa, A.M., Weippl, E. (eds.) CD-MAKE 2020. LNCS, vol. 12279, pp. 343–363. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-57321-8_19
Mayr, F., Yovine, S.: Regular inference on artificial neural networks. In: Holzinger, A., Kieseberg, P., Tjoa, A.M., Weippl, E. (eds.) CD-MAKE 2018. LNCS, vol. 11015, pp. 350–369. Springer, Cham (2018). https://doi.org/10.1007/978-3-319-99740-7_25
Mayr, F., Yovine, S., Visca, R.: Property checking with interpretable error characterization for recurrent neural networks. Mach. Learn. Knowl. Extr. 3(1), 205–227 (2021)
Merten, M.: Active automata learning for real life applications. Ph.D. thesis, Dortmund University of Technology (2013)
Okudono, T., Waga, M., Sekiyama, T., Hasuo, I.: Weighted automata extraction from recurrent neural networks via regression on state spaces. In: Proceedings of AAAI 2020, pp. 5306–5314. AAAI Press (2020)
Omlin, C.W., Giles, C.L.: Extraction of rules from discrete-time recurrent neural networks. Neural Netw. 9(1), 41–52 (1996)
Peled, D.A., Vardi, M.Y., Yannakakis, M.: Black box checking. J. Autom. Lang. Comb. 7(2), 225–246 (2002)
Ryou, W., Chen, J., Balunovic, M., Singh, G., Dan, A.M., Vechev, M.T.: Fast and effective robustness certification for recurrent neural networks. CoRR abs/2005.13300 (2020)
Schulz, K.U., Mihov, S.: Fast string correction with Levenshtein automata. Int. J. Document Anal. Recogn. 5(1), 67–85 (2002)
Weiss, G., Goldberg, Y., Yahav, E.: Extracting automata from recurrent neural networks using queries and counterexamples. In: Proceedings of ICML 2018. Proceedings of Machine Learning Research, vol. 80, pp. 5244–5253. PMLR (2018)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 Springer Nature Switzerland AG
About this paper
Cite this paper
Khmelnitsky, I. et al. (2021). Property-Directed Verification and Robustness Certification of Recurrent Neural Networks. In: Hou, Z., Ganesh, V. (eds) Automated Technology for Verification and Analysis. ATVA 2021. Lecture Notes in Computer Science(), vol 12971. Springer, Cham. https://doi.org/10.1007/978-3-030-88885-5_24
Download citation
DOI: https://doi.org/10.1007/978-3-030-88885-5_24
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-88884-8
Online ISBN: 978-3-030-88885-5
eBook Packages: Computer ScienceComputer Science (R0)