Abstract
This paper presents a summary and meta-analysis of the first three iterations of the annual International Verification of Neural Networks Competition (VNN-COMP), held in 2020, 2021, and 2022. In the VNN-COMP, participants submit software tools that analyze whether given neural networks satisfy specifications describing their input-output behavior. These neural networks and specifications cover a variety of problem classes and tasks, corresponding to safety and robustness properties in image classification, neural control, reinforcement learning, and autonomous systems. We summarize the key processes, rules, and results, present trends observed over the last three years, and provide an outlook into possible future developments.
Article PDF
Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.Avoid common mistakes on your manuscript.
References
Simple Adversarial Generator. https://github.com/stanleybak/simple_adversarial_generator. Accessed: 2022-09-13
VNN-COMP2020 report. https://www.overleaf.com/project/5f0c85e8d15dc10001749fa9. Accessed: 2022-08-28
Bai, J., Lu, F., Zhang, K., et al.: Onnx: open neural network exchange (2019). https://github.com/onnx/onnx
Bak, S.: Execution-guided overapproximation (ego) for improving scalability of neural network verification (2020)
Bak, S., Liu, C., Johnson, T.: The second international verification of neural networks competition (VNN-COMP 2021): summary and results (2021). https://doi.org/10.48550/ARXIV.2109.00498
Balunovic, M., Vechev, M.T.: Adversarial training and provable defenses: bridging the gap. In: 8th International Conference on Learning Representations, ICLR 2020, Addis Ababam, Ethiopia, April 26–30, 2020 (2020). https://openreview.net/forum?id=SJxSDxrKDr
Bojarski, M., Del Testa, D., Dworakowski, D., Firner, B., Flepp, B., Goyal, P., Jackel, L.D., Monfort, M., Muller, U., Zhang, J., Zhang, X., Zhao, J., Zieba, K.: End to end learning for self-driving cars (2016). https://doi.org/10.48550/ARXIV.1604.07316
Botoeva, E., Kouvaros, P., Kronqvist, J., Lomuscio, A., Misener, R.: Efficient verification of neural networks via dependency analysis. In: Proceedings of the 34th AAAI Conference on Artificial Intelligence (AAAI20). AAAI Press, Menlo Park (2020)
Brix, C., Noll, T.: Debona: decoupled boundary network analysis for tighter bounds and faster adversarial robustness proofs. CoRR (2020). arXiv:2006.09040 [abs]
Bunel, R., De Palma, A., Desmaison, A., Dvijotham, K., Kohli, P., Torr, P.H., Kumar, M.P.: Lagrangian decomposition for neural network verification. In: Conference on Uncertainty in Artificial Intelligence (2020)
Bunel, R., Lu, J., Turkaslan, I., Kohli, P., Torr, P., Kumar, M.P.: Branch and bound for piecewise linear neural network verification. J. Mach. Learn. Res. 21, 1574–1612 (2020)
Bunel, R., Turkaslan, I., Torr, P.H.S., Kohli, P., Mudigonda, P.K.: A unified view of piecewise linear neural network verification. In: Bengio, S., Wallach, H., Larochelle, H., Grauman, K., Cesa-Bianchi, N., Garnett, R. (eds.) Advances in Neural Information Processing Systems, vol. 31, pp. 4795–4804. Curran Associates, Red Hook (2018). https://proceedings.neurips.cc/paper/2018/hash/be53d253d6bc3258a8160556dda3e9b2-Abstract.html
Ehlers, R.: Formal verification of piece-wise linear feed-forward neural networks. In: International Symposium on Automated Technology for Verification and Analysis, pp. 269–286 (2017). https://doi.org/10.1007/978-3-319-68167-2_19
Ferlez, J., Shoukry, Y.: AReN: assured ReLU NN architecture for model predictive control of LTI systems. In: Proceedings of the 23rd International Conference on Hybrid Systems: Computation and Control, HSCC ’20. ACM, New York (2020). https://doi.org/10.1145/3365365.3382213
Ferrari, C., Müller, M.N., Jovanovic, N., Vechev, M.T.: Complete verification via multi-neuron relaxation guided branch-and-bound. In: 10th International Conference on Learning Representations, ICLR 2022, Virtual Event, April 25–29, 2022 (2022). https://openreview.net/forum?id=l_amHf1oaK
Fischer, M., Sprecher, C., Dimitrov, D.I., Singh, G., Vechev, M.T.: Shared certificates for neural network verification. In: Shoham, S., Vizel, Y. (eds.) Computer Aided Verification – 34th International Conference, CAV 2022, Proceedings, Part I, Haifa, Israel, August 7–10, 2022. Lecture Notes in Computer Science, vol. 13371, pp. 127–148. Springer, Berlin (2022). https://doi.org/10.1007/978-3-031-13185-1_7
Gehr, T., Mirman, M., Drachsler-Cohen, D., Tsankov, P., Chaudhuri, S., Vechev, M.T.: AI2: safety and robustness certification of neural networks with abstract interpretation. In: 2018 IEEE Symposium on Security and Privacy, SP 2018, Proceedings, San Francisco, California, USA, 21–23 May 2018, pp. 3–18. IEEE Comput. Soc., Los Alamitos (2018). https://doi.org/10.1109/SP.2018.00058
Goodfellow, I.J., Shlens, J., Szegedy, C.: Explaining and harnessing adversarial examples. In: Bengio, Y., LeCun, Y. (eds.) 3rd International Conference on Learning Representations, ICLR 2015, Conference Track Proceedings, San Diego, CA, USA, May 7–9, 2015 (2015). http://arxiv.org/abs/1412.6572
Gowal, S., Dvijotham, K., Stanforth, R., Bunel, R., Qin, C., Uesato, J., Arandjelovic, R., Mann, T.A., Kohli, P.: On the effectiveness of interval bound propagation for training verifiably robust models. CoRR (2018). arXiv:1810.12715 [abs]
He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 770–778 (2016). https://doi.org/10.1109/CVPR.2016.90
Henriksen, P., Lomuscio, A.: Efficient neural network verification via adaptive refinement and adversarial search. In: Proceedings of the 24th European Conference on Artificial Intelligence (ECAI20) (2020)
Henriksen, P., Lomuscio, A.: Deepsplit: an efficient splitting method for neural network verification via indirect effect analysis. In: Proceedings of the 30th International Joint Conference on Artificial Intelligence (IJCAI21) (2021). https://doi.org/10.24963/ijcai.2021/351
Huang, X., Kwiatkowska, M., Wang, S., Wu, M.: Safety verification of deep neural networks. In: Majumdar, R., Kunčak, V. (eds.) Computer Aided Verification, pp. 3–29. Springer, Cham (2017)
Julian, K.D., Lopez, J., Brush, J.S., Owen, M.P., Kochenderfer, M.J.: Policy compression for aircraft collision avoidance systems. In: 2016 IEEE/AIAA 35th Digital Avionics Systems Conference (DASC), pp. 1–10 (2016). https://doi.org/10.1109/DASC.2016.7778091
Katz, G., Barrett, C., Dill, D.L., Julian, K., Kochenderfer, M.J.: Reluplex: An efficient SMT solver for verifying deep neural networks. In: Majumdar, R., Kunčak, V. (eds.) Computer Aided Verification, pp. 97–117. Springer, Cham (2017)
Katz, G., Huang, D.A., Ibeling, D., Julian, K., Lazarus, C., Lim, R., Shah, P., Thakoor, S., Wu, H., Zeljić, A., et al.: The Marabou framework for verification and analysis of deep neural networks. In: International Conference on Computer Aided Verification, pp. 443–452. Springer, Berlin (2019)
Khedr, H., Ferlez, J., Shoukry, Y.: Effective formal verification of neural networks using the geometry of linear regions. arXiv preprint (2020). arXiv:2006.10864
Kouvaros, P., Lomuscio, A.: Towards scalable complete verification of ReLU neural networks via dependency-based branching. In: Proceedings of the 30th International Joint Conference on Artificial Intelligence (IJCAI21) (2021). https://doi.org/10.24963/ijcai.2021/364
Liu, C., Arnon, T., Lazarus, C., Kochenderfer, M.J.: Neuralverification.jl: algorithms for verifying deep neural networks. In: ICLR 2019 Debugging Machine Learning Models Workshop (2019). https://debug-ml-iclr2019.github.io/cameraready/DebugML-19_paper_22.pdf
Liu, C., Arnon, T., Lazarus, C., Strong, C., Barrett, C., Kochenderfer, M.J.: Algorithms for verifying deep neural networks. Found. Trends Optim. 4(3–4), 244–404 (2021). https://doi.org/10.1561/2400000035
Lopez, D.M., Althoff, M., Benet, L., Chen, X., Fan, J., Forets, M., Huang, C., Johnson, T.T., Ladner, T., Li, W., Schilling, C., Zhu, Q.: Arch-comp22 category report: artificial intelligence and neural network control systems (AINNCS) for continuous and hybrid systems plants. In: Frehse, G., Althoff, M., Schoitsch, E., Guiochet, J. (eds.) Proceedings of 9th International Workshop on Applied Verification of Continuous and Hybrid Systems (ARCH22). EPiC Series in Computing, vol. 90, pp. 142–184 (2022). https://doi.org/10.29007/wfgr
Lopez, D.M., Musau, P., Tran, H.D., Dutta, S., Carpenter, T.J., Ivanov, R., Johnson, T.T.: Arch-comp19 category report: artificial intelligence and neural network control systems (ainncs) for continuous and hybrid systems plants. In: Frehse, G., Althoff, M. (eds.) ARCH19. 6th International Workshop on Applied Verification of Continuous and Hybrid Systems. EPiC Series in Computing, vol. 61, pp. 103–119 (2019). https://doi.org/10.29007/rgv8
Lu, J., Kumar, M.P.: Neural network branching for neural network verification. In: International Conference on Learning Representations (2020)
Madry, A., Makelov, A., Schmidt, L., Tsipras, D., Vladu, A.: Towards deep learning models resistant to adversarial attacks. In: 6th International Conference on Learning Representations, ICLR 2018, Conference Track Proceedings, Vancouver, BC, Canada, April 30–May 3, 2018 (2018). https://openreview.net/forum?id=rJzIBfZAb
Mirman, M., Gehr, T., Vechev, M.T.: Differentiable abstract interpretation for provably robust neural networks. In: Dy, J.G., Krause, A. (eds.) Proceedings of the 35th International Conference on Machine Learning, ICML 2018, Stockholmsmässan, Stockholm, Sweden, July 10–15, 2018. Proceedings of Machine Learning Research, vol. 80, pp. 3575–3583 (2018). http://proceedings.mlr.press/v80/mirman18b.html
Müller, M.N., Brix, C., Bak, S., Liu, C., Johnson, T.T.: The third international verification of neural networks competition (VNN-COMP 2022): summary and results (2022). https://doi.org/10.48550/arXiv.2212.10376
Müller, M.N., Eckert, F., Fischer, M., Vechev, M.T.: Certified training: small boxes are all you need. CoRR (2022). https://doi.org/10.48550/arXiv.2210.04871
Müller, M.N., Makarchuk, G., Singh, G., Püschel, M., Vechev, M.: Prima: precise and general neural network certification via multi-neuron convex relaxations. arXiv preprint (2021). arXiv:2103.03638
Isac, O., Barrett, C., Zhang, M., Katz, G.: Neural network verification with proof production. In: 22nd International Conference on Formal Methods in Computer-Aided Design (FMCAD) (2022)
De Palma, A., Behl, H.S., Bunel, R., Torr, P.H.S., Kumar, M.P.: Scaling the convex barrier with active sets. In: 9th International Conference on Learning Representations, ICLR 2021, Conference Track Proceedings, May 3–7, 2021 (2021). https://openreview.net/forum?id=uQfOy7LrlTR
De Palma, A., Behl, H.S., Bunel, R., Torr, P.H.S., Kumar, M.P.: Scaling the convex barrier with sparse dual algorithms. CoRR (2021). https://doi.org/10.48550/arXiv.2101.05844
De Palma, A., Bunel, R., Desmaison, Alban., Dvijotham, K., Kohli, P., Torr, P.H.S., Kumar, M.P.: Improved branch and bound for neural network verification via lagrangian decomposition. CoRR (2021). https://doi.org/10.48550/arXiv.2104.06718
De Palma, A., Bunel, R., Dvijotham, K., Kumar, M.P., Stanforth, R.: IBP regularization for verified adversarial robustness via branch-and-bound. (2022). https://doi.org/10.48550/arXiv.2206.14772
Serre, F., Müller, C., Singh, G., Püschel, M., Vechev, M.: Scaling polyhedral neural network verification on GPUs. In: Proc. Machine Learning and Systems (MLSys) (2021)
Shi, Z., Wang, Y., Zhang, H., Yi, J., Hsieh, C.: Fast certified robust training with short warmup. In: Ranzato, M., Beygelzimer, A., Dauphin, Y.N., Liang, P., Vaughan, J.W. (eds.) Advances in Neural Information Processing Systems 34: Annual Conference on Neural Information Processing Systems 2021, NeurIPS 2021, Virtual, December 6–14, 2021, pp. 18335–18349 (2021). https://proceedings.neurips.cc/paper/2021/hash/988f9153ac4fd966ea302dd9ab9bae15-Abstract.html
Shriver, D., Elbaum, S., Dwyer, M.B.: DNNV: a framework for deep neural network verification. In: Silva, A., Leino, K.R.M. (eds.) Computer Aided Verification, pp. 137–150. Springer, Cham (2021)
Shriver, D., Elbaum, S.G., Dwyer, M.B.: Reducing DNN properties to enable falsification with adversarial attacks. In: 43rd IEEE/ACM International Conference on Software Engineering, ICSE 2021, Madrid, Spain, 22–30 May 2021, pp. 275–287. IEEE (2021). https://doi.org/10.1109/ICSE43902.2021.00036
Singh, G., Ganvir, R., Püschel, M., Vechev, M.: Beyond the single neuron convex barrier for neural network certification. In: Advances in Neural Information Processing Systems, vol. 32, pp. 15098–15109. Curran Associates, Red Hook (2019)
Singh, G., Gehr, T., Mirman, M., Püschel, M., Vechev, M.: Fast and effective robustness certification. In: Bengio, S., Wallach, H., Larochelle, H., Grauman, K., Cesa-Bianchi, N., Garnett, R. (eds.) Advances in Neural Information Processing Systems, vol. 31, pp. 10802–10813. Curran Associates, Red Hook (2018). http://papers.nips.cc/paper/8278-fast-and-effective-robustness-certification.pdf
Singh, G., Gehr, T., Püschel, M., Vechev, M.: An abstract domain for certifying neural networks. Proc. ACM Program. Lang. 3(POPL), 41:1–41:30 (2019)
Singh, G., Gehr, T., Püschel, M., Vechev, M.: Boosting robustness certification of neural networks. In: Proc. International Conference on Learning Representations (ICLR) (2019)
Szegedy, C., Zaremba, W., Sutskever, I., Bruna, J., Erhan, D., Goodfellow, I.J., Fergus, R.: Intriguing properties of neural networks. In: Bengio, Y., LeCun, Y. (eds.) 2nd International Conference on Learning Representations, ICLR 2014, Conference Track Proceedings, Banff, AB, Canada, April 14–16, 2014 (2014). http://arxiv.org/abs/1312.6199
Tacchella, A., Pulina, L., Guidotti, D., Demarchi, S.: The verification of neural networks library (VNN-LIB) (2019). https://www.vnnlib.org
Tjeng, V., Xiao, K.Y., Tedrake, R.: Evaluating robustness of neural networks with mixed integer programming. In: ICLR (2019)
Tran, H.D., Bak, S., Xiang, W., Johnson, T.T.: Verification of deep convolutional neural networks using imagestars. In: 32nd International Conference on Computer-Aided Verification (CAV). Springer, Berlin (2020)
Tran, H.D., Musau, P., Lopez, D.M., Yang, X., Nguyen, L.V., Xiang, W., Johnson, T.T.: Parallelizable reachability analysis algorithms for feed-forward neural networks. In: Proceedings of the 7th International Workshop on Formal Methods in Software Engineering (FormaliSE’19), FormaliSE ’19, pp. 31–40. IEEE Press, Piscataway (2019). https://doi.org/10.1109/FormaliSE.2019.00012
Tran, H.D., Musau, P., Lopez, D.M., Yang, X., Nguyen, L.V., Xiang, W., Johnson, T.T.: Star-based reachability analysis for deep neural networks. In: 23rd International Symposium on Formal Methods (FM’19). Springer, Berlin (2019)
Tran, H.D., Yang, X., Lopez, D.M., Musau, P., Nguyen, L.V., Xiang, W., Bak, S., Johnson, T.T.: NNV: the neural network verification tool for deep neural networks and learning-enabled cyber-physical systems. In: 32nd International Conference on Computer-Aided Verification (CAV) (2020)
Vincent, J.A., Schwager, M.: Reachable polyhedral marching (RPM): a safety verification algorithm for robotic systems with deep neural network components (2021)
Wang, S., Zhang, H., Xu, K., Lin, X., Jana, S., Hsieh, C.J., Kolter, Z.: Beta-CROWN: efficient bound propagation with per-neuron split constraints for complete and incomplete neural network verification. arXiv preprint (2021). arXiv:2103.06624
Xiang, W., Tran, H., Johnson, T.T.: Output reachable set estimation and verification for multilayer neural networks. IEEE Trans. Neural Netw. Learn. Syst. 29(11), 5777–5783 (2018)
Xu, K., Shi, Z., Zhang, H., Wang, Y., Chang, K.W., Huang, M., Kailkhura, B., Lin, X., Hsieh, C.J.: Automatic perturbation analysis for scalable certified robustness and beyond. In: Advances in Neural Information Processing Systems, vol. 33 (2020)
Xu, K., Zhang, H., Wang, S., Wang, Y., Jana, S., Lin, X., Hsieh, C.J.: Fast and complete: enabling complete neural network verification with rapid and massively parallel incomplete verifiers. In: International Conference on Learning Representations (2021). https://openreview.net/forum?id=nVZtXBI6LNn
Xu, L., Hutter, F., Hoos, H.H., Leyton-Brown, K.: SATzilla: portfolio-based algorithm selection for SAT. J. Artif. Intell. Res. 32(1), 565–606 (2008)
Zhang, H., Chen, H., Xiao, C., Gowal, S., Stanforth, R., Li, B., Boning, D.S., Hsieh, C.: Towards stable and efficient training of verifiably robust neural networks. In: 8th International Conference on Learning Representations, ICLR 2020, Addis Ababa, Ethiopia, April 26–30, 2020 (2020). https://openreview.net/forum?id=Skxuk1rFwB
Zhang, H., Wang, S., Xu, K., Li, L., Li, B., Jana, S., Hsieh, C., Kolter, J.Z.: General cutting planes for bound-propagation-based neural network verification. CoRR (2022). https://doi.org/10.48550/arXiv.2208.05740
Zhang, H., Weng, T., Chen, P., Hsieh, C., Daniel, L.: Efficient neural network robustness certification with general activation functions. In: Bengio, S., Wallach, H.M., Larochelle, H., Grauman, K., Cesa-Bianchi, N., Garnett, R. (eds.) Advances in Neural Information Processing Systems 31: Annual Conference on Neural Information Processing Systems 2018, NeurIPS 2018, Montréal, Canada, December 3–8, 2018, pp. 4944–4953 (2018). https://proceedings.neurips.cc/paper/2018/hash/d04863f100d59b3eb688a11f95b0ae60-Abstract.html
Zhang, H., Weng, T.W., Chen, P.Y., Hsieh, C.J., Daniel, L.: Efficient neural network robustness certification with general activation functions. Adv. Neural Inf. Process. Syst. 31, 4939–4948 (2018). https://arxiv.org/pdf/1811.00866.pdf
Zhou, C.: Computation of optical flow using a neural network. In: IEEE 1988 International Conference on Neural Networks, vol. 2, pp. 71–78 (1988). https://doi.org/10.1109/ICNN.1988.23914
Funding
Open Access funding enabled and organized by Projekt DEAL.
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher’s Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.
About this article
Cite this article
Brix, C., Müller, M.N., Bak, S. et al. First three years of the international verification of neural networks competition (VNN-COMP). Int J Softw Tools Technol Transfer 25, 329–339 (2023). https://doi.org/10.1007/s10009-023-00703-4
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10009-023-00703-4