Verifying Neural Networks by Approximating Convex Hulls
Author: 
Zhongkui MaAuthors Info & Claims
Zhongkui MaAuthors Info & ClaimsPages 261 - 266
Abstract
The increasing prevalence of neural networks necessitates their verification in order to ensure security. Verifying neural networks is a challenge due to the use of non-linear activation functions. This work concentrates on approximating the convex hull of activation functions. An approach is proposed to construct a convex polytope to over-approximate the ReLU hull (the convex hull of the ReLU function) when considering multi-variables. The key idea is to construct new faces based on the known faces and vertices by uniqueness of the ReLU hull. Our approach has been incorporated into the state-of-the-art PRIMA framework, which takes into account multi-neuron constraints. The experimental evaluation demonstrates that our method is more efficient and precise than existing ReLU hull exact/approximate approaches, and it makes a significant contribution to the verification of neural networks. Our concept can be applied to other non-linear functions in neural networks, and this could be explored further in future research.
References
[1]
Bunel, R., Turkaslan, I., Torr, P.H., Kohli, P., Kumar, M.P.: A unified view of piecewise linear neural network verification. In: Proceedings of the 32nd International Conference on Neural Information Processing Systems, pp. 4795–4804 (2018)
[2]
Cao, Y., et al.: Invisible for both camera and lidar: security of multi-sensor fusion based perception in autonomous driving under physical-world attacks. In: 2021 IEEE Symposium on Security and Privacy (SP), pp. 176–194. IEEE (2021)
[3]
He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 770–778 (2016)
[4]
Katz G, Barrett C, Dill DL, Julian K, and Kochenderfer MJ Majumdar R and Kunčak V Reluplex: an efficient SMT solver for verifying deep neural networks Computer Aided Verification 2017 Cham Springer 97-117
[5]
Li, L., Xie, T., Li, B.: SoK: certified robustness for deep neural networks. In: 2023 IEEE Symposium on Security and Privacy (SP), pp. 94–115. IEEE Computer Society (2022)
[6]
Lomuscio, A., Maganti, L.: An approach to reachability analysis for feed-forward ReLU neural networks. arXiv preprint: arXiv:1706.07351 (2017)
[7]
Müller, M.N., Makarchuk, G., Singh, G., Püschel, M., Vechev, M.: PRIMA: general and precise neural network certification via scalable convex hull approximations. Proc. ACM Programm. Lang. 6(POPL), 1–33 (2022)
[8]
Pulina L and Tacchella A Touili T, Cook B, and Jackson P An abstraction-refinement approach to verification of artificial neural networks Computer Aided Verification 2010 Heidelberg Springer 243-257
[9]
Singh, G., Ganvir, R., Püschel, M., Vechev, M.: Beyond the single neuron convex barrier for neural network certification. In: Advances in Neural Information Processing Systems, vol. 32 (2019)
[10]
Singh, G., Gehr, T., Mirman, M., Püschel, M., Vechev, M.: Fast and effective robustness certification. In: Advances in Neural Information Processing Systems, vol. 31 (2018)
[11]
Singh, G., Gehr, T., Püschel, M., Vechev, M.: An abstract domain for certifying neural networks. Proce. ACM Programm. Lang. 3(POPL), 1–30 (2019)
[12]
Szegedy, C., et al.: Intriguing properties of neural networks. In: 2nd International Conference on Learning Representations, ICLR 2014 (2014)
[13]
Vaswani, A., et al.: Attention is all you need. In: Advances in Neural Information Processing Systems, vol. 30 (2017)
[14]
Weng, L., et al.: Towards fast computation of certified robustness for ReLU networks. In: International Conference on Machine Learning, pp. 5276–5285. PMLR (2018)
[15]
Zhang, H., et al.: General cutting planes for bound-propagation-based neural network verification. In: Advances in Neural Information Processing Systems, vol. 35, pp. 1656–1670 (2022)
Index Terms
- Verifying Neural Networks by Approximating Convex Hulls
Index terms have been assigned to the content through auto-classification.
Recommendations
Convex hulls of spheres and convex hulls of disjoint convex polytopes
Given a set @S of spheres in E^d, with d>=3 and d odd, having a constant number of m distinct radii @r"1,@r"2,...,@r"m, we show that the worst-case combinatorial complexity of the convex hull of @S is @Q(@__ __"1"=<"i"<>"j"=<"mn"in"j^@__ __^d^2^@__ __), ...
Convex hulls of spheres and convex hulls of convex polytopes lying on parallel hyperplanes
SoCG '11: Proceedings of the twenty-seventh annual symposium on Computational geometryGiven a set Σ of spheres in Ed, with d≥3 and d odd, having a fixed number of m distinct radii ρ1,ρ2,...,ρm, we show that the worst-case combinatorial complexity of the convex hull CHd(Σ) of Σ is Θ(Σ{1≤i≠j≤m}ninj⌊ d/2 ⌋), where ni is the number of ...
The quickhull algorithm for convex hulls
The convex hull of a set of points is the smallest convex set that contains the points. This article presents a practical convex hull algorithm that combines the two-dimensional Quickhull algorithm with the general-dimension Beneath-Beyond Algorithm. It ...
Comments
Information & Contributors
Information
Published In
© The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd. 2023.
Publisher
Springer-Verlag
Berlin, Heidelberg
Publication History
Published: 21 November 2023
Author Tags
Qualifiers
- Article
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 0Total Downloads
- Downloads (Last 12 months)0
- Downloads (Last 6 weeks)0
Reflects downloads up to 16 Oct 2024
Other Metrics
Citations
View Options
View options
Get Access
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in