Drift Detection for Black-Box Deep Learning Models
Authors: 
Luca Piano, Fabio Garcea, Andrea Cavallone, Ignacio Aparicio Vazquez, Lia Morra, Fabrizio LambertiAuthors Info & Claims
Luca Piano, Fabio Garcea, Andrea Cavallone, Ignacio Aparicio Vazquez, Lia Morra, Fabrizio LambertiAuthors Info & ClaimsAbstract
Dataset drift is a common challenge in machine learning, especially for models trained on unstructured data, such as images. In this article, we propose a new approach for the detection of data drift in black-box models, which is based on Hellinger distance and feature extraction methods. The proposed approach is aimed at detecting data drift without knowing the architecture of the model to monitor, the dataset on which it was trained, or both. The article analyzes three different use cases to evaluate the effectiveness of the proposed approach, encompassing a variety of tasks including document segmentation, classification, and handwriting recognition. The use cases considered for the drift are adversarial assaults, domain shifts, and dataset biases. The experimental results show the efficacy of our drift detection approach in identifying changes in distribution under various training settings.
References
[1]
L. Floridi, M. Holweg, M. Taddeo, J. Amaya Silva, J. Mökander, and Y. Wen. “CapAI - A procedure for conducting conformity assessment of AI systems in line with the EU artificial intelligence act.” SSRN. [Online]. Available:.
[2]
J. Lu, A. Liu, F. Dong, F. Gu, J. Gama, and G. Zhang, “Learning under concept drift: A review,” IEEE Trans. Knowl. Data Eng., vol. 31, no. 12, pp. 2346–2363, Dec. 2019.
[3]
S. Rabanser, S. Günnemann, and Z. Lipton, “Failing loudly: An empirical study of methods for detecting dataset shift,” in Proc. Adv. Neural Inf. Process. Syst., 2019, vol. 32, pp. 1396–1408.
[4]
L. Piano, F. Garcea, V. Gatteschi, F. Lamberti, and L. Morra, “Detecting drift in deep learning: A methodology primer,” IT Prof., vol. 24, no. 5, pp. 53–60, Sep./Oct. 2022.
[5]
T. Cerquitelli et al., “Data-driven predictive maintenance: A methodology primer,” in Proc. Predictive Maintenance Smart Factories, 2021, pp. 39–73.
[6]
S. Greco and T. Cerquitelli, “Drift lens: Real-time unsupervised concept drift detection by evaluating per-label embedding distributions,” in Proc. Int. Conf. Data Mining Workshops, 2021, pp. 341–349, doi:.
[7]
J. Demšar and Z. Bosnić, “Detecting concept drift in data streams using model explanation,” Expert Syst. Appl., vol. 92, pp. 546–559, Feb. 2018.
[8]
V. Cerqueira, H. M. Gomes, A. Bifet, and L. Torgo, “STUDD: A student–teacher method for unsupervised concept drift detection,” Mach. Learn., vol. 112, no. 11, pp. 1–28, Nov. 2023.
[9]
J. Quiñonero-Candela, M. Sugiyama, N. D. Lawrence, and A. Schwaighofer, Dataset Shift in Machine Learning. Cambridge, MA, USA: MIT Press, 2009.
[10]
J. Gama, P. Medas, G. Castillo, and P. Rodrigues, “Learning with drift detection,” in Proc. Brazilian Symp. Artif. Intell., 2004, pp. 286–295.
[11]
S. Disabato and M. Roveri, “Learning convolutional neural networks in presence of concept drift,” in Proc. Int. Joint Conf. Neural Netw., 2019, pp. 1–8.
[12]
G. Ditzler and R. Polikar, “Hellinger distance based drift detection for nonstationary environments,” in Proc. IEEE Symp. Comput. Intell. Dyn. Uncertain Environ., 2011, pp. 41–48.
[13]
T. Che et al., “Deep verifier networks: Verification of deep discriminative models with deep generative models,” in Proc. AAAI Conf. Artif. Intell., 2021, vol. 35, no. 8, pp. 7002–7010.
[14]
P. Wang, N. Jin, D. Davies, and W. L. Woo, “Model-centric transfer learning framework for concept drift detection,” Knowl.-Based Syst., vol. 275, Sep. 2023, Art. no. 110705.
[15]
C. Geng, S.-J. Huang, and S. Chen, “Recent advances in open set recognition: A survey,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 43, no. 10, pp. 3614–3631, Oct. 2021.
[16]
I. J. Goodfellow, J. Shlens, and C. Szegedy, “Explaining and harnessing adversarial examples,” 2014,.
[17]
T. Viehmann, L. Antiga, D. Cortinovis, and L. Lozza, “TorchDrift: Drift detection for PyTorch,” TorchDrift, 2019. [Online]. Available: https://torchdrift.org/
[18]
D. E. Rumelhart, G. E. Hinton, and R. J. Williams, “Learning representations by back-propagating errors,” Nature, vol. 323, no. 6088, pp. 533–536, Oct. 1986.
[19]
A. Ilyas, S. Santurkar, D. Tsipras, L. Engstrom, B. Tran, and A. Madry, “Adversarial examples are not bugs, they are features,” in Proc. Adv. Neural Inf. Process. Syst., 2019, vol. 32, pp. 125–136.
[20]
C. Schuhmann et al., “LAION-5B: An open large-scale dataset for training next generation image-text models,” 2022,.
[21]
C. Chen, G. Yao, C. Wang, S. Goudos, and S. Wan, “Enhancing the robustness of object detection via 6G vehicular edge computing,” Digit. Commun. Netw., vol. 8, no. 6, pp. 923–931, Dec. 2022.
Recommendations
Concept drift detection via competence models
Detecting changes of concepts, such as a change of customer preference for telecom services, is very important in terms of prediction and decision applications in dynamic environments. In particular, for case-based reasoning systems, it is important to ...
Brute force concept drift detection
AbstractWe present a brute-force approach to detect concept drift behind time sequence data. This approach, named Select-Starţ searches for start points of concept drift to minimize error. In other words, Select-Start searches for the start points of new ...
Model-centric transfer learning framework for concept drift detection
AbstractConcept drift refers to the inevitable phenomenon that influences the statistical features of the data stream. Detecting concept drift in data streams quickly and precisely remains challenging, and failure to detect it will render model trained ...
Comments
Information & Contributors
Information
Published In
1520-9202 © 2024 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See https://www.ieee.org/publications/rights/index.html for more information.
Publisher
IEEE Educational Activities Department
United States
Publication History
Published: 01 May 2024
Qualifiers
- Research-article
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 0Total Downloads
- Downloads (Last 12 months)0
- Downloads (Last 6 weeks)0
Reflects downloads up to 14 Oct 2024
Other Metrics
Citations
View Options
View options
Get Access
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in