Machine Learning Lineage for Trustworthy Machine Learning Systems: Information Framework for MLOps Pipelines
Abstract
We describe machine learning (ML) lineage as a framework to holistically capture and connect the required information about ML model development and operations. ML lineage distinguishes between model and prediction levels, conceptually encompassing separate yet interconnected core domains for the project, experiment, model, and prediction.
References
[1]
T. Mikkonen, J. Nurminen, M. Raatikainen, I. Fronza, N. Mäkitalo, and T. Männistö, “Is machine learning software just software: A maintainability view,” in Proc. Int. Conf. Softw. Qual., New York, NY, USA: Springer-Verlag, 2021, pp. 94–105.
[2]
D. Kreuzberger, N. Kühl, and S. Hirschl, “Machine learning operations (MLOps): Overview, definition, and architecture,” IEEE Access, vol. 11, pp. 31,866–31,879, 2023.
[3]
D. Sculley et al., “Hidden technical debt in machine learning systems,” in Proc. Int. Conf. Neural Inf. Process. Syst., vol. 2, Cambridge, MA, USA: MIT Press, 2015, pp. 2503–2511.
[4]
M. Mäntymäki, M. Minkkinen, T. Birkstedt, and M. Viljanen, “Defining organizational AI governance,” AI Ethics, vol. 2, no. 4, pp. 603–609, 2022.
[5]
N. Mehrabi, F. Morstatter, N. Saxena, K. Lerman, and A. Galstyan, “A survey on bias and fairness in machine learning,” ACM Comput. Surv. (CSUR), vol. 54, no. 6, pp. 1–35, 2021.
[6]
L. Baier, F. Jöhren, and S. Seebacher, “Challenges in the deployment and operation of machine learning in practice,” in Proc. Eur. Conf. Inform. Syst. (ECIS), vol. 1, 2019, pp. 1–19.
[7]
F. Königstorfer and S. Thalmann, “AI documentation: A path to accountability,” J. Responsible Technol., vol. 11, 2022, Art. no. 100043.
[8]
M. Steidl, V. Golendukhina, M. Felderer, and R. Ramler, “Automation and development effort in continuous AI development: A practitioners’ survey,” in Proc. Euromicro Conf. Softw. Eng. Adv. Appl. (SEAA), 2023, pp. 120–127.
[9]
M. Mitchell et al., “Model cards for model reporting,” in Proc. Conf. Fairness, Accountability, Transparency, 2019, pp. 220–229.
[10]
M. Arnold et al., “FactSheets: Increasing trust in AI services through supplier’s declarations of conformity,” IBM J. Res. Develop., vol. 63, no. 4/5, pp. 6:1–6:13, Jul./Sep. 2019.
[11]
T. K. Gilbert, N. Lambert, S. Dean, T. Zick, A. Snoswell, and S. Mehta, “Reward reports for reinforcement learning,” in Proc. AAAI/ACM Conf. AI, Ethics, Soc., 2023, pp. 84–130.
[12]
R. Ikeda and J. Widom, Data Lineage: A Survey. Redwood City, CA, USA: Stanford Univ. Publications, 2009, vol. 8090, no. 918, p. 1. [Online]. Available: http://ilpubs.stanford.edu
[13]
A. I. Canhoto and F. Clear, “Artificial intelligence and machine learning as business tools: A framework for diagnosing value destruction potential,” Bus. Horiz., vol. 63, no. 2, pp. 183–193, 2020.
[14]
R. Tatman, J. VanderPlas, and S. Dane, “A practical taxonomy of reproducibility for machine learning research,” presented at the 2nd Reproducibility Mach. Learn. Workshop Int. Conf. Mach. Learn., Stockholm, Sweden, 2018.
[15]
Y. Luo, M. Raatikainen, and J. Nurminen, “Autonomously adaptive machine learning systems: Experimentation-driven open-source pipeline,” in Proc. 49th Euromicro Conf. Softw. Eng. Adv. Appl. (SEAA), Piscataway, NJ, USA: IEEE Press, 2023, pp. 44–52.
Index Terms
- Machine Learning Lineage for Trustworthy Machine Learning Systems: Information Framework for MLOps Pipelines
Index terms have been assigned to the content through auto-classification.
Recommendations
Machine Learning: The State of the Art
The two fundamental problems in machine learning (ML) are statistical analysis and algorithm design. The former tells us the principles of the mathematical models that we establish from the observation data. The latter defines the conditions on which ...
Comments
Information & Contributors
Information
Published In
© 2024 The Authors. This work is licensed under a Creative Commons Attribution 4.0 License. For more information, see https://creativecommons.org/licenses/by/4.0/.
Publisher
IEEE Computer Society Press
Washington, DC, United States
Publication History
Published: 01 January 2025
Qualifiers
- Research-article
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 0Total Downloads
- Downloads (Last 12 months)0
- Downloads (Last 6 weeks)0
Reflects downloads up to 01 Feb 2025