Abstract
Machine learning (ML) researchers and practitioners are building repositories of pre-trained models, called model zoos. These model zoos contain metadata that detail various properties of the ML models and datasets, which are useful for reporting, auditing, reproducibility, and interpretability. Unfortunately, the existing metadata representations come with limited expressivity and lack of standardization. Meanwhile, an interoperable method to store and query model zoo metadata is missing. These two gaps hinder model search, reuse, comparison, and composition. In this demo paper, we advocate for standardized ML model metadata representation, proposing Macaroni, a metadata search engine with toolkits that support practitioners to obtain and enrich that metadata.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
Notes
- 1.
- 2.
- 3.
The prototype is available at metadatazoo.io.
- 4.
References
Coleman, C.: Dawnbench: an end-to-end deep learning benchmark and competition. Training 100(101), 102 (2017)
Deshpande, A., et al.: A linearized framework and a new benchmark for model selection for fine-tuning. arXiv preprint arXiv:2102.00084 (2021)
Li, Z., Hai, R., et al.: Metadata representations for queryable ML model zoos. https://doi.org/10.48550/ARXIV.2207.09315, https://arxiv.org/abs/2207.09315
Mitchell, M., et al.: Model cards for model reporting. In: Proceedings of the Conference on Fairness, Accountability, and Transparency, pp. 220–229 (2019)
Vanschoren, J.: Meta-learning. In: Automated Machine Learning: Methods, Systems, Challenges, pp. 35–61 (2019)
Wu, Y., Lentz, M., Zhuo, D., Lu, Y.: Serving and optimizing machine learning workflows on heterogeneous infrastructures
Yang, Z., et al.: Optimizing machine learning inference queries with correlative proxy models. arXiv preprint arXiv:2201.00309 (2022)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Li, Z., Kant, H., Hai, R., Katsifodimos, A., Bozzon, A. (2023). Macaroni: Crawling and Enriching Metadata from Public Model Zoos. In: Garrigós, I., Murillo RodrÃguez, J.M., Wimmer, M. (eds) Web Engineering. ICWE 2023. Lecture Notes in Computer Science, vol 13893. Springer, Cham. https://doi.org/10.1007/978-3-031-34444-2_31
Download citation
DOI: https://doi.org/10.1007/978-3-031-34444-2_31
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-34443-5
Online ISBN: 978-3-031-34444-2
eBook Packages: Computer ScienceComputer Science (R0)