On the Way to SBOMs: Investigating Design Issues and Solutions in Practice
Article No.: 149, Pages 1 - 25
Abstract
The increase of software supply chain threats has underscored the necessity for robust security mechanisms, among which the Software Bill of Materials (SBOM) stands out as a promising solution. SBOMs, by providing a machine-readable inventory of software composition details, play a crucial role in enhancing transparency and traceability within software supply chains. This empirical study delves into the practical challenges and solutions associated with the adoption of SBOMs through an analysis of 4,786 GitHub discussions across 510 SBOM-related projects. Through repository mining and analysis, this research delineates key topics, challenges, and solutions intrinsic to the effective utilization of SBOMs. Furthermore, we shed light on commonly used tools and frameworks for SBOM generation, exploring their respective strengths and limitations. This study underscores a set of findings, for example, there are four phases of the SBOM life cycle, and each phase has a set of SBOM development activities and issues; in addition, this study emphasizes the role SBOM play in ensuring resilient software development practices and the imperative of their widespread adoption and integration to bolster supply chain security. The insights of our study provide vital input for future work and practical advancements in this topic.
References
[1]
The State of Software Bill of Materials (SBOM) and Cybersecurity Readiness. (n.d.). Retrieved from https://www.linuxfoundation.org/tools/the-state-of-software-bill-of-materials-sbom-and-cybersecurity-readiness/
[2]
2021. Executive Order on Improving the Nation’s Cybersecurity. Retrieved from https://www.whitehouse.gov/briefing-room/presidential-actions/2021/05/12/executive-order-on-improving-the-nations-cybersecurity/
[3]
2023. The Minimum Elements for a Software Bill of Materials (SBOM). Retrieved from https://www.ntia.doc.gov/files/ntia/publications/sbom_minimum_elements_report.pdf
[4]
2023. Types of Software Bill of Materials (SBOM). Retrieved from https://www.cisa.gov/resources-tools/resources/types-software-bill-materials-sbom
[5]
Musard Balliu, Benoit Baudry, Sofia Bobadilla, Mathias Ekstedt, Martin Monperrus, Javier Ron, Aman Sharma, Gabriel Skoglund, César Soto-Valero, and Martin Wittlinger. 2023. Challenges of producing software bill of materials for Java. arXiv preprint arXiv:2303.11102 (2023).
[6]
Iain Barclay, Alun D. Preece, Ian J. Taylor, Swapna Krishnakumar Radha, and Jarek Nabrzyski. 2023. Providing assurance and scrutability on shared data and machine learning models with verifiable credentials. Concurr. Comput. Pract. Exp. 35, 18 (2023).
[7]
Iain Barclay, Alun Preece, Ian Taylor, and Dinesh Verma. 2019. Towards traceability in data ecosystems using a bill of materials model. arXiv preprint arXiv:1904.04253 (2019).
[8]
Tingting Bi, Xin Xia, David Lo, John Grundy, Thomas Zimmermann, and Denae Ford. 2022. Accessibility in software practice: A practitioner’s perspective. ACM Trans. Softw. Eng. Methodol. 31, 4 (2022), 1–26.
[9]
Miles Brundage, Shahar Avin, Jasmine Wang, Haydn Belfield, Gretchen Krueger, Gillian Hadfield, Heidy Khlaaf, Jingying Yang, Helen Toner, Ruth Fong, et al. 2020. Toward trustworthy AI development: Mechanisms for supporting verifiable claims. arXiv preprint arXiv:2004.07213 (2020).
[10]
Seth Carmody, Andrea Coravos, Ginny Fahs, Audra Hatch, Janine Medina, Beau Woods, and Joshua Corman. 2021. Building resilient medical technology supply chains with a software bill of materials. NPJ Digit. Med. 4, 1 (2021), 1–6.
[11]
Casey Casalnuovo, Bogdan Vasilescu, Premkumar Devanbu, and Vladimir Filkov. 2015. Developer onboarding in GitHub: The role of prior social links and language experience. In Proceedings of the 10th Joint Meeting on Foundations of Software Engineering. 817–828.
[12]
Sunita Chulani, Clay Williams, and Avi Yaeli. 2008. Software development governance and its concerns. In Proceedings of the 1st International Workshop on Software Development Governance. 3–6.
[13]
Klaas Andries de Graaf, Peng Liang, Antony Tang, Willem Robert van Hage, and Hans van Vliet. 2014. An exploratory study on ontology engineering for software architecture documentation. Comput. Industr. 65, 7 (2014), 1053–1064.
[14]
Xinxing Ding, Feng Zhao, Lijuan Yan, and Xiaodong Shao. 2019. The method of building SBOM based on enterprise big data. In Proceedings of the 3rd International Conference on Electronic Information Technology and Computer Engineering (EITCE’19). IEEE, 1224–1228.
[15]
Steve Easterbrook, Janice Singer, Margaret-Anne D. Storey, and Daniela E. Damian. 2008. Selecting empirical methods for software engineering research. Guide to Advanced Empirical Software Engineering (2008), 285–311.
[16]
Robert J. Ellison, John B. Goodenough, Charles B. Weinstock, and Carol Woody. 2010. Evaluating and Mitigating Software Supply Chain Security Risks. Technical Report. Carnegie-Mellon University, Software Engineering Institute, Pittsburgh, PA.
[17]
William Enck and Laurie Williams. 2022. Top five challenges in software supply chain security: Observations from 30 industry and government organizations. IEEE Secur. Privac. 20, 2 (2022), 96–100.
[18]
Gang Fan, Chengpeng Wang, Rongxin Wu, Xiao Xiao, Qingkai Shi, and Charles Zhang. 2020. Escaping dependency hell: Finding build dependency errors with the unified dependency graph. In Proceedings of the 29th ACM SIGSOFT International Symposium on Software Testing and Analysis. 463–474.
[19]
A. J. Grotto and James Dempsey. 2021. Vulnerability disclosure and management for AI/ML systems: A working paper with policy recommendations. ML Systems: A Working Paper with Policy Recommendations (November 15, 2021) (2021).
[20]
Martin Host, Austen Rainer, Per Runeson, and Bjorn Regnell. 2012. Case Study Research in Software Engineering: Guidelines and Examples. John Wiley & Sons.
[21]
Nasif Imtiaz, Seaver Thorn, and Laurie Williams. 2021. A comparative study of vulnerability reporting by software composition analysis tools. In Proceedings of the 15th ACM/IEEE International Symposium on Empirical Software Engineering and Measurement (ESEM’21). Association for Computing Machinery, New York, NY, Article 5, 11 pages. DOI:DOI:
[22]
Ritu Jain and Ugrasen Suman. 2015. A systematic literature review on global software development life cycle. ACM SIGSOFT Softw. Eng. Notes 40, 2 (2015), 1–14.
[23]
Dennis Kengo Oka. 2021. Software Composition Analysis in the Automotive Industry. Wiley, 91–110. DOI:DOI:
[24]
Herb Krasner. 2021. The Cost of Poor Software Quality in the US: A 2020 Report. Consortium for Information and Software Quality.1–46.
[25]
Piergiorgio Ladisa, Henrik Plate, Matias Martinez, and Olivier Barais. 2023. SoK: Taxonomy of attacks on open-source software supply chains. In Proceedings of the IEEE Symposium on Security and Privacy (SP’23). IEEE, 1509–1526.
[26]
John M. Longshore and Angela L. Cheatham. 2022. Managing Logistics Systems: Planning and Analysis for a Successful Supply Chain. Routledge.
[27]
Tim Mackey. 2018. Building open source security into agile application builds. Netw. Secur. 2018, 4 (2018), 5–8.
[28]
Nabil M. Mohammed, Mahmood Niazi, Mohammad Alshayeb, and Sajjad Mahmood. 2017. Exploring software security approaches in software development lifecycle: A systematic mapping study. Comput. Stand. Interf. 50 (2017), 107–115.
[29]
Shripad Nadgowda. 2022. Engram: The one security platform for modern software supply chain risks. In Proceedings of the 8th International Workshop on Container Technologies and Container Clouds. 7–12.
[30]
Sabato Nocera, Simone Romano, Massimiliano Di Penta, Rita Francese, and Giuseppe Scanniello. 2023. Software bill of materials adoption: A mining study from GitHub. In Proceedings of the IEEE International Conference on Software Maintenance and Evolution (ICSME’23). IEEE, 39–49.
[31]
Marc Ohm, Henrik Plate, Arnold Sykosch, and Michael Meier. 2020. Backstabber’s knife collection: A review of open source software supply chain attacks. In Proceedings of the 17th International Conference on Detection of Intrusions and Malware, and Vulnerability Assessment (DIMVA’20). Springer, 23–43.
[32]
Philippe Ombredanne. 2020. Free and open source software license compliance: Tools for software composition analysis. Computer 53, 10 (2020), 105–109.
[33]
OpenAI. 2023. March 20 ChatGPT outage: Here’s what happened. (2023). Retrieved from https://openai.com/blog/march-20-chatgpt-outage
[34]
P. K. Ragunath, S. Velmourougan, P. Davachelvan, S. Kayalvizhi, and R. Ravimohan. 2010. Evolving a new model (SDLC Model-2010) for software development life cycle (SDLC). Int. J. Comput. Sci. Netw. Secur. 10, 1 (2010), 112–119.
[35]
Mohammad Masudur Rahman and Chanchal K. Roy. 2014. An insight into the pull requests of GitHub. In Proceedings of the 11th Working Conference on Mining Software Repositories. 364–367.
[36]
Per Runeson and Martin Höst. 2009. Guidelines for conducting and reporting case study research in software engineering. Empir. Softw. Eng. 14, 2 (2009), 131–164.
[37]
Bruce Schneier. 2019. Every part of the supply chain can be attacked. New York Times (2019). Retrieval from https://www.nytimes.com/2019/09/25/opinion/huaweiinternet-security.html
[38]
Ravi Sen, Siddhartha S. Singh, and Sharad Borle. 2012. Open source software success: Measures and analysis. Decis. Supp. Syst. 52, 2 (2012), 364–372.
[39]
Yong Shi, Mingzhi Wen, Filipe R. Cogo, Boyuan Chen, and Zhen Ming Jiang. 2021. An experience report on producing verifiable builds for large-scale commercial systems. IEEE Trans. Softw. Eng. 48, 9 (2021), 3361–3377.
[40]
Hartmut Stadtler, Christoph Kilger, and Herbert Meyr. 2015. Supply Chain Management and Advanced Planning: Concepts, Models, Software, and Case Studies. Springer.
[41]
Trevor Stalnaker, Nathan Wintersgill, Oscar Chaparro, Massimiliano Di Penta, Daniel M. German, and Denys Poshyvanyk. 2023. BOMs away! Inside the minds of stakeholders: A comprehensive study of bills of materials for software systems. arXiv preprint arXiv:2309.12206 (2023).
[42]
Anselm Strauss and Juliet M. Corbin. 1997. Grounded Theory in Practice. Sage.
[43]
Jason Tsay, Laura Dabbish, and James Herbsleb. 2014. Let’s talk about it: Evaluating contributions through discussion in GitHub. In Proceedings of the 22nd ACM SIGSOFT International Symposium on Foundations of Software Engineering. 144–154.
[44]
Boming Xia, Tingting Bi, Zhenchang Xing, Qinghua Lu, and Liming Zhu. 2023. An empirical study on software bill of materials: Where we stand and the road ahead. In Proceedings of the IEEE/ACM 45th International Conference on Software Engineering (ICSE’23). 2630–2642. DOI:DOI:
[45]
Boming Xia, Dawen Zhang, Yue Liu, Qinghua Lu, Zhenchang Xing, and Liming Zhu. 2023. Trust in software supply chains: Blockchain-enabled SBOM and the AIBOM future. arXiv preprint arXiv:2307.02088 (2023).
[46]
Yanming Yang, Xin Xia, David Lo, Tingting Bi, John Grundy, and Xiaohu Yang. 2022. Predictive models in software engineering: Challenges and opportunities. ACM Trans. Softw. Eng. Methodol. 31, 3 (2022), 1–72.
Index Terms
- On the Way to SBOMs: Investigating Design Issues and Solutions in Practice
Recommendations
SBOM Ouverture: What We Need and What We Have
ARES '24: Proceedings of the 19th International Conference on Availability, Reliability and SecurityA Software Bill of Materials (SBOM) is an inventory of the software components used to build a product, which can help customers track security risks throughout the development lifecycle. The popularity of SBOMs grew in May 2021 when the White House ...
BOMs Away! Inside the Minds of Stakeholders: A Comprehensive Study of Bills of Materials for Software Systems
ICSE '24: Proceedings of the IEEE/ACM 46th International Conference on Software EngineeringSoftware Bills of Materials (SBOMs) have emerged as tools to facilitate the management of software dependencies, vulnerabilities, licenses, and the supply chain. While significant effort has been devoted to increasing SBOM awareness and developing SBOM ...
SBOM Generation Tools Under Microscope: A Focus on The npm Ecosystem
SAC '24: Proceedings of the 39th ACM/SIGAPP Symposium on Applied ComputingGenerating accurate Software Bill of Materials (SBOM) is challenging due to the complex dependencies in the diverse components used in software and also the way software is built into executables. A handful of tools claim the capability of automatic SBOM ...
Comments
Information & Contributors
Information
Published In
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 27 June 2024
Online AM: 26 March 2024
Accepted: 13 March 2024
Revised: 09 February 2024
Received: 22 August 2023
Published in TOSEM Volume 33, Issue 6
Check for updates
Author Tags
Qualifiers
- Research-article
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 697Total Downloads
- Downloads (Last 12 months)697
- Downloads (Last 6 weeks)143
Reflects downloads up to 04 Oct 2024
Other Metrics
Citations
Cited By
View allView Options
Get Access
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in