Software Strategy for Reuse Study, Final Report

National Aeronautics and Space Administration REPORT NO: AC1L1-52 DATE: October 1, 2005 Constellation Program Systems Engineering Exploration Systems Mission Directorate National Aeronautics and Space Administration, Headquarters Washington DC 20546-0001 Software Strategy for Reuse Study Final Report CONSTELLATION PROGRAM SYSTEMS ENGINEERING CONSTELLA NEERING Title: Software Strategy for Reuse Final Report Document No.: AC1L1-52 Effective Date: October 1, 2005 Revision: 0 Page i Document History Log Status (Baseline/ Revision/ Canceled) Document Revision Effective Date Description Baseline per Directive # CONSTELLATION PROGRAM SYSTEMS ENGINEERING CONSTELLA NEERING Title: Software Strategy for Reuse Final Report Document No.: AC1L1-52 Effective Date: October 1, 2005 Revision: 0 Page ii Software Strategy for Reuse Study—Final Report Constellation Program Systems Engineering Prepared by: ______________________________________ Dr. David J. Atkinson Study Lead Jet Propulsion Laboratory _____________________ Date Approved by: ______________________________________ Dr. David Atkinson Chair Computers, Software and Automation Integrated Discipline Team Constellation Program Systems Engineering _____________________ Date Accepted by: ______________________________________ Geoffrey Yoder Chief Engineer (Acting) Constellation Program Systems Engineering _____________________ Date CONSTELLATION PROGRAM SYSTEMS ENGINEERING CONSTELLA NEERING Title: Strategy for Software Reuse Study Final Report Document No.: AC1L1-52 Effective Date: October 1, 2005 Revision: 0 Page iii Table of Contents EXECUTIVE SUMMARY............................................................................................................8 INTRODUCTION .......................................................................................................................10 Purpose...............................................................................................................................10 Study Objectives ................................................................................................................10 Relationship to Exploration Objectives .................................................................10 Relationship to Constellation Modeling and Simulation.......................................11 Relationship of This Study to Other IDT Studies ..................................................11 CCC IDT AC1L1-85 Command and Control Interoperability Study ..................12 AFM IDT Study AC1L1-18 Automated Rendezvous and Docking .....................12 AFM IDT Study AC1L1-19 Mated Flight Control Operations ............................13 CSA IDT Study AC1L1-56 Integrated Systems Health Management Study .......14 CSA IDT Study AC1L1-55 Autonomy and Automation Roadmap for SRR........14 Ground Rules and Assumptions ........................................................................................15 Study Task Description ......................................................................................................15 Key Issues for Constellation Reuse Strategy ....................................................................16 SOFTWARE REUSE...................................................................................................................18 Section Purpose and Scope ................................................................................................18 Background........................................................................................................................18 Various Definitions of Software Reuse..............................................................................19 Constellation and Reuse.....................................................................................................21 Reuse Strategy ...................................................................................................................22 Architecture .......................................................................................................................23 Product Lines .....................................................................................................................24 Software Reuse Success ....................................................................................................26 CASE STUDIES ..........................................................................................................................28 Introduction........................................................................................................................28 NASA Case Studies ...........................................................................................................28 AMMOS Multi-Mission System Reuse Case Study .............................................28 Mission Operations Services Provided.................................................................28 AMMOS as a Software Product Line ..................................................................30 Return on Investment ...........................................................................................30 Lifecycle Considerations ......................................................................................32 Other Challenges of a Product Line Approach .....................................................35 Future Vision ........................................................................................................35 AMMOS Conclusions ..........................................................................................36 ROSE—Flight Analysis and Design Reengineering Case Study ..........................36 Project Description ...............................................................................................36 Project Timeframe ................................................................................................37 CONSTELLATION PROGRAM SYSTEMS ENGINEERING CONSTELLA NEERING Title: Strategy for Software Reuse Study Final Report Document No.: AC1L1-52 Effective Date: October 1, 2005 Revision: 0 Page iv Process 37 Organization .........................................................................................................38 Training 38 Technical ..............................................................................................................38 Lessons Learned ...................................................................................................38 Checkout and Launch Control System (CLCS) Case Study .................................39 Impact of Cultural Differences on Software Product Specification .....................39 Subsystem Status via Service Provider Model .....................................................40 Framework Encourages Reuse .............................................................................41 Configuration Complexities of Software Reuse ...................................................41 Summary of SEEDS Formulation Team Final Recommendation Report .............42 Introduction ..........................................................................................................42 Definitions ............................................................................................................42 Summary of Findings and Recommendation of the Study ..................................43 GSFC Ground System Reuse Case Study .............................................................45 Non-NASA Case Studies ...................................................................................................45 Introduction 45 Bold Stroke Avionics Software Family, Boeing ....................................................46 A-7E Operational Flight Program, U.S. Naval Research Laboratory ...................47 ShipSystem 2000, CelsiusTech Systems AB .........................................................47 United States National Reconnaissance Office (NRO) Control Channel Toolkit (CCT) 47 NRO CCT and Similarity to Constellation ..........................................................48 Managing Several Teams with SPL .....................................................................48 Need for an Early Architecture .............................................................................49 Technical and Cultural Challenges Using SPL for Constellation.........................49 Constellation Scope and Complexity ...................................................................49 Leadership in Management within SPL Environment .........................................50 Lessons Learned on NRO CCT Project ...............................................................50 Discussion and Analysis ....................................................................................................50 Overview of NASA Case Lessons .........................................................................50 Overview of Non-NASA Case Lessons.................................................................51 Summary of Case Study Challenges and Solutions...............................................52 POTENTIAL BENEFITS OF REUSE.......................................................................................55 Discussion..........................................................................................................................55 Increased Reliability and Overall Quality .............................................................55 Reduced Development Time..................................................................................56 Increased Productivity ...........................................................................................56 Reduced Cost 56 Reduced Development Process Risk .....................................................................59 CONSTELLATION PROGRAM SYSTEMS ENGINEERING CONSTELLA NEERING Title: Strategy for Software Reuse Study Final Report Document No.: AC1L1-52 Effective Date: October 1, 2005 Revision: 0 Page v Increased Maintainability ......................................................................................59 Continuous Improvement and Increased Evolvability ..........................................59 More Effective and Efficient Use of Specialists....................................................60 Easier Compliance with Standards ........................................................................60 Increased Flexibility ..............................................................................................60 Findings .............................................................................................................................60 RISKS, OBSTACLES, AND MITIGATIONS ..........................................................................62 Introduction........................................................................................................................62 Technical Risks ..................................................................................................................62 Software Inheritance ..............................................................................................62 System Failure ......................................................................................................62 Cost Growth .........................................................................................................63 Legacy Software ....................................................................................................63 Reuse Process Risks ..............................................................................................64 Non-Technical Risks..........................................................................................................64 Organization-Level Reuse .....................................................................................64 Business and Economic Factors ............................................................................64 Legal Factors 65 Psychological Factors and Incentives ....................................................................66 Inadequate Training................................................................................................67 Additional Risk Findings .......................................................................................68 Mitigation Strategies..........................................................................................................68 Mitigation for Inherited Software Risks ................................................................69 Mitigation for Legacy Software Risks...................................................................69 Mitigation for Business and Economic Factors.....................................................70 Mitigation for Psychological Factors: Incentives .................................................71 Additional Mitigation Findings .............................................................................71 SUMMARY OF FINDINGS .......................................................................................................73 Opportunity to Realize Major Benefits from Reuse ..........................................................73 Confidence from a Rich History and Broad Current Work on Software Reuse ................73 Requirement for Success: a Systemic Engineering Technical Approach ..........................73 Importance of and Necessity of Both Technical and Non-Technical Factors....................74 RECOMMENDATIONS ............................................................................................................76 Non-Technical Recommendations .....................................................................................76 Technical Recommendations .............................................................................................79 NEXT STEPS—IMPACT ON REQUIREMENTS ..................................................................85 Traceability to Requirements .............................................................................................85 CONSTELLATION PROGRAM SYSTEMS ENGINEERING CONSTELLA NEERING Title: Strategy for Software Reuse Study Final Report Document No.: AC1L1-52 Effective Date: October 1, 2005 Revision: 0 Page vi Impact on SEI Constellation Responsible Assignment Matrix (CRAM) products ...........88 Software Management Plan (SMP)—DSM&A ....................................................88 Other Constellation Program Systems Engineering Products ...............................88 Follow-On Studies .............................................................................................................90 ACKNOWLEDGEMENTS ........................................................................................................93 Appendix A: Glossary, Acronyms, and Abbreviations ............................................................94 A.1 Glossary .................................................................................................................94 A.2 Acronyms and Abbreviations.................................................................................96 Appendix B: Bibliography .......................................................................................................100 Appendix C: Study Team Participants ...................................................................................107 C.1 Participants ..........................................................................................................107 C.2 Study Team Bios ..................................................................................................108 Appendix D: D.1 D.2 D.3 D.4 D.5 D.6 Study Team Approach .......................................................................................125 Study Team and Logistics ....................................................................................125 Initial Team Meeting............................................................................................125 Regular Meetings.................................................................................................125 Data Development ...............................................................................................125 Analysis ...............................................................................................................125 Synthesis ..............................................................................................................125 Appendix E: Reuse Metrics Deserve a Warning Label: The Pitfalls of Measuring Software Reuse ...................................................................................................................127 Appendix F: Software Reuse Study Report—A Report for the NASA IV&V Facility ......132 Appendix G: Quantitative Benefits Achieved through Software Reuse ..............................135 Appendix H: Recommended Online Reading ........................................................................137 CONSTELLATION PROGRAM SYSTEMS ENGINEERING CONSTELLA NEERING Title: Strategy for Software Reuse Study Final Report Document No.: AC1L1-52 Effective Date: October 1, 2005 Revision: 0 Page vii List of Figures Figure 3-1. Market for Reusable Software Artifacts ...............................................................18 Figure 4-1. Mission System Elements .......................................................................................27 Figure 4-2. Dual Lifecycle Approach ........................................................................................35 Figure 5-1. Relative Development Cost vs. Reuse Fraction by Reuse Type (from Jensen, 2004)..............................................................................................54 List of Tables Table 4-1. Impact of Software Reuse on Deep Space Mission Ground Systems ...................29 Table 4-2. Mature System Reuse Advantages ..........................................................................31 Table 4-3. Challenges in System Level Reuse ...........................................................................32 Table 4-4. Product Line Successes .............................................................................................36 Table 4-5. Product Line Challenges ..........................................................................................36 Table 4-6. Case Studies—Lessons Learned* ............................................................................50 Table 5-1. Reported Benefits From Reuse ................................................................................53 Table 5-2. Return on Investment by Reusable Artifact ...........................................................54 Table 6-1. Local Reuse Costs vs. Benefits (Fichman and Kemerer, 2001) .............................63 Table 6-2. Targets for Early Investment to Enable Reuse ......................................................66 Table 9-1. Reuse Impact on Program Systems Engineering Products...................................82 CONSTELLATION PROGRAM SYSTEMS ENGINEERING CONSTELLA NEERING Title: Software Strategy for Reuse Final Report Document No.: AC1L1-52 Effective Date: October 1, 2005 Revision: 0 Page 8 1.0EXECUTIVE SUMMARY This study investigated and develops a strategy for software reuse for the Constellation Program. The study team consisted of a group of multidisciplinary experts from across the NASA Centers, with collective expertise spanning the lifecycle of software engineering from research through operations and software obsolescence. The team investigated what software reuse would mean in the context of the Constellation Program, the benefits that might be earned, and the risks to be mitigated. The team found that “software reuse” is usually defined as either the application of software to a problem other than the original problem the software was implemented to solve; or the designing of the software (and other artifacts) from the beginning for reuse within multiple products, projects, or domains with similar requirements. The first definition of reuse was found to be very limited in potential benefit and otherwise not suitable for a project with the large scale of Constellation, and was not further addressed with the exception of reuse of “legacy” software. The study team focused on the second type of reuse since it rapidly became obvious that these concepts have been applied beneficially to a large number of large-scale commercial, defense, and aerospace projects over the last decade or so. Many case studies have shown that the systematic development and reuse of core assets substantially reduces implementation time and increases product quality. Core assets include requirements, architecture, documentation, code, test plans, experience, management processes, and other products of the software engineering product lifecycle. The Software Engineering Institute has captured and refined the best practices of large-scale reuse for the Department of Defense. The practice is called Software Product Lines. The case studies show that it is critical that the foundations for software product lines are laid early in the project lifecycle, and that the practice receives support from senior project management. The foundations must be laid early, as otherwise, decisions will be made and actions taken that will make it difficult or practically impossible to implement reuse later in the lifecycle. For example, the system architecture must be developed early and with support for reuse as a key objective. If it is not developed with reuse as an objective, is allowed to evolve in an ad-hoc manner, or is partitioned amongst multiple contractors with no contractual incentive to discover and exploit commonality, then it will not support large-scale, project-wide reuse. If the management support for reuse is not established early and supported from the top, then investment, governance, and incentive strategies will evolve that do not support, or even suppress, project-wide reuse. In either case, the benefits of product lines will be lost. The team thus recommends the establishment of a software systems engineering and management office at the Constellation program level that is broadly chartered to execute software product lines within the project. Effective utilization of large-scale systematic reuse within Constellation will require the NASA engineering and management culture to take an evolutionary step forward. The team thus focused much of its time on the risks and obstacles to effective reuse and identified possible mitigations. Several case studies of the development of successful software product lines within CONSTELLATION PROGRAM SYSTEMS ENGINEERING CONSTELLA NEERING Title: Strategy for Software Reuse Study Final Report Document No.: AC1L1-52 Effective Date: October 1, 2005 Revision: 0 Page 9 NASA were developed and studied, yielding important lessons learned. These cases included both flight and ground software systems from four mission Centers. The report concludes with the unanimous recommendation that the Constellation program adopt the software product line engineering methodology as the basis of all software development within the project. The team developed multiple recommendations and identified the next steps to be taken towards this objective. The time to start is now. CONSTELLATION PROGRAM SYSTEMS ENGINEERING CONSTELLA NEERING Title: Strategy for Software Reuse Study Final Report Document No.: AC1L1-52 Effective Date: October 1, 2005 Revision: 0 Page 10 2.0INTRODUCTION 2.01 Purpose The purpose of this study was to develop a set of guidelines and policies governing the appropriate strategy for reuse of software to ensure the most cost effective, schedule-efficient, and productive use of development efforts across all elements/projects of the Constellation Program. This was undertaken in support of the level 0 requirements for an “affordable” and “sustainable” exploration program. 2.02 Study Objectives The initial study objectives, which were to investigate and create a “software systems strategy” addressing the Constellation goals of reuse, interoperability, compatibility, and improvement, became too broad for this study. Recognizing that solutions to these goals are strongly interrelated, and that the concept of “reuse” is central to an overall solution, the study team elected to focus the available time and resources on the objective of developing the programmatic and technical precursors to the development of a “Strategy for Software Reuse,” because doing so would enable the timely and efficient development of an actionable strategic reuse plan. It was anticipated that follow-on work would be conducted to develop the actionable strategic reuse plan for Constellation. Therefore the principal objective of this study has been to identify the key benefits, costs, and risks of a reuse-based software engineering strategy for the Constellation Program Systems Engineering Office (with flow-down of requirements and recommendations to Constellation programs) that will help enable interoperability, compatibility, and improvement through effective software engineering for reuse, and will help the Program realize significant benefits from reuse of software engineering artifacts across the multi-mission lifecycles of the various Constellation systems. This report is comprised of an initial set of best practices, risks, recommendations, and linkages to key program systems engineering products with special focus on deliverables for the System Requirements Review (SRR) and consequent requirements for further studies. 2.02.1 Relationship to Exploration Objectives The results of this study will support Constellation policy decisions and architecture development for a software strategy that will be instrumental in “implementing a sustainable and affordable human and robotic program.” In order to “extend human presence across the solar system and beyond,” Constellation systems will require multiple diverse vehicles and systems. Each of the exploration vehicles will require significant software to perform needed tasks from low-level command and control to autonomous operations, and will need to support extensive interoperability and coordination. Software lifecycle costs will only be affordable and sustainable if a strategic architecture for reuse is developed and adopted at the onset so that the development costs of core assets can be amortized across multiple uses of those assets within the vehicles and systems of Constellation. Software reuse, interoperability, and compatibility can only be CONSTELLATION PROGRAM SYSTEMS ENGINEERING CONSTELLA NEERING Title: Strategy for Software Reuse Study Final Report Document No.: AC1L1-52 Effective Date: October 1, 2005 Revision: 0 Page 11 achieved if the program can “develop supporting innovative technologies, knowledge, and infrastructures.” A fully defined strategy for software reuse, interoperability and compatibility will enhance “international participation in exploration” by allowing software from diverse sources to integrate in a predictable manner and with predictable and high-resultant system dependability. Finally, the requirements generation process will be impacted by this study since the software strategy for reuse, interoperability, and compatibility will be a factor in the set of requirements approved at SRR. 2.02.2 Relationship to Constellation Modeling and Simulation Constellation Modeling and Simulation (formerly known as Simulation-Based Acquisition [SBA]) is intended to support program decisions with accurate and timely data. The modeling and simulation activity ensures that the infrastructure of tools, policies, and organizational working groups required for modeling and simulation are built, maintained, and improved as necessary. As an analysis effort generated by Constellation Integrated Discipline Team (IDT) efforts, this study has been mindful of and compliant with policies regarding analysis tool development and use. There are two complementary aspects to the relationship between modeling and simulation and the area of reuse. One is that, as Constellation evolves, the tools developed for modeling and simulation must evolve as well, and reuse and compatibility of those tools will be an issue. The other is that the tools will be used to test systems that include reused software. Adoption of an architecture and framework for reuse will provide a vehicle for effective evolution of supporting tools and provide a basis for testing of reused software and newly developed software that is intended for future reuse. 2.02.3 Relationship of This Study to Other IDT Studies There are several other IDT studies that are also concerned with the major aspects of software reuse. These studies include but are not limited to: • Command Control and Communications (CCC) IDT AC1L1-85 Command and Control Interoperability Study (C2I) • Aerosciences and Flight Mechanics (AFM) IDT AC1L1-18 Automated Rendezvous and Docking (AR&D) • AFM IDT AC1L1-19 Mated Flight Operations (MFC) • Computers, Software and Automation (CSA) IDT AC1L-55 Autonomy and Automation Roadmap for SRR Each of these has important software reuse aspects related to the focus of their study. In particular, each study is developing elements of the overall architecture that must in the end be made compatible with the objective of facilitating reuse. These IDT studies are described in the following subsections. CONSTELLATION PROGRAM SYSTEMS ENGINEERING CONSTELLA NEERING Title: Strategy for Software Reuse Study Final Report 2.02.3.1 Document No.: AC1L1-52 Effective Date: October 1, 2005 Revision: 0 Page 12 CCC IDT AC1L1-85 Command and Control Interoperability Study The C2I and the Autonomy and Automation Roadmap for SRR studies (see subsection 2.2.3.5) are both analyzing the sets of software product lines for systems and subsystems. Exploration missions pose new challenges for command and control, because they combine operation of unmanned automated robotic spacecraft with relatively autonomous manned spacecraft. The C2I study collected historical data from a set of manned and unmanned spacecraft. For the X-15, Mercury, Gemini, and Apollo programs, control was either hardwired into the spacecraft or performed by the crew. Although these were built as custom systems, there has been a pattern of reusing computer systems and software programs in migrating from Apollo to Shuttle and Station. The number of lines of code and number of available commands increases for newer spacecraft in the manned space programs. Unmanned spacecraft tend to use more complex commands. Manned spacecraft have at least an order-of-magnitude more telemetry parameters than unmanned spacecraft from the same time era. The study also looked into locus-of-control issues, i.e., the designation of what entity (ground, element automation, or crew) is authorized to issue commands to a spacecraft. When two vehicles rendezvous (such as Shuttle docking at the ISS or Shuttle retrieving a payload), one spacecraft is designated as the controlling entity while the other spacecraft is the passive target. To support Exploration missions, inter-element command and control autonomy and automation will be needed that requires shifting the locus of control among the Constellation elements, depending upon the mission phase and configuration of vehicles. This drives a substantial need for a software reuse strategy that allows software to be shared between Constellation elements that are designed and built by different contractors during different time frames. The study recommends further historic research (particularly for data involving the complexity of commands used for different spacecraft) in order to (a) form a clearer idea of the trends in locus of control, and (b) identify the need to distribute and shift the locus of control to support the needed inter-element command and control for autonomous and automated Exploration missions. 2.02.3.2 AFM IDT Study AC1L1-18 Automated Rendezvous and Docking Successful execution of the Constellation Program will require extensive utilization of automated and autonomous rendezvous and docking capability and operations infrastructure for both crewed and un-crewed missions. This requires a significant element of interoperable command and control. This study provides the trade results and quantitative data required for programmatic decisions, and the requirements development effort to produce a Constellation rendezvous and docking Interface Requirements Document (IRD) as specified in the Exploration level 1 requirements release. It also establishes the basic interface requirements that envelope the performance and operations for rendezvous and docking between various Constellation elements. This study is a critical component of the overall NASA strategy of implementing a sustainable and affordable CONSTELLATION PROGRAM SYSTEMS ENGINEERING CONSTELLA NEERING Title: Strategy for Software Reuse Study Final Report Document No.: AC1L1-52 Effective Date: October 1, 2005 Revision: 0 Page 13 human and robotic program. And again, this drives a substantial need for a software reuse strategy that allows software to be shared between Constellation elements that are designed and built by different contractors during different time frames. 2.02.3.3 AFM IDT Study AC1L1-19 Mated Flight Control Operations The Exploration missions require the capability for two or several vehicles to be operated together as a single flight unit. This requires command and control intercommunication between mated elements, which again drives a substantial need for a software reuse strategy that allows software to be shared between Constellation elements that are designed and built by different contractors during different time frames. The software reuse requirements include command and control authority distribution options, information flow across mated elements, changes in vehicle static and dynamic characteristics, verification and validation (V&V) of mated vehicle dynamics; and guidance, navigation and control (GN&C) configuration changes, interface mechanism variability, articulation of vehicle components, mass redistribution, and adaptation to multiple flight configurations and operating modes. Each element will have its own GN&C subsystem, capable of executing GN&C functions and with some degree of built-in redundancy for that element, but not necessarily adequate for the mated vehicles. Exploration mated-element command and control software will require shifting the locus of control among mated elements, depending upon the mission configuration and phase. Mated flight command and control software requirements for Crew Exploration Vehicles (CEV) may require over-design to accommodate controlling various configurations of mated flight control challenges. Command and control interoperability for mated flight control (MFC) requires a methodical approach to managing GN&C subsystems redundancies for multiple mated configurations, defining essential components (hardware/software) for each GN&C subsystem, defining MFC GN&C performance requirements for the mated vehicles, identifying all possible GN&C subsystems (feasible set) that can satisfy MFC GN&C performance requirements, and selecting an optimal GN&C configuration from the feasible set. Each element will have independent actuator sets capable of controlling that element, but not necessarily adequate for the mated vehicles. The CEV’s C&DH subsystem must be capable of communicating information across the mated interfaces for MFC operations. A candidate information set across mated interfaces includes mass, center of mass, moments of inertia in three degrees of freedom (3DOF), position, attitude, attitude rates, velocity, acceleration, and deltas, consumables inventory and rate of use; particularly propellant; and shared electrical power, including generation, usage, and battery charging rates and state-of-charge. The dynamic model of all mated configurations involving CEV must be sufficiently accurate to reliably predict MFC responses to within a “to-bedetermined” specification under both nominal and off-nominal conditions. The GN&C subsystem performance must be sufficiently robust during all MFC operations including both nominal and off-nominal conditions, driving requirements for software reuse across a complex of Constellation elements. CONSTELLATION PROGRAM SYSTEMS ENGINEERING CONSTELLA NEERING Title: Strategy for Software Reuse Study Final Report 2.02.3.4 Document No.: AC1L1-52 Effective Date: October 1, 2005 Revision: 0 Page 14 CSA IDT Study AC1L1-56 Integrated Systems Health Management Study Integrated Systems Health Management (ISHM) is a key component for software reuse and interoperability across Constellation elements. ISHM is a system engineering discipline that addresses the design, development, operation, and lifecycle management of components, subsystems, vehicles, and other operational systems with the purpose of maintaining nominal system behavior and function and assuring mission effectiveness under off-nominal conditions. For the Constellation Program, ISHM will enable (1) autonomous and automated launch-abort and crew-escape capability; (2) efficient testing and checkout of ground and flight systems during various mission stages; (3) monitoring, verification, and trending of ground and flight system operations and performance; (4) enhanced situation awareness and control options for ground personnel and crew; (5) vehicle autonomy (self-sufficiency) in responding to off-nominal conditions during long-duration and distant exploration missions; (6) in-space maintenance and repair; and (7) efficient ground processing of reusable systems. Many of these scenarios require rapid response to prevent or mitigate mission- or life-threatening conditions. Effective health management requires integration of command and control information from multiple systems and subsystems developed by a multitude of contractors over a period of decades, thus driving requirements for software reusability. It is necessary to approach ISHM as an integrated systems engineering discipline. Traditional health management functional needs are allocated to individual subsystems and components with no requirements for commonality, interoperability, or even data interchange among implementations. This haphazard approach to health management results in inefficiencies in system design and operation, inconsistent coverage of mission- and life-critical failure modes, and inability to recover from failures in a predictable manner. Human involvement is often required to interpret and integrate pieces of information obtained from disparate subsystems in order to make mission-critical health and safety decisions at the systems level. Experience with aircraft and spacecraft shows that an integrated approach to development, deployment, and management of ISHM requirements, architectures, and systems is critical to mission success. Such integration can be achieved only by defining and enforcing software reuse standards and best practices for ISHM design, certification, operations, interoperability, supportability, and knowledge management across Constellation programs. 2.02.3.5 CSA IDT Study AC1L1-55 Autonomy and Automation Roadmap for SRR This study includes investigation of both autonomous and automated software capabilities as they apply to the Constellation Program. Crewed as well as robotic missions are covered in the study. Today’s mission management roles and responsibilities between onboard crew and mission control need to evolve to accommodate missions beyond the Earth-moon system. With increased travel times for communication and telemetry, a great deal of authority for decisionmaking will be transferred to the onboard crew. As such, onboard expert systems, automation, and insight into spacecraft systems will be required to provide the tools for the crew to make CONSTELLATION PROGRAM SYSTEMS ENGINEERING CONSTELLA NEERING Title: Strategy for Software Reuse Study Final Report Document No.: AC1L1-52 Effective Date: October 1, 2005 Revision: 0 Page 15 educated and informed decisions. There must be appropriate balance between human versus computer functions (automation) and ground versus onboard functions (autonomy). The strategy for autonomy and automation will have major impact on the design of these systems as the capability for manual control evolves into automatic control. Schedule and cost constraints will dictate the continued software reuse of legacy capabilities, leading to interface issues, model and data compatibilities, continuous maintenance of redundant (but different) information sets, and reverse engineering of legacy systems for reuse. Designing automation and autonomy software systems for reuse and evolution requires hardware independence, language and development infrastructure independence, and placing appropriate performance requirements on supporting hardware and software infrastructure. 2.03 Ground Rules and Assumptions There are no simple answers for the core themes of Constellation Program Systems Engineering: reuse, interoperability, compatibility, and improvement. Experience has shown that effective software engineering strategies for reuse must be planned and in place very early in the Program lifecycle. For example, Clements and Northrop (2001) show that early commitment to software product lines (the most comprehensive form of reuse) is a key success factor. A study that examines the requirements of the Constellation Program missions and product lifecycles versus the best practices in software engineering for reuse, is essential. The Constellation Program will develop a series of vehicles with space and ground systems that will be refined repeatedly through incremental development. The spacecraft will have extensive computing capabilities and require large and complex on-board software systems with millions of lines of code. Constellation system software will need to be continuously written, rewritten, modified, repaired, tested, verified, validated, deployed, and obsolesced during sustained operations. It will not be possible to sustain such a system over time without applying software engineering principles to effectively modularize the system such that system elements can be added, removed, and evolved independently for both flight and ground. There is a clear requirement for software systems to be sustainable and interoperable for the lifetime of Constellation elements, possibly up to 20–30 years for certain reusable in-space systems. In the arena of software, with its accelerating progress, concepts of software reuse will be enabling for this requirement. This study will help drive technology requirements for the Earth-local and lunar missions. In fiscal 2005, this topic is the responsibility of the Computers, Software and Automation IDT, with support from several other related IDTs. In future years, the Program Systems Engineering activities will further address this topic. 2.04 Study Task Description The study team sought to answer the questions: • What is software reuse and what does it mean in the context of Constellation? CONSTELLATION PROGRAM SYSTEMS ENGINEERING CONSTELLA NEERING Title: Strategy for Software Reuse Study Final Report Document No.: AC1L1-52 Effective Date: October 1, 2005 Revision: 0 Page 16 • What are the potential benefits? • What are the risks and obstacles? • What can be done to mitigate risks and avoid or overcome obstacles? Deliverable content in this Final Report of the study includes: • Data and analysis regarding these questions • Recommendations in multiple technical and non-technical areas • Figures of Merit, metrics and measurements relevant to software reuse processes and products across the software system and mission lifecycle(s) • Identification of Constellation program systems engineering work products that are impacted by the study, with a focus on the major products required for SRR • Recommended next steps • Recommended follow-on studies After the first draft of this study was released, a subset of the original team broadened the scope of the study from the original scope of “software reuse” to the larger range of “reuse,” “interoperability,” “compatibility,” and “improvement.” This was done in response to a review that noted that Software Product Lines, a mature reuse discipline developed for use on largescale projects, was a superset of the study recommendations and a comprehensive solution. Section 3.7 of this report outlines the Software Product Line practice areas. 2.05 Key Issues for Constellation Reuse Strategy Early in the conduct of this study, it became apparent that the study team had a broad set of concerns, issues, ideas, and forward-looking approaches to the basic challenge of effective software reuse. Four unifying themes that emphasize the importance of reuse strategy are highlighted below: • The greatest return from software reuse occurs when software artifacts are planned from the beginning as reusable assets. Hence, this report focuses on planned reuse; ad hoc or opportunistic reuse is assumed to be part of the normal development process and is not emphasized as a key element of Constellation reuse strategy. • Perhaps the most important part of creating a planned reuse strategy is the focus on upfront programmatic planning and investment in reusable assets (software artifacts, tools, infrastructure). Once the software architecture and product line plans are in place, programmatic plans can be made for developing key assets. This requires more up-front investment in software than is typical for a NASA mission. It is this upfront investment that will provide the greatest return in terms of cost, reliability, and schedule savings (see discussion in Sections 3.0 and 4.0 and recommendations in Sections 8.0 and 9.0). CONSTELLATION PROGRAM SYSTEMS ENGINEERING CONSTELLA NEERING Title: Strategy for Software Reuse Study Final Report Document No.: AC1L1-52 Effective Date: October 1, 2005 Revision: 0 Page 17 • The product line approach appears to have significant advantages for the Constellation Program. This approach is discussed in multiple places in this report and is an important part of the recommendations in Section 8; however, the productline concept as an integrating theme for the Constellation Reuse Strategy has not been fully developed. Hence, this topic is proposed as a follow-on study (Section 9.0). • In order to make software reuse a part of the Constellation Program from its inception, it is critical to include the CEV project and to ensure that project’s compliance with the adopted recommendations of this report. The next window of opportunity to impact CEV is the upcoming Call-for-Improvement (CFI) to be released by early FY06. In this CFI, specific sections devoted to software reuse policy need to be included. For instance, it is recommended that the CEV Project’s Software Management Plan and the applicable Constellation SEI standards and other products passed as requirements to the CEV project, should include provisions based on an explicit Software Reuse Plan (see Section 8.1.3). CONSTELLATION PROGRAM SYSTEMS ENGINEERING CONSTELLA NEERING Title: Strategy for Software Reuse Study Final Report Document No.: AC1L1-52 Effective Date: October 1, 2005 Revision: 0 Page 18 3.0SOFTWARE REUSE 3.01 Section Purpose and Scope This section introduces the fundamental concepts of software reuse and how software reuse should be incorporated into the Constellation Program lifecycle. After a brief discussion on the history of software reuse, a working definition of software reuse is provided, along with the scope of the concept. Subsequent subsections describe key aspects of software reuse within the context of the Constellation Program lifecycle. First, some background is presented. The notion of a reuse strategy is then presented, and shown to be a key element to effective and high-payoff utilization of software reuse. Next, the concept of software architecture is described, and again shown to be a key element to reuse within the program. The concept of a software product line is described and shown to enable a comprehensive and systematic reuse program over the total lifecycle of the Constellation Program. Subsequent sections describe additional practices that have been found to be beneficial and others that have been nonproductive. 3.02 Background The reuse of software code in order to improve software productivity was first proposed by M. D. McIlroy (1968) in an invited address that he presented at the groundbreaking NATO Conference on Software Engineering. More recently, IEEE Transactions on Software Engineering dedicated a special section to the same topic (TSE, 2005). The general claims and advantages for software reuse have remained remarkably stable over the decades: cost, schedule, and reliability may be improved—but are not necessarily improved—by reusing various software development assets (code, requirements, specifications, tests, knowledge, etc.) rather than recreating them over and over again. There are three core issues that are central to the practice of software reuse: 1. Establishing and refining the reuse strategy. The software reuse strategy lays out the plan for investing in reusable assets (software artifacts, tools, infrastructure) in order to achieve long-term benefits. It includes the specification of a software architecture, product line plans, policies and incentives for reuse, as well as a business plan that shows the future return to be achieved from reuse investments. The business case motivates investment now for benefits accrued later (Nazareth, 2003). 2. Improving the technical elements of the reuse strategy. The technical challenge of software reuse is, in essence, how to make finding and incorporating reusable core assets more efficient than recreating those assets from scratch. This includes questions of how to create reusable core assets in the first place (investment), as well as how to find and utilize them later (accrued benefit), how to maximize the technical opportunities for reuse (typically via software product lines [Lutz, 2002]), and how to quantify and manage the risks of reuse (Yao, 2004). CONSTELLATION PROGRAM SYSTEMS ENGINEERING CONSTELLA NEERING Title: Strategy for Software Reuse Study Final Report Document No.: AC1L1-52 Effective Date: October 1, 2005 Revision: 0 Page 19 3. Improving the non-technical elements of the reuse strategy. Research and practical experience have shown that software reuse may not occur, even when a rational business case seems to indicate that it would be profitable to do so. Various reasons for this have been proposed, including cultural inertia, inappropriate reward system, unquantified or unquantifiable risk (“not invented here” syndrome), and the like (Menzies, 2003). This area of reuse strategy focuses on managing the non-technical incentives and disincentives for reuse. The remainder of this section explores the opportunities and implications of software reuse within the context of the Constellation Program. Figure 3-1. Market for Reusable Software Artifacts 1 3.03 Various Definitions of Software Reuse The definition of software reuse varies among organizations and across development communities. Some communities practice software reuse by utilizing commercial software libraries as a matter of routine, and thus, do not define that practice as “software reuse” and do not explicitly represent the use of the library code in their cost models. This means the accrued advantage is apparently tacitly distributed across the costing of the other elements of the work. This view on software reuse is ubiquitous in the commercial software sector. In other communities of practice, for example safety-critical software development, this practice is more limited. 1 Courtesy NASA GSFC [http://softwarereuse.gsfc.nasa.gov/] CONSTELLATION PROGRAM SYSTEMS ENGINEERING CONSTELLA NEERING Title: Strategy for Software Reuse Study Final Report Document No.: AC1L1-52 Effective Date: October 1, 2005 Revision: 0 Page 20 Some communities distinguish between “accidental” and planned reuse. Accidental or “ad hoc” reuse occurs when an individual informally reuses artifacts that were not designed for reuse and without the use of reuse-oriented processes. Planned reuse means that software artifacts are designed from the beginning for the purpose of being used in different domains or projects. The focus of this report is on planned reuse. Software reuse is not just limited to reusing the code assets but it includes everything that is required to provide a technically complete system:2 1. Software libraries 9. Analysis 2. Software components 10. Test 3. Application libraries 11. API specifications 4. Design patterns 12. Protocols 5. Parametric programming 13. Architecture frameworks 6. Aspect-oriented programming 14. Training 7. Requirements 15. Knowledge 8. Design The most comprehensive definition of reuse is that of software product lines: A software product line is a set of software-intensive systems sharing a common, managed set of features that satisfy the specific needs of a particular market segment or mission and that are developed from a common set of core assets in a prescribed way. (P. Clements and L. Northrop, Software Product Lines: Practices and Patterns, [Boston: Addison-Wesley, 2002], p. 5) A comprehensive program must involve program-level activities in order to enable reuse on a scope that will improve costs and efficiency at the program level. In general, the program-level activities include (Kemerer, 1987): 2 • Facilitate construction of artifacts that are more effectively reused. • Develop a program system-of-systems architecture that maximizes opportunities for reuse. • Promote formal and informal incentives that encourage sharing and reuse. • Develop and maintain a software reuse cost model to guide investments. • Implement a metrics program that allows reuse costs and benefits to be quantified and fed back into calibration of the cost model. This is an extension of the list found in the Software Engineering (Sommerville), http://www.comp.lancs.ac.uk/ computing/resources/IanS/SE6/ CONSTELLATION PROGRAM SYSTEMS ENGINEERING CONSTELLA NEERING Title: Strategy for Software Reuse Study Final Report 3.04 Document No.: AC1L1-52 Effective Date: October 1, 2005 Revision: 0 Page 21 Constellation and Reuse Constellation is a very large many-tiered program with many stakeholders that have mature competitive-advantage software development capabilities, and has a very long lifecycle. This dictates certain properties of any viable software reuse program: 4. The software reuse strategy for Constellation must be developed, refined, and maintained so that it can be used to guide program investments in software reuse. 5. The reuse program must have a phased implementation. 6. The program must accommodate legacy, commercial, and designed-for-reuse elements so that the reuse program does not require unnecessary reconstruction of assets. 7. The program must have a flexible investment (cost) profile so that inevitable fluctuations over the life of the program can be accommodated. 8. The program must include development of a set of domain-specific software architectures (DSSA) to maximize potential commonality amongst reusable elements, and to increase return on investment (ROI) by engineering-in flexibility in the face of obsolescence. 9. The program must facilitate institutional self-improvement in the domain of reuse, since a large reuse program like this is certain to extend the state-of-the-art and the state-of-the-practice of software reuse. 10. The interfaces and practices that must be common amongst stakeholders should be negotiated in a neutral manner, e.g., within a standards body, perhaps the Object Management Group (OMG) and/or IEEE. 11. The incentive mechanisms must be engineered, based on the business case, to cause rational self-interest on the part of all involved parties to motivate actions that coincide with the objectives of the Constellation reuse program. These measures are necessary because of the tremendous scope of the Constellation Program. There will be a vast amount of new and legacy software; there will be many political, regulatory, and corporate stakeholders; much of the software will be safety-critical; there is a lot of money involved; and the program will exist for several decades. If the nation’s taxpayers are to gain maximum benefit from software reuse within Constellation, then the reuse program must be national in scope. Most of the research done on software reuse is performed within the context of industrial software, and consequently, the primary focus is on profit. In particular, improvements in software quality and reliability are generally noted as side effects of reuse, rather than primary objectives. There is also little discussion on the topic of reuse for safety-critical software, and not much practical experience, though the topic has come up (Lougee, 2004, 2005). Constellation will CONSTELLATION PROGRAM SYSTEMS ENGINEERING CONSTELLA NEERING Title: Strategy for Software Reuse Study Final Report Document No.: AC1L1-52 Effective Date: October 1, 2005 Revision: 0 Page 22 require large amounts of extremely reliable software built and tested to formal standards for safety-critical software (DO-178B/ED-12B, 1992). It is likely that software reuse can be used to manage and improve software reliability in its own right, but further original research will probably be necessary if maximum benefit is to be accrued. 3.05 Reuse Strategy The Constellation Reuse Strategy lays out the programmatic plans for the implementation of reusable software across the Constellation Program. The entire document is focused on articulating the key concepts to be addressed by any software reuse strategy for Constellation; this subsection provides an introduction to the elements that make up that strategy. All of these elements are discussed in more detail elsewhere in this report. Major elements include: • Policies, processes, and practices • Software architecture • Programmatic plan for investing in reusable assets • Policies and incentives for reuse • Business case Because of the importance of developing the software architecture and product line plans, these elements are discussed in separate subsections (3.6 and 3.7 below). Once these plans are in place, programmatic plans can be made for developing key assets. Perhaps the most important part of creating a reuse strategy is the focus on up-front programmatic planning (development of policies, processes, and practices) and investment in reusable assets (software artifacts, tools, infrastructure). As discussed in several sections of this report, the greatest return from software reuse occurs when software artifacts are planned from the beginning as reusable assets. Many of these assets should be developed prior to their first use by a project and delivered to the project user (see discussion in Subsection 4.2, the AMMOS case). This requires more up-front investment in software than is typical for a NASA mission. Although the return on this investment will not be immediate, it is this up-front investment that will provide the greatest long-term return in terms of cost, reliability, and schedule savings (see recommendations in Sections 8.0 and 9.0). Policies and incentives for reuse are discussed throughout this entire report. The previous two subsections (3.3 and 3.4) review policies that are derived from the nature of the Constellation Program. Incentives must be provided to the individual projects to encourage reuse (see Figure 3-1) and funding must be provided for infrastructure and tools that support the reuse effort (see Subsection 3.3). Contracts must be written to incentivize reuse. Specific attention must be paid to organizational or cultural disincentives for reuse (see Sections 4.0 and 6.0). Section 8.0 provides a summary of the policies that are recommended by the study team. By establishing these policies early in the program, individual project plans can be developed consistent with program plans and contracts can be written that support the reuse program. CONSTELLATION PROGRAM SYSTEMS ENGINEERING CONSTELLA NEERING Title: Strategy for Software Reuse Study Final Report Document No.: AC1L1-52 Effective Date: October 1, 2005 Revision: 0 Page 23 The business case shows the value of the reuse strategy. It documents the savings to be achieved by individual projects and shows the costs of programmatic investment in reuse. It will highlight improvements in reliability and reductions in schedule to be achieved through reuse. The business case establishes the relationships across time and between costs, benefits, schedule, functionality, and risks. In the case of a multi-tier program like Constellation, it would take political and organizational boundaries into account as well. The business case is used to direct investments to where they will yield the best return. These elements, along with Architecture and Product Lines discussed in the next two subsections, comprise the reuse strategy. The rest of this report provides additional detail and further insight into these elements of reuse strategy. 3.06 Architecture The software architecture of a program or computing system is the structure or structures of the system, which comprise software elements, the externally visible properties of those elements, and the relationships among them. By “externally visible” properties, we are referring to those assumptions other components can make of a component, such as it provided services, performance characteristics, fault handling, shared resource usage, and so on. (P. Clements and L. Northrop, Software Product Lines: Practices and Patterns, [Boston: Addison-Wesley, 2002], p. 57) Constellation must have a comprehensive and well-defined software architecture if it is to have a far-reaching software reuse program (there are many other reasons as well, and if the architecture is not developed, then costly consequences are likely to ensue). The software architecture provides the “context” for the elements of the software systems: how the elements must interact (and must not interact) to accomplish the objectives of the system, how resources are managed, what functionality goes where, etc. If the architecture does not define how the elements interact, then they will inevitably interact in different ways. Each different pattern of interaction will become part of the operational specification of the relevant software, and will hinder its reuse in other contexts. If interaction patterns are standardized, and unnecessary interaction is minimized, then opportunities for reuse will increase and the cost of developing reusable elements will decrease. Thus, the upper bound on the ROI is increased by the existence of a strong architecture. Architecture is key to the success of any software project, not just a software product line. Architecture is the first design artifact that begins to place requirements into a solution space. Quality attributes of a system (such as performance, modifiability, [dependability] and availability) are in large part permitted or precluded by its architecture—If the architecture is not suitable from the beginning for these qualities, don’t expect to achieve them by some miracle later… The right architecture is absolutely essential for smooth sailing. The wrong one is a recipe for disaster. (P. CONSTELLATION PROGRAM SYSTEMS ENGINEERING CONSTELLA NEERING Title: Strategy for Software Reuse Study Final Report Document No.: AC1L1-52 Effective Date: October 1, 2005 Revision: 0 Page 24 Clements and L. Northrop, Software Product Lines: Practices and Patterns, [Boston: Addison-Wesley, 2002], p. 57) The large scale of the Constellation software systems suggests that it may be best organized as a number of layers of architecture: a system-of-systems for the overall program, an architecture for the various flight/ground elements, and so on. Development of this architecture is a key element of the Software Research Strategy (see recommendations in Sections 8.0 and 9.0). As the above quote indicates, this work must also begin early in the project lifecycle. The development of the architecture is also a Software Product Line key practice. 3.07 Product Lines The purpose of a software product line is to exploit economies of scope, by taking advantage of commonalities between the many elements of a project. The economies of scope are engineered into the product line—they don’t just happen. In a sense, software product lines represent the end game of software reuse: under the software product line paradigm, the system itself is designed from the beginning to offer maximum opportunities for reuse, and the software engineering, and non-technical management is structured to take maximum advantage of those opportunities. This goes far beyond simple reuse of code. Software Product Lines are the distillation of best practices by the Software Engineering Institute at Carnegie Mellon University for the Department of Defense. The SEI examined billions of dollars of reuse-oriented software development in the defense, commercial, and aerospace sectors to develop the practices. The study team found that the “Software Product Lines” practices are the most comprehensive and most broadly used framework for a large-scale reuse program in existence today. The Constellation Program mitigates risk by using a spiral development model, each spiral undertaking a more ambitious goal, while building upon the work done in previous cycles. Each cycle provides an opportunity to learn from the previous cycle, and modify the approach as necessary. That is an ideal environment for software product lines, as the project itself is in fact a product line. What is a Software Product Line? A software product line (SPL) is a set of software-intensive systems that share a common, managed set of features satisfying the specific needs of a particular market segment or mission and that are developed from a common set of core assets in a prescribed way. Why are Software Product Lines Important? Software product lines are rapidly emerging as a viable and important software development paradigm allowing companies to realize order-of-magnitude improvements in time to market, cost, productivity, quality, and other business drivers. Software product line engineering can also enable rapid market entry and CONSTELLATION PROGRAM SYSTEMS ENGINEERING CONSTELLA NEERING Title: Strategy for Software Reuse Study Final Report Document No.: AC1L1-52 Effective Date: October 1, 2005 Revision: 0 Page 25 flexible response, and provide a capability for mass customization. (Carnegie Mellon University, “Software product lines,” online document) Product lines save time and money: systems that use the same components and offer the same features require less time for planning and rework, have lower production costs, and require fewer staff members. Because the shared components of a product line are reused from system to system (rather than built from scratch every time), the systems in software product lines are also highly reliable. (Carnegie Mellon University, “Product line systems program,” online document) The software product line captures both the architecture and the managed evolution of the system. Product lines facilitate software reuse across elements of the program, and reuse across the life of the program. There are a number of software product line practice areas. It is out of scope of this report to go into detail about them all. However, it is useful to present an outline of the areas in order to convey the scope of the practice of product lines (Clements and Northrop, 2001). 12. Software Engineering Practice Areas • Architecture Definition • Architecture Evaluation • Component Development • COTS Utilization • Mining Existing Assets • Requirements Engineering • Software System Integration • Testing • Understanding Relevant Domains 13. Technical Management Practice Areas • Configuration Management • Data Collection, Metrics, and Tracking • Make/Buy/Mine/Commission Analysis • Process Definition • Scoping • Technical Planning • Technical Risk Management • Tool Support CONSTELLATION PROGRAM SYSTEMS ENGINEERING CONSTELLA NEERING Title: Strategy for Software Reuse Study Final Report Document No.: AC1L1-52 Effective Date: October 1, 2005 Revision: 0 Page 26 14. Organizational Management Areas • Building a Business Case • Customer Interface Management • Developing an Acquisition Strategy • Funding • Launching and Institutionalizing • Market Analysis • Operations • Organizational Planning • Organizational Risk Management • Structuring the Organization • Technical Forecasting • Training 3.08 Software Reuse Success The case studies in this report show that many organizations have searched for and found ways to reduce the cost of software development efforts through the use of software reuse. To achieve this, much attention has been given to a combination of technical, non-technical, and management approaches. Technical approaches have included the use of automatic code generation and more esoteric approaches such as algorithm synthesis. Non-technical approaches have included both organizational-wide and ad-hoc software repositories, and use of shared libraries of vendor-supplied code. From a management level, one sees everything from a casual position on software reuse to edicts that “Thou shall reuse whenever possible.” Some people are resistant to any form of reuse because of the “not invented here” syndrome. That is why a successful program must involve non-technical management at the highest level that will nurture and maintain the program. Software reuse is most successful when it is designed-in from the beginning of the project as part of a software product line process. Such a process includes, but is not limited to, the development of system architectures, software architectures, common standards, protocols, and interfaces. The many case studies in the literature clearly demonstrate that trying to retroactively add in (or slam in) a policy of reuse later in the lifecycle ends up costing more than rebuilding from scratch. In addition to setting out on the right path for software reuse at the onset of the project, to succeed in the large, reuse efforts must address a spectrum of both technical (tools and techniques) and non-technical (corporate, management, policies) issues. A strong software architecture, well thought out product line plan, and an effective reuse strategy are essential to achieving successful software reuse. CONSTELLATION PROGRAM SYSTEMS ENGINEERING CONSTELLA NEERING Title: Strategy for Software Reuse Study Final Report Document No.: AC1L1-52 Effective Date: October 1, 2005 Revision: 0 Page 27 CONSTELLATION PROGRAM SYSTEMS ENGINEERING CONSTELLA NEERING Title: Strategy for Software Reuse Study Final Report Document No.: AC1L1-52 Effective Date: October 1, 2005 Revision: 0 Page 28 4.0CASE STUDIES 4.01 Introduction The purpose of analyzing case studies is to focus on lessons learned from prior reuse efforts that are directly applicable to the Constellation Program effort. This section consists of two types of case studies: NASA and Non-NASA case studies. There are a total of nine separate studies followed by a Discussion and Analysis subsection that emphasizes the lessons of importance to Constellation Systems and a set of product line solutions for the challenges derived from the cases. 4.02 NASA Case Studies Four NASA case studies and one summary report were documented or reviewed specifically for this Software Strategy for Reuse Study Final Report: 15. AMMOS Multi-mission System Reuse Case Study 16. ROSE—Flight Analysis and Design Reengineering Case Study 17. Checkout and Launch Control System (CLCS) Case Study 18. Summary of SEEDS Formulation Team Final Recommendation Report 19. GSFC Ground System Reuse Case Study The AMMOS case study illustrates the value of the product line approach and the importance of programmatic planning and guidance to the success of a system-level reuse effort. The ROSE case study describes a product line approach to reengineering and shows the need for detailed technical planning and training in executing such an approach. The CLCS case study shows the impact of cultural and organizational complexities on reusable product design and demonstrates the value of frameworks in software reuse efforts. The SEEDS report provides an overview of the reuse recommendations for the Earth Science Enterprise Data System. The GSFC Ground System case study describes some important product line concepts in the context of developing ground system software. 4.02.1 AMMOS Multi-Mission System Reuse Case Study The Advanced Multi-Mission Operations System (AMMOS) has been in use at JPL since 1989. Since that time it has been used by approximately 30 missions as the primary source of ground data system (GDS) software, hardware, procedures, and personnel. Although AMMOS is a true multi-mission system, it is comprised of individual elements (shown in Figure 4-1) that must be adapted, integrated, tested, and deployed for each individual mission. Lessons learned over the last 16 years of adaptation and core system development make an excellent case study of systemlevel reuse. 4.02.1.1 Mission Operations Services Provided A significant advantage of AMMOS is that broad Mission Operations Systems functionality is available for new missions. AMMOS provides a well-defined set of mission services: CONSTELLATION PROGRAM SYSTEMS ENGINEERING CONSTELLA NEERING Title: Strategy for Software Reuse Study Final Report Document No.: AC1L1-52 Effective Date: October 1, 2005 Revision: 0 Page 29 • Navigation • Planning and Sequencing • Spacecraft Analysis (includes spacecraft modeling and simulation tools) • Mission Control • Data Management and Archiving • Telemetry and Command • Computing and Communications • Instrument Operations The system elements comprising these services and their interrelationships are shown in Figure 4-1.3 Ready availability of these elements is a significant aid to mission planning in addition to their substantial reuse value. Figure 4-1. Mission System Elements 3 Unpublished presentation, “AMMOS Roadmap,” 2005. CONSTELLATION PROGRAM SYSTEMS ENGINEERING CONSTELLA NEERING Title: Strategy for Software Reuse Study Final Report 4.02.1.2 Document No.: AC1L1-52 Effective Date: October 1, 2005 Revision: 0 Page 30 AMMOS as a Software Product Line AMMOS was planned from the beginning as a reusable multi-mission system that required relatively small adaptations to create a mission-specific GDS. An important part of this strategy is that core GDS capability grows over time with later missions gaining from the experiences (and code) of earlier missions. Reuse impacts outside the area of code development are substantial: planning, operations, and risk reduction benefits are discussed below. AMMOS is a software-intensive system. Although there are significant hardware elements in any specific project implementation, it is the software portion that is inheritable. It is useful to view AMMOS as an early example of a Software Product Line (SPL). A mapping to current SPL terminology would be: Products → Individual Project GDS’s Product Line → Set of Project GDS’s Core Assets → AMMOS Subsystems (see Figure 4-1) AMMOS Architecture → Documents, Testing Plans, etc. Production Plan → Parameterization and Adaptation by Projects With this mapping it is easy to make comparisons to the typical SPL structure discussed in Clements and Northrop (2001). The lessons learned from AMMOS provide key insights into the benefits and challenges of the product line approach. 4.02.1.3 Return on Investment Calculating a meaningful return on investment (ROI) figure for AMMOS requires a detailed estimate of what mission-specific development would have cost for the 30 missions that have used AMMOS. Because of the difficulty of obtaining accurate, independent estimates of these 30 non-existent custom systems, few measures of ROI exist. Two approaches that demonstrate the significant value of multi-mission system reuse for current and future users of AMMOS are described in this section. Bottoms-up estimation. A 2004 presentation4 by Charles Klose of the AMMOS program office shows substantial savings provided by the multi-mission approach especially during the later years of his 17-year study. Klose uses a detailed bottoms-up approach to estimate the annual cost of each phase of a mission during the mission lifetime. This study indicates dramatic savings ($180–$250M per year) for the years 2005–2011. He also shows the significant reduction in flight operations costs that is possible if a multi-mission operations team is employed along with the AMMOS multi-mission GDS. These savings compare with an AMMOS program cost of approximately $40M per year. 4 Unpublished presentation, “AMMOS 2004 Update, ROI Benefit Model,” Feb. 2004. CONSTELLATION PROGRAM SYSTEMS ENGINEERING CONSTELLA NEERING Title: Strategy for Software Reuse Study Final Report Document No.: AC1L1-52 Effective Date: October 1, 2005 Revision: 0 Page 31 Lines-of-code estimation. An alternative parametric approach to estimating value for current missions is based on lines-of-code estimation: comparing lines-of-code needed for custom development with the alternative of significant inheritance from AMMOS. This approach is summarized in Table 4-1. Custom ground data systems for deep space missions are complex software-intensive systems comprising 5–10 million source lines of code (SLOC). Software inherited by current major users of AMMOS is given in Table 4-1.5 For any specific mission, one can compare the estimated cost (using any of the usual cost models) of generating a custom GDS with cost of a multi-mission system based on the inheritance from AMMOS. The cost of the AMMOS-based GDS is either known historical data or estimated for ongoing missions—see footnote (**) in Table 4-1. This estimation method has the added value of easy testing of its basic assumptions: SLOC for the custom system and the cost of development (per line-of-code). Sensitivity analysis of a wide range of reasonable values of these basic assumptions shows consistent savings generated by the multi-mission system. Table 4-1. Impact of Software Reuse on Deep Space Mission Ground Systems Project Genesis Size (MSLOC) 12.2 GDS with ith AMMOS % From Cost AMMOS $M (FY05) 93 7.5 Custom GDS GD Size* GDS Cost (MSLOC) $M (FY05) 3.5 87.5 Percent Savings 91.4 Mars Exploration Rover Deep Impact 15.1 75 26.2 5.5 137.5 80.9 14.4 78 9.4 4.5 112.5 91.6 Mars Reconnaisance Orbiter Spitzer Space Telescope Dawn 11.5 98 11.5 2.5 62.5 81.6 14.9 76 18.8 5.0 125.0 85.0 12.0 93 9.0 3.0 75.0 88.0 Custom GDS S Productivity: Productivity:*** ($/SLOC) 25.0 Average Savings 86.4 GDS Cost (with ith AMMOS) is s based on historical h orical values from rom the ESD Cost ost Reposit Repository, ry, the Proje Project and Financial Workforce rkforce Data Database, se, and the JPL Software Metrics Reposi Repository. ry. Where d data ta was not a available, estimated cost st is based o on a mixture of historical storical actu actuals, s, current bu budgets, ets, and est estimated mated data. **Custom GDS S size has b been en adjusted to reflect the fa fact ct that navigation naviga n software w would ould be inhe inherited with little modification. cation. **The cost perr SLOC assu assumes mes a doublin doubling in productivi productivityy due to COT COTS/GOTS S/GOTS or other ot er inheritance. inheritance This is in addition to o the naviga navigation n software iinheritance. eritance. A caveat to this LOC approach: substantial use of “government-off-the-shelf” (GOTS) or “commercial-off-the-shelf” (COTS) software might reduce costs of building a custom system to a level comparable to an AMMOS-based GDS. No one has yet demonstrated the practicality of doing this. However, navigation software would almost certainly be reused. The SLOC requirements for custom systems in Table 4-1 have been adjusted to reflect this fact, and the 5 Unpublished presentation, “AMMOS Roadmap,” 2005. CONSTELLATION PROGRAM SYSTEMS ENGINEERING CONSTELLA NEERING Title: Strategy for Software Reuse Study Final Report Document No.: AC1L1-52 Effective Date: October 1, 2005 Revision: 0 Page 32 productivity factor assumes that significant inheritance would occur even in custom system development. Even with these fairly conservative assumptions about custom GDS cost, this method provides savings estimates that are reasonably consistent with the Klose study. Both bottoms-up and LOC approaches to estimating ROI indicate significant value to the multimission approach for the long term. As the number of missions using AMMOS increases, this return becomes even more dramatic. Return in the early years is less clear—see Early User Considerations in Subsection 4.2.1.4. 4.02.1.4 Lifecycle Considerations Core software assets have a lifecycle; they will evolve and erode over time. This subsection explores the value stemming from 16 years of AMMOS use and then looks at problems (i.e., erosion) associated with this maturity. The discussion concludes with problems faced by early users. Advantages of a Mature Product Line Much of the value provided by AMMOS is derived from its evolution over 16 years of use by multiple missions. Advantages already discussed include providing substantial functionality at the beginning of new projects and significant cost advantages. Other advantages include the ability to plan new missions much more quickly and accurately due to a well-defined set of mission services and much greater predictability of GDS development and testing costs. This planning capability is enhanced by a valuable set of standard products and templates for required mission documents. Some of the many available are: • Software Management Plan • Requirements documents • Test documents • GDS Architecture • Security plans • Training documents • KSC support plans • Standard interfaces Mature product lines tend to provide high reliability (AMMOS has been operationally tested by 20+ missions) and shorter GDS development times. Lessons learned from past missions are automatically incorporated in future missions. Because AMMOS is a multi-mission system, it allows significant savings through shared operations and development teams. Examples in JPL projects include shared flight control teams, sequence integration teams, GDS system engineering and test engineering teams. All of these advantages provide greatly reduced GDS risk (schedule and cost) and allow project personnel to focus on mission objectives rather than GDS development. These advantages are summarized in Table 4-2. CONSTELLATION PROGRAM SYSTEMS ENGINEERING CONSTELLA NEERING Title: Strategy for Software Reuse Study Final Report Document No.: AC1L1-52 Effective Date: October 1, 2005 Revision: 0 Page 33 These powerful advantages are derived from long-term use but are offset to some degree by challenges that come with maturity, by disadvantages facing early users, and by other system reuse challenges. Table 4-3 summarizes these challenges. Mature Product Line Concerns One of the challenges facing AMMOS is the erosion in quality of code over time. Mature systems (such as AMMOS) require maintenance and new investment if they are to continue as an efficient source of inherited software. For example, much of the AMMOS code was written fifteen years ago in the “C” programming language. Current designers might prefer C++ or other object-oriented language. Although there have been significant efforts aimed at continuous improvement in functionality and structure, tight program budgets have restricted the amount that could be spent to upgrade existing code. An approach of “if it’s working, don’t fix it” will guarantee obsolescence of the product line. Table 4-2. Mature System Reuse Advantages Substantial functionality available for new missions Well-defined set of mission services Prior investment in capability development provides significant value High leverage for current missions Reduced planning uncertainty Better predictability of GDS costs (both development and operations) Shorter development cycle High reliability (reduced risk) Years of testing by multiple (20+) missions Testing (including MOS V&V) costs are lower and more predictable Valuable set of standard products and templates Lessons learned from past missions automatically incorporated in future missions True multi-mission operation dramatically reduces flight team costs Innovation cost is spread across multiple missions Well-documented, well-understood values and benefits Allows focus on mission objectives rather than GDS development A related problem with mature product lines is that users who arrive late in the lifecycle sometimes assume the software product is “free.” Budget and schedule are still required for adaptation, maintenance, testing, and operations. Programs employing a product line approach must still budget for these items. AMMOS is developing future plans to alleviate these concerns and enhance the mature product line benefits described above (see Subsection 4.2.1.6). CONSTELLATION PROGRAM SYSTEMS ENGINEERING CONSTELLA NEERING Title: Strategy for Software Reuse Study Final Report Document No.: AC1L1-52 Effective Date: October 1, 2005 Revision: 0 Page 34 Early User Considerations Many of the benefits described above for mature product lines were available to a lesser extent or simply not available to early users of AMMOS. To avoid early users paying a disproportionate share of the cost, AMMOS was planned with substantial programmatic investment (i.e., early development) up-front. AMMOS, in effect, delivered to the early users. In addition, adaptation expenses were paid for the first five missions. This allowed AMMOS to keep the focus on a multi-mission product line rather than being dominated by the needs of the first users. In spite of these efforts, functionality available to early users (especially the first user) was limited and precise capabilities were unknown or even unplanned at the time project planning was taking place. This parallel planning was a tax on both projects and AMMOS developers. Cost risk was not well understood. Some users did not need all of the capabilities of the multimission system; these extra features may have actually increased project risk. For all of the above reasons, some early project users of AMMOS felt they could have developed a custom GDS that better met their needs. Table 4-3. Challenges in System Level Reuse Mature System Concerns The core software has a lifecycle; it will evolve and erode over time. Core assets require maintenance and new investment to avoid obsolescence. SW is not “free” to a user; it requires adaptation, maintenance, and testing. Early User Considerations Solid programmatic planning and significant up-front investment is required. Reusable software should be delivered to not developed by the first user. No one wants to be the first user. Early users may pay a disproportionate share of the cost. Functionality available to early users may be limited. Parallel planning (projects and product line developers) may be difficult. •Unknown capability requirements Multi-mission features not needed by the project may increase project risk. Standardization Projects are adapting a standard GDS—it is not optimized for specific mission. Standardized spacecraft make project adaptation easier. Missions with special requirements might do better with a custom system. Configuration Management CM is difficult when operating multiple missions in parallel with product upgrades. •Multiple versions of software are under tight project CM. •Encouraging upgrades to a project-CM controlled system can be difficult. CONSTELLATION PROGRAM SYSTEMS ENGINEERING CONSTELLA NEERING Title: Strategy for Software Reuse Study Final Report •Each Document No.: AC1L1-52 Effective Date: October 1, 2005 Revision: 0 Page 35 mission becomes a separate upgrade, handled independently. Organizational Challenges Organizational incentives and disincentives for product use must be managed. •Program plans must be specifically designed to support the product line. A well thought-out program with solid programmatic planning and significant up-front investment alleviated many early-user concerns. However, even with an excellent program plan some issues remained. The big payoff in product lines is long term. 4.02.1.5 Other Challenges of a Product Line Approach Other issues have created challenges for AMMOS users and developers. These issues will be important to other systems where adaptation to individual projects is required. Standardization. Projects adapting AMMOS must use a standard GDS (albeit tailored or adapted). Hence, it is not optimized for a specific mission. Missions with special or lesser requirements might do better with a custom-designed system. For example, Earth orbital missions with limited Command requirements (e.g., TOPEX, Jason-1 and -2) have found it desirable to design a custom GDS to meet their needs. In general, more standardized spacecraft make adaptation of AMMOS easier. Configuration Management (CM). Mission adaptation can cause a CM nightmare as upgrades are developed in parallel with multiple ongoing missions. Voyager, Ulysses, Galileo, and Magellan each had different versions of AMMOS under tight project CM while the AMMOS developers were trying to upgrade to the next version of the multi-mission system. Encouraging projects to upgrade under these circumstances can be difficult. As a result, developers had to maintain four different CM paths rather than concentrating on multi-mission development. In recent years, projects have tended to see the value of upgrading to the latest version of the multi-mission system—the exception being projects nearing end-of-mission or facing severe budgetary constraints. 4.02.1.6 Future Vision In order to take full advantage of the benefits described above, alleviate some of the disadvantages, and achieve the operations cost reductions implied by even optimistic budget scenarios, new adaptation paradigms are being developed. The vision for the AMMOS of the future6 includes the following elements: Plug and play – Projects will be able to select the AMMOS elements that meet their needs and plug them into a basic GDS with relatively little adaptation. Project choice – Projects will be free to choose other suppliers or commercial products for some or all of the system elements. 6 Unpublished presentation, “AMMOS Roadmap,” 2005. CONSTELLATION PROGRAM SYSTEMS ENGINEERING CONSTELLA NEERING Title: Strategy for Software Reuse Study Final Report Document No.: AC1L1-52 Effective Date: October 1, 2005 Revision: 0 Page 36 Fair market paradigm – Project choice will provide a fair estimate of market value. Full cost recovery – Projects will pay for use of system elements. Innovation – Major investments will be paid by the AMMOS program with particular attention to ROI. Thus, innovation cost will be spread across multiple missions. These new paradigms should significantly enhance the value for AMMOS users. Project choice combined with a market paradigm not only provides discipline for developers, it encourages projects to control their appetites for new capability. Plug and play not only allows projects to efficiently build the system they want, it enhances reusable design and allows modules to be continuously improved for the benefit of future users. These enhancements will, of course, require continuing investment in the AMMOS core assets. 4.02.1.7 AMMOS Conclusions Although early project users of AMMOS may not have achieved the full cost and risk reduction benefits of the product line approach, long-term results have been dramatic. AMMOS has provided significant advantages for many years: lower development and operations cost, shorter development cycle, robust GDS, and reduced risk (Table 4-2). It has also demonstrated some significant factors that should be carefully considered in setting up the Constellation Systems Program (Table 4-3). Solid programmatic planning and significant up-front investment is needed to alleviate the concerns for early users. Organizational incentives and disincentives for product line use must be managed and program plans must be specifically designed to support the product line approach. 4.02.2 ROSE—Flight Analysis and Design Reengineering Case Study 4.02.2.1 Project Description The ROSE (Reusable Object Software Engineering) Project was aimed at reengineering a large legacy system—the Flight Analysis and Design System (FADS) for the Space Shuttle program. ROSE was planned as an economical and effective approach to reengineering and maintaining Shuttle Flight Design and Dynamics (FDD) systems. The FDD software was developed over many years, and has been continuously evolving. Because the software was developed under a variety of methodologies and languages, it was difficult to use and expensive to maintain. The Flight Analysis and Design System (FADS) development was originally targeted to address the software in addition to upgrading the hardware platform. FADS is composed of 2.5 MLSOC of mostly FORTRAN that has undergone several “minimum impact” changes in a variety of methodologies and languages and several programs with overlapping or redundant functionality. For a variety of reasons most of the software was rehosted to the new platform with minimal change. Therefore, the problems associated with operations and maintenance of the software persisted. The ROSE Project intended to bring the FDD software into a consistent architecture and design framework. To accomplish this, the existing functionality was to be reengineered into a modern architecture based on Object Oriented Software Engineering (OOSE) techniques, CONSTELLATION PROGRAM SYSTEMS ENGINEERING CONSTELLA NEERING Title: Strategy for Software Reuse Study Final Report Document No.: AC1L1-52 Effective Date: October 1, 2005 Revision: 0 Page 37 implemented in a language that supports those techniques, and accomplished through a repeatable, defined process. 4.02.2.2 Project Timeframe • 1/1/93 — Pilot start • 8/31/93 — Pilot review, “go” given for secondary pilot • 4/5/94 — Pilot review, “go” for project • 9/9/94 — NASA merges ROSE with MOC rehost effort, team size doubles • 3/6/95 — Delivery Increment-1 (Navigation) delivered • 6/95 • 7/31/95 — Project retargeted at TOTM–DI-2 (DOPS/LANDOPTS application) still on track • 3/12/96 — ROSE-TOTM cancelled 4.02.2.3 — Massive re-planning of Trajectory-Off-the-MOC (TOTM) Process A product line approach employing a dual lifecycle was selected. The dual lifecycle approach separates the activities involved in software development and management into two distinct groups within the organization: those concerned with the domain as a whole (product line), and those concerned with particular applications (products) within the domain. The domain and application engineering activities are distinct, yet related endeavors, which proceed in parallel (see Figure 4-2). Figure 4-2. Dual Lifecycle Approach The choice of a product line approach was driven strongly by a large amount of redundancy both within FADS itself and with other closely related systems within NASA/JSC. The additional CONSTELLATION PROGRAM SYSTEMS ENGINEERING CONSTELLA NEERING Title: Strategy for Software Reuse Study Final Report Document No.: AC1L1-52 Effective Date: October 1, 2005 Revision: 0 Page 38 effort expended in generating a generic solution (domain engineering) rather than a point solution for FADS, was perceived as beneficial, and was key in the decision to expand the project scope to include the Mission Operations Center Trajectory System. The process was based on an iterative approach, where iterations were planned and executed in about 6 weeks. 4.02.2.4 Organization ROSE began as a six-month, 20-person initial pilot to demonstrate proof-of-concept for the approach. The sixth-month review resulted in the expansion of the team to 40 people and extension of the pilot to a full year. This year-long activity generated enough information to assess probable success of the full project, and in the year-end review, the team was given the entire flight analysis and design system as its scope, with a set of phased delivery increments to successively replace major FADS subsystems. The team was then expanded to 60 people. Once the Trajectory part of the Mission Operations Center was included in the project, the team was increased again to a total of 80 developers. The team was split into multiple groups, each responsible for a different area. Areas of focus included: reverse engineering techniques, tool customization, process definition, domain engineering, application engineering, etc. 4.02.2.5 Training Extensive training was provided to all the team members. Since the team had to go through multiple learning curves in parallel (OO, reuse, reengineering, toolset, etc.) the training covered all these aspects. Formal training classes were augmented with mentoring sessions. 4.02.2.6 Technical The ROSE team members were all familiar with the Flight Design domain. They were all software developers involved in the use and maintenance of the legacy software. The team decided to use C++ for the project. The project used the OMT method (Rumbaugh et al., 1991) as the Object-Oriented Analysis and Design method. The tool chosen to support the modeling was Paradigm Plus. 4.02.2.7 Lessons Learned The ROSE case provides several keys to successful use of the product line approach (Table 4-4) and also reveals several challenges that can derail that success (Table 4-5): Table 4-4. Product Line Successes The product line approach was viewed as very beneficial by the ROSE team. Training was key—both formal training classes and informal mentoring program. The pilot project was important to debug the process and start to build the core team. A strong architecture team is key to the success of Domain Engineering. CONSTELLATION PROGRAM SYSTEMS ENGINEERING CONSTELLA NEERING Title: Strategy for Software Reuse Study Final Report Document No.: AC1L1-52 Effective Date: October 1, 2005 Revision: 0 Page 39 Table 4-5. Product Line Challenges The product line approach must be supported by the entire organization. • ROSE faced too many conflicting objectives. The customer did not understand the importance of developing the core assets (domain engineering) and wanted the focus to be on application production. There were too many learning curves at the same time (process, reverse engineering, OO paradigm, iterative/incremental lifecycle, C++ language, toolset, etc.). When a real-time system was inserted in the ROSE project, the design constraints were significantly different from the ones relevant to Flight Design. The ROSE Project was terminated due to schedule pressure generated by a rapidly expanding scope (inadequate product line scope control). 4.02.3 Checkout and Launch Control System (CLCS) Case Study Software reuse can be attempted at multiple levels and through multiple phases of the software lifecycle. This case study looks at several approaches attempted during the CLCS project. The CLCS project ran from 1998 through August 2002 when it was cancelled. The project deployed a new control room at the Hypergolic Maintenance Facility (HMF) and was approximately one year away from deploying for Space Shuttle Orbiter checkout in the Orbiter Processing Facility (OPF). 4.02.3.1 Impact of Cultural Differences on Software Product Specification Application software development was organized around vehicle subsystems as this was the organization of the legacy system and represented how the operations community was organized. One area of reuse was in the software that represents valves. The fluid systems on the orbiter and their associated Ground Support Equipment (GSE) utilize many types of electrically actuated pneumatic valves. These valves range from simple valves with one command and no position indicators to complex valves with multiple commands and multiple position indicators. Since the same physical valve is used within multiple systems, this appeared to be an obvious area of reuse potential. Both the Main Propulsion System (MPS) and the Orbital Maneuvering System (OMS) use the same valve for GSE, which is Normally Closed (NC), and has a single OPEN command and a single OPEN position indicator, a micro switch that closes when the valve physically reaches the OPEN position. During the domain analysis and requirements phase, it was discovered that the two engineering groups supporting the MPS and OMS subsystems held diametrically opposed philosophies concerning how to handle off-nominal cases in determining the state of the valve. Both parties were in agreement that if the command were OPEN and the valve indicated OPEN, then the state was OPEN. Likewise if the command were CLOSED and the valve indicated CLOSED, then the state was CLOSED. However, if the command were OPEN and the valve indicated CLOSED, disagreement ensued. The MPS engineers would call this state CLOSED-ERROR, because the valve indicated CLOSED so the problem most likely was that the command was not reaching the valve. The OMS engineers would call this state OPEN-ERROR because indicators have failed in the past (they’re only micro switches) and since it is commanded OPEN it is most likely OPEN and the problem is in the indication. CONSTELLATION PROGRAM SYSTEMS ENGINEERING CONSTELLA NEERING Title: Strategy for Software Reuse Study Final Report Document No.: AC1L1-52 Effective Date: October 1, 2005 Revision: 0 Page 40 After much discussion, the groups were unable to reach an agreement on the determination of the state of the same physical valve. There were several other operational differences in how the valve was used by the system engineers including: timing constraints, required sample rates for position indicators, and the test and checkout philosophy that led to the realization that multiple software representations were required of the same physical piece of hardware. The lesson learned here is that differences in operational use and philosophy must be addressed prior to software product specification. 4.02.3.2 Subsystem Status via Service Provider Model The existing legacy system was a federated architecture where each of the subsystems, or predefined groups of subsystems, were allocated their own CPU and developed monolithic applications that minimized interactions with other subsystems. One observation early in the project was that 80% of the existing systems had duplicated code in their own applications to determine if power was available, if the avionics system was in the correct configuration, and other integrated configuration information (such as the location of the orbiter or which launch pad the orbiter was on). The determination was made for the project that configuration, health, and status information would be provided by the subsystem owner and published or made available via remote access to anyone who needs the information. In this way the code can be maintained in one place. However, this had an unforeseen consequence in the testing and validation of the individual subsystem’s software. Previously, subsystems (such as MPS) could test and validate their software independent of other subsystem software. With this new approach many dependencies were created between subsystems. While schedule issues could be worked around through manually setting published values or building test proxy software that emulated the data provider, there was the problem of trust. Engineers working on Ground Launch Sequencer (GLS) believed they test their software to the best of their ability, and do not believe that other subsystems test their software to the same level. This is not isolated to GLS as there are other trust issues that exist between various subsystems; some even believe their valves are OPEN even when their indicators tell them they are CLOSED. There was widespread concern that individual subsystem application software functionality and safety were being put at risk by relinquishing control of these functions to the owning organization. One of the fundamental operations philosophies is “test your software like you will use your software.” However, there are ramifications that involve dependencies on other software or subsystems. For example, which systems application software is running, and where that software is running for specific integrated testing purposes. Dependencies determine when those tests can be performed and increases the level of support needed when running those integrated tests. The lesson learned here is that reuse through a service provider model creates dependencies in testing and integration that, in some circumstances, may outweigh the intended cost savings. CONSTELLATION PROGRAM SYSTEMS ENGINEERING CONSTELLA NEERING Title: Strategy for Software Reuse Study Final Report 4.02.3.3 Document No.: AC1L1-52 Effective Date: October 1, 2005 Revision: 0 Page 41 Framework Encourages Reuse Within the software engineering domain some level of software reuse was enforced through an application framework based on extending the ACE framework. Through use of the framework, common methods of application initialization, messaging, debug tracing, reading data points, and sending commands were standardized through a set of application programming interfaces (API). These framework APIs provided higher-level capabilities that met the needs of 80% of the various application subsystem domains. In addition to the standard APIs, the framework also provided a consistent concurrency model that simplified the complexities associated with concurrent programming into a simple threading model. Some of the capabilities provided by the framework involved utilizing C++ templates to take the various system-provided primitives for command issuance, derived from the legacy system (APPLY for analogs, SET for discretes, ISSUE for digital patterns), into a single SET interface that was used for all command types. Another higher-level capability was provided through an analysis of the domain space; most often, when a measurement was read it was compared to a limit and verified to be within limits. The framework provided a common VERIFY method that performed the data read and then compared the value to a set of limits. Standardized messaging was provided through the framework SEND_MSG API, with the option of outputting a standard message when a measurement was out-of-limits as part of the VERIFY API. In addition, the framework also provided the core capability to perform unit-level debug and testing. Providing a common mechanism enabled successfully promotion of unit-level testing prior to application-level integrated testing. The lesson learned here is that by utilizing standard capabilities through the framework API and framework-provided services and capabilities, we were able to achieve high levels of application software reuse. 4.02.3.4 Configuration Complexities of Software Reuse Configuration complexities are often overlooked in software reuse applications. For example, an important consideration should be: which version of the reusable software component is running and where it is being run. This complexity may be minimized in an operational environment where the specific configuration is known and documented. However, in development and validation, this can lead to increased debug and testing times and increase the risks that software, which already passed testing, could fail in the operational environment. The details of this are development environment-specific. In the CLCS case development was on UNIX (SUN Solaris to be specific) and deployment on Solaris as well as VxWorks operating systems. Many of the reusable software components were stored in reusable, shared library files. The specific applications have environment variables of which values pointed to the specific shared library to use. Management and configuration of the environment variables and the paths that are searched to find the specified library files can lead to extended debug time, as the shared library (that the program is using) may not be the library that one might expect. This is especially true if the libraries are shared objects where an CONSTELLATION PROGRAM SYSTEMS ENGINEERING CONSTELLA NEERING Title: Strategy for Software Reuse Study Final Report Document No.: AC1L1-52 Effective Date: October 1, 2005 Revision: 0 Page 42 application that was previously running may have loaded a different version of the library than that one needed. Hence, a subsequent application uses the earlier library version since it is already in memory. The lesson here is that reuse can lead to increased configuration complexity and that configuration control and versioning is especially important as the dependencies increase. 4.02.4 Summary of SEEDS Formulation Team Final Recommendation Report 4.02.4.1 Introduction The purpose of SEEDS (Strategic Evolution of Earth Science Enterprise Data System) is to (1) establish a framework for distributed data management to maximize availability and utility of Earth Science Enterprise (ESE) products; (2) leverage community expertise, ideas, and capabilities; and (3) improve overall effectiveness of ESE-funded systems and services. The study was lead by Gail McConaughy, NASA GSFC, to determine if software reuse with a reference architecture can reduce the cost and improve the delivery of information services needed by future NASA Earth Science Enterprise missions. Team members carried out this study (SEEDS, 2003) by gathering, assembling, analyzing, and interacting with community experts. This summary focuses on the reuse aspects of the SEEDS final report. 4.02.4.2 Definitions SEEDS defined reuse as “the act of taking a functional capability used in or provided by one system or mission and employing in another system or mission.” Reference architecture was defined as “a generic architecture that provides coherent design principles for use in a particular domain. It can have various levels of specificity and granularity.” The SEEDS’ team identified the following key software reuse and reference architecture approaches: • Clone & Own: Copying code and associated artifacts of one system for use in another system, where they may be independently modified and maintained. • Improved Clone & Own: Extending Clone & Own practices to enable developers to identify existing assets, subsequently copy those assets, and modify and integrate them more easily as needed for use in new systems. • Open Source: Engaging developers across missions to collaboratively develop and update selected components or systems stored in a central repository. • Service Encapsulation: Wrapping a complete, operational capability of a system with a network-accessible interface so that the capability can be used by others in-place, rather than re-implemented. • Product Lines: Reusing a set of core software components intentionally designed for a family of systems; the components are modified and maintained only by the organization responsible for the core components. • Notional Reference Architecture: Architecture that defines the subsystems or subcomponents of a system and allocates requirements/functionality to each. CONSTELLATION PROGRAM SYSTEMS ENGINEERING CONSTELLA NEERING Title: Strategy for Software Reuse Study Final Report Document No.: AC1L1-52 Effective Date: October 1, 2005 Revision: 0 Page 43 • Specific Reference Architecture: Architecture that defines the services (including all parameters) and service invocation mechanisms of each component of a system in precise enough terms to build interfaces. • Coarse-Grained Reference Architecture: Architecture that defines external interfaces to major subsystems only. • Fine-Grained Reference Architecture: Architecture that defines internal interfaces within applications or functional components. SEEDS grouped their user community into two categories: 1. Those involved in mission critical activities driven by launch schedule and the need for highly reliable production, distribution, and user services. 2. Those involved in mission success activities driven by need for more innovation in research and science applications. 4.02.4.3 Summary of Findings and Recommendation of the Study • The mission-critical community is strongly in support of an Improved Clone & Own approach for mission-critical development efforts. • The mission success community favors the Open Source and Service Encapsulation approach for mission success development effort. • Establish a body such as reuse integration office wherein responsibilities include: – Prioritize & approve reuse initiatives. – Select & guide reuse projects. – Develop, institute, and administer reuse incentives. – Conduct some reuse outreach and education activities. – A small technical team to support all reuse-related activities and evaluate cost saving and impacts. – Make policy changes where there are impediments to reuse—particularly policies related to intellectual property rights. • Competitively select, evaluate, and fund community reuse implementation projects. • Develop reference architecture as part of the support to encourage reusability. • Significant opportunity exists in NASA to realize additional benefits from software reuse. Current impediments are lack of support mechanism for reuse, schedule, and quality concerns. • Guiding principles: – Encourage reusability to be built into original code. CONSTELLATION PROGRAM SYSTEMS ENGINEERING CONSTELLA NEERING Title: Strategy for Software Reuse Study Final Report Document No.: AC1L1-52 Effective Date: October 1, 2005 Revision: 0 Page 44 – Create an Open Source authoring environment. – Identify functional areas appropriate for reuse, and focus reuse efforts on those areas. – Training and knowledge accompanies a software asset. – Define a peer-review process for selecting components for Open Source. • Technology strategies for reuse: – Establish a reusable component library. – Establish a testbed to help identify and qualify reusable components. – Incorporate reuse in NASA development standards. – Develop a software experience library with links to experts, assets, and other resources. – Document architecture/design patterns to enable design reuse. – Focus on reusable tools. – Focus on general software that has broad opportunity for reuse by the community. – Use formal methods and application generators to assemble systems from existing assets. • Program strategies to encourage reuse: – Establish teams to mine assets. – Have authoring organizations provide reuse support. – Fund authoring organizations to package, document, and advertise the software assets. – Provide institutional support by soliciting software from the community, polling the community for validation of components, designing testbeds for testing components. – Provide additional funding to cover overhead costs associated with making code reusable. – Fund activities that support/enable software reuse. – Establish/revise policies related to reuse. – Fund actual software reuse efforts/activities primarily concerned with taking and employing reusable assets in an operational environment and documenting the results. Currently there is a group at GSFC supporting the development and promotion of reusable software applicable to the Earth Science Data System. Project SHAirED (Services for Helping the Air-quality community use ESE Data) CONSTELLATION PROGRAM SYSTEMS ENGINEERING CONSTELLA NEERING Title: Strategy for Software Reuse Study Final Report Document No.: AC1L1-52 Effective Date: October 1, 2005 Revision: 0 Page 45 aims to extend Web infrastructure development for sharing distributed Earth science data and creating tools for data processing and analysis. The project will seek existing data processing components that can assist in attaining its technology development objectives. 4.02.5 GSFC Ground System Reuse Case Study The journal article by Selby (2005) analyzed productivity and quality of ground system products from a NASA software development environment at GSFC that actively reuses software (average reuse is 32% per project). The study looked at module design and implementation factors for 25 projects for ground support software for unmanned spacecraft control. The projects were developed from the 1970s to the 1990s, and were from 3KLOC to 112KLOC of FORTRAN code. In that environment, the approach to implement a new software development was to first select one of their reference architectures to meet the needs of the system requirements. They then populated the architecture by using their repositories to determine which existing modules could be reused or modified. For new capabilities, they created new modules and ensured that they were designed for reusability in future projects. Thus, the development environment benefits from the maturity of the domain and the multi-project planning horizon of project managers and architects. The study found that average module development effort decreased with more reuse. Modules used with major revision were the largest; those completely reused, the smallest. Modules used without revision had the fewest faults and low fault correction effort; newly developed modules or those with major revision had the most faults and the most fault isolation effort. Modules used without revision were small, well-documented modules with simple interfaces and little I/O. The article concludes, “integrating flexible combinations of architectures and components can yield sustainable, multi-project benefits in well-defined problem domains … in terms of productivity and quality.” 4.03 Non-NASA Case Studies 4.03.1 Introduction Much of the literature available today on SPL discusses how Software product lines are rapidly emerging as a viable and important software development paradigm in the private sector. Claims of companies realizing order-of-magnitude improvements in time to market, cost, productivity, quality, and other business drivers are not uncommon. This section examines the non-NASA, not necessarily non-Government, cases and examples of SPL. The main source of comprehensive case examples cited here are from Carnegie-Mellon University’s Software Engineering Institute (SEI) software product line case studies (ND). At SEI the Product Line Systems Program, through the work of the Product Line Practice Initiative, has been the leader in many ideas and pioneered many of the practices that characterize the software product line approach to software development. In the SEI SPL Program, they are working to make software product line practice a systematic, dependable, low-risk, high-benefit practice that combines the necessary business and technical approaches to achieve success. CONSTELLATION PROGRAM SYSTEMS ENGINEERING CONSTELLA NEERING Title: Strategy for Software Reuse Study Final Report Document No.: AC1L1-52 Effective Date: October 1, 2005 Revision: 0 Page 46 Therefore, the study team examined the SEI’s SPL Program case studies because they are well documented and aligned with the effort of promoting sound SPL development and engineering. The cases selected provide a good basis for Constellation to develop a solid SPL approach by examining how other non-NASA organizations implemented SPL. It is useful to examine the technical and management approach, the benefits gained from SPL and also the potential pitfalls and challenges that are presented. By doing this, Constellation can development a meaningful SPL approach with a solid plan for software development and re-use strategy that carefully balances benefit and risks while maximizing success. There are a total of four cases discussed here. The first three cases were chosen to show successful non-NASA implementations of SPL that have similarities to the Constellation Program and therefore can yield beneficial lessons learned that are directly applicable to the program. The last case on NRO’s CCT effort is also a non-NASA case NOT from the Hall Of Fame but has strong correlations to the type of SPL software development effort that can be expected on Constellation These cases selected from the Software Product Line Hall of Fame were chosen due to the selection criteria. The criteria are that members had to serve as models of what a software product line should be, exhibiting a majority of the following characteristics: • The family that constitutes the product line is clearly identified, i.e., there is a way to tell whether or not a software system is a member of the product line, either by applying a known rule or a known enumeration. • The family that constitutes the product line is explicitly defined and designed as a product line, i.e., the commonalities and variabilities that characterize the members of the product line are known and there is an underlying design for the product line that takes advantage of them. • The product line has had a strong influence on others who desire to build and evolve product lines, and has gained recognition as a model of what a product line should be and how it should be built. Others have borrowed, copied, and stolen from it in creating their product lines or in expounding ideas and practices for creating product lines. • The product line has been commercially successful. • Interestingly, the Hall of Fame cases must have sufficient documentation about the product line that one can understand its definition, design, and implementation without resorting solely to hearsay. 4.03.2 Bold Stroke Avionics Software Family, Boeing The Bold Stroke Software Product Line is comprised of a wide range of artifacts required to create Operational Flight Programs for a variety of Boeing military fighters. A well-defined software architecture and carefully designed approaches to handle commonality and variability were crucial to the success of this product line. The architecture is heavily based on and CONSTELLATION PROGRAM SYSTEMS ENGINEERING CONSTELLA NEERING Title: Strategy for Software Reuse Study Final Report Document No.: AC1L1-52 Effective Date: October 1, 2005 Revision: 0 Page 47 expressed via object-oriented patterns. These patterns were leveraged to convey both the architecture and its rationale to a large community of software engineers previously experienced primarily with military standard assembly language systems. CORBA was selected as the communication middleware for Bold Stroke. The product line exploits commercial standards, technologies, and products as much as possible, using an open source CORBA real-time Object Request Broker—the ACE ORB-developed in partnership with Washington University in St. Louis and a commercial real-time operating system. The Bold Stroke Software Product Line is the foundation for an increasing number of production and research programs including several funded by the Air Force Research Laboratory and the Defense Advanced Research Projects Agency. 4.03.3 A-7E Operational Flight Program, U.S. Naval Research Laboratory The A-7E operational flight program (OFP) is the software that assists the pilot of the Navy’s A7E aircraft to operate the airplane. The OFP was redesigned by the Software Cost Reduction project at the Naval Research Laboratory to show how to apply family-based software development principles in the development of a hard real-time system. Commonalities and variability’s were explicitly identified starting in the requirements specification for the family, and were a strong driving factor in the modular design of the OFP. The OFP design, including a modular structure, a process structure, and a uses relation, was explicitly created and documented to be an engineering model that others could follow. It has had a strong influence on the field of both software engineering and of product line engineering. 4.03.4 ShipSystem 2000, CelsiusTech Systems AB ShipSystem 2000 is a family of naval shipboard comment and control systems produced by CelsiusTech Systems AB of Sweden since the late 1980s. Begun in 1985 as a business and technical response to two large contracts awarded simultaneously, ShipSystem 2000 is based upon a robust architecture that was designed to handle both of those initial systems as well as the more than fifty variants that followed. Family members include systems for ships from coastal corvettes to cruisers to submarines, for navies all over the world. These systems comprise 1–1.5 million SLOC of Ada code, are hard-real-time, embedded, and safety-critical. CelsiusTech has been able to slash production time, build more systems with fewer people, and increase quality. The story of ShipSystem 2000 was one of the first and most important case studies in successful software product line engineering. 4.03.5 United States National Reconnaissance Office (NRO) Control Channel Toolkit (CCT) The last non-NASA case presented here is the Control Channel Toolkit (CCT) developed by the United States National Reconnaissance Office (NRO). NRO recognized that much is the same comparing one ground-based spacecraft command and control system to another. The details surrounding the NRO CCT case can be found at the SEI website and so will not be discussed here. However, there are striking similarities between that case and the challenges we face at NASA and specifically on Constellation. CONSTELLATION PROGRAM SYSTEMS ENGINEERING CONSTELLA NEERING Title: Strategy for Software Reuse Study Final Report Document No.: AC1L1-52 Effective Date: October 1, 2005 Revision: 0 Page 48 To reiterate, a software product line is a set of software-intensive systems sharing a common, managed set of features that satisfy the specific needs of a particular market segment or mission and that are developed from a common set of core assets in a prescribed way (Clements and Northrop, 2001). As it turns out, NRO made some good decisions by recognizing that there is great commonality in their delivered systems and that they too could benefit from the advantages of applying SPL thereby taking advantage of the commonality to build a product line asset base for their ground-based spacecraft command and control software. The first delivered system in their product line showed overall reductions in cost of 50% with a greatly reduced workforce. In addition, software defects went down significantly. 4.03.5.1 NRO CCT and Similarity to Constellation NRO set out to develop CCT which is a software asset base, funded and developed by one organization and built under contract with Raytheon Company as an industry partner. This software supplier contractor arrangement and the interaction between the two organizations are also common in NASA. Many spaceflight missions are, for example, are managed by one NASA center, while the spacecraft is built by an industry partner such as Lockheed Martin, Ball Aerospace, etc. NRO did not develop the entire system from architecture through final products; rather, they provided overall core assets containing an architecture, requirements and development processes and then disseminated the product development to other teams. On Constellation, the complexity and challenges are even greater than this. Here we have many NASA centers and several contractors that need to work together in developing perhaps the largest and most complex software system since the Apollo program. 4.03.5.2 Managing Several Teams with SPL Therefore, much like NRO, it is conceivable that Constellation will not develop a complete software system or complete SPL. Rather, Constellation will develop an asset base which consists of generalized requirements, domain specifications, a software architecture, a set of reusable software components, test procedures, a development environment definition, and the very important documentation needed for reusing the architecture and components. The Constellation users will build products. The users described here are the individual teams and government contractors under direction of a NASA Center to build spacecraft flight and ground software assets. As NRO did, Constellation may decide to have product development and the core asset development divided among potentially different teams/contractor organizations, where users (team/contractors) share the management responsibilities. A core asset is a software artifact that is used in the production of more than one product in a software product line. A core asset may be a software component, a process model, a plan, a document, or any other useful result of building a system. One of the most important core assets is the software architecture for the product line. The scope of the core asset differs from traditional reuse in that the scope must span the needs of the entire set of products, and provide a context in which other assets, such as code and test artifacts are related. The core asset must also be developed with the right flexibility to satisfy the products in the product line. The scope of the core asset needs to clearly state what CONSTELLATION PROGRAM SYSTEMS ENGINEERING CONSTELLA NEERING Title: Strategy for Software Reuse Study Final Report Document No.: AC1L1-52 Effective Date: October 1, 2005 Revision: 0 Page 49 products the core assets are intended to support. The scope defines the commonality between the various products and the ways in which they differ. For Constellation, the SPL will represent a significant departure from traditional attempts of software reuse schemes in which well intended architects have made assets as general as possible (domain light) without the context provided by an architecture and a scope definition. These approaches typically designate some repository of components or frameworks that are too general, lack domain depth, have insufficient descriptions, lack good documentation and require way too much adaptation to claim valuable reuse. 4.03.5.3 Need for an Early Architecture Product lines pay off because the investment in the core assets is allocated across the entire family of products, and as NRO has discovered, it’s good to have an architecture (a core asset) early in the lifecycle in order to provide a clear production capability that allows rapid construction of high-quality products. Lessons learned from NRO indicate that perhaps Constellation should create an Architecture Group early on. This group will advocate the creation of an Architecture Model for Constellation to clearly communicate Constellation architectural concepts and decisions to Constellation users who would build future products. Learning from NRO’s CCT case where it was important to communicate architectural features early on to the builders it will be even more paramount on Constellation with the myriad of teams and contractors. 4.03.5.4 Technical and Cultural Challenges Using SPL for Constellation A major challenge for the Constellation team is two-fold. There is the newer technical side of SPLs where a well thought out architecture and reuse strategy is needed, along with the numerous tools, methods and processes to ensure successful development and delivery. But also, there is the cultural paradigm change that will need to occur which includes leadership and solid management, careful planning, a clear view of the approach, training, collaborating and obtaining the necessary buy-in to bring together many software teams from various NASA centers. Skilled organizational management is needed to adopt software product line engineering, achieve the correct organizational structure, and establish an effective customer interface. These and other practice areas are described in the SEI SPL Framework for Software Product Line Practice. The challenge of NRO to develop new systems with shrinking budgets was probably the original trigger for turning to software product lines as a cost-saving system acquisition strategy. NASA, and Constellation as a project within NASA, must operate in a new era of greater technical and fiscal responsibility as well. Projects are constrained more and more today by cost caps in which projects must not exceed them without risk of major descoping or possibly cancellation. 4.03.5.5 Constellation Scope and Complexity However, unlike NRO, Constellation faces the issue of greater scope and a plethora of partners. This may be the single greatest reason for the SPL approach. Constellation will set out early to define “what” kind of products will be needed to reach programmatic needs. From here, solid CONSTELLATION PROGRAM SYSTEMS ENGINEERING CONSTELLA NEERING Title: Strategy for Software Reuse Study Final Report Document No.: AC1L1-52 Effective Date: October 1, 2005 Revision: 0 Page 50 requirements and an architecture developed early by the Constellation Architecture Group will allow all teams to better understand the core assets needed to further the design of their systems to develop products. Learning from the NRO case, it is recommended that a team be formed early to explore architecture definition, process definition, operations and maintenance definition, cost analysis, schedule assessment, and risk assessment. 4.03.5.6 Leadership in Management within SPL Environment For NRO’s CCT effort, it has been noted that successful management of a product line effort requires sophisticated management skills and well-honed leadership qualities. While some of the leadership was attributed to strong personalities with great drive to succeed, it is recommended that a well-conceived training program be developed to ensure all Constellation teams receive excellent training in Software Product Lines (SPLs) and project leadership courses. CCT was completed on schedule and within budget and with no outstanding risks. There are only a few organizations developing software these days that can make such a claim and represents a significant change in the software success arena. NRO originally set out pursuing a product line approach to reduce development costs, increased quality, improve the evolution of software for future use, decrease time-to-field, and obtain savings in sustainment costs, all of which were achieved. 4.03.5.7 Lessons Learned on NRO CCT Project Constellation can greatly benefit from NRO’s CCT effort through the Software Object Technology Group, a joint government and industry group led by NASA and the NRO, which already has applied the CCT architectural concepts to the definition of an object/interface standard for space-related applications. 4.04 Discussion and Analysis This subsection summarizes the lessons learned from the case studies with special emphasis on the software product line approach and challenges in implementing this approach; it concludes with an overview of potential solutions for these challenges. 4.04.1 Overview of NASA Case Lessons The AMMOS study shows there can be substantial long-term benefits to system-level reuse. These benefits are far more than cost savings—better planning, shorter development cycle, and high reliability (see Table 4-2). To achieve these benefits, however, Constellation must overcome the challenges associated with managing a software product line: mature system issues, early user considerations, standardized products, early investment, and configuration management. Solid programmatic planning and significant up-front investment is needed to alleviate the concerns for early users. Organizational incentives and disincentives must be managed and program plans must be specifically designed to support the product line approach. The ROSE case describes a product-line approach to reengineering the Shuttle Flight Analysis and Design System (FADS). It illustrates the use of Object Oriented Software Engineering CONSTELLATION PROGRAM SYSTEMS ENGINEERING CONSTELLA NEERING Title: Strategy for Software Reuse Study Final Report Document No.: AC1L1-52 Effective Date: October 1, 2005 Revision: 0 Page 51 techniques and employs a dual-lifecycle approach—an approach that allows a strong architectural team to focus on overall product line design while individual implementation teams focus on applications. This case emphasizes the importance of training when employing several new techniques (multiple learning curves) and when teams are operating in parallel. A pilot effort helped to eliminate process bugs. Process discipline is viewed as critical in implementing a product line effort. The ROSE Project emphasizes the importance of all stakeholders understanding and supporting the product line approach. The project was terminated when the scope was expanded too rapidly and in a way that was inconsistent with product line scope. The CLCS study focuses on how different team cultures and organizational interaction can affect reuse strategies. The study provides examples of how differences in terminology between organizational elements can result in software product specification problems. These issues need to be addressed before reuse efforts are undertaken. The CLCS study also provides examples of how using common service-provider software can complicate testing and communication across diverse groups. In addition, the study shows how frameworks can help achieve high levels of software reuse at the application level. The GSFC Ground Systems Study shows that reuse of architectures and components can achieve both productivity and quality improvements in relative proportion to the degree of reuse. It also indicates that these core assets have improved over time (with use) and because of the multiproject planning horizon—two important tenets of the product line approach. The SEEDS report provides well thought-out definitions of reuse and related topics such as reference architecture. The report recommends the establishment of a Reuse Integration Office to manage all reuse activities in the organization. It also encourages the use of reuse support groups —the SHAirED group at Goddard has been established to encourage reuse within the Earth Science Enterprise Data System. In addition to general guidelines for reuse, the SEEDS report provides recommendations for reuse technologies and program strategies. Many of these will be directly applicable to Constellation Systems. One finding is that different approaches to reuse (e.g., Clone and Own vs. Product Line) may be appropriate for different user groups. This conclusion needs to be carefully considered in establishing a software product line. Programmatic support and stakeholder buy-in are vital as evidenced in the ROSE case. 4.04.2 Overview of Non-NASA Case Lessons Each of the three cases selected from the Hall-Of-Fame expose an advantage to using SPL on large software-intensive projects that relate to Constellation. Key in the use of SPL during the planning and development on Constellation should be availability to lessons-learned from other government and non-government involvement with SPL as well access to the materials and experts in the SPL subject area at Carnegie-Mellon University. The Bold Stroke case focuses on the positive aspects of SPL implementation by having a welldefined architecture early on that is heavily based on well-expressed object-oriented patterns that could be communicated to a large development community. CONSTELLATION PROGRAM SYSTEMS ENGINEERING CONSTELLA NEERING Title: Strategy for Software Reuse Study Final Report Document No.: AC1L1-52 Effective Date: October 1, 2005 Revision: 0 Page 52 The A7E Operational Flight Program case illustrates the gains in SPL realized from having a well-defined process and engineering model for a large to team to follow. The ShipSystem 2000 case shows success in using SPL for the development of real-time safetycritical development effort on a very large scale. Finally, the NRO CCT case shows several parallels to Constellation and lessons-learned that feed directly to the Constellation Program. NRO is funded and developed by one organization and built under contract with Raytheon Company as an industry partner. This model can be expanded and applied to Constellation and to management and technical relationships with other NASA Centers and industry partners. NRO had a supplier-contractor arrangement where the interaction between the two organizations was essential to the final products. The same will be true for Constellation. NRO needed to leverage from the commonality of their ground-based spacecraft command and control software and therefore, built a product line asset base for final product development. Constellation faces the same kind of challenge in developing an integrated flight and ground software system. NRO also developed an early architecture as the preliminary core asset and disseminated it to the other teams that could then carry out the development of the final product. Constellation will need to convey early architectural assets to other teams for final product development, test, integration and delivery. Lessons learned from NRO indicate that perhaps Constellation should create an Architecture Group early on to communicate architectural ideas and obtain buy-in by various stakeholders. Constellation has a myriad of partners to share technical information with and can greatly benefit from such a working group. 4.04.3 Summary of Case Study Challenges and Solutions Lessons learned from the NASA and non-NASA case studies that are applicable to the Constellation Systems effort are summarized in Table 4-6. The benefits have been discussed above. Many of the solutions shown in the table are discussed more fully in Sections 6.0 through 8.0. They are highlighted here to emphasize their relationship to the cases and provide motivation for the recommendations that follow. If constellation chooses a product line approach, it is essential to get the buy-in of the entire organization. This requires strong programmatic leadership and a well-defined architecture. Achieving stakeholder buy-in will require appropriate education as well as significant management attention. Core assets are the lifeblood of the product line and Constellation must nurture these resources. A major aspect of this challenge is managing the evolution and erosion of software over time (see Subsection 4.2.1.4). This means continuous investment in core asset software improvement throughout the life of the code. Product lines must be planned with evolution in mind and implemented with appropriate technology (e.g., object-oriented paradigms, frameworks, etc.)— see discussion of technical factors and recommendations in Sections 7.0 and 8.0. CONSTELLATION PROGRAM SYSTEMS ENGINEERING CONSTELLA NEERING Title: Strategy for Software Reuse Study Final Report Document No.: AC1L1-52 Effective Date: October 1, 2005 Revision: 0 Page 53 One of the strongest arguments for a product line approach is the huge scale of Constellation and the need for interoperability and compatibility. The non-NASA cases have demonstrated considerable success in achieving commonality of systems and architectures and in achieving very large scale reuse (see Subsection 4.3). These organizations have also demonstrated the ability to manage complex contracting relationships (note especially the NRO case). A product line that is developed for multi-mission use is, by definition, not optimized for a specific mission or narrow domain. Reuse will be more efficient to the extent that missions and domains adopt the same standards that were used to design the product line. The product line should, of course, be planned to support future missions. The point here is that common standards make both project adaptation and product line design easier. Table 4-6. Case Studies—Lessons Learned* Benefits • • • • • • • • • • • High return from product line approach roach (long-t (long-term) Shorter development cycle for new w missions Reduced development cost for new w missions High reliability/reduced risk through h testing in multiple missions/domains Reduced testing time and cost New missions benefit from the availability vailability of su substantial functionality Lessons learned from prior missions ns are automa automatically incorporated Planning uncertainty is reduced byy predictabil predictability of product costs Allows focus on mission objectivess rather than software development Innovation costs spread across multiple missi missions Product line discipline is valued byy team memb members Challenges Solutions Entire organization support for product line approach Buy-in by stakeholders Support for core assets and domain engineering Strong programmatic leadership Well defined architecture Strong architecture team Education and training: the how of reuse Core software has a lifecycle; it will evolve and erode Continued programmatic investment over time. Product line approach: Core assets require maintenance and new investment Planning for evolution to avoid obsolescence. Dual lifecycle approach Modern reuse technology: Frameworks and design patterns Object-oriented techniques and languages Commonality of systems and architectures Product line approach Very large scale reuse Complex contracting relationships See NRO case for model solutions No one wants to be the first user of new products. Early users’ functionality may be limited. Early users may face increased risk. Standard products Domain differences/team cultures Product line configuration management is crucial. Programmatic planning and up-front investment Product software should be delivered to not developed by early users. Product line planning (incentives and standards) Reuse Integration Office Incentives Standardized requirements across domains Early attention to cultural differences Product line planning Clone and own (for some users) CONSTELLATION PROGRAM SYSTEMS ENGINEERING CONSTELLA NEERING Title: Strategy for Software Reuse Study Final Report Document No.: AC1L1-52 Effective Date: October 1, 2005 Rapidly expanding project scope Mismatch of product and application Too many new processes and learning curves Revision: 0 Page 54 Product line scope control Product line definition Clear definition of product line approach Education and training Pilot projects *It is important to note that most of these lessons are applicable pplicable at multiple reuse levels (system, program, module) and multiple development stages (requirements, design, n, code). Early product users need special attention. Product development should be funded separately from individual missions and planned for delivery to the first user. Use of incentives and standards are vital (for early and later users)—see Sections 7.0 and 8.0. Both the AMMOS and CLCS studies emphasize the importance of configuration management (CM) and illustrate how version control can become complex when core software is under continuous development while products are being used in multiple applications. Configuration control and versioning is especially important as the dependencies increase. To reduce complexity, product line evolution should be planned to release versions at times consistent with mission needs and plans—see product line planning recommendations in the following sections. The system architecture should be developed early and with great attention to detail so that major architectural changes are not required as the system evolves: the architecture should facilitate evolution, not hinder it. Finally, Constellation must pay special attention to product line definition. Several cases have shown the importance of maintaining a clear focus on the defined scope while allowing it to evolve over time to take advantage of new opportunities. Both NASA and non-NASA cases provide key insights and are sources of information for Constellation planners. This concluding section has focused on identifying product line challenges and solutions that were developed from the case studies. The remaining sections of this report are devoted to developing these and other solutions that form the basis of a reuse strategy for Constellation Systems. CONSTELLATION PROGRAM SYSTEMS ENGINEERING CONSTELLA NEERING Title: Strategy for Software Reuse Study Final Report Document No.: AC1L1-52 Effective Date: October 1, 2005 Revision: 0 Page 55 5.0POTENTIAL BENEFITS OF REUSE This section describes benefits that accrue from reuse of software development artifacts. Artifacts that can be reused include plans, cost estimates, schedules, requirements models, specifications, designs, human/machine interfaces, documentation, code, data, test cases, and application generators. 5.01 Discussion Systematic reuse is viewed as a possible means to reduce software development costs while improving software quality. The benefits of software reuse are substantial. For a particular project, the dollar savings can be readily calculated by estimating the cost for development without reuse and subtracting the cost for development with the reuse of existing software components. Because the quality of the software is normally known and is generally higher than that of a newly developed software component, a high level of confidence in the software is achieved through familiarity with it. A list of potential benefits for Constellation include: • Increased productivity • Shorter project schedule • Reduced net costs over time • Increased quality and reliability • Earlier requirements validation • Lower risk due to more accurate estimates • Higher ability to leverage expertise in individual domains The largest savings in reuse are anticipated through leveraged reuse and not opportunistic reuse. In 1992, the Advanced Research Projects Agency (ARPA) projected software costs and savings for the next 16 years. The savings anticipated in reuse-based software development equals the combined savings of process improvement and software development tools. Section 4.0 on NASA case studies documents these benefits in prior missions (see Table 4-2). Section 4.0 also supports the benefits described below. 5.01.1 Increased Reliability and Overall Quality Ideally, a software artifact that is developed for reuse would be verified to be correct, and would contain no defects. In reality, with reuse, defects are found and eliminated, and as a result, an artifact’s reliability is improved. Over time, the artifact can become virtually defect-free (or sometimes it merely reaches an equilibrium with respect to the number of defects as shown in Table 5-1 (Matsumoto, 1995).7 This is especially important for upstream artifacts such as 7 Personal communication, Barry Boehm to a study member regarding an OS/360 example. CONSTELLATION PROGRAM SYSTEMS ENGINEERING CONSTELLA NEERING Title: Strategy for Software Reuse Study Final Report Document No.: AC1L1-52 Effective Date: October 1, 2005 Revision: 0 Page 56 requirements and designs because these errors are more expensive to correct later in the lifecycle than are programming errors. Table 5-1. Reported Benefits From Reuse Organization Nippon Electric Productivity 6.7x Cost Development Time 75 Ada Projects 10x Toshiba 20%–30% fewer defects DEC 1.25x 0.33x–0.2x HP1 1.5x 0.57x HP2 1.4x Raytheon 1.5x 9 companies 0.16x 0.3x 1.2x 0.75x 1.15x–1.5x 0.9x–0.5x US Navy simulator 2x NASA 76% fewer defects 24% fewer defects 312 aerospace projects Japanese industry 20%–35% fewer customer complaints 0.25x AT&T SofTech Quality 2.8x, 5x–10x 0.5x 10x–20x 5.01.2 Reduced Development Time Development time is reduced because it is not necessary to spend the time to develop and unittest the reused artifacts. The reused artifacts incorporate lessons learned and avoid the time for reinvention or rediscovery. 5.01.3 Increased Productivity When reusable artifacts are applied throughout the software process, less time and effort is spent creating the artifacts that are required to create a deliverable system. Development is simplified. Therefore, the same level of functionality is delivered to the customer with less input effort, improving productivity. Rough estimates suggest that 30% to 50% reuse can yield productivity improvements in the 25% to 40% range. Others experienced from 7× to 20× improvement (see Table 5-1). Leverage is obtained by employing commercial and open-source components. The use of application generators can achieve order-of-magnitude productivity gains (Tracz, 1995), and may also have the benefit of enabling domain experts to specify the component, rather than requiring programmers as intermediaries. 5.01.4 Reduced Cost A reduction in cost results from less time and effort being spent developing the software product; also testing costs are lower. An increase in the quality of the software means less expense CONSTELLATION PROGRAM SYSTEMS ENGINEERING CONSTELLA NEERING Title: Strategy for Software Reuse Study Final Report Document No.: AC1L1-52 Effective Date: October 1, 2005 Revision: 0 Page 57 detecting and correcting defects, especially the expensive ones discovered late in the development process. Table 5-2 contains information from military projects, and depicts the return on a $1.00 investment after four years and 24 uses for each of the artifacts noted above (Jones, 1994). Quantitative evidence indicates that the effort it takes to reuse a piece of software is 20% of the cost to develop it for one-time use (Poulin, 1997). Table 5-2. Return on Investment by Reusable Artifact Reusable Artifact Four-Year Return ($) Project plans 2.00 Cost estimates 3.00 Architecture 1.50 Requirements models / specs 3.00 Designs 5.00 Source code 6.00 User/technical documentation 1.50 Human interfaces 1.00 Data 3.50 Test cases 3.50 Reuse can occur at many levels of the software system development process, as discussed earlier. A simple list would include requirements, design, code, and validated code. Jensen (2004) performed an economic analysis of software reuse and demonstrated that results were independent of the particular software cost modeling tool used (e.g., COCOMO). His model provides a simple yet realistic prediction of software product costs in an environment containing reused software artifacts. Although his results assume that a mature reuse organization and a mature body of reusable assets that are readily available, the results provide a useful baseline model. Figure 5-1 shows the relative development cost versus reuse fraction by reuse type. CONSTELLATION PROGRAM SYSTEMS ENGINEERING CONSTELLA NEERING Title: Strategy for Software Reuse Study Final Report Document No.: AC1L1-52 Effective Date: October 1, 2005 Revision: 0 Page 58 Figure 5-1. Relative Development Cost vs. Reuse Fraction by Reuse Type (from Jensen, 2004) A realistic full-lifecycle cost model with no externalized costs is important. Most techniques for increasing reusability, such as modularity, high functional cohesion, low module coupling, and use of standards, are good software engineering techniques and should be used regardless of the reusability issue. The cost required to implement a cataloging scheme for retrieval of software from reuse libraries can be significant. Domain engineering is a new concept and requires training and manpower. Disagreements over ownership rights and liability issues of reused software components can result in costly legal services. Enthusiasm for software reuse can lead to overestimating cost savings and underestimating cost increases, thus biasing the cost-benefit analysis. A list of costs include: • Domain analysis and modeling • Domain architecture development • Certification and quality assurance of reusable components • Increased documentation to facilitate reuse • Maintenance and enhancements of reuse artifacts • Royalties or licensing fees for externally acquired components • Operation of a reuse repository • Resolution of ownership and liability • Additional time to negotiate interfaces suitable to multiple users of the interface • Negotiating component evolution with many users of that component CONSTELLATION PROGRAM SYSTEMS ENGINEERING CONSTELLA NEERING Title: Strategy for Software Reuse Study Final Report • Document No.: AC1L1-52 Effective Date: October 1, 2005 Revision: 0 Page 59 Training of personnel in design and coding for reuse 5.01.5 Reduced Development Process Risk Because known artifacts are being reused, there is less uncertainty in the product, leading to better schedule and cost predictability. The uncertainty in the cost of reusing an artifact is less than that of developing it, especially in the case of large subsystems (Sommerville, 2004). In addition, the capabilities of the system can be planned more accurately (see Table 4-2). Also, the risk of innovation is spread over more applications. 5.01.6 Increased Maintainability Maintenance cost reductions of up to 90% have been reported when reusable code, code templates, and application generators have been used to develop new systems. Maintenance consists of fixing errors and implementing new functionality. The benefits for reliability of reuse have been mentioned above. The ease with which new functionality can be implemented is dependent on how modular the system is relative to the needed changes. To the extent that modifications can be foreseen, the point of domain engineering is to understand the problem domain so that its reusable modular components are identified. At the extreme end of maintenance, product-line engineering enables cost-effective development of an envisioned suite of products. As an example, Constellation ground software will have to support a variety of mission architectures. Investing in domain engineering up front could allow the ground system to endure the evolution of the missions. 5.01.7 Continuous Improvement and Increased Evolvability Reuse of interfaces defined by an abstraction layer enhances continuous improvement and evolvability. Defining a reusable interface insulates applications from their implementation platform, allowing for substitution of the platform implementing the interface, and preventing obsolescence. Examples of major sources of obsolescence are operating systems, COTS products, hardware, and data formats. To ameliorate the problem with operating systems, a platform-independent layer can be specified which provides certain system services. It can be implemented by invoking calls to a particular operating system. If that operating system eventually is no longer viable (because the vendor went out of business, or it doesn’t meet timeliness requirements), another operating system can be substituted by re-implementing the layer using the new operating system.8 For hardware, defining a byte-code interface language (as with Java) allows portability among different hardware. For data formats, formats defined in XML allow for regeneration of the programs that process the data as the requirements of the information stream evolve. 8 This approach was taken for the TOPEX and Jason satellites: a layer was defined that provided services for memory allocation, buffer management, name services, file access control, concurrency control, and inter-process communication. CONSTELLATION PROGRAM SYSTEMS ENGINEERING CONSTELLA NEERING Title: Strategy for Software Reuse Study Final Report Document No.: AC1L1-52 Effective Date: October 1, 2005 Revision: 0 Page 60 Another example is middleware, which insulates applications from each other by interposing an intermediary system, thus allowing the two applications to evolve separately. CORBA is an example of a middleware standard with implementations available from multiple vendors. It has been used in a number of large and mission-critical systems. Evolvability is a critical issue for Constellation because of the program’s intended long life span. Defining appropriate reusable interfaces could ensure evolution within reasonable cost. 5.01.8 More Effective and Efficient Use of Specialists The specialized knowledge needed to construct an artifact is encapsulated in the artifact itself; that knowledge is reused when the artifact is reused, and thus it is not necessary to obtain the labor of the specialist again for the new application. In many cases, especially those cases where reliability or performance are critical, the user of reusable code typically seeks to acquire the developer of that reusable code, and if that specialist is available, productivity and quality will benefit. Class A code generally isn’t as “opaque” as is portrayed here. The real benefit comes incrementally as a decreasing reliance on the original creators (if available) of software when there is a desire to reuse it. 5.01.9 Easier Compliance with Standards To the extent that the artifacts embody standards, or were developed in accordance with standards, the new applications making use of those artifacts inherit those standards. This is applicable to software development process assets that may support a CMMI maturity-level rating, as well as code that satisfies coding standards. This is a very important point. In the case of Application Program Interfaces (API), such standards open the door to broader participation by the software development community in industry and other organizations and may very well help determine new standards. 5.01.10Increased Flexibility There is flexibility as to how a product is developed. In periods of high demand, developers would have a higher level of reuse in their designs (design with reuse), leading to higher productivity. A “design for reuse” philosophy can be applied when demand is low. 5.02 Findings Constellation has the opportunity to realize major benefits from software reuse in a number of areas as discussed in this section. The degree to which these benefits are realized will be a function of the effectiveness of the overall implementation of the technical and non-technical aspects of reuse policies and procedures. With respect to software systems engineering practice, most techniques for increasing reusability —such as modularity, high functional cohesion, low module coupling, and use of standards—are good software engineering techniques and should be used regardless of the reusability issue. Based on the study team’s analysis of NASA and external case histories, a few areas realizing benefit are noteworthy: CONSTELLATION PROGRAM SYSTEMS ENGINEERING CONSTELLA NEERING Title: Strategy for Software Reuse Study Final Report Document No.: AC1L1-52 Effective Date: October 1, 2005 Revision: 0 Page 61 • Productivity: Reduced time to deliver the same functionality: 25% to 40%. • Quality: Reduced defects via reuse of iteratively refined and matured software. • Cost: Reduced development time and improved quality yield return on investment for a variety of reusable products ranging from 100% to 600%. • Flexibility: Software (all artifacts) designed for reuse enable more rapid response to demand and systematic adaptation to a variety of requirements. CONSTELLATION PROGRAM SYSTEMS ENGINEERING CONSTELLA NEERING Title: Strategy for Software Reuse Study Final Report Document No.: AC1L1-52 Effective Date: October 1, 2005 Revision: 0 Page 62 6.0RISKS, OBSTACLES, AND MITIGATIONS 6.01 Introduction There are problems in deciding what should be reusable and at what level of granularity reuse should be applied, but when properly done, then significant gains can be realized (see Appendix G). For reuse to be successful and cost effective, it needs to be applied from a higher level than that of identifying common subroutines. When the project is viewed as an evolving software product line composed of COTS, GOTS, and NDIs, then there are benefits. There are issues about at what level of abstraction the components should exist. When the same component is used more than once and its API changes, then there are ramifications to every instance where it is used. In order for a software reuse strategy to be effective, it means the components that are reused must be fairly static and not undergoing revisions. Management is sometimes reluctant to support a reuse approach because reusable software components usually cost more to develop. This is because rather than implementing the component for a single use, the big picture of how this asset fits in must be considered with possibly broader functionality. Additionally, a unified API needs to be developed and coordinated with all the users that intend to use the asset, requiring more complex testing and support. For software reuse to be truly successful, organizations must make an early commitment and investment in a reuse approach. Sometimes organizations are unenthusiastic because the returns are often not immediately realized or easy to quantify. 6.02 Technical Risks In reusing a component, what assumptions are necessary for the component to function correctly in a new context? Are the assumptions exhaustively known and specified? Can they be checked in the new context? 6.02.1 Software Inheritance Oftentimes projects will try to make explicit reuse of technologies, products, or processes that were developed for previous projects. The decision to do so is a function of determining whether the previous product was “good enough” or “close enough” to meet the new requirements. Unfortunately, more often than not, a real analysis is not done of the suitability of the reuse of existing software products. In the absence of such an analysis, the results are unpredictable and can result in multiple risks to the project, the most significant of which are system failure and cost growth. 6.02.1.1 System Failure A major reliability risk is undocumented or poorly understood dependencies embedded in the legacy software that are peculiar to the application system (e.g., CEV for ISS missions) that would not carry over to a new vehicle (e.g, CEV for lunar missions). Such dependencies may exist at any level in the software system and manifest themselves in multiple ways. CONSTELLATION PROGRAM SYSTEMS ENGINEERING CONSTELLA NEERING Title: Strategy for Software Reuse Study Final Report Document No.: AC1L1-52 Effective Date: October 1, 2005 Revision: 0 Page 63 An extreme example of this kind of risk is the Ariane 501 mishap—where a supposedly noncritical portion of the flight code from the Ariane 4 was reused without adequate analysis for the maiden flight of Ariane 5. A series of poorly documented assumptions dating back a decade led to the loss of Ariane 501 in the first minutes of the ascent phase. Specifically, in order to fit within a computational envelope, exception handlers were removed from parts of the Ariane 4 code. One of these removed exception handlers was for arithmetic overflows related to the conversion from floating point to integer of the “horizontal bias” variable—which measured the horizontal component of velocity up to the first minute after launch. There was a dependency between this exception handler and the thrust of the Ariane first stage. This dependency was forgotten over the course of a decade from the original development of the code for the Ariane 4 to its reuse on the Ariane 5. The more powerful thrust of the Ariane 5 literally caused a floating point to integer conversion overflow that led to loss of the maiden mission (Leveson, 2004). 6.02.1.2 Cost Growth Project cost growth can occur as a result of several technical and non-technical factors associated with the “inheritance” of code. This may occur at any level in the software system, with concomitant magnitude of impact at the software system level. Cost growth risks may be due to • inheritance of code that does not behave as advertised, • inheritance code that is poorly documented, or • inheritance code that required more modification than expected. The cost-growth risk can be mitigated by a number of methods, each of which contributes to a solution but may not be sufficient to completely mitigate the risk. Mitigations include (1) a thorough software inheritance review (see also criteria for such a review, discussed elsewhere in this report), (2) ensuring that engineers who are knowledgeable about the inherited software come with the software or are otherwise available, and (3) ensuring that software to be reused, or “inherited” is designed from the onset for this purpose and the appropriate infrastructure is created to maintain the code in a managed and certified condition. 6.02.2 Legacy Software Unlike software inheritance, where there is an explicit decision to capture and reuse existing software, “legacy software systems” are those that are considered out-of-date or are too specialized to be carried forward into a new design. They are often viewed as a hold-over until a complete system replacement occurs that may completely replace or eliminate the functions performed. However, the existence of legacy software is frequent enough that it should be considered as a special case for reuse. The risks involved are similar to inherited software, but typically plans are made to explicitly replace the legacy software. The risks become very significant when these plans for replacement go awry, and the legacy software now becomes “good enough.” CONSTELLATION PROGRAM SYSTEMS ENGINEERING CONSTELLA NEERING Title: Strategy for Software Reuse Study Final Report Document No.: AC1L1-52 Effective Date: October 1, 2005 Revision: 0 Page 64 6.02.3 Reuse Process Risks Frakes and Fox (1996) identify seven reuse “failure modes,” each corresponding to a step in the process where reuse can fail. These include: (1) reuse was not attempted, (2) part does not exist, (3) part is not available, (4) part is not found, (5) part is not understood, (6) part is not valid; and (7) part is not integratable. The end result of any of these failure modes is a missed reuse opportunity. Failures that occur late in the process of attempting reuse are far more expensive than those occurring early in the process. These costs are analogous to those of any major software rework that occurs due to other failures, and are highly context-dependent. 6.03 Non-Technical Risks Non-technical risks refer to those factors that affect the planned reuse of software, but do not pertain to the technology of software engineering. These factors can be organizational, administrative, psychological, or legal and can sometimes overwhelm the technical challenges. If not addressed in policies and guidelines, they can often spell the difference between cost effectiveness and cost ineffectiveness. Kalagiakos (2003) writes: Current reuse support systems allow developers to gain some of the benefits of reusability but, in order for reusability to reach its full potentiality special attention has also to be given to the non-technical factors of software reuse. The greatest reuse technical solutions may fail if the human factor together with the contractual and legal issues are not addressed successfully. 6.03.1 Organization-Level Reuse Developing, deploying, and supporting systematically reusable software assets requires a deep understanding of application developer needs and business requirements. As the number of developers and projects employing reusable assets increases, it becomes difficult to structure an organization to provide effective feedback loops between these constituencies. Groups that develop reusable middleware platforms are often viewed with suspicion by application developers who resent the fact that they may no longer be empowered to make key architectural decisions. Likewise, internecine rivalries among business units may stifle reuse of assets developed by other internal product groups that may be perceived as a threat to job security or corporate influence. 6.03.2 Business and Economic Factors Supporting organization-wide reusable assets requires an economic investment, particularly if reuse groups operate as cost centers. Many organizations find it difficult to institute appropriate taxation or charge-back schemes to fund their reuse groups. Sound business practices must be applied when assessing the effectiveness of planned software reuse. If the Return on Investment (ROI) is poor, it may not justify the expense of reuse. Of course there are other factors that may be considered such as the need to maintain an effective skill base. CONSTELLATION PROGRAM SYSTEMS ENGINEERING CONSTELLA NEERING Title: Strategy for Software Reuse Study Final Report Document No.: AC1L1-52 Effective Date: October 1, 2005 Revision: 0 Page 65 More Resources and Time Required Reusable software requires more resources and a time factor that is often not taken into consideration during the planning process. For a typical project there is often little incentive for project managers to develop reusable software. One reason is, unlike hardware, there really is no easily accessible catalog of software components, therefore, developers continue to repeatedly “reinvent the wheel.” Contractual Incentives/Disincentives For most of the organizations within a corporate structure, the contract award process and individual rewards system are generally oriented against software reuse. Cost-plus contracts are disincentive to contractors to make use of reusable products. On a large project where numerous companies are involved, licensing issues and ownership of intellectual property rights make the reuse of software across the entire project extremely difficult. It is probably easier to develop reusable components of software within a corporation where these issues are not important. 6.03.3 Legal Factors Intellectual Property Rights Software is protected legally as intellectual property through laws that control its dissemination and use. These laws relate to the exclusive ownership of the idea, the form of expression of the idea, and the use of the idea and its expression. There are three basic methods to protect software: patents, copyrights, and trade secrets. Patents protect the rights to the idea itself, while copyrights protect the rights to the expression of the idea. Trade secret laws protect the rights to confidential business information. However, in many cases laws are not clear about the enforcement of intellectual property rights (Institute for Defense Analysis, 1991). ITAR International Traffic in Arms Regulations (ITAR) apply when development teams deal with foreign nationals and partners. Legislative experts must be consulted to ensure that appropriate international agreements can be drawn up to avoid the onerous restrictions of simplistic ITAR application. Today, many export items would be covered under an existing ITAR license for ISS. The cognizant NASA Center’s Export Control Office will determine if the export item(s) are covered by this license base on the technical information provided by the technical team or individual. If items are not covered by the existing licenses, the center’s Export Control Office will work with the NASA HQ Export Control Office and the U.S. State Department to apply for a new license. Depending on the export items, the process can take up to six months. If International Partners are part of the Constellation Program, it is recommended that the HQ Export Control Office apply for the “umbrella” license that would cover the data, technology, and knowledge exchange to ensure the cooperation necessary. Sharing data or technology does not mean that one must relinquish ownership of the intellectual property. The inventor or developer of the exchange items retains ownership of the items. The receiving party will have to comply with the copyright license. CONSTELLATION PROGRAM SYSTEMS ENGINEERING CONSTELLA NEERING Title: Strategy for Software Reuse Study Final Report Document No.: AC1L1-52 Effective Date: October 1, 2005 Revision: 0 Page 66 Liability In the context of software, liability refers to the legal responsibility for harm attributable to software components. Liability may affect not only the supplier of the component but also the user. For example, software suppliers could be liable for submitting defective software components that fail to meet performance standards or cause a software system malfunction. Liability is the responsibility of both the original and the “reused” developers. The original developers must document all the design architecture, capability, and sign effects (if any) of the object. The “reused” developers must make sure the object performs according to the document(s) and complies with the new system’s requirements. It is also important to thoroughly test the object as if it were developed in-house. It can be assumed as unnecessary to re-certify a module or object that has been certified for a certain critical level if it is reused for a new system with the same critical level. That may be true if the module alone is the new system. However, it would not be true if the module is only part of an integrated system. The integrated verification and validation process should be the same as if no reused part were included. The legal ramifications of software liability are fairly new. As a consequence, there is still some uncertainty (such as whether software is a product or service) about the nature of liability that may accompany software. For this reason, experts believe that organizations interested in reuse need to address liability issues, otherwise suppliers may be reluctant to submit components for reuse, repositories may limit the availability of components, and users may be unwilling to use the components in the repository. Responsibility Legal responsibilities often cause software reuse to not be considered viable for a project. One question arises as to who is responsible when a reused software asset fails. An even more complex question is, who is responsible when that asset is modified? Does this mean a purchaser of a reusable asset has the right to make changes when it is being used in a critical application? Using commercial aviation as an example, a common presumption is that some entity (typically the airframer) is responsible for ensuring the integrity of all modifications to a system. Thus reuse and interoperability arguments always hinge upon an overall organization maintaining a global safety argument. Unfortunately, this tends to limit opportunities for planned reuse in the aviation sector. 6.03.4 Psychological Factors and Incentives Although the benefits of planned software reuse may appear to net a positive return across a project or organization, the benefits may not always be perceived by the very people who are charged with developing the software. In “Why Software Reuse has Failed and How to Make It Work for You,” (Schmidt, ND) states, “Application developers may perceive ‘top down’ reuse efforts as an indication that management lacks confidence in their technical ability. The ‘not invented here’ syndrome is ubiquitous in many organizations, particularly, among highly talented programmers.” If the benefits are not perceived at the developer level, it will be very difficult for a given project (as well as for individual developers) to embrace reuse-compatible strategies. CONSTELLATION PROGRAM SYSTEMS ENGINEERING CONSTELLA NEERING Title: Strategy for Software Reuse Study Final Report Document No.: AC1L1-52 Effective Date: October 1, 2005 Revision: 0 Page 67 The uneven distribution of reuse costs and benefits across projects generates the bulk of incentive problems at the project team-level. Table 6-1 illustrates how reuse costs and benefits might be perceived locally by a software development team. Note that, in the absence of incentives—specifically at the producer-level—local costs clearly outweigh the local benefits. 6.03.5 Inadequate Training Schmidt goes on to say: As if these non-technical impediments aren’t daunting enough, reuse efforts also frequently fail because developers lack technical skills and organizations lack core competencies necessary to create and/or integrate reusable components systematically. For instance, developers often lack knowledge of, and experience with, fundamental design patterns in their domain, which makes it difficult for them to understand how to create and/or reuse frameworks and components effectively. Developers have often been observed putting too much faith in language features, such as inheritance, polymorphism, templates, and exception handling as the primary means to foster reuse. Unfortunately, languages alone don’t adequately capture the commonality and variability of abstractions and components required to build and systematically apply reusable software in complex application domains. Table 6-1. Local Reuse Costs vs. Benefits (Fichman and Kemerer, 2001) Reuse Costs Reuse Producer Costs: • Cost of performing cost-benefit analysis • Cost of performing domain analysis • Cost of designing reusable parts • Cost of modeling/design tools for reusable parts • Cost of implementing reusable parts • Cost of testing reusable parts • Cost of documenting reusable parts • Cost of obtaining reuse library tools • Cost of added equipment for reuse library • Cost of resources to maintain reuse library • Cost of management for development, test and library support groups • Cost of producing publications • Cost of maintaining reusable parts • Cost of marketing reusable parts • Cost of training in software reuse Reuse Benefits Reuse Producer Benefits: • Added revenue due to income from selling reusable information • Added revenue from fees or royalties resulting from redistribution of information CONSTELLATION PROGRAM SYSTEMS ENGINEERING CONSTELLA NEERING Title: Strategy for Software Reuse Study Final Report Document No.: AC1L1-52 Effective Date: October 1, 2005 Reuse Consumer Costs: • Cost of performing cost-benefit analysis • Cost of performing domain analysis • Cost of locating and assessing reusable parts • Cost of integrating reusable parts • Cost of modifying reusable parts • Cost of maintaining modified reusable parts • Cost of testing modified reusable parts • Fees for obtaining reusable parts • Fees or royalties for reusing parts • Cost of training on software reuse Revision: 0 Page 68 Reuse Consumer Benefits: • Reduced cost to design • Reduced cost to document (internal) • Reduced cost to implement • Reduced cost to unit test • Reduced cost to design tests • Reduced cost to document tests • Reduced cost to execute testing • Reduced cost to product publications • Added revenue due to delivering product sooner to market place • Reduced maintenance costs • Added revenue due to improved customer satisfaction with product quality • Reduced cost of tools • Reduced cost of equipment • Reduced cost to manage development and test 6.03.6 Additional Risk Findings The following findings point to additional risks that must be taken into consideration in a planned software reuse development program. 1. A cross-cutting reliability risk is the reuse of software on platforms for which it was not originally designed. 2. Without careful design in advance for portability and reuse, there are likely to be dependencies built into the software peculiar to the legacy platform (e.g., aerospace vehicle). 3. A major reliability risk is undocumented or poorly understood dependencies embedded in the legacy software that would not transfer to a new application. 4. The use of reuse components, COTS, GOTS, or any other non-developmental items (NDI) is a risk that can be addressed by thorough risk management processes. 6.04 Mitigation Strategies Mitigating the technical risks of planned software reuse is largely based on effectively addressing the previous hard-won lessons of software engineering and computer science such as the strategies of structured programming, information hiding, modularization, and planned software reuse.9 Structured programming is the “adherence to a strict discipline of a few wellunderstood control structures. The lesson could be summarized as: ‘Give careful thought 9 Mitigation Strategy, citations from Software Program Managers Network, 2005. CONSTELLATION PROGRAM SYSTEMS ENGINEERING CONSTELLA NEERING Title: Strategy for Software Reuse Study Final Report Document No.: AC1L1-52 Effective Date: October 1, 2005 Revision: 0 Page 69 to the structuring of your software.’ Its goal is software that is easier to construct, prove correct and maintain, by keeping the structure clean and simple.” Information hiding is the “separation of concerns. The lesson here is: ‘Don’t bother users of your code with what happens inside, since they shouldn’t need to know.’ The goal is to hide unnecessary details, making the total software package a group of black boxes, each performing its specified function. The process involved is data abstraction and specification.” Modularization is the “structuring of code, grouping related pieces into modules. The lesson is: ‘Group logically related parts, so as to group related problems and to make their being solved easier.’ The goal is to collect related problems, easing again both construction and maintenance. The process involved is functional abstraction.” Planned software reuse implies using previously existing software assets to build new software. The lesson is: To modularize your software so it can be reused in other applications. The goal is using those assets to reduce the development effort and improve the quality of the software rather than creating it from scratch. Although, these four strategies are very much related, the difference is in their emphasis (Laverman, 1995): Strategy Emphasis Structured programming Design & implementation Information hiding Data abstraction Modularization Functional abstraction Planned software reuse Effort & quality Mitigations that address some of the specific risks identified earlier are described in the following subsections. 6.04.1 Mitigation for Inherited Software Risks Without careful design in advance for portability and planned reuse, there are likely to be dependencies built into the software peculiar to the legacy platform (e.g., aerospace vehicle). Reuse then requires careful analysis, removal of the dependencies, testing, and recertification. This effort is facilitated if the software, design, and code are well-documented; but for humanrated software, the reliability standard is necessarily high: all dependencies need to be tracked down and analyzed. 6.04.2 Mitigation for Legacy Software Risks There are four main ways of successfully reusing legacy software components and systems: • Reuse legacy code on top of new hardware that emulates the functioning of a legacy computer. CONSTELLATION PROGRAM SYSTEMS ENGINEERING CONSTELLA NEERING Title: Strategy for Software Reuse Study Final Report Document No.: AC1L1-52 Effective Date: October 1, 2005 Revision: 0 Page 70 • Reuse the legacy source code using new compilers/assemblers targeting new hardware platforms. • Translate legacy source code to new source code on new hardware. • Mine selected artifacts related to the software for reuse—design, requirements, algorithms, etc. These options are actually technical solutions in nature, but because they entail development costs that are sometimes unforeseen or unplanned, the use of legacy software can become a significant factor in the business case decision for planned reuse. 6.04.3 Mitigation for Business and Economic Factors Previous reuse histories and case studies may yield important signs of whether a particular reuse path is worth pursuing. Potential risks and mitigations of those risks must be assessed prior to deciding on reuse. (This is one of the purposes of this study, itself.) Figures of merit (FOM) must be constructed that will enable management to judge the potential worthiness of reuse. Examples for FOMs may include: • Modularity of reusable elements • Proven reusability of candidate reusable elements (%) • Complexity of reusable elements • Revision history of reusable elements A number of early investments are needed to make reuse viable even if case histories, figures of merit, and other analytical tools are already in place. One early investment is a sharable archive and retrieval system for reusable components and systems that provide for associated documentation. Table 6-2 shows typical targets of early reusable software investments (Boehm, 1997). Table 6-2. Targets for Early Investment to Enable Reuse COST FACTORS • • • • • • • • • Domain engineering Architecture determination Reusable components development Legacy software reengineering Process redefinition Training, teambuilding Repository development and operations Component certification Component maintenance MAJOR COST DRIVERS • • • • • • • • • • Breadth, maturity of domain Architecture maturity; diversity of legacy, COTS software Breadth of reuse; other cost drivers Legacy SW structure, understandability Process maturity Novelty of PLM approach; number of trainees Number of components; users Number of components; degree of certification Stability of application, technology, environment, and infrastructure Type of product (package, service) CONSTELLATION PROGRAM SYSTEMS ENGINEERING CONSTELLA NEERING Title: Strategy for Software Reuse Study Final Report Document No.: AC1L1-52 Effective Date: October 1, 2005 Revision: 0 Page 71 Early investments in these cost-factor areas will help mitigate some of the typical costs of planned reuse which arise from: • Locating and determining reusable software components • Modifying reusable software components • Integrating and testing assets as well as products that use reusable software components • Product/asset maintenance • Software asset acquisition and maintenance fees 6.04.4 Mitigation for Psychological Factors: Incentives The software development environment must be positive in order to encourage planned reuse of software. Developers and software project teams must be able to realize rewards for appropriately and correctly making the decision to reuse rather than to reinvent software. Even in the best cases, some original designing and coding will be necessary to make best use of reusable components. The measurement of productivity performance should include the effectiveness of the architecture and structure in which the reusable components are placed. Developers must be cautioned about “2nd systems affect”—the tendency to try to include every feature that was left out of the first incarnation. A “reuse compatible” incentive program has the goal to maximize the probability that, for any given project, the benefits to that project will outweigh the costs as perceived by the project team. Some heuristics for such incentives are: • Encourage reuse where it has the highest probability of a net positive local payoff. • Seek ways to shift the extra costs and risks of reuse production out of individual projects. • Recognize and encourage all forms of reuse that have net benefits at a project level, including less leverageable forms (e.g., within-project factoring to isolate common code, and reuse of components previously developed by the project team itself). • Focus incentives to reduce the incidence of reuse failures that are the most costly to the consumers of products built with reused components. 6.04.5 Additional Mitigation Findings The following recommendations address additional methods of mitigating risks inherent in a planned software reuse program. 20. Reuse is most successful when designed in from the beginning in a “product line” oriented process. 21. Definition of the software product line architecture, specifically the definition of common components versus adapted or domain-specific components, has both an CONSTELLATION PROGRAM SYSTEMS ENGINEERING CONSTELLA NEERING Title: Strategy for Software Reuse Study Final Report Document No.: AC1L1-52 Effective Date: October 1, 2005 Revision: 0 Page 72 immediate impact on viability of early and continuing reuse over the product line lifecycle. 22. Additional methods of addressing the technical risks of reuse include: • Standardization of reusable software components • Providing tools and training to support a development approach based on reuse • Providing a library of reusable parts that can easily be accessed and used • Using a software development methodology that fosters reuse 23. Non-technical risk factors can begin to be addressed in two simple ways, both of which are necessary but not sufficient for success: • Enlisting management support and enforcement for planned reuse • Emulating the behavior and actions of organizations that have to some extent overcome the difficulties of planned reuse CONSTELLATION PROGRAM SYSTEMS ENGINEERING CONSTELLA NEERING Title: Strategy for Software Reuse Study Final Report Document No.: AC1L1-52 Effective Date: October 1, 2005 Revision: 0 Page 73 7.0SUMMARY OF FINDINGS 7.01 Opportunity to Realize Major Benefits from Reuse 1. Productivity: Reduced time to deliver the same functionality: 25% to 40%. 2. Quality: Reduced defects via reuse of iteratively refined and matured software. 3. Cost: Reduced development time and improved quality yield return on investment for a variety of reusable products ranging from 100% to 600%. 4. Flexibility: Software (all artifacts) designed for reuse enable more rapid response to demand and systematic adaptation to a variety of requirements. 7.02 Confidence from a Rich History and Broad Current Work on Software Reuse 1. Maturity: Case studies, lessons learned, and life-cycle information readily yield common technical and non-technical factors that can spell success or failure. 2. Enabling technology and standards: Software engineering methodologies, tools, languages, architectures, development environments, and standards are available; appropriate selection and use is directly correlated with success and value returned. 7.03 Requirement for Success: a Systemic Engineering Technical Approach 1. Staff must be trained in the discipline of developing reusable core assets. 2. Reuse is most successful when “designed in” from the beginning as part of a “software product line”-oriented process. “Design for reuse” includes the development of software architectures, common standards and interfaces. Attempts at reuse later in the lifecycle, or reuse of legacy systems, are virtually guaranteed to fail. 3. Effective reuse in complex systems benefits greatly from the use of middleware, for example CORBA. Ease of development and deployment, and reduced project cost are just two of the important benefits. Middleware functionally bridges the gap between application programs and the lower-level hardware and software infrastructure. By providing this architectural mediating function, middleware coordinates how parts of applications are connected and how they interoperate. It also enables and simplifies the integration of components developed by multiple technology suppliers. Standards are embodied and enforced by middleware. 4. An iterative development cycle allows time to learn from the past then architecture and core assets are developed. 5. Standardization is another important aspect and all areas of the software lifecycle need to be gradually standardized, for example, standardizing the components themselves as well as the methodologies of identifying, classifying, modifying and incorporating these components into a wide range of systems. Establishing a classification scheme requires combining the knowledge inherent in the components with the knowledge about the application domain where the components are going to CONSTELLATION PROGRAM SYSTEMS ENGINEERING CONSTELLA NEERING Title: Strategy for Software Reuse Study Final Report Document No.: AC1L1-52 Effective Date: October 1, 2005 Revision: 0 Page 74 be used. Common characteristics of the components are then grouped and organized into a structure that can be easily understood. 6. The repository of core assets is the enabling tool for software reuse in a “software product line” approach. The repository builds on the standards and product line structure defined through domain analysis. It must be widely available, easy, and efficient to use. As a consequence, the storage, definitions, searching, accessing, and representation of components are just some of the factors that need to be considered in creating an “easy-to-use” repository. Such a repository must be built slowly and carefully, with components that are high quality and well documented. This approach is more likely to lead to (1) greater trust by reuse asset producers and consumers, (2) increased usefulness to software project teams, and (3) reduced risk in reuse with a corresponding lower average costs of reuse failure. 7. System level reuse may pay big dividends for Constellation. This requires up-front programmatic investment and attention to the Constellation Reuse Strategy, especially the software architecture and the establishment of an effective system level product line. 7.04 Importance of and Necessity of Both Technical and Non-Technical Factors 1. “To succeed in-the-large, reuse efforts must address both technical and non-technical issues. Focusing on non-technical issues alone neglects important skills-building education that must occur to succeed with non-trivial reuse efforts. Likewise, focusing solely on technological issues neglects the deep impact that social and economic forces and processes have on the success or failure of technology adoption.” Project policies, plans, commitments and management support must be in place early and consistently applied across the project. Non-technical issues in legal, procurement, intellectual property and related areas must be addressed. Management must make investments and support, enforce, and incorporate systematic reuse in their organizations and processes. 2. Significant investment must be expended up-front to produce efficient, flexible, and well-documented reusable software assets before they can be leveraged in subsequent generations of a product line. Therefore, organizations must support an appropriate software development process that allows systematic reuse to flourish. 3. It is not a question of whether Constellation should or should not reuse “software.” Reuse includes “non-developmental items” such as: legacy software systems, COTS, GOTS, MOTS, and a multiplicity of software engineering artifacts, processes, and procedures. The question is how best to practice reuse for both non-developmental items and project-developed software artifacts. 4. The importance of software reuse issues in the acquisition process cannot be emphasized enough. Some of the issues include (1) how reuse should be considered in the request-for-proposal process, (2) what criteria to use for evaluating proposals, CONSTELLATION PROGRAM SYSTEMS ENGINEERING CONSTELLA NEERING Title: Strategy for Software Reuse Study Final Report Document No.: AC1L1-52 Effective Date: October 1, 2005 Revision: 0 Page 75 (3) how costs of reuse will be evaluated and estimated, and (4) what incentives are needed in solicitation documents to promote reuse. CONSTELLATION PROGRAM SYSTEMS ENGINEERING CONSTELLA NEERING Title: Strategy for Software Reuse Study Final Report Document No.: AC1L1-52 Effective Date: October 1, 2005 Revision: 0 Page 76 8.0RECOMMENDATIONS This section provides a set of team recommendations that, if adopted and carefully implemented, will enable the Constellation Program to realize and maximize the benefits of software reuse. Fully aware that there are a great many details to be developed, a second major purpose is to lay out the groundwork for a set of next-steps and follow-on studies. The team’s recommendations are grouped into two primary areas that are reflected elsewhere in this report. Non-Technical Recommendations address those factors that affect the reuse of software, but do not have to do with the technology of software engineering itself. Technical Recommendations address those factors that affect the processes, methodologies, and technical products of software engineering. Non-technical factors can easily overwhelm technical factors. The technical approach to reuse can be 100% correct and reuse can still fail if the non-technical factors are not handled properly. The team believes that it is essential to address both areas, and to address them early in the project lifecycle. For Constellation, this means that the Program must immediately undertake multiple actions to create the required technical and non-technical systems engineering products for reuse that in turn will make software reuse effective, readily usable, and a preferred implementation mode by Constellation programs and primes. Successful reuse efforts general execute these recommendations out of an office specifically chartered and empowered by the project to oversee the execution of the product line reuse effort. 8.01 Non-Technical Recommendations 24. Immediately establish a team to develop a comprehensive actionable product line reuse strategy that addresses the recommendations herein. The team should be empowered to guide other study teams to begin organizing Constellation in a manner that facilitates a product line approach. 25. Adopt and enforce management and organizational policies, processes and procedures that require and promote effective reuse across all Constellation elements. It is essential to ensure continuous, top-to-bottom management support for reuse policies, plans, and activities. In so doing, it must be recognized that the benefits of effective reuse may not become realized in an organization or product line for several years. The benefits of software reuse can only be realized and maximized if software reuse is formalized at the program level, and adopted or mandated at project level. The team proposes the following approach to help achieve awareness: • Brief Constellation management and highlight the executive summary with technical recommendations. • Brief stakeholders and others who will be responsible for implementing the recommendations (as adopted) of this report, including project managers, software system engineers, and software engineers with the results of this report (NASA CONSTELLATION PROGRAM SYSTEMS ENGINEERING CONSTELLA NEERING Title: Strategy for Software Reuse Study Final Report Document No.: AC1L1-52 Effective Date: October 1, 2005 Revision: 0 Page 77 internal first; primes and subcontractors as identified by NASA management and stakeholders). • Offer website with report results and available reference links. 26. Establish an initiative to disseminate information on software reuse and the related Constellation policies. The technical recommendations listed in this section and elsewhere in this report are key factors to consider when writing a plan or strategy for achieving success in software reuse, operability, and compatibility. Another important factor that would contribute to success is the acceptance of the strategy by NASA’s management and software community. The software reuse benefit discussed in this report needs to be appreciated across the agency. The benefit can only be realized and maximized if software reuse is formalized at program-level, and adopted or mandated at projectlevel. To help achieve awareness, the team proposes the same approach as for Recommendation #2. 27. Ensure software reuse-related requirements are addressed and incorporated into Constellation procurements, including the CEV “Call for Improvement.” Historical data indicates that past development programs have benefited greatly in terms of cost, schedule, and risk reduction through the use of planned software reuse methodologies. Such methodologies would include commitment of high-level management, development of reuse infrastructure such as automated program libraries, and full documentation of software elements that encourage adaptation and use of the software in and out of product lines. Because of the large volume of software expected in the Constellation Program and the necessity to share this software across multiple project element boundaries, it would be highly beneficial for the developers of the software to work within a consistent software reuse environment. This environment would ensure maximum interoperability between all the entities contributing to the development of Constellation systems. Providing a contractual incentive to encourage software reuse in a consistent environment will be a prudent business practice that will enhance reuse and provide long-term benefit to Constellation as a whole. In order to make software reuse a part of the Constellation Program from its inception, it is critical to include the CEV project and to ensure that project’s compliance with the adopted recommendations of this report. The next window of opportunity to impact CEV is the upcoming Call-for-Improvement (CFI) to be released by early FY06. In this CFI, specific sections devoted to software reuse policy need to be included. For instance, it is recommended that the CEV Project’s Software Management Plan and the applicable Constellation SEI standards and other products passed as requirements to the CEV project, should include provisions based on an explicit Software Reuse Plan. CONSTELLATION PROGRAM SYSTEMS ENGINEERING CONSTELLA NEERING Title: Strategy for Software Reuse Study Final Report Document No.: AC1L1-52 Effective Date: October 1, 2005 Revision: 0 Page 78 All additional Constellation proposal solicitation should require the responder to identify the software that will be considered for reuse on the project. Proposals should include a section on the applicable software, if any, that will be made available for reuse in the proposed “Software Reuse Catalog”. Thus all software acquisition and development activities must explicitly be required to consider the reuse on overall project and future payoff. 28. Construct an incentive program that will promote and reward effective reuse by Constellation products and prime contractors. Further study is recommended. 29. Ensure NASA and the Constellation Program have addressed the legal issues related to software reuse. Further study is recommended. 30. Establish a “Software Reuse Tiger Team” to identify high leverage reuse opportunities. The establishment of a software tiger team with responsibility for the identifying early high-leverage reuse opportunities for Constellation, with initial emphasis on CEV, Launch and Mission Systems (LMS) and Exploration Communications and Navigation Systems (ECANS) projects. 31. Require and create plans for software reuse, at each Constellation level, that address the key technical and non-technical factors for successful reuse while at the same time insisting on sound business practices in assessing opportunities and effectiveness of reuse at each level. Further study is recommended. 32. Monitor the effectiveness of Software Reuse Plans across the project lifecycle. Further study is recommended. 33. Undertake periodic, thorough cost estimation, monitoring and assessing reusable software engineering assets. This could be included in a “software product line” strategy. The application of sound business practices in assessing opportunities and the effectiveness of software reuse will be paramount in determining the actual benefits realized by NASA, also in the monitoring and adjustment (if needed) of contractual business incentives. 34. Develop and specify Constellation engineering and management processes that promote reuse, support the business case, and make software reuse the readily adoptable, preferred approach. This will require the creation of policies and plans that promote reuse, create compelling incentives, and reward effective reuse by projects and prime contractors. CONSTELLATION PROGRAM SYSTEMS ENGINEERING CONSTELLA NEERING Title: Strategy for Software Reuse Study Final Report Document No.: AC1L1-52 Effective Date: October 1, 2005 Revision: 0 Page 79 The introduction of a reuse infrastructure within an existing organization and the associated modification of employee behavior and processes is a complex interdisciplinary task. Additionally, the Constellation Program has its own specific characteristics that need to be understood in order for the technology infusion to be successful. A number of examples in the literature are showing that unless a real effort is put into the insertion activities, low adoption and minimal success can be expected. For instance, a reuse technology insertion approach requires motivation of all involved, namely the professionals who have to change their practices, and the management who control the resources. Constellation should undertake a study to identify the key success criteria for software reuse technology infusion during the early phases of the project and Constellation-specific actions to meet those success criteria. 35. Develop a “Reuse Maturity Model” for organization elements (NASA and nonNASA) and require continuous improvement. In order to assess the reuse maturity level of the Constellation Program, the team recommends adopting a Reuse Maturity Model. Reuse Maturity is defined as the extent to which a company’s reuse practices are defined, managed, measured, controlled, and are generally effective. There are various reuse maturity models proposed in the literature. These models consist of a set of key areas that capture different aspects of the introduction and improvement of a company’s reuse maturity. Constellation should develop a model to encourage and evaluate continuous improvement in organization, workgroup, and individual readiness and practices. Effective reuse will require change in organizational, workgroup, and individual behavior. • Use this model for assessment of Constellation organizations and their capability for capturing the benefits of reuse for Constellation. • Require continuous improvement in Reuse Maturity. 8.02 Technical Recommendations 1. Establish a policy and develop the technical requirements for “Design for Reuse.” A “Design for Reuse” policy provides a conceptual framework for the design of reusable components. Details should embody the best practices described in this study report and include those technical processes, procedures, standards, etc. that mitigate specific risks of reuse. A follow-on study should focus on producing the set of guidelines for “design-for-reuse” that will define the key principles to be followed by the Constellation Program. The next level of details would address specific concepts such as component structures, interfaces, parameterization, and composition. This follow-on study would also allow the project to analyze and select alternatives at the next level of detail. CONSTELLATION PROGRAM SYSTEMS ENGINEERING CONSTELLA NEERING Title: Strategy for Software Reuse Study Final Report Document No.: AC1L1-52 Effective Date: October 1, 2005 Revision: 0 Page 80 2. Constellation Software Management Plans should explicitly address the software reuse strategy. The Constellation Program Software Management Plan should include a complete and thorough strategy and requirements regarding software reuse, interoperability, and compatibility. The strategy will vary depending on the different level of software criticality, as defined in Appendix B of NPR 7150.2. Human Rated Flight Software is classified as the highest criticality level software class and General Purpose Desktop Software is classified as the lowest criticality level software class. The software reuse strategy must clearly place requirements on how each software class should be handled in terms of adaptation, application, documentation, and verification and validation. Class G S/W—General Purpose Computing Software and class H S/W— General Purpose Desktop Software probably do not engender any formal requirements for reuse strategy, except in specific cases where they are used to develop higher level-of-criticality software class. The software reuse strategy also needs to address requirements on how to treat other software types, such as contracted software, off-the-shelf software (COTS, GOTS), or glueware, in terms of adaptation, application, documentation, verification and validation, and legal implication. 3. Implement changes to the technical plan to reach readiness for the Systems Requirements Review. Several specific reused-related activities should be incorporated immediately into the near-term activities of Constellation SEI. • During Requirement Analysis Cycle 1 (RAC1) The Software Requirements Document (SRD) should be modified to include software reuse. The changes to the Constellation Responsible Assignment Matrix (CRAM) specified in Section 9.0, including those to the Software Management Plan, should be implemented. As part of the RAC1 integration review activities, the project should review the status of progress in the adoption of a reuse plan. • During Requirement Analysis Cycle 2 (RAC2) The project needs to refine and update the Systems Requirements Review (SRR), and review the status of progress in the adoption of a reuse plan as part of RAC2 integration review. – Establish a software tiger team with responsibility for identifying of early highleverage reuse opportunities for Constellation, with initial emphasis on CEV, Launch and Mission Systems (LMS) and Exploration Communications and Navigation Systems (ECANS) projects. – Develop a software cost model parameterized to Constellation that will be used to determine the most highly leveraged and minimally risky investments in software reuse. CONSTELLATION PROGRAM SYSTEMS ENGINEERING CONSTELLA NEERING Title: Strategy for Software Reuse Study Final Report Document No.: AC1L1-52 Effective Date: October 1, 2005 Revision: 0 Page 81 4. Develop and maintain a Software System Architecture at the Constellation Program Systems Engineering level based on “software product lines.” The objective is to identify specific [or] specify architecture and design patterns to be reused by the Constellation Program. A common architecture enables the creation of product platforms (product lines): sets of similar products that are developed rapidly from a common starting point. It defines rules for developing reusable components by defining interface, control, and data format standards, and it speeds development by defining design patterns and application frameworks. One very effective form of software reuse is the reuse of software architectures. A follow-on study focused on establishing an architectural style, assessing the most relevant domain-specific design patterns and application frameworks, defining the control and data format standards, and middleware specifications would allow the Constellation Program to start reaping the benefits of reuse early in the lifecycle. 5. Develop software engineering technical requirements for creation of reusable artifacts. Require a modular, model-based engineering and component-based development (CBD) approach, use of design patterns, and application frameworks. Additional details should be developed in a follow-on systems engineering activity. 6. Specify standards, requirements, and guidelines for middleware or platform independent constructs to isolate software components from hardware peculiarities. This recommendation provides strong technical support to realize the benefits of reuse and mitigate numerous risks. Independent of the topic of software reuse, it is also a recommendation based on software systems engineering state-of-the-practice. 7. Adopt engineering policies, processes, and procedures that promote effective reuse across Constellation elements. The processes for effective software engineering and software systems engineering share a great deal with those for systems engineering. These two “families” of systems engineering should be brought into closer alignment. • Criterion for software design, review, and evaluation must be modified to infuse the practices that lead to development of reusable software. • The system engineering process that identifies the software requirements also must be modified to take into account reusable artifacts. The software reuse process is an important step of the software systems engineering lifecycle and should be formalized and made a requirement. • Use architectural and design definitions based on approved higher-level requirements in the selection of reusable products. CONSTELLATION PROGRAM SYSTEMS ENGINEERING CONSTELLA NEERING Title: Strategy for Software Reuse Study Final Report Document No.: AC1L1-52 Effective Date: October 1, 2005 Revision: 0 Page 82 8. Develop metrics for software reusability. A Constellation Software Reuse Plan will need to prove its effectiveness through the use of specific metrics. A follow-on study should be performed to define the detail measurements to put in place from the inception of the reuse program. Based on the wealth of published research in this area, this study should identify those metrics that are relevant and well targeted for the Constellation Program. Constellation should develop measurements, metrics, and decision process with focus on rapid deployment in current Constellation activities. Examples of topic areas worthy of follow-on study for the development of software reuse metrics are: • Reuse by organization (people) not involved in development • Granularity of reuse—software vs. only software artifacts • Changes required to reusable software or application software • Portability to new platforms • Plan for technology insertion • Increase in productivity measured in terms of cost savings & schedule gain • Reduction in repair time, reduction in customer complaints • Lifecycle cost of acquiring reusable software • Technical, liaison, and maintenance support for reusable software • Ease of availability/visibility through catalogs • Usage by multiple users/organizations • Ease of incorporation into different architectural environments • Level of dependency on operating system • Use of good programming practices during development such as modularity, cohesion, etc. 9. Manage (re)use of non-developmental items. The use of reusable components, commercial off-the-shelf (COTS), government offthe-shelf (GOTS), modified off-the-shelf (MOTS), legacy software, or any other nondevelopmental items (NDI) should be treated as a risk and managed through risk management processes. The reuse of COTS, GOTS, and any other NDI requires the definition of precise acceptance criteria based on a number of parameters such as the target software criticality, the required certification process, the potential benefits of obtaining it. The follow-on study should allow the Constellation Program to categorize the NDI, to refine the acceptance criteria, elaborate risks, and identify potential mitigation strategies. CONSTELLATION PROGRAM SYSTEMS ENGINEERING CONSTELLA NEERING Title: Strategy for Software Reuse Study Final Report Document No.: AC1L1-52 Effective Date: October 1, 2005 Revision: 0 Page 83 Application of reuse components, COTS, GOTS, or any other NDI should be made only after successful completion of an NDI acceptance inspection. This inspection needs to consider a standard list of reuse criteria. These will include the process used to develop it, how it was documented, the number of users, user experience, and compliance with essential program technical and non-technical considerations. Before a decision is made to reuse a product or to acquire COTS, GOTS, or NDI, a complete cost trade-off should be made considering the full lifecycle costs, update requirements, maintenance costs, warranty and licensing costs, and any other considerations that impact use of the product throughout its lifecycle. All reuse products, COTS-, GOTS-, or NDI-decisions should ensure compliance with Constellation architectural and design definitions. The selection of all reuse items must be traceable back to an approved user requirement. All reuse components, COTS, and GOTS need to be tested individually, first against program requirements, and then in an integrated software and system configuration prior to release for testing according to the program test plan. Reuse COTS-, GOTS-, and NDI-decisions should be continuously revisited as program conditions change. 10. Establish a “software product line” component library. The major purpose of this recommendation is to make reusable artifacts readily available to Constellation development across institutional and organizational boundaries. Recommendations related to the establishment of a Software Product Line Component Library are as follows: • The reusable components in the Library must be under strict configuration management. • To speed adoption and remove some of the costs of reuse from the user, the metadata associated with each component should be complete and the component itself should be “certified” (guaranteed to meet a set of pre-determined criteria). • A software taxonomy should be developed that will allow better definition and classification of metadata for software for easy deposition and search from the Reusable Software library. • Operations of the library should include promotion of reuse across all organizations participating in software development activities, inspection and audit responsibilities, and maintenance of software reuse policies and plans at the Constellation SEI level. • The library should provide some level of support for COTS, GOTS, MOTS, legacy, and open-source. CONSTELLATION PROGRAM SYSTEMS ENGINEERING CONSTELLA NEERING Title: Strategy for Software Reuse Study Final Report Document No.: AC1L1-52 Effective Date: October 1, 2005 Revision: 0 Page 84 • The library should identify Constellation software artifacts, developed and available throughout the project lifecycle at a given level of maturity. • A valuable extension of the library functions could include verification and validation of standard libraries widely used in software engineering (e.g., MatLab, WindRiver products). 11. Provide tools and techniques to development groups that promote and enable reuse. For the reuse process to be successful, the availability of tools to enable the rapid identification, adaptation, and deployment of reusable software assets can be a critical factor. Constellation should identify specific tools that will enable the most effective use of Constellation software assets and the relationship of these tools to the proposed “Software Product Line Asset Repository.” “Tools” are broadly defined as instruments that facilitate the reuse of products or byproducts during the software lifecycle, including reuse libraries, application templates and generators, common testbed environments, etc. Tool categories include Analysis and Design, Asset Constructors, Asset Testers, Reuse Management. A reuse-based software development model (a.k.a. “Software Product Line”) follows the assembly-line analogy. It is built around a reuse infrastructure. A follow-on study of the infrastructure to put in place for the specific characteristics of Constellation would allow the program to identify early how to build an appropriate infrastructure. The objective would be to develop expanded Constellation-specific detail for the infrastructure recommendations identified in this report. Domain engineering has been noted earlier herein as a necessary component of an effective software product-line approach. This type of systems engineering has its impact on software systems engineering via a well-defined process and vocabulary. This can be embodied in a “domain specific language” (DSL). The basic idea of a DSL is a computable language that is targeted to a particular kind of problem, rather than a general purpose language that is aimed at any kind of software problem. This relatively new field of research could be of high interest to Constellation in the systematic representation and software systems implementation of domain engineering requirements. Some DSL tools are starting to appear and could be leveraged for the Constellation Program. A follow-on study on this topic could make a recommendation to Constellation on the best method for using DSL to address the domain engineering process requirements. CONSTELLATION PROGRAM SYSTEMS ENGINEERING CONSTELLA NEERING Title: Strategy for Software Reuse Study Final Report Document No.: AC1L1-52 Effective Date: October 1, 2005 Revision: 0 Page 85 9.0NEXT STEPS—IMPACT ON REQUIREMENTS Software has played a major role in most parts of space flight programs at NASA. The adoption of the “Software Strategy for Reuse,” as recommend in this study, will change a large part of the Constellation Program’s software requirements and development. These changes will impact across the software system lifecycle—from planning, requirements, development, and testing, to sustaining. The degree of impact will vary depending on whether or not the software is destined for reuse or the software itself is being reused. The degree of impact also depends on the criticality level of the software (see NASA Procedural Requirements NPR 7150.2). This section will identify ESMD requirements and Constellation Responsible Assignment Matrix (CRAM) products that will be impacted by the strategy for software reuse. An approach to extend communication of the software reuse strategy to the NASA software community is needed. This section will also discuss how the result of this study will affect the Call-ForImprovement for CEV. The road to SRR will be addressed. In addition to those new studies identified earlier in Section 8.0 Recommendations, this section will suggest follow-on studies specifically related to steps identified here. 9.01 Traceability to Requirements The design and development of the EDMS flight software requirements, as listed below, will be affected by the software reuse strategy. These requirements include monitoring, command and control, automations, and system health management. Whether or not these requirements will be included in further revisions of the requirement documents, they will be superseded by future revisions that incorporate any change required as a result of the ESAS (60 days) study. Design solutions for the requirements identified in this section, are governed by high level requirements, guidelines, and standards, as defined in NPR 7150.2—NASA Procedural requirements, NASA NPR 8750.2—Human Rating Requirements and Guidelines for Space Flight Systems, NPR 8715.3—NASA Safety Manual, and NASA-std 8719.13—Software Safety Standard, for all safety-critical software. The “Reuse Strategy” recommended in this report must also be in compliance with these NASA requirements. The following requirements are from the NASA Exploration Crew Transportation System Requirements Document (ESMD-RQ-001, Rev E, 30 Mar 2005). The document’s section headings are shown in gray to indicate the context; when no headings are shown, the prior heading applies. 3 System Requirements 3.3 Performance Requirements CTS0050H The CTS shall provide the capability to monitor, command, and control, its system segments and elements. CTS0075H The CTS shall provide automated control linked to the mission phase and function, with an objective of automated operations throughout the mission. CTS0300H The CTS shall provide autonomous operations linked to the mission phase and function with an objective of autonomous operations throughout the mission. CONSTELLATION PROGRAM SYSTEMS ENGINEERING CONSTELLA NEERING Title: Strategy for Software Reuse Study Final Report Document No.: AC1L1-52 Effective Date: October 1, 2005 Revision: 0 Page 86 CTS0320H The CTS shall provide manual intervention of automated functions critical to mission success and crew safety. 3.3.5 Software CTS0270H The CTS shall comply with NPR 7150, NASA Software Engineering Requirements. 3.4 Safety 3.4.3 Vehicle Health CTS0190H The CTS shall automatically detect and annunciate conditions which could result in loss of human life, loss of vehicle, loss of mission, or significantly impact mission capability. CTS0200H The CTS shall provide autonomous fault detection, isolation and recovery, with an objective of automated isolation and recovery, from conditions which could result in loss of human life or loss of vehicle. CTS0230H The CTS shall provide isolation and recovery from conditions which could result in loss of mission or significantly impact mission capability. CTS0045H The CTS shall capture, archive, and make retrievable, mission and safety critical performance data. 3.7 Subordinate Elements 3.7.2 Crew Exploration Vehicle (CEV) 3.7.2.3 Performance Characteristics 3.7.2.3.5 Software CEV0170H The CEV shall comply with NPR 7150, NASA Software Engineering Requirements. 3.7.2.4 Safety 3.7.2.4.1 General CEV0180H The CEV shall comply with NASA NPR 8750.2, Human Rating Requirements and Guidelines for Space Flight Systems CEV0190H The CEV shall comply with NPR 8715.3 NASA Safety Manual. CEV0200H The CEV shall comply with NASA-std 8719.13, Software Safety Standard, for all Safety-Critical Software. 3.7.2.4.3 Vehicle Health CEV0240H The CEV shall provide autonomous isolation and recovery, with an objective of automated fault isolation and recovery, from conditions which could result in loss of human life or loss of vehicle. CEV0290H The CEV shall provide isolation and recovery from conditions which could result in loss of mission or significantly impact mission capability. CEV0047H The CEV shall capture, archive, and make available for retrieval, mission and safety critical performance data. The following requirements are from the NASA Exploration System of Systems Programmatic Requirements and Guidelines Document (ESMD-RQ-0021, Rev E, 24 Mar 2005). The CONSTELLATION PROGRAM SYSTEMS ENGINEERING CONSTELLA NEERING Title: Strategy for Software Reuse Study Final Report Document No.: AC1L1-52 Effective Date: October 1, 2005 Revision: 0 Page 87 document’s section headings are shown in gray to indicate the context; when no headings are shown, the prior heading applies. 3 System Requirements 3.1 Exploration Programmatic Requirements EPR0670 The program will establish "Design for Minimum Risk" criteria for components/ systems which do not meet the applicable failure tolerance requirements through the use of the following process: (process not quoted here) EPR0560 The responsible program will develop interface standards at the Exploration System of Systems level. EPR0500 The responsible program will develop a Human Rating Plan to address compliance with NASA 8705.2 Human Rating Requirements and Guidelines for Space Flight Systems. This is parent of CEV0180H. 3.2 Exploration Programmatic Guideline EPG0810 The responsible program should simplify and minimize the number of interfaces between systems, segments, and elements, and between organizations where such simplification will increase reliability, reduce risk, and minimize lifecycle costs. Software reuse should provide opportunities to simplify and minimize interfaces. EPG0820 The responsible program should use common subsystems, interfaces, and software across the Exploration System of Systems where such commonality will reduce risk and minimize cost. One of the major principles driving software reuse is the reduction of risk and cost. EPG0840 The responsible program should, to the maximum extent practical, design the Exploration System of Systems to allow for the use of commercially provided items and services. Third party software and commercially available computing systems fall under this requirement. The following requirements are from the NASA Exploration System of Systems Technical Requirements Document (ESMD-RQ-0010, Rev E, 30 Mar 2005). The document’s section headings are shown in gray to indicate the context; when no headings are shown, the prior heading applies. 3 System Requirements 3.5 NASA Procedural Requirements ESS0350 Manual. The Exploration System of Systems shall comply with NPR 8715.3, NASA Safety This is the parent of CEV0190H. ESS0370 The Exploration System of Systems shall comply with NPR 7150, NASA Software Engineering Requirements. This is the parent of CTS0270H and CEV0170H. ESS0360 The Exploration System of Systems shall comply with NASA-STD-8719.13, Software Safety Standard for all Safety-Critical Software. This is the parent of CEV0200H. CONSTELLATION PROGRAM SYSTEMS ENGINEERING CONSTELLA NEERING Title: Strategy for Software Reuse Study Final Report 9.02 Document No.: AC1L1-52 Effective Date: October 1, 2005 Revision: 0 Page 88 Impact on SEI Constellation Responsible Assignment Matrix (CRAM) products 9.02.1 Software Management Plan (SMP)—DSM&A The Software Management Plan should include a complete and thorough discussion of strategy and requirements regarding software reuse, interoperability, and compatibility. The strategy will vary depending on the different level of software criticality, as defined in Appendix B of NPR 7150.2. Human Rated Flight Software is classified as the highest criticality level software class and General Purpose Desktop Software is classified as the lowest criticality level software class. The software reuse strategy must clearly place requirements on how each software class should be handled in term of adaptation, application, documentation, and verification and validation. Class G S/W—General Purpose Computing Software and Class H S/W—General Purpose Desktop Software probably do not engender any formal requirements for reuse strategy; except in specific cases where they are used to develop higher level of criticality software class. The software reuse strategy also needs to have requirements on how to treat other S/W types, such as Contracted Software, Off-The-Shelf (COTS, GOTS), or Glueware, in term of adaptation, application, documentation, verification & validation, and legal implication. 9.02.2 Other Constellation Program Systems Engineering Products Table 9-1 identifies the Constellation Program systems engineering products, in addition to the Software Management Plan, that will also be impacted by a software reuse strategy. Products are identified by their number as designated in the Constellation Responsibility Allocation Matrix (CRAM). Table 9-1. Reuse Impact on Program Systems Engineering Products IPT T&V T&V Product Number Product Title Product Description Milestone C0254P Master Validation, The purpose of the CR Integration, and Constellation MVI&V Plan Verification Plan is to document the approach, philosophy, requirements, guidelines, processes, and best practices associated with the validation, integration, verification and certification of the elements that comprise the System. CxxxxP Test & Verification This document will detail Strategy top-level strategies and philosophies for testing and verification. Strategy will be developed utilizing data, products, and other input provided by virtually all other IPTs. Impact Should include plan to T&V reused components. Should include strategy to T&V reused software component. (continued) CONSTELLATION PROGRAM SYSTEMS ENGINEERING CONSTELLA NEERING Title: Strategy for Software Reuse Study Final Report Document No.: AC1L1-52 Effective Date: October 1, 2005 Revision: 0 Page 89 Table 9-1. Reuse R Impact act on Progr Program Systems ms Engineering Engineerin ng Products—Continued Products— IPT T&V Product Number Product Title CxxxxP Constellation Systems Verification Plans and Matrix DSM&A C0042A DSM&A C0162A DSM&A CxxxxA RD&FA C0155P Product Description Milestone Impact Outline of verification Should include activities from an verification activity for integrated system level. reused software Document will detail component, and a individual tests to be detail test plan to performed, test objectives, perform on reused test configurations, ICDs, software component requirements, and specifications needed, HW and SW products involved. Command, See DR SDR Design and Control, Development will be Communications, impacted by software Computers, and reuse, interoperability, Intelligence and compatibility Support Plan strategy. Software Requirements (where SRR Software reuse policy Development applicable) for software and processes will be Guidelines/ development processes, key factors of the Standards. coding standards, data software development dictionaries, delivery guidelines. media, support role. ISHM Integrated Systems Health Design and Management development will be impacted by software reuse, interoperability, and compatibility strategy System Requirements must Requirements reflect software reuse, Documents interoperability, and compatibility. Test & Verification (T&V) products will have significant impact. The T&V strategy will need to include a process to test and verify software systems with reuse components. The process of testing and verification of software that is built to be reused will also need to be part of the T&V strategy. The process of T&V of a software system is assumed to be shortened or simplified if the software system employs a software component from an equal or higher software class. In the opposite case, when a higher software class reuses a component from a lower software class, the T&V process needs to be as stringent as if the component is built from scratch. Integration testing will always be performed regardless of whether reusable software components have been employed. The Design Synthesis Manufacturing & Assembly (DSM&A) products related to software (e.g., Software Development Guideline/Standard or Command, Control, Communications, Computers, and Intelligence Support Plan) will also have an impact. Software development guidelines must include steps to guide software system development in order to leverage on software reuse, as well as to generate reusable components to share across other Constellation programs. To comply CONSTELLATION PROGRAM SYSTEMS ENGINEERING CONSTELLA NEERING Title: Strategy for Software Reuse Study Final Report Document No.: AC1L1-52 Effective Date: October 1, 2005 Revision: 0 Page 90 with the software development guideline, standard, or SMP, software systems like Integrated System Health Management (ISHM) or Command and Control software must consider the software reuse strategy in the design and implementation phases. As discussed in Section 9.1—Traceability to Requirements, the System Requirements Document in the Requirement Definition & Functional Allocation (RD&FA) IPT will also be affected by software reuse strategy. The software requirements should reflect the result of this study. The software lifecycle needs to address software reuse, especially in the planning phase. 9.03 Follow-On Studies A number of follow-on studies should be considered for the short term in order to improve the chance of this study to positively impact the Constellation Program from the beginning. Brief descriptions of the proposed studies are provided below: 36. Develop a Software Product Line Strategy The product line concept as an integrating theme for the Constellation Reuse Strategy has not been fully developed in this report. A study aimed at understanding and developing the software product line as the backbone for the Constellation Reuse Strategy is needed. This study should include a review of the software architecture to identify potential product lines and select candidates for initial programmatic investment. 37. Domain Specific Language Domain engineering has been noted earlier in this report as a necessary component of an effective software product line approach. This type of systems engineering has its impact on software systems engineering via a well-defined process and vocabulary. This can be embodied in a “Domain Specific Language” (DSL). The basic idea of a DSL is a computable language that is targeted to a particular kind of problem, rather than a general purpose language that's aimed at any kind of software problem. This relatively new field of research could be of high interest to Constellation in the systematic representation and software systems implementation of domain engineering requirements. Some Domain Specific Language tools are starting to appear and could be leveraged for the Constellation program. Objective: A follow on study on this topic could make a recommendation to the Constellation on the best method for using DSL to address the domain engineering process requirements. 38. Tools for Constellation Software Asset Reuse For the reuse process to be successful, the availability of tools to enable the rapid identification, adaptation and deployment of reusable software assets can be a critical factor. CONSTELLATION PROGRAM SYSTEMS ENGINEERING CONSTELLA NEERING Title: Strategy for Software Reuse Study Final Report Document No.: AC1L1-52 Effective Date: October 1, 2005 Revision: 0 Page 91 Objective: Identify specific tools that will enable the most effective use of Constellation software assets and the relationship of these tools to the proposed “Software Product Line Asset Repository.” 39. Design for Reuse Technical Requirements A follow on study focused on producing the potential set of guidelines for “designfor-reuse” will define the key principles to be followed by the Constellation Program. The next level of details would address specific concepts such as component structures, interfaces, parameterization and composition. This will also allow the project to analyze and select alternatives at next level of detail. Objective: Provide a conceptual framework for the design of reusable components. 40. Software Systems Architecture One very effective form of software reuse is the reuse of software architectures. A follow on study focused on establishing an architectural style, assessing the most relevant domain specific design patterns and application frameworks, defining the control and data format standards and middle-ware specification would allow the Constellation Program to start reaping the benefits of reuse early on in the lifecycle. Objective: Identify specific architecture and design patterns to be reused by the Constellation Program. 41. Reuse of Non-Developmental Items The reuse of COTS, GOTS and any other non-developmental items (NDI) requires the definition of precise acceptance criteria based on a number of parameters such as the target software criticality, the required certification process, the potential benefits to obtain. The follow on study should allow the Constellation program to categorize the NDI, to refine the acceptance criteria, to elaborate risks and identify potential mitigation strategies. Objective: Initiate the decision process related to the reuse of NDI. 42. Software Product Lifecycle Reuse Metrics Constellation Program reuse plan will need to prove its effectiveness through the use of specific metrics. A follow on study should be performed to define the detail measurements to put in place from the inception of the reuse program. Based on the wealth of published research results in this area, this study should identify those metrics that are relevant and well targeted for the Constellation program. Objective: Develop measurements, metrics, and decision process with focus on rapid deployment in current Constellation activities. 43. Reuse Maturity Model CONSTELLATION PROGRAM SYSTEMS ENGINEERING CONSTELLA NEERING Title: Strategy for Software Reuse Study Final Report Document No.: AC1L1-52 Effective Date: October 1, 2005 Revision: 0 Page 92 In order to assess the reuse maturity level of the Constellation Program, we assess and adopt a Reuse Maturity Model. Reuse Maturity is defined as “the extent to which a company’s reuse practices are defined, managed, measured, controlled and generally effective.” There are various reuse maturity models proposed in the literature. These models consist of a set of key areas that capture different aspects of the introduction and improvement of a company’s reuse maturity. This study will select the appropriate model for the Constellation program. Objective: Develop a model to encourage and evaluate continuous improvement in organization, workgroup and individual readiness and practices. 44. System Level Reuse Because of the nature of the Constellation Program, system level reuse appears to have high potential benefits. The follow on study should be aimed at reviewing the software architecture to identify potential system reuse candidates and their need for early programmatic investment and attention. This is the first step in establishing a system level product line. CONSTELLATION PROGRAM SYSTEMS ENGINEERING CONSTELLA NEERING Title: Strategy for Software Reuse Study Final Report Document No.: AC1L1-52 Effective Date: October 1, 2005 Revision: 0 Page 93 10.0ACKNOWLEDGEMENTS This effort and the study described in this report was lead and performed in part by the Jet Propulsion Laboratory, California Institute of Technology under contract with the National Aeronautics and Space Administration. Contributing NASA Centers: Ames Research Center Goddard Space Flight Center Jet Propulsion Laboratory Johnson Space Center Kennedy Flight Center Marshall Space Flight Center Document preparation and editorial support was provided by members of the JPL Documentation Services Group. A complete list of study participants and contact information can be found in Appendix C. Note from the Study Lead (David Atkinson): I especially wish to acknowledge the editorial as well as technical contributions of Mark James, Kirk Reinholtz, Charlie Beswick and Harry Woo. Mark James’ leadership was especially important in managing the final editing and publication of this document. CONSTELLATION PROGRAM SYSTEMS ENGINEERING CONSTELLA NEERING Title: Strategy for Software Reuse Study Final Report Document No.: AC1L1-52 Effective Date: October 1, 2005 Revision: 0 Page 94 Appendix A: Glossary, Acronyms, and Abbreviations A.1 Glossary Application product lines: An application type generalized around a common architecture so that it can be adapted in different ways for different customers. Architecture: The software architecture of a program or computing system. The structure or structures of the system, which comprise software elements, the externally visible properties of those elements, and the relationships among them. An architecture largely permits or precludes a system’s quality attributes such as performance or reliability. Architecture represents a capitalized investment, an abstract reusable model that can be transferred from one system to the next. Aspect-oriented software development: Shared components woven into an application at different places when the program is compiled. Component-based development systems: Developed by integrating components (collections of objects) that conform to component-model standards. Configurable vertical application: A generic system designed so it can be configured to the needs of specific system customers. COTS software: Operating systems, libraries, applications, and other software purchased from a commercial vendor. Not customized for a particular project. Access to source code and documentation are often limited. COTS integration: Systems developed by integrating existing application systems. Design patterns: Generic abstractions that occur across applications, represented as design patterns that show abstract and concrete objects and interactions. Derivative reuse: Domain Specific Software Architecture: An architecture designed to solve a set of problems within a specific domain (area of related problems). A DSSA is an example of a generic architecture. A DSSA is a software architecture based on the actual and projected commonalities and differences of applications in a domain. It is a reusable design component providing the basis for other reusable design and code components. Legacy system wrapping: Legacy systems that can be “wrapped” by defining a set of interfaces and providing access to these legacy systems through these interfaces. Leveraged reuse: Reuse with modifications. Locked reuse: Reusable content that cannot be changed except by the original author. Locked reuse is used when to ensure that content is not changed. Middleware: Software that resides between applications and the underlying operating systems, network protocol stacks, and hardware. CONSTELLATION PROGRAM SYSTEMS ENGINEERING CONSTELLA NEERING Title: Strategy for Software Reuse Study Final Report Document No.: AC1L1-52 Effective Date: October 1, 2005 Revision: 0 Page 95 Opportunistic reuse: Refers to when the author makes a conscious decision to find an element, retrieve it, and reuse it. Opportunistic reuse requires that the author be aware of the opportunities for reuse and be motivated to search for and retrieve reusable content. Opportunistic reuse is the most common form of reuse. Opportunistic reuse does not rely on specific technology; it can be done without a content management system, although a CMS is advisable. Program generators: A generator system that embeds knowledge of a particular types of application and can generate systems or system fragments in that domain. Program libraries class and function libraries: Implementing commonly used abstractions are available for reuse. Reusable software: Software that is implemented in such a way as to facilitate reuse, as opposed to software that could be reapplied but which was not implemented with reuse in mind. Reusable software is actually an extension of the concept of maintainability, where maintainability implies that the software will be able to evolve with the problem, and reusability implies that the software will also be applicable to other related problems. Reuse methods: There are two methods for reuse: opportunistic reuse (an author makes a conscious decision to reuse content) and systematic reuse (planned reuse where content is automatically inserted). Within each method of reuse are three options: locked reuse (reused content cannot be changed), derivative reuse (reused content can be changed), and nested reuse (multiple versions of the content are contained within the same element). SBA Support Office: Office that is chartered with ensuring that the infrastructure of tools, policies, and organizational working groups required to realize simulation-based acquisition (SBA) are built, maintained, and improved as necessary. Service-oriented systems: Systems developed by linking shared services that may be externally provided. Simulation-based acquisition: Refers to supporting program decisions with accurate and timely data. Software Product Line: “A SPL is a set of software-intensive systems that share a common, managed set of features satisfying the specific needs of a particular market segment or mission and that are developed from a common set of core assets in a prescribed way.” [Reference: http://www.sei.cmu.edu/] Software reuse: A software product developed for one use but having other uses or one developed specifically to be usable on multiple projects or in multiple roles on one project. Examples include, but are not limited to, COTS products, acquirer-furnished software products, software products in reuse libraries, and pre-existing developer software products. Each use may include all or part of the software product and may involve its modification. This term can be applied to any software product (such as requirements and architectures), not just to software code itself. Often this applies to software previously written by an in-house development team and CONSTELLATION PROGRAM SYSTEMS ENGINEERING CONSTELLA NEERING Title: Strategy for Software Reuse Study Final Report Document No.: AC1L1-52 Effective Date: October 1, 2005 Revision: 0 Page 96 used on a different project. GOTS software would come under this category if it is supplied from one government project to another government project. [Reference: NASA-GB-8719.13, NASA Software Safety Guidebook] Software reuse: The application of software to a problem other than the specific original problem the software was implemented to solve. Systematic reuse: Planned reuse. Specific content is identified as reusable in a specific location. Then the content management system automatically inserts (auto-populates) the reusable content in the appropriate locations in the document. The author does not have to determine whether the reusable content exists or search for and retrieve it. Systematic reuse ensures that content is reused and reduces the burden on the author of knowing that reusable content exists, of finding the reusable content, and of inserting it appropriately. Verbatim reuse: Reuse without modifications. A.2 Acronyms and Abbreviations 3DOF ACE AFM AMMOS ANSI API AR&D AT & T C&DH C2I CASE tools CCC CEV CFI CLCS CM CMMI COCOMO COREM COTS CPU Three degrees of freedom An expert in a specialized area. Aerosciences and Flight Mechanics The Advanced Multi-Mission Operations System American National Standards Institute Application programming interfaces Automated Rendezvous and Docking American Telephone and Telegraph Company Command and data handling Command and Control Interoperability Study Computer-Aided Systems Engineering tools Command Control and Communications Crew Exploration Vehicle Call-for-Improvement Checkout and Launch Control System Configuration management Capability Maturity Model Integration Constructive Cost Model Capsule oriented reverse engineering for software reuse Commercial-Off-The-Shelf Central processing unit CONSTELLATION PROGRAM SYSTEMS ENGINEERING CONSTELLA NEERING Title: Strategy for Software Reuse Study Final Report CRAM CRIS CSA DEROS DoD DOPS/LANDOPTS DSM&A DSSA ECANS EDSAC ESD ESE FADS FDD FOM GDS GLS GN&C GOTS GSE GSFC HMF HQ IDT ISHM ISS IT ITAR JASON JSC KBEmacs KLOC LMS LOC MFC Document No.: AC1L1-52 Effective Date: October 1, 2005 Revision: 0 Page 97 Constellation Responsible Assignment Matrix Customer resource information systems Computers, Software and Automation Design Environment for Reusable Object-oriented Software US Department of Defense Desirable Objectives and Practices/ Design Synthesis Manufacturing & Assembly Domain-specific software architecture Exploration Communications and Navigation Systems Electronic Delay Storage Automatic Computer Electronic Services Delivery Earth Science Enterprise Flight Analysis and Design System [Shuttle] Flight Design and Dynamics Figures of merit Ground data system Ground Launch Sequencer Guidance, navigation and control Government Off-The-Shelf Ground Support Equipment Goddard Space Flight Center Hypergolic Maintenance Facility Headquarters Integrated Discipline Team Integrated Systems Health Management Study International Space Station Information Technology International Traffic in Arms Regulations Successor to TOPEX Johnson Space Center Knowledge-Based Editor in Emacs Thousand lines of code Launch and Mission Systems Lines of code Mated Flight Control Operations CONSTELLATION PROGRAM SYSTEMS ENGINEERING CONSTELLA NEERING Title: Strategy for Software Reuse Study Final Report MLSOC MOC MPS NASA NATO NC NDI OMS OMT OO OOSE OPF OS PAC ROI ROSE RSL RSL S/W SBA SDA SDLC SEEDS SEI SLOC SMP SPL SRR SRR STAR STL T&V TCM TOPEX TOTM Document No.: AC1L1-52 Effective Date: October 1, 2005 Revision: 0 Page 98 Million lines of source code Mission Operation Computer Main Propulsion System National Aeronautics and Space Administration North Atlantic Treaty Organisation Normally Closed Non-developmental items Orbital Maneuvering System Object Modeling Technique Object oriented Object Oriented Software Engineering Orbiter Processing Facility Operating system Presentation, Abstraction and Control Return on investment Reusable Object Software Engineering Reusable Software Libraries Traditional reusable software libraries Software Simulation-Based Acquisition Software Development Assets Software Development Life Cycle Strategic Evolution of Earth Science Enterprise Data System Software Engineering Institute Source lines of code Software Management Plan Software product line System Requirements Review. System Requirements Review Software Technology Standard Template Library Test & Verification Topography Experiment Trajectory-Off-the-MOC CONSTELLATION PROGRAM SYSTEMS ENGINEERING CONSTELLA NEERING Title: Strategy for Software Reuse Study Final Report V&V XML Document No.: AC1L1-52 Effective Date: October 1, 2005 Verification and validation Extensible Markup Language Revision: 0 Page 99 CONSTELLATION PROGRAM SYSTEMS ENGINEERING CONSTELLA NEERING Title: Strategy for Software Reuse Study Final Report Document No.: AC1L1-52 Effective Date: October 1, 2005 Revision: 0 Page 100 Appendix B: Bibliography “16 critical software management practices,” Software Managers Network, [Online document], 2005, [2005 Sep. 9], Available http://www.spmn.com/16CSP.html A. Abran and J. Desharnais, “Measurement of functional reuse in maintenance,” Journal of Software Maintenance: Research and Practice, vol. 7, no. 4, July–Aug., pp. 263–277, 1995. “Aerospace Industries Association Position Statement on Ada,” [Online document] July 12, 1996, [2005 Aug. 26], Available http://sw-eng.falls-church.va.us/aia_position_on_ada.html W. Agresti and W. Evanco, “Projecting software defects from analyzing Ada designs,” IEEE Trans. Software Engineering, vol. 18, no. 11, Nov., pp. 988–997, 1992. A. Albrecht, “Measuring application development productivity,” in Proc. IBM Applications Development Symposium, Oct. 1979, pp. 83–92. “Ariane 5 flight 501 failure [Online document] ND [2005 Aug. 30], Available http:// www.ima.umn.edu/~arnold/disasters/ariane5rep.html G. Asrar, “Data and knowledge system for mission to planet Earth,” Remote Sensing Reviews, vol. 13, pp. 1–25, 1995. J. Baldo, J. Moor, and D. Rine, “Software reuse standards,” Standard Views: ACM Perspectives on Standardization, vol. 5, no. 2, Jun., pp. 50–57, 1997. R. Banker, R. Kauffman, C. Wright, and D. Zweig, “Automating output size and reuse metrics in a repository-based computer aided software engineering CASE environment,” IEEE Trans. on Software Engineering, vol. 20, Mar., pp. 169–187, 1994. V. Basili and J. Musa, “The future engineering of software: a management perspective,” IEEE Computer, vol. 24, no. 9, Sep., pp. 90–96, 1991. T. Berners-Lee, R. Cailliau, J. Groff, and B. Pollermann, “World-Wide Web: the information universe,” Electronic Networking: Research, Applications and Policy, vol. 1, no. 2, Spring, pp. 74–82, 1992. T. Berners-Lee, R. Cailliau, A. Luotonen, H. Frystyk Nielen, and A. Secret, “The World-Wide Web,” Communications of the ACM, vol. 37, no. 8, Aug., pp. 76–82, 1994. B. Boehm, Software Engineering Economics, Englewood Cliffs, NJ: Prentice Hall, 1981. B. Boehm and P. Papaccio, “Understanding and controlling software costs,” IEEE Trans. Software Engineering, vol. 14, no.10, Oct., pp. 1462–1477, 1987. T. Bollinger and S. Pfleeger, “Economics of reuse: issues and alternatives,” Information and Software Technology, Dec., vol. 32, no. 10, pp. 643–652, 1990. G. Booch, Object Solutions, Managing the Object-Oriented Project, Menlo Park, CA: AddisonWesley, 1996. J. Carbonell, “Derivational Analogy: A Theory of Reconstructive Problem Solving and Expertise Acquisition,” in Machine Learning—An Artificial Intelligence Approach, Vol. II, R. Michalski, J. Carbonell, and T. Mitchell, Eds., San Francisco: Morgan Kaufmann Publishing, 1986, pp. 371–392. J. Carbonell and M. Harris, “Open systems architecture for legacy aircraft,” Open Systems Project Engineering Conference, [Online document], 1988, [2005 Aug. 30], Available http:// www.acq.osd.mil/osjtf/pdf/osala.pdf CONSTELLATION PROGRAM SYSTEMS ENGINEERING CONSTELLA NEERING Title: Strategy for Software Reuse Study Final Report Document No.: AC1L1-52 Effective Date: October 1, 2005 Revision: 0 Page 101 D. Card, and E. Comer, “Why do so many reuse programs fail?” IEEE Software, vol. 11, no. 9, Sep., pp. 114–115, 1994. S. Christensen, “Software reuse initiatives at Lockheed,” Lockheed Horizons, [Online document], Dec. 1994, [29 Aug. 2005]. Available http://www.stsc.hill.af.mil/crosstalk/1995/05/ Lockheed.asp A. Clark, The Biology of Business, San Francisco, CA,: Jossey-Bass, 1999. P. Clements and L. Northrop, Software Product Lines: Practices and Patterns, Boston, MA: Addison-Wesley, 2001. R. Cruickshank and J. Gaffney, “An Economics Model of Software Reuse,” in T. Gulledge and W. Hutzler (ed.), Analytical Methods in Software Engineering Economics, Springer-Verlag, 1993. Cusumano, M., Japan’s Software Factories: A Challenge to U.S. Management, New York: Oxford University Press, 1991. DO-178B/ED-12B, “Software considerations in airborne systems and equipment certification,” RTCA, Inc., Washington, DC, Dec. 1992. C. Donaldson, “InQuisiX: An electronic catalog for software reuse,” SIGIR Forum, vol. 28, no. 1, Spring, pp. 8–12, 1994. D. Eichmann, T. McGregor, and D. Danley, “Integrating structured databases into the Web: The MORE System,” in Selected Papers of the First Conference on World-Wide Web, Amsterdam, The Netherlands: Elsevier Science Publishers, pp. 281–288, 1994. D. Fafchamps, “Organization factors and reuse,” IEEE Software, Sep., pp. 31–41, 1994. M. Fayad, W. Tsai, and M. Fulghum, “Transition to object-oriented software development,” Communications of the ACM, vol. 39, no. 2, Feb., pp. 109–121, 1996. N. Fenton and S. Pfleeger, Software Metrics: A Rigorous Approach, New York: Chapman and Hall, 1991. A. Ferrentino, SNAP: The Use of Reusable Templates to Improve Software Productivity and Quality, Herndon, VA: Template Software, Inc., 1994. R. Fichman and C. Kemerer, “Object-oriented and conventional analysis and design methodologies, comparison and critique, IEEE Computer, vol. 25, no. 10, Oct., pp. 22–39, 1992. P. Fonash, “Metrics for reusable software code components,” diss, Fairfax, VA: George Mason University, 1993. W. Frakes, “Software reuse empirical studies,” in Software Reusability, W. Schafer, R. Prieto-Diaz, and M. Matsumoto, Eds. New York: Ellis Horwood, 1994. W. B. Frakes and C. Fox, “Sixteen questions about software reuse,” Communications of the ACM, vol. 38, no. 6, Jun., pp. 75–87, 1995. W. Frakes and S. Isoda, “Success factors of systematic reuse,” IEEE Software, Sep., pp. 15–19, 1994. W. Frakes and C. Terry, “Software reuse: metrics and models,” ACM Computing Surveys, vol. 28, no. 2, June, pp. 415–435, 1996, Available http://delivery.acm.org/10.1145/240000/ 234531/ p415-frakes.pdf?key1=234531&key2=6118136211&coll=GUIDE&dl=GUIDE& CFID=52602224&CFTOKEN=73012708 W. Gibbs, “Software’s chronic crisis, Scientific American, Sep., pp. 86–95, 1994. CONSTELLATION PROGRAM SYSTEMS ENGINEERING CONSTELLA NEERING Title: Strategy for Software Reuse Study Final Report Document No.: AC1L1-52 Effective Date: October 1, 2005 Revision: 0 Page 102 J. Gold, “Reusabilty promise hinges on libraries,” IEEE Software, vol. 13, no. 1, Jan., pp. 86–92, 1993. M. Griss and M. Wosser, “Making Software Reuse Work at HP,” Hewlett Packard Laboratories, [Online document], 1995, [2005 Sep. 9], Available http://www.hpl.hp.com/techreports/95/ HPL-95-04.html M. Griss, “Software reuse: from library to factory,” IBM Systems Journal, vol. 32, no. 4, Oct., pp. 548–566, 1993. IEEE std 1517-1999 IEEE standard for information technology—software life cycle processes— reuse processes—description [Online document], 1999, [2005 Aug. 31], Available http:// standards.ieee.org/reading/ieee/std_public/description/se/1517-1999_desc.html R. Fichman and C. Kemerer, “Incentive compatibility and systematic software reuse,” Journal of Systems and Software, vol. 57, no. 1, Apr., pp. 45–60, 2001. Function Point Counting Practice Manual Release 4.0, Westerville, Ohio: International Function Point User Group, 1994. J. Gaffney and R. Cruickshank, “A general economics model of software reuse,” in Proc. 14th International Conference on Software Engineering, pp. 327–337, New York: ACM Press, 1992. Institute for Defense Analysis, 1991. Islam, Choudhury, “A perspective on software reuse,” [Online document], ND, [2005 Sep. 9], Available http://www.scism.sbu.ac.uk/cios/islam/Reuse_p1.htm I. Jacobson, M. Griss, and P. Jonsson, “Making the reuse business work,” IEEE Computer, vol. 30, no. 10, Oct., pp. 36–42, 1997. R. Jensen Software Technology Support Center, “An economic analysis of software reuse,” CROSSTALK: The Journal of Defense Software Engineering [Online document], Dec. 2004, [2005 Aug. 29], Available http://www.stsc.hill.af.mil/crosstalk/2004/12/0412Jensen.html C. Jones, “The economics of object-oriented software,” American Programmer, vol. 7, no. 10, Oct., pp. 28–35, 1994. R. Joos, “Software reuse at Motorola,” IEEE Software, Sep., pp. 42–47, 1994. P. Kalagiakos, “The non-technical factors of reusability,” in Proc. 29th Euromicro Conference, p. 124, 2003. C. Kemerer, “An empirical validation of software cost estimation models,” Communications of the ACM, vol. 30, no. 5, pp. 416–429, 1987. D. Kiely, “Are components the future of software?” IEEE Computer, vol. 31, no. 2, Feb., pp. 1011, 1998. D. Lamb, Software Engineering: Planning for Change, Upper Saddle River, NJ: Prentice-Hall, 1988. B. Lampson, “How software components grew up and conquered the world,” [Online document], May 21, 1999, [cited 2005 Aug 30], Available http://research.microsoft.com/Lampson/ Slides/ ReusableComponentsAbstract.htm R. Leach, Software Reuse: Methods, Models and Costs, New York: McGraw-Hill, 1997. M. Lenz, H. Schmid, and P. Wolf, “Software reuse through building blocks,” IEEE Software, vol. 4, no. 4, Jul., pp. 34–42, 1987. N. Leveson, “The role of software in spacecraft accidents,” AIAA Journal of Spacecraft and Rockets, vol. 41, no. 4, July 2004. http://sunnyday.mit.edu/papers/jsr.pdf CONSTELLATION PROGRAM SYSTEMS ENGINEERING CONSTELLA NEERING Title: Strategy for Software Reuse Study Final Report Document No.: AC1L1-52 Effective Date: October 1, 2005 Revision: 0 Page 103 N. Leveson and K. Weiss, “Making embedded software reuse practical and safe,” ACM SigSoft 2004/FSE-12, [Online document], 2004, [2005 Aug. 30], Available http://sunnyday.mit.edu/ papers/fse04.pdf W. Lim, “Effects of reuse on quality, productivity, and economics, IEEE Software, vol. 11, no. 5, Sep., pp. 23–30, 1994. W. Lim, Managing Software Reuse, Upper Saddle River, NJ: Prentice Hall, 1998. Y. Maarek, D. Berry, and G. Kaiser, “An information retrieval approach for automatically constructing software libraries,” IEEE Trans. Software Engineering, vol. 17, no. 8, Aug., pp. 800–813, 1991. M. Matsumoto, “Research to shed light on software shadows,” IEEE Software, vol. 12, no. 5, pp. 16, 1995. C. McClure, The Three Rs of Software Engineering: Reengineering - Repository - and Reusability, Englewood Cliffs, NJ: Prentice-Hall, 1992. M. D. McIlroy, “Mass-produced software components,” Proc. NATO Software Engineering Conference, pp. 138–155, 1968. A. Meyer, “Software reuse: why you need a well managed repository,” Flashline Technology, [Online document] Feb. 2002 [2005 Aug. 30], Available http://www.flashline.com/Content/ Meyer/impact_analysis.jsp?&sid=1125442154273-2303618884-101 M. Morisio, M. Ezran, and C. Tully, “Success and failure factors in reuse,” IEEE Trans. Software Engineering, vol. 28, no. 4, pp. 340–357, 2002, Available http://csdl2.computer.org/ persagen/ DLAbsToc.jsp?resourcePath=/dl/trans/ts/&toc=comp/trans/ts/2002/04/ e4toc.xml&DOI=10.1109/TSE.2002.995420 E. Morris, L. Levine, C. Meyers, P. Place, and D. Plakosh, “System of Systems Interoperability (SOSI): final report,” Pittsburgh, PA: Carnegie Mellon University, CMU/SEI-2004-TR-004, April 2004. N. Nada, “Software reuse-oriented functional framework,” diss, Fairfax, VA: George Mason University, 1997. K. Obraczka, P. Danzig, and S.-H. Li, “Internet Discovery Services,” IEEE Computer, vol. 25, no. 5, Sep., pp. 8–22, 1993. J. O’Connor, C. Mansour, J. Turner-Harris, and G. Campbell, Jr., “Reuse in command-and-control systems,” IEEE Software, vol. 11, no. 5, Sep., pp. 70–79, 1994. D. Parnas, “On the design and development of program families,” IEEE Trans. Software Engineering, vol. 2, no. 1, Mar., pp. 1–9, 1976. F. Patterson, “Reengineerability: metrics for software reengineering for reuse,” diss, Fairfax, VA: George Mason University, 1995. S. Pfleeger, Software Engineering, The Production of Quality Software, Indianapolis, IN, Macmillan Publishing, 1987. S. Pfleeger, “Design and analysis in software engineering, part 3: types of experimental design,” Software Engineering Notes, vol. 20, no. 2, Oct. pp. 14–16, 1994. S. Pfleeger, “Design and analysis in software engineering, part 5: analyzing the data,” Software Engineering Notes, vol. 20, no. 5, pp. 14–16, 1994. S. Pfleeger, “An investigation of cost and productivity for object-oriented development,” diss, Fairfax, VA: George Mason University, 1995. CONSTELLATION PROGRAM SYSTEMS ENGINEERING CONSTELLA NEERING Title: Strategy for Software Reuse Study Final Report Document No.: AC1L1-52 Effective Date: October 1, 2005 Revision: 0 Page 104 T. Pole, “It’s all in the timing,” Flashline Technology, [Online document], ND, [2005 Aug. 30], Available http://www.flashline.com/Content/tpole/decisionModel.jsp? sid=1122760619257-1084354677–102 J. Poulin, “Populating software repositories: incentives and domain-specific software,” Journal of Systems and Software, vol. 30, no. 3, Sep., pp. 187–199, 1995. J. Poulin, “The economics of product line development,” International Journal of Applied Software Technology, vol. 3, no. 1, Mar., pp. 20–34, 1997a. J. Poulin, Measuring Software Reuse: Principles, Practices, and Economic Models, Reading, MA, Addison-Wesley Longman, 1997b. J. Poulin and K. Werkman, “Software Reuse Libraries with Mosaic,” Oswego, New York: Loral Federal Systems, [Online document], ND, [2005 Aug. 30], Available http:// archive.ncsa.uiuc.edu/SDG/IT94/Proceedings/DDay/werkman/www94.html J. Poulin and K. Yglesias, “Experiences with a faceted classification scheme in a large reusable software library (RSL),” Proc. Seventeenth Annual International Computer Software and Applications Conference, pp. 90–99, 1993. J. Poulin, D. Hancock, and J. Caruso, “The business case for software reuse,” IBM Systems Journal, vol. 32, no. 4, pp. 567–594, 1993. R. Pressman, Making Software Engineering Happen: A Guide for Instituting the Technology, Englewood Cliffs, NJ, Prentice Hall, 1988. Pressman, R., Software Engineering: A Practitioner’s Approach, Boston, MA: McGraw-Hill, 1992. “Product line systems program,” Software Engineering Institute, Carnegie Mellon University, [Online document], ND, [2005 Sep. 9], Available http://www.sei.cmu.edu/progams/pls/ pl_program.html R. Prieto–Diaz and P. Freeman, “Classifying software for reusability,” IEEE Software, vol. 4, no. 1, Jan., pp. 6–16, 1987. L. Putnam and W. Myers, Five Core Metrics: The Intelligence Behind Successful Software Management, New York: Dorset House Publishing, 2003. D. J. Reifer, Practical Software Reuse: Strategies for Introducing Reuse Concepts in Your Organization, IEEE Standard 1517, New York: John Wiley and Sons, 1997. “Report on independent research study of software reuse (using frame technology),” Pittsfield, MA: Quality Software Management, Inc., [Online document], Sep. 1994, [2005 Aug. 30], Available http://www.qsm.com/independent.pdf “Reusable software components,” Federal Aviation Administration Advisory Circular AC 20-148, [Online document], Dec. 7, 2004, [2005 Aug. 31], Available http://www.airweb.faa.gov/ Regulatory_and_Guidance_Library/rgAdvisoryCircular.nsf/ Reuse Adoption Guidebook, version 02.00.05, Herndon, VA: Software Productivity Consortium, SPC-92051-CMC, Nov. 1993. W. Riddle, “Instituting and maturing a practical reuse program: an interview with Dr. William Frakes,” International Perspectives in Software Engineering, vol. 1, Summer, pp. 18–26, 1993. D. Rine, “Supporting reuse with object technology,” special issue on software reuse processes and products,” IEEE Computer, vol. 30, no. 10, Oct., pp. 43–45, 1997. CONSTELLATION PROGRAM SYSTEMS ENGINEERING CONSTELLA NEERING Title: Strategy for Software Reuse Study Final Report Document No.: AC1L1-52 Effective Date: October 1, 2005 Revision: 0 Page 105 D. Rine and R. Sonnemann, “Investments in reusable software: a study of software reuse investment success factors,” Journal of Systems and Software, vol. 41, no. 1, Apr., pp. 17–32 1998. J. Rumbaugh, M. Blaha, W. Premerlani, F. Eddy, and W. Lorensen, Object-Oriented Modeling and Design, Englewood Cliffs, NJ: Prentice-Hall, 1991. J. Rushby, “Modular certification,” Washington, DC, National Aeronautics and Space Administration, NASA/CR-2002-212130, [Online document], Dec. 2002, [2005 Aug. 30], Available http://techreports.larc.nasa.gov/ltrs/PDF/2002/cr/NASA-2002-cr212130.pdf D. Schmidt, “Why software reuse has failed and how to make it work for you,” [Online document], ND, [2005 Aug. 31], Available http://www.cs.wustl.edu/~schmidt/reuse-lessons.html R. Selby, “Enabling reuse-based software development of large-scale systems,” IEEE Trans. Software Engineering, vol. 31, no. 6, June, pp. 495–510, 2005. “Services for Helping the Air-quality community use ESE data,” [Online document], ND, [2005 Sep. 9], Available http://capita.wustl.edu/ShAirED/ “Software approval guidelines,” Federal Aviation Administration Order 8110.49, [Online document], June 3, 2003, [31 Aug. 2005], Available http://www.airweb.faa.gov/ Regulatory_and_Guidance_Library/rgOrders.nsf/ “Software architecture for software-intensive systems,” Software Engineering Institute, Carnegie Mellon University, [Online document], ND, [2005 Sep. 9], Available http://www.sei.cmu.edu/ architecture/index.html “Software product line case studies,” Software Engineering Institute, Carnegie Mellon University, [Online document], ND, [2005 Sep. 21], Available http://www.sei.cmu.edu/productlines/ spl.case_studies.html “Software product lines,” Software Engineering Institute, Carnegie Mellon University, [Online document], ND, [2005 Sep. 9], Available http://www.sei.cmu.edu/productlines/index.html Software Reuse Major Issues Need to Be Resolved Before Benefits Can Be Achieved: Report to the Chairman, Subcommittee on Defense, Committee on Appropriations, House of Representatives (SuDoc GA 1.13:IMTEC-93-16), Washington, DC: Government Accounting Office, 1993. “Software Reuse Study Report: A Report for the NASA IV&V Facility, Fairmont, West Virginia,” [Fairmont, WV, 2005]. I. Sommerville, Software Engineering, 5th ed., New York: Addison-Wesley, 1996. I. Sommerville, “Software Reuse,” in Software Engineering, 7th ed., Addison-Wesley, 2004. R. Sonnemann, “Exploratory study of software reuse success factors,” diss, Fairfax, VA: George Mason University, 1995. W. Staringer, “Constructing applications from reusable components, IEEE Software, vol. 11, no. 5, Sep., pp. 61–68, 1994. “Strategic evolution of Earth Science Enterprise Data System (SEEDS) Formulation Team final recommendation report, Vol. I,” [Online document], Jul. 2003, [2005 Aug. 31], Available http:// seeds.gsfc.nasa.gov/SFormHome.html “Supporting Reuse with Object Technology,” special issue, IEEE Computer, vol. 30, no. 10, Oct. 1997. J. Tirso and H. Gregorious, “Management of Reuse at IBM,” IBM Systems Journal, vol. 32, no.4, pp. 612–615, 1993. CONSTELLATION PROGRAM SYSTEMS ENGINEERING CONSTELLA NEERING Title: Strategy for Software Reuse Study Final Report Document No.: AC1L1-52 Effective Date: October 1, 2005 Revision: 0 Page 106 W. Tracz, “Domain-specific software architecture frequently asked questions, Software Engineering Notes, vol. 19, no. 2, April, pp. 52–56, 1994. W. Tracz, Confessions of a Used Program Salesman: Institutionalizing Software Reuse, Addison Wesley Longman, 1995. W. Ulrich, “Collaborative business & IT infrastructures: creating a culture of reuse,” Flashline Technology, [Online document], ND, [2005 Aug. 31], Available http://www.flashline.com/ Content/Ulrich/reuse_culture.jsp?sid=1122760619257-1084354677-102 M. Wasmund, “Implementing critical success factors in software reuse,” IBM Systems Journal, vol. 32, no. 4, pp. 595–611, 1993. J. Withey, “Implementing model based software engineering in your organization: an approach to domain engineering,” Pittsburgh, PA: Carnegie Mellon University Software Engineering Institute,. Tech. Rep.CMU/SEI-93-TR, November 1993. K. Yglesias, “Limitations of certification standards in achieving successful parts retrieval,” Proc. 5th International Workshop on Software Reuse, 1992. CONSTELLATION PROGRAM SYSTEMS ENGINEERING CONSTELLA NEERING Title: Strategy for Software Reuse Study Final Report Document No.: AC1L1-52 Effective Date: October 1, 2005 Revision: 0 Page 107 Appendix C: Study Team Participants C.1 Participants David Atkinson (JPL, Study Lead) Anil Agrawal (JPL) Charles Beswick (JPL) Lisa Blue (MSFC) Tom Cochrane (ARC) Keith Cornett (MSFC) Len Day (JPL) S. Ellenberger (JSC) M. Hinchey (GSFC) Michael Izygon (JSC) Mark L. James (JPL) Patricia Langwost (KSC) Michael Lowry (ARC) Jeff Lubelczyk (GSFC) Huyen-Anh “Bebe” Ly (JSC) L. Markosian (ARC) P. Miner (LaRC) Eugene Miya (ARC) Thomas Pressburger (ARC) Steven Purinton (MSFC) Kirk Reinholtz (JPL) Luis Trevino (MSFC) Robert Waterman (KSC) Harry Woo (JPL) CONSTELLATION PROGRAM SYSTEMS ENGINEERING CONSTELLA NEERING Title: Strategy for Software Reuse Study Final Report C.2 Document No.: AC1L1-52 Effective Date: October 1, 2005 Study Team Bios D. Atkinson JPL, Study Lead Revision: 0 Page 108 CONSTELLATION PROGRAM SYSTEMS ENGINEERING CONSTELLA NEERING Title: Strategy for Software Reuse Study Final Report Document No.: AC1L1-52 Effective Date: October 1, 2005 Revision: 0 Page 109 Dr. David J. Atkinson is Program Manager nager of the Cons Constellation Systems Engineering Support Office in JPL’s Exploration Systems and nd Technology Office O where he works with NASA’s Project Constellation. As Program Manager, nager, he is responsible re for all of JPL’s systems engineering support to the Constellation on Program Systems S Engineering. Additionally, David serves as the Chair of the “Computers,, Software and Automation Integrated Discipline Team.” As the Chair of this group, he leads a multidisciplina ultidisciplinary group of experts from across NASA to develop the foundational systems engineering neering in this thi discipline area for Project Constellation. Previously, Dr. Atkinson was Deputy Division ivision Mana Manager of the Information Technologies and Software Systems Division at NASA’s Jet Propulsion L Laboratory, California Institute of Technology. As deputy manager of JPL’s ’s lead organization organi for computing, software engineering and information technologies, he was responsible for oversight ove of the development, delivery and operations of major information systemss and software softwa for JPL’s flight projects and Deep Space Network as well as advanced computing ng and information inform systems for other government agencies. Recent major deliveries by the he Division have ha included software architecture and engineering for the on-board flight software oftware systems system on the Mars Exploration Rover (MER) project, the mission control centers for MER Entry, Descent D and Landing, Engineering, and Science Operations, and major software re system upgrades upgra to the uplink, monitor and control and ground data systems of the Deep Spacee Network. Earlier at JPL, Dr. Atkinson headed the Section responsible respons for development of key software subsystems of the AMMOS multi-mission ion operations system. As a Section Manager, he also oversaw the development of both mission-specific ion-specific and a multi-mission science data systems, such as the Topex SDS and the Planetary Data ata System. Throughout T his career at JPL, Dr. Atkinson has had additional duties as a Program Manager. In collaboration c with NASA HQ and Centers, he supervised NASA technology R&D at JPL and numerous num other institutions in such areas as autonomous systems, operations automation, ation, artificial artific intelligence, and robotics. He has also participated in mission studies, including ng the Mars Rover Rove Sample Return, Europa and Titan missions. Dr. Atkinson is an expert in systems autonomy utonomy and automation, a a notable early contribution being the invention of the SHARP (now SHINE) system sys which provided real-time monitoring and diagnosis operations support for Voyager oyager and Magellan M telecommunications as well as the Galileo power subsystems. Dr. Atkinson on has received receive the NASA Exceptional Service Medal as well as numerous NASA Group Achievement vement Awards Awards, Space Act Awards, and multiple recognitions by the NASA Inventions Board. Dr. Atkinson At is a member of the IEEE, AAAI, and ACM, and has over 45 publications. Hee serves as a Member M of the Industrial Relations Board for California State University at San Luis uis Obispo (CalPoly) (Ca Computer Engineering Program. Dr. Atkinson holds a Ph.D in Computerr Systems Engineering Engi from Chalmers University of Technology, Göteborg, Sweden (where he also served serve as a Visiting Scientist) the M.S. and M. Phil degrees in Computer Science from Yale University Univers and the B.A in Psychology from the University of Michigan. CONSTELLATION PROGRAM SYSTEMS ENGINEERING CONSTELLA NEERING Title: Strategy for Software Reuse Study Final Report Document No.: AC1L1-52 Effective Date: October 1, 2005 Revision: 0 Page 110 A. Agrawal JPL Dr. Anil Agrawal is Member of Technical cal Staff in the Ground Data System section at Jet Propulsion Laboratory. He has overr 25 years experience experi in the area of software development, system engineering and development of satellite ground data da systems. Dr. Agrawal was responsible for developing oping the major ma segments of the ground data system for TOPEX and JASON project. Currently y he is providing providi technical support for the NASA’s Project Constellation Computers Software and nd Automation Integrated Int Discipline Team (CSAIDT) in the area of risk management, software reusability usability and automation. a He received his Ph.D. in Electrical Engineering ngineering from Southern Methodist University. C. Beswick JPL Dr. Charles Beswick has managed large ge IT organizations organiza at the Jet Propulsion Laboratory for 20 years. Most recently, he managed a 400 person section sect that builds, maintains, and operates the Lab’s computing infrastructure: networks, orks, workstations, workstat desktop applications, and supercomputing. He also served as manager nager of the IT Partnering Office (managing $400Million worth of IT support contracts). Dr. Beswick was Deputy Division Manager nager of the Information Inform Systems Division (750 people, $120M budget) for 10 years. In this capacity, pacity, he directed dire the implementation and operation of large ground data systems, telecommunication unication systems, syste and administrative computing systems. He led the first re-evaluation of the Deep ep Space Network Net information systems architecture in more than twenty years and co-led the development of the JPL IT Strategic Plan. Prior positions at JPL include serving ng as technical section s manager of two different sections, one focused on artificial intelligence, science nce data systems, syste and software engineering (JPL software development standards) and the otherr on building large la military simulation and decision support systems for DOD. Dr. Beswick managed ged the developm development of the Corps Battle Simulation (CBS), which was featured on the cover over of TIME and made m major contributions to Desert Storm and Iraqi Freedom. Prior to joining JPL, Dr. Beswick held d a faculty position pos at the University of South Carolina (Assistant Professor of Management Science) cience) and worked w for Olin Corporation as a systems engineer (information systems). Dr. Beswick received the NASA Exceptional ptional Service Servic Medal in 1986 for “exceptional achievements in large scale simulation, on, computer graphics, gra and artificial intelligence.” L. Blue MSFC CONSTELLATION PROGRAM SYSTEMS ENGINEERING CONSTELLA NEERING Title: Strategy for Software Reuse Study Final Report Document No.: AC1L1-52 Effective Date: October 1, 2005 Revision: 0 Page 111 Ms. Lisa Blue has 22 years of experience nce in Flight Computers Com and Data Systems. Her experience includes being the responsible ble Avionics Lead L Engineer for the International Space Station Nodes 2 & 3 Project as well as the ISS Command Com and Data Handling (C&DH) system expert at MSFC. She was responsible for the Flight Test Telemetry System for the Proto-type Clipper-Graham vehicle, where she also o dealt with human hum rated requirements for evolving the Clipper-Graham into a space vehicle. T. Cochrane ARC CONSTELLATION PROGRAM SYSTEMS ENGINEERING CONSTELLA NEERING Title: Strategy for Software Reuse Study Final Report Document No.: AC1L1-52 Effective Date: October 1, 2005 Revision: 0 Page 112 Mr. Tom Cochrane is co-lead and systems ms architect for SimStation, S a virtual vehicle that is a knowledge-management system for the he International Internationa Space Station. SimStation is a 3D CADbased visualization model integrated with ith functional, functiona behavioral and procedure models, with access to multiple data resources and streaming treaming telemetry tele and with interactive crew avatars. It is intended to assist vehicle system engineers, neers, flight operations ope and crew, as they research and evaluate design and operational tradeoffs. offs. The technology technol is being expanded to future NASA exploration vehicles. Mr. Cochrane’s work has included participation rticipation in developing de the structure for NASA’s high performance computing and communications cations projects, projec next generation space transportation systems, and on a project team to design gn a spatial modeling m system for the National Airspace Management System. He played a leading ding role in a Space S Computing Working Group chartered by NASA headquarters to evaluate NASA’s SA’s future needs ne for computing in space. He also participated in a JSC committee to evaluate luate on-orbit on-orbi failures of Command and Control computers on the International Space Station. tation. Mr. Cochrane is a pathfinder in knowledge-integrat edge-integrating systems for aerospace computing and geographic information systems and nd has made numerous num presentations on Capital Hill, the Pentagon, NASA, DOE, EPA, DoD, and nd DOI. He ha has been responsible for developing systems ranging across many application areas,, including: energy e and aerospace, federal, state and local environmental and natural resources management, anagement, educational e computer games, automotive manufacturing, and motion pictures production. Mr. Cochrane worked for several years for DOI as Interior Int Basins RS/GIS Coordinator; designing and managing remote sensing, ng, GIS and database da systems for natural resource planning. Responsibilities include conducting onducting remote remot sensing analysis of habitat, biological, hydrologic and other resources. His work ork included constructing c and maintaining RS/GIS layers and databases throughout the interior or basins region of the t western United States. Mr. Cochrane served as systems integration gration manager manage for the congressionally mandated SERDP Program (Strategic Environmental Resources ources Development Devel Program), a joint DoD/DOE/EPA life-cycle-analysis program for agile, clean, lean, manufacturing manufa technology. In this capacity he acted for DOE headquarters as the technical liaison between betwe software development teams from several national laboratories. Mr. Cochrane served as technical reviewer wer for DOE headquarters of the Integrated Computer Control System (ICCS), for the National onal Ignition Facility Fa (NIF) at Lawrence Livermore Laboratories (LLNL), for generating ng fusion events in i the laboratory. He contributed system architecture design concepts, and served d as technical technica liaison for the LLNL NIF ICCS team. Mr. Cochrane has supported U.S. EPA and U.S. DO DOJ with modeling and data quality management for contaminated Superfund rfund sites. He produced produc large 3D scientific visualizations for a number of complex SUPERFUND D sites, and worked w as a technical reviewer for scoring national priority list sites. He also performed rformed an extensive ext evaluation for U.S. EPA on the data management systems and GIS software re tools used within w the SUPERFUND program. CONSTELLATION PROGRAM SYSTEMS ENGINEERING CONSTELLA NEERING Title: Strategy for Software Reuse Study Final Report Document No.: AC1L1-52 Effective Date: October 1, 2005 Revision: 0 Page 113 K. Cornett MSFC Mr. Keith Cornett began here at MSFC C in 1980. Initially, Ini his work involved Spacelab Mission Operations and he was a member of two o flight ops teams t that conducted the on-orbit payload operations. After these two missions,, he transitioned over ove to development of mission operations ground systems. He worked specifically ly with the verification ve of the integrated ground systems development activities (hardware and nd software) for the t Spacelab Payload Operations Control Center and the Shuttles Engineering Support upport Center in i the HOSC. The tasks eventually transitioned into actual software and nd hardware development deve of these systems. Mr. Cornett became Branch Chief in n 1990 and the branch bra had the responsibility to build the Chandra Operations Control Center (OCC) CC) and the International Int Space Station Payload Operations Integration Center (POIC) IC) and the United Unite States Operations Center (USOC). The group was responsible for the system engineering ngineering and a the hardware/software development of these systems. In 1993, he became Division Chief responsible ponsible for the t activities above coupled with the facility development activities necessary to accommodate commodate these t systems. In late 2000, he became Deputy Project Manager of the Interim Control Module Modul project and within 6 months became the Project Manager. In 2002, Mr. Cornett was part of the SLII program office of and developed program requirements for software and for mission operations. ons. After the SLI S Program transitioned to the Orbital Space Plane (OSP) Program, he became a member mber of the OSP O Acquisition Management Office and developed various components of the OSP SP RFP, including inc software and mission operations requirements. L. Day JPL CONSTELLATION PROGRAM SYSTEMS ENGINEERING CONSTELLA NEERING Title: Strategy for Software Reuse Study Final Report Document No.: AC1L1-52 Effective Date: October 1, 2005 Revision: 0 Page 114 Mr. Len Day is presently a senior software are engineer at a the Jet Propulsion Lab. Recent accomplishments include the leadership p of the team that delivered boot and filesystem code to the Deep Impact project and boot code to the Mars Reconnaissance Re Orbiter project. He also acted as a system engineer and led the low-level software soft team for the JPL Multi-Mission System Architecture Platform. He is currently urrently the System S Engineer/Chief Engineer for an Integrated Vehicle Health Management nt technology demonstration de task that is being done in cooperation by JPL, Ames Research h Center and Dryden Dryde Flight Research Center. Mr. Day’s first involvement with a space ce flight project proj was as a member of the operations team for the Viking Mars project. Since then n he has participated partic in a number of other flight projects including Seasat, MILSTAR, Galileo and nd Cassini. He H has also participated in problem resolution teams for a number of other flight projects ects including MISR, M QuickSCAT, Genesis and MER and has three times received the Space Act Award for significant si contribution to the space program. Mr. Day’s career has not been limited d to Aerospace and a he has worked for a number of companies through his career including ng Printrak, Inc. Inc (fingerprint matching systems), Hughes (factory automation), Networking Dynamics ynamics (FAX applications) a and he did the computer system for the first Cyber Café in Los Angeles County. Count He brings a broad experience base at all levels of software engineering from GUI UI user interfaces interfa through systems-level and device drivers. He has performed in all phasess of software development de including systems engineering, architecture development, software development, velopment, test/validation te and technical management. Mr. Day has a particularly strong background kground in embedded embedde systems and space applications. S. Ellenberger JSC M. Hinchey GSFC Dr. Michael G. Hinchey is currently Director irector of the NASA Software Engineering Laboratory, located at Goddard Space Flight Center. r. Prior to joining joi the US Government, he held academic positions at the level of Full Professor or in the USA, UK, U Ireland, Sweden and Australia. He is the author of move than 120 technical papers, pers, and 15 books. books His current research interests are in the areas of formal methods, system correctness, ctness, and agent-based age technologies. Dr. Hinchey is a Senior Member of the IEEE, a Fellow Fell of the IEE and the British Computer Society. He is a Chartered Engineer,, Chartered Professional Profe Engineer, Chartered Information Technology Professional and Chartered d Mathematician. Mathematic Dr. Hinchey is currently Chair of the IEEE Technical Committee on Complexity xity in Computing, Comput and is the IEEE Computer Society’s voting representative to IFIP TC1, C1, for which he has been elected Chair for 2006 to 2008. He is also Vice Chair of the IEEE Computerr Society Technical Techni Activities Board and of the newly established IEEE Task Force on Autonomous onomous and Autonomic A Systems. Dr. Hinchey received the Ph.D. in Computer puter Science Scienc from University of Cambridge, UK, the M.Sc. degree in Computation from University niversity of Oxford, O UK, and the B.Sc. degree in Computer Science from University of Limerick, k, Ireland. M. Izygon JSC CONSTELLATION PROGRAM SYSTEMS ENGINEERING CONSTELLA NEERING Title: Strategy for Software Reuse Study Final Report Document No.: AC1L1-52 Effective Date: October 1, 2005 Revision: 0 Page 115 Dr. Michel Izygon came to the Johnson on Space Center Cent as a post-doctoral research scientist under the National Research Council program in 1990. Since Sinc then he has continued to work at JSC toward the insertion of advanced software are technologies technologi into operational programs. Today he is the CTO and co-founder of Tietronix Software, oftware, a small sm business participating in multiple projects at NASA/JSC. As a research scientist, Dr. Izygon researched arched ways to use artificial intelligence technologies to solve NASA-JSC problems. His responsibilities ponsibilities inc included the development of an intelligent user interface for complex scientific programs, ms, the use of Artificial Intelligence in Software Engineering, and development of a Visual ual Programming Program tool for a Reuse-based Domain Engineering Environment. Dr. Izygon served as a software technology hnology consultant consulta in the ROSE project which aim was to reengineer millions of lines of code supporting upporting the Trajectory Tra simulation of the JSC Mission Control Center. His support to the ROSE E project included inc overall software architecture development, training of more than n one hundred engineer engi on Object-Oriented Methods, use of the Object Modeling Technique (OMT), ), development developme of Software Development Processes based on the OO concept, customization on of the CASE CAS Tool, evaluation of tools and Class Libraries, and development of a Pen-based n-based application applicat for group activity support. He acted as project lead on the development of an n Intelligent Computer Com Aided Training system for the OMT method, as well as for the development nt of a simulation simulat tool aimed at verifying and validating OMT models. Dr. Izygon was in charge of the insertion on of advanced advance software technology at the NASA-JSC. His responsibilities included the use of Object-Oriented Object-Orie Methods to reengineer Mission Control Center software, training of two Rockwell ell groups on the t Object Modeling Technique (OMT) method, assessment of Object-Oriented d Analysis and Design D Methods for software development, development of a tool to o support Total Tota Quality Management method at JSC, software reengineering methods and tools ools evaluation. Dr. Izygon served as technical lead in n the Web-based Web-base electronic workflow system, which aimed at deploying a number of workflow automating utomating different diffe mission critical processes of the NASA —Johnson Space Center. These include ude the Space Station S Procedures Change Request System, the Space Shuttle Flight Rule Change Request System, Syste and the Internal Proposal Technical Assessment Process of the NASA—JSC C Mission O Operation Division. Dr. Izygon acted as the main software architect of the Roboticss Situational Awareness Display, an on-board software that helps the astronauts manipulates the he robotics arm on the Space Shuttle. M. James JPL CONSTELLATION PROGRAM SYSTEMS ENGINEERING CONSTELLA NEERING Title: Strategy for Software Reuse Study Final Report Document No.: AC1L1-52 Effective Date: October 1, 2005 Revision: 0 Page 116 Mr. Mark L. James is a member of the Advanced Computing Com Algorithms and IVHS Group in the Flight Software and Data Systems Section tion within NASA’s N Jet Propulsion Laboratory, Pasadena, CA. At JPL Mr. James is Principal Investigator and Task Manager in real-time inference and knowledge-based systems. His primary ry research focus foc is on high-speed inference systems and their application to planetary and exploration oration and military m systems, and, automatic software debugging of large software systems. Mr. James has received a number of JPL L and NASA awards which include: Space Act Award for 2005, Space Act Award for 2003, Major or Scientific Achievement A Award for 2002, NOVA award for Leadership in 2000, Team Excellence nce Award for 1990, JET J Productivity Award for 1990, Major NASA Monetary Award rd for 1990, NASA Ce Certificate of Recognition for 1990, NASA Group Achievement Award for 1990, NASA ASA Exceptional Except Service Medal for 1989, NASA Exceptional Service Plaque for 1989, NASA ASA Manned Manne Flight Awareness award for 1989 and NASA Manned Flight Awareness award rd for 1988. He is author of 1 patent, 8 provisional patents, 41 scientific sc and technical papers, and 17 technical manuals. Mr. James has over thirty years experience ence in design, implementation, i and integration of large software systems with core competencies ies in applied computer c science and core artificial intelligence technology. His special areas as of expertise experti include: artificial intelligence technology, high-speed real-time expert systems, image age processing process and scene analysis, pattern recognition, planning and simulation, real-time control ontrol systems, programming progra languages and compilers, symbolic mathematics, operating systems, ms, programming program environments, robotics, virtual reality systems, with additional expertise in medicine, edicine, biology, biol and analog and digital electronics. Prior to coming to JPL Mr. James was the former Chief Chi Scientist and Board Member of System Cognition Corporation (now Inference Corporation). He H contributed to the development of the MIT LISP Machine when it was commercialized ercialized as the LMI LISP Machine. He contributed to the development of Symbolic Manipulation ation Program (SMP), later to become Mathematica. He also held a consulting position at Hughes ughes Research Lab, L Systems programmer at the University of Southern California, Researcher at the he Information Sciences S Institute (ISI) and consultant to XEROX PARC. Mr. James worked with Dr. David Atkinson kinson in the invention inve of the SHARP (now SHINE) system which provided real-time monitoring toring and diagnosis di operations support for Voyager and Magellan telecommunications as well as the Galileo power pow subsystems. He is co-author of STAR*TOOL, a sophisticated knowledge dge representation represent system. He is co-author of SHINE which has been licensed to numerous companies ompanies involving invol applications that include business forecasts, biotech, robotics, smell detection, ction, etc. He is author of the Programmers’ Assistant, an automatic debugging tool that locates anomalies in software. s P. Langwost KSC CONSTELLATION PROGRAM SYSTEMS ENGINEERING CONSTELLA NEERING Title: Strategy for Software Reuse Study Final Report Document No.: AC1L1-52 Effective Date: October 1, 2005 Revision: 0 Page 117 Ms. Patricia Langwost received a BS in n Electrical Engineering E with a minor in Space Studies from the University of North Dakota and nd a MS in Industrial Indus Engineering from the University of Florida. Ms. Langwost has worked in the ISS Payload ayload Proce Processing Directorate developing video and audio requirements, test procedures,, problem reports, report trouble-shooting plans, and interface drawing between GSE and flight hardware are for Multi-Element Mult Integration Test 3 (MEIT 3 included Node 2, Japanese Experiment nt Module (JEM), (JE and the Flight Emulator), Node 2 Standalone test, and Node 2 Traveled Work ork Systems System Test (TWST). She was the KSC Experiment Engineer for Space Dynamically ically Responding Respondi Ultrasonic Matrix System (SpaceDRUMS) and Expedite the Processing essing of Experiments Expe to the Space Station (EXPRESS) Engineer for ULF1.1 Mission. She wass also the Electrical Ele Power Distribution and Control (EPD&C) lead for the Orbital Space Plane lane (OSP) Launch L Site Integration Office. M. Lowry ARC CONSTELLATION PROGRAM SYSTEMS ENGINEERING CONSTELLA NEERING Title: Strategy for Software Reuse Study Final Report Document No.: AC1L1-52 Effective Date: October 1, 2005 Revision: 0 Page 118 Dr. Michael Lowry is the area lead d for Automated Software S Engineering within the Computational Sciences Division and nd is also the project proj manager for Automated Software Engineering Technologies within the ITSR program. program He serves as the NASA lead for the HighDependability Computing Program, and nd is the NASA NAS program manager for the joint NSF/ NASA HDCSSR program. Dr. Lowry’s main technical interest is in applying automated a reasoning to Software Engineering. After joining NASA Ames in 1992, he was the principle princ investigator on the Amphion project. Amphion automates the construction on of programs from domain-oriented user specifications by composing subroutines from a reuse library. brary. Amphion Amphi is a generic architecture which is specialized to a new domain by developing oping a declarative declara domain theory. Domain theories for several NASA applications have been n successfully built, bui including Computational Fluid Dynamics, Space Observation Geometries, ries, and Space Spa Shuttle Navigation. Meta-Amphion was developed in 1995 as a means of generating rating additional additiona Amphion applications, receiving the best paper award at the Knowledge-Based d Software Engineering Engi conference. Dr. Lowry has since lead his group in n developing innovative innova applications of automated reasoning for software engineering based ed on experience experie with NASA domains. This includes software model checking (2003 TGIR IR award), the before/after be experiments with Deep-Space one, and new thrusts such as certifiablee program synthesis synt and automated testing. Under his leadership, the automated software engineering ngineering group has ha grown from three people to fifteen people, and has developed collaborations ons with a number num of NASA application projects including Space Shuttle SAFM, JPL Planetary Data ata Systems, JPL J MER and MSL projects, and NASA Ames aviation capacity projects. He recently cently organized organi a NASA-wide workshop in April 2004 on reliable software, involving both NASA ASA centers and a the chief engineers office. Dr. Lowry is a member of the editorial board for the Journal of Automated Software Engineering (published by Kluwer), and nd was the program progra chair of the Automated Software Engineering conference held in fall 1997 (formerly KBSE'97). K He is co-editor of the book Automating Software Design (1991) available vailable from MIT Press, and was the principle author of a tutorial on knowledge-based software re engineering that t was part of the Handbook of Artificial Intelligence. Lowry has reviewed d proposals for NASA, NA Darpa and NSF, and is currently the technical advisor for NASA Ames on many any university univers grants on software engineering technologies. He has served on numerous rous conference conferenc program committees. Dr. Lowry received his MS/BS from MIT IT and his PhD P in 1989 from Stanford University in computer science. His PhD dissertation, on, “Algorithm Synthesis through Problem Reformulation,” focused on formalizing ng and automating automa algorithm design and problem reformulation/abstraction. From 1989 to 1992 he was w a computer scientist at the Kestrel Institute, and was principle investigator or on several projects proj related to AI and Software Engineering. During his graduate studies udies he also did research re in image understanding and speech understanding, and digital signal gnal processing. J. Lubelczyk GSFC CONSTELLATION PROGRAM SYSTEMS ENGINEERING CONSTELLA NEERING Title: Strategy for Software Reuse Study Final Report Document No.: AC1L1-52 Effective Date: October 1, 2005 Revision: 0 Page 119 Mr. Jeff Lubelczyk has more than n 16 years of NASA experience working with software, systems engineering, and project management on a variety of programs progra ranging from technology assessment and insertion into operational onal systems (such (s as spacecraft operations and the space network) to science systems development. ent. He currently curre supports multiple Earth science related missions including NPOESS/NPP and nd LDCM, and ha has begun supporting NASA’s exploration initiative. Mr. Lubelczyk holds an M.S. in electrical cal engineering engineeri from the Johns Hopkins University. H. “Bebe” Ly JSC Ms. Huyen-Anh “Bebe” Ly has worked d at NASA Johnson Johns Space Center since January 1986. Ms. Ly has worked on many software projects which whic range from artificial intelligent projects to flight support software systems development. opment. She developed de the X-window Graphical User Interface for the Expert System Shell,, C Language Integrated Int Production System (CLIPS, the knowledge capture tool named Task Analysis/ nalysis/ Rule Generation Tool (TARGET), and CLIPS expert system Verification and validation on tool, Cross Cros Reference & Verification System (CRVS). Ms. Ly was the project lead of the Onboard nboard Short Term T Plan (OSTP) and Onboard Data File (ODF) Crew Interface (OOCI) project,, which provided provi the software package for the Internal Space Station Crew and the flight controller roller team to view vi the manual procedures and the onboard short term plan. Ms. Ly also managed anaged the High H Volume Data Management Project which developed an advanced set of tools ools to support distributed handling of documents and the integration of electronic tools for authoring, horing, revising, reviewing, re controlling, and releasing operations books controlled by the system. tem. This development deve includes automation of the process definition engine used to build d process flows. flow L. Markosian ARC Mr. Lawrence Markosian is a Computerr Scientist employed em by QSS Group, Inc. at NASA ARC, where he led a team developing practical al model checking che tools based on Java PathFinder. Over the past three years he has been active in the NASA Software Engineering Initiative’s Research Infusion activity, where he is the contractor ractor lead on a process for tech transfer of software engineering research into NASA development opment organizations. organi More recently he has contributed to the Constellation ISHM study, focusing ing on V&V of ISHM. Prior to joining NASA, Mr. Markosiann held various positions at Reasoning, Inc. where, as VP of Applications Development, he managed d technology transfer t of advanced software engineering tools for software synthesis, transformation, ation, and defect defe detection. He has authored many presentations and publications on code synthesis, reengineering, re and software defect detection. Mr. Markosian has an undergraduate degree in mathematics mathe from Brown University and has done graduate work at Stanford University rsity in logic and a applications of artificial intelligence. P. Miner LaRC CONSTELLATION PROGRAM SYSTEMS ENGINEERING CONSTELLA NEERING Title: Strategy for Software Reuse Study Final Report Document No.: AC1L1-52 Effective Date: October 1, 2005 Revision: 0 Page 120 Dr. Paul Miner is a Senior Research Engineer ngineer in the Safety-Critical Avionics Systems Branch at NASA Langley Research Center. His primary research resea interests are fault-tolerant systems, design and digital systems, and design gn assurance using us formal methods. He is the principal architect of the Scalable Processor-Independent -Independent Design Des for Extended Reliability (SPIDER). Dr. Miner’s recent responsibilities have ve focused on formal form analysis and certification considerations for fault-tolerant integrated grated modular avionics. a He is an active participant on RTCA Special Committee 200, developing oping the document docum “Integrated Modular Avionics (IMA) Development Guidance and Certification on Considerations.” Considera Dr. Miner has been a NASA employee since 1989 and a holds a PhD in Computer Science from Indiana University. E. Miya ARC Most recently Mr. Eugene Miya worked d as one of NASA’s N liaisons/technical reviewers to the NASA Computers, Information, and nd Communications Technologies (CICT) Program Office on the Interagency Digital Libraries Initiative tive which begat be the Google search engine and other Internet. Mr. Miya is known as “the” Unix evangelist ngelist inside NASA. N This advocacy placed Unix on the Cray-2 supercomputer thus keeping ng Cray Research, Inc. Inc a viable firm (and contributing to the demise of competitor ETA Systems) as well as the early e adoption of networked workstations as other Ames people advocated for TCP/IP. /IP. This work consisted c of supercomputer and support processor acquisition, mass storage system ystem evaluation, evaluati and other systems metrics. Mr. Miya is known for starting the FAQ Q concept on the t Internet in the early 1980s. He maintains the world’s bibliographic database on parallel pa computing. He is Associate Editor for Publications and Software Review for the journal Scientific Programming, and moderator of the comp.parallel omp.parallel news ne group. Mr. Miya has served as staff on various ous ACM and IEEE IE Conferences as well as an ACM SIG chapter President. Mr. Miya’s work at JPL consisted of systems ystems work for S Seasat and VOIR as well as software conversion cost estimation for Voyagerr using software softw metrics. He is an ANSI Standards Committee member, Consultant with the he DOE and DTRA D and other agencies, and is a member of the US Antarctic Program (holderr of Antarctic Service Se Medal). Mr. Miya got online starting in n 1973, host #3 on the ARPAnet. T. Pressburger ARC CONSTELLATION PROGRAM SYSTEMS ENGINEERING CONSTELLA NEERING Title: Strategy for Software Reuse Study Final Report Document No.: AC1L1-52 Effective Date: October 1, 2005 Revision: 0 Page 121 Mr. Thomas Pressburger is in the Robust Software Engineering E group led by Dr. Michael Lowry at NASA Ames. He serves on the Amess Engineering Process P Group and is the alternate Ames representative to the Software Working ng Group. He led l the subgroup of the Software Working Group whose task it is to get mature software oftware engineering engine research used on NASA projects; 9 infusion projects either finished or are in progress, and a 4-5 more will be started in 2006. The subgroup received a “Best New Research” rch” award at OSMA’s Software Assurance Symposium in 2004. Mr. Pressburger’s expertise is in the area rea of software engineering; in particular, automatic code generation. He was a principle programmer mer of the Amphion A system developed under Dr. Lowry at Ames which automatically constructed ted programs from specifications that solved solar system geometry problems, making use of a reusable usable software softw library called SPICE from JPL’s NAIF group. He started as a contractor at Ames es in 1992 and became be a civil servant in 2000. Previously, he worked at the Kestrel Institute and Reasoning, Re Inc. on program synthesis and transformation, and software reengineering. ring. He holds a B.S. in Mathematics from CalTech and an M.S. in Computer Science from Stanford. nford. S. Purinton MSFC Mr. Steven Purinton is the Team Lead, d, Software System Sys Engineering (2+ Years): In this capacity Mr. Purinton performs requirements allocation, location, analysis, ana preliminary schedule development, metric determination, and trade studies in addition to t insight for embedded software programs at MSFC. Project Engineer (2+ Years): Mr. Purinton nton served as a subsystem manager for Command & Data Handling, Guidance Control and Navigation, gation, and Software Soft Integration in the International Space Station Propulsion Module Project, ect, an element eleme of the International Space Station Program and Software Systems in the Chandra Program. Progra Chief Engineer (11 Years): Mr. Purinton on served in Chief Chi Engineer Office for the Chandra Observatory, Chandra Science Centerr and the Aero assist a Flight Experiment with oversight responsibilities for end-to-end system engineering of all a flight software, flight software support systems, ground support equipment software, oftware, and electrical e ground support equipment. Developer (22 Years): Mr. Purinton developed veloped simulator simul environments for Space Telescope, Solar Magnetograph, X-ray Telescope,, SRB thermal therma protection system and the Spacelab Payload Crew Training Complex. He converted onverted and developed de real-time operating systems including multiprocessors. He developed oped a robot control cont program and models for rocket engines and rocket engine controllers. He verified fied a Fortran compiler c and library for use in testing of the Space Shuttle Main Engine Controllerr software. K. Reinholtz JPL CONSTELLATION PROGRAM SYSTEMS ENGINEERING CONSTELLA NEERING Title: Strategy for Software Reuse Study Final Report Document No.: AC1L1-52 Effective Date: October 1, 2005 Revision: 0 Page 122 Mr. Kirk Reinholtz is the Project Software are Systems Engineer for the Mars Science Laboratory (MSL) mission. He is also the software re manager and Chief Chi Programmer for the Mission Data System (MDS), a large software and systems ystems engineering engine effort instituted at JPL to prepare the laboratory to meet the challenges of tomorrows omorrows advanced adva missions. He is the P.I. for several research efforts, including “Multi-view modeling and Analysis of Mission Data System Applications” and “Verifiable C++.” The he objective of the t research in Verifiable C++ is to reduce the cost of safety-critical software by allowing llowing productive produc programming languages to be applied in the safety-critical domain, or equivalently lently to allow larger systems to be developed for the same cost. The approach is based d upon the development developm of coding techniques that work in conjunction with analysis tools to allow deeper analysis anal for faults of much larger programs than would otherwise be possible. Mr. Reinholtz’s research interests include ude large-scale large-scal software engineering and verification techniques, and the prudent and nd productive application applicat of powerful programming languages and tools to large mission-critical and safety-critical y-critical software soft systems. Mr. Reinholtz has a Masters degree in Computer Science from the University niversity of Southern S California. L. Trevino MSFC CONSTELLATION PROGRAM SYSTEMS ENGINEERING CONSTELLA NEERING Title: Strategy for Software Reuse Study Final Report Document No.: AC1L1-52 Effective Date: October 1, 2005 Revision: 0 Page 123 Dr. Luis Trevino, Marshall Space Flight ght Center, is currently c an AI researcher in the Advanced Sensors and Health Management Systems ms Branch within w the Spacecraft and Vehicle Systems Department investigating new advanced d computing technologies t for Health Management Systems. Specifically he’s investigating ng advanced software s architectures and requirements for spacecraft systems. Other areas of research arch interest are a in prognostics, diagnostics, fault detection, mitigation strategies, and nd brain theory related rel to the development of intelligent machines. For the prior year he served d as lead for the Avionics Department’s Advanced Computing Office investigating applications ations of advanced adva computing techniques in MSFC projects. For the past three years Dr. Trevino hass served on NASA’s N Software Working subgroup for Software Technology Infusion involving ng members from Ames Research Center, JPL, Langley Research Center, and Goddard Space Flight light Center. This group independently researches technologies and uses available resources ources for furthering furthe the infusion of potential technologies into NASA software projects for improving proving the production produc of software. From mid-1999 to early-2004, he served as Software Design ign Team Leader Lea for the Flight Software Group within the Avionics Department. He served on a software oftware review revie board overseeing all embedded flight software projects and addressing issues. s. He helped the t Flight Software Group reach CMM Level III certification by representing ng the Design and Implementation Im Capability during the audit. For the prior 11 years, Dr. Trevino worked orked in the Propulsion Propul Laboratory at MSFC as a controls engineer for Technology Test Bed SSME E Testing and SSME S and X-33 Aero-Spike engine health management. Dr. Trevino holds a BSEE from Texas A&M and a MSEE M and a PhD EE from University of Alabama, Huntsville. He has applied d for a patent on “Implementing “ Bayesian Analysis Through Recursion” (for real time engine analysis ysis software) for rocket roc engine start-up. R. Waterman KSC CONSTELLATION PROGRAM SYSTEMS ENGINEERING CONSTELLA NEERING Title: Strategy for Software Reuse Study Final Report Document No.: AC1L1-52 Effective Date: October 1, 2005 Revision: 0 Page 124 Mr. Robert Waterman is responsible for Command and a Control, Avionics, Integrated Systems Health Management, and Software for the NASA KSC K Constellation Systems Office. Previously he was the technology area champion for supportability s of non-terrestrial system at the Kennedy Space Center. He was the Command and a Data Handling (C&DH) as well as the Flight Software lead for the Orbital Space pace Plane (OSP) (O Launch Site Integration Office and was an active member of the agency wide team eam for Integrated Integra Health Management (IHM). He was the Spaceport Technology Development nt Manager for command, c control and monitoring technologies where he led the KSC strategic rategic planning and a new business development activities in addition to addressing the high priority ority needs of the t center within the command, control and monitor thrust area. Previously, Mr. Waterman was the technical hnical lead for real-time re control software being developed for the ground Checkout and Launch h Control System (CLCS) where he was responsible for the technical direction of the application software oftware development devel and conflict resolution between the development organization and the operations rations organization. organi Mr. Waterman has fifteen years experience ence as the system sys specialist for the Space Shuttle Main Engine (SSME) avionics system where he was responsible respons for developing the certificate of flight readiness for the SSME avionics system and leading l the anomaly resolution activities during launch countdown. He was the NASA point of contact c for SSME Controller (SSMEC) software problem analysis and resolution, on, coordinating coordinat troubleshooting results with various NASA centers and vendors. Before joining ning NASA, Mr. M Waterman was with Lockheed Space Operations Company, where he was a launch certified certifi console engineer on the SSME avionics system. He was responsible for the day-to-day y-to-day maintenance maint and diagnostic testing of the SSME and related Space Shuttle Orbiter hardware. rdware. H. Woo JPL Mr. Harry Woo has nearly 30 years experience xperience in mission m operations and the design and implementation of ground data systemss for planetary spacecraft. s He has worked in various systems engineering and management capacities on Voyager, Ulysses, Galileo, Magellan, Stardust, and Cassini. He established the he Operations Engineering Laboratory at JPL enabling improved mission operations performance nce through development de and application of software tools. Mr. Woo has served in a numberr of line and program progra management positions at JPL and participates on the mission operations review boards of several current flight projects. CONSTELLATION PROGRAM SYSTEMS ENGINEERING CONSTELLA NEERING Title: Strategy for Software Reuse Study Final Report Document No.: AC1L1-52 Effective Date: October 1, 2005 Revision: 0 Page 125 Appendix D: Study Team Approach D.1 Study Team and Logistics The Software Reuse, Interoperability, and Compatibility Trade Study team has representatives from across NASA and across the Constellation IDTs. The members of the team were chosen to provide a variety of skills backgrounds. A complete list of study participants appears in Appendix C. D.2 Initial Team Meeting An initial team meeting was conducted to lay out the study approach, identify key study milestones, assignments, and communicate the study schedule. The effort was phased so that not all participants would provide support for the full duration of the study. A specific meeting agenda was developed for initiating the study. D.3 Regular Meetings The team conducted weekly telecon and WebEx meetings to issue assignments, and to report on progress, identify issues, and develop solutions. An agenda was developed for the recurring meetings to facilitate and streamline communications and to set expectations for the study team. D.4 Data Development Team members individually identified key topics or concerns in their special areas of expertise and interest. Literature surveys: The team reviewed over a hundred published documents, NASA memos and data, conference proceedings and presentations, and other written reports. Interviews and consultations: The team added non-IDT personnel who had first hand experience with reuse and/or major technical expertise in the area. Targeted interviews with internal-NASA and external experts were performed. Case studies: The team identified four diverse cases of previous NASA activities directed at or including software reuse, and studied these in depth. D.5 Analysis Team members were assigned specific study topic areas for individual analysis. Data from areas of investigation were shared and discussed in multiple venues, and all work products made available to the team via shared Windchill folders and email. Weekly study team telecons focused the team on emerging “tall-pole” issues and relevant data, and served as a forum for technical discussion as well as study guidance by the Study Lead. D.6 Synthesis Small teams were assigned to create findings by consolidating analysis data in key topic areas, abstracting common elements, and identifying key factors and lessons-learned. CONSTELLATION PROGRAM SYSTEMS ENGINEERING CONSTELLA NEERING Title: Strategy for Software Reuse Study Final Report Document No.: AC1L1-52 Effective Date: October 1, 2005 Revision: 0 Page 126 These teams and all study members were invited to make preliminary recommendations. Periodic “consensus” polls were taken on key topic areas as well as study recommendations, and the results targeted for further elaboration. The Study Lead analyzed overall progress and results and prepared consolidated findings and recommendations for discussion by the team. CONSTELLATION PROGRAM SYSTEMS ENGINEERING CONSTELLA NEERING Title: Strategy for Software Reuse Study Final Report Document No.: AC1L1-52 Effective Date: October 1, 2005 Revision: 0 Page 127 Appendix E: Reuse Metrics Deserve a Warning Label: The Pitfalls of Measuring Software Reuse 10 Jeffrey S. Poulin Lockheed Martin Federal Systems Almost everyone agrees that reuse involves using something that someone originally developed for “someplace else.” However, when it actually comes time to calculate reuse levels and reuse benefits, people often define reuse with a remarkable amount of flexibility. This article describes some of the difficulties in measuring software reuse. In a thorough and well-executed reuse survey of industry and government organizations by the Defense Information Systems Agency (DISA), software development leaders consistently reported impressive results. Managers gave reuse returns on investment a ratio as high as 4-to-1 after two years, cycle time improvements of more than 30 percent, and “a significantly increased competitive edge” [1]. But when the survey team sought to understand how the survey participants measured success, they found that the participants had provided testimonials and anecdotal results rather than objective evidence derived from quantitative measures. How can we have faith in anecdotes? This article explains how reuse metrics, like all software metrics, can leave a lot of open questions [2]. Leaving Room for Interpretation The values derived from any software metric depend on the data gathered to put into the metric. Unfortunately, developers usually exercise considerable discretion and creativity when deciding how to count different classes of code. To understand the true value of software development practices such as reuse, the data must be evaluated objectively and consistently. To complicate efforts, most people want to use traditional metrics to show increased productivity, e.g., lines of code (LOC) per labor-month, and improved quality, e.g., errors per thousand LOC. This creates an intuitive tendency to “look better by writing more,” which conflicts with the reason for reuse: to avoid software development costs by “writing less.” Unless acknowledged, this conflict will arise in subtle ways as people attempt to show excellence in reuse by using the same type of data used for traditional, productivity-sensitive metrics. Common Counting Strategies Management has considerable influence over how reuse is calculated. If management emphasizes reuse, a variety of common practices will suddenly appear under the reuse banner to artificially raise reuse levels. In my experience, developers often use the following methods to distort their reuse metrics and corresponding reuse benefits. 10 Verbatim text: http://www.stsc.hill.af.mil/crosstalk/1997/07/reuse.asp Note: “Acknowledgments” and “About the Author” have not been included in this appendix. CONSTELLATION PROGRAM SYSTEMS ENGINEERING CONSTELLA NEERING Title: Strategy for Software Reuse Study Final Report Document No.: AC1L1-52 Effective Date: October 1, 2005 Revision: 0 Page 128 The Code Generator Method Generating code can greatly help us during software development. However, developers will also use code generators to artificially achieve extremely high levels of “reuse.” For example, graphical user interface (GUI)-intensive applications commonly use code generators. After designing the GUI with a graphical layout editor, the developers merely push a button to create thousands of LOC that implement the GUI. Likewise, the developers might also count the generated code to inflate their productivity metrics. (Some show productivity rates as high as 15,000 LOC per labor-month—a superhuman feat by any standard.) Finally, the developers might include the generated code (usually close to defect-free) to distort their quality metrics with uncommonly good defect rates. Code generators yield a significant advantage when we can use them. However, from a metrics point of view, the code they generate requires special consideration when calculating metrics for productivity, quality, and reuse. We do this by reporting the generated code separately from all other code. The Baseline Code Method Groups performing maintenance can also significantly impact software metric values depending on how they count the code that they maintain. When making a new version of an application via changes to a large software baseline, the group might claim all of the software from that baseline as “reuse” or “produced” in the subsequent version of the application. In short, making minor modifications to a 1 million LOC product does not mean the team “reused” or “wrote” 1 million LOC. As with the generated code example, to include baseline code will severely inflate reuse, productivity, and quality metrics. The Porting Method In an “open systems environment,” the same application often runs on various hardware, operating systems, and flavors of the same operating system. Porting software to similar platforms usually requires little more than recompiling the source code for the new targets. However, some developers will report reuse levels of 99.9 percent for each “port,” and claim a corresponding financial benefit from “reuse.” Because overly high reuse levels may attract unwanted attention and scrutiny, reuse metrics may be collected and combined in many interesting and creative ways. For example, if a low reuse level on the initial development is averaged with a couple of 99 percent values from porting, one can arrive at a plausible and impressive value of 65 percent to 85 percent. This, of course, involves the mathematically unsound practice of averaging percents. The Programming Language Differences Method Language features provided by [insert your favorite programming language here] can make it easier to use code that someone originally wrote for use someplace else. But while procedures, functions, generics, templates, etc., provide highly effective mechanisms for reuse [3], these features do not guarantee reuse, nor does their use automatically mean reuse has taken place. The following set of examples shows how the use of programming language features can help programmers claim reuse even when no true reuse has occurred: CONSTELLATION PROGRAM SYSTEMS ENGINEERING CONSTELLA NEERING Title: Strategy for Software Reuse Study Final Report Document No.: AC1L1-52 Effective Date: October 1, 2005 Revision: 0 Page 129 Traditional example: Programmer J. Average sits down to design and implement a couple of thousand LOC for a new personnel roster and duty assignment system. Average decides to use the C language. Since many of the functions performed by officers, non-commissioned officers (NCOs), and enlisted members do the same thing, Average codes these functions as C functions. The program calls these C functions to process data records for officers, NCOs, and enlisted members. It takes Average three months to complete the program. When asked about reuse, Average responds, “I didn't do any; I wrote the whole program myself.” Object-oriented example: Programmer J. Average sits down to design and implement a couple of thousand LOC for a new personnel roster and duty assignment system. He decides to use the C++ language. Since many of the functions performed by officers, NCOs, and enlisted members do the same thing, Average codes these functions as C++ methods. Classes in the program inherit these methods to process data objects for officers, NCOs, and enlisted members, and it takes Average three months to complete the program. When asked about reuse, Average responds, “At least 60 percent, because I reused the same code over and over.” If both efforts took three months, what happened to the benefits of reuse in the second example? In fact, no reuse took place in either example, because both programmers did all the work themselves. The programmers merely placed different labels on what they did, while correctly using the features provided by their chosen programming language. The Problem of Units Reuse metrics do not necessarily require large influences to cause their values to vary. The previous examples illustrate factors that can have a major impact on metric values. However, metric values can vary significantly due to inconspicuous and seemingly innocuous factors, such as the units used in the metrics. Most organizations use LOC (which have many welldocumented advantages and disadvantages) or units such as function points or objects. How should we account for the wide variation in the size of objects? Should we make an allowance, since the use or reuse of a small object counts the same as the use or reuse of a large object? I have seen reuse levels vary by as much as a factor of 8, depending on whether a group calculated reuse by LOC or by object. The choice of units can affect code metric values, but the need to remove ambiguity in “what to count“ extends beyond code. We must also determine and quantify how to reuse lifecycle products such as requirements, documentation, and designs. What units should we use for these lifecycle phases? How should we count words, scripts, or graphics? Does a picture equal a thousand words? With the exception of the design phase—where some design environments clearly identify measurable “design objects”—standard, agreed-upon units of measure do not exist in the software industry. CONSTELLATION PROGRAM SYSTEMS ENGINEERING CONSTELLA NEERING Title: Strategy for Software Reuse Study Final Report Document No.: AC1L1-52 Effective Date: October 1, 2005 Revision: 0 Page 130 Reengineering Pitfalls Just as we expect developers to create routines when they have to do the same thing many times, we also expected them to copy and modify software when possible. This widespread practice, called “reengineering,” routinely saves developers significant implementation effort. But how should we track the amount of modified software in each routine? In addition to the difficulties of measuring reengineered software, how does it compare to the disciplined use of unmodified software that we have designed and implemented for reuse? Note that a plethora of evidence shows that reengineering can actually cost more than new development. This penalty can occur if developers modify as little as 20 percent of the code, and almost certainly occurs if they change more than half [4]. To include modified software in reuse metrics without adjusting for these penalties will inflate the apparent benefits from “reuse.” Altogether, these facts make a cogent argument for the design and reuse of unmodified software. Possible Solutions Management needs to give specific directions when requesting metric data; otherwise, developers will almost certainly use some latitude to interpret the request to their advantage. Developers do not necessarily manipulate the data, but they will present what they have done in the best possible light. For example, many practitioners do not often recognize the difference between “use” and “reuse.” When programmers need to do the same thing many times, they naturally code it into a procedure, function, method, subroutine, remote call, or similar language feature. When asked for reuse data, groups commonly exaggerate their reuse level by reporting every invocation of these routines as “reuse.” Management should specifically state that multiple “uses” do not count as “reuse.” More often than not, reuse metrics should include a warning. Knowing to watch for the “reuse warning label” will help everyone better understand the true benefits of reuse and ultimately help them make the best possible business decisions involving their reuse investments. Reuse metrics make an effective technology insertion tool, even if people will always try to find ways to make the numbers sing their praises. The following suggestions may help you with your reuse program and reuse metrics: • Have a management structure that supports reuse. Dedicate a team of programmers to the task of building and supporting code that all your teams can use. All developers can use this team as a resource during application design and implementation. • Develop and enforce a software architecture for all your teams. Provide the resources early in your project for domain analysis and to develop a common software architecture. • Define what you will measure as reuse and how you will measure it. Make sure everyone understands what you will count and why. • Regularly track your reuse metrics. Use metrics as a technology insertion tool to improve all of your processes and methods. CONSTELLATION PROGRAM SYSTEMS ENGINEERING CONSTELLA NEERING Title: Strategy for Software Reuse Study Final Report Document No.: AC1L1-52 Effective Date: October 1, 2005 Revision: 0 Page 131 Conclusions We have all seen experience reports that claim glowing results from reuse; however, reuse does not automatically provide the panacea that some reports seem to indicate [5]. Experience reports that do not mention where and how they obtained the data probably rely on testimonials and anecdotes rather than on objective evidence. Armed with a little knowledge about how metric values can vary based on who does the counting, we can better understand how to measure reuse activities and thereby maximize the benefits of reuse in our organizations. References 1. DISA/SRI, “Software Reuse Benchmarking Study: Learning from Industry and Government Leaders,” Defense Information Systems Agency Center for Software, Software Reuse Initiative, Document DCA100-93-D-0066, Delivery Order 0037, Nov. 10, 1995. 2. Poulin, Jeffrey S., Measuring Software Reuse: Principles, Practices, and Economic Models, Addison-Wesley, Reading, Mass., 1997. 3. Biddle, Robert L. and Ewan D. Tempero, “Understanding the Impact of Language Features on Reusability,” Fourth International Conference on Software Reuse, Orlando, Fla., April 23-26, 1996, pp. 52-61. 4. Stutzke, Richard D., “Software Estimating Technology: A Survey,” Crosstalk, STSC, Hill Air Force Base, Utah, May 1996, pp. 17-22. 5. Pfleeger, Shari Lawrence, “Measuring Reuse: A Cautionary Tale,” IEEE Software, July 1996, pp. 118-127. CONSTELLATION PROGRAM SYSTEMS ENGINEERING CONSTELLA NEERING Title: Strategy for Software Reuse Study Final Report Document No.: AC1L1-52 Effective Date: October 1, 2005 Revision: 0 Page 132 Appendix F: Software Reuse Study Report—A Report for the NASA IV&V Facility After the team completed this study, the team became aware of a software reuse study that was developed for the NASA Independent Verification and Validation (IV&V) Facility. That study and report were done by a company called TMC Technologies. The PI gave a presentation about the report at the IV&V Facility in Fairmont, West Virginia, in August 2005. That presentation and report provided important results that needed to be discussed herein. The NASA POC for the report, Gerald Gilley, graciously provided soft copies of that report to this study team. This appendix provides a summary of that report and its impact on this software reuse study (TMC, 2005). TMC undertook their reuse project for the NASA IV&V Facility to determine the impact of using legacy software on NASA spacecraft, and to help mitigate the risks of using legacy spacecraft software. The project was in four phases. The purpose of the Phase 1 study was to document the “as-is” state of NASA legacy spacecraft flight-software reuse. They reviewed literature on software reuse, and interviewed NASA IV&V personnel. TMC’s literature review led to definitions of different kinds of reuse: ad hoc, which is unplanned reuse of artifacts, also known as accidental reuse; opposed to that is systematic reuse, which is planned from reusable software components; black-box reuse is where components are reused unmodified, as opposed to white-box reuse, where the components are modified in the new use. (Frakes, Williams, and Terry, 1996; Poulin, 1997; Reifer, 1997). TMC’s literature review included mention of the regulations as described in Chapter 12 of FAA Order 8110.49, Software Approval Guidelines, June 2003, for reusable software components. Reusable components may be approved for integration in a larger system if “there is no adverse effect on original system safety margins, and 2) there is no adverse effect on original operational capability UNLESS accompanied by a justifiable increase in safety.” There is also an FAA Advisory Circular AC 20-148, Reusable Software Components, December 2004, supplementing FAA Order 8110.49. This advisory suggests a process that developers and integrators could follow to receive FAA acceptance of reusable software components. It also addresses reverification, and changes to reusable components; that is, how white-box reuse affects certification of the component. TMC’s literature review concluded with the following factors that seem to contribute to successful reuse projects (Joos, 1994; Morisio, Ezran, and Tully, 2002; Griss and Wosser, 1995): • Top management involvement and support • Changing the organization’s culture to reuse • Reuse-driven incentives • Training on reuse • Use of standards CONSTELLATION PROGRAM SYSTEMS ENGINEERING CONSTELLA NEERING Title: Strategy for Software Reuse Study Final Report Document No.: AC1L1-52 Effective Date: October 1, 2005 • Reuse within a well-defined domain • A library or centralized component repository • Reuse related tools • Well-defined software engineering methods Revision: 0 Page 133 The TMC report noted that loss of Mars Climate Orbiter was in part due to reusing its “Small Forces File,” which was in English units, while introducing a new vendor-supplied thruster equation, which returned data in metric units. This led them to consider the following questions: • Could the NASA IV&V facility have found the problems that resulted in the catastrophic loss, had the mission gone through their IV&V process? • Could proper reuse practices have prevented the loss? • Were contextual changes properly accounted for in the reuse scenario? The approach to answering these questions was to study the state-of-the-practice of software reuse in order to mitigate risk and increase benefits. TMC investigated 23 missions looked at by the IVV Facility: 19 IV&V efforts and 4 Independent Assessment efforts, and interviewed 12 IV&V project managers. They did not have access to the software developers. One result of the project manager interviews was that there were no standard definitions of software reuse among the project managers. For example, some defined reuse as black-box reuse, while another allowed up to 25% modifications from a previous mission. There were also no standard metrics for measuring the extent of software reuse on a NASA mission: some measured lines-of-code reused, while others counted the number of unmodified modules. The managers themselves did not measure software reuse; they accepted the claims of the software developers. Similarly, there was no standard approach among developers to implement reuse. Reuse claims were often overestimated and unjustified. For example, it appears that the amount of reuse is usually less than what is estimated at the beginning of the project. Also, the amount of rework needed for that reused code is usually more than what is estimated at the onset of the project: 6 out of 7 projects investigated required more changes to reused software than was originally anticipated. A focus of the report was on how reuse affected the use of IV&V. It was found that the reused portions of the software undergo the same IV&V activities, usually in less depth, but not always: one manager experienced the same or increased level-of-effort on the IV&V of reused software. The amount of IV&V is based on the criticality of the component, not on reuse. They found inconsistency or contradiction in the IV&V opinion on the impact of reuse on cost, schedule, and risk. This finding is important: Risk of failure is another factor that can be linked to NASA software reuse claims. Project Managers think that risk impact depends on various factors like component modification, quality of the interface analysis, changes in context, and whether the parent mission underwent IV&V at the IV&V facility or not. In most cases, they CONSTELLATION PROGRAM SYSTEMS ENGINEERING CONSTELLA NEERING Title: Strategy for Software Reuse Study Final Report Document No.: AC1L1-52 Effective Date: October 1, 2005 Revision: 0 Page 134 believe software reuse elevated the mission's risk because it was white-box, the documentation supporting the reuse was not provided, and the legacy software was usually not IV&V’ed and sometimes software claimed heritage was not even flown (TMC, 2005). TMC’s report recommended standards for both the developer and the IV&V practitioner, imposing limits on what should and should not be considered as valid reuse, which would allow a more consistent sharing of experiences and lessons learned from manager to manager. The report developed 10 genealogies of missions interrelated by reuse of artifacts. GN&C and C&DH were the two most commonly reused subsystems. Source code and requirements were the most commonly reused software artifacts. Only one genealogy involved systematic reuse. One of them involved parallel evolution of missions, where the missions’ software was exchanged while both missions’ software was being developed. It was thought this would confuse the development teams’ reuse strategy and its impact on risk. The level-of-risk of reuse was believed to be impacted by whether the missions involved had the same context, that is, involved similar space science goals such as exploring planets, or deep space, or the Sun. The TMC team recommends further study to identify patterns that could help mitigate the risks of software reuse in NASA spacecraft systems. In particular, the team recommends a more extensive study of projects where reuse was systematic to determine if these have fewer reuserelated problems. CONSTELLATION PROGRAM SYSTEMS ENGINEERING CONSTELLA NEERING Title: Strategy for Software Reuse Study Final Report Document No.: AC1L1-52 Effective Date: October 1, 2005 Revision: 0 Page 135 Appendix G: Quantitative Benefits Achieved through Software Reuse This is a verbatim excerpt, with format changes, from “Software acquisition gold practice ™ assess reuse risks and costs,” Available http://www.goldpractices.com/practices/arrc/index.php [Poulin, 1997], [Putnam, 2003], [Leach, 1997] and [Lim, 1998] have reported some of the quantitative benefits achieved through software reuse: • Nippon Electric Company: Achieved 6.7 times higher productivity and 2.8 times better quality through 17% reuse. They improved software quality 5-10 times over a 7-year period through the use of unmodified reuse components in the domain of basic system software development and in the domain of communication switching systems. • GTE Corporation: Saved $14 million in costs of software development with reuse levels of 14%. GTE Data Services benefited from $1.5M in cost savings in 1988 for 20–50% reuse. • Multiple Ada Projects: A study of 75 Ada projects in 15 firms totaling 30M LOC found reuse resulted in 10 times higher quality with reuse levels of 10-18%. • Toshiba saw a 20–30% reduction in defects per line of code with reuse levels of 60%. • DEC reported cycle times that were reduced by a factor of 3–5 through reuse levels of 50–80% and an increase of 25% in productivity through software reuse. • Hewlett-Packard (HP) cited quality improvement on two projects of 76% and 24% defect reduction, 50% and 40% increases in productivity, and a 43% reduction in time to market with reuse levels up to 70%. ROI ranged from 215% for one development to 410% for the other. • Raytheon achieved a 50% productivity increase in the MIS domain from 60% reuse using COBOL. • A study of nine companies showed reuse led to 84% lower project costs, cycle time reduction of 70%, and reduced defects. • 312 projects in aerospace industry: [“Showed an”] average 20% increase in productivity; 20% reduction in customer complaints; 25% reduction in time to repair; 25% reduction in time to produce the system. CONSTELLATION PROGRAM SYSTEMS ENGINEERING CONSTELLA NEERING Title: Strategy for Software Reuse Study Final Report Document No.: AC1L1-52 Effective Date: October 1, 2005 Revision: 0 Page 136 • Japanese industry study: 15–50% increase in productivity; 20–35% reduction in customer complaints; 20% reduction in training costs; 10–50% reduction in time to produce the system. • Simulator system developed for the US Navy: Increase of approximately 200% in number of SLOC produced per hour. • NASA Report: Reduction of 75% in overall development effort and cost. • AT&T reported a 50% decrease in time-to-market for 40–90% reuse. • Raytheon Missile Systems experienced a 1.5 times increase in productivity from 40– 60% reuse. • SofTech had a 10-to-20 times increase in productivity for reuse greater than 75%. CONSTELLATION PROGRAM SYSTEMS ENGINEERING CONSTELLA NEERING Title: Strategy for Software Reuse Study Final Report Document No.: AC1L1-52 Effective Date: October 1, 2005 Revision: 0 Page 137 Appendix H: Recommended Online Reading This is a verbatim excerpt with format changes from resources cited in http:// www.goldpractices.com/practices/arrc/index.php. Defense Information Infrastructure (DII) Common Operating Environment (COE) Upon successful login, you can view information on COE Releases, download COE Reference Materials, request COE Security Products, and interact with COE developers through the COE Forums. URL: https://coe.mont.disa.mil/login/login.asp Reuse Home Page The Reuse Group is comprised of experienced professionals with extensive expertise in establishing successful software reuse programs in major corporations. Its mission is to enable organizations to reduce time-to-market while improving system quality and organizational productivity through the systematic institutionalization of reuse practices. They emphasize a pragmatic approach to implementing and improving a reuse program. URL: http://www.reuse.com/ Software Dioxide: Reuse A portal into numerous software reuse resources, including papers and tech reports; books and publications; experts; conferences and seminars; tools; best practices; news and FAQs. URL: http://www.softwaredioxide.com/Channels/Reuse.asp?CDParent=Reuse Reusable Software Research Group (RSRG) The work of the group deals with the disciplined engineering of component-based software systems and the software components (a.k.a. reusable software components) from which they can be built. RSRG’s “home base” is The Ohio State University, and the information accessible through the links on this site deals primarily with OSU RSRG work. URL: http://www.cse.ohio-state.edu/rsrg/index.html Software Reuse: Acquisition Community Connection at Defense Acquisition Univ. (DAU) A group of questions and responses compiled on the topic of Software Reuse. CONSTELLATION PROGRAM SYSTEMS ENGINEERING CONSTELLA NEERING Title: Strategy for Software Reuse Study Final Report Document No.: AC1L1-52 Effective Date: October 1, 2005 Revision: 0 Page 138 URL: http://acc.dau.mil/simplify/ ev.php? URL_ID=1491&URL_DO=DO_TOPIC&URL_SECTION=201&reload Software Program Manager’s Network (SPMN)—Assess Reuse Risks and Costs The SPMN mission is to identify proven industry and government software best practices and convey them to managers of large-scale software-intensive acquisition programs. URL: http://www.spmn.com/16CSP.html#reuse Software Productivity Consortium (SPC) The purpose of the Software Productivity Consortium is to o serve its members, affiliates, and the national interest by providing highly leveraged system and software technology and services to increase productivity, profitability, and competitiveness. Through membership, it offers several reports, tools, etc., in the area of software reuse. URL: http://www.software.org/ Lombard Hill Group The Lombard Hill Group mission is to enable organizations to reduce time-to-market while improving system quality and organizational productivity through the systematic institutionalization of reuse practices. URL: http://www.lombardhill.com Data and Analysis Center for Software (DACS) Risk Management Topic Area This site contains hyperlinks that deal with DoD Risk Management resources; case studies; education, training and conferences; experts; literature; related sites; service providers; and tools and methods. URL: http://www.dacs.dtic.mil/databases/url/key.hts?keycode=270 Data and Analysis Center for Software (DACS) Reuse of Software Assets Topic Area This site contains hyperlinks that are dedicated to the DoD Software Vision and Strategy, as well as citations and abstracts of literature related to the reuse of software architectures and product lines. URL: http://www.thedacs.com/databases/url/key.hts?keycode=15 CONSTELLATION PROGRAM SYSTEMS ENGINEERING CONSTELLA NEERING Title: Strategy for Software Reuse Study Final Report Document No.: AC1L1-52 Effective Date: October 1, 2005 Revision: 0 Page 139 The DACS “Formal Risk Management” Gold Practice™ This is one of a number of practices that form part of the DACS Gold Practice initiative, which strives to promote the understanding and effective use of software acquisition and development methodologies and processes. URL: http://www.goldpractices.com/practices/frm/index.php View publication stats