Does your product or system meet the requirements? Find out in this webinar. Your host will discuss a verification and validation process that has worked for hundreds of our clients to answer this question. Then you will learn how to use a model-based systems engineering tool, Innoslate, to develop your own V&V process document. Then your host will dive into Test Center. Test Center allows for easy test case capture and traceability to requirements and provides the ability to run test cases within one easy-to-use view.
This webinar will cover:
1. V&V process
2. Test Center
3. Documents View
4. Traceability Matrix
2. AskUsYourQuestions
• Ask us your questions using
the panel on the right
• This presentation is being
recorded and will be made
available to you.
• Contact us after the webinar
through
• support@Innoslate.com
• Call 571.485.7800
• LinkedIn Innoslate User Group
• Twitter
Proprietary of SPEC Innovations® 05/26/2022 at 11:00 am ET, Dr. Steven Dam
3. MeetYourHost
• President and Founder of SPEC
Innovations
• Participated in the development of C4ISR
and the DoDAF
• Expert Systems Engineering Professionals
Certificate
• steven.dam@specinnovations.com
• @stevenhdam
Proprietary of SPEC Innovations® 05/26/2022 at 11:00 am ET, Dr. Steven Dam
4. WhatYouWillLearnToday
• How Different Customers View V&V
• How to Develop Your V&V Requirements, V&V Plans, and Test Suites from
the Beginning
• How to Capture and Report Test Results in Innoslate®
• How to Evaluate Processes using Sopatra®
Proprietary of SPEC Innovations® 05/26/2022 at 11:00 am ET, Dr. Steven Dam
5. V&VinDoD
• DoD focuses on “operational effectiveness,
operational suitability and survivability
(including cybersecurity), or lethality.”
[Defense Acquisition Guidebook (DAG),
Chapter 8]
• Organizationally, DoD divides into
Developmental Test and Evaluation (DT&E)
and Operational Test and Evaluation (OT&E)
• Development of a Test and Evaluation Master
Plan (TEMP) is considered critical
• DoD supports major test ranges and facilities,
worldwide
• An entire command focuses on
interoperability (Joint Interoperability Test
Command – JITC)
Proprietary of SPEC Innovations® 05/26/2022 at 11:00 am ET, Dr. Steven Dam
6. V&VinNASA
• According to NASA:
• “Product verification proves that an end product
(whether built, coded, bought, or reused) for any
element within the system structure conforms to its
requirements or specifications.”
• “While verification proves whether ‘the product was
done right,’ validation proves whether ‘the right
product was done.’”
• These definitions then map back to the DoD
approach:
• DT&E == Verification
• OT&E == Validation
• So, frequently different customers will frame
V&V in different terms and have different
processes and procedures, but the end goals
are the same
Proprietary of SPEC Innovations® 05/26/2022 at 11:00 am ET, Dr. Steven Dam
From NASA Systems Engineering Handbook
7. EachOrganizationWillDefineTheirOwnProcessesforCM
• Our process has many of the same
elements as the DoD and NASA
processes
• We include the use of our “SE
Knowledgebase” as a key part of our
process
• We currently use Innoslate® as our
primary SE Knowledgebase and tool
• It can ingest or capture all the work
products and may even be the
source of many of those products
• We can also use it to conduct
analyses and create any necessary
reports
Proprietary of SPEC Innovations® 05/26/2022 at 11:00 am ET, Dr. Steven Dam
SE Knowledgebase
Review Test Results with ERB
Develop Deployment Plan and
Conduct Beta Testing
Assess Risk and Plan Mitigation
Updated SE
Knowledgebase
Return Item to “Integrate Solution”
(if necessary)
Beta Test
Report
Deficiencies?
YES
NO
Workaround
Identified
Verify/Validate to Requirements
Verification/
Validation
Requirements
From Design &
Analysis Phase
Perform Regression Testing
Acceptable
Regression Tests?
YES
NO
Deployment Plan
Submit Changes to CM Process
Regression Test
Report
Inspection and Test
Report
Updated Test
Plan
10. CreateVerification/ValidationRequirementsandDocuments
• Start with Validation
Requirements
• Customer needs get translated into
implementation independent
requirements
• Determine then how you will validate
that these requirements have been
met
• May require a separate set of
requirements
• Once system requirements have
been decomposed to specification
level, create the verification
requirements as well
Proprietary of SPEC Innovations® 05/26/2022 at 11:00 am ET, Dr. Steven Dam
11. CreateTestPlansandTestSuites
• Test Plans can be developed
using built-in templates or
create your won
• Essential to be done early in the
lifecycle to identify long lead
items, such as test facilities and
automated test equipment (ATE)
• Preliminary Test Suites can also
be created as verification
requirements are developed
• Just about everything can be
planned well ahead of the actual
V&V execution activities
• This upfront planning ensures
better requirements and much
quicker V&V
Proprietary of SPEC Innovations® 05/26/2022 at 11:00 am ET, Dr. Steven Dam
12. EnsureRequirementsTraceabilityEarly
• The Traceability Matrix
provides a way to trace
requirements to Test
Cases
• The traceability assist
provides a check on the
probability that a
requirement and test
case should be
connected
Proprietary of SPEC Innovations® 05/26/2022 at 11:00 am ET, Dr. Steven Dam
13. IncludeVerificationMethodsforRequirements
• Verification Methods are
identified as labels in Innoslate®
• Analysis
• Demonstration
• Inspection
• Modeling & Simulation
• Test
• Additional labels can be added, if
needed by ensuring that the
“Verification Method” label is
used as a parent
• These labels are provided at the
Organization level
Proprietary of SPEC Innovations® 05/26/2022 at 11:00 am ET, Dr. Steven Dam
15. UseTestCenterandTestSuitestoCaptureResults
• Test Center provides a means to
define Test Suites for each major
V&V activity, using all methods
(analysis, demonstration,
inspection, simulation, test)
• Innoslate® can support the
simulation portion of V&V as well
• Data recorded in files can be
stored as Artifacts or a “File”
attribute can be added to the Test
Case class to associate files with
each Test Case
Proprietary of SPEC Innovations® 05/26/2022 at 11:00 am ET, Dr. Steven Dam
16. UseAlltheTestCaseAttributesandDiagrams
• Test Cases are a subclass of
Actions
• This feature enables the Test
case to be use in Action
Diagrams, Timelines, and other
project planning and execution
activities
• Test Cases have extra
attributes to capture V&V
related information
• Expected Result
• Actual Result
• Setup
• Add others as needed, e.g.
• Result Link
• File
Proprietary of SPEC Innovations® 05/26/2022 at 11:00 am ET, Dr. Steven Dam
17. CreateTestReportsandVerificationMatrices
• Test Reports are another
document type in
Innoslate®
• No templates are currently
available
• Add your own custom
templates
• Verification Matrices are
reports in the Document
View or can be developed
in Database View
Proprietary of SPEC Innovations® 05/26/2022 at 11:00 am ET, Dr. Steven Dam
19. Sopatra®WasDesignedtoEvaluateProcessesandProcedures
• Sopatra® translates a text-based
Standard Operating Procedure
into an Action Diagram
• It uses other information (input
devices, output displays,
acronyms, and time distribution
databases) to better develop the
process diagram
• Best practice actions (Hear, See,
Decide) are automatically added
• Special metrics were defined to
identify the Available Operational
Time Window (AOTW) and Time
of Procedure (ToP) to calculate
the Procedure Buffer Time (PBT)
Proprietary of SPEC Innovations® 05/26/2022 at 11:00 am ET, Dr. Steven Dam
20. Sopatra®’sMonteCarloSimulatorProvidestheanalysisResults
• A well developed procedure will
show “all green”
• Any failed runs will show up in
“red”
• Some procedures may be able to
have an acceptable failure rate and
the check for that will highlight the
fail rate in red when exceeded
Proprietary of SPEC Innovations® 05/26/2022 at 11:00 am ET, Dr. Steven Dam
21. LiveDemo
Let’s dive on in.
Proprietary of SPEC Innovations® 05/26/2022 at 11:00 am ET, Dr. Steven Dam
22. PredicttheFuture!
A well calibrated simulation can do this
• Remember what we do and how we do it matters
• High-quality analyses save projects, products, systems,
and lives
• Develop high-quality simulations by using Innoslate’s
discrete event and Monte Carlo Simulators
• Reduce operational errors by using Sopatra
Proprietary of SPEC Innovations® 05/26/2022 at 11:00 am ET, Dr. Steven Dam
23. Questions and
Answers
Use the panel on the right to ask your questions
Proprietary of SPEC Innovations® 05/26/2022 at 11:00 am ET, Dr. Steven Dam
24. Next Webinar
in “How to MBSE”
“Webinar: How to Verify and Validate”
May 26, 2022, 11:00 am ET
Go To Webinar
Proprietary of SPEC Innovations® 05/26/2022 at 11:00 am ET, Dr. Steven Dam
25. MoreResources
SPEC Innovations offers training, books, videos, documentation, trials, and more
Training: specinnovations.com/training
Books: “Real MBSE” textbook and lab manual available on
Amazon
Videos: Visit the SPEC Innovations YouTube channel
LinkedIn: Innoslate and Systems Engineers User Group
Documentation: help.Innoslate.com
Trial: cloud.Innoslate.com
Proprietary of SPEC Innovations® 05/26/2022 at 11:00 am ET, Dr. Steven Dam
26. SPEC Innovations offers training, books, videos, documentation, trials, and more
Thank you!
Visit cloud.innoslate.com for a trial.
SPEC Innovations
@Innoslate
Innoslate User Group
Innoslate.com/blog
571.485.7800
innoslate.com
Proprietary of SPEC Innovations® 05/26/2022 at 11:00 am ET, Dr. Steven Dam