Veachelp
Veachelp
Veachelp
Update packages may be used between editions. The manual printing date changes when a
new edition is printed. The contents and format of this manual are subject to change without
notice.
Rev: 447e070
© Copyright 2024, Vector Informatik, GmbH All rights reserved. No part of the material
protected by this copyright notice may be reproduced or utilized in any form or by any means,
electronic or mechanical, including photocopying, recording, or by any informational storage
and retrieval system, without written permission from the copyright owner.
This computer software and related documentation are provided with Restricted Rights. Use,
duplication or disclosure by the Government is subject to restrictions as set forth in the
governing Rights in Technical Data and Computer Software clause of
Vector Informatik reserves the right to make changes in specifications and other information
contained in this document without prior notice. Contact Vector Informatik to determine
whether such changes have been made.
Third-Party copyright notices are contained in the file: 3rdPartyLicenses.txt, located in the
VectorCAST installation directory.
2
TABLE OF CONTENTS
Introduction 14
VectorCAST Overview 15
VectorCAST Automation 15
Key Terminology 15
Key Concepts 17
VectorCAST Components 18
Starting VectorCAST 21
To Start VectorCAST and Open an Environment 21
To Start VectorCAST, Build, and Open an Environment 21
To Start VectorCAST in C++ or Ada "Mode" 21
To Set the Product Mode 22
To Exit VectorCAST 22
VectorCAST Interface 22
To Specify the Language for the VectorCAST GUI 24
Multibyte Characters 24
To Set the Industry Mode for Coverage 24
Changing Industry Mode 26
To Collapse / Expand the Message Window 27
To Hide the Message Window 28
To Clear or Copy Text in Message Window 29
To Hide the Environment View 29
To Move Docking Windows 29
To Float Docking Windows 30
To Return Docking Windows to Default Locations 31
To Save the Window Configuration 31
The Toolbar 32
The Status Bar 33
To Open a File Named in a Report 33
The MDI Window: Groups 34
To Open a Group 34
To Close a Group 35
To Maximize a Window 35
To Close a Tab 35
To Arrange Groups in a Cascade 35
To Tile Groups 36
To Bring a Tab to the Top 38
To Close the Current Window (Tab) 38
To Close All Windows 38
VectorCAST Keyboard Shortcuts 38
Getting Help 41
To Determine Available Licenses 41
3
To View Online User Guides 42
To Contact Technical Support 43
To Send a Test Environment to Technical Support 43
To Determine the VectorCAST Version and Customer Number 43
Using the Jobs Window and Jobs Monitor 45
The Jobs Window 45
Opening the Job Monitor 46
Job Status Viewer 47
Execution Status Viewer 50
Changing Application Preferences 53
To Set the Style of the Main Window 53
To Set Up an External Text Editor 53
To Edit User Code with the External Editor 56
To Set Up an External Browser 56
To Toggle Gridlines in the Test Case Editor 57
To Alphabetize Test Cases 57
To Change the Main Window’s Font 58
To Automatically Save Test Cases Before Execution 59
To Turn on a Reminder Before Closing Multiple Tabs 59
Customizing Reports 59
Customizing Existing Reports 60
Customizing Templates 62
Customizing Report Sections 63
Customizing Report Style 65
Creating New Reports 68
Editing, Searching, and Printing 73
To Create a New Text File 73
To Open a Script or Report File 73
To Save a Script or Text File 74
To Save the Open Window As 74
To Save All Open Windows 74
Print Setup 74
To Print an Open Window 75
To Preview Before Printing 76
To Undo a Recent Change 76
To Redo 76
To Copy, Cut, and Paste 76
To Search for Text Using the Find Banner 77
To Apply a Search Filter to the Test Case Tree 80
To Apply a Search Filter to the Parameter Tree 85
To Repeat a Search 86
To Goto a Line 86
Browsing Configuration Options 87
Using the Configuration Options Viewer 87
4
Using the Wizard to Create an Environment 90
To Set the Working Directory 90
Troubleshooting the Working Directory 91
To Start the Wizard 92
To Save the Settings in the Wizard 92
Step 1: Choose Compiler 93
Step 2: Name the Environment 93
Step 3: Build Options 95
Whitebox Restrictions and Limitations 96
To Not Allow Recursive Calls in the Harness 96
Step 4: Locate Source Files 97
Step 5: Choose UUTs & Stubs 98
To Specify a Stubbing Strategy for Individual Units 101
To Specify Custom Stubbing Interactively 103
Step 6: User Code (Optional) 103
Step 7: Summary 105
To Build the Environment 105
To View the Environment Overview Report 105
Files Created by the Environment 107
Troubleshooting Environment Creation 108
To Create a System Testing Environment 109
Setting Compiler Options 111
Options: Ada Tab 111
Options: Wizard Tab 114
Setting Builder Options 118
Options: Builder Tab 118
Deprecated Builder Options 126
Working with a Test Environment 126
To Open an Environment 126
To Close an Environment 127
To Rename an Environment 127
To Update an Environment 127
To Create Regression Scripts for an Environment 129
To Post-Process the Regression Scripts 133
To Integrate Regression Scripts with ClearCase™ 135
To Save and Load a Post-Processing Script 136
To Incorporate Source Code Changes into Environment 136
To Rebuild the Environment 137
Troubleshooting Environment Rebuild 138
To Delete an Environment 138
To Re-create a Deleted Environment 138
Other Environment Tools 139
To Create an Environment Script 139
To Create an Environment by Running a Script 139
To Recompile the Test Harness 140
To Recompile the Instrumented Test Harness without Re-instrumenting It 140
To Create a Test Harness Recompile Script 140
5
Recompile the Test Harness by Running a Script 143
To Relink an Environment 143
To Refresh Type Range Data 144
6
To Enter Input and Expected Values 173
To Enter Test Case Requirements or Notes 173
To Enter Values for Global Data 173
To Enter a Range 173
To Enter a List 174
The List Values Tab 174
To Pass a Null String 175
To Raise an Exception from a Stub 175
Working with In/Out Parameters in Stubs 176
Data Types 176
To Enter Values for Enumeration Types 176
To Enter a Number for an Enumeration 177
To Enter Integer Values in Different Bases 177
To Enter Real Numbers using Scientific Notation 177
To Enter Values for Record Types 178
Variant Record Types 178
Character Types 178
To Enter a String 178
System.Address Types 179
Access Types 179
Special Kinds of Stubs 180
Testing Out of Range Values 182
VectorCAST and "Class" based Testing 183
Using the Multi-unit Whitebox feature to Manipulate Private Types 188
Working with Arrays 189
To Enter Values for Constrained Array Types 189
To Expand All Elements of an Array 189
To Expand the First Element of an Array 190
To Collapse Unused Elements of an Array 191
To Expand Certain Elements of an Array 191
To Apply Values to an Array 194
To Clear Values from an Array 195
Using the Array Properties Dialog 196
Multi-Dimensional Arrays 197
Unconstrained Arrays 197
Working with Unconstrained Array Parameters 198
Entering Data in the Parameter Dialog Box 199
To Use a Range Expression in an Expected Value List 199
To Use the <<Any>> Tag in a List 200
To Access the Parameter Dialog 200
The Scalar Values Tab 201
To Apply Values to an Array 202
To Clear Values from an Array 204
The Range Values Tab 204
To Set Up an Input Range or List for More Than One Parameter 205
To Use Ranges as Expected Values 205
The List Values Tab 206
Controlling Values in the List 206
7
To Repeat Values in the List 207
To Use a Range Expression in an Expected Value List 208
To Use the <<Any>> Tag in a List 209
Test Case Scripting 210
To Export Test Cases to a Test Script 210
To Import Test Cases from a Test Script 211
To Create a Test Case Template 211
Troubleshooting Test Script Template 212
Automating Test Script Maintenance 213
Generating Test Cases Using CSV- or Tab-Delimited Files 218
To Create a Template for a CSV- or Tab-Delimited File 219
To Create a Map from an Existing CSV or Tab-Delimited File 224
To Create Test Cases from a MAP Test Case 228
To Edit an Existing Map Test Case 229
To Create and Edit a Test Script from a Map File 230
VectorCAST Tools 233
To Set VectorCAST Options 233
Tools Menu 234
View Harness Source 234
Basis Path 235
MC/DC 236
Working With Custom Tools 236
To Create a Diagnostic Report 243
Troubleshooting VectorCAST 243
8
View a Test Case’s Raw Data Set 276
Setting Execution Options 276
Setting Report Options 285
Report Content Options 285
Report Format Options 297
The Testcase Options Tab 303
Deprecated Report Options and Commands 309
9
To Set a Breakpoint 350
Setting Coverage Options 351
Options: Coverage Tab 351
Coverage Viewer Options 358
To Format the Text in the Coverage Viewer 359
Importing Coverage Results 361
Preparing to Import Results 361
Importing the Results 362
To Export a Script of Coverage 363
To Import a Coverage Script 366
To Delete Imported Results 366
To View the Coverage Import Log 367
10
To Recompile Environment User Code 395
To Remove Environment User Code 395
Example: To Perform Timing Analysis 395
Example: To Read Test Data from a Database 396
User Parameters 397
Example: Chaining Data Values With User Parameters 399
Test Case User Code 400
To Enter Test Case User Code 400
Test Compile Testcase User Code 401
To Clear Test Case User Code from Test Harness 402
Add Test Case User Code Back Test Harness 402
Parameter User Code 402
To Enter Parameter User Code 403
To Test Compile Parameter User Code 403
Input User Code Example 403
Expected User Code Example 404
Another User Code Example 404
Stub User Code 407
To Enter Configure Stub User Code 408
To Test Compile Stub User Code 409
To Save Stub User Code 411
To Recompile Stub User Code 412
To Remove Stub User Code 412
Example of Stub User Code 413
11
Step 6 - Run the Test Case 436
Step 7 - Configure the Export Settings 436
Step 8 - Export the Test Data 437
12
Appendix E: Environment Variables 481
Index 546
13
Introduction
VECTORCAST OVERVIEW 15
VectorCAST Overview
VectorCAST is a suite of tools for automating the entire process associated with conducting unit and
integration testing:
> VectorCAST/C++ and VectorCAST/Ada automatically generate executable test harnesses for
one or more units of application source code written in C, C++ or Ada. In addition, they generate
and execute unit test cases and report on results and metrics.
> VectorCAST/QA is a code-coverage analysis tool that identifies which areas of an application
have been exercised by test executions. It supports C, C++ or Ada, and supplements the unit
coverage provided by VectorCAST/C++ and VectorCAST/Ada.
VectorCAST Automation
You can use VectorCAST to conduct unit or integration testing on applications written in the C, C++, or
Ada programming languages. VectorCAST automates the following unit and integration test functions:
Three types of input data can affect the processing of a unit under test (UUT):
VectorCAST provides direct control over all three types of data. It does this by automatically creating a
unique test harness for each software unit to be tested.
Key Terminology
You should be familiar with the following terms before you begin using the tutorials:
Test Harness – A test harness is the executable program created when the test driver, the units under
test, and the dependent units or stubs of dependent units are linked together.
Dependent Unit – A file that contains a function called by a UUT, or data objects used by a UUT. A
dependent unit is an Ada package that is 'withed' by a unit. Dependent units can be stubbed or used 'as
is'.
Event – An event is a change in the flow of control that VectorCAST can monitor. Each of the following
is an event:
> A call from the test harness to a UUT
> A call from a UUT to a stubbed dependent in the test harness
> A return from a UUT to the test harness
Test Case – A collection of input and output data, defined with the purpose of exercising specific
software and verifying results. This data is commonly comprised of the formal input and output
parameters of the dependent subprograms, and input and output parameters of the subprogram under
test. The Test Data controls the execution of the unit under test as well as values for global objects
visible in the unit under test or stubbed subprograms.
Test Driver – VectorCAST generates the main program, or driver, necessary to exercise all visible
subprograms in all units under test. This consists of calls to each visible subprogram in a unit under
test.
Parent Library (Ada) – The directory that contains the units under test and dependent units. In Ada
environments, the Parent Library is a single directory in the hierarchy of libraries.
Stub by Function Units – Prior to VectorCAST version 5.0, only subprograms in dependent units of
the UUT were allowed to be stubbed. In fact, all subprograms in stubbed units had to be stubbed. In
VectorCAST 5.0, you have the choice of making a Unit Under Test "stub by function (SBF)." When a
UUT is made "SBF", this means that functions in the unit can be stubbed individually, while others in
the unit are not stubbed.
Key Concepts
You should be familiar with the following concepts before you begin using the tutorials:
Execution Closure – An execution closure consists of all the units required for an application to
successfully link. Figure 1 shows a closure consisting of a single UUT, three direct dependent units,
and several sub-dependents.
Stubbing Units in a Closure – When building a VectorCAST environment, you designate stubbing
from the top-most level of dependency downward to the bottom-most level in each branch.
In the closure illustrated in Figure 1, for example, dependents Dep Unit A, Dep Unit B, and Dep Unit C
would be the primary candidates for stubbing. If you were to leave one of these dependents unstubbed,
the dependents immediately under it would then become the primary candidates for stubbing, and so on
to the bottom of each branch.
Figure 2 shows the example closure after stubbing has been designated.
The scheme allows a flexibility spanning from testing a single unit in a standalone environment in which
every dependent is stubbed, to testing groups of units with only the dependents on the periphery
(bottom) of the closure stubbed.
Ignoring Units in a Closure (Ada only) – As mentioned above, normally when you choose not to
stub a unit, any dependents below this unit remain in the closure. In VectorCAST/Ada, however, you
can choose to leave a unit unstubbed but effectively remove all the downstream dependents from the
list of stub candidates; all downstream dependents will not be stubbed. You do this by directing
VectorCAST/Ada to ignore the unit.
Note: In Ada, ignoring dependents applies only to the dependents of the 'body' of the ignored
unit; it does not apply to the dependents of the 'spec'. The reason for this is that VectorCAST
needs to parse the dependents of the 'spec' in order to resolve types derived from the
dependents.
This option is useful when you want to avoid doing any stubbing beyond a certain point. For example, in
Figure 3, suppose Unit C is an interface to a subsystem already tested. There is no need to stub beyond
Unit C. By directing VectorCAST to ignore Unit C, it will use this entire branch as if you had chosen not
to stub the units.
This option can in some cases dramatically reduce the time it takes for VectorCAST to build an
environment.
Note: In choosing to ignore a unit, downstream dependents will not be ignored if these units are
brought into the closure through an alternate dependency.
VectorCAST Components
VectorCAST consists of four major functional components:
> It generates a framework (environment) consisting of all the necessary resources to build a test
harness
> It generates (or regenerates) the test harness itself
The Environment Builder parses the designated files and, from the information gathered, generates a
complete test harness.
Unlike a manually generated test harness, a VectorCAST test harness can be automatically
regenerated whenever changes are made to a UUT or stub included in the environment (for example,
when a subprogram is fully coded).
In addition, test cases can be added to an existing environment without any need for manual coding,
recompiling, etc.
Execution Manager
The Execution Manager invokes the test harness created by the Environment Builder, and executes the
test cases built by the Test Case Generator.
Report Generator
The Report Generator produces a variety of reports based on information maintained in every
environment database. This information includes:
Starting VectorCAST
Prior to starting VectorCAST, you should ensure that VectorCAST is installed. See the VectorCAST
Installation Guide for details. Also, the VectorCAST Quick Start Guide has additional information for
starting VectorCAST.
Linux
$VECTORCAST_DIR/vcastqt -e <environment_name>
Linux
$VECTORCAST_DIR/vcastqt -b <env script>
The path to <env script> can be relative to the directory from which VectorCAST is invoked, or a
full path. The filename can be specified with or without the .env extension.
Linux
$> $VECTORCAST_DIR/vcastqt -lc (to open VectorCAST/C++)
$> $VECTORCAST_DIR/vcastqt -lada (to open VectorCAST/Ada)
If you already have an environment open, the File => Set Product Mode menu item is dim.
To Exit VectorCAST
The File => Exit command enables you to exit the VectorCAST application. You can also exit by using
the shortcut of clicking the "X" in the top right-hand corner of the main window.
VectorCAST Interface
When you start VectorCAST, the VectorCAST interface (main window) appears. This window can vary
slightly in detail on the basis of the licenses held. For users with only VectorCAST/Ada licensed, the
VectorCAST/Ada window opens. For users with only VectorCAST/C++ licensed, the
VectorCAST/C++ window opens. For users with licenses for multiple VectorCAST products, a product-
neutral VectorCAST window opens.
> The Environment View is located on the left-hand side of the main window. It provides a high
level view of the project structure. The Environment View pane contains the Test Case Tree.
> The Message Window is located along the bottom left of the main window. It contains tabs for
informational messages and for error messages. The Message window can be expanded and
collapsed using the small button located on the upper right corner of the message window.
> The Multiple Document Interface (MDI) Window is located to the right of the Project Tree. It
displays a variety of windows, including Test Case editors, Coverage Viewers, Report Viewers
and Text Editors. Windows are collected into groups.
> The Jobs Window is located on the bottom of the main window. It displays the status of jobs as
they execute and exposes the associated back-end commands.
The Environment View and the Message Window are docking windows: That is, you can move these
from the context of the main window to any other location on your desktop.
The Jobs Window is an auto-hide window. Hover over the Jobs tab to open it and click on
the pin icon to keep it open.
The VectorCAST tools and functions are available from a menu bar and a toolbar located at the top of
the main window.
By default, the VectorCAST main window displays text in English, using the Latin1 character encoding.
You can set the main window to display in Japanese or Chinese by selecting the desired language from
the drop-down menu. You must exit VectorCAST and restart it for the change to take effect.
This information is saved to a file named .vcast-qt in the user’s HOME directory.
Multibyte Characters
If source code files use the character encoding Shift JIS, select the option Wide-character support,
located in the Options dialog. Choose Tools => Options, and click the Builder tab.
To select the industry mode, select the Tools=>Industry Mode command and then select the specific
Industry Mode from the context menu.
When initializing or selecting a coverage type, the industry appropriate coverage types are shown.
If you do not set an Industry Mode, the Default mode is used and you will be presented with the Default
coverage types when choosing or initializing coverage.
Once an Industry Mode is selected, it is stored in the .vcast-qt file and the chosen Industry Mode
remains in effect until a different Industry is selected.
When coverage is initialized, the industry appropriate coverage type is displayed in the status bar at the
bottom.
Coverage reports reflect the associated coverage categories, Statements, Branches, MC/DC Pairs, for
the coverage type chosen. In addition, the Industry Mode will be shown on the report.
For existing environments that have a coverage type that is contained in the table below, changing the
Industry Mode will result in an immediate translation of coverage type from one Industry Mode to
another. This immediate translation is reflected in the status bar of the application and in any test
reports generated after the change.
Statement Level C Unit Level ASIL A SIL 1/2 SIL 1/2 Class A
Statement + Branch Level B Unit Level ASIL B/C SIL 3 SIL 3 Class B
For example, if the Industry Mode currently is set to Automotive and the coverage type is ASIL A, if the
Industry Mode is changed to Avionics, using the table, the coverage type will be translated to Level C.
If the current Industry Mode is Default, and the current coverage type is Branch, Basis Path or MC/DC,
when the Industry Mode is changed the current coverage type is retained. However, the coverage
choices in Coverage => Initialize or in the Build Options step of the Update Wizard will reflect the
industry specific coverage types.
If the environment is rebuilt (Environment => Rebuild Environment) after changing the Industry
Mode from default, the Branch, Basis Path or MC/DC coverage type is retained.
If the environment is updated (Environment => Update) after changing the Industry Mode from
default, the Branch, Basis Path or MC/DC coverage type is set to None.
For environments that were created with a version of VectorCAST prior to 6.0, Level A, Level B, and
Level C coverage types will be mapped using the table above as if the Industry Mode was set to
Avionics prior to the Industry Mode change.
To expand the Message window to see all of the text in the Message window, click the small
button located on the right side of the single line message. This action can be done almost at any time,
and is specifically permitted between execution of multiple tests.
The expanded Message window consists of the following tabs: Message and Error. The Message tab
displays user status messages. The Error tab displays VectorCAST diagnostic messages.
To make it visible again, choose View => Dock Control => Messages, or View => Default Layout.
To make it visible again, choose View => Dock Control => Environment View, or View => Default
Layout.
To move the Message Window or the Test Case Tree to a new location, grab the window by its top Title
bar and drop the window within the boundaries of the VectorCAST main window.
For example, to make more room in the MDI window, you can drag the Message window and Test Case
Tree and dock them above the MDI window. To do this, drag the Title bars of the Message window and
Test Case Tree and drop them above the MDI window. Use the splitter to adjust the window size.
Use View => Default Layout to return the main window to its default layout, if desired.
Use View => Default Layout to return the window to its default layout, if desired.
By default, VectorCAST saves the current configuration for use in all subsequent sessions. In other
words, if you change the window configuration, exit VectorCAST, and then return, the window
configuration will be the same as it was when you exited the previous session.
To save the current window configuration for use only in subsequent sessions involving the same
environment, choose Tools => Options => GUI tab, then check Remember window sizes per
environment.
This information is saved to a file named .vcast-qt in the user’s HOME directory.
The Toolbar
The Toolbar shows all icons associated with the various VectorCAST commands. The Toolbar includes
icons for third-party tools integrated with VectorCAST. The icons are functional only if you are licensed
for these features.
By default, the Toolbar is visible. If you move the Toolbar you can return it to its default position by
choosing View => Default Layout.
The environment’s status should be “Normal”; if an error occurs when building or rebuilding an
environment, the status changes to one of the following:
> Compile Error – An error occurred while compiling the test harness. You will need to correct the
problem and recompile the environment. Choose Environment => View => Compile Errors to
view the compile errors.
> Link Error – An error occurred while linking the test harness. You will need to correct the problem
and relink the environment. Choose Environment => View => Link Errors to view the linker
errors.
> Data Interface Error – The test harness linked but generated a catastrophic error during program
elaboration.
If coverage has not been initialized, then the Coverage status is “None.” If you have initialized any of
the types of coverage (statement, branch, etc.), then the status is changed to “Type,” where Type is
one of the types of coverage. Note that if you disable coverage, the status changes to “Type (disabled)”.
Enabling coverage removes “disabled” from the status.
The right-most section of the status bar shows the full path to the current working directory. It is the
default directory in the File Save dialog when saving a report or exporting a script to a file, and the
default directory in the File Open dialog when opening a file or importing a script.
You can switch between different windows in the group by clicking the named tab at the top.
To Open a Group
The lower half of the Window menu contains a list of open windows. To switch between windows,
choose Window => <window name>.
To Close a Group
To close a group, click the X in the upper right corner of the group tab. Closing a group closes all the
tabs in that group. If any tabbed window requires saving before it can be closed, you are asked if you
want to save it.
To Maximize a Window
You can maximize a window by un-docking it and clicking the maximize button in the title bar of the
window.
To Close a Tab
To close a tab in a group, click the X in the tab. Alternatively, right-click a tab and choose Close from
the context menu.
Alternatively, selecting the File => Close command from the Menu Bar enables you to close the
current active tab in the MDI window. Note that It does not close the environment, only the tab.
To Tile Groups
The Window => Tile => Grid menu item arranges all open windows in a grid pattern. Each window is
reduced to a rectangle, and set next to another window. There is no overlap or layering. The keyboard
shortcut is Ctrl+Alt+G.
Choose Window => Tile => Horizontal to arrange the groups horizontally. The figure below shows
two groups tiled horizontally. The keyboard shortcut is Ctrl+Alt+H.
Choose Window => Tile => Vertical to arrange the groups vertically. The keyboard shortcut is
Ctrl+Alt+V.
window is in focus, its tab is highlighted. If a tab is not visible, use the left and right arrows to
scroll the tabs to the left or right.
Available in a Unit Test environment or Cover environment that has been opened from within a
VectorCAST project:
Expand / Collapse Project View Ctrl+Alt+p
Getting Help
To Determine Available Licenses
To determine which VectorCAST products your organization is licensed to use, choose Help =>
Available Licenses:
An Available Licenses Report is displayed in the MDI window, listing all available licenses.
Alternatively, to determine which features and targets are licensed, choose Help => Diagnostics =>
Create Diagnostic Report:
Check Check Licenses on the Diagnostics popup, click OK, and then scroll down the report to the
Licensing section.
> Release Notes, which include the changes and improvements to VectorCAST since the last
released version.
> User Guides for all VectorCAST products.
The User Guide .PDF files are located in the VectorCAST installation directory, /docs/PDF sub-
directory, and can be accessed by selecting Help =>PDF User Guides from the Menu Bar. Note that
only User Guides for licensed products are available from the menu.
In addition to the PDF versions of the User Guides, VectorCAST includes HTML Help. Use the Help
=>VectorCAST Help menu item to access the HTML guides. After selection, the guide opens in a
VectorCAST browser window which is detached from the main VectorCAST GUI. If multiple help
guides are opened, each will exist in a separate tab in the browser window.
Email: support@vector.com
Web: https://support.vector.com
If your environment is named TEST, for example, you should send the entire contents of the directory
named TEST along with the file TEST.vce. Under Windows, you can use PKZip or WinZip; under
Linux, you can use tar and compress or gzip to create an archive file that can be emailed to Tech
Support.
If the size of the compressed file is greater than 5 MB, please send an email to support@vector.com
and they will instruct you on how to send the file.
The Jobs window displays the full command line for every invocation of an executable by
VectorCAST's back end. Such executables include the compilers, .bat files, python and any other
commands called while building the environment.
Single-clicking on a command line highlights the associated line in the Messages window. Hovering
over a command line shows the exit code and stdout and stderror for that line.
For each command, the status, execution time elapsed and Process ID (PID) is displayed. Icons in the
Status column represent the following:
A command taking more than 5 seconds to execute displays a yellow background. Click the Abort
button to kill the command if desired.
To open the Job Monitor and the Job Status viewer, double-click a command line in the Jobs window or
right-click a command line and select Job Status from the context menu. The Job Status Viewer opens
in the MDI window.
The Job Status viewer is primarily used to diagnose how VectorCAST makes calls to the Target
compiler, and provides the user with helpful information about what a command is and where it is run
from.
The Job Status viewer displays the executable called, any arguments and full stdout and stderror
information. Click a running command to see the stdout and stderror in real time.
This debugging feature provides a way to diagnose failing commands. It does not affect the state of the
environment or resolve errors caused by the original command failing to run.
For example, say the Environment User Code is modified and a coding error introduced. When the
change is saved and linked, the coding error is detected and the Job Status viewer automatically opens.
You can use the information in the standard error tab to determine which test harness file has the
problem, make changes to the file, save, and then click the Test Command button to try the command
again.
Once satisfied that you have the solution to the problem, you still need to make the change in the user
code or Compiler template before attempting the test execution again.
The Execution Status viewer opens automatically when a test case is executed. Click the Show
Details button to display the full details of the execution in real time.
Note: When running hundreds of tests, hiding details can reduce total execution time.
A progress bar is displayed on the top left showing the number of tests remaining to execute. The name
of the currently executing test is displayed beneath. An Abort button is available to halt the test case
execution if desired and return control to VectorCAST.
When viewing full execution details, as each test completes it is listed on the bottom left along with its
pass/fail status. Hover over the name of a test case to see the name, unit and subprogram.
A deleted test displays an icon to the left of its execution status indicating that it is an invalid result
now.
Hover over the status of a test case to see details of the execution. If a test aborts during execution,
that information is displayed in the tooltip.
If the environment has coverage and the test passed, double-clicking on a passed status opens the
Coverage Viewer. If the environment has no coverage and the test passed, double-clicking on a passed
status opens the Execution Report.
Failing test cases are displayed on a red background. Double-clicking on a fail status opens the
Execution Report, and the report scrolls to the first failure instance.
The full command line and the stdout and stderror details are displayed in the right column of the
Execution Status viewer.
Choose Tools => Options, and click the GUI tab. When you have made your selection, click Apply or
click OK to close the dialog.
Note: These options are for text files and text reports only. To use an external browser for HTML
reports, see "To Set Up an External Browser" on page 56.
The Editor drop-down list contains a list of predefined commonly used text editors which can be
selected as a VectorCAST external editor for text files. Optionally, based on the setting of the "Use as
text report viewer" option, the external editor can be used to view text reports.
> <custom> (this option allows users to specify their own external editor)
> VS code (Visual Studio Code)
> Emacs
> Notepad
> vi
The executable file for the editor must exist on your PATH environment variable. In cases where the
executable of the editor is not on your path, a warning appears:
In this case, you can either modify your PATH environment variable or configure VectorCAST with the
full path to the editor executable using the Editor Command option.
Click Use DOS Formatted Files if your choice of editor requires that all lines be terminated with a CR
+ LF (i.e. Notepad).
If you want to use the same editor for viewing text reports, click the checkbox next to Use as text
report viewer. If this option is set, and you have chosen “Text” as the report format in the Reports tab,
then any time you choose a “View” action, the file chosen is opened in the external editor.
When you have made your selection(s), click Apply or click OK to close the dialog.
When opening a file using an external editor in a context where no particular location is requested, such
Linux Example
On Linux, a vi user creates a bash script named my_start_editor.sh, whose contents include:
#!/bin/bash
vi "$@" +$VCV_EDIT_LINE_NUMBER
In VectorCAST's Options dialog, GUI tab, Text Editor Options sub-tab, the user sets the option
Editor command to:
xterm -e %s
Then, in a VectorCAST unit test environment, when right-clicking on a UUT's subprogram name and
choosing Open Source, xterm opens with the vi editor correctly scrolled to the first line of the selected
subprogram.
Windows Example
where <path to> is written similar to "c:\Program Files (x86)\Notepad++" and enclosed in
quotes.
In VectorCAST's Options dialog, GUI tab, Text Editor Options sub-tab, the user sets the option
<path to>\my_start_editor.bat
Then, in a VectorCAST unit test environment, when right-clicking on a UUT's subprogram name and
choosing Open Source, Notepad++ opens with source file correctly scrolled to the first line of the
selected subprogram.
First, click the checkbox next to Use External Browser to enable the feature. Note that checking this
box without specifying a browser causes VectorCAST to display all reports, including the test case
Execution Results, in a tab in the Report group.
If you would like all reports to be displayed in a separate browser window, enter the path to your HTML
browser in the edit box next to Browser Command. If none is specified, VectorCAST’s internal
browser is used. You can set up a browser but temporarily turn it off by un-checking the Use External
Browser checkbox.
When you have made your selection, click Apply or OK to close the dialog.
This information is saved to a file named .vcast-qt in the user’s HOME directory.
The Show gridlines in test case option enables you to view the Test Case Editor either with gridlines
(if you prefer a spreadsheet look to the editor) or without gridlines. By default, the Test Case Editor
displays with gridlines.
When you have made your selection, click Apply or OK to close the dialog.
This information is saved to a file named .vcast-qt in the user’s HOME directory.
In the Test Case Tree, clicking on the title bar of the Test Cases column will toggle the display from an
alphabetical listing of Unit, Subprogram, and Test Case names to a listing based on the order of
definition of those items. "Order of definition" means the order that subprograms are defined in your
source file under test, or the order in which a test case is added to the environment.
To alphabetize in the Test Case Editor Parameter Tree, choose Tools => Options, and click the GUI
tab and click the Test Case Editor Options sub-tab. Select the Alphabetize parameters option.
When this option is set, the Unit names, Subprogram names, Global Object names and Parameter
names are alphabetized in the Parameter Tree. When this option is cleared, the Parameter Tree items
are displayed in the order they are defined in the source code.
Note: If you turn this option on or off, you must close and re-open any existing Test Case Editor
windows for the change to take effect.
This information is saved to a file named .vcast-qt in the user’s HOME directory.
Click either OK or Apply in the Tools => Options dialog, and the new font appears.
The “Automatically Save Test Cases Before Execution” option saves the current test case prior to
execution. If this option is not selected, you are prompted to save a modified test case when execute is
chosen. By default, modified test cases are automatically saved before executing.
When you have made your selection, click Apply or OK to close the dialog.
This information is saved to a file named .vcast-qt in the user’s HOME directory.
If you would like VectorCAST to remind you that more than one tab is being closed, turn on the “Ask
before closing multiple tabs” option. This option is off by default, meaning that when you click the close
X of a tag group, all the tabs close without a reminder.
When you have made your selection, click Apply or OK to close the dialog.
Customizing Reports
VectorCAST's reporting system allows reports to be customized to contain any information with any
layout. Reports can be produced as either HTML or text. To generate a PDF of a report, use the HTML
version.
Note that the full data API into all statistics is only available with the VectorCAST/Ada Enterprise
edition.
The option VCAST_RPTS_CUSTOM_CONFIG is used to customize the sections used in reports and
to replace the contents of a template wihout needing a customization directory. The option can point to
a file containing the configuration or the JSON configuration itself.
{
"reports": {
"<report_name|*>" : {
"<HTML|TEXT>": {
"<sections|extra_sections|remove_section":[<data>]
}
}
}
}
Where:
extra_sections - allows the addition of sections without having to list out all of the sections. Use
the option '+' to add the new section after the existing section. Use the option '-' to add the new section
before the existing section.
The example below shows adding the Overall Results and Metrics tables after the Configuration Data in
the Execution Report:
{
"reports": {
"<ExecutionResultsReport>" : {
"HTML": {
"extra_sections":
"[["CONFIG_DATA", "+OVERALL_RESULTS"],{"OVERALL_
RESULTS","+METRICS"]]
}
}
}
}
The following example shows removing the Metrics Table from the Full Report:
{
"reports": {
"<FullReport>" : {
"HTML": {
"remove_sections": ["METRICS"]
}
}
}
}
{
"sections": {
"<section_name>" : {
"<HTML|TEXT_FILE|TEXT>":"<path to file|text>"
}
}
}
Where:
HTML | TEXT_FILE | TEXT - specifies whether the following string is the actual HTML or text
template or a file name containing the template.
The following example shows replacing the Metrics Table with a text file:
{
"sections": {
"<METRICS>" : {
"TEXT_FILE": "C:\template.txt"
}
}
}
{
"reports": {
"<report_name |*>" : {
"<HTML | TEXT>" : {
"<sections | extra_sections | remove_section": [<data>]
}
}
}
}
Where:
extra_sections - allows the addition of sections without having to list out all of the sections.
For example,
[ 'EXISTING_SECTION', '<option>NEW_SECTION’ ]
– where option is ‘+’ or ‘-’ to add the new section after or before the existing section
The example below shows adding a "new section" after Configuration Data and "another section"
before the Metrics Table to all reports:
{
"reports": {
"*" : {
"TEXT" : {
"extra_sections" : [["CONFIG_DATA", "+NEW_SECTION"], ["METRICS", "-
ANOTHER_SECTION"]]
}
}
}
}
Customizing Templates
All report templates reside in the product installation directory located at:
%VECTORCAST_DIR%\python\vector\apps\ReportBuilder\templates\*
Best practice is to use a Configuration Directory to override the default templates. If a template exists in
the Configuration Directory, it will be used over the template located in the installation directory.
The Configuration Directory can also be the location to create new reports.
This command copies all the templates and adds a .orig extension.
Template files are located in the templates\<section_name> directory. All template files have a
.orig extension, so by default, they are not used. To use the template in the custom report directory,
you must remove the .orig extension.
Template files use a full-featured template engine for Python called Jinja ( http://jinja.pocoo.org/ ),
which allows:
To accomplish this, users need to configure VectorCAST to use custom templates. The following steps
are used to set up the configuration:
1. First, to override the default report templates, it is recommended that you create a configuration
directory to store and create new reports. For our example, we are creating a new configuration
directory named CustomDirectory. This will copy all the default report template directories into our
new directory and add a .orig extension to each file.
To customize reports, locate the report template in the configuration directory, remove the .orig
extension and modify the file as needed.
the file in your editor and add the desired footer text after the closing </div> tag, and before the closing
</body> tag, as shown below.
Save the changes and generate a report. The footer is displayed at the bottom of the report.
Open the file in your editor and override the default VectorCAST logo with your own Base64 encoded
image.
Save changes and generate a report. Your logo is displayed in the title bar of the report.
The styles contained in this style sheet are embedded into the <style> section of the HTML
VectorCAST reports when they are generated.
Your custom style sheet is appended when reports are generated and the contents of the custom style
sheet are embedded in the <style> section following the default styles, as shown in the example
below:
<!DOCTYPE html>
<!-- VectorCAST Report header -->
<html lang="en">
<head>
<title>Report</title>
<meta charset="utf-8"/>
<style>
default styles...
user's custom styles...
</style>
</head>
...
Any user defined styles will follow standard cascading style sheets rules and will correctly override the
defaults, meaning that only the specific attributes of a style must be specified.
In the example below, we are modifying our custom style sheet to change the report header background
color to green and make it twice as wide.
To add a custom style sheet, Choose Tools => Options and click the Report tab => Format sub-tab
=> HTML sub-tab. Use the browser button to select the location of your custom style sheet.
Environment variables in the path to the custom style sheet are supported, using the syntax $(ENV_
VAR). Click OK to apply the custom style sheet.
Once the custom style sheet is added, it is applied when generating a report. In the example below, we
see our custom-style.css is applied with its modifications to the title bar.
Embedding can cause problems with some web servers. To address this, VectorCAST provides the
ability to set the option to False. When the option is set to False, the styles contained in the default
style sheet (and any custom style sheet) are generated into a local style file referenced by the report.
Style sheets and images are stored in the same directory as the report, which is useful when serving
reports from a web server.
When True, HTML reports are created as a single, self-contained file with an
embedded stylesheet and embedded images (for all built-in, non user-modified
reports). Set the option to True (default value) when planning to email
reports. When False, HTML reports are created as multiple files, storing
stylesheets and images in the same directory as the report. Set the option to
False when serving reports from a webserver.
Report Components
There are three components to writing a new report: a main report file, a section file, and a template
layout file. The table below lists the files used to create an example of a new report, named My Report,
along with the location of each file and its purpose. Each component is discussed in more detail in the
following sections.
Report File
The report file, my_report.py, is the main entry point for the report. This file sets up any initial data,
defines the report name, and describes the sections of the report and the order in which they appear.
import os
from vector.apps.ReportBuilder.custom_report import CustomReport
Section File
The section file, example_sections.py, contains the data that goes into the report. Report data is
typically extracted using the Manage Data API (although it is possible to retrieve the data from
elsewhere).
class ExampleSection(ReportSection):
# Define the title (this shows in the config data and the Table of Contents)
title = 'Results Section'
# Sections can be conditional on type of environment - in this case, this will
be used for both
supported_environments = ('COVER', 'UNIT')
# prepare_data is called to setup any data required for generating the report
def prepare_data(self):
self.section_context['rep']="Execution Results Summary"
self.section_context["results"]=[]
api=Api('.')
a=api.TestCase.all()
for test in a:
result={}
result["name"]=test.name
result["verdict"]="PASS" if test.passed else "FAIL"
self.section_context["results"].append(result)
The layout template file, main.html.template, is a Jinja template that describes the layout of the
data. Note that:
> Elements in double curly braces {{}} are Python data objects from the section file
> Can also contain regular HTML
>> But do not include styles. Presentation should be placed in the CSS file.
<table style="width:100%">
{%- for result in results %}
{%- if result.verdict == "PASS" %}
<tr class="success">
{%- else %}
<tr class="danger">
{%- endif %}
<td>{{result.name}}</td>
<td>{{result.verdict}}</td>
</tr>
{%- endfor %}
</table>
For example:
clicast -e my_env report user my_report
To generate new reports from the GUI, the report can be configured as a custom tool which will run the
clicast command.
The script can accept any report name, so it can be used for any customised report.
The following figure illustrates how the components used to create a custom report work together to
produce the final custom HTML report.
If you have specified an external editor on the Tools => Options dialog, GUI tab, then the text file
opens in that editor.
The command may also be launched by clicking on the Open button in the toolbar.
By changing the “Files of Type” selection, you can also use File => Open... to open other kinds of files
such as source files. Files are opened in a Text Editor or the external editor, if specified.
C or C++ source code file C/C++ Source Files (*.cpp, *.cc, *.cxx, *.c,
*.c++)
Ada Source code files Ada Source Files (*.adb, *.ada, *.ads)
You can save a changed test case if it is in the MDI window. If a source code file, test script, or
configuration file is currently open in the MDI window, then it is saved. If a results file report, data file, or
anything listed in the Environment => View or Test => View menu is saved, then the Save As dialog
opens, allowing you to give the file a name.
The command may also be invoked by clicking on the Save button on the Toolbar .
The command may also be invoked by clicking on the Save All button in the toolbar.
The command may also be invoked by clicking on the Save All button in the toolbar. When you
use the Save All button, each tabbed window is saved. If you have several different windows open and
tiled or cascaded, then each tab in each visible window is saved. In cases where a filename is required,
you are prompted once for each tabbed window that needs a filename.
Print Setup
File => Print Setup... enables you to customize your printing options, such as headers, footers and
margins.
Header and Footer: Add a header and/or footer text string to your printed report.
Margins: Customize the size of the margins in the report output. The margins options are: Left, Right,
Top and Bottom. You can enter the desired margin value for each option. You can also use the spin
arrows to select the margin values.
Use Colors: Print coverage reports with the colors chosen in Tools => Options, Coverage tab,
Coverage Viewer sub-tab. Colors can be chosen for covered lines, partially covered lines, uncovered
lines, and uncoverable lines.
Use Backgrounds: Print the background color specified in Tools => Options dialog, Coverage tab,
Coverage Viewer sub-tab. The default is to print using the foreground color only. Printing with
background colors enabled can use a lot of ink or toner.
Reverse Foreground and Background: Reverse foreground and background colors when printing
coverage reports. It is useful when the selected coverage foreground colors are light and the
background colors are dark, saving on ink or toner.
This command can also be accessed by clicking the arrow on the right of the Print button on
the Toolbar and then selecting Print Preview… from the drop-down menu.
To Redo
Edit => Redo reverses the action of the Undo button. If you select Undo by mistake, Redo will
correct the error.
You can also move text using cutting and pasting. This means you can remove text from one place and
move it to another. To cut and paste do the following:
The Find Banner appears at the bottom of the window, and is off by default.
1. Open a file in an Editor, a report in a Viewer, or open a Parameter or Test Case Tree.
2. Click the mouse within the window where you wish to perform the search.
3. Choose Edit => Find... from the Menu Bar (or press Ctrl+F) and a Find Banner appears at the
bottom of the Text Editor.
Execute a Search
1. Type the text you are looking for in the Find text edit box. To choose text you have typed before,
click the drop-down arrow, and select text from the list.
2. To specify that the case is matched exactly as you typed it, click the checkbox next to Match
Case.
3. When executing a search in the Test Case Tree, you must first select the columns to be included
in the search. Click the Find button drop-down menu, and from the menu, select the column
names.
> In the Text Editor, Parameter Tree and Test Case Tree, the search will continue to cycle
through from the beginning.
6. To search for the previous instance of the text, click the green up arrow .
> In the Coverage Viewer, Report Viewer and User Code Editor, if the search text between
the current line and the top of the file is not found, you are prompted to search from the
bottom. Click OK to continue searching, or Cancel to quit the search.
> In the Text Editor, Parameter Tree and Test Case Tree, the search will continue to cycle
through from the bottom.
7. In the Parameter Tree, Text Editor, and Test Case Tree, if the text is not found, the text entry field
displays a red outline.
8. In the Coverage Viewer, if the text is not found you are notified that the search failed.
Repeat a Search
To search again without using the Find dialog, choose Edit => Find Again from the Menu Bar, or press
the F3 key.
To dismiss the Find Banner, click the button located next to Find on the banner.
To apply a filter to the Test Case Tree, first select the columns to be included in the search. Click the
Find button drop-down menu, and from the menu, select the column names.
The Find drop down menu is a tear off menu. Click once on the dotted line, and a floating menu stays on
the screen.
Next, select the Filter box on the Find Banner. Notice that after selecting Filter, two filter indicators are
displayed. The menu bar Test item is now displayed as Test *, and the Test Cases header in the Test
Case Tree is now displayed as Test Cases (*Filter Active). These indicators make it easy to detect if
a filter has been applied to the tree.
Enter the search value and press the Enter key. The Test Case Tree will now filter to only show nodes
containing the search value.
To change the search term and reapply the filter, enter the new term in the search text box, click Filter
to uncheck, and click Filter again to check it. The filter for the new term is applied and the results will be
displayed.
In this example, the TUTORIAL_ADA environment is executed and a Test Case Data Report is
generated (Test => View => Test Case Data). A portion of the Test Case Data Report is shown
below. The report contains information for all test cases.
A filter is applied for PLACE_ORDER.001 in the Test Case Tree, and the report is re-executed. (Test
=> View => Test Case Data). The report now shows information for only the test case that is
displayed for the filter.
A test script is then exported from the TUTORIAL_ADA environment level of the tree. TUTORIAL_
ADA is selected, then, Test => Scripting => Export Script is selected. The following test script is
generated. Note that the only test case in the test script is for the test case returned by the filter. In an
unfiltered tree, the test script would contain all test cases for the environment.
-- VectorCAST 20 (04/05/20)
-- Test Case Script
--
-- Environment : TUTORIAL_ADA
-- Unit(s) Under Test: MANAGER
--
-- Script Features
TEST.SCRIPT_FEATURE:MULTIPLE_UUT_SUPPORT
TEST.SCRIPT_FEATURE:MIXED_CASE_NAMES
TEST.SCRIPT_FEATURE:ADA_DIRECT_ARRAY_INDEXING
--
-- Test Case: PLACE_ORDER.001
TEST.UNIT:MANAGER
TEST.SUBPROGRAM:PLACE_ORDER
TEST.NEW
TEST.NAME:PLACE_ORDER.001
TEST.NOTES:
This test is the same as the one created by following all of
the steps in the "Ada Tutorials -> Basic Tutorial" from the
VectorCAST Getting Started manual.
It shows the basic concepts associated with setting input and
expected values for both the Unit Under Test and Stub Functions.
TEST.END_NOTES:
TEST.VALUE:MANAGER.PLACE_ORDER.TABLE:2
TEST.VALUE:MANAGER.PLACE_ORDER.SEAT:1
TEST.VALUE:MANAGER.PLACE_ORDER.ORDER.SOUP:ONION
TEST.VALUE:MANAGER.PLACE_ORDER.ORDER.SALAD:CAESAR
TEST.VALUE:MANAGER.PLACE_ORDER.ORDER.ENTREE:STEAK
TEST.VALUE:MANAGER.PLACE_ORDER.ORDER.BEVERAGE:MIXED_DRINK
TEST.VALUE:DATABASE.GET_TABLE_RECORD.DATA.NUMBER_IN_PARTY:0
TEST.VALUE:DATABASE.GET_TABLE_RECORD.DATA.CHECK_TOTAL:0
TEST.EXPECTED:DATABASE.UPDATE_TABLE_RECORD.DATA.IS_OCCUPIED:TRUE
TEST.EXPECTED:DATABASE.UPDATE_TABLE_RECORD.DATA.NUMBER_IN_PARTY:1
TEST.EXPECTED:DATABASE.UPDATE_TABLE_RECORD.DATA.ORDER(1).DESSERT:PIE
TEST.EXPECTED:DATABASE.UPDATE_TABLE_RECORD.DATA.CHECK_TOTAL:12..16
TEST.END
The environment level is executed and the filter is then removed to display the entire Test Case Tree.
Notice that coverage is available only for the test that was selected by filtering. This is indicated by a
Check Box next to the previously filtered test case name.
To apply a filter to the Parameter Tree, select the Filter box on the Find Banner. Enter the search value
and press the Enter key. The Parameter Tree will display the unit node associated with the filtered
results. Note that (Filter Active) is displayed next to the Parameter column heading, making it easy to
identify when a filter is being applied.
A filter can be applied to any column or combination of columns in the Parameter Tree. To select
column filters, click the Find button drop-down menu. From the menu, select the column names. Note
that an asterisk appears in the selected column heading to indicate that the column is used during
search/filter actions. Alternatively, you can use Alt+Click on the column headings to toggle the filter on
and off.
To Repeat a Search
The Edit => Find Again command enables you to repeat a search for previously specified text.
The shortcut for the Edit => Find Again command is F3.
To Goto a Line
The Edit => Goto Line command enables you to go to a specific line number in User Code and source
code in Text Editors. The file line numbers start at 1. This command is available in any text file: script
files, source files, text reports, and results files.
The shortcut for the Edit => Goto Line command is Ctrl+G.
At the bottom of the Text Editor window a Goto dialog opens. Insert the line number and hit Enter. The
specific line is highlighted in the code.
Note: The file line number is the source file line number, not the executable source code line
number.
The Configuration Options viewer provides a filter to search for relevant options. As the user types, the
viewer dynamically filters the list of settings, presenting only those configuration options that match the
input criteria. Highlighting an option in the list displays the description of the selected option in the lower
pane, where the matches to the search criteria are highlighted in red.
Upon selection of an option in the list, the corresponding Options editor interface is immediately brought
into focus and the option field is highlighted in yellow. This allows the user to edit or review the current
value of an option without the need to navigate through multiple layers of nested tabs.
The File => Set Working Directory command is used to change the current working directory before
creating a new environment. This command is not available when an environment is open. When you
select this command, a dialog appears enabling you to navigate to the directory of choice or create a
new directory.
If you select a directory which contains spaces or does not have read / write permissions, you will
receive the following error and be prompted to return to the Set Working Directory dialog to set a valid
directory.
The “Remember last working directory” option is located on the Tools => Options dialog, GUI tab.
Whenever this option is enabled, VectorCAST opens in the last-used working directory, regardless of
how the VectorCAST application is invoked.
When you start the VectorCAST application from the command line, the working directory is the
directory from which you invoked the application, unless the “Remember last working directory” option
is set.
When you start VectorCAST from the Windows Start menu, the working directory is the Environments
directory in the VectorCAST installation directory, unless the “Remember last working directory” option
is set. On Windows, you can change the default working directory by modifying the Windows properties
of the VectorCAST shortcut.
Note: The working directory must not contain spaces and must have Read and Write
permissions. If you start VectorCAST from such a directory, the status bar has a red outline. The
Set Working Directory error dialog appears, prompting you to enter a valid directory.
If you install VectorCAST in the C:\Program Files\ directory, then the Environments directory
will have spaces in its path and therefore will be invalid. If this happens, set the working directory
to a different location on your drive.
The current working directory is indicated in the status bar in the main application window.
If you start VectorCAST from such a directory, the status bar has a red outline. The Set Working
Directory dialog appears, prompting you to enter a valid directory.
See also "To Set Defaults for the Wizard" on page 114
Alternatively, the New button on the toolbar has a small arrow to the right. Clicking this arrow
causes a popup menu to appear, which lists the types of environments that can be built.
In the figure below, the user has licensed: VectorCAST/Project, VectorCAST/C++, VectorCAST/Ada,
one or more Ada targets and VectorCAST/System Testing. Your menu may differ, depending on the
products you have licensed from Vector.
If you have only one product licensed from Vector, then clicking the New button on the toolbar brings up
the wizard to create a new environment of that product type.
Tip: The Create New Environment wizard reflects the current settings on the Tools => Options
dialog, Builder tab. If you consistently want to create environments with Whitebox checked or
Coverage on, for example, choose Tools => Options, and go to the Builder tab to set these
options. Then, the next time you choose File => New => Environment, those settings are
already specified, saving you time.
If saving the environment script would modify the existing environment script of the same name,
VectorCAST asks if you want to overwrite the original environment script with the changed version.
To create an environment script from a currently open environment, use Environment => Scripting
=> Create.
On this page, you select the compiler. The preprocessor and compiler commands must be on your
default system PATH.
GNAT options are enabled only when the compilation system is set to GNAT.
Environment Name: Enter a unique name you wish to give to the new environment. Any space
characters you type are converted to “_” characters. A directory is created with this name, and a file is
created with this name and a “.vce” extension. If a directory with that name already exists in the working
directory, the name entry is outlined in red. The tool-tip asks you to choose a unique name for the
environment.
Tip: When loading an environment script that was saved with Relative Paths, ensure that your
current working directory is the same as it was when the script was saved.
If you need to modify an open environment, say, to turn on Whitebox, add a Search directory/Parent
Library, or change the stubbing strategy, use Environment => Update Environment. See also, "To
> Coverage type (optional) - Although coverage can be added at any time after the environment has
been built, it is recommended that you select coverage here in the wizard for increased speed and
efficiency.
> Whitebox (optional) - A Whitebox environment provides visibility to data defined in the package
body, as well as the package spec. Unchecking this box allows visibility only to data defined in
the package spec.
> Do not allow recursive calls in the test harness (optional)
To Turn on Coverage
Coverage Type: Choose a type of code coverage for the environment from the dropdown menu, if
desired. To build an environment without code coverage, set the Coverage Type to NONE.
To turn off coverage in an open environment, use the Environment => Update Environment
command.
To set a Coverage Type as the default setting when you open the Create New Environment wizard, use
the Tools => Options dialog, Wizard tab, Coverage Type option.
You can initialize code coverage at any time after the environment is built.
To Turn on Whitebox
Whitebox: If you wish to perform a Whitebox conversion, check the box marked Whitebox. The
unchecked state is the default, unless you have checked "Whitebox" on the Tools => Options dialog,
Wizard tab.
Whitebox gives you access to data visible in the UUT package body (objects and subprograms) and
includes access to private types in the UUT. Whitebox therefore allows access to everything at the
package level, both spec and body, and everything in packages nested one level down.
> Inserts a nested package definition to a UUT specification to allow manipulation of private types
and access to subprograms defined in the body of a UUT.
> Inserts a package body separate definition to a UUT body to correspond to a UUT specification
change.
Out-of-date units VectorCAST makes every effort to re-compute the compilation order and recompile
any out-of-date units that result from re-compiling the UUT specification and body. In rare cases, link
errors from out-of-date units may occur. If they do, you must recompile the out-of-date units into the
VectorCAST Ada library. For specific details regarding your compiler, please contact VectorCAST
Technical Support: support@vector.com.
Circular References In some cases, a whitebox conversion may result in a circularity during link. This
is generally caused by a unit in the dependency list of the UUT "with"ing the UUT. Because
VectorCAST-created stubs depend on harness files, and these harness files depend on the Unit Under
Test, a circular dependency can result. This problem can be avoided by creating stubs for dependent
units.
Due to the inherent invasiveness of the Whitebox technique, the following limitations should be noted:
> Nested packages must use named notation for the end statement
> A source file can only contain a single package body
> Private types defined in nested package specification are not supported
As an alternative to Whitebox and its modifications to source files, try using the Builder option “Multiunit
whitebox” instead.
> Green Hills 95 or ObjectAda compiler – the text on this page reads “Select Directory Containing
ADA.LIB.” Navigate to the location of the ADA.LIB file.
> ActivAda compiler – the text reads “Select Parent Library Directory.” Navigate to the directory
which is the parent library directory. This Parent Library can be over-ridden with the environment
variable VCAST_PARENT_LIBRARY. See Appendix E.
> GNAT compilers – select the Library specification method by selecting either the GPR file or
Directory radio button. Selecting GPR file, the text reads “Select GPR file”. Selecting Directory,
the text reads “Select Directory Containing .ali files”.
> DACS compiler – the text reads “Select Ada Library file.” Navigate to the directory that contains
the .alb file.
> SCORE compiler – the text on this page reads "Select Ada Libraries." Click the Add Library
button, navigate to the location of a SCORE library file (.alb) and click Open.
> Library specification method – select either GPR file or directory
To use relative paths as the default setting when you open the Create New Environment wizard, use
the Tools => Options dialog, Wizard tab, “Use relative paths for parent library” option.The target I/O
directory will also use relative paths is this option is selected.
Note: The Library/Directory path may be entered with a relative or absolute path consistent with
the default directory. See Appendix D for details specific to a particular compiler.
Note: :On platforms where the operating system is case sensitive (i.e. Linux) the exact case
matched path must be entered.
Specifying Unit(s) Under Test: The units found in the directories specified in the Source directories
list are displayed on the left, under the tab labeled “Unit Names.” Removing or adding a directory to the
Source directories list changes the items in this list.
To specify a selected unit as a Unit Under Test, move it from the left to the right. It appears in the
UUT selection list under the UUT icon ( ). You may specify more than one unit as a Unit Under
Test.
In the following example, the units MANAGER and DATABASE are the Units Under Test. They are
positioned under the UUT node ( ) of the UUT Selection list.
Stubbing Type: When unit testing, it is often desirable to stub functions called by the unit you are
testing. Stubbing allows you to test without waiting for dependent units to be completed and tested.
VectorCAST stubs enable you to capture parameters passed in, control parameters passed out, and
raise exceptions.
When you first open the Create New Environment wizard, the default stubbing strategy is to stub All of
the dependent units. The radio button selected in the Stub Dependencies group reflects the stubbing
strategy chosen for all of the dependents of the Units Under Test. (You can change the default by going
to the Tools => Options dialog, Wizard tab, and changing it to None or Custom. Thereafter, when
you open this dialog, it will default to that setting.) Choose Ignore if you want to use the real source code
for each dependent, but save time by not parsing the dependents or any of their dependents. You can
get code coverage data on Ignored units.
There are several choices for the stubbing strategy for all dependant units of the UUTs:
>
> All - Stub all dependent units for all UUTs. Every subprogram in the file is stubbed; the default
> None – Do not stub any dependent units of the UUTs.
> Ignore - Do not stub the dependent units or any of their dependent subprograms. This option is
useful when, for example, a specific unit is the interface to a set of dependent units that have
already been tested. Using Ignore can sometimes dramatically reduce the time it takes
VectorCAST to build an environment because it will not parse dependents of an ignored unit.
> Custom - Stub only those dependent units indicated interactively; ignore or don't stub others. see
below for more information.
> If you click the radio button to stub “None” or “Custom,” all the dependents of the UUTs are not
stubbed, except for the ones you place under the STUB or IGNORE nodes.
> If you click the radio button for “Ignore,” all the dependents of the UUTs are ignored, except for the
ones you place under the STUB node.
> If you click the radio button for “Stub All,” all the dependents of the UUTs are stubbed, except for
the ones you place under the IGNORE node.
In the example below, EARTH is a UUT. We have indicated to Stub None, but we plan to “override”
that setting to stub one unit, FRANCE, and ignore one unit, GERMANY.
In our example, we drag over the unit FRANCE and drop it onto the STUB node.
Once this environment is built, VectorCAST writes the following to the environment script (.env).
ENVIRO.UUT: EARTH
ENVIRO.STUB:FRANCE
ENVIRO.IGNORE:GERMANY
Note: If “Ignore” is selected in the Stub Dependencies group, any units under the IGNORE node
are superfluous. Similarly, if “All” is selected in the Stub Dependencies group, any units under
the STUB node are superfluous.
> Yes, stub – Stub this unit; any dependents are lost.
> No, don’t stub – Do not stub this unit; any dependents are parsed and can have their own setting.
> Ignore – Do not stub this unit or parse any of its dependents. The remainder of the closure for this
dependent is ignored.
Also, the environment variable can be set to the full path to a filename. If this file exists, units will
be read from this file, with one unit per line. This is especially useful when the list of units is long,
or when sharing lists among users.
This section briefly describes the types of environment user code. For a complete description, see
"User Code" on page 380.
User Globals - User Globals provide a mechanism for user-defined types and objects to be included in
the test harness. By default, there are five integer objects, five floating point objects, five string objects,
and an array of 200 integer elements. All data objects that are defined in User Globals when the
environment is built can be manipulated as test data when building a test case.
Preface any variables defined here with VCAST_USER_GLOBALS_EXTERN to ensure that only one
definition of the variable is created in the test harness. They will be listed in the Parameter Tree under
USER_GLOBALS_VCAST.
User Params - User Params are used to instruct VectorCAST to use a user-defined object for a
parameter, rather than a VectorCAST-generated object. User Params are used to provide the
association between the user-defined parameter and the subprogram's parameter. A syntax example is:
<<manager.Place_Order.Table>> VECTORCAST_INIT1
User Code - User Code is best suited for operations that relate to the harness as a whole; loading data
from a file or initializing a database unit are two examples. It enables you to write source code to handle
some of the harness tasks that are not easily accomplished with static data. With user code, you can
write code to read data from a file, call initialization routines, or assign and verify data objects based on
dynamic criteria.
Unit Appendix User Code - Unit Appendix User Code is user code that is appended to the end of the
specified UUT (and therefore added to the test harness). Because Unit Appendix User Code is
considered part of the specified UUT, it is useful for #including a concrete subclass for an abstract
class, as well as many other purposes.
You have access to all types of environment user code after the environment is built by choosing
Environment => User Code => Edit.
If you are not familiar with Environment User Code, see "Types of Environment User Code".
Step 7: Summary
This page of the wizard, called Summary, displays a summary of your settings. If there is any missing
information, the problem is displayed in red text. In particular, it checks that the compiler and
preprocessor are on the system PATH and that there are no empty fields. Missing information must be
supplied before the Build button is enabled.
If you chose “Custom” stubbing, a dialog box presents you with the choice to stub or not stub or ignore
any dependent units that contain subprograms.
See also “To Specify Custom Stubbing Interactively” on page 94 for information about Custom
stubbing.
You can also use *.env for <scriptfile>. CLICAST is passed a command
line with each <filename>.env found in the directory. On Linux, to avoid
having the command line get too long after expansion, you can quote the
argument to CLICAST, using “*.env” for <scriptfile>. This form passes
CLICAST the actual wildcard string for it to expand.
If the compiling or linking of the test harness component fails, the diagnostic messages for the compiler
are displayed, and the Message Window displays “Environment Built but not compiled”, or
“Environment Built but not linked” or “Environment Built but run-time errors exist.” After correcting the
errors, choose Environment => Rebuild Environment.
> Environment Name – the name of the environment, the VectorCAST internal version number,
and the environment status.
-------------------------------------------------------------
Environment Overview
-------------------------------------------------------------
Environment Name: ADA_1
Environment Status: Normal
Environment Version: REVISION_2020
Environment Build Type: Full
Coverage Status: Not Initialized
Language: Ada
Compiler Name: GNAT
Parent Lib: C:\VCAST\Examples\environments\tutorial_ada
Units Under Test: MANAGER
DATABASE
Stub List: None
Ignored List: None
The following table lists some of the files that are contained in the working directory. Under normal
conditions, you will not need to open or edit these files.
Note: These files do not need to be maintained for configuration management or version
management. The Environment => Create Regression Scripts command creates the three
files that need to be archived for each environment (unless you have imported coverage results,
which adds one more).
Env.vce
clicking this file or its icon launches VectorCAST and
opens the environment.
For example, in the TEXT Compiler Output listing shown below, we have highlighted filename data_if_
adacast_pkg.ads and right-clicked on it. Selecting Open data_if_adacast_pkg.ads from the popup
menu opens that file. (This feature does not work for HTML reports on Windows, because VectorCAST
has an embedded IE browser.)
This file can be viewed from within VectorCAST using the Environment => View =>
Environment Build Log command. The log data includes information written to the Message window.
by clicking the arrow ( ) to the right of the New button on the toolbar.
The Ada tab consists of configurable items applicable to your compiler. The options defined here should
be analogous to the options you use for source code compilation, when using the command line
interface to the compiler. Pass your cursor over any of the edit boxes to see an explanation of that
option in a tool-tip.
Compilation System
The Ada compiler to be used to compile and link the test harness.
Compiler Options
Library Options
Command line options to be used when creating the test Ada library.
Binder Options
GNAT only. Specify command line options to be used when invoking the binder.
Linker Options
Preprocessor Options
Command line options to be used when invoking the Ada preprocessor (only
available for some compilers).
Gnatchop Options
GNAT only. Use this field to enter options to be used by VectorCAST whenever
"gnatchop" is called, either during environment build/compile or coverage
instrumentation.
Debugger Command
This option overrides the default debug command that VectorCAST uses to
invoke the debugger.
Other Options
These options are not available in the VectorCAST Options dialog.
Assign a preprocessor for VectorCAST to use on Ada files when using the Green
Hills compiler.
Set this option to an appropriate .BLD file so that VectorCAST will compile
and link using the “Build” command instead of compiling each file separately.
Stub Dependencies
Choose Tools => Options, and click the Builder tab.
The Stub Dependencies feature provides the following choices for the default stubbing type displayed in
the Create New Environment wizard:
> All
> None
> Custom
Default setting for Stub Dependencies option in the environment builder. ALL
means stub all dependencies, NONE means do not stub any dependencies. Its
default value is YES (All).
Unit Type
Choose Tools => Options, and click the Wizard tab.
The default setting for Unit type in the Create New Environment wizard. Choose one of the following:
> UUT – allows stubbing only for functions external to the unit being tested
> SBF (default) – allows stubbing of functions within the unit being tested
The default setting for Unit type in the Create New Environment wizard.
UUT allows stubbing only for functions external to the unit being tested.
Stub By Function (SBF) allows stubbing of functions within the unit being
tested. The default value is SBF.
Coverage Type
Choose Tools => Options, and click the Wizard tab.
This option sets the type of coverage instrumentation to perform immediately after building the
environment. By setting this option to a value other than None, coverage will be initialized each time
you build a new environment.
The choices provided in the context menu are dependent upon the current Industry Mode (see "To Set
the Industry Mode for Coverage" on page 24 of this manual). If no Industry Mode is set, the Default set
of coverage types are used.
Here is a list of coverage types listed by Industry Mode and the clicast commands to set them:
Once the environment is built, the environment script has the line:
ENVIRO.COVERAGE_TYPE: <type>
where <type> is None | LEVEL_A | LEVEL_B | LEVEL_C
Automotive (ISO-26262)
> Unit Level ASIL D (Statement, Branch, MC/DC)
> Unit Level ASIL B/C (Statement, Branch)
> Unit Level ASIL A (Statement)
Once the environment is built, the environment script has the line:
ENVIRO.COVERAGE_TYPE: <type>
where <type> is None | ASIL_D | ASIL_B/C | ASIL_A
Industrial (IEC-61508)
> SIL4 (Statement, Branch, MC/DC),
> SIL3 (Statement, Branch)
> SIL 1/2 (Statement)
Once the environment is built, the environment script has the line:
ENVIRO.COVERAGE_TYPE: <type>
where <type> is None | SIL_4 | SIL_3 | SIL_1/2
Railway (EN-50128)
> SIL4 (Statement, Branch, MC/DC),
> SIL3 (Statement, Branch)
> SIL 1/2 (Statement)
Once the environment is built, the environment script has the line:
ENVIRO.COVERAGE_TYPE: <type>
where <type> is None | SIL_4 | SIL_3 | SIL_1/2
Medical (IEC-62304)
> Class C (Statement, Branch, MC/DC)
> Class B (Statement, Branch)
> Class A (Statement)
Once the environment is built, the environment script has the line:
ENVIRO.COVERAGE_TYPE: <type>
where <type> is None | CLASS_C | CLASS_B | CLASS_A
Default
> Statement
> Branch
> Basis Paths
> MC/DC
> Statement+Branch
> Statement+MC/DC
Once the environment is built, the environment script has the line:
ENVIRO.COVERAGE_TYPE: <type>
where <type> is None | STATEMENT+MC/DC | STATEMENT+BRANCH | MC/DC | BASIS_PATHS
| BRANCH | STATEMENT
Whitebox
Choose Tools => Options, and click the Wizard tab.
Default settings for the Whitebox checkbox in the Create New Environment wizard.
Default settings for the “Whitebox” checkbox in the environment builder. Its
default value is YES (whitebox) for C environments, NO (blackbox) for other
environments.
Once the environment is built, the environment script has the line: ENVIRO.WHITE_BOX: YES | NO.
Do not prompt for stubbing ancestors of the UUT. Its default value is false.
When a dependent unit is stubbed, VectorCAST creates a “dumb” stub for any unit “withed” by the
stubbed units’ specification. By setting this option, these “dumb” bodies will be compiled and linked into
the harness, allowing the execution closure to be completed with less effort.
This option tells VectorCAST that you want to recompile the body of the Unit Under Test before linking,
but after all stubs are compiled. This is useful when the Unit Under Test calls in-lined subprograms from
stubbed units.
Recompile the body of the UUT before linking, but after all stubs are
compiled. Default value is false.
Multiunit Whitebox
Choose Tools => Options, and click the Builder tab.
Set this option to allow manipulation of private types from all units in the environment, not just the Unit
Under Test.
If Multiunit Whitebox is not set, VectorCAST builds dumb stubs for private ancestors of the UUT.
Therefore, you will not be able to set any values for the parameters in the stub. If Multiunit White box is
set, VectorCAST build smart stubs for all private units, including ancestors of the UUT.
Set this option to TRUE to enable manipulation of private types from all
units in the environment, not just the UUT. The default value is false.
When this option is set, VectorCAST will process source files found in the mutual imports of the parent
view.
This option instructs VectorCAST to build stubs for subprograms that are defined using pragma import.
The pragma tells the compiler and linker that the source code for this subprogram is provided by an
external source; you cannot supply an Ada body for this subprogram. VectorCAST builds the stub for
the imported routine and then exports it as appropriate. This option enables you to manipulate the
parameters of an imported subprogram as though the subprogram were a native Ada subprogram.
This option tells VectorCAST to build stubs for subprograms that are defined
using ‘pragma import’. The default value is false.
When this option is True, VectorCAST builds the harness using a different mechanism which allows
VectorCAST to get the appropriate index for almost every array, regardless of target limitations.
This option is used to build the harness so that direct array indexing can be
used on targets that do not support 'image' and 'value'. The default value is
false.
Newer versions of GNAT use different syntax for the ‘gnatchop’ command. Disable this option if ‘gcc --
version’ returns 3.4.3 or earlier.
Indicates if your version of gcc is later than 3.4.4 Default value is TRUE.
When an exception is propagated out of a UUT call, setting this flag to TRUE
will cause ADA.EXCEPTIONS.EXCEPTION_MESSAGE to be called. Default value is
false.
This option sets the default size of "extern" array objects in which the index range is not specified. For
example, if VectorCAST encounters an extern (e.g. extern int A[];) and does not find the actual
definition of A, it generates a definition of the size specified in this option.
Default size for unconstrained arrays where the index range is unknown or
very large. May be specified as [lower bound]:[upper bound] or just [upper
bound]. The default value is 10 for upper bound.
This option provides an upper limit for the length of an unconstrained string parameter.
This command is executed prior to full compilations of the test harness. It can be used to make
modifications to harness files before the initial compilation. Note that this command may also be called
for subsequent harness compilations, but it is not called before every compilation.
Wide-character Support
Choose Tools => Options, and click the Builder tab.
This option disable support for wide character when parsing Ada source code for environment building
and coverage instrumentation. When set, this option causes VectorCAST to parse Ada source code
using ADA.WIDE_TEXT_IO. When this option is off, VectorCAST reverts to using ADA.TEXT_IO.
If your source code contains the form feed character, turn this option off to properly build and instrument
an Ada unit test environment.
To disable wide-character support in order to instrument Ada source code with a formfeed character in a
Use Gnatmake
Choose Tools => Options, and click the Builder tab.
This option forces VectorCAST to use gnatmake to compile, bind, and link the test harness. If this
option is not set, VectorCAST will use “gcc” to compile the test harness files, “gnatbind” to bind, and
“gnatlink” to link.
If you set this option, VectorCAST will compile stubs that are created for generic packages.
Set this option to tell VectorCAST to generate either Ada83 or Ada95 compliant test harness.
When this option is False, VectorCAST inserts the Environment User Code using the old methodology
(first UUT). When the option is True and the environment is whitebox, environment user code is parsed
/ stored as follows:
> Any code at the beginning of the section is written to the USER_CODE_ADACAST package
> When the token --VCAST_UNIT: <unitname> is found in the Environment User Code, code
after that line is written to the appropriate unit body user code section.
> If <unitname> is not found, code is written to the last valid package
1 ENVIRO.USER_CODE_ONE_SHOT_INIT:
2 USER_GLOBALS_VCAST.INT1 := 1234;
3 --VCAST_UNIT: UUT1
4 UUT1_SPEC := 100;
5 UUT1_BODY := 1000;
6 --VCAST_UNIT: UUT2
6 --VCAST_UNIT: UUT2
8 UUT2_BODY := 2000;
9 ENVIRO.END_USER_CODE_ONE_SHOT_INIT:
Line 2 is inserted into the "Harness Initialization" subprogram in the common user code package.
Lines 4 and 5 are inserted into the "Harness Initialization" subprogram in the first UUT.
Lines 7 and 8 are inserted into the "Harness Initialization" subprogram in the second UUT.
This option causes the Ada environment builder to use gprbuild instead of gnatmake to compile and link
the test harness for Ada environments built with GPR files. Using this option may slow the build
process, as this command compiles objects in other languages that are part of your source code build
process.
This option allows the GNAT Project file to use 'package NAMING' to override GNAT's default file
naming convention.
The GNAT Project file uses 'package NAMING' to override GNAT's default file
naming conventions. Default value is false.
Set this option to cause VectorCAST to compile source code files without debug information.
By default, the Multiunit Whitebox test harness redefines TRUE and FALSE by using
STANDARD.BOOLEAN. Setting this option disables that override, necessary when user source files
use ‘STANDARD’ in a different context.
When this option is set, the environment builder does not generate the basis paths information
automatically, saving some time during environment build. The complexity column in the Metrics report
indicates "(not computed)" and all basis path report items are gray. Once the environment is built, the
basis paths can be computed at any time by choosing Tools => Basis Path => Generate.
This option enables preprocessing of the Unit Under Test (and any non-stubbed unit instrumented for
code coverage) when the preprocessor is configured in the GNAT Project file (.gpr) specified when
building the environment.
This option allows VectorCAST to use the GPR file to preprocess the Unit(s)
under test before parsing or instrumenting. Its default value is False.
Set this option to tell VectorCAST that Ada83-compliant source code should be generated even though
the compiler supports Ada 95.
> Click the Open button in the toolbar or use File => Open if you need to navigate to find the
environment you want.
> Choose File => Recent Environments to open a recently-opened environment from a list. The
most recently-opened environment is at the top of the list. Coverage environments are shown with
a green icon ; Ada environments with an orange icon ; and C environments with a blue
icon .
> Double-click the environment’s icon (Windows)
To use this method, the VECTORCAST_DIR environment variable must be set at the system level.
The Windows installation program does this for you.
> Use the command line:
Once you open an environment, the working directory is set to the location of the newly-opened
environment.
To Close an Environment
The File => Close Environment command closes the environment and any open files in the MDI
window. If any file is unsaved, you are prompted to save before it closes.
To Rename an Environment
Choose File => Rename Environment to give the current open environment a new name. A dialog
appears with the current name at the top and the new name is at the bottom. If the new name is not
unique, then it is outlined in red. Type a new name and click OK or Apply.
Once you click OK or Apply, VectorCAST renames the environment directory and the .vce file.
To Update an Environment
The Update Environment command provides the opportunity to easily make changes to an open
environment, such as adding another UUT, adding a source directory, making the environment
Whitebox, or changing the stubbing strategy. Updating the environment rebuilds the environment, and
re-initializes coverage, if applicable.
When you choose Environment => Update Environment, VectorCAST first creates environment and
test case scripts for the existing environment. It then moves the existing environment to a backup
directory. The Update Environment dialog appears, with all the information filled in for the current
environment. Here you can enter new information or click to a different step. Note that the Load button
is not available.
Click the Update button to begin building the environment with the changed settings. If coverage was
initialized in the former environment, it is initialized in the updated environment, and the following
message appears:
If there is some reason that VectorCAST cannot create the backup directory, then the user is alerted
and offered the choice to Retry or Cancel.
Once you have corrected the problem, click the Retry button and the rebuild process continues. Click
the Cancel button to stop the rebuild process.
See also "Using the Wizard to Create an Environment" on page 90 for more information on the various
steps in the wizard.
This command enables you to generate the minimum set of files that allow for the regeneration of the
tool options, test environment, and all test cases. The test environment directory does not need to be
maintained when you are not actively testing. These regression files contain everything you need to
recreate the test environment. You simply run the batch file or shell script and VectorCAST rebuilds the
environment, loads and runs the test cases, and creates a test case management report (by default).
When you choose Environment => Create Regression Scripts, the following dialog appears.
The destination directory is relative to the current working directory. You can use relative or full paths.
For example, entering “.” causes the scripts to be created in the current working directory. Once a valid
directory is entered, the Create button enables. Click Create to write the regression script files to the
directory specified. The Create button is dimmed if you choose the environment directory. The Results
tab shows that the scripts have been built successfully. Click Done to close the dialog.
Build the environment script, test script, and shell script (Linux) or batch
file (Windows) used to perform a complete regression test. The optional post-
processing script is invoked after the regression scripts have been created.
When the regression script is run to recreate the environment, a temporary file is created, named
commands.tmp This file contains the list of commands to build the environment, run all tests, and to
create a management report. The last line of the regression script uses commands.tmp as an
argument for the clicast command shown below. This clicast command runs all of the commands in a
single invocation, using a single license checkout, and single “open” of the underlying project, resulting
in enhanced performance.
The following are excerpts from the regression scripts for a Windows environment called TUTORIAL.
tutorial.bat (Windows)
del commands.tmp
false
commands.tmp
clear_default_source_dirs
tutorial.env
ENVIRO.NEW
ENVIRO.NAME:TUTORIAL
ENVIRO.COVERAGE_TYPE: NONE
ENVIRO.STUB_BY_FUNCTION:database
ENVIRO.STUB_BY_FUNCTION:manager
ENVIRO.STUB_BY_FUNCTION:manager_driver
ENVIRO.STUB_BY_FUNCTION:whitebox
ENVIRO.WHITE_BOX:NO
ENVIRO.MAX_VARY_RANGE: 20
ENVIRO.STUB: ALL_BY_PROTOTYPE
ENVIRO.SEARCH_LIST: C:\VCAST\Environments\
ENVIRO.TYPE_HANDLED_DIRS_ALLOWED:
ENVIRO.LIBRARY_STUBS:
ENVIRO.LINK_OPTIONS:
ENVIRO.END
tutorial.tst
--
-- Environment : TUTORIAL
--
-- Script Features
TEST.SCRIPT_FEATURE:C_DIRECT_ARRAY_INDEXING
TEST.SCRIPT_FEATURE:CPP_CLASS_OBJECT_REVISION
TEST.SCRIPT_FEATURE:MULTIPLE_UUT_SUPPORT
TEST.SCRIPT_FEATURE:STANDARD_SPACING_R2
TEST.SCRIPT_FEATURE:OVERLOADED_CONST_SUPPORT
--
TEST.UNIT:manager
TEST.SUBPROGRAM:(cl)Manager::PlaceOrder
TEST.NEW
TEST.NAME:(CL)MANAGER::PLACEORDER.001
TEST.VALUE:manager.(cl)Manager::PlaceOrder.Table:1
TEST.VALUE:manager.(cl)Manager::PlaceOrder.Seat:4
TEST.VALUE:manager.(cl)Manager::PlaceOrder.Order.Soup:Onion
TEST.VALUE:manager.(cl)Manager::PlaceOrder.Order.Salad:Caesar
TEST.VALUE:manager.(cl)Manager::PlaceOrder.Order.Entree:Steak
TEST.VALUE:manager.(cl)Manager::PlaceOrder.Order.Dessert:Pie
TEST.VALUE:manager.(cl)Manager::PlaceOrder.Order.Beverage:Wine
TEST.VALUE:whitebox.<<GLOBAL>>.
(cl).WhiteBox.WhiteBox.<<constructor>>.WhiteBox().<<call>>:0
TEST.END
--
You can then save these post-processing steps to a script or just perform them once. The following
instructions explain how to perform post-processing on your regression scripts.
This functionality is controlled by the checkbox “Additionally, post-process using the script.” After
checking this option, you can then type or load a script into the Script tab. When you click the Create
button, the regression scripts are created and then manipulated according to your instructions in the
script.
To see an example script, click the Show Example button. An example batch file (Windows) or shell
script (Linux) is displayed in the Script tab.
You can use this script, modify it, or clear it (click Clear Script) and write your own.
When you provide a post-processing script and click the Create button, the Results tab displays the
standard output generated by post-processing the regression scripts. Note that the script itself is not
saved by the Create action.
Build the environment script, test script, and shell script (Linux) or batch
file (Windows) used to perform a complete regression test. The optional post-
processing script is invoked after the regression scripts have been created.
See also "To Save and Load a Post-Processing Script" on page 136 for information on saving to a script
and loading in a script.
5. Change the script so that it runs the ClearCase commit call on the regression script files:
DEST_DIR = \home\user_name\project\regression\%1
if not exist $DEST_DIR mkdir –p $DEST_DIR
copy %2 $DEST_DIR
copy %3 $DEST_DIR
copy %4 $DEST_DIR
cd $DEST_DIR
cleartool commit *
6. To create the regression scripts and commit them using ClearCase, click the Create button.
VectorCAST creates the three (or four) regression script files for the environment and post-
processes them using the script you entered. You can save this script by clicking the Save
button.
7. Close the dialog by clicking Done.
To do this, click the Save icon . In the Save Script dialog, give the file a name and extension
appropriate for your platform. Once saved, the name appears in the edit box below Additionally, post-
process using the script.
The script name and its path is preserved in the registry (Windows) or in the .vcast-qt file (Linux). This
file name is loaded in the Create Regression Scripts dialog as the default post-processing script. Once
a post-processing script is saved, you can create regression scripts and post-process them with one
click.
You can load an existing script file (such as a project-level file created by someone else) by using the
browse button to the right of the edit box for the script. The post-processing script you load in
becomes the new default script.
See "To Post-Process the Regression Scripts " on page 133 to learn how to create a script that post-
processes your regression script files.
To save you time, VectorCAST can relink both blackbox and whitebox testing environments, without
requiring you to rebuild the whole environment after making changes to the source code. Keep in mind
that if you make changes to the source code other than logic, such as changing subprogram names, or
removing parameters, objects, or types, then compile errors may occur and you will need to rebuild. If
subprograms or objects are added, the environment will compile correctly, but these new items will not
be visible. In this case also you will need to rebuild the environment (using Environment => Rebuild
Environment).
Note: Before using the Relink command, the changed code must be compiled into the parent
library manually.
Note: If the program specification for the unit under test or a dependent unit is modified, a new
environment must be created.
If there is some reason that VectorCAST cannot create the backup directory, then the user is alerted
and offered the choice to Retry or Cancel.
Once you have corrected the problem, click the Retry button and the rebuild process continues. Click
the Cancel button to stop the rebuild process.
To Delete an Environment
The File => Delete Environment command deletes the current open environment or enables you to
select an environment to delete. Deleting an environment deletes the sub-directory that was created for
that environment and all files contained in the sub-directory. It leaves the .env file that can be used to re-
create the environment later. Note that the test cases are not saved for you. To avoid losing any work,
you should create Regression Scripts before deleting.
Note: The directory created by VectorCAST when building a new environment should only
contain VectorCAST-created files. Do not store any files that are not tool-specific in these
directories. When the File => Delete Environment command is invoked, all files contained in
the specified directory are deleted.
If you wish to delete the current open environment, choose Environment =>Delete Environment. You
are asked to confirm deleting the open environment.
If no environment is open when you choose File => Delete Environment, the standard Choose File
dialog appears, providing the opportunity to navigate to find any environment to delete. Locate the .vce
or .vcp file for the environment you want to delete. Click the Open button.
Build a new environment from the specified script file. You can also use
*.env for <scriptfile>. CLICAST is passed a command line with each
<filename>.env found in the directory. On Linux, to avoid having the command
line get too long after expansion, you can quote the argument to clicast,
using “*.env” for <scriptfile>. This form passes CLICAST the actual wildcard
string for it to expand.
If you delete the environment script, it can be re-created using Environment => Scripting => Create.
The menu item is dimmed unless you have an environment open. This command enables you to create
a script file that has all the necessary information to rebuild the current environment, including user
globals, stubbed units, and whitebox indication. To create a script, choose Environment => Scripting
=> Create. A Save File dialog appears, prompting you for a file name. Regardless of the file name, the
ENVIRO.NAME line in the script reflects the name of the currently open environment.
If an environment is currently open when you choose this command, it is closed and the new one is built
and opened.
When you choose Environment => Scripting => Run, the Open File dialog appears, prompting you to
open an environment script file (.env). These files are created by VectorCAST, but sometimes you may
want to copy one and slightly modify it as a text file. This command enables you to build an environment
by running an environment script.
If you attempt to build an environment from a script and a directory already exists with the same name
as the ENVIRO.NAME line in the script, the environment will not be overwritten and the build process
will terminate.
Build a new environment from the specified script file. You can also use
*.env for <scriptfile>. CLICAST is passed a command line with each
<filename>.env found in the directory. On Linux, to avoid having the command
line get too long after expansion, you can quote the argument to clicast,
using “*.env” for <scriptfile>. This form passes CLICAST the actual wildcard
string for it to expand.
Note: No actual compiling is performed by this command. The commands that would normally
be executed are dumped to a file. Execution of the script is performed with the Environment =>
Recompile => Run Script command.
Select Environment => Recompile => Create Script from the Main menu. Enter a filename to store
the compile script. If you are running under Windows, give the filename an extension .bat. If you are
running under Linux, give the filename the extension .sh. Click Save.
You will see this dialog box when the script has finished building.
rem # (S)
gnatchop -r -w vcast_ada_options.ads
rem # (B)
gnatchop -r -w vcast_ada_options.adb
gnatchop -r -w S0000007.ADA
gnatchop -r -w B0000007.ADA
gnatchop -r -w S0000008.ADA
gnatchop -r -w S0000002.ADA
gnatchop -r -w S0000004.ADA
gnatchop -r -w data_if_spec.ada
gnatchop -r -w B0000002.ADA
gnatchop -r -w B0000001.ADA
gnatchop -r -w S0000003.ADA
gnatchop -r -w B0000004.ADA
gnatbind -v -x -g -I\\\\dataserver/vcast_tutorial/ADA/ \
-aO\\\\dataserver/vcast_tutorial/ADA/uut_interface_adacast.ali
gnatchop -r -w I0000009.ADA
gnatchop -r -w I0000010.ADA
gnatchop -r -w S0000003.ADA
gnatbind -v -x -g -I\\\\dataserver/vcast_tutorial/ADA/ \
-aO\\\\dataserver/vcast_tutorial/ADA/ \
uut_interface_adacast.ali
gnatlink -g -aO\\\\dataserver/vcast_tutorial/ADA/ \
-o UUT_INST.EXE uut_interface_adacast.ali
\\\\dataserver
cd \vcast_tutorial\ADA\TUTORIAL
If an environment is currently open, select Environment => Recompile => Run Script. The following
dialog window opens, in which you may select the script you wish to run.
Once you have selected the script you wish to run, click Open. If you are on the Windows platform,
VectorCAST opens a Command Prompt (DOS box) and runs the batch file. If you are on the Linux
platform, VectorCAST runs a shell script. The Recompile is complete when you see the following text
in the Message window:
The Scripting Log is displayed in the MDI window, and it shows the output from the recompile operation.
See "To Create a Test Harness Recompile Script" on page 140 for information on how to create a
recompile script.
To Relink an Environment
The Environment => Relink command enables you to relink the object files of a previously created
test environment. This command should be used any time modifications are made to the source code of
the unit under test. You would then recompile your source files, and choose Environment => Relink to
By making modifications to the original source code file, compiling the file, and re-linking the
VectorCAST environment, you will be able to test your changes with your existing test cases. This
command is useful when you are operating in a find-and-fix environment and debugging problems with a
UUT.
If you make changes to the source code other than logic, such as changing subprogram names, or
removing parameters, objects, or types, then compile errors may occur and you will need to rebuild. If
subprograms or objects are added, the environment will compile correctly, but these new items will not
be visible. In this case also you will need to rebuild the environment (using Environment => Rebuild
Environment).
Note: Before using the Relink command, the changed code must be compiled into the parent
library manually.
Note: If the program specification for a unit under test or a dependent unit is modified, a new
environment must be created.
Many of the menu items in the Test menu, Tools menu, and of course the right-click menu, operate on
the item or items that are selected in the Test Case Tree.
At the top of the hierarchy is the environment name. Click this node if you want your next action to
affect the whole environment.
Next are the <<COMPOUND>> and <<INIT>> items. Click one of these special items if you
want your next action to affect only the COMPOUND test cases or INIT test cases, respectively.
The Units Under Test are listed at the next level of the hierarchy. They are listed alphabetically.
Click this a unit node if you want to your next action to affect all subprograms in the unit.
The subprograms for the unit are listed below the unit in the hierarchy, in the order they are specified
in the source code. Click a subprogram node if you want your next action to apply to that subprogram
only.
The lowest level of the hierarchy is the test case level. Click a test case if you want your
next action to apply only to that test case.
When selecting items in the Test Case Tree, you can combine items at different levels using
Shift+click or Ctrl+click actions.
name to differentiate between overloaded subprograms. Example: store (integer), store (float), store
(boolean).
To expand a node:
> Mouse: click the icon to the left of the node
> Keyboard: after placing the keyboard focus on that node, press the right-arrow key
> Menu: right-click the node and choose Expand All Children
To collapse a node:
> Mouse: click the icon to the left of the node
> Keyboard: press the left-arrow key
> Menu: right-click the node and choose Collapse All Children
Additionally, use the keyboard’s up- and down-arrow keys traverse the expanded nodes. You can also
type the first few letters of an expanded node name to jump to that node.
The same process is used to multi-select subprograms, units, or a mixture of test cases, subprograms,
and units.
A compound test case is a collection of simple test cases that perform a series of calls. A compound
test case provides the ability to invoke a UUT multiple times with a variety of data within a single
execution. Each test case in a compound test case can be executed one or more times. Compound test
cases can be used to test processing which is dependent on multiple calls to different procedures of a
UUT. For example, you may have processing that implements a database with one subprogram to write
to the database, and another subprogram to read from the database. In this case a compound test could
be developed to invoke a call to the "write" subprogram, followed immediately by a call to the "read"
subprogram. In this way you can verify the integrity of the stored data.
When executing a compound test case the Event header contains information about the compound test
case that is executing, the slot number executing, its name, and its iteration. For example, the following
Event header is displayed for event 1 when <<COMPOUND>>.001 is executed:
Event 1
<<COMPOUND>>.001 Slot 1 ((CL)MANAGER::PLACEORDER.001) Iteration 1
All simple test cases are associated with a subprogram of a Unit Under Test. Prior to building simple
test cases, it is necessary to select a subprogram to test. The test cases built for this subprogram are
applicable to this subprogram only.
When building a new test case, you may assign values to parameters of the subprogram under test, any
parameters of the stubbed dependent subprograms, and any global data objects.
Values are entered for individual scalar data items. VectorCAST takes you through a simple-to-use tree
expansion of the type mark for each data item until a scalar data type is encountered. You are then able
to enter a value for the item directly, or double-click to bring up a dialog box that is specific to that scalar
type.
> Min Mid Max - Min Mid Max tests stress a function at the bounds of the input data types. Min,
Mid, and Max test cases are useful when unit testing because they often uncover hidden
assumptions present in the code under test.
> Partitioned - Partitioned tests create "partitions" for each data type and test the min and max
value from each partition. The assumption is that values from the same partition will stimulate the
application in a similar way.
> MC/DC - MC/DC tests use the MC/DC analysis to examine the unique paths that exist through a
procedure. MC/DC tests can automatically create a high level of path coverage.
> Basis Path - Basis Path tests use the basis path analysis to examine the unique paths that exist
through a procedure. Basis Path tests can automatically create a high level of branch coverage.
For example, PLACE_ORDER.008 is the eighth test case for the subprogram Place_Order.
The two portions of the automatically generated name are separated by a period ('.'). Note that test case
names are limited to 80 characters in length.
The automatically generated name may be edited or completely replaced by right-clicking on the test
case name and choosing Rename, as described in "To Rename a Test Case" on page 149.
Test case names must contain only alphanumeric characters or punctuation (i.e., hyphens,
underscores, or periods). Percent, backslash and single or double quotes are not supported and cause a
message to pop up and the new name to be discarded.
If you rename a test case that is part of a compound test case, VectorCAST renames it in the
compound test automatically.
To sort the test cases alphabetically (for each subprogram), choose the Tools => Options dialog,
GUI, and set the option “Alphabetize test cases” on. As a result, as each test case is created or
imported, it is positioned in the list alphabetically, for each subprogram.
Select a test case to duplicate, right-click, and choose Duplicate Test Case. The new test case is
automatically named based on the name of the original test case.
For example, if you duplicate a test case for the subprogram Place_Order named STEAK$14, then the
new test case takes the name STEAK$14.001.
If a test case is used in a compound test case, then deleting the test case affects the compound test
case. In this situation, VectorCAST puts up another dialog message, as shown below. Click Yes to
delete the test case and remove it from any compound test cases. Click No to cancel the delete
operation.
Delete the specified test case. If <testcase> is part of a compound, add the
word "yes" to the command.
Delete all test cases from the subprogram in the specified unit. If any test
case being deleted is part of a compound, add the word "yes" to the command.
Delete all test cases from the specified unit. If any test case being deleted
is part of a compound, add the word "yes" to the command. Note, if <unit> is
the first UUT in the environment, then all test cases for <<COMPOUND>> and
The “Save backup copies” option is selected by default as an added failsafe to prevent accidental
deletion of test cases. This “backup script” is saved in the environment directory and named TEST_
DELETE.SAV.
Alternatively, right-click the top node of the Test Case Tree hierarchy (the environment name), and
choose Delete. Doing so deletes the test cases from the environment level, which includes all test
cases. You are asked to confirm the delete operation, but a backup file is not created.
Delete all test cases from the environment. The word “yes” must be added to
the command as confirmation.
See "To Restore Deleted Test Cases" on page 151” on page for information on restoring test cases.
The Delete All Test Cases dialog has a checkbox labeled “Save backup copies.” By default, this
checkbox is selected. When selected, a backup test case script is automatically generated to preserve
the deleted test cases. This test script file is called TEST_DELETE.tst and is created inside the
environment directory. If you need to retrieve the deleted test cases, simply import the test script (Test
=> Scripting => Import Script => env_dir/TEST_DELETE.tst) and all deleted test cases will be
restored.
See"To Rename a Test Case" on page 149 on page for information on renaming a test case.
Another useful feature for the <<INIT>> test case is as a setup step for compound test cases. You can
set the <<INIT>> test case to initialize global data, and then include that test case as the first slot in a
compound test. Doing so enables large amounts of initialization data to be entered one time for multiple
compound tests.
To create all three types, click a node in the Test Case Tree (subprogram, unit, or environment). Right-
click and choose Insert Min Mid Max from the drop-down menu.
Build the <<MIN>>, <<MID>>, and <<MAX>> special test cases for the
environment, a specified unit, or a specified subprogram.
To create only one type (min, mid, or max), right-click and choose Insert Min Mid Max from the popup
menu, and then choose Min (or Mid or Max) from the submenu.
Min-Mid-Max test cases are given the special names <<MIN>>, <<MID>>, and <<MAX>> in the Test
Case Tree. These special test cases cannot be edited, but you can derive new test cases from them.
To derive a test case from one of the special <<MIN>>, <<MID>>, or <<MAX>> test cases, double-
click on it. A new test case is created. For example, double-clicking on <<MIN>> that appears below
the subprogram Place_Order creates the test case PLACE_ORDER_MIN.001. In this test case, all
input values for the subprogram and stubs are already set to the minimum value, but you can override
specific values by entering data.
When you view the test case data listing, it indicates that all input data values are set to min, mid, or
max, as shown in the following example.
Test cases are added to the environment, named BASIS-PATH-001, -002, -003, etc., which correspond
to the basis paths for the subprogram. Because of unreachable paths and because some conditions
may be based on intermediate computations, the test cases that VectorCAST creates may not be
complete. After the basis path test cases are created, a Test Script Log is displayed, showing the test
case processing.
The Message window displays a summary of the automatic test case generation. The generated test
cases are complete, partial, or template tests.
> Complete – the test case was completely generated, with an input value for each data item
necessary to traverse the test path
> Partial – the test case was partially generated. The test case’s name has “-PARTIAL” appended to
it. The test case’s Notes tab contains a full description of which portions of the test case are not
complete and why. Example: a dependency on an intermediate computation (c=foo(); if
(c)).
> Template – VectorCAST was unable to determine any information necessary to traverse the test
path. The test case’s name has “-TEMPLATE” appended to it. As with partial test cases, the test
case Notes tab contains a full description of the reasons.
There are a number of reasons why VectorCAST may not be able to determine input values:
computations embedded in the conditional, local variables, etc. In cases where complete test cases
cannot be generated, the test case Notes tab describes any missing data that you must set up to
implement the test.
Basis paths are determined by parsing the code and applying the basis path algorithm to the list of
branch point nodes that result. As a result, it may be impossible to satisfy a particular branch point
when building test cases. In cases where a conditional is based on an intermediate computation,
VectorCAST is not able to control the decision point. The following is an example of this case:
Example 1:
ROAD_SPEED_LIMIT: constant := 55;
TRUCK_SPEED_LIMIT: constant := 50;
Example 2:
for INDEX in INDEX_TYPE' first..INDEX_TYPE' last
loop
RESET (INDEX);
end loop;
Because INDEX_TYPE' first and INDEX_TYPE' last are constants, the loop will always get executed
at least once. The basis path calling for the loop to not be entered cannot be satisfied.
Generate a test script containing one testcase for each basis path in the
environment for the specified unit, subprogram in that unit, or whole
environment.
To solve this problem, click the Parameter Tree tab to return to the Test Case Editor. If you know that
the subprogram under test would never get a value that is too large or too small, then enter your own
input values for any parameters that may be getting values that are too large or too small. On the other
hand, you may want to modify your source code to handle values that are too large or too small.
A compound test case consists of a series of "slots." Each slot contains the following information:
The name defaults to <<COMPOUND>>.001. You can change this by right-clicking on the test
case name at any time and selecting Rename from the context menu.
2. With your mouse, click-and-drag one or more test cases to the Compound Test Editor located in
the MDI area. You can drag and drop a test case to a new position within the Compound Test
Editor.
Each test case in a compound is called a slot. The unit name, subprogram name, test case name,
number of iterations, delay, and report status is displayed for each slot. The slots are executed in order
when the compound test case is executed. See " To Reorder Test Cases in a Compound" on page 162
to change the execution order.
A setting of 0 Iterations causes the test case to be skipped during execution of the compound. A
notation, "Slot <#n> not executed, iteration count is zero", is added to the Execution Report.
If you want global data to be displayed in the Execution Report after each slot iteration, select Tools =>
Options from the Menu Bar to open the Options dialog. Select the Report tab, Content sub-tab and go
to the Display global data after group. Click the radio button next to Each slot iteration and select
the Apply button.
Example: If you want to delay 2.5 seconds after the first test case slot is finished, enter 2.5 in the Delay
column for the first slot.
The default delay is zero. The delay feature is useful when testing processing related to synchronizing
calls.
In this example, <<COMPOUND>>.001 has been marked as an Automatic Initialization test. Tests set
as automatic tests have their name underlined in the Test Case Tree.
Alternatively, you can drag and drop a test case to a new position within the Compound Test Editor.
Multi-selection drag and drop is supported. When you select multiple slots and drop them in a new
position, the slots are dropped in the order in which they were selected. In this way you can either
duplicate slots or duplicate and re-order slots at the same time.
The test case opens in the Test Case Editor, located below the Compound Test Editor. Once changes
are made to the individual test case and saved, they are part of the Compound too.
Type the text you are looking for in the Find text edit box and press Enter.
The search looks in all columns (slot number, unit name, subprogram name, test case name, iterations,
and delay). So searching for the number 3 will find both slot 3 and any slot with Iterations set to 3, for
example.
For more details, see "To Search for Text Using the Find Banner" on page 77.
Change the specified test case to a “Compound Only” test case or back again.
"Compound Only" test cases are not executed in normal or in batch mode, but
only as part of a compound test case.
When a test case is "Compound Only" its name is italicized, and there is a check mark next to
Compound Only in the right-click context menu.
To specify that a test case is no longer Compound Only, right-click the test case and choose
Compound Only again, such that the check mark is removed.
To do so, simply drag an existing compound test case into a new compound test case. A compound
test can be limited to run only within a nested compound test case by setting it to "Compound Only."
See "Compound Only Test Cases" above.
In this example, PLACE_ORDER.001 and GET_TABLE_RECORD.001 are the only slots in the
compound test case named INNER. The number of iterations for GET_TABLE_RECORD.001 is set to
2. Another compound test case is inserted in the Test Case Tree, named OUTER, and then INNER is
dragged into OUTER.
Double-clicking on INNER in the OUTER Compound Tree displays the Compound Tree for INNER.
Execution report event headers for a nested compound event reflect the nesting relationship. Below,
Event 3 of the execution report for OUTER shows that OUTER executes INNER and INNER executes
the first iteration of GET_TABLE_RECORD.001. Scrolling down the report shows that in Event 5, the
second iteration of GET_TABLE_RECORD.001 in INNER is executed.
A compound test cannot be a nested slot of itself. If you drag a compound onto itself, the following error
dialog appears:
Nested compound tests do not support test recursion. For example, if OUTER nests INNER, and
INNER nests OUTER, the relationship is a recursive one.
The recursive relationship is detected when the change to the compound test is saved, not when the
nested compound is created. When a recursive relationship is detected, an error dialog appears when
To specify a compound test case as a slot in a compound test script, use syntax similar to the following
examples:
-- COMPOUND TESTS
TEST.SUBPROGRAM:<<COMPOUND>>
TEST.NEW
TEST.NAME:INNER
TEST.COMPOUND_ONLY
TEST.SLOT: "1", "MANAGER", "PLACE_ORDER", "1", "PLACE_ORDER.001"
TEST.SLOT: "2", "DATABASE", "GET_TABLE_RECORD", "2", "GET_TABLE_RECORD.001"
TEST.END
--
-- COMPOUND TESTS
TEST.SUBPROGRAM:<<COMPOUND>>
TEST.NEW
TEST.NAME:OUTER
TEST.SLOT: "1", "<<COMPOUND>>", "<<COMPOUND>>", "1", "INNER"
TEST.END
--
By default, reporting is enabled and the Enable Reporting icon is displayed in the far-right column
of the Compound Editor. To prevent the harness from reporting on executing data in the Execution
Report, click on the icon to toggle to the Disable Reporting icon .
From a test script, disable reporting by adding "PRINT=FALSE" to the end of the TEST.SLOT line. For
example:
Note: If the slot data contains expected results (either for the particular test case, or, in the case
of a nested compound, within the execution of the slot), then the compound test will fail to run
and will notify the user of the reason.
A warning dialog appears indicating the number of test cases that will be changed by the action. You
have the opportunity to abort the action by selecting the No button. Selecting the Yes button will allow
the operation to proceed and modify the test cases. Note that this action cannot be undone.
All Expected Values listed in the Execution Report are set to their actual values.
See Controlling Report Information for more information on capturing output for variables.
clicast -l<lang> -e <env> [-u <unit> [-s <sub> [-t <testcase>]]] TESt
Actuals_to_expected
This will set the actual values for a test case as the expected values. Thus
100% of the expected results will match for that test case.
There are two ways to enter test case data. Enter test data directly into the Parameter Tree by selecting
the appropriate spot in the “Input Value” or “Expected Value” column next to the parameter name.
Alternatively, double-click on any parameter or global object from within the Parameter Tree to bring up
the Parameter dialog box for the data object. The two methods are described below.
When entering parameter values for the subprogram being tested, all parameter values that will be
accessed during subprogram execution must be set. If no value is provided for a parameter or object, its
value is indeterminate. Some compilers zero the stack heap and therefore parameter values before
allocating data but this should not be relied upon.
Note: If you hover the mouse cursor over a cell in the Parameter Tree, VectorCAST displays the
legal range of values for that parameter in a tool-tip.
The Parameter Tree also contains a column to control which parameters are to be captured in the Test
Execution Report. This column is indicated by the icon. For more information on using this
control, see "To Enter Input and Expected Values" on page 173and "Controlling Report Information" on
page 250
Using the Tutorial environment as an example, the UUT is manager, and one of its subprograms is
Place_Order. Place_Order’s parameters consist of an integer type for the Table, an integer type
for the Seat, and a structure type for the Order. The dependent unit database is also a UUT. In this
case, the Parameter Tree looks similar to the following screen shot:
See also "Data Types" on page 176 for information on entering Input and Expected data for specific
data types.
To expand a node
Mouse: click the icon to the left of the node
Keyboard: after placing the keyboard focus on that node, press the right-arrow key
To collapse a node
Mouse: click the icon to the left of the node
Keyboard: press the left-arrow key
Additionally, use the keyboard’s up- and down-arrow keys traverse the expanded nodes. You can also
type the first letter of an expanded node name to jump to that node.
When you arrive at the node you would like to edit, simply press Enter. This will edit either the Input or
Expected value (depending on which can be edited at the time).While editing a value, press the Tab key
to continue to the next value, or use the up- and down-arrow keys to continue to the next editable node
in the same column.
Close any open test cases. Open a test case and all items for each node of the Parameter Tree are
To restore the order as originally processed by VectorCAST, uncheck Alphabetize Parameters in Tools
=> Options => GUI. Close any open test cases. Open a test case and all items for each node of the
Parameter Tree are displayed as originally processed by VectorCAST.
Input Values to a Unit Under Test are “in” parameters; Expected Values are returns or pointers.
Parameters passed to a subprogram in a stubbed unit are Expected Values; values returned from a stub
are Input Values.
Entering an Input Value or Expected value into the Parameter Tree automatically enables the data to be
displayed in the Test Execution Report. This is indicated by the in the parameter tree. For Input
Values, the data may be optionally hidden (See Controlling Report Information). Entered expected
values are always shown on the Test Execution Report and may not be hidden.
To Enter a Range
For any data type, you can enter a range for an Input or Expected Value in the Parameter Tree. In the
Input Values column, the range syntax is low..high/delta. The range causes the test case to
iterate over the input values starting at low, going to high, by delta.
In the Expected Values column, the range syntax is low..high and defines a range of values that will
match an input. The acceptable range includes low and high.
To Enter a List
For any data type, you can enter a list for an Input or Expected Value in the Parameter Tree. In the Input
Values column, the list syntax is value-1,value-2,value-n. The list causes the test case to
iterate over the input values starting with value-1, and continuing to value-n.
The special input value <<KEEP>> indicates that the input object should keep its current value from
the previous iteration, rather than being modified. This special input value is useful when an item needs
an initial value but an iteration is occurring and you don't want a new value set on each iteration. For
example, the input might look like:
123,<<KEEP>>,<<KEEP>>
In the Expected Values column, the list syntax is value-1,value-2,value-n and defines a list of
values that will match an input, one at a time. There is no space between the items. To specify that an
item be repeated, use parentheses around the number of times to repeat the value: (repeat)value-
1,(repeat)value-2,(repeat)value-n
Repeating and non-repeating list items can be mixed in the list. A range can be an item in an Expected
Values list.
When you add items to the Input Values list for a parameter, you are assigning a series of input values
to that parameter. Adding more than one item to the list causes an iteration, that is, the test case is
executed once for each item. Repeating a value in the list causes additional iterations.
Using an Input Values list for a stub parameter means that the stubbed subprogram returns one item
from the list each time it is called. If a stub is called more time than you have provided values for in your
list, then the last value in the list is used repeatedly, once the list items are exhausted. Each time a test
case iterates, the Input Values list for a stub is reset to the beginning.
When you add items to the Expected Values list for the same or a different parameter, you are
assigning a series of expected values for that parameter. One expected value is compared to the
parameter’s value per test iteration. The expected value can be a range of acceptable values for that
iteration, as described below.
For example:
package UUT is
procedure CONVERT (S : in out string);
function GET return string;
<<UUT.CONVERT.S>>_L := 0;
To check that GET returns a null string, use the following user code:
{{ <<UUT.GET.return>>_L =0 }}
By coordinating the list of data values with the list of exceptions you can achieve a variety of interesting
results. For instance, if you want the stub to return normal data on the first two calls and then to raise an
exception on the third and subsequent calls, you would set up test case data similar to the following
script example:
TEST.VALUE:DATABASE.GET_VALUE.<<raise>>:NONE,NONE,STORAGE_ERROR
TEST.VALUE:DATABASE.GET_VALUE.DATA:12,17
This test case data will result in the values 12 and 17 being returned on the first two calls to GET_
VALUE and then an exception STORAGE_ERROR being raised on the third and subsequent calls. If
you only want the exception to be raised on the third call and then data to again be returned on the fourth
call, add NONE after STORAGE ERROR in the list of exceptions.
Note: Only the "in" value will appear in the execution results.
Data Types
The parameter types referred to in this user's guide are defined as "Scalar" and “Composite", with
Scalar types being broken down into Integer and Floating Point types. Integer types can be further
subdivided into Enumerated and Numeric types. Composite types can be subdivided into array,
structure (including class), and pointer types.
If you enter an illegal value, VectorCAST outlines the cell in red and the tool-tip provides information.
When entering a range for an enumeration type, the syntax is value1..value2, where value1 and value2
are enumerals defined in the type. VectorCAST auto-completes the first value as you type it. After you
type “..” a red outline appears, until you finish typing the second value. VectorCAST does not auto-
complete the second value in the range.
To change the base, right-click an input or expected value in the parameter tree and select the desired
base from the context menu.
A base may also be specified as part of the value, using the standard C syntax of a “0” prefix for octal or
a “0x” prefix for hexadecimal, or Ada syntax of “8#” prefix for octal, “16#” prefix for hexadecimal, or “2#”
prefix for binary. When using the Ada syntax, any value between 2 through 16 is valid.
For example:
As a record type is a composite data type, you do not enter values for the record directly. You choose a
member of the record by first expanding our the list of members and then setting a value for any scalar
members. If a member is a record, you would expand that member using the button again.
When working with variant structures, you will need to change the value of the discriminant before
working with fields whose availability is controlled by the discriminant.
Character Types
If the parameter or object chosen is of "character" type, you can enter the character in the "Input Value"
or "Expected Value" area.
Note: Enter non-printable characters (i.e. carriage returns, line feed.) by using the hex value for
the character in standard Ada syntax (i.e. 16#0d#)
To Enter a String
To enter a value for parameter of type string, type the string without quotes into the Input Values or
Expected Values column. To include a comma character in the string, use the escape character before
it. For example, the string bob\,tom is a string of one item.
To type a list of strings, use the comma character to separate the items of the list. For example,
entering bob,tom in the Input Values column for a string parameter enters a list of two string items,
causing the test case to iterate with “bob” as the first input and “tom” as the second input.
You can enter the following special characters without escaping them:
~ ! ` @ # $ ^ & * ( ) - _ + = { } [ ] | : ; “ ‘ < > ? / <space>
To enter the character \ (backslash), escape it with another \ first. To enter the comma, escape it with a
backslash first.
To enter the % character, use the syntax \045, as the % character itself is used internally by
VectorCAST as a string delimiter.
Note: When setting values of any constrained string type or sub-type, VectorCAST will blank
pad the entered string value to match the size of the parameter type, or truncate the entered
value to fit the appropriate size.
System.Address Types
System.Address types are most commonly used in Ada to pass the address of some data object to a
procedure. Once inside the procedure the address parameter is cast to an Ada type and then used. The
following code fragment illustrates this point:
If the parameter or object chosen is of "address" type, you can choose from a list of object names that
are defined in the USER_GLOBALS_VCAST package. Setting the data value involves 2 steps. First,
you need to make an association between the parameter and the address of a particular global as
shown here:
Then, you must set the value of INT1 in the unit USER_GLOBALS_VCAST as you would for any other
integer value. By doing these two steps, VectorCAST will pass the “’address” of INT1 to the procedure
and when the procedure does a type cast of the address it will be pointing at integer-formatted data.
Access Types
If the parameter or object chosen is an “access” type, you can set the value of the pointer to be null or to
point to an allocated object. You choose null by selecting NULL from the drop-down menu. You choose
to allocate an object by choosing VALUE. If you choose to allocate an object you will be able to then
Protected type definitions - The stub body for the protected type will allow the user to control the
values returned by the entries defined in the protected type in the normal way that this is allowed for
other stubbed subprograms.
Generic packages - The stubbed bodies are not automatically compiled into the harness because the
compilation of the generic body may put instantiations of that generic out of date. To compile the stubs
into the test harness, turn on the option "Compile generic stubs". The only parameters that can be set or
expected are for types defined external to the generic package.
Task Types - The stub body for the task type will allow the user to control the values returned by the
task entries in the normal way that this is allowed for other stubbed subprograms.
For example, your package contains a protected type called PROTECTED_EXAMPLE with
subprograms WRITE and READ.
end;
You will see the following two entries in the list of subprograms:
(PT) PROTECTED_EXAMPLE.WRITE, and
(PT) PROTECTED_EXAMPLE.READ. Data can be set for the parameters associated with the entry or
subprogram in the same way as for any other subprogram parameter.
Generic Packages
To test a generic package you must manually create a stand-alone generic instantiation compilation unit
for the generic package and then make this instantiation the Unit Under Test.
The following example will illustrate this concept. Given the following generic specification to be tested:
generic
type FIRST_GENERIC_TYPE is private;
type SECOND_GENERIC_TYPE is private;
package GENERIC_PKG is
type TYPE_IN_GENERIC_PKG is new integer;
type DERIVED_FROM_GENERIC is new FIRST_GENERIC_TYPE;
type RECORD_IN_GENERIC_PKG is
record
field : FIRST_GENERIC_TYPE;
end record;
type PTR_IN_GENERIC_PKG is access RECORD_IN_GENERIC_PKG;
type PTR_TO_GENERIC_TYPE is access FIRST_GENERIC_TYPE;
procedure DO_GENERIC_PROCESSING (
PARAM1 : FIRST_GENERIC_TYPE;
PARAM2 : SECOND_GENERIC_TYPE);
end GENERIC_PKG;
package GENERIC_TYPES_PKG is
type DERIVED_FROM is new integer range 20..50;
type RECORD_TYPE is
record
FIELD : float;
end record;
end GENERIC_TYPES_PKG;
with GENERIC_TYPES_PKG;
with GENERIC_PKG;
package GENERIC_INSTANTIATION is new GENERIC_PKG (
GENERIC_TYPES_PKG.DERIVED_FROM,
SECOND_GENERIC_TYPE => GENERIC_TYPES_PKG.RECORD_TYPE);
The parameter profile for the generic package may or may not use named association for the generic
parameters as in the above example. A VectorCAST imposed restriction is that the actual parameters
must be in the same order as the generic formal parameters whether or not they use named notation.
After creating and compiling the GENERIC_INSTANTIATION package we are ready to begin testing.
Simply use VectorCAST to build a test environment with GENERIC_INSTANTIATION as the Unit
Under Test. This will allow testing of the generic subprograms including the manipulation of objects or
parameters defined in the generic package.
There are two options that are relevant to testing out-of-range values. These are the "Compiler options"
and the "Range check" options. To ensure that the VectorCAST-generated test harness components
are compiled with run-time checks disabled you will need to modify the VectorCAST "compile options"
to use the compiler-specific command line option to force checks off. The syntax for this varies from
compiler to compiler. Please consult your compiler documentation for the relevant command. (As an
example Green Hills Ada uses "-no_check" on the command line to suppress run-time checks).
Once the harness is built and compiled with "checks off", you can turn on and turn off the VectorCAST
imposed range checking on data values by using the "Range Check Option" in the Options Dialog box.
To access the Options Dialog use the TOOLS => OPTIONS choice in the main menu. The Range
Check Option is available in the "Execute" tab and has three choices: None, Disable, All. The default
value for this option is All. Each of the choices results in slightly different processing by VectorCAST.
The All choice is the default. When range checking is ON, VectorCAST will range check every scalar
value that is entered and disallow the entry of any out of range values. This range checking applies to
data that is entered interactively through the GUI or via a test case script.
When range checking is in the Disable position, VectorCAST will not enforce any scalar ranges. The
scalar input dialogs will still show the applicable Ada type and its associated range, but you will be
allowed to enter data values that are outside of this range. The only caveat to this is that you will, for
obvious reasons not be able to enter data values that are larger than the physical memory the compiler
allocates to objects of this type (either the default or the size provided by a 'size attribute). If a value is
entered that is larger than the physical memory size, the value will be truncated at run-time and the
smaller, truncated value will show up in the VectorCAST test results. You should consult your compiler
documentation for details on the default memory allocation for various Ada types.
The None position is similar to Disable with the added restriction that VectorCAST will not show the
type range in any of the scalar data entry dialog boxes. This choice will not be used in the normal course
of testing, it should be used only if directed by VectorCAST Technical Support.
The VectorCAST test harness is completely data driven. This means that the test data is de-coupled
from the harness source code. The importance of this is that it enables you to build a suite of test data
that is persistent across multiple versions of the application source code. Because the test data is
persistent and the test harness code can be automatically regenerated for any version of the application
code as necessary, regression testing can be accomplished for each and every "build" of your
application totally automatically!
This discussion will focus on the following example Ada 95 code which uses the concepts of type
extension, and child packages to implement one base class and two sub-classes. The following is
similar to an example in Chapter 13 of Programming in Ada 95 by John Barnes.
Assume you have a package called OBJECTS as follows that declares a type OBJECT with two
primitive operations associated with a single dimension object.
package objects is
type object is tagged
record
xpos : float;
ypos : float;
end record;
function range_in_yards (center : object) return float;
function bearing_in_degrees (center : object) return float;
end objects;
Also assume that you have a child package called OBJECTS.DIMENSION2, that extends the type
OBJECTS for specific two dimensional geometric shapes and also adds the AREA operations for these
extended types.
package objects.dimension2 is
end objects.dimension2;
Finally assume that you have a child package called OBJECTS.DIMENSION2.DIMENSION3, that
extends the types CIRCLE and RECTANGLE into three dimensional geometric shapes and also adds
the VOLUME operation for these extended types.
package objects.dimension2.dimension3 is
type box is new rectangle with
record
height : float;
end record;
end objects.dimension2.dimension3;
When developing a test strategy for this implementation, you will want to test the operations associated
with the base class OBJECT, and then move on to test each of the descendent classes. It is not
necessary to test the function BEARING_IN_DEGREES with objects of the type BOX for instance
because once these primitive operations are tested they are assumed correct for all descendants. This
is a basic foundation of Object Oriented Programming.
test would normally be a package. With the introduction of child packages in Ada 95, the VectorCAST
unit under test will many times be a child package. The reason for this is that the child package is a
basic building block for implementing the OOP concept of CLASS in Ada 95.
The following sections contain a partial listing of the test artifacts that VectorCAST would generate for
each class. For each class we show pseudo code for the driver, along with a single test case and the
test results. Normally you would create several test cases for each operation.
CLASS: OBJECTS
case SUBPROGRAM is
when 1 =>
{result} :=
OBJECTS.RANGE_IN_YARDS (
CENTER => {data value});
when 2 =>
{result} :=
OBJECTS.BEARING_IN_DEGREES (
CENTER => {data value});
Test Case Data
Test Results
-------------------------------------------------------------------
Event 1
-------------------------------------------------------------------
-------------------------------------------------------------------
Event 2
-------------------------------------------------------------------
CLASS: OBJECTS.DIMENSION2
case SUBPROGRAM is
when 1 =>
{result} :=
OBJECTS.DIMENSION2.AREA (
R => {data value});
when 2 =>
{result} :=
OBJECTS.DIMENSION2.AREA (
C => {data value}1);
Test Case Data
Test Results
-------------------------------------------------------------------
Event 1
-------------------------------------------------------------------
-------------------------------------------------------------------
Event 2
-------------------------------------------------------------------
CLASS: OBJECTS.DIMENSION2.DIMENSION3
case SUBPROGRAM is
when 1 =>
{result} :=
OBJECTS.DIMENSION2.DIMENSION3.VOLUME (
B => {data value});
when 2 =>
{result} :=
OBJECTS.DIMENSION2.DIMENSION3.VOLUME (
Test Results
-------------------------------------------------------------------
Event 1
-------------------------------------------------------------------
-------------------------------------------------------------------
Event 2
-------------------------------------------------------------------
capability, allowing the tester to manipulate private types that are defined in the Unit Under Test.
The “Multiunit whitebox” option provides the ability to manipulate private types that are defined in the
specification of any package, not just the unit under test. This enables the user to set up objects using
the standard GUI and test scripting available with VectorCAST, rather than having to write User Code
to set / examine these types. This capability, called Multiunit Whitebox, is not a replacement for the
existing whitebox capability – both methods of building test harnesses are supported.
To access this new capability, the tester must activate this option prior to building the environment via
one of the following methods:
From the GUI: Go to Tools => Options dialog, Builder tab, select the Multiunit Whitebox checkbox
Once an environment is built with the Multiunit Whitebox capability, data entry for objects of private
types (via the GUI or test scripting) is no different than data entry for objects of normally visible types.
Note: The Multiunit Whitebox feature is implemented using Ada 95 child package constructs
and, as such, is available only to compilers / targets that support Ada 95. If this option is set and
the compiler or target does not support Ada 95, or the option to force Ada 83 compliance is set,
the Multiunit Whitebox option will be ignored.
This array has three elements. You can enter an input and expected value for a specific array element,
or set multiple array elements to the same value. The display of array parameters and global objects is
“collapsed” by default, meaning that only a single entry appears in the Parameter Tree. You can
“expand” this display and then enter data for each expanded elements, or you can specify a data value
and apply it to specific elements. These methods are discussed below.
The purpose of expanding or collapsing array data is to enable you to control the size of the Parameter
Tree.
<<Expand Indices: Size n>>, where n is the number of elements in the array.
To expand all elements of the array, right-click on <<Expand Indices>> and choose Expand All Array
Indices.
To collapse unused array elements, right-click on <<Expand Indices>> and choose Collapse Unused
Children.
The array elements that do not have Input or Expected data are removed from the Parameter Tree. In
the example below, all 200 elements of the VECTORCAST_BUFFER had been expanded, but data
was entered for the element VECTORCAST_BUFFER [01]. When the unused array elements were
collapsed, all array elements are closed except for index [01], which has data.
There are several ways to expand (or display) certain elements of this array. In the Input Value or
Expected Value edit box in the Parameter Tree tab, you can type a single index, a range, or a list. In the
example below, the odd array elements from 1 to 199 are specified.
Click OK to exit the Array Indices dialog and expand the array.
type Enumeration_Index_Type is (
EONE,
ETWO,
ETHREE,
EFOUR,
EFIVE );
Another way to expand array elements is to right-click on <<Expand Indices>> to display the pop-up
menu:
> Selecting Properties from this pop-up menu takes you to the Array Indices dialog described
above.
> Selecting Collapse Unused Children causes any elements that have no data to collapse,
minimizing the size of the Parameter Tree.
> Selecting Expand First ArrayIndex causes the first element of the array to expand.
> Selecting Expand All ArrayIndices causes all elements of the array to expand.
For example, in the figure below, the global array BUFFER has been expanded twice. The first array
element was expanded, and then elements 5..10 were expanded. Data can now be entered into these
elements.
In some special cases, VectorCAST is not able to determine the index type for the array. These cases
usually involve constants or expressions in the definition of the array in the package. Therefore, the
array indices are presented in the ‘pos notation. This notation is also used in cases where the Advanced
option “Disable Direct Array Indexing” is on.
In this case the array index values selected would be 1 through 5, as shown below.
Once you have enabled Input or Expected values and filled in the data, you can easily apply these
values to multiple elements of the array. The choices are:
> This (default) to set the one element that you expanded and double-clicked
> All to set all elements of the array
> Range to set a range of elements, (x..y, or 1,3,5,7, or 1..200/2 etc.).
The Apply to expanded array indices only checkbox modifies the “All” or “Range” options by limiting
their effect to the currently expanded elements in the Parameter Tree.
For example, if you expanded elements 2 to 200, even only, using the Expand Indices dialog, you could
then assign a value using the “All” choice in the Parameter dialog. It would only apply to the even
elements.
Then, double-click on any element to bring up the Parameter dialog. Set an input value, then click the
“All” radio button and the checkbox next to “Apply to expanded array elements only.”
Alternatively, you could skip the step where you expand the even elements and instead expand the first
element, double-click it to go directly to the Parameter dialog, and use the Range radio button with the
expression 2..200/2.
Doing so clears the data from all of the indicated range of array elements.
> in the Parameter Tree, double-click on an array’s <<Expand Indices: Size n>>
> in the Parameter Tree, right-click on an array’s <<Expand Indices: Size n>> and choose
Properties...
You can enter an expression to control which array elements are expanded or to expand the entire array,
click the All Values button. The syntax of expressions that can be entered is described in the previous
section.
Click OK to exit the Array Indices dialog and expand the array.
Multi-Dimensional Arrays
The index values for multi-dimensional arrays should be selected via the comma-separated
expressions: [1, 4]
Unconstrained Arrays
VectorCAST supports dynamic modifications to the size of unconstrained array parameters for single
dimension arrays only. This functionality enables you to define, for any subprogram parameter that is a
one-dimension unconstrained array, a slice that you want to consider as active for that test case.
You can construct test cases which pass different sized arrays into COMPUTE. For each test case
you will select lower and upper bounds for the VALUE parameter.
In cases where the array size is unknown or very large, you can set the upper-bound or both the upper-
and lower-bounds. The setting is called “Unconstrained Array Size,” on the Tools => Options dialog,
Builder tab.
Note: To avoid constraint errors, it is important that you define the lower and upper array
constraints for "in" AND "out" parameters. For instance, if you know that a subprogram is going
to return a parameter which is ARRAY_TYPE (2..6) for a particular test case, you need to set the
array constraints for that parameter with VectorCAST before running the test.
In VectorCAST, when an environment is built, an Ada object is defined for each parameter that exists in
the particular closure being tested. In the case of unconstrained array types, that object is always
constrained with the maximum size of the index type that was used to define the type. For example
given the following declarations:
VectorCAST would define an object to map to the parameter for subprogram PROCESS_ARRAY. This
object would be declared as the size of the parameter based on the Ada type. For example, in this case
the following would be generated:
This creates a problem though when the compilation system restricts the maximum size of objects. In
this example the object declaration requires 4 bytes per array item and the array has an index range of -
2**31-1 .. 2**31-1 which is over 4 billion items. This will, with most compilations systems, generate a
compiler warning, or a run time exception. To accommodate this limitation, VectorCAST restricts the
size of unconstrained array objects of this type to 2000 items. The actual declaration that VectorCAST
generates for the above example would be:
This automatic range restriction occurs whenever the index type of the array is one of the following
package STANDARD types: NATURAL, INTEGER, POSITIVE.
The 1000 used in this example is the VectorCAST default. This value can be modified by the user via
the Tools => Options dialog.
You will however still get the compiler warning is cases where there are user-defined types that have
sizes comparable to integer, because VectorCAST is unable to automatically restrict the index range in
these cases.
So, for this example, you would set VCAST_USER_CONSTRAINTS to: INDEX_TYPE.
Note: If the array object index uses full dot notation, you must use full dot notation when setting
the variable (e.g. MY_TYPES.INDEX_TYPE).
Ranges are not permitted in the Input Values list. If you need an input range, you can express it as an
input list, or just go to the Range Values tab and add the input range there.
A value of <<ANY>> does not get compared; it is merely a placeholder, so that other values are
compared on the desired iteration.
For example, double-clicking the parameter Beverage, in the data object Order, in the subprogram
Place_Order, in the UUT manager brings up the Parameter dialog as shown below:
All tabs contain a header which shows the base type of the data object as well as the valid range. Note
that the valid range saves you from having to search through your source code files to find the range of a
particular data item.
In all of the tabs, the small checkboxes next to the data values are used to "enable" the data value for
the test case. Before entering an Input or Expected value, you must click on the checkbox to activate
that value. Similarly, if you "unchecked" a value, it is deactivated and the value is removed from the test
case.
When you enable the Expected checkbox, the Any menu item also becomes available. It is used only
with expected values, and indicates that you want to allow any value for that parameter’s expected
value. Since <<Any>> is a data value, it is recorded in the test execution results. Use <<Any>> if you
want to see the value of a non-global parameter in the test execution results without making an actual
comparison.
The items in the right-click menu depend on the data type of the parameter. Floats have <<ZERO>>,
<<POS_INF>>, <<NEG_INF>>, and <<NAN>>. Strings only have <<ANY>> (for expected values).
Enumerals do not have <<MID>> or <<ZERO>>.
Once you have enabled Input or Expected values and filled in the data, you can easily apply these
values to multiple elements of the array. The choices are:
> This (default) to set the one element that you expanded and double-clicked
> All to set all elements of the array
> Range to set a range of elements, (x..y, or 1,3,5,7, or 1..200/2 etc.).
The Apply to expanded array indices only checkbox modifies the “All” or “Range” options by limiting
their effect to the currently expanded elements in the Parameter Tree.
For example, if you expanded elements 2 to 200, even only, using the Expand Indices dialog, you could
then assign a value using the “All” choice in the Parameter dialog. It would only apply to the even
elements.
Then, double-click on any element to bring up the Parameter dialog. Set an input value, then click the
“All” radio button and the checkbox next to “Apply to expanded array elements only.”
Alternatively, you could skip the step where you expand the even elements and instead expand the first
element, double-click it to go directly to the Parameter dialog, and use the Range radio button with the
expression 2..200/2.
Doing so clears the data from all of the indicated range of array elements.
The From edit box is the first value used, and the To edit box contains the last value used each time the
subprogram is called. These two numbers represent the range, and the Delta is used to get from the
first number to the second. For example, to vary an input from 1 to 6 by 2, use:
From: 1
To: 6
Delta: 2
Input values used for parameter: 1, 3, 5
From: 10
To: 1
Delta: -1
Input values used for parameter: 10, 9, 8, 7, 6, 5, 4, 3, 2, 1.
When you add items to the Input Values list for a parameter, you are assigning a series of input values
to that parameter. Adding more than one item to the list causes an iteration, that is, the test case is
executed once for each item. Repeating a value in the list causes additional iterations.
Using an Input Values list for a stub parameter means that the stubbed subprogram returns one item
from the list each time it is called. If a stub is called more time than you have provided values for in your
list, then the last value in the list is used repeatedly, once the list items are exhausted. Each time a test
case iterates, the Input Values list for a stub is reset to the beginning.
When you add items to the Expected Values list for the same or a different parameter, you are
assigning a series of expected values for that parameter. One expected value is compared to the
parameter’s value per test iteration. The expected value can be a range of acceptable values for that
iteration, as described below.
of the list.
To insert a new entry into an existing list, select an item in the list, enter a new value, and click the
Insert button. The new entry appears below the selected one. To change an entry already in the list,
first select it. Make the change in the text area below (either in the repeat box, the value box, or both),
and click the Replace button. The entry in the list is replaced with the new values.
An entry can be moved up or down in the list by selecting the entry and clicking the Up or Down button.
An entry can be deleted by clicking the Remove button. When you click the Remove All button, all
entries are deleted from the list.
Ranges are not permitted in the Input Values list. If you need an input range, you can express it as an
input list, or just go to the Range Values tab and add the input range there.
A value of <<ANY>> does not get compared; it is merely a placeholder, so that other values are
compared on the desired iteration.
Note: Test Case Scripting is an advanced workflow which can have unintended side effects and
should be used with caution.
VectorCAST Scripting complements the normal test case creation process. With Scripting you can
create a single test case using the VectorCAST GUI, export the script file, edit the file to contain
multiple similar test cases, and import the script to quickly generate tens or hundreds of test cases.
Additionally, test case scripting enables you to import data from other tools, such as MATLAB™, that
may contain high precision models or data sets.
The Export Script command is sensitive to what is currently selected in the Test Case Tree. You can
click (or multi-select) nodes from the Test Case Tree, then choose Test menu => Scripting =>
Export Script. When you click Save, VectorCAST saves the file in the working directory (by default)
with the extension .tst. The file contains test data for all selected test cases.
clicast -e <env> [-u <unit> [-s <sub> [-t <test>]]] TESt Script CReate
<script file>
Create (export) test script file from existing test case(s). If only the
environment parameter is specified, the script will contain data for all test
cases (including <<init>> and <<compound>>). If the unit parameter is
specified, the script will contain data for all testcases for that unit. If
the unit and subprogram parameters are specified, the script will contain
data for all testcases for the specified subprogram. If unit, subprogram, and
testcase parameters are specified, the script will contain data for the
specified test case.
When a test script is imported, whether by you or as part of another operation (such as Updating or
rebuilding an environment), the Test Script Log appears in the MDI window. If any test cases had to be
renamed or other errors occurred, they are documented in the Test Script Log.
The Test Script Log is displayed in the MDI window after a test script is imported, using Test =>
Scripting => Import Script or during another operation, such as rebuilding or updating the
environment. To view the Test Script Log at any other time, choose Test => View => Scripting Log.
If an imported compound test case includes an individual test case that must be renamed, the new
name is propagated to the compound test case, so that the imported test cases remain consistent.
Display to standard output the Test Script Log that was generated by the last
test script import operation.
and output data for a particular environment. The templates contain all global data, formal parameters of
the unit(s) under test, and formal parameters of any stubbed dependent unit. The purpose of the test
script template is to show you the correct syntax for every possible TEST.VALUE line for the
environment. In addition to showing the command syntax, each line shows the range of allowable
values for each parameter.
Create a test script template containing entries for each parameter and
global object in the environment, along with range information, and save it
in <scriptfile>.
TEST.SUBPROGRAM:
TEST.NEW
TEST.NAME:
TEST.NOTES:
TEST.END_NOTES:
TEST.VALUE:MANAGER.PLACE_ORDER.TABLE: <<unknown>>..<<unknown>>
TEST.VALUE:MANAGER.PLACE_ORDER.SEAT: <<unknown>>..<<unknown>>
TEST.VALUE:MANAGER.PLACE_ORDER.ORDER.SOUP: <<unknown>>..<<unknown>>
TEST.VALUE:MANAGER.PLACE_ORDER.ORDER.SALAD: <<unknown>>..<<unknown>>
...
The reason there are <<unknown>> entries for the values is because VectorCAST does not determine
the range for parameters if Range Checking is set to None. Change the setting to All or Disable,
rebuild the range data, and generate the template again.
The lack of TEST.VALUE and TEST.EXPECTED lines, or having only a few of them, means that the
setting for the maximum number of lines in the range file was too low, so the range file could not be
expanded. To overcome this situation, set the value of the option higher, rebuild the range data
(Environment => Rebuild Range Data), and then generate the template again.
To automatically convert test scripts, you must first have an existing environment with test cases.
Next, upload a new version of the source code and select Environment => Rebuild Environment
from the Menu Bar.
As the environment rebuilds, the Message window shows the progress of the automated test script
conversion.
Upon completion of the rebuild, both the Test Script Log and the Test Script Maintenance Difference
Listing are displayed. The Test Script Log can be accessed at any time from the Menu Bar by selecting
Test => Scripting => Import Log.
The Test Script Maintenance Difference Listing shows the changes made in the original script. Note in
our example that the seven translated script lines are listed and that the parameter "ORDER" has been
converted to "MYORDER". The Test Script Maintenance Difference Listing can be accessed at any
time from the Menu Bar by selecting Test => Scripting => Test Script Maintenance Difference
Listing. This GUI-only report is located in the environment directory and is named convertScript-
differences.html.
The original test script file is saved in the environment directory as convertScript-
original.tst. The new converted test script file is also saved in the environment directory as
convertScript-translated.tst.
OLD: MANAGER.ADD_INCLUDED_DESSERT.ORDER.BEVERAGE.enumeration
NEW: MANAGER.ADDNEW_INCLUDED_DESSERT.ORDER.BEVERAGE.enumeration
NEW: MANAGER.PLACE_MYORDER.ORDER.BEVERAGE.enumeration
The Test Script Maintenance Utility recognizes that there are two possible translations,
MANAGER.ADDNEW_INCLUDED_DESSERT.ORDER.BEVERAGE.enumeration or
MANAGER.PLACE_MYORDER.ORDER.BEVERAGE.enumeration, but does not know which one to
use.
When using the Test Script Maintenance Utility in the GUI, a prompt is displayed allowing the user to
resolve the ambiguity and select the translation. Each choice is displayed along with a score indicating
the likelihood of the choice being correct. The list is sorted with the highest score first.
Note: The highest score is auto-selected when using the command line tools for regression
testing.
The user enters the number of the desired choice, or, if no selection is made and the user hits Return,
the highest score is automatically selected. The message window remains open and cycles through
until all the ambiguities are resolved.
To see the full set of status messages after the message window closes, select Test => View =>
Scripting => Messages from last translation from the Menu Bar.
This command imports the test script and converts it to accommodate source
code changes in the environment.
Selecting Import Log opens the Test Script Log. The Test Script Log reports on the processing of the
test cases and any warnings or errors that occurred.
Selecting Messages from last translation opens the Messages From Last Translation report. This
report provides status messages for the Test Script Maintenance Utility.
Selecting Test Script Maintenance Difference Listing opens the Test Script Maintenance Difference
Listing, and shows the changes between the original and translated scripts.
imported by first creating an association between the columns in the CSV file with scalar data item in
the test case parameter tree. After creating this association, the CSV data file can be imported, and
each row in the CSV file will become a separate test case in VectorCAST.
If you do not already have test data in this format, VectorCAST can be used to create the mapping, and
then dump a "template". Once created, the template is filled with data, one row for each set of test case
data using an external tool. The completed CSV file is then imported into VectorCAST to create the test
cases.
The first step in either importing an existing file or in creating a template file is to create a map test case
in VectorCAST. This special test case maps the columns of the file to any VectorCAST subprogram
test case data item. Procedures for both creating a template file and for using a pre-existing file are
discussed below. Note that once created, a map test case is imported and exported the same as any
other test case.
1. Right-click a subprogram in the Test Case tree and select Generate CSV Map.
The CSV Options panel opens and below the CSV Options panel a Parameter tree opens. This
tree looks similar to the Test Case Parameter tree. When creating a template file, the Parameter
tree is used to select test case data items that will become columns in the template file. It is also
used to set the order of the columns within the template.
3. Select the Create Template radio button in the CSV Options panel.
When this option is selected, each input and expected value data item in the Parameter Tree
contains an associated checkbox control. The check boxes are used to select the data items that
will become columns in the template file. Notice that check boxes appear not only for the
parameters of the subprogram, but also for global data items and for SBF data items contained in
the tree. In addition, if a data item is a pointer type, you can optionally use the Parameter Tree to
create an instance of the pointer data item type. You can then select instance members of that
type to be included as columns in the template file.
In the CSV Options panel, select a column separator by selecting either COMMA, TAB or
<custom> from the drop-down menu for the delimiter you wish to use. Select COMMA if your
data file or template uses the comma character to separate the field in a row. Select TAB if your
data file or templates uses the tab character to separate the fields in a row. Select <custom> and
enter any other character that your data file uses to separate fields in a row.
Enter a name for the template file. You may wish to enter an extension for the file name, i.e.,
.csv. By doing so, an external program, Microsoft Excel for example, will recognize the file type
and open it appropriately so that its contents can easily be edited. You can optionally navigate to a
specific directory by using the file browse button.
4. Next, click the arrow button, found in the upper left of the MDI window.
5. Next, select the test case data items to be included in the template file by clicking the check
boxes in the Parameter tree. As each data item is selected, a header entry is added to the
Mapping View. The header information consists of five rows of information for each data item and
will be included in the template file when it is generated. For each column, the template header
information consists of:
> Input or Expected
> Unit name
> Subprogram name
> data item Type (int, float, enum, etc.)
> data item’s Name
6. As input or expected values are selected for inclusion, a number appears in the associated
Parameter tree cell.
7. The same number also appears next to the column header name in the Mapping View. These
numbers indicate the column number that the test data item is mapped to. The column order can
be easily changed using either of the following methods:
> The first method is to change the value in the Test Tree Editor. Click the desired cell in
the editor and change the column number to the desired number and press Enter. The
new column order will be immediately reflected in both the Mapping View and the
Parameter Tree. All columns are renumbered appropriately based on the change.
> The second method is to drag and drop columns within the Mapping View column title
row. Holding the CTRL key, select and drag a column title in the column title row to
the desired column position. The column numbers are automatically updated in both
the Mapping View and the Parameter Tree using this method as well.
8. When you are finished selecting columns and ordering them, save the map by clicking the Save
button on the Toolbar, or selecting File => Save. Note that this saves the mapping created
above, but a template file has not yet been generated.
9. To create the template file from the map, right-click on the (MAP) test case in the test case tree
and choose Create CSV Template.
This generates the template file using the name specified in the CSV Options menu and will place
the file in the directory you specified. If no path was given, the file is placed in the working
directory. The file contains the five rows of header information that were shown in the Mapping
10. Open the generated file using an external tool. The first column of the file represents the test
name, and the following columns represent the test data items that were selected when the map
was created. Below, two test cases have been added to the template file. The first test case is
named PLACE_ORDER_TEST_1, and the second is PLACE_ORDER_TEST_2. Input
parameters for Table and Soup have been entered and an expected return value has also been
provided. When you are done adding data, save the file, and the populated test file is now ready to
be imported.
The first row of the file is a header row containing the names of the input and expected values for each
test. Each of the following rows represents the test data for each test case to be created. The function
findY will be tested using the values in “table.csv”. The source code for findY is as follows:
> Column A, labeled “Y,” is the expected value returned from findY.
> Column B, labeled “Slope,” is the input parameter “m”.
> Column C, labeled “X,” is the input parameter “x”.
> Column D, labeled “Y-Intercept,” is the input parameter “b”).
1. Create a test environment for line.c that is located in the Examples/c/csv_example directory in the
VectorCAST installation directory. Right-click on the findY subprogram in the test tree and select
Generate CSV Map. A (MAP) test case is created for findY and a CSV options panel appears.
2. In the CSV Options panel select the Use Existing data file radio button
3. Select the type of delimiter used to separate values. The example uses a CSV file, so select
Comma-separated values.
4. Select the file "CSV Filename" by browsing for the file "table.csv"
5. The controls below "CSV Filename" in the CSV Options panel provide the facility to load files that
are custom tailored.
> Number of Rows to skip - enter the number of rows of header information at the top of the file.
"Table.csv" has 1 header row, so enter 1. If there were no headings, this value would be set to
zero. For template files created with VectorCAST, this value is 5.
> Row containing column headings - is the row that contains the data item name. This
information is used by the Mapping View as the heading for each column. In the example this is
row 1. If there were no heading rows, this value would be set to zero. When using a template file
created with VectorCAST, this value is 5.
> Rows per testcase - for this example set the value to 1. Typically, each row represents a single
testcase, but if you wish to combine multiple rows into one test case (the test parameters are
combined as a list), set this value to the number of rows to be combined into one test case. If for
example, we selected a value of 2, each parameter would have a list containing two values and
two rows would be used per test case. This feature is only used with very large data files.
> Test name column – in the example, there is no column with test names, so the value is set to 0.
The test names for this example will automatically be generated by VectorCAST. When using a
template file created by VectorCAST, the name column is 1.
> Test notes column – in the example, there is no column containing test notes, so the value is set
to 0. When test notes are available, they will be added to the “Notes” tab of the test. Place the
column number containing the notes in this field. When using a template file created by
VectorCAST, the name column is 0.
> Load all rows - selecting this option will load all of the rows from the file into the Mapping View.
> Load rows from/to – this allows a subset of rows to be loaded into the Mapping View, specifying
a starting and ending row. If you have a very large CSV or tab-delimited file and only want to a see
a small subset in the Mapping View after loading, select this option and specify the range of rows
to load.
6. Click Load to open the Mapping View using the current settings. After clicking Load, the CSV
Mapping View appears. It contains the rows selected for loading and is used to map the file
columns to test data items of the subprogram.
7. To map a column in the Mapping View to a test case data item, drag any value in a column in the
Mapping View and then drop it to a cell in the Parameter Tree.
When you select a cell in the mapping view, all cells with a matching data type in the Parameter
Tree are highlighted in yellow. This helps you select an appropriate cell for mapping. You can drag
a value from any row. The number in the parentheses in the Parameter Tree is the value contained
in the cell selected in the Mapping View that was dragged into the Parameter Tree. The number to
the right of that is the column number in the Mapping View that is mapped to the test data item.
8. When you have completed mapping all of the columns to input or expected values, click the Save
icon in the toolbar or choose File => Save.
9. Mapping is now complete, and the map may be used to create tests.
All tests are created and appear in the Test Case tree, and a test script log is displayed. You may now
select and execute the tests.
clicast -lc -e <env> [-u <unit> [-s <sub> [-t <test>]]] TESt CSv2tst Load
Generate a test case containing a test case for each row in the CSV file
identified by map test case(s) in the environment.
The cells that are highlighted in yellow in the Parameter Tree are available for editing. The value in
parenthesis in the Parameter Tree cell is the column value for the data item in the map. For example in
the picture above, “x” in the Parameter Tree contains (10). This is the same value for “X” in the first row
of the Mapping View. The number in the cell outside of the parenthesis is the column number of the
Mapping View that the data item is mapped to. In this example, “X” is column 3 in the Mapping View,
and cell in the Parameter Tree also contain a 3. This allows you to easily verify the current mapping and
to make changes if necessary.
To remove a data item from the map, simply click in the Parameter Tree cell and delete the column
number entry and press Enter.
To map a column that is currently not mapped to the Parameter Tree or to change a currently mapped
column, click a value in the column and drag and drop it into the desired cell of the Parameter Tree. You
may also select a Parameter Tree cell and type a column number directly and pressing Enter. You will
see the value and its column position immediately updated in the Parameter Tree upon releasing the
mouse button or pressing Enter.
When you are finished editing the map file, click the Save button on the Toolbar or select File =>
Save.
A dialog box appears indicating the number of tests generated from the associated CSV or tab-delimited
file. Click OK.
A test script, containing all of the generated tests appears in a test script editor.
Review the tests and make any changes you wish to the script file. The test script can be imported by
selecting Test => Scripting => Import Script
clicast -lc -e <env> [-u <unit> [s- <sub> [-t <test>]]] TESt CSv2tst CReate
<test script>
Generate a test script containing a test case for each row in the CSV file
identified by map test case(s) in the environment. Output file <test script>
must be specified.
VectorCAST Tools
To Set VectorCAST Options
The Tools => Options dialog is used to access the many options and settings available to configure
VectorCAST. To access the dialog, choose Tools => Options... or click the Options button on
the toolbar. The options are grouped by category on the following tabs:
> GUI
> Ada
> Wizard
> Builder
> Execute
> Report
> Target
> Coverage
When you click the OK or Apply button, VectorCAST writes the options to the ADACAST_.CFG file,
located in the current working directory. This directory is noted in the status bar of the main window.
Any changes made to the configuration file in the working directory affect other environments in the
directory.
If you run VectorCAST in command line mode (clicast), the same file is accessed for Options settings.
When the VectorCAST application is launched, it looks for a configuration file in the following locations:
Options are read in the order specified with each file taking precedence over previous files.
If you want some options to apply to all environments (e.g., the compiler you are using), you can set
these options in the VectorCAST installation directory.
If you have VectorCAST installed on a network and each user has a client installation, you can create a
configuration file that applies to all users of VectorCAST. To do this, set the working directory to
network drive, and set any options you want all users to have, such as compiler settings. This
configuration file is used unless settings from an environment-specific configuration file overrides them.
If you add a line or make a change to your configuration file external to VectorCAST (while it is running),
you can read those values back into VectorCAST. To do this, click the Load button. The changes that
you made to the file are now reflected in the Options dialog box.
The default values for the current active tab can be reset by clicking the Default button.
Clicking Cancel closes the dialog box, without saving any changes since the last time you clicked the
Apply button.
See "To Set the Working Directory" on page 90 for information on changing the current working
directory.
Tools Menu
The following sections cover the Tools Menu selections. The selections on the menu that are discussed
are:
The pop-up window contains the UUT and any dependent units, along with all the harness source files.
Selecting one of them opens up the source code file. This feature provides a convenient way to browse
the source code as tests are being developed. The figure below shows the pop-up window for an
environment having MANAGER as the UUT and having DATABASE as a dependent unit.
To aid in cross-referencing the source filenames with the VectorCAST unit numbers, the VectorCAST
unit numbers are displayed in the Harness Files window.
To view one of the files, select the name, right-click, and click Open Source File. Clicking on the
header bars at the top sorts the files in reverse order by Unit #, alphabetically by description. Clicking
again reverses the order. To dismiss the window, click on the button located in the upper right corner.
Once you have chosen a harness source file to view, it appears in the MDI window. You can view the
source file or even edit it here. If you change the source code file, and want the changes incorporated
into the test harness, choose Environment => Recompile. Keep in mind that any changes you make
to the test harness source files are not reflected in your original source code files, as these files are
separate and distinct from the test harness.
See also "To View the Source for a Subprogram" on page 152.
Basis Path
This menu item has four options:
The number of paths identified equals the code complexity, also referred to as V(g). The code
complexity corresponds to the number of test cases that must be developed to exercise the unique
paths in a subprogram.
See also "To Insert Basis Path Test Cases" on page 155 for information on having VectorCAST
automatically generate a test case for each basis path.
MC/DC
This menu has one option: View Equivalence Matrices.
> Bug Tracking - Allows users to create bugs via an integrated bug tracking tool.
> Command Prompt - On Windows, a DOS command prompt is available. On Linux, an XTerm
window is available.
> Generate CSV Execution Report - Dumps the Execution Report for the currently selected test
case in CSV format, and on Windows, opens the resultant file in Microsoft Excel. By default, this
custom tool is not shown in the Toolbar or the Custom Tools menu.
> Settings Database - Provides a visual editor for use with a vcshell database file.
> RSP Configurator - Allows users to select the compiler family and modify and save associated
arguments.
> RSP Installer - Allows users to download and install customer-specific VectorCAST/RSPs.
Default custom tools are displayed with a gray background color, and are not editable by the user.
To add a tool, enter the label you want to appear in the Custom Tools menu into the Name field. The
name is also used as the tool-tip for the icon.
Optionally, enter or browse for the path to the icon image file in the Icon field.
Enter or browse for the path to the relevant application file. The path to the executable file, shell script,
or batch file is entered into the Target field.
Enter any arguments needed for the target in the Arguments field.
The new custom tool is added to the list of Custom Tools. The background for the tool is temporarily set
to green, to aid the user in identifying and tracking newly added items. Clicking the Apply button adds
the tool to the list and the background turns to white.
Use the associated check boxes to change the display of the link. By default, the tool's link is displayed
in the both the Custom Toolbar and the Custom Tools menu.
To test the tool to make sure it works, press the Test button. The tool opens in a separate window.
Highlight the tool in the Display Name column. The settings for the tool are displayed in the Attributes
pane.
To update the settings for a tool, in the Attributes pane, make changes in the Name, Icon, Target, or
Arguments fields. Click the Update button. Click OK to close the dialog.
Select the tool to delete and then click the Delete button. Click OK to close the dialog.
To export an existing custom tool, highlight the tool and select the Export button. Note that default
custom tools cannot be exported. Browse to the location where you will store the .ini settings file and
enter the file name. In our example, the file is named ToolExport.ini.
To import a custom tool, open the Custom Tools dialog and select the Import button. Browse to the
location of the .ini settings file and select Open. The tool is added to the Custom Tools list, and the
background for the tool is temporarily set to yellow, to aid the user in identifying and tracking newly
imported items.
If VectorCAST detects a duplicate name attribute in the list, the background for the tool is set to red to
alert the user of the conflict.
Troubleshooting VectorCAST
Note: This feature should be used only as directed by VectorCAST's Technical Support to
resolve issues in the field.
To troubleshoot VectorCAST and create a debug log, choose Help => Diagnostics =>
Troubleshooting. A dialog box appears, allowing you to check which items to want to see in the log.
Click the Browse button to specify a location for the file. When you click Enable Debug, a debug log is
written while you run VectorCAST.
When debug logging is enabled, the default log output location defaults to a file with the name prefix
To view the log later, choose Help => Diagnostics => Troubleshooting. Click the View Log File
button. In the Open File dialog, navigate to the debug.log.* file, and click Open. The debug log is
displayed in the MDI window.
> Right-click on a test case and select Execute from the context menu.
> Select a test case to run, then click this toolbar icon:
> Select a test case to run, then press the F5 key on the keyboard.
> Choose Test => Batch Execute All
You can multi-select any combination of test cases, subprograms, and units, before choosing Execute.
When a test case is executing, the Message window gives you feedback on the progress. It informs
you as it processes each event and when test execution is complete.
The Execution Status viewer automatically opens when a test case is executed. The Execution Status
viewer provides detailed real time test case execution information. See the "Execution Status Viewer"
on page 50 for more information.
If a test case contains expected values, those values are compared to the actual values when a test is
executed and the Test Case Tree is updated with a PASS or FAIL indication.
The time of test execution appears in the Date column. If you open an environment with a test case that
you executed a day or more ago, then the date, rather than the time, is displayed in the Date column.
Clicking this checkbox opens the Coverage Viewer and displays the unit's coverage achieved by the
selected test cases.
> Execute a test case (the Execution Report tab opens automatically)
> Open a test case editor and click the Execution Report tab, or
> Select a test in the Test Case Tree and choose Test => View => Execution Results.
The Execution Report tab is automatically opened after a single test case is executed. If you execute
multiple test cases, the Test Case Management report is displayed.
If you turn on the “Use external browser” option, located on the GUI tab of the Tools => Options dialog,
and specify an external browser (IE, FireFox, etc.), then all reports, including the Execution Results,
are displayed in a separate browser window. If you would like the Execution Results to be displayed in
a report tab, you can turn on the “Use external browser” option but leave the browser unspecified.
The maximum number of events is set at 1000 by default, but can be changed for an individual test case
or for all test cases.
Expected Values
If a parameter has an input value or expected value, it is included in the Execution Report. If a
parameter’s value at the time of the call matches the Expected Value or falls within an expected range,
then the Expected Values column is colored green for that parameter. If it doesn’t match or fall within an
expected range, then the column is colored red. If all parameters with Expected Values matched, then
the test case status is set to PASS. If any expected values do not match, the test case status is set to
FAIL.
A section called “User Code Expected Values” is included at the end of the Execution report when User
Code contains an expected value. The expected value for the item is compared to the actual value and
if they match, a line is added to the section and colored green. The item is displayed in the left column of
the report, and the status, <<match>> or [fail] is displayed in the right column. If the item being
compared is lengthy, then the display is truncated to the page width. In HTML, this width is 80
characters. In TEXT format, this width is configurable, using the “Page width” option on the Reports tab,
Format tab, Text sub-tab.
To cause a parameter to be included in the report without having to set an Input Value or a specific
Expected Value, enter <<ANY>> as the Expected Value. Doing so causes the parameter to match any
value, and also to be included in the report.
If a test case has no Expected Values, then there are no values to match, and so the status is neither
PASS nor FAIL.
In the Parameter Tree, the expected values are also colored to indicate PASS or FAIL. If an expected
values is provided and it matches at some events in the Execution Report but fails in others, then it is
colored yellow in the Test Case Editor.
For example, suppose a parameter has a range of input values 1..2, which causes two iterations of the
test case. If the expected value for this parameter is set to the range 1..2, then both Input Values
match.
If we change the Expected Value for this parameter to the range 2..3, then the first Input Value (1) does
not match, but the second (2) does. This situation results in the expected value cell being colored
yellow. (if a test case partially passes and it is part of a compound, then the slot is considered failed.
When there are multiple iterations of a slot and some pass in a compound then that slot partially
passes.)
For compound test cases, the unused expected results are grouped by slot.
By default, all parameters, both input and expected, that have values specified in the Parameter Tree,
are included on the Test Execution Report.
Optionally, Input Values populated within the Parameter Tree can be hidden on the Test Execution
report. To hide the Input Value, right-click the icon next to the parameter on the Parameter Tree
and select Hide from Report on the context menu. This option is not available if the Expected Values
column contains data.
The icon is then displayed indicating that the Input Value is now hidden on the report.
To re-enable the Input Value on the report, right-click the icon and select Show in Report. Selecting
the Clear Attribute allows the value to observe the attribute state for its parent.
For Expected Results with a specified value, the icon appears on the Parameter Tree to indicate that
the value will be printed on the Test Execution report, but it is not user controllable and is always
included in the report.
For both Input Values and Expected Results parameters that have no values specified, right clicking in
the report control column will display the following context menu.
Select Show in Report and the force print icon appears indicating that the parameter is currently
included in the report.
To remove the parameter from the report, right-click the force print icon. Choose Hide from Report, to
suppress printing, and the icon is displayed. Selecting Clear Attribute at any time will suppress
printing and remove the icon from the Parameter Tree.
In addition, the attribute items are available on composite parameters as well. When the user selects a
composite object (e.g. "struct" object) and sets the attribute, all members (recursively) will receive the
selected attribute unless directly overridden (with the added limitation that a parameter with an
expected value will not inherit the "Hide" attribute).
> If you set an input or an expected value for a parameter, VectorCAST will capture the output. This
includes the use of the expected value <<ANY>>.
> If you set an input value, but you then use the printer option and select Hide in Report,
VectorCAST will not capture the output. (Note that VectorCAST needs to capture the output in
order to perform the test for the expected value. This scenario sets up a contradiction by hiding the
variable and preventing capture.)
> If you do not set an input or an expected value, then you can use the printer option Show in
Report and VectorCAST will capture the output.
clicast execute extract coverage was deprecated in VectorCAST version 4.0. Use
clicast reports custom coverage.
clicast execute extract results was deprecated in VectorCAST version 4.0. Use clicast
reports custom actual.
clicast reports report was deprecated in VectorCAST version 4.0. Use clicast reports
custom full.
together. An <<INIT>> test case in the first slot can be used to initialize variables or instantiate class
pointers that subsequent test cases access.
By default, each slot in a compound is executed once, unless the number of iterations is specified. If
the cumulative number of events exceeds the environment-wide Event Limit (specified on the Tools =>
Options dialog, Execute tab), compound test case execution terminates, with a message in the
Execution Report. For test cases that have their own Event Limit specified on their Options tab, then
that Event Limit is used when comparing the cumulative number of events. Following that, the
environment-wide Event Limit is used.
The Execution Report for a compound test case is very similar to that of a simple test case. The only
difference is in the Event header for an Event that marks the start of a new slot. The first slot is
executed starting with Event 1, and the second slot’s start is indicated as “Event n, Slot 2, Iteration 1.”
If you have specified that a slot iterate, then the Execution Report reflects the slot being called multiple
times.
See " To Specify the Number of Iterations of a Test in a Compound" on page 159 for information on
specifying slot iterations.
For example, suppose we have a compound test case with three slots: a test case that passes, a test
case that fails, and a test case that has no expected values, so it neither passes nor fails. When this
compound is executed, it is given the status of FAIL as a whole, due to slot 2 failing.
The slots in the Compound Test Case are colored to reflect the pass or fail status of each individual
slot. A slot that passes is colored green; a slot that fails is colored red, and a slot that has no expected
values is left uncolored.
If a slot has more than one iteration specified, then all iterations of that test case must pass in order for
that slot to be considered PASSed. In the example below, a test case was added to the compound. One
of the parameters in the test case has an Expected List of 2 values, but only 1 input value. The first
input will pass, but the second expected value won’t match. In the compound, the number of iterations
is set to 2. When the compound is executed, this 4th slot is colored yellow, because the first iteration
passed, but the second iteration failed, as shown in the tool-tip.
Alternatively, you can right-click the top node of the Test Case Tree (the environment name) and
choose Execute, which also executes all test cases. If you click a node other than the top node of the
Test Case Tree, then only the test cases under that node are executed.
If a test case is marked “Compound Only” it is only executed when its compound test is executed.
Execute multiple test cases. To run all test cases for a subprogram, specify
the subprogram name on the command. To run all tests for an environment, do
not specify the subprogram name.
test cases, the following dialog appears. Click the Abort button to halt the test case execution and
return control to VectorCAST. This is useful for test cases that are running indefinitely, not responding,
or when target communication is suspect.
To expand all nodes of the tree, right-click anywhere in the Subprograms panel and select Expand All
from the context menu. To collapse the tree, select Collapse All from the context menu.
If the Control Flow panel is empty, a control flow was not created or saved for the test case. See “To
Save the Control Flow” on page 257.
You can manually add a subprogram to the control flow by double-clicking any subprogram listed in the
Subprogram panel. The item will be added to the bottom of the list in the Control Flow panel
You can edit the control flow by right-clicking on any item in the Control Flow panel. Using the context
menu you can:
> Insert Selected – this menu item will duplicate the selected subprogram and place a copy directly
below the selected subprogram in the Control Flow panel.
> Move-Up – the selected subprogram will move up one position in the control flow. This selection
choice is inactive if the subprogram selected is at the top of the list.
> Move-Down - the selected subprogram will move down one position in the control flow. This
selection choice is inactive if the subprogram selected is at the bottom of the list.
> Move-Selected – this menu item allows you to select and move one or more subprograms to a
different position in the Control Flow. To activate this feature, first, select the subprograms you
wish to move by CTRL-clicking each subprogram. Next, select the “move to” position by
CTRL+right-clicking the new position. When the context menu appears, select Move Selected.
The subprograms will be moved just above the CTRL+right-click position.
> Remove Selected– delete the selected subprogram or subprograms from the control flow.
> Clear All – delete all subprograms in the control flow. See also, “To Clear the Control Flow”.
Stubbed subprograms listed in the Control Flow panel are identified with the prefix identifier “uut_
prototype_stubs.” for easy recognition.
By double-clicking a subprogram in the control flow list, you can edit the name of the subprogram. If you
accidentally enter a subprogram name that is either misspelled or does not exist, the entry will have a
light red background to alert you to the error.
Control flow can change as a result of new source code being introduced into the test environment or as
a result of manually editing the control flow.
If a control flow has changed, the Test Execution Report identifies the event where the control flow
mismatch occurred as well as providing a list missing control flow events.
comparison against future runs. To use this command you first create and execute a test case. If, after
viewing the results, you wish to save the order of the calls as an expected result, select Test =>
Control Flow => Save. Future runs of the same test case result in a comparison of the call sequence
to the saved call sequence. Using this command results in an additional comparison of expected and
actual results for pass/fail metrics.
A warning dialog appears indicating the number of test cases that will be changed by the action. You
have the opportunity to abort the action by selecting the No button. Selecting the Yes button will allow
the operation to proceed and modify the test cases. Note that this action cannot be undone.
All Expected Values listed in the Execution Report are set to their actual values.
See Controlling Report Information for more information on capturing output for variables.
clicast -l<lang> -e <env> [-u <unit> [-s <sub> [-t <testcase>]]] TESt
Actuals_to_expected
This will set the actual values for a test case as the expected values. Thus
100% of the expected results will match for that test case.
VectorCAST provides the ability to control what coverage data appears in reports containing the
Aggregate Coverage Report or Metrics Report. To set the coverage data used, select Test => View =>
Coverage data used in reports from the Menu Bar. Then, select one of the sub-menu options:
> All results (default) - Use coverage data from all test results in the environment, even if
unchecked (unselected) when generating reports.
> Only checked results - Use coverage data only from the checked (selected) test results when
generating reports.
The setting is saved in the user's HOME directory, and thus applies to all unit test environments.
Note: This option does not apply to reports generated via clicast, which uses all results.
If you switch your Report Format from the default HTML to TEXT, then you can view the test reports in
an external text editor. If you specified an external editor (notepad or emacs, for example) and indicated
that it should be used as the File Viewer in the Tools => Options dialog, GUI tab, then the test
reports are displayed in that external editor.
Create a Test Case Data report and output to HTML file <env>_test_case_data_
report.html, standard output as text, or to the specified output file. If the
Unit parameter is specified, the report includes the Test Case Data for all
test cases for the specified unit. If the Unit and Subprogram parameters are
specified, then the report includes the Test Case Data for only those test
cases for that subprogram of the specified unit. If the Unit, Subprogram, and
Test Case parameters are specified, then the report includes the Test Case
Data for only the specified test case. Otherwise, the report includes the
execution results for all test cases in the environment, including
<<COMPOUND>> and <<INIT>>.
The following report is the result of the Test => View => Test Case Data command for a simple test
case.
menu item is dimmed, this means the test case has not been executed.
The contents of this report reflects the items selected in the Test Case Tree. If you click on a
subprogram name or a unit name, then the Execution Report contains results for all test cases for that
subprogram or unit. If you click on the name of the environment, then the report contains results for all
compound test cases, simple test cases, and compound-only test cases.
Similarly, if you click on <<COMPOUND>> or <<INIT>>, then the report contains results for all
Compound test cases or Init test cases, respectively.
See also "To Execute a Test Case and View Results" on page 246 for more information on
Execution Reports.
4. Exit VectorCAST.
5. From a shell window in the working directory, execute clicast -e <env> reports
alternate <outputfile>
The default separating character is a comma (,), but this can be changed using the “Delimiter” option on
the Tools => Options dialog, Report tab, Format sub-tab, Text sub-sub-tab. To enter a different
character as the delimiter, type it in the edit box. To enter a tab, use “\t”.
The contents of this report reflects the items selected in the Test Case Tree. Much of the full report is
the same for all test cases. The test case data and execution results are specific to the test case(s)
included in the report.
See also "To Execute a Test Case and View Results" on page 246.
Create a Test Case Management report and output to HTML file <env>_
management_report.html, standard output as text, or to the specified output
file. If the Unit parameter is specified, the report includes the Overall
Execution and Coverage summary for the specified unit, the Pass/Fail status
for all test cases in that unit, and the Metrics report for only the
specified unit. Otherwise, the report includes Overall Execution and Coverage
summary for the environment, the Pass/Fail status for all test cases,
including <<INIT>> and <<COMPOUND>>, and the Metrics report for all units.
To override tracking of selected items in the Test Case Tree, open the drop-down menu for the Data
Summary Report and select Options => Track Current Selection. Remove the check next to the
option. The tracking icon on the Summary table will change to gray to indicate that the summary is
now tracking all items in the Test Case Tree.
The Test Data Summary table is dynamic. When the Track Current Selection option is enabled, as
nodes are selected and deselected in the Test Case Tree, the Test Data Summary table updates in real
time reflecting the selections.
The data displayed in the Test Data Summary includes the Unit name, Subprogram name, Test Case
name, most recent Execution Date/Time, Status and Events for each tracked selection. Additional
columns are added to the Test Data Summary when that data is available. These additional columns
are: Expected Values, Matched Values, Control Flow, Signals, Exceptions and Abnormal Termination.
Totals are provided for the data being tracked. The Totals row at the top of the table displays the totals
for each data column. In the example above, note that we have a total of 9 test cases executed, and of
the 8 total expected values, 7 were matched.
The Summary table updates whenever test data is updated. For example, the table refreshes when test
cases are inserted, deleted, or duplicated, or following test execution.
Double-clicking on a line in the Test Data Summary opens the corresponding test case in the Test Case
editor.
Test data can be accessed all the way down to the individual test case level by filtering. Access the
filter by typing into the top row of any column. Clear the filter by right-clicking in the top row and
selecting Clear Filter from the context menu. In the example below, the table has been filtered to only
show data for test cases with more than 3 events.
Filtering supports the following symbols: <, >, =. Examples of filtering inputs are:
10 - lists test cases matching the specific value of "10" in the selected column
>50 - lists test cases greater than the value of "50" in the selected column
<90 - lists tests cases less than the value of "90" in the selected column
=100 - lists test cases matching the specific value of "100" in the selected column
< - lists test cases with empty values in the selected column
> - lists test cases with non-empty values in the selected column
= - lists test cases with non-empty values in the selected column
Printing the contents of the Test Data Summary is supported. See "To Print an Open Window" on page
To open the Test Comparison Report, select two or more test cases under a single subprogram node,
and right-click. Select Compare Test Values from the right-click context menu.
Note: Test cases from different subprograms cannot be compared. You must select at least two
test cases from the same subprogram or <<INIT>> node (including specialized test cases and
test cases in a TDD environment) to generate the Test Comparison Report.
Selection of (MAP)CSV test cases and <<MIN>>, <<MID>>, and <<MAX>> test cases is not
supported.
Parameter Section
> Row - Row number, starting with 2 (the filter occupies row 1 of the table)
> File - Unit name or stubbed unit name. For a <<GLOBAL>> parameter, USER_GLOBALS_
VCAST is displayed. For <<INIT>> test cases, the unit is displayed as <<INIT>>. Note that in
the header for the File column the total number of parameters in the table is displayed in
parentheses. For example, File (15).
> Subprogram - For constructors and destructors, displayed as <<global>>. For <<SBF>>
constructors and subprograms, displayed as class::subprogram. For example,
Manager::PlaceOrder.
> Parameter - For constructors, the long name is displayed. For an <<SBF>> constructor or
subprogram, the parameter name is "stub". For stub-by-implementation
(ENVIRO>STUB:database) , the parameter name is "stub".
> In/Out - Values are "input" or "expected".
In the Test Case section, one test is designated as the Baseline Test, which the other tests are
compared to. By default, the Baseline test is the first test created and is displayed in the first column
following the In/Out column in the Parameter section. A black dot in the column header and green cell
background indicate the location of the Baseline test in the report.
To change which test case is the Baseline test, right-click in any cell under a different test and select
Display Differences from the context menu, or press Alt+D.
The Display Differences setting can be toggled on and off by using the right-click context menu or
pressing the Esc key. When toggled off all cells have a white background and the table is not doing a
comparison, but is just listing the values.
When a test case is compared to the Baseline test, if there are differences in values, the cell
background is blue. If there are no differences, the cell background remains white.
Parameter User Code is displayed as <user>. If both tests have any text in Parameter User Code, they
are treated as a match. Note that Test Case User code is skipped.
The Column Controller is used to show/hide the columns contained in the Test Comparison Report. To
open the Column Controller, right-click on any cell in the Report and select Column Controller from
the context menu.
The Column Controller panel opens to the right of the Report. Place a checkmark in the boxes of each
column to be shown in the Report. Unselected boxes hide the column from view. Hiding columns does
not impact the data contained in the Report.
You may also highlight the columns that you want to select or deselect, and right-click to use the
context menu to Check selected or Uncheck selected.
Close the Column Controller panel by right-clicking in any cell in the Report and selecting Column
Controller from the context menu to toggle the Controller off.
> Right-click on a cell in a test case column and select Open Test Case from the context menu.
(This will leave keyboard focus in the cell in the Test Case Editor.)
> Highlight a cell in a test case column and select Enter. (This will leave keyboard focus in the cell
in the Test Case Editor.)
> Highlight a cell in a test case column and select Alt+Enter. (This will leave keyboard focus in the
cell in the Report, but using the up-arrow and down-arrow keys will move the selection to the
associated parameter value in the Test Case Editor.)
> Double-click on a cell in a test case column. (This will leave keyboard focus in the cell in the Test
Case Editor.)
The Test Case Editor opens in a horizontal pane under the Report. The cell used to perform the Open
action is highlighted in blue in the Editor, making it easy to locate the value to be edited.
To reload the comparison data, click the Reload button. Alternatively, right-click on any cell in the
Report and select Reload from the context menu. The values for the test case update. Note that
reloading may result in a change in comparison, and cell colors (white = matching, blue = difference)
may change.
Renaming a test case does not trigger a reload and renamed tests are left out of the reload. Reloading
recreates the Report using the original arguments (test case names) and does not recognize the
The Execution options tab is used to change the way VectorCAST executes test cases, on an
environment-wide scale. To change execution options for an individual test case, use Test Case
Options. Pass your cursor over any of the options to see an explanation of that option in a tool-tip.
Range Check
The Range Check option provides the following choices:
> All (default) – determines valid ranges of data types, and enforces input and expected values
within valid range
> Disable – determines ranges of data types, but does not enforce
> None – does not determine ranges of data types
VectorCAST provides for full range checking of test data during test cases building. This is reflected in
the default range check setting of "All".
However, when building test cases it is often desirable to be able to specify out-of-range test values for
parameters and global data. The Range Check option provides this capability with the “Disable” option.
When range checking is disabled, the dialog boxes will still show you the data range that is allowed, but
will not enforce that range. This will allow you to enter values that are beyond the data range for the
particular parameter or object (including enumerated values).
In order to determine data type ranges, VectorCAST must execute a small program. You can disable
this processing by setting the Range Check option to None. The “None” position is similar to
“Disable” with the added restriction that VectorCAST will not attempt to determine scalar ranges. The
“None” option is normally used in target testing where execution speed is an issue.
Indicates whether range checking should be performed when accepting input and
expected values. NONE indicates harness should not perform processing used to
verify ranges (useful on slow targets). The default value is Full.
Event Limit
The Event Limit option enables you to control the maximum number of events per test harness
execution. The event limit defaults to 1000. If this option is set to 0, no results data is captured,
although coverage data is captured. If the event limit is exceeded, VectorCAST automatically stops
test execution if your compiler supports signals, and the execution report shows:
The event limit applies to each test case. Each time a test executes, it resets the event counter to 1. If
you execute several test cases at once using Ctrl+Shift click to select them, or execute them all by
selecting the environment name and choosing Test => Execute, the counter starts over with each test
case (because the test harness is being executed multiple times).
Note: An individual test case can have its own event limit, which overrides the environment
event limit when that test case is executed. This option is accessed via the Options tab in the
Test Case Editor.
When executing a compound test case, the event limit is not reset for each slot; the Event Limit setting
applies to the Compound Test Case as a whole.
Note: The Event Limit feature is implemented through the raising of signals. If your target does
not support signals, the event limit capability will not be available.
Maximum number of events for the harness to process. If this option is set to
0, no results data is captured, although coverage data is captured. Its
default value is 1000.
Percentage that a floating point actual value can vary from its expected
value and still be considered a match. <percentage> is a floating point
number. Its default value is 0.000000.
TEST.FLOATING_POINT_TOLERANCE:<tolerance>
Note: For very small tolerances, the floating point representation of the value may not be exact
due to the resolution of floating point numbers; in this case, the floating point tolerance may
show more digits than the user expects. For example, a tolerance of 0.000000001 may actually
be displayed as 0.00000000099999.
The status of the test case will be FAIL until you edit and save the test case in the Test Case Editor. If
you export the test case to a test script before correcting the problem, “TEST.IMPORT_FAILED” is
written to the test script for that test case.
Any errors encountered during test script import cause the test case's status
to be set to Fail. Its default value is FALSE.
If this option is set, then test cases that have no expected results or
expected user code will be marked as failed. Its default value is FALSE.
Sets test case status to Fail if an exception is raised during test case
execution but no exception was expected. Its default value is TRUE.
code for the user globals unit is automatically cleared as well. This option is helpful when you need to
keep the executable size small.
When set to TRUE, user code is cleared prior to executing tests. The default
value is False.
Disable this option if you have added parameter user code to a stub that you expect to never get called.
(You might add code to an uncalled stub if you want the test to show a failure if the stub ever gets
called.)
Set this option to True to detect when any test-specific user code that
checks expected values with the "{{...}}" syntax has not been executed. In
that case, the test will be flagged as having unused expected results. Such
code is most likely to be associated with a stub that has not been called.
Any test case or parameter user code, either Input or Expected, that uses the
conditional syntax "{{...}}" is considered when the option is True. Set this
option to False if you have added parameter user code to a stub that you
expect to never get called. (You might add code to an uncalled stub if you
want the test to show a failure if the stub ever gets called. The default
value is False.
If the option “Redirect Standard Output” is on as well, then standard output continues to be directed to
the execution reports, and standard input is ignored. Therefore, turn off “Redirect Standard Output”
when using this option. Under Linux, this option is dimmed, as harness standard input and standard
output are by default available in the console window from which VectorCAST was started.
Combination Testing
If a test case has a parameter with an input range or list, then the test harness executes the test case
once for each iteration. When two or more parameters have a range iteration, then each parameter’s
iteration count is incremented at the same time, by default.
Combination testing can be turned on for an individual test case, using Test Case Options, or for the
whole environment, using the Tools => Options dialog, Execute tab. If combination testing is
enabled, VectorCAST will determine the combinations of input values and use those values as stimulus
values.
For example, if parameter A has a range 1 to 3 with a delta of 1 and parameter B has a range 2 to 5 with
a delta of 1, and Combination Testing is off, then the test case executes this way:
If the combination testing option is enabled, then the test case executes this way:
A is 1
B is 2
Range Iteration #2:
A is 1
B is 3
Range Iteration #3:
A is 1
B is 4
Range Iteration #4:
A is 1
B is 5
Range Iteration #5:
A is 2
B is 2
Range Iteration #6:
A is 2
B is 3
Range Iteration #7:
A is 2
B is 4
Range Iteration #8:
A is 2
B is 5
Range Iteration #9:
A is 3
B is 2
Range Iteration #10:
A is 3
B is 3
Range Iteration #11:
A is 3
B is 4
Range Iteration #12:
A is 3
B is 5
This feature is not available with an environment built with VectorCAST version 3.2 or earlier.
Rebuilding the environment will enable this feature’s functionality.
Use all combinations of the values in the range expressions for test stimulus
values. The default value is False.
If this option is set, then then testcases that have no expected return value
or expected user code for the return value will be marked as failed.
Sets test case status to Fail if a signal is raised during test case
execution. Its default value is TRUE.
Limits script logs to show errors only. The default value is False.
This option tells VectorCAST to discard those partial or template tests when importing the generated
test script.
When building Basis Path or MCDC test cases, there are three outcomes for
each test: Complete, Partial, and Template. When this option is on,
VectorCAST will discard the Partial and Template tests, and only load the
Complete tests. The default value is False.
The Content sub-tab is used to change the way VectorCAST displays test execution reports and
coverage reports on an environment-wide scale. Pass your cursor over any of the options to see an
explanation of that option in a tool-tip.
By default, the Metrics report lists all the units in the environment alphabetically. Setting this option
causes the report to have one section per directory, sorted alphabetically. For each section, the full
search directory path and the units from that directory are listed.
Set to true if you want the Metrics report to be sorted by search directory
rather than by unit alphabetically (default). The default value is False.
By default, the Aggregate Coverage Report includes all units in the environment and if they have no
coverage data, “No coverage data exists” is written for that unit. Turning on this option causes the
annotated source code for all units to be included in the Aggregate Coverage Report, even if a unit has
no coverage data.
Extra column that reports if a function was entered. This option is only used
if the coverage type is not Function. The default value is False.
This option enables you to reduce the detail that is displayed in the Metrics report. By default the option
is set to "Include subprograms and units," (No_Filtering) which generates the familiar Metrics report
with all units and subprograms included.
The other choices for filtering reduce the amount of detail in the report. Choosing "Include only units"
(Subprogram_Detail) causes the Metrics report to show only the unit names and their totals for the
number of subprograms, complexity, and coverage.
Choosing "Show only Grand Totals" (Unit_Detail) causes the Metrics report to show only the very last
Note: If the option "Sort Metrics report by directory" is also set, then the "Show only Grand
Totals" setting causes the Metrics report to have a single GRAND TOTAL line for each
directory.
Specify which result to filter out of the Metrics Report. The default value
is No_Filtering.
The Display Global Data After feature gives you control over when global data objects are displayed in
the Test Execution report. If an expected value for the global object is set, then that expected value is
compared to its actual value at the same time.
Ranging from displaying more frequently to less frequently, you can choose to display global data after:
Each Event displays the current value(s) of global data objects at every transfer of control flow.
Each range iteration displays the current value(s) of global data objects after every iteration caused by
a parameter’s range of input values. If no parameter has an iteration, then the globals are displayed at
the end of the test execution.
Each slot iteration displays the current value(s) at the end of each iteration of a test case in a
compound test case. A compound can have one or more test cases in it, each with one or more
iterations. This setting causes the globals to be displayed (and compared) when each test case finishes
an iteration. If you apply this setting to a test case and then execute that test case alone (i.e. not in a
compound), then the globals are displayed at the end of test execution, because it is the only “slot.” In
the example below, the global values would be displayed twice, at the end of each slot iteration.
The default Each test case displays the current value(s) of global data at the end of a slot in a
compound test case. A compound can have one or more test cases in it; this setting causes the globals
to be displayed and compared when each test case in a compound finishes executing all of its
iterations. If you apply this setting to a test case and then execute that test case alone (i.e. not in a
compound), then the globals are displayed at the end of test execution, because it is the only “slot.”
When to display and check global values in execution results: at each event;
at the end of each range iteration; at the end of each slot iteration; at the
end of each slot. Its default value is Testcase.
Input Expected
VCAST _USER_GLOBALS
<<GLOBALS>>
VECTORCAST _INT 1 44 55
Get_Check_Total
Table 2..3/1
Execute the test case GET_CHECK_TOTAL.001. In the execution results, you see 6 events, three for
the iteration in which Table is 2, and three for the iteration in which Table is 3. Because Each Test Case
is the default setting, the global data value is displayed following Event 6, at the end of the test case.
The following table shows when the global data is displayed based on the setting of the “Display Global
Data After” option. Global data is never compared to its expected value at the start of an iteration.
In some cases, a test may contain a huge number of global data input, but you may not want to capture
all of this data in the Execution Reports. In the test case execution report, global and parameter data
will only be displayed in the test execution report if expected values are provided; globals and
parameters with input values only are not displayed.
When this option is set, global and parameter data will only be displayed in
the test execution report if expected values are provided. This option will
result in smaller execution reports and faster test execution in cases where
there is a lot of global and / or parameter data input. The default value is
False.
TEST.VALUE:<<OPTIONS>>.SHOW_ONLY_DATA_WITH_EXPECTED_RESULTS:TRUE | FALSE
If specified in a test case script, this option causes only events that have
Expected values to be included in the Execution Report. It overrides the
environment-wide setting for “Show only events with expected results.” If not
specified in a test script, the default value is the environment-wide
setting, accessible on the Report tab, Content sub-tab of the Tools =>
Options dialog.
In previous versions of VectorCAST, this option was named “No global inputs in results.” VCAST_
Setting this option causes VectorCAST to leave out any events in the Execution Report that do not
have expected data for any parameter or global value. This option will result in smaller execution reports
in cases where there is a lot of global and /or parameter data input.
When this option is set, events will only be displayed in the test execution
report if expected values exist for that event. This option will result in
smaller execution reports in cases where there is a lot of global and / or
parameter data input. The default value is False.
TEST.VALUE:<<OPTIONS>>.SHOW_ONLY_EVENTS_WITH_EXPECTED_RESULTS:TRUE | FALSE
If specified in a test case script, this option causes only events that have
Expected values to be included in the Execution Report. It overrides the
environment-wide setting for “Show only events with expected results.” If not
specified in a test script, the default value is the environment-wide
setting, accessible on the Report tab, Content sub-tab of the Tools =>
Options dialog.
This option is used to cause VectorCAST to make comparisons of expected values to actual values
before the UUT is even called. It is useful if a stub is called before you get to the UUT call (from user
code in an <<INIT>> test case, for example), and you want to verify that a value was passed in
correctly. By default, this option is false. Set it to true if you want to compare expected values before
the UUT call.
Setting this option enables comparisons between expected values and actual
values to occur before the call to the UUT. The default value is False.
By default, strings with unprintable characters are displayed in octal in the Execution report. To display
these strings in hex, select Tools => Options = Execute tab and set the option “Hex notation for
unprintable chars.”
Setting the option, “Convert octal/hex-encoded strings to ASCII” under Execution Report Options
causes the printable portions of these strings to be displayed in ASCII in the Execution report, while the
Report Options
When True, this option copies the contents of the 'Test Notes' box in the Test Case Editor to the Test
Case Management Report. One possible use for this is for test requirement numbers to be correlated to
test cases in the Test Case Management Report.
Copies the contents of the 'Test Notes' box in the Test Case Editor to the
Test Case Management Report. Its default value is
When this option is set to True, the Test Case Management Report uses the "old-style" pass/fail
results in the main table of the report. The Overall Results section does not change.
The "old-style" Test Case Management Report combined the Expected Values matched with the
Control Flow results, signals, exceptions, and terminations in the Pass/Fail column. Enable this option
to once again generate the Test Case Management Report with the data displayed in this way.
When set, this option reverts the Test Case Management Report to the behavior
found in VectorCAST version 6.0 and prior, in which the main body of the
report combined the Expecteds and Control Flow in one fraction in the
"Pass/Fail" column. The default value is False.
When this option is set, the version and build date of VectorCAST is added to the Configuration section
of the following reports:
Show the VectorCAST version in the configuration section of the reports. The
default value is False
By default, the Notes / Requirements section is included in the Full Report. When cleared, the option
"Show notes / requirements in the Full Report" excludes the Notes / Requirements in the Full Report.
Note that the section is always included in the Test Case Data Report.
This option will add a section to the Full Report listing test notes and
requirements. The default value is True.
This value controls exactly how the decimal part of a floating point number will be displayed. It modifies
the amount of precision that VectorCAST will use to print a floating point number.
Controls the number of digits used when the test harness prints floating
point Actual Values for the Execution Report or Range Data. This option's
setting is significant in the comparison of Expected Values for floating
point numbers as well as determining the min and max values.
If a command is supplied, the command is executed on each UUT source file and any non-stubbed
units. The results of the command are included in the Test Case Management Report and the Full Test
Report.
For example, setting this option to ls -l (on Linux) causes the following information to be added to the
two reports:
The full path to each UUT is derived from the Source directories used in the environment.
Command executed for each source file in the test environment, and a section
added to the Test Case Management and Full reports listing the name of each
unit in the test environment and the information generated by this command.
If this command is empty, then the File Version section fo the report will
not be created. <command> is a quoted string. It has no default value.
In the Test Case Notes tab, Parameter User Code, and Test Case User Code, you can import a text
template. This feature is useful if you want to document the requirements tested by each test case
following a standard format. Before importing the template, first specify the template file. To do this,
choose Tools => Options dialog, Report tab, Content sub-tab. Enter the full path to the template file
for the option “Notes section template.”
To import the template at the cursor location in a User Code editor, first click the checkbox next to
Enable. Right-click in the blank text area and choose Import template. The text from the template file
appears in the text area.
To use the template in test case Notes, first specify the template file. If the Notes tab of a test case is
empty and a template file is specified, then the template is automatically loaded when the test case is
opened. You can also right-click in the Notes tab and choose Import template. The text from the
template file appears in the Notes tab at the cursor location.
This option allows you to enter a path to a text file that contains a
template for the Notes section of the test case.
This option specifies a text file containing text you want VectorCAST to place at the top of exported test
scripts. Use the full path to the file, or a path relative to the environment directory (not working
directory). In the exported test script, the text is preceded with a comment marker (--) automatically, in
order to retain the custom text when the environment is rebuilt or updated.
This option allows you to enter a path to a text file that contains a custom
header for all generated test scripts.
Report Header
This option prepends a text string to the title or adds a new section to the Full Report, just below the
title, to include the contents of a file in TEXT or HTML format. Input can be a text string, or a file
containing several lines.
If an HTML file is provided, the contents should provide a <div> section with a table. For example:
<div class='report-block'>
<h2>Additional Data</h2>
<table class='table table-small'>
<tr><th>Tests created by</th><td>Fred Williams</td></tr>
<tr><th>Source revision</th><td>123abc789</td></tr>
</table>
</div>
Prepend a text string to the title or add a new section to the Full Report,
just below the title, to include the contents of a file in TEXT or HTML
format. Input can be a text string, or a file containing several lines. If an
HTML file is provided, the contents should provide a <div> section with a
table.
This option directs VectorCAST to include a text string in the Execution Report and Test Case Data
Report, as well as those sections in the Full Report, in place of empty cells in the Input and Expected
Value tables.
For example, use the text "N/A" for blank cells in reports to indicate that no Input Value or Expected
Value is intended in this position.
Text to show in blank cells in the Execution and Test Case Data report
sections. The default value is unset.
The Format sub-tab is used to change the formatting options for HTML and text reports. Pass your
cursor over any of the options to see an explanation of that option in a tool-tip.
Background color used for passing data cells in tables in the UI. The default
color is #ccffcc, a light green.
Background color used for failing data cells in tables in the UI. The default
color is #ffffcc, a light pink.
Background color used for partially passing data cells in tables in the UI.
The default color is #ffffcc, a light yellow.
Report Format
Choose Tools => Options and click the Report tab, then click the Format sub-tab.
The Report Format option determines if reports are displayed in HTML or Text. The default setting is
HTML. If you choose HTML, you can view the reports within VectorCAST or in an external browser. If
you choose text, you can view them within VectorCAST or in an external text editor.
Note that a test does not need to be re-executed in order to see the Execution report in a different
format.
Output format for VectorCAST reports: HTML or TEXT. The default value is
HTML.
To apply a custom CSS file to use in the HTML reports, enter the path to the custom CSS file.Your
custom style sheet is appended when reports are generated and the contents of the custom style sheet
are embedded in the <style> section following the default styles, when the reports are generated.
If the option is not set or the file cannot be found, the default CSS is used. The default CSS is located at
$VECTORCAST_DIR/python/vector/apps/ReportBuilder/css.
The Text sub-tab has options that are honored when the Report Format is Text.
The default Unit column width is 19 characters. To change the width, specify another size in the spin
box.
Width (in characters) of unit column in text reports. Its default value is
19.
The default Subprogram column width is 21 characters. To change the width, specify another size in the
spin box.
The default Testcase column width is 24 characters. To change the width, specify another size in the
spin box.
Width (in characters) of testcase column in text reports. Its default value
is 24.
The default Date column width is 11 characters. To change the width, specify another size in the spin
box.
Width (in characters) of date column in text reports. Its default value is
11.
The default Results column width is 13 characters. To change the width, specify another size in the
spin box.
Width (in characters) of result columns in text reports. Its default value is
13.
The default Complexity column width is 10 characters. To change the width, specify another size in the
spin box.
Width (in characters) of complexity column in text reports. Its default value
is 10.
The default Coverage Results column width is 18 characters. To change the width, specify another size
in the spin box.
Width (in characters) of coverage result columns in text reports. Its default
value is 18.
The default Notes column width is 30 characters. To change the width, specify another size in the spin
box.
Width (in characters) of notes columns in text reports. Its default value is
30.
Delimiter
The character separator for the Cover report created by the CLICAST report commands:cover
report csv_metrics and reports alternate. Enter the character in quotes, as in “@”. To enter
a tab, use “\t”. Valid delimiters are: ? , ' ; | { } [ ] @ ~ # $ _ \t \n
Character separator used in two CLICAST report commands cover report csv_
metrics and reports alternate. The default value is comma “,”.
This option will cause the text in the notes section to wrap at the first
space before 80 characters. The default value is FALSE.
By choosing this option, the actual results for Integer types are displayed in hexadecimal rather than in
the default decimal format in the execution reports.
Percentage that a floating point actual value can vary from its expected
value and still be considered a match. <percentage> is a floating point
number. Its default value is 0.000000.
TEST.FLOATING_POINT_TOLERANCE:<tolerance>
Note: For very small tolerances, the floating point representation of the value may not be exact
due to the resolution of floating point numbers; in this case, the floating point tolerance may
show more digits than the user expects. For example, a tolerance of 0.000000001 may actually
be displayed as 0.00000000099999.
Event Limit
The Event Limit option enables you to control the maximum number of events per test harness
execution. The event limit defaults to 1000. If this option is set to 0, no results data is captured,
although coverage data is captured. If the event limit is exceeded, VectorCAST automatically stops
test execution if your compiler supports signals, and the execution report shows:
The event limit applies to each test case. Each time a test executes, it resets the event counter to 1. If
you execute several test cases at once using Ctrl+Shift click to select them, or execute them all by
selecting the environment name and choosing Test => Execute, the counter starts over with each test
case (because the test harness is being executed multiple times).
Note: An individual test case can have its own event limit, which overrides the environment
event limit when that test case is executed. This option is accessed via the Options tab in the
Test Case Editor.
When executing a compound test case, the event limit is not reset for each slot; the Event Limit setting
applies to the Compound Test Case as a whole.
Note: The Event Limit feature is implemented through the raising of signals. If your target does
not support signals, the event limit capability will not be available.
Maximum number of events for the harness to process. If this option is set to
0, no results data is captured, although coverage data is captured. Its
default value is 1000.
Combination Testing
If a test case has a parameter with an input range or list, then the test harness executes the test case
once for each iteration. When two or more parameters have a range iteration, then each parameter’s
iteration count is incremented at the same time, by default.
Combination testing can be turned on for an individual test case, using Test Case Options, or for the
whole environment, using the Tools => Options dialog, Execute tab. If combination testing is
enabled, VectorCAST will determine the combinations of input values and use those values as stimulus
values.
For example, if parameter A has a range 1 to 3 with a delta of 1 and parameter B has a range 2 to 5 with
a delta of 1, and Combination Testing is off, then the test case executes this way:
A is 2
B is 3
Range Iteration #3:
A is 3
B is 4
Range Iteration #4:
A is 3 again
B is 5
If the combination testing option is enabled, then the test case executes this way:
This feature is not available with an environment built with VectorCAST version 3.2 or earlier.
Rebuilding the environment will enable this feature’s functionality.
Use all combinations of the values in the range expressions for test stimulus
values. The default value is False.
Setting this option enabled comparisons between expected values and actual
values to occur before the call to the UUT. Its default value is FALSE.
When this option is set, global and parameter data will only be displayed in
the test execution report if expected values are provided. This option will
result in smaller execution reports and faster test execution in cases where
there is a lot of global and / or parameter data input. The default value is
False.
TEST.VALUE:<<OPTIONS>>.SHOW_ONLY_DATA_WITH_EXPECTED_RESULTS:TRUE | FALSE
If specified in a test case script, this option causes only events that have
Expected values to be included in the Execution Report. It overrides the
environment-wide setting for “Show only events with expected results.” If not
specified in a test script, the default value is the environment-wide
setting, accessible on the Report tab, Content sub-tab of the Tools =>
Options dialog.
In previous versions of VectorCAST, this option was named “No global inputs in results.” VCAST_
SUPPRESS_GLOBAL_INPUTS was deprecated in VectorCAST version 4.2k
When this option is set, events will only be displayed in the test execution
report if expected values exist for that event. This option will result in
smaller execution reports in cases where there is a lot of global and / or
parameter data input. The default value is False.
TEST.VALUE:<<OPTIONS>>.SHOW_ONLY_EVENTS_WITH_EXPECTED_RESULTS:TRUE | FALSE
If specified in a test case script, this option causes only events that have
Expected values to be included in the Execution Report. It overrides the
environment-wide setting for “Show only events with expected results.” If not
specified in a test script, the default value is the environment-wide
setting, accessible on the Report tab, Content sub-tab of the Tools =>
Options dialog.
Ranging from displaying more frequently to less frequently, you can choose to display global data after:
The default Each Event displays the current value(s) of global data objects at every transfer of control
flow.
Each range iteration displays the current value(s) of global data objects after every iteration caused by
a parameter’s range of input values. If no parameter has an iteration, then the globals are displayed at
the end of the test execution.
Each slot iteration displays the current value(s) at the end of each iteration of a test case in a
compound test case. A compound can have one or more test cases in it, each with one or more
iterations. This setting causes the globals to be displayed (and compared) when each test case finishes
an iteration. If you apply this setting to a test case and then execute that test case alone (i.e. not in a
compound), then the globals are displayed at the end of test execution, because it is the only “slot.” In
the example below, the global values would be displayed twice, at the end of each slot iteration.
Each test case displays the current value(s) of global data at the end of a slot in a compound test case.
A compound can have one or more test cases in it; this setting causes the globals to be displayed and
compared when each test case in a compound finishes executing all of its iterations. If you apply this
setting to a test case and then execute that test case alone (i.e. not in a compound), then the globals
are displayed at the end of test execution, because it is the only “slot.”
When to display and check global values in execution results. Its default
value is EACH_EVENT.
TEST.VALUE:<<OPTIONS>>.GLOBAL_DATA_DISPLAY.
Its default setting is the environment-wide setting for the Display global
data option, accessible on the Report tab of the Tools => Options dialog.
TEST.VALUE:<<OPTIONS>>.DISPLAY_INTEGER_RESULTS_IN_HEX
clicast reports report was deprecated in VectorCAST version 4.0. Use clicast reports
custom full.
Change-Based Testing results in greatly reduced test time, allowing for more frequent testing, and
ensuring that bugs are fixed when they are introduced, instead of during a full test cycle at a later date.
This section describes the Incremental Rebuild feature that the VectorCAST Project Manager uses in
Change-Based Testing. Incremental Rebuild can be used in a unit test environment by the user in the
same manner that the VectorCAST Project Manager uses it.
If the environment does not have coverage, a full rebuild is required because the changes are
considered global in scope.
To perform an incremental rebuild, first make any modifications to a UUT or dependent body. Next,
save the changes and recompile the source. Then select Environment => Incremental Rebuild from
the Menu Bar.
Upon selection of the Incremental Rebuild option, if VectorCAST detects a change to UUTs or non-
stubbed dependents, it determines which package bodies or specs have changed, updates the
associated portions of the test harness, and recompiles and then re-instruments those portions. A pop-
up message is displayed indicating the incremental rebuild is complete. Select the OK button and the
Incremental Rebuild Report is displayed in the MDI window.
The Incremental Rebuild Report indicates which units were modified and which subprograms were
affected by the change. If execution results exist, the report also indicates which test cases were
affected.
Execution and coverage status are selectively removed following an incremental rebuild. Assuming the
environment has coverage instrumented, the test cases that call the modified function (and those
whose non-stubbed subprograms call the modified function) are considered "affected," and their
coverage data and execution data are removed from the environment. Unaffected test cases retain their
execution and coverage data.
Note: The Parameter Tree does not reflect changes to the source code after an Incremental
Rebuild; only the test harness is affected. To see changes such as renaming an enumerated
value, changing a parameter name, etc. reflected in the Parameter Tree, you must fully rebuild
the environment.
Global source code changes require a full rebuild. If a source code modification adds a new
dependency, such as a new parameter, when incremental rebuild is initiated the user is notified that the
change is too large to be rebuilt incrementally. The Incremental Rebuild Error Report prompts the user
to do a full rebuild of the environment to incorporate the source code changes into the test harness.
Code Coverage
The Code Coverage tool determines which lines of source code (statement), which branches of source
code (branch), or which equivalence pairs (MC/DC) have been executed by one or more sets of test
case data. The reports show you the completeness of your test suite. By analyzing the untested code,
you can easily work backward designing test cases to test these portions of your unit.
> Statement - reports on which executable lines of source code are being executed as the program
runs.
> Branch - displays which outcomes have occurred for each branch point in the program.
> MC/DC - similar to branch coverage, but in addition to reporting on the outcomes of all Boolean
expressions, it also reports on the values of all Boolean components of a complex conditional.
> Statement + MC/DC - reports on which executable lines of source code are being executed as
the program runs and on the outcomes of all Boolean expressions, and reports on the values of all
Boolean components of a complex conditional.
> Statement + Branch - reports on which executable lines of source code are being executed as
the program runs and displays which outcomes have occurred for each branch point in the
program.
> Basis Paths - identifies all possible paths of execution and consists of two parts: the list of basis
paths and an associated annotated source listing.
The current Industry Mode (see “To Set The Industry Mode” in Section 1) determines the set of
coverage choices. If no Industry Mode is set, Default is used. The following is a list of coverage types
listed by Industry Mode:
Initializing a type of coverage results in the instrumentation, compilation, and linking of an instrumented
version of the test harness which enables VectorCAST to monitor coverage during the execution of test
cases.
Note: Because code coverage runs in parallel with normal VectorCAST functionality, test cases
run with coverage ON can always be re-run on the non-instrumented test harness to verify real-
time performance.
1 5 * switch (Order.Entree)
{
case NO_ENTREE :
1 6 break;
case STEAK :
1 7 * Table_Data.Check_Total = Table_Data.Check_Total + 14.0;
1 8 * break;
case CHICKEN :
1 9 * Table_Data.Check_Total = Table_Data.Check_Total + 10.0;
1 10 * break;
...
To initialize Statement coverage, select a source file and choose Coverage => Initialize =>
Statement.
Statement coverage maps to the Industry Modes as shown in the following table:
Statement Level A Unit Level ASIL A SIL 1/2 SIL 1/2 Class A
Level B Unit Level ASIL B/C SIL 3 SIL 3 Class B
Level C Unit Level ASIL D SIL 4 SIL4 Class C
To initialize Statement coverage when using Industry Modes, choose Coverage => Initialize =>
<appropriate type> based on the current Industry Mode.
As VectorCAST instruments the test harness, a process dialog appears and a message is displayed in
the Message window:
Before you can inspect coverage results, you need to execute a test case.
Each branch point will have space for either one or two condition values. Boolean decision points (i.e., if
statements) are displayed with two place holders, because the condition can be either true or false, as
in the following example:
1 0 (T) Add_Included_Dessert
1 1 (T) (F) if((Order->Entree == STEAK &&
Order->Salad == CAESAR &&
Order->Beverage == MIXED_DRINK))
{
Order->Dessert = PIE;
}
else
1 3 (T) (F) if((Order->Entree == LOBSTER &&
Order->Salad == GREEN &&
Order->Beverage == WINE))
{
Order->Dessert = CAKE;
}
}
Case statements are handled by treating each “when” as a single decision branch point, as in the
following example:
1 1 case ORDER.ENTREE is
To initialize Branch coverage, choose Coverage => Initialize => Branch. Once VectorCAST has
completed instrumenting the test harness, the following message is displayed:
Branch coverage maps to the Industry Modes as shown in the following table:
Note: Before you can inspect coverage results, you need to execute a test case.
Boolean expressions, it also reports on the values of all Boolean components of a complex conditional.
A complex conditional is one in which there is more than one "term" in the conditional. For example: if (a
&& b) is a complex conditional with two terms.
Note: VectorCAST displays a progress bar when initializing MC/DC or Level A coverage on a
source file having an expression with more than 10 sub-expressions.
MC/DC coverage maps to the Industry Modes as shown in the following table:
Pairs Status
When MC/DC coverage is initialized, an additional “Pairs” metric is available in the coverage Metrics
Report, Test => View => Metrics Report and the Test Case Management Report, Test => View =>
Test Case Management Report
Pairs Status consists of the total number of MC/DC pairs satisfied compared to the total number of
MC/DC pairs in the unit.
To initialize MC/DC coverage, select a source file, and choose Coverage => Initialize => MC/DC.
As VectorCAST instruments the test harness, a process dialog appears and a message is displayed in
the Message window:
Before you can inspect coverage results, you need to execute a test case.
lowercase.
Note: For Ada source code, the comments should be inside the begin block. There can be no
spaces after the comment characters.
For example:
-- VCAST_DONT_DO_MCDC
specific statement to which you do not want MC/DC coverage applied.
-- VCAST_DO_MCDC
specific statement to which you DO want MC/DC coverage applied
To Initialize Statement+MC/DC
Statement+MC/DC coverage maps to the Industry Modes as is shown in the following table:
To initialize Statement+MC/DC coverage when using Industry Modes, choose Coverage => Initialize
=> <appropriate type> based on the current Industry Mode.
As VectorCAST instruments the test harness, a process dialog appears and a message is displayed in
the Message window:
Before you can inspect coverage results, you need to execute a test case.
To Initialize Statement+Branch
Statement+Branch coverage maps to the Industry Modes as is shown in the following table:
To initialize Statement + Branch coverage when using Industry Modes, choose Coverage => Initialize
=> <appropriate type> based on the current Industry Mode.
As VectorCAST instruments the test harness, a process dialog appears and a message is displayed in
the Message window:
Before you can inspect coverage results, you need to execute a test case.
In the figure below, the MANAGER unit is a UUT ( ), and DATABASE is a non-stubbed unit ( ). Any
unit that is not-stubbed or ignored is given the icon .
You may select individual units for coverage by checking the box next to the unit name or right-clicking
and choosing Check Selected. After selecting the units, choose a coverage type from the drop-down
list, and click the Initialize button to initialize coverage for the selected units.
Once you initialize a coverage type for real units (that is, non-stubbed or ignored units), any subsequent
use of Coverage => Initialize affects those units as well.
Note: It is not possible to specify different types of coverage for different units.
To exit the dialog without initializing coverage, click the Cancel button.
To disable coverage for a particular UUT, use the following CLICAST command:
You can instruct VectorCAST to ignore sections of your source code by delimiting the section with the
following source code comment markers:
(Ada)--VCAST_DONT_INSTRUMENT_START
--VCAST_DONT_INSTRUMENT_END
The text can be upper- or lowercase, but there can be no spaces between the comment characters and
the text of the delimiter. Care must be used to ensure that the start and end delimiters are within the
same scope. That is, if you have a case statement, you do not insert the delimiters such that half of the
case statement is not instrumented and half is instrumented for code coverage.
Note: For Ada source code, the comments should be inside the begin block. There can be no
spaces after the comment characters.
To Uninstrument an Environment
To completely remove coverage instrumentation from unit test environments, select Coverage =>
Uninstrument from the Menu Bar. Selecting the Uninstrument command removes all the coverage
data and the environment is no longer instrumented.
Note: Execution results are not removed when the environment is uninstrumented.
Remove coverage from all UUTs and non-stubbed units with custom coverage in a
unit test environment.
To Enable Coverage
The Coverage => Enable command turns on code coverage for the current open environment. This
command is only valid for environments that previously had coverage initialized and disabled.
Subsequent to enabling coverage, all test cases will be run against the instrumented test harness
(UUT_INST.EXE).
Run subsequent test cases with coverage monitoring enabled, using the
instrumented test harness.
To Disable Coverage
The Coverage = > Disable command turns off coverage for the current environment. This command is
only available for environments that have coverage initialized and enabled. All subsequent test case
execution will use the non-instrumented harness. The purpose of disabling coverage is to enable you to
run test cases using the original, non-instrumented test harness (UUT_INTE.EXE), so that you can
ensure that the coverage instrumentation does not affect the test results.
If this menu item is dimmed, coverage is already disabled or has not been initialized.
Run subsequent test cases with coverage monitoring disabled, using the un-
instrumented test harness.
coverage. Each unit has its own Coverage Viewer which contains an annotated version of the source
file, colorized to indicate the coverage level achieved.
The Coverage Viewer is opened by highlighting unit, subprogram or test case nodes in the Test Case
Tree and clicking the View Coverage icon in the Toolbar.
You can also open the Coverage Viewer for any number of units by selecting those units in the Test
Case Tree, right-clicking, and selecting View Coverage from the context menu.
To view coverage achieved by multiple test cases, first select multiple test case, subprogram, or unit
nodes in the Test Case Tree. Then right-click and choose Select Coverage, which activates the
checkboxes next to all selected test cases. The Coverage Viewer reflects the coverage achieved by
the selected test cases.
Statement Coverage
When Statement coverage is active, the Coverage Viewer looks like the following example:
The numbers along the left-hand margin indicate the subprogram number and the statement number. An
asterisk (*) indicates that the statement was covered (along with the color-coding).
Branch Coverage
When Branch coverage is active, the Coverage Viewer looks like the following example:
The left-most number indicates the subprogram number, and the second number indicates the decision
number. For Boolean decision points, two sets of parentheses are visible and are filled in with a T or F
when the True or False outcomes have been tested. For switch statements, one set of parentheses is
visible and a (T) is filled in when that branch outcome has been tested.
MC/DC Coverage
When MC/DC coverage is active, the Coverage Viewer looks like the following example:
The left-most number indicates the subprogram number, and the second number indicates the decision
number. For Boolean decision points, two sets of parentheses are visible and are filled in with a T or F
when the True or False outcomes have been tested. For decision points that have multiple components
(e.g., a || b), each sub-condition is displayed on a separate line. The sub-conditions are numbered
.1, .2 to .n, where n is the total number of sub-conditions.
Note: When you re-initialize coverage, rebuild or update the environment, or regenerate basis
paths, the coverage results are removed.
For MC/DC or STATEMENT+MC/DC coverage, the left margin includes a red arrow that, when
clicked, opens the Equivalence Matrices below the associated subprogram in the Coverage Viewer.
When the arrow buttons are clicked, the coverage source window scrolls to show the next covered or
uncovered line. Partially-covered lines are considered both covered and uncovered. Contiguous blocks
of lines are skipped over, until a line of a different block type is found or a line off-screen is found.
If there are no lines found, the message “Search reached bottom” or “Search reached top” appears,
depending on which direction you were searching.
In addition, pressing the Home key or End key (or alternatively, pressing Ctrl+Home or Ctrl+End)
brings the cursor to the beginning or end of the file, respectively.
To customize the display in the Coverage Viewer, right-click anywhere in the window. The context
menu appears allowing you to expand or collapse all subprograms in the viewer.
Use the Expand and Collapse menu items as “filters” or “layers” to get the exact display you want in the
Coverage Viewer.
You can also expand and collapse individual subprograms by clicking the (collapse) and (expand)
icons located on the left of the viewer.
Collapsing and expanding the subprograms in the Coverage Viewer does not affect any of the Coverage
reports.
By default, selecting a line in the Coverage Viewer, Original Source text editor or CBA Editor will
automatically scroll to the corresponding line in any other open editor. To change the behavior, right-
click within the Coverage Viewer or CBA Editor and uncheck the Map Selection to Original Source
View option on the context menu.
Alternatively, in the Original Source text editor, right-click and uncheck the Map Current Line to
Coverage View option on the context menu.
Right-click a covered line in the Coverage Viewer to see a pop-up menu listing the test case(s) that
cover it. True (T), False (F) or both True and False (TF) branch coverage is also indicated. To open that
test case in the Test Case Editor, right-click, and select the test case name from the pop-up list.
Note: Long test case names are truncated in the pop-up menu. Select the Show full names
option to display the full test case name. This option is only available when test case names are
truncated.
Clicking Yes in this dialog removes coverage checkboxes from the Test Case Tree, as shown below.
Coverage remains initialized.
The report is sensitive to what level is selected in the Test Case Tree. If you select the top-level node
for the environment, the report includes coverage achieved by test cases executed in all UUTs and non-
stubbed units that have coverage instrumented using Custom Initialize. You can narrow the content
down by selecting particular units in the Test Case Tree for which you want coverage achieved. Once
you have made a unit selection, choose Test => View => Aggregate Coverage Report.
The report contains coverage data from all test cases within the selection, regardless of whether they
are toggled on to show coverage or not.
The annotated source code is color-coded to mark the exercised lines (green), the un-exercised lines
(red), lines covered by analysis (blue) and partially exercised lines (yellow), similar to what is seen in
the Coverage Viewer.
The report is sensitive to what level is selected in the Test Case Tree. If you select the top-level node
for the environment, the Metrics table includes coverage achieved by test cases executed in all UUTs
and non-stubbed units that have coverage instrumented using Custom Initialize. You can narrow the
content down by selecting particular units in the Test Case Tree for which you want coverage achieved.
Once you have made a unit selection, choose Test => View => Metrics Report.
The report contains coverage data from all test cases within the selection, regardless of whether they
are toggled on to show coverage or not.
If coverage data is not available, then only the Unit, Subprogram, and Complexity columns are present
in the Metrics Report.
If the report format is HTML, the Metrics Report looks similar to the following:
Pairs Status
When MC/DC or STATEMENT+MC/DC coverage is initialized, an additional “Pairs” column is added
to the Metrics Report. MC/DC Pairs consists of total number of MC/DC pairs satisfied compared to the
total number of MC/DC pairs in the unit.
If the report format is HTML, the report looks similar to the following:
Alternatively, from the Menu Bar, select Coverage => Code Coverage Summary => Code
Coverage Summary.
To override tracking of selected UUTs in the Test Case Tree, open the drop-down menu for the Data
Summary Report and select Options => Track Current Selection. Remove the check next to the
option. The tracking icon on the Summary table will change to gray to indicate that the summary is
now tracking all units in the Test Case Tree.
The Code Coverage Summary table is dynamic. When the Track Current Selection option is enabled,
as units are selected and deselected in the Test Case Tree, the Code Coverage Summary table
updates in real time reflecting the selections.
The data displayed in the Code Coverage Summary includes the Unit name, the Subprogram name, the
Cyclomatic Complexity (Vg), the Test Cases Count (showing the number of unique test cases that
have hit a particular subprogram during execution, with compound tests being counted as one test
regardless of the number of slots in the compound), and the achieved coverage for each coverage type.
Note: When I/O type is set to "Animation" (e.g. when using Basis Path coverage), the Test
Cases Count column indicates the number of times a function is entered. This total can also
include multiple slot iterations of a compound test.
The Totals row at the top of the table displays the totals for each data column. In the example above,
note that in the unit MANAGER, we have a total of 29 branches of which 4 branches are covered and 25
are uncovered.
The Summary table updates whenever coverage data is updated. For example, the table refreshes
when coverage is initialized, or following test execution.
Double-clicking on a line in the Code Coverage Summary opens the corresponding UUT in the
Coverage Viewer.
Coverage data can be accessed all the way down to the individual subprogram level by filtering. Access
the filter by typing into the top row of any column. Clear the filter by right-clicking in the top row and
selecting Clear Filter from the context menu. In the example below, the table has been filtered to only
show subprograms with Cyclomatic Complexity less than 4.
Filtering supports the following symbols: <, >, =. For example, the summary can be filtered to show
only subprograms with a Cyclomatic Complexity greater than 2. Other examples of filtering inputs are:
10 - lists subprograms matching the specific value of "10" in the selected column
>50 - lists subprograms greater than the value of "50" in the selected column
<90 - lists subprograms less than the value of "90" in the selected column
=100 - lists subprograms matching the specific value of "100" in the selected column
< - lists subprograms with empty values in the selected column
> - lists subprograms with non-empty values in the selected column
= - lists subprograms with non-empty values in the selected column
Printing the contents of the Code Coverage Summary is supported. See "To Print an Open Window" on
page 75 for more information.
To enable the Function Coverage option, select Tools => Options from the Menu Bar. Select the
Coverage => Options => Instrumentation Options tab. Check the box for 'Always display function
coverage' and select the Apply button.
In the Function Coverage column, 100% coverage is displayed in green and no coverage is displayed in
red.
The Function Coverage column is displayed in each report instance that includes the Metrics report
(Full Report, Test Case Management Report, and Aggregate Coverage Report).
Test Path 1
(1) if ( value < min ) ==> FALSE
(3) if ( value > max ) ==> FALSE
Test Path 2
(1) if ( value < min ) ==> FALSE
(3) if ( value > max ) ==> TRUE
Test Path 3
(1) if ( value < min ) ==> TRUE
Complexity: 3
Using the above example, Test Path 1 specifies that value < min should be False, and value >
max should also be False. If we use 4 for min and 10 for max, setting value to 5 satisfies this Test
Path.
Test Path 2 specifies that value < min should be False, and value > max should be True. If we
use 4 for min and 10 for max, setting value to 11 satisfies this Test Path.
Test Path 3 specifies that value < min should be True. We can use 4 for min and 3 for value.
There is no need to set max; it is irrelevant.
Executing these three test cases gives us 100% branch coverage and 100% basis paths coverage.
VectorCAST can automatically build the basis path test cases. For details, see "To Insert Basis Path
Test Cases" on page 155
The Tools => Basis Path => View Basis Path command enables you to view the results of the
computation of test paths for test case building. The contents of this report reflect the items selected in
the Test Case Tree.
The number of paths identified equals the code complexity, also referred to as V(g). The code
complexity corresponds to the number of test cases that must be developed to exercise the unique
paths in a subprogram.
This information can also be viewed by accessing the Basis Paths tab of the Coverage window.
Extract the basis path analysis for all units or the specified units to
standard output or the specified file. If VCAST_CUSTOM_REPORT_FORMAT is HTML
and <outputfile> is not specified, the file is saved to <env>_basis_path_
report.html.
MC/DC Coverage
When you have the Coverage Viewer open, and MC/DC or STATEMENT+MC/DC coverage is
initialized, if you hover your cursor over any fully or partially covered line of code, a tooltip displays the
test case(s) which cover the line.
For MC/DC or STATEMENT+MC/DC coverage, the left margin includes a red arrow that, when
clicked, opens the Equivalence Matrix below the associated subprogram in the Coverage Viewer.
Clicking the arrow again closes the Equivalence Matrix. The matrices can also be viewed by selecting
Tools => MC/DC => View Equivalence Matrices and viewing the MC/DC Condition Tables Report.
See"To View the Equivalence Matrices Report" on page 345 for more information.
For each Boolean conditional there is a corresponding matrix. Each row in the matrix shows a unique
combination of values for the components of the complex conditional, as well as the result of the
condition for each combination. The equivalence pairs are pairs of rows which demonstrate that the
result of the conditional changes from True to False as one component changes, while all other
components are held constant.
The Equivalence Pair Matrix shows both the static pair analysis and the pair coverage achieved. The
Pair A through Pair n columns (where n is the number of sub-conditions) show candidate row pairs for
proving condition components. An asterisk next to a row number indicates that the row has been
covered. When an equivalence pair has been covered, the Pa through Pn summary lines reflect that
information.
The Equivalence Pair Matrix shows only rows having equivalence pair information. Rows that do not
have any pair infomation are eliminated from the display because they add no useful information.
The following example shows a single complex conditional with the corresponding equivalence pair
matrix.
This source listing shows that the “if” condition has been executed with A=True, B=True (Row 1);
A=False, B=True (Row 3); and A=False, B=False (Row 4). The MC/DC branch coverage is 6/6. Six out
of a total of six outcomes have been satisfied, and the pair coverage is 1 of 2 pairs satisfied.
The MC/DC Condition Tables report shows only rows having equivalence pair information. When
VectorCAST instruments for MC/DC or Statement+ MC/DC coverage, it calculates the Equivalence
Pair matrix. Each row represents a combination of possible outcomes for each condition in the
expression, and the resulting output. A pair is a set of two rows which demonstrate that the result of the
conditional (Rslt column) changes from True to False as one component (Ca or Cb) changes, while all
other components are held constant.
For example, in the matrix below, rows 1 and 3 form a pair, and rows 1 and 2. However, row 4 does not
form a pair with any other row because changing one of the components, such as Ca, from F to T does
not change the result of the conditional (F) to True.
VectorCAST displays the rows that do not have any pair information as having blank boxes under the
Pair column (Pa or Pb, in the example). These rows are eliminated from the display of the Equivalence
Pair matrix because they add no useful information.
Extract MC/DC Equivalence matrices for all units or the specified unit to
standard output or the specified output file. If VCAST_CUSTOM_REPORT_FORMAT
is HTML and <outputfile> is not specified, the file is saved to <env>_mcdc_
tables_report.html.
If the report format is HTML, the report looks similar to the following:
The Source line # refers to the source code line in the Coverage Viewer that corresponds to the
expression being evaluated in the matrix. After noting the line number, right-click the unit in the Test
Case Tree, and choose View Coverage. Then, in the Coverage Viewer, type Ctrl+F (or choose Edit
=> Find from the Menu Bar), and type in the line number.
Coverage Animation displays the flow of control from one unit to another, using the test’s coverage
data. Calls to stubbed units are not included in Animation. Not-stubbed units are included if they have
coverage initialized using Coverage => Initialize Custom.
Coverage can be animated for only one test case in the Test Case Tree at a time. The item currently
activated for Animation has a check in the box to the left of its name in the Test Case Tree. If you check
another item, the Animation toolbar dims until you activate another test result for animation.
Note: If the Animation toolbar is dim, activate a test case for animation.
After setting the Coverage I/O type to Animation, instrument for a type of coverage. If you want to see
each statement being executed, you would instrument for Statement coverage; if you want to see the
choice made at each branch point, you would instrument for Branch coverage. To see each entry point,
decision point, and statement covered, choose MC/DC, STATEMENT+MC/DC, or
STATEMENT+BRANCH.
Next, execute a test case. Right-click the test case in the Test Case Tree, and then choose Activate
Coverage Animation from the popup menu:
The Coverage Viewer for the unit containing the first covered line in the test result opens, with the
current line on the first covered line. The coverage checkbox next to the test case is checked.
If not already open, the Coverage Animation toolbar (pictured below) appears in the main toolbar.
Click the Play button to begin the animation. In the Coverage Viewer, the current line progresses
steadily from the first covered line to the last. The speed of each step is 1 second. The source code in
the Coverage Viewer scrolls up, keeping the current line in the middle. If a subprogram in another unit is
called and that unit has coverage initialized, then its tab comes to the front of the Coverage Viewer and
animation continues in that tab.
> the blue arrow indicates the current line in the animation of control flow
Animation. If there is a checkmark, the toolbar is visible; otherwise it is hidden. Alternatively, choose
View => Dock Control and toggle the setting for Coverage Animation.
Fast Forward Moves the current-line indicator to the last line executed.
To Set a Breakpoint
Setting a breakpoint causes the animated display of the execution flow to pause on the line marked with
the breakpoint . To set a breakpoint, select the line to make it current, then click the Set Breakpoint
button
> jump to the next breakpoint (or the end, if there is none)
Three coverage I/O types are supported: Real-time, Buffered, and Animation. Changes to the Coverage
I/O type take effect upon coverage instrumentation.
> Real-time: Default. Coverage data is gathered as the test runs, and output to the results file
immediately. This I/O type is optimized.
> Buffered: Coverage data is stored and then output when the test finishes executing. This method
is useful is your target platform is slow. This I/O type is optimized. Buffered I/O gives better
performance, but requires more memory than Real-time coverage I/O.
> Animation: This I/O type is not optimized. Instead, coverage data is gathered in the order
encountered, in preparation to animate the test result’s coverage in the Coverage Viewer. The
animated coverage results reveal the flow of control.
Type of coverage I/O that the instrumented harness will perform. The default
value is Real_Time.
The “Coverage field width” option specifies the width, in characters, of the far left column in the
Coverage tab in the Coverage Viewer. Increase this number if you have a large number of subprograms
The number of digits to allow for the numeric identifier displayed in the
left column of a coverage report. The default is 8. Increase this number if
you have a large number of subprograms or a subprogram with a large number of
statements or branches. Its default value is 8 characters.
This option allows you to adjust the coverage database cache size. Increasing the value may increase
performance, but care should be taken not to set the value greater than available memory. Changes to
this option take effect when re-opening an environment.
Increasing the maximum coverage database cache size may increase performance,
but take care not to set it to a value greater than the amount of available
system memory. For a system with 2GB, a 1000 MB maximum cache is probably
sufficient. Changes to this option take effect when reopening an environment.
This number tells VectorCAST to expect no more than this many subconditions
in an MC/DC expression. Its default value is 8.
You can override the default behavior on a statement-by-statement basis, so that coverage is disabled
on a conditional statement that would, by default, have instrumentation applied, or enabled on
assignment statements that would by default, not have instrumentation applied.
To enable coverage for a specific statement, precede the statement with the comment VCAST_DO_
MCDC. The comment must exist on lines with no other text, immediately preceding the source code line
for which you want to modify the coverage. The text can be upper- or lowercase, but there can be no
spaces after the comment characters.
foo(a&&b);
Set this option to prevent VectorCAST from casting conditional expressions to boolean when
instrumenting for Branch or MC/DC coverage.
is False.
When the “Use Static Memory Allocation on the Target” option is checked, VectorCAST uses a static
memory model when allocating memory for coverage data. By default, the instrumentation routines
allocate memory as needed (dynamic). If you choose the static memory model, you can specify limits
for the maximum number of subprograms and MC/DC expressions for which memory should be pre-
allocated.
This option tells VectorCAST that only static data allocation can be used on
the target platform. This is used when collecting coverage data. If this is
set to True, then use the options VCAST_MAX_MCDC_STATEMENTS and VCAST_MAX_
COVERED_SUBPROGRAMS to configure the static allocation. Its default value is
FALSE.
To set the “Maximum covered subprograms” option, you must check the box next to “Use static
memory allocation” and use the Buffered Coverage I/O type.
The “Maximum covered subprograms” option tells VectorCAST how many unique subprogram calls it
should allocate memory for. If a test case or compound test case execution encounters 500 different
subprograms calls, and the option is set to 499 or lower, then a text error message is added to the
coverage data indicating that the limit was reached. This text error message is used to generate popup
messages and CLICAST output messages.
The “Maximum MC/DC expressions” option tells VectorCAST how many unique MC/DC expressions it
should allocate memory for. If a test case or compound test case execution encounters 500 MC/DC
expressions with unique values for its parameters, and the option is set to 499 or lower, then a text error
message is added to the coverage data indicating that the limit was reached. This text error message is
used to generate popup messages and CLICAST output messages.
The “Run script after instrumentation” option provides a means to execute a batch file (Windows) or
shell script (Linux) once after each unit in the environment is instrumented. This feature is useful for
performing post-processing on the unit’s instrumented source code. Specify the full or relative (to the
environment directory) path to the batch file or shell script in the option. The full path to the unit is
passed as an argument to the script.
Once the path to the example script is specified in the “Run script after instrumentation” option and the
environment is instrumented for coverage, this example script causes the file path to the unit and the
instrumented file itself to be appended to the file report.txt, located in the environment directory.
The first few lines of report.txt are shown below. These lines are repeated once for each unit in the
environment.
c:\vcast\Environments\TUTORIAL_C\manager_inst.c
/* VectorCAST/Cover */
#ifndef VCAST_CONDITION_TYP
#define VCAST_CONDITION_TYP int
#endif
#ifdef __cplusplus
extern "C" {
#endif
#ifdef VCAST_USE_STD_STRING
#include <string.h>
#endif
/*
---------------------------------------
-- Copyright 2017 Vector Informatik GmbH --
-- North Kingstown, Rhode Island USA --
---------------------------------------
*/
...
If you have an external text editor defined on the GUI tab, clicking the Edit File button ( ) opens the
script in the specified external text edit.
This script will be run after VectorCAST has instrumented a file. The path to
the source file is passed as an argument to the script. It has no default
value.
The “Animation play speed” option enables you to change the speed of coverage animation progress.
By default, coverage animation proceeds at the rate of one line per second. To go faster, increase the
value. To go slower, decrease the value. The ratio 1.0 represents 1 line / 1 second.
The speed ratio for coverage animation. The default value is 1.0, which is 1
line per 1000 msec.
The “Uncovered line indicator” option allows the selection of a character to be used to mark "uncovered"
lines in the Coverage Viewer and coverage reports. It allows easy identification of uncovered lines in
the VectorCAST reports outside of the VectorCAST GUI.
The feature can be used with Statement, DO-178B Level A, B, and C coverage types.
Valid character choices are: '#', '$', '%', '&', or '+'. The default is "no character".
The uncovered line indicator. The default value is space, that is none.
Reinstrumentation is necessary after changing this option. Coverage => Initialize, and select
coverage type
The Coverage Viewer tab enables you to control the fonts and colors used in the Coverage Viewer and
the various coverage reports. By default, covered lines are shown in green, uncovered lines are shown
in red, and non-executable statements are shown in black. It is recommended that you use a
monospaced font for the Coverage Viewer (e.g. Courier) as that allows the tables and report to be
aligned properly.
Note: These options do not have a CLICAST counterpart; they are strictly for the Coverage
Viewer.
Set this option to prevent VectorCAST from casting conditional expressions to boolean when
instrumenting for Branch or MC/DC coverage.
Partially Covered Lines – lines that have multiple outcomes where some but not all outcomes have
been tested.
Uncovered Lines – lines or branches that have not yet been executed.
Click the Default Fonts button. This option returns the font and color for each line type to the default
settings: Courier font, Normal Font style, 10 pt Size.
The Font... button for each line type and the Change All Fonts... button enable you to select the font,
font style and size for one or all line types in the Coverage Viewer. The Default Fonts button enables
you to revert to the default font settings. Default font settings are: Courier font, Normal Font style, 10 pt
Size.
The Text Color... buttons enable you to change the text color for each of the five line types.
The Background Color... buttons enable you to change the background color for the five line types.
To import coverage results, choose Coverage => Import Results from Environment. If the menu
item is dimmed, first initialize coverage by choosing Coverage => Initialize => coverage-type. If
coverage has been initialized but disabled, you must enable it for the menu item to be available.
> The same files must be used in both environments. VectorCAST compares the filenames and file
checksums to ensure that the files are the same or copies of each other.
> “Instrument Files in Place” in the VectorCAST/Cover environment may be used with VectorCAST
version 5.0+, otherwise, “Instrument Files in Place” may not be used.
> The same type of coverage must be used in both environments.
Select the results files from that environment to import. You can choose to import the other
environment’s test execution results or imported results or both.
A filter is provided at the top of the pane making it easier to filter out some results when working with a
large set of test results. By entering text or a regular expression in the <<filter>> field, the user can filter
by name. Clear the filter by right-clicking in the top row and selecting Clear Filter from the context
menu.
When you are ready to import the coverage results, click the Import button. To exit the dialog box
without importing, click Cancel.
After the import is complete, a Coverage Import/Export log is displayed showing a series of status (S)
or error (E) messages. Some of the messages you may see in the log file:
initialized for that unit. An example is when there are non-stubbed dependent units in the
environment but only the UUT was initialized for coverage. Use Initialize Custom to enable
coverage for non-stubbed dependent units.
> (E) No match found for source file
Although the file names may be the same, the checksums of the source files differ.
> (E) Coverage types differ
The type of coverage in the importing environment is different than the type of coverage in the
environment to which results are being imported. They must match for a successful import.
> (E) No translatable data for result
A result was found in the importing environment, but the data did not match any source file in the
current environment.
> (S) Coverage data was loaded
A result file was successfully imported.
The Coverage Import/Export Log file is named IMPORT.LOG and resides in the environment directory.
You can view it at any time by selecting Coverage => View Import Log.
The imported results appear in the Test Case Tree, near the top. They are listed in a folder named
"Imported Results", as shown below.
After the script file is exported, a log is displayed showing a series of status (S) or error (E) messages.
Some of the messages you may see in the log file:
> (S) Script creation started – the process of creating the script file (.cvr) is beginning.
> (S) Looking for results directory IMPORTED_RESULTS – VectorCAST uses the results in the
directory IMPORTED_RESULTS, located in the environment directory, to create the script file
containing imported results.
> (E) Error reading imported results directory – there was an error reading the directory IMPORTED_
RESULTS, located in the environment directory. Make sure the results have not been deleted or
corrupted.
> (E) No imported results found – VectorCAST could not find any results file in the directory
IMPORTED_RESULTS. The script file could not be successfully created.
> (E) Script creation aborted – the script file could not be created due to other errors.
> (S) Script creation completed – the script file was created successfully.
RESULT.DATA:2 2 17
RESULT.END
RESULT.NEW
RESULT.NAME:IMPORTED_RESULTS\PLACE_ORDER-PATH-2
RESULT.DATA:1 1 1
RESULT.DATA:1 2 1
RESULT.DATA:2 1 1
RESULT.DATA:2 1 3
RESULT.DATA:2 2 1
RESULT.DATA:2 2 2
RESULT.DATA:2 2 3
RESULT.DATA:2 2 4
RESULT.DATA:2 2 5
RESULT.DATA:2 2 6
RESULT.DATA:2 2 7
RESULT.DATA:2 2 16
RESULT.DATA:2 2 17
RESULT.END
RESULT.NEW
RESULT.NAME:IMPORTED_RESULTS\PLACE_ORDER-PATH-3
RESULT.DATA:1 1 1
RESULT.DATA:1 2 1
RESULT.DATA:2 1 1
RESULT.DATA:2 1 3
RESULT.DATA:2 2 1
RESULT.DATA:2 2 2
RESULT.DATA:2 2 3
RESULT.DATA:2 2 4
RESULT.DATA:2 2 5
RESULT.DATA:2 2 6
RESULT.DATA:2 2 8
RESULT.DATA:2 2 9
RESULT.DATA:2 2 16
RESULT.DATA:2 2 17
RESULT.END
RESULT.NEW
RESULT.NAME:IMPORTED_RESULTS\PLACE_ORDER-PATH-4
RESULT.DATA:1 1 1
RESULT.DATA:1 2 1
RESULT.DATA:2 1 1
RESULT.DATA:2 1 3
RESULT.DATA:2 2 1
RESULT.DATA:2 2 2
RESULT.DATA:2 2 3
RESULT.DATA:2 2 4
RESULT.DATA:2 2 5
RESULT.DATA:2 2 6
RESULT.DATA:2 2 10
RESULT.DATA:2 2 11
RESULT.DATA:2 2 16
RESULT.DATA:2 2 17
RESULT.END
RESULT.NEW
RESULT.NAME:IMPORTED_RESULTS\PLACE_ORDER-PATH-5
RESULT.DATA:1 1 1
RESULT.DATA:1 2 1
RESULT.DATA:2 1 1
RESULT.DATA:2 1 3
RESULT.DATA:2 2 1
RESULT.DATA:2 2 2
RESULT.DATA:2 2 3
RESULT.DATA:2 2 4
RESULT.DATA:2 2 5
RESULT.DATA:2 2 6
RESULT.DATA:2 2 12
RESULT.DATA:2 2 13
RESULT.DATA:2 2 16
RESULT.DATA:2 2 17
RESULT.END
IMPORT.END
Create a coverage script containing test execution results that are present
in <env>, including imported results. Coverage scripts have the extension
.cvr.
Create a coverage script containing only imported coverage results that are
present in <env>.
Any errors that occur during importing of a coverage script are logged in a file named Import.log. To
view this file, choose Coverage => View Import Log.
appears asking you to verify that you want to continue with this operation, as shown below.
Clicking No cancels the operation without removing the imported coverage results. Clicking Yes
removes the imported coverage results from the environment and the IMPORTED_RESULTS directory
from the environment directory. There is no clicast command to delete all imported results.
VectorCAST CBA provides the ability to do code analysis directly within VectorCAST using the
Coverage Analysis Editor and combines the test and analysis coverage metrics in a single report.
VectorCAST CBA can also import analysis files, including those generated by third party tools.
UUT or non-stubbed unit. To add Coverage Analysis for a selected unit, select the CBA button on
the Toolbar. Clicking the CBA button requires a unit to be selected. Alternatively, right-click a UUT in
the Test Case Tree and select Add Coverage Analysis from the context menu.
The Coverage Analysis Editor opens in the MDI Window and a CBA node displays in the Test Case
Tree. A CBA data file is created for the selected UUT and this Analysis result displays beneath the
CBA node. When an Analysis result is first created, it is empty and contains no data. Empty Analysis
results display the icon, even if they contain notes or requirements. Each CBA data file
corresponds to a single UUT, but you can create multiple Analysis results per UUT.
In the Coverage Analysis Editor, the Notes tab on the right is a free-form text editor allowing you to
annotate the associated analysis. Use the Requirements pane to trace the Project Requirements and
Test Case Requirements associated with the selected code. The Requirements tab is populated by
VectorCAST Requirements Gateway. Use the Save button to save inputs.
Statement Coverage
With Statement coverage instrumented, the Coverage Analysis Editor displays boxes on the left for
statement outcomes that are uncovered. To mark a statement or condition as "considered covered",
select the check box. Lines covered by analysis are displayed in blue.
In the Coverage Analysis Editor, a check box does not appear next to a line if it is already covered by a
test case execution result. Regular test case results, if they are present in the environment, take
precedence over CBA results.
Branch Coverage
When Branch coverage is instrumented in the environment, the Coverage Analysis Editor displays
each subprogram with a True branch (T) for the entry result, and True and False branches (T) (F) for
each expression.
To mark a condition as having the True branch covered, select the check box in the (T) column. The "(T)
" is displayed in blue. The branch is now partially covered because the True branch is Covered By
Analysis, and the False branch is not covered.
To mark an expression as having the False branch covered, select the check box in the (F) column in
the Coverage Analysis Editor. The "(F)" is displayed in blue.
As with Statement coverage, if either or both of the True and False branches is already covered by
regular test case execution results, then the check box is not available in the Coverage Analysis Editor,
and the expression shows yellow if partially covered, or green if already fully covered.
MC/DC Coverage
To add coverage analysis for a condition with multiple sub-conditions when MC/DC Coverage is
instrumented, you annotate that one or more rows of the Equivalence Pairs table is covered.
To access the equivalence pair table, click the arrow to the left of the condition. The truth table
opens and a check box is displayed for each row. When a checkbox is selected, the associated sub-
condition is considered "covered" and displayed in blue.
Double-click on an existing Analysis result in the Test Case Tree to open the Coverage Analysis Editor.
Right-clicking on the Analysis result provides a contextual menu allowing you to Rename, Delete,
Select Coverage and Deselect Coverage. Select and Deselect turns On/Off the Analysis result in the
Coverage Viewer. Deleting an Analysis result removes its data, notes, and requirements from the
environment.
Lines covered only by CBA results are displayed in green (indicating the line is covered) with a blue "A"
(indicating the line is covered only by CBA).
Lines covered by both regular execution results and CBA results are displayed in green (indicating the
line is covered) and with a blue asterisk "*" (for statement) or a blue "T" or "F" (for branch), which
indicates that it is also covered by CBA.
Use the buttons provided in the Covered-By-Analysis group box to change the font, text color and
background color. When changes are complete, select the Apply button to save the changes to the
.CFG file.
Subprogram
Unit Number Line Number
Number
9 1 1
9 1 2
The .dat file above covers the first two lines of Subprogram 1 of Unit 9.
Subprogram
Unit Number Condition Outcome Covered
Number
9 1 0 T
9 1 1 T
In the .dat file above, 9 1 0 T is the entry point to the first Subprogram of Unit 9. 9 1 1 T is the True
Unit Test environments always have Unit 9 as the first UUT, Unit 11 as the second UUT, Unit 12 as the
third, etc. The user can refer to the Coverage Viewer to locate the Subprogram number and Condition
number, whether T or F branch is covered, and then fill in the Unit number.
with the icon indicating it is empty , double-click the icon to view the information in the Notes.
To access the Covered By Analysis Report, select Test => View => Covered By Analysis Report
from the Menu Bar.
The CBA Report has two sections: The Covered By Analysis, Per Line section and the Covered By
Analysis Result File section(s).
The Covered By Analysis, Per Line section lists each covered line in the unit, and identifies which
CBA result file covers that line. The Subprogram ID corresponds to the left-hand number in the
Coverage Viewer and CBA Editor. The Line number corresponds to the right-hand number in the
Coverage Viewer and CBA Editor when the coverage type is Statement. The Line number represents
the branch or decision number when the coverage type is other than Statement. There may be more
than one CBA result covering a line. If the CBA result covers the True (T) outcome of a branch or
decision, "(T)" is displayed in the line column. Similarly, if it covers the False (F) outcome, "(F)" is
displayed.
The Covered By Analysis Result File section includes one table for each CBA result file. The table is
similar to the Metrics Report in that it shows the number of statements and/or branches covered by that
CBA result. For Statement and Branch coverage, only the number of lines or conditions that are
Covered by Analysis are shown.
For MC/DC or Level A coverage, the number of conditions covered by CBA are shown. However, for
the MCDC Pairs column, both CBA results and execution results are considered. An MC/DC Pair is
considered satisfied if at least one of the pair components (or row) is a CBA result. The remaining
component may be either a CBA result or an execution result.
In the example below, two pairs are covered. Each pair has one component covered by the CBA result
file ADD_INCLUDED_DESSERT (rows 2 and 3). The other component (row 1) is covered by an
execution result.
Metrics Report
In the Metrics Report, when CBA results are present in the environment, they are displayed in italics in
the row below the subprogram and the coverage achieved by test execution is displayed below the CBA
results.
There are several types of user code: environment user code, test case user code, individual parameter
user code, and stub user code.
> Environment User Code is best suited for operations that relate to all test cases; loading data from
a file or initializing a database are two examples.
> Stub User Code is added to functions that have been stubbed, allowing you to specify extra logic
to be executed when the stub is called.
> Test Case User Code is used to include input value or expected value user code which is not
associated with a specific parameter. It applies to simple test cases, not compound test cases. It
is accessed by clicking the Testcase User Code tab in the Test Case Editor window.
> Parameter User Code is used to set a parameter value based on a dynamic expression, or to
verify a parameter value against some expression.
Order of Execution
User code is executed in the following order during test case execution:
>> Environment User Code (tags only) – Stub Entry User Code
>> Environment User Code (tags only) – Stub Exit User Code
> Environment User Code is accessed by choosing Environment => User Code => Edit.
> Stub User Code is accessed by choosing Environment => Configure Stubs => Edit.
> Test Case User Code is accessed in the Testcase User Code tab of the Test Case Editor.
> Parameter User Code is accessed by right-clicking on a parameter in the Parameter Tree and
selecting “Properties…” or double-clicking the parameter and selecting the User Code tab.
Some examples of the tag syntax follow (see the next section for information on obtaining the correct
tag syntax automatically):
<<unitname.subprogram.parameter>>
<<unitname.subprogram.array_name>>.element
<<unitname.subprogram.parameter>>[index].element
<<unitname.subprogram.parameter>>.Array[index].element
<<unitname.subprogram.parameter>>.Array[index].element
uut_prototype_stubs.
<<unitname.<<GLOBAL>>.parameter>> ...
In addition to assigning values to parameters and global objects, you can also compare values against
an expected expression. To have the result of a user code comparison show up in the test results, you
must enclose the comparison in double curly braces, {{ }}. For example, {{
<<unitname.subprogram.parameter>> == value }} would result in a comparison being
performed between the value of the parameter and value. Of course, a dynamic expression of any sort
can be substituted for value.
When specifying Expected User Code, the code between the double braces ({{...}}) must evaluate
to a boolean expression. The boolean expression will be evaluated as the test runs, and the value will
be reported in the test results listing. The result of the evaluation of the boolean expression will be
included in the pass/fail analysis of the test case. A boolean expression can be combined with other
Ada-code to accomplish a more complex comparison.
To make it easier to enter user code tags for objects, you can click the User Code Tags button in
any of the user code dialogs or on the toolbar. Alternatively, select View => Dock Control => User
Code Tags from the Menu Bar. A new window opens in non-docked mode, which displays a hierarchal
tree of objects in the environment. Using this tree, navigate to the object whose tag you want to insert.
A Find Banner is available to assist in searching and filtering the text. The Find Banner appears at the
bottom of the window and is off by default. Access the Find Banner using the Ctrl + F keyboard shortut.
For more information on using this feature, see "To Search for Text Using the Find Banner" on page 77.
In the screen shot below, the MANAGER file is a UUT in the environment, and DATABASE is stubbed.
The TABLE parameter in the subprogram GET_TABLE_RECORD is expanded. For each object in the
tree, you can:
> right-click the object and choose Copy – puts both the assignment and conditional forms into your
copy buffer. For example:
<<DATABASE.GET_TABLE_RECORD.TABLE>>:=(expression);
{{ <<DATABASE.GET_TABLE_RECORD.TABLE>> = (expression) }}
> right-click the Assignment node and choose Copy – puts the assignment form into your copy
buffer. For example:
<<DATABASE.GET_TABLE_RECORD.TABLE>>:=(expression);
> right-click the Conditional node and choose Copy – puts the conditional form into your copy
buffer. For example:
{{ <<DATABASE.GET_TABLE_RECORD.TABLE>> = (expression) }}
For Configure Stubs user code only, the following two additional insertion types are available:
> Insert Parameter param – inserts the actual parameter name. For the subprogram Get_Table_
Record, parameter Table, the text inserted looks similar to the following:
Table
> Insert Conditional for Parameterparam– inserts a VectorCAST user code conditional
expression for the parameter. For the subprogram sum, parameter R, the text inserted looks
similar to the following:
{{ Table == (expression) }}
> Insert VCAST Object for Parameterparam is the same as right-clicking Assignment for param
and choosing Copy, then Paste
> Insert VCAST Object Conditional for Parameterparam is the same as right-clicking
Conditional for param and choosing Copy, then Paste.
Choose Environment => User Code => Edit to enter or modify the environment user code.
The Environment User Code dialog enables access to User Globals, User Parameters, Environment
User Code, Unit Appendix User Code, and Unit Prefix User Code.
Expand the node for “User Code” to see the input boxes for each type of Environment User Code.
Header: Source code that normally appears before “package body” declaration. Usually consists of
“with” clauses, as well as “use” or “use type” clauses.
By default, the “with” or “use” clauses are inserted in several places: the User Code package, the
nested User Code package in a whiteboxed UUT, and the stub bodies. In some cases, it is not
desirable to put the same “with” statements in every location. You can control where these “with” and
“use” clauses are inserted by adding an Ada comment before the “with” clause: --unit_name, where
unit_name is the name of the UUT or any stub, for which you want the “with” clause to apply. The
comment –-uut may be used to specify the UUT. If you use –-user_code_adacast, then the “with” clause
is inserted only in the declarative section of the User Code package.
For example,
Data – Source code that normally occurs near the top of a source file. This could consist of type and
data declarations, as well as any support subprograms that might be needed.
Harness Init – Source code in this area gets executed once per harness execution, immediately after
elaboration has completed. This could include calling initialization routines.
Test Case Init – Source code in this area gets executed immediately after the test case’s data is
loaded, but before the UUT is called by the harness. This usually includes initializing data for a specific
test case. If your test case sets global values, the Test Case Init code has the opportunity to write over
those values.
UUT Timer Start – Source code in this area gets executed just prior to entering the call to the UUT.
Used to start timer.
UUT Timer Stop – Source code in this area gets executed just after returning from the call to the UUT.
Used to stop timer.
Stub Entry – Source code in this area gets executed upon entry to a stubbed subprogram, after
Configure Stubs Beginning User Code.
Stub Processing – Source code in this area is executed every time any stub is called.
Stub Exit – Source code in this area gets executed upon exit from a stubbed subprogram, before
Configure Stubs Ending User Code
Test Case Term – Source code in this area gets executed immediately after the UUT returns control to
the harness. This usually includes examining data affected by specific test cases.
Harness Term – Source code in this area gets executed once per harness execution, immediately
before exiting. This could include saving data to a file or other cleanup processing.
Additional Unit Bodies – Implementation portion of additional code unit, to be added to test harness.
There are two additional types of stub processing that you can add for all stubs; user code to process
upon entry, and user code to process upon exit.
To enter user code for a particular stubbed subprogram, see "To Enter Configure Stub User Code" on
page 408.
See "Environment Script Language" on page 447 for more information on the user code tags in the
environment scripting language.
An edit box opens in which you can type, copy/paste, and test compile source code.
To test compile this code, click the Test Compile button . Test compiling in one cell of
Environment User Code incorporates the source code in all other Environment User Code cells.
Therefore if you have an error in any cell, the Test Compile Errors report is displayed.
– needs saving
– Save, or Save, Compile and Link (from the drop-down menu) the unit, but does not affect the
Parameter Tree of a test case until environment is rebuilt
– pop cell out into its own window for easier editing
Compile button to compile the code before it is saved to the test harness. Or, right-click a node
and choose Test Compile. Either way, all environment user code is test compiled, not just the individual
cell.
If there are compile errors, a dialog informs you and a window explains the errors. For example,
suppose the following user code is entered in the Header section. Note the missing semicolon at the
end of the line.
When the test compile button is clicked, the MDI window shows the following:
The top pane shows the compiler error and the line in which the error occurs.
The lower pane shows the user code file with the temporary user code inserted. Use Ctrl+G (Goto Line)
to go to the line with the error.
with DATA_PKG_ADACAST;
with ADACAST_IO;
with USER_GLOBALS_VCAST;
with MANAGER;
with TYPES;
with DATABASE;
-- BEGIN USER_CODE_DEPENDENCIES_4 --
with database
-- DONE USER_CODE_DEPENDENCIES_4 --
package body USER_CODE_ADACAST is
-- BEGIN USER_CODE_OBJECTS_4 --
-- DONE USER_CODE_OBJECTS_4 --
To close the Test Compile Errors window, use the in the upper right corner or File => Close.
To save all sections of Environment User Code, click the Save button on the Toolbar , right-click a
node and choose Save, or choose File => Save. The following dialog appears.
If you are ready to compile the user code and link it into the test harness, leave the radio button on Link
and click OK.
If you plan to rebuild the environment anyway, put the radio button on Do Nothing, and click OK or
Cancel.
If you attempt to close the Environment User Code window (using the window control) and there are
any unsaved sections, the same dialog appears, but this time indicating that a Save needs to be
performed.
Choosing the Save and Link radio button saves, compiles and links the user code into the test harness
before closing the window. Choosing the Save radio button saves your changes before closing the
window. Choose Environment => User Code => Compile and Link later. Choosing the Link radio
button when some cells have been modified discards the changes and compiles and links the user
code. Clicking the Do Nothing radio button discards your changes and closes the Environment User
Code window.
User Globals
User Globals is a collection of global objects for use by the User Code functionality and for manipulation
of objects of SYSTEM.ADDRESS type. You can also use this area to define temporary data objects.
These objects can also be accessed in the Create New Environment wizard, User Code page, and are
visible in the Parameter Tree of a test case.
Choose Environment => User Code => Edit to enter or modify the user globals. If you want to see
the change in the Parameter Tree of a test case, you will need to rebuild the environment. Otherwise,
compiling and linking the change into the test harness may suffice.
The Environment User Code dialog enables access to User Globals, User Parameters, Environment
User Code, Unit Appendix User Code and Unit Prefix User Code.
Expand the node for “User Globals” to see the edit box of default user globals. The User Globals section
provides a mechanism for user-defined types and objects to be included into the test harness. By
default, the file has five integer objects, five floating point objects, five string objects, and an array of
200 integer elements. All data objects that are defined in the User Globals file when the environment is
built can be manipulated as test data when building a test case.
You can add to these objects at the time you create the environment using the User Code page in the
Create New Environment wizard.
The User Globals file is accessible in the Create New Environment dialog, the Environment User Code
dialog, and the USER_GLOBALS_VCAST section of a test case’s Parameter Tree. The following list
explains how to move between the three:
> Any User Globals added in the Create New Environment wizard, User Code page, are seen in the
Environment User Code dialog, User Globals section and in the Parameter Tree of a test case.
> Adding any User Globals in the Environment User Code dialog, User Globals section, requires
you to rebuild the environment before they are visible in the Parameter Tree of a test case. To see
them in the Update Environment wizard, choose Environment => Update Environment.
> The default version of the User Globals is delivered in source code format in the VectorCAST
Installation Directory, subdirectory DATA, filename GLOBALS.ADA. If you modify the default file,
the changes are reflected in the User Code page in the Create New Environment wizard for each
new environment.
An edit box opens in which you can type, copy/paste, and test compile source code.
To test compile this code, click the Test Compile button . Test compiling in one cell of
Environment User Code incorporates the source code in all other Environment User Code cells.
Therefore if you have an error in any cell, the Test Compile Errors report is displayed.
– needs saving
– Save, or Save, Compile and Link (from the drop-down menu) the unit, but does not affect the
Parameter Tree of a test case until environment is rebuilt
– pop cell out into its own window for easier editing
To save the User Globals, click the Save button on the toolbar , right-click the User Globals node
and choose Save, or choose File => Save. The following dialog appears.
To see the changes to the User Globals in the Parameter Tree of a test case, you’ll need to rebuild the
environment. Put the radio button on Do Nothing, and click OK. Choose Environment => Rebuild
later. If you have other Environment User Code changes, you probably want to use Link.
If you only need access to the new User Globals (without seeing them in the Parameter Tree), leave the
radio button on Link and click OK.
To cancel the save operation without saving, click the Cancel button.
If you attempt to close the Environment User Code window (using the window control) and there are
any unsaved sections, the same dialog appears, but this time indicating that a Save needs to be
performed.
Choosing the Save and Link radio button saves, compiles, and links the all environment user code into
the test harness before closing the window. Choosing the Save radio button only saves your changes
before closing the window; it does not compile and link. Choose Environment => User Code =>
Compile and Link later. Choosing the Link radio button when some cells have been modified
discards their changes and compiles and links the user code. Clicking the Do Nothing radio button
discards your changes and closes the Environment User Code window.
would like to know how long it takes for the harness to execute, as well as how long each test case
takes to execute. The following code would be added to each area of the Environment User Code
dialog:
Header
with CALENDAR; use type CALENDAR.TIME;
with TEXT_IO;
with USER_GLOBALS_VCAST; use USER_GLOBALS_VCAST;
Data
package DURATION_IO is new TEXT_IO.FIXED_IO(CALENDAR.DAY_DURATION);
HARNESS_START : CALENDAR.TIME;
TESTCASE_START : CALENDAR.TIME;
Harness Init
HARNESS_START := CALENDAR.CLOCK;
Harness Term
TEXT_IO.PUT ( “Time of harness: “ );
DURATION_IO.PUT ( CALENDAR.CLOCK - HARNESS_START );
TEXT_IO.NEW_LINE;
In this case it is may be desirable for one person on the project to write a database interface that will
access the data and to include this interface into all VectorCAST test harnesses, via the user code
package. Consider this following example:
package TARGET_DATABASE is
type TARGET_RECORD is
record
ID : integer;
X_POSITION : YARDS_TYPE;
Y_POSITION : YARDS_TYPE;
COURSE : DEGREES_TYPE;
SPEED : KNOTS_TYPE;
end record;
TARGETS : TARGET_ARRAY;
end TARGET_DATABASE;
In this case you could set up a portion of the target database using the VectorCAST test case utility,
however it is unlikely that you would set up all 1000 items in the target array this way. To read the target
data from a database you could write some code that could read the target data from a database and
populate the TARGETS data object. We will not expand all of this code here except to show how it
relates to the VectorCAST user code capability.
The following code would be added to two areas of the Environment User Code dialog:
Header
with DATABASE_IF;
with TARGET_DATABASE;
Harness Init
for I in TARGET_DATABASE.TARGETS'range loop
TARGET_DATABASE.TARGETS (I) :=
DATABASE_IF.READ_TARGET_ITEM (I);
end loop;
User Parameters
User Parameters enable the tester to specify an object defined elsewhere in the harness and use it in
place of a VectorCAST-generated parameter. The User Params section of the Environment User Code
editor is where you indicate the association between the two objects. The syntax is:
<<unit.subprogram.parameter>> user_parameter
where unit, subprogram, and parameter are all VectorCAST-generated. For example,
<<uut.Create_For_Write.File>> User_Globals_Vcast.My_File
<<uut.Write_Line.File>> User_Globals_Vcast.My_File
<<uut.Close.File>> User_Globals_Vcast.My_File
1. From the Menu Bar, select Environment => User Code => Edit.
2. Expand the node User Params.
3. Double-click in the yellow cell to open the edit box.
In our example, we will cut and paste the example code above into the edit box:
In addition, (for this example) you must declare your parameter in the User Globals section and then
rebuild the environment.
As a result, the User Parameter MY_FILE is available in the USER_GLOBALS_VCAST section of the
test cases in the environment. When you set an input or expected value for MY_FILE, you are actually
setting it for the “File” parameter for the Create_For_Write, Write_Line, and Close functions.
In this example, if the package specification for the Unit Under Test looked like:
with ADA.TEXT_IO;
package UUT is
procedure CREATE_FOR_WRITE ( FILE : in out ADA.TEXT_IO.FILE_TYPE; FILENAME : in
string );
procedure WRITE_LINE ( FILE : in ADA.TEXT_IO.FILE_TYPE; LINE : in string );
procedure CLOSE ( FILE : in out ADA.TEXT_IO.FILE_TYPE );
end UUT;
ENVIRO.NEW
ENVIRO.NAME:ENV
ENVIRO.COMPILER:GNAT
ENVIRO.PARENT_LIB:.
ENVIRO.UUT:UUT
ENVIRO.USER_PARAMETERS:
<<UUT.CREATE_FOR_WRITE.FILE>> user_globals_vcast.my_file
<<UUT.WRITE_LINE.FILE>> user_globals_vcast.my_file
<<UUT.CLOSE.FILE>> user_globals_vcast.my_file
ENVIRO.END_USER_PARAMETERS:
ENVIRO.USER_GLOBALS:
with Ada.Text_Io;
package USER_GLOBALS_vCAST is
My_FILE : Ada.Text_Io.File_Type;
end USER_GLOBALS_vCAST;
ENVIRO.END_USER_GLOBALS:
ENVIRO.END
In this scenario, all three subprograms would use the my_file object as a file pointer.
Use the User Code Tags window to easily insert the symbol for a parameter, using the correct notation.
To use this feature in either the Input or Expected edit box, click the User Code Tags button to
open the window.
Once the Test Compile has completed, a message window is displayed at the bottom of the dialog. You
may need to re-size the dialog or scroll the window to see this message window. You can move the
splitter between sections of the window by dragging it up or down. If the test compile was successful,
the message “User Code Compiled Successfully” is displayed. If there are errors, the compiler
diagnostic messages are displayed.
To delete all the text in the Input User Code section and start over, click the Clear Input button. To
delete all the text in the Expected User Code section and start over, click the Clear Expected button.
The purpose of this command is to allow you to reset the user code unit in the test harness to its default
(empty) state, and then selectively add the user code back in as you run tests. This process may be
useful in cases where you experience unexpected results because of conflicts in your user code.
When you choose Test => User Code => Clear All, the following dialog box appears:
If you wish to remove all the user code from the harness, click Yes.
If you wish to add all the user code to the harness, click Yes.
See also "Setting Input and Expected Values" for details on the notation for parameter object names.
User code may consist of any number of lines of source code. For example, you may set an array
parameter value using the following user code:
A boolean expression that is inside the {{ and }} markers is translated into a test expression that will
output its results to the test results report. Variables are expanded as in input user code, and any user
not inside {{ and }} markers is executed as (non-conditional) normal code.
For example, you could check the value of an array parameter using the following user code:
This code will test all 200 values of the array “a”.
The package specification for the unit under test in this example might look like:
package DATA_FILE_OPERATIONS is
type FILE_TYPE is private;
procedure OPEN_DATA_FILE (
NAME : in string;
FILE : in out FILE_TYPE);
procedure WRITE_DATA_TO_FILE (
FILE : in out FILE_TYPE;
DATA : in integer);
procedure CLOSE_DATA_FILE ( FILE : in out FILE_TYPE );
function IS_OPEN ( FILE : FILE_TYPE ) return boolean;
private
type FILE_TYPE is new integer;
end DATA_FILE_OPERATIONS;
The package body for the unit under test in this example might look like this:
with TEXT_IO;
package body DATA_FILE_OPERATIONS is
FILES : array ( FILE_TYPE range 1 .. 10 ) of TEXT_IO.FILE_TYPE;
CURRENT : FILE_TYPE := 0;
procedure OPEN_DATA_FILE (
NAME : in string;
FILE : in out FILE_TYPE) is
begin
CURRENT := CURRENT + 1;
FILE := CURRENT;
TEXT_IO.CREATE ( FILES(FILE), TEXT_IO.OUT_FILE, NAME );
end OPEN_DATA_FILE;
procedure WRITE_DATA_TO_FILE (
FILE : in out FILE_TYPE;
DATA : in integer) is
begin
TEXT_IO.PUT_LINE ( FILES(FILE), integer'image ( DATA ) );
end WRITE_DATA_TO_FILE;
procedure CLOSE_DATA_FILE ( FILE : in out FILE_TYPE ) is
begin
TEXT_IO.CLOSE ( FILES(FILE) );
end CLOSE_DATA_FILE;
function IS_OPEN ( FILE : FILE_TYPE ) return boolean is
begin
return TEXT_IO.IS_OPEN ( FILES(FILE) );
end IS_OPEN;
end DATA_FILE_OPERATIONS;
You would create a test case for OPEN_DATA_FILE, which, when called as the first slot in a
Compound test case, would create a file with the specified name, which is one of the parameters.
Next, create a test case for WRITE_DATA_TO_FILE. For the File parameter, we want to pass the file
pointer from OPEN_DATA_FILE. This can be done by double-clicking on the File parameter, then using
Input User Code as shown:
<<DATA_FILE_OPERATIONS.WRITE_DATA_TO_FILE.FILE>> := @
( <<DATA_FILE_OPERATIONS.OPEN_DATA_FILE.FILE>> );
The Parameter User Code for a CLOSE_DATA_FILE test case is similar. We want to pass the same
The test cases for IS_OPEN are simple; pass the file pointer (again), and set the Expected value to
True for one test case, and False for the other.
When the Compound test case is executed, the Execution Results shows that the file was open after
the WRITE_DATA_TO_FILE subprogram returned, and was not open after the CLOSE_DATA_FILE
subprogram returned.
> Parameter User Code – To enter expected data or input values for a stubbed subprogram, use
the Parameter Tree as you would for a UUT. The user code entered here us implemented via two
procedure calls (one at the start of the stub, which is expected user code, and one at the end of the
stub, which is input user code). Parameter User Code is entered in the User Code tab of the
Parameter Dialog. User Code that is entered in this way will only be executed when the stubbed
function is called. An application of this type of user code would be to set the value of a global
variable when a particular stubbed subprogram is called, or to set the value returned from a stub to
a dynamic expression (e.g. foo()).
> Environment User Code – To enter user code for all stubbed subprograms, use Environment
User Code, Stub Processing. The Stub Processing section of the Environment => User Code
=> Edit dialog is implemented in the same was as Parameter and Test Case User Code, but it is
executed whenever any stubbed subprogram in the environment is called. For example, if you
added stub processing user code that incremented a counter, at the end of each test execution
you would have the total number of stubbed subprograms that were called during the test.
> To add user code for all stubbed subprograms to be processed upon entry to or exit from the stub,
use the Stub Entry and Stub Exit section of the Environment => User Code => Edit dialog.
> Configure Stubs – Configure Stubs user code enables you to insert code directly into the
definition of the stubbed subprogram. As a result, you have direct access to the named
parameters of the function. An application of this type of user code is to cause a function to return
a value based on the value of the input parameters.
There are two editable fields on the Configure Stubs dialog. The first is marked “Beginning of Stub”, the
second is marked “End of Stub.” Any code that is put into “Beginning of Stub” is placed before any other
code in the stubbed function, and is the first code executed in the stub.
Any code that is put into “End of Stub” will be placed after all other code in the stubbed function
(immediately preceding the return() statement).
An edit box opens in which you can type, copy/paste, and test compile source code.
– needs saving
– Save, or Save, Compile and Link (from the drop-down menu) the unit, but does not affect the
Parameter Tree of a test case until environment is rebuilt
– pop cell out into its own window for easier editing
Compile button to compile the code before it is saved to the test harness. Or, right-click a node
and choose Test Compile. Either way, both the Beginning of Stub and End of Stub user code is test
compiled, not just the individual cell.
You can also right-click a node higher in the hierarchy, such as the unit or the top level, Stubbed
Subprograms. In this case, choosing Test Compile performs a test compilation on all of the cells in the
scope of the right-click. The light-yellow shading of the cells indicates the current scope of a right-click.
For example, in the following screen shot, the user has right-clicked on the unit manager.c. Therefore,
the test compile is performed over both subprograms in the unit, and both the Beginning of Stub and End
of Stub columns. As a result, the bad code in the second subprogram will cause a compile error.
If there are errors in your user code, clicking the Test Compile button opens a Stub Test Compile Errors
window. The top pane shows the compile error; the lower pane shows the stub file. In the example
below, the code “this will cause a compile error” is present. By using Ctrl+G in the lower pane you can
easily go to the line number where the error occurred. To correct the error, close the Stub Test Compile
Errors window by clicking the X in the upper right corner, edit the user code, and click the Test Compile
button again until it compiles successfully.
To save all sections of Configure Stubs User Code, click the Save button on the Toolbar , right-
click a node and choose Save, or choose File => Save. The following dialog appears.
If you are ready to compile the user code and link it into the test harness, leave the radio button on Link
and click OK.
If you plan to rebuild the environment anyway, put the radio button on Do Nothing, and click OK.
To cancel the save operation without saving, click the Cancel button.
If you attempt to close the Stub User Code window (using the window control) and there are any
unsaved sections, the same dialog appears, but this time indicating that a Save needs to be performed.
Choosing the Save and Link radio button saves, compiles and links the user code into the test harness
before closing the window. Choosing the Save radio button saves your changes before closing the
window. Choose Environment => Configure Stubs => Compile and Link later. Choosing the Link
radio button when some cells have been modified discards the changes and compiles and links the user
code. Clicking the Do Nothing radio button discards your changes and closes the Stub User Code
window.
Compile and link Configure Stub User Code into test harness.
VectorCAST will automatically re-compile the user code file after the user code has been removed.
uut.adb
with STUB;
package body UUT is
function FUNC return integer is
RET : integer := 0;
begin
for I in 1 .. 10 loop
RET := RET + STUB.SUM ( I, I );
end loop;
return RET;
end FUNC;
end UUT;
uut.ads
package UUT is
function FUNC return integer;
end UUT;
stub.ads
package STUB is
function SUM ( R : integer;
L : integer )
return integer;
end STUB;
3. Choose Environment => Configure Stubs => Edit. Double-click in the End of Stub cell to open
it for editing.
4. Click the User Code Tags button . Click the arrow to see the sub-menu.
<<STUB.SUM.return>>
<<STUB.SUM.return>> := R + L:
Note: Rather than typing “R” and “L,” you can have VectorCAST insert the parameter
names automatically. To do this, place your cursor ":=", click the arrow in the User Code
Tags button, and choose R => Insert Parameter R.
7. Click the Test Compile button . If you have any compile errors, fix them.
8. Once the user code compiles successfully, click the Save button on the Toolbar .
9. In the message that appears, Click the Save and Link button.
10. Execute the test case FUNC.001. At the end of the execution results file, you see that the stub
user code summed the inputs R and L each time it was called, and the returned value was 110,
which is what was expected.
Tip: To see the returned value after each event, choose Tools Options => Execute
Tab. Click the checkbox Expand Report Parameters and execute the test case again.
For assistance in migrating any RGW databases to the new RGW 3 data model (using .json files),
contact Vector Tech Support at support@vector.com or visit https://support.vector.com/.
VectorCAST Requirements Gateway (RGW) provides traceability between software requirements and
test cases and allows the import and mapping of requirements to test cases. Requirements Gateway
provides a way to:
> Import testing requirements from your Requirements Management Tool (RMT) to a VectorCAST
RGW repository
> Assign specific requirements to testcases in a unit test environment
> Export the resulting pass or fail status back to the RMT tool
In RGW, the requirements database is implemented using JSON, allowing the user to commit the
repository to Source Control Management (SCM).
In addition to CSV files, VectorCAST RGW supports the following Requirements Management
Systems:
> codebeamer
> Jama
> Polarion®
> IBM CLM
> Visure
> IBM® Rational® DOORS®
The location of the repository can be assigned to a directory by selecting Tools => Requirements
Gateway => Options... from the Menu Bar.
The RGW credentials file can store the username and password for a specific user, to save time when
logging in to the RMT. The location of the credentials file can be assigned to a .json file by selecting
Tools => Requirements Gateway => Options... from the Menu Bar.
Use the Browse button to navigate to either the location of the RGW repository or the location of the
credentials file and then select the OK button. The paths to these values are typically specified as a full
path, but can be a relative path.
A repository location can also be specified by selecting Tools => Requirements Gateway =>
Options... from the Menu Bar. In the RGW Repository Location field, enter the path to the repository
and select the OK button.
> Select the drop-down menu next to the Open Requirements Gateway on the Toolbar and
select the name of the gateway.
> Select Tools => Requirements Gateway => <gateway name> from the Menu Bar.
Configure the system to identify the data attributes to be imported. Available attributes vary according
to the gateway selected. See "Working With Supported Gateways" on page 427 to identify the
attributes required for your gateway. Our example below is based on a CSV gateway.
Note that the imported requirement data may be filtered by selecting the Specify what requirement
data to import checkbox. When this option is selected, only objects which have an attribute with a
specified Attribute name and Attribute value are imported.
Select the Import button to import the requirement data from the requirements subsystem. In our
example, the option to specify what data to import was not selected, and all four attributes were
imported.
Available export settings vary according to the gateway selected. See "Working With Supported
Gateways" on page 427 to identify the settings available for your gateway.
Our example shows export settings for a CSV gateway. The following test data can be exported:
Use the browser dialog to enter the path to the export file.
Select the Export button. The test data is appended to the export file.
Single requirements, multiple requirements and entire requirement groups can be deleted from the
repository. To delete a single requirement, select the requirement and select Delete from the right-click
menu. To delete multiple requirements from the repository, use ctrl-click or shift-click to select the
requirements and select Delete from the right-click menu. An entire group of requirements can be
deleted by selecting the group node and selecting Delete from the right-click menu.
Once edits are complete, select the Save button to save changes and reload the Python file. If
VectorCAST is unable to reload the file after editing, error messages are displayed in the message box
below the script.
Note: Best practice is to not edit this file directly. The file may be overwritten by future
VectorCAST installations. Instead, make a copy of the file and save it in the $VECTORCAST_
DIR/python/vcast/requirements directory. Modify the file as required. Your custom file
can then be specified in the Requirements Gateway options.
To link a requirement to a test case, double-click on the requirement in the Project Requirements
pane. The requirement will be added to the list in the Test Case Requirements pane. In our example,
we have linked the requirement FR27 Adding free dessert to test case PLACE_ORDER.001.
To remove a requirement from a test case, double-click on the requirement in the Test Case
Requirements pane.
Once requirements have been associated with a test case, run the test case to generate a result. The
result is then linked to that requirement in the repository.
Requirements that are associated with a test case are printed in the Test Case Management Report,
which can be accessed by selecting Test => View => Test Case Management Report from the Menu
Bar.
The work flow for tracking the impact of requirement changes on test cases consists of the following:
1. Configure the Requirements Gateway and import your requirements for the first time.
3. Make a change to some requirements in your chosen Requirement Management Tool (RMT).
If VectorCAST detects that a requirement has been updated and it is linked to a test case in a
VectorCAST unit test environment, then the requirement is considered "impacted". Any test cases
linked to impacted requirements are then marked as needing review.
In the GUI, test cases linked to impacted requirements that need review are displayed in the Test Case
Tree with a yellow "!" icon . The test's tooltip also indicates how many requirements need review.
Note that the user is then prevented from changing the list of linked requirements and from exporting
test case pass / fail data back to the RMT.
5. Verify that the test case still satisfies the test's associated requirement(s), perhaps modifying Input
or Expected Values and executing as needed.
When such a test case is opened in the Test Case Editor, the impacted requirements are displayed as
having a "Needs Review" status, and a Mark All As Reviewed button is enabled.
Clicking the Mark All As Reviewed button clears the "Needs Review" status of the test case, and
marks all impacted requirements as "Done" in that test case.
This action then frees up the restrictions on the test case, allowing the user to once again change the
list of linked requirements and export the pass / fail status back to the RMT.
In the GUI, test cases linked to impacted requirements that have been reviewed are displayed in the
Test Case Tree with a green check mark icon icon .
The Mark As Reviewed CLICAST command marks a specified test case as having been reviewed and
approved, and clears the "Marked for Review" flag.
VectorCAST provides two scripts allowing the user to remove historical data and ensure that the latest
set of requirements or test case data is protected.
> rgw_clean.py - This script allows the user to clean out historical RGW and test execution data
from an RGW Repository database.
> rgw_report.py - This script allows the user to query the Repository database and produce .xml
reports of historical RGW and test execution data. This report is useful for examining historical
data prior to removal.
Note: The rgw_clean script only clears historical data. The most recent set of requirements
and test execution data is protected and will never be removed.
See RGW Utility Scripts for details on the command line syntax.
> codebeamer
> IBM CLM
> IBM® Rational® DOORS®
> Jama
> Polarion®
> Visure
Instructions for these integrations are published as Application Notes and are available on
www.vector.com.
The following sections discuss in detail how to set up and use the Requirements Gateway with the
CSV Gateway.
The Authentication tab allows the user to specify configuration options. Configuration options vary
according to the gateway selected. Use the Gateway Profile drop down menu to select the CSV
gateway.
The CSV file path, as shown in the example below, is configurable for a CSV subsystem.
Click the Get Fields button to display the column names of the .csv fields.
Import Requirements
Now that all the required system settings have been configured, import the requirements from the CSV
Requirements gateway. Select the Import tab at the bottom of the Requirements Gateway dialog.
Note that the imported requirement data may be filtered by selecting the Specify what requirement
data to import checkbox. When this option is selected, only objects which have an attribute with a
specified Attribute name and Attribute value are imported. For our example, we will not use the filter
and will import all four attributes.
Select the Import button to import the requirement data from the requirements subsystem.
Single requirements, multiple requirements and entire requirement groups can be deleted from the
repository. Select the requirement(s) or requirements group node to be deleted and right-click and select
Delete.
To link a requirement to a test case, double-click on the requirement in the Project Requirements
pane. The requirement will be added to the list in the Test Case Requirements pane. In our example,
we have linked the requirement FR27 Adding free dessert to test case PLACE_ORDER.001.
To remove a requirement from a test case, double-click on the requirement in the Test Case
Requirements pane.
Once requirements have been associated with a test case, run the test case to generate a result. The
result is then linked to that requirement in the repository.
Requirements that are associated with a test case are printed in the Test Case Management Report,
which can be accessed by selecting Test => View => Test Case Management Report from the Menu
Bar.
First, configure the export settings by clicking the checkbox next to the test data to export. The
Use the browser dialog to enter the path to the export file.
Select the Export button. The test data is appended to the export file.
This work flow uses the VectorCAST example environment Tutorial for Ada. Before starting the work
flow example, you can automatically build the environment by selecting Help => Example
Environments => Ada => Tutorial for Ada from the Menu Bar. Be sure that the TUTORIAL_ADA
environment is closed before beginning the work flow. Close the environment by selecting File =>
Close Environment from the Menu Bar.
The .csv file used in our example is located in the VectorCAST installation directory: $VECTORCAST_
DIR/Examples/RequirementsGW/CSV_Requirements_For_Tutorial.csv.
$ %VECTORCAST_DIR%\mkdir REPO3
Note that for the purposes of our example, the path to the new repository is located in the working
directory. In actual practice, the repository is normally located in a different directory.
**Version 21 (%H%)
Processing options file C:\VCAST\examples\environments\tutorial_
ada\ADACAST_.CFG
Key 'title_attribute' set to 'Title' for Subsystem Profile 'csv'.
'FR11'.
ada\ADACAST_.CFG
Current Gateway was set to 'CSV'.
This message indicates that an error was made in the installation of VectorCAST. Check all
environment variables, symbols and path links that are required.
License Messages
FLEXlm: Licensed number of users already reached
This message indicates that an attempt was made to use more VectorCAST licenses than are
authorized.
This message indicates that license processing could not be completed. Check all environment
variables, symbols and path links that are required.
This message indicates that some license anomaly was detected subsequent to the initial invocation of
VectorCAST and will not allow VectorCAST to continue. All data will be saved and VectorCAST will be
automatically terminated. In general, a FLEXlm error number is displayed. The meaning of the error
code is available in the FLEXlm End User’s Guide: VectorCAST installation directory => DOCS =>
fnp_LicAdmin.pdf.
This message indicates that the source for a dependent of the UUT could not be found. The
environment construction will continue but you will not have the option of creating a stub for this unit.
Also, you will not be able to modify any data values which rely on types from this dependent unit.
This message indicates that there is an exception being raised during the elaboration processing of the
unit under test. This is most often caused by function calls that are made by the code under test during
program elaboration.
(Apex only) This message is displayed before VectorCAST calls the Apex copy_view command to
create a new view in the current sub-system for test purposes.
(Apex only) This message is displayed performs a private check-out of the source files that exist in the
test view.
These message are displayed during the environment construction when VectorCAST ios performing
the parsing and conversion of the unit under test that is required for whitebox testing.
This message indicates that the parsing of the Unit Under Test is being accomplished by the
VectorCAST environment builder.
This message indicates that the indicated unit is being parsed and added to the current environment.
This message will be displayed if the unit under test is a generic instantiation and you choose the
whitebox test option.
This message will be displayed if the unit under test is a private package and you DO NOT choose the
whitebox test option.
Building Driver
This message indicates that the Ada language driver program is being built by the VectorCAST
environment builder and generator.
This message is displayed when VectorCAST is building the data handling functions for a particular
unit.
This message indicates that elements of the environment have been created and they are now being
compiled into the Test Ada Library by VectorCAST.
Linking Environment
This message indicates that all elements of the environment have been created and they are now being
linked into the Test Ada Library by VectorCAST.
This message indicates that all elements of the VectorCAST environment have been successfully
constructed.
This message indicates that the source file for a Unit Under Test spec or body could not be found. This
error is fatal for the unit under test and informational for dependent units due to the fact that the UUT
source is required to build a test harness.
This message indicates that the source for a dependent of a UUT could not be found. The environment
construction will continue but you will not have the option of creating a stub for this unit. Also, you will
not be able to modify any data values which rely on types from this dependent unit.
This message indicates that there is an Ada construct not recognized by the VectorCAST parser. The
next screen output by VectorCAST will be a scrollable buffer window containing parser diagnostic
message.
This message indicates that there is some construct not being parsed correctly by VectorCAST. This
error is fatal, no recovery is possible. After the error message is cleared, a scrollable buffer window will
be presented with amplifying information about the parser error. This information will also be written to a
file. The file will be stored in the default directory from which VectorCAST was invoked. The filename
will be "VEIPARSE.ERR". Print this file and submit the listing with a description of the error to
Customer Support support@vector.com.
This message indicates that when the operator chose to create a stub for a dependent unit the creation
failed. This message results from an unexpected error while the VectorCAST code generator executed.
This error is not fatal; environment creation will continue.
This message is displayed to indicate that the compilation of a particular harness source file has failed.
The compiler diagnostic message will be displayed to help resolve the issue.
This message indicates that during the compilation of the VectorCAST Test Harness components, a
compile error occurred. The compiled error listing will be displayed in the VectorCAST display and will
also be written to the file ACOMPILE.LIS.
This message is displayed when VectorCAST encounters an unexpected error during the compilation or
link of the test harness. For example this message may reflect the inability to create a file because of
disk space or permission problems.
This message indicates that some sort of operating system or protection error occurred when
VectorCAST attempted to create the disk directory that will contain the environment files
This message indicates that you have attempted to create a VectorCAST environment in a situation
where a directory already exists with the same name. The directory may or may not be a VectorCAST
environment.
This message indicates that the UUT is a generic package or subprogram. The environment creation
process will be abandoned. You cannot test a generic unit directly, you must use a stand-alone
instantiation to test a generic. Instantiate this unit and choose the instantiation as the VectorCAST unit
under test.
These messages indicate that an executable program created by VectorCAST did not link properly. The
linker diagnostic listing will be displayed and will be written to the file AALINKER.LIS.
These messages indicates that VectorCAST cannot execute one of the executables that it builds.
Probable Cause: a previous error during environment creation, an operating system protection error on
program execution, or "." is not on the user's path.
This message indicates that the VectorCAST environment builder was not able to complete its
processing normally. Nothing is usable in the partially created environment, and it will be deleted.
This message indicates that a serious error occurred during environment creation. File protection or limit
conditions may exists.
This message indicates that there are unexpected files in the environment subdirectory or for some
other reason the files in the environment subdirectory could not be deleted.
This message indicates that the environment selected either does not exist or has been corrupted.
This message indicates that the environment creation is complete, however there was a compile failure
during creation that must be resolved before continuing.
This message indicates that the environment creation is complete, however, there was a link failure
during creation that must be resolved before continuing.
This message indicates that there is an exception being raised during the elaboration processing of the
unit under test. This is most often caused by function calls that are made by the code under test during
program elaboration. The following Ada source fragment illustrates this point:
package unit_under_test is
object : integer := get_object_value;
This message is displayed if the environment creation aborts for any reason. VectorCAST will
automatically delete all files that were created during the aborted environment build.
This message indicated that the execution of the test harness did not produce the expected output files
for VectorCAST to produce the test reports. Common causes are a fatal error in the initialization of the
test harness, or problems with the configuration of the target if you are using VectorCAST/RSP. To
determine the cause, run the test case using the VectorCAST option: “Execute with debug” and step
through the code under test.
This message is displayed during the creation of a test case, this error message is displayed to notify
the operator that the entered change was not affected.
This message indicates that an operator entered parameter value entered during test case generation is
not consistent with the numeric value required by the context.
This message indicates that the displayed type mark is of a type that could not be found in the
environment.
This message indicates that a particular test data item is outside the allowable range for that type, and
that no further processing will be performed.
This indicates that the a field in a variant structure has been selected for modification, but that the field
cannot be changed because the current value of the discriminant makes the field inactive.
This message indicates that an operator entered array index value is invalid for the array specified.
This indicates that the a field in a variant structure has been selected for modification, but that the field
cannot be changed with in a VectorCAST test case because the type definition did not provide a default
value for the discriminant.
This message indicates that the selected test case name contains illegal characters or space
characters.
This message indicates that you have selected an operation dependent on the expected results file.
However, no expected results exist for this test case.
This message indicates that you have selected an operation dependent on the actual results file.
However, no expected results exist for this test case.
This message is displayed when a type is encountered that is not supported by VectorCAST. See
"Limitations and Restrictions" on page 472 for more information.
This message indicates that the ASCII data is being created which will allow viewing and printing of
test cases.
General Messages
Building Environment Script file
These message are displayed when VectorCAST is building the three script files required to for
regression testing.
This message is displayed when VectorCAST is done building a test case script or an environment
script.
This message is displayed while VectorCAST is creating an test case script template
This message is displayed when you initialize coverage, or re-link the test environment and the
instrumented test harness is being linked.
This message is displayed when VectorCAST is building its database of range data for all of the various
parameters, global objects and data types that are part of the test environment.
These message report on the progress of computing the correct data ranges for global objects and
parameters. This processing is the processing that enables VectorCAST to display the specific range
of legal values for all data types used in the test environment.
This message is displayed when VectorCAST is performing “make” processing to make the test ada
library up-to-date when there are obsolete units. This processing is only required in the case of whitebox
testing.
(Target Only) These message are displayed while VectorCAST is connecting to the target and
downloading the test harness executable.
This message indicates that the entered environment name contains a space or some other character
that is an illegal directory name on the host platform.
This message indicates that the environment chosen for deletion has been completed.
This message is displayed when there is an error opening an environment. Possible reasons are disk
space and file protection problems.
This message is displayed when VectorCAST is performing the basis path computation for a particular
unit.
This message is displayed when VectorCAST cannot compute the basis path listings for a module.
Errors of this type should be reported to VectorCAST Technical Support (support@vector.com)
This message is displayed when VectorCAST cannot perform the coverage instrumentation processing
for a module. Errors of this type should be reported to VectorCAST Technical Support
(support@vector.com)
This message is displayed when the user selects the “re-link” command and coverage is enabled.
VectorCAST will automatically retrieve the latest version of the code under test, and re-instrument the
code.
This message is displayed when the compilation of an instrumented source module fails. Errors of this
type should be reported to VectorCAST Technical Support (support@vector.com).
This message is displayed while VectorCAST is generating the Min, Mid or Max test cases.
This message is displayed when an environment has been opened in VectorCAST and there are
unresolved compile errors. This message indicates that no testing can be performed until the compile
errors are resolved.
This message is displayed when an environment has been opened in VectorCAST and there are
unresolved link errors. This message indicates that no testing can be performed until the link errors are
resolved.
This message is displayed when an environment has been opened in VectorCAST and there are
unresolved run-time errors. This message indicates that no testing can be performed until the run-time
errors are resolved.
This message is displayed after a test case execution when VectorCAST is updating the coverage data
to reflect the test that was just run.
This message indicates that there was some kind of unexpected error during the generation of the
parameter and global data range information. As a result, testing may be performed, but the generation
of Min, Mid, Max test cases is unavailable.
These message are displayed to provide an update to the user when large coverage data files are being
processed . The line numbers will update in 1000’s as the file is processed.
This message indicates a serious error occurred during the test execution and the expected output data
did not get produced by the test harness. With Target testing this can indicate that the target
communication failed. With host testing this can indicate a file lock, disk space, or permission problem.
CLICAST Messages
Environment must be specified on command line
These messages are displayed when you issue a clicast command that requires the name of an
environment, and you do not provide an environment name or provide an invalid name as a command
option.
This message is displayed when you issue a clicast command that requires the name of a subprogram,
and you do not provide a subprogram name or you provide an invalid name as a command option.
This message is displayed when you issue a clicast command that requires the name of a test case,
and you do not provide a test case name or you provide an invalid name as a command option.
No files specified
This message is displayed when a clicast command requires an output filename and no filename is
provided.
This message is displayed when an unknown compiler name is provided for a clicast command
This message is displayed when you attempt to invoke clicast with a combination of parameters and
options that VectorCAST cannot understand.
This message is displayed when the execution of a clicast command fails and there is no additional
information available.
The Environment Scripting language corresponds to the data required to construct an Environment
using the GUI. Each command is entered on a single line. The script commands are not case sensitive,
though the data may be.
> All commands start with "ENVIRO." to indicate that an Environment command follows.
> All commands that require data must use a colon (:) to indicate that start of the associated data.
> Comments may be inserted, and are indicated by inserting the VectorCAST comment delimiter "--
" as the first item on the line.
The following is a list of the valid script commands with a description of each command.
ENVIRO.NEW
Required. This command indicates that a new environment is to be created. ENVIRO.NEW must be
the first non-comment line in the environment script.
ENVIRO.NAME:
Required. This command is used to provide a name for the Environment. Unlike test case scripts, the
"name" command is mandatory. Since the Environment name will ultimately be the name of a sub-
directory, it must be unique within its parent directory and not contain spaces.
ENVIRO.UUT:
This command indicates a Unit Under Test filename. If you have more than one UUT, list each unit
name separately, one per line, each with its own ENVIRO.UUT command. If a unit is listed twice, in
two different commands (i.e. ENVIRO.UUT and ENVIRO.STUB), then the first time the unit’s name is
encountered takes precedence over other appearances.
The environment script must include at least one ENVIRO.UUT line or ENVIRO.STUB_BY_
FUNCTION line.
ENVIRO.COMPILER:
This command tells VectorCAST which compiler to use when building the Environment. The supported
compilers and corresponding script names are as follows:
Unit-specific coverage enable flag. Supports the specification of coverage on non-stubbed dependent
units in test environments. Custom coverage on these units can be applied during environment build
and retained during environment rebuild.
ENVIRO.PARENT_LIB:
Required. This command is used to tell VectorCAST the name of the parent library where the UUT
exists. If the parent library is not a subdirectory of the current default directory, then the full path must
be given. On platforms where the operating system is case sensitive (i.e. Linux) the exact case
matched path must be entered.
The environment variable VCAST_PARENT_LIBRARY can be used to override the value set in the
environment script, which can be useful in running regression tests with different libraries for
comparison.
ENVIRO.WHITE_BOX:
This command is optional and if not given, defaults to blackbox. If the Whitebox option is desired then
the command "YES" is used.
ENVIRO.COVERAGE_TYPE: type
This command is optional and if not given, defaults to no coverage (None). To specify a coverage type,
type can be None, STATEMENT+MC/DC, STATEMENT+BRANCH, MC/DC, BASIS_PATHS,
BRANCH, STATEMENT .
ENVIRO.STUB:
ENVIRO.STUB:ALL
ENVIRO.STUB:NONE
ENVIRO.STUB: unit name to stub
This command is used to tell VectorCAST which dependent units it should stub by implementation
during the environment creation. The syntax for this command is one value per line. If all of the
dependent units are to be stubbed, then use ENVIRO.STUB:ALL. If none of the dependent units are to
be stubbed, then use ENVIRO.STUB:NONE.
To specify individual units to stub, each unit is named, one per line. By default, units that are not
specified by name are not stubbed.
ENVIRO.IGNORE:
ENVIRO.IGNORE:ALL
ENVIRO.IGNORE: unit name to ignore
This command is used to tell VectorCAST to ignore all dependent units or to ignore particular ones
when building the environment. Ignoring a unit causes VectorCAST to not parse it or any of its
dependents, thus shortening the time needed to build an environment. If all of the dependent units are to
be ignored, then use ENVIRO.IGNORE:ALL. To specify individual units to ignore, each unit is named,
one per line.
Import the specified subsystem into the environment’s work view. For Apex users.
ENVIRO.MAX_VARY_RANGE: num
This command is used to tell VectorCAST how many scalars can be varied at one time. The default is
20. The setting is indicated in the Tools => Options dialog, Builder tab.
ENVIRO.USER_GLOBALS:
ENVIRO.END_USER_GLOBALS:
This file can be permanently altered by editing the GLOBALS.ADA file in the DATA directory in the
VectorCAST installation directory. The User Globals are also accessible by choosing Environment =>
User Code => Edit, and then clicking the User Globals node.
package USER_GLOBALS_vCAST is
type float_type is digits 6;
INT1 : integer;
INT2 : integer;
INT3 : integer;
INT4 : integer;
INT5 : integer;
FLT1 : float_type;
FLT2 : float_type;
FLT3 : float_type;
FLT4 : float_type;
FLT5 : float_type;
STR1 : string_type;
STR2 : string_type;
STR3 : string_type;
STR4 : string_type;
STR5 : string_type;
BUFFER : BUFFER_TYPE;
end USER_GLOBALS_vCAST;
ENVIRO.USER_PARAMETERS:
ENVIRO.END_USER_PARAMETERS:
The code you add between this pair of commands creates a user-defined parameter to use in place of a
parameter, instead of a VectorCAST-generated parameter. The syntax consists of a pair on each line:
the case-sensitive VectorCAST-generated parameter, followed by your parameter object.
For example:
ENVIRO.USER_PARAMETERS:
<<MANAGER.PLACE_ORDER.TABLE>> USER_GLOBALS_vCAST.MY_TABLE
ENVIRO.END_USER_PARAMETERS:
User Parameters can be created by selecting Environment => User Code => Edit from the Menu Bar
and expanding the User Params node. Double-click to open the edit box. Alternatively, User
Parameters can be created in the Create New/Update Environment wizard by selecting Step 6 User
Code and opening the User Params node.
ENVIRO.USER_CODE_DEPENDENCIES:
ENVIRO.END_USER_CODE_DEPENDENCIES:
These commands delimit the dependencies (#include statements) for environment user code. They
are accessible in the Create New/Update Environment wizard, Step 5 User Code, User Code node, or
by choosing Environment => User Code => Edit, and opening the Header section.
ENVIRO.USER_CODE_DEPENDENCIES:
Header section
ENVIRO.END_USER_CODE_DEPENDENCIES:
ENVIRO.USER_CODE_OBJECTS:
ENVIRO.END_USER_CODE_OBJECTS:
These commands delimit the object definitions (type definitions, globals object, and subprograms) for
environment user code. They are accessible in the Create New/Update Environment wizard, Step 5
User Code, User Code node, or by choosing Environment => User Code => Edit, and opening the
Data section.
ENVIRO.USER_CODE_OBJECTS:
Data section
ENVIRO.END_USER_CODE_OBJECTS:
ENVIRO.USER_CODE_ONE_SHOT_INIT:
ENVIRO.END_USER_CODE_ONE_SHOT_INIT:
These commands delimit the harness initialization environment user code. This user code is processed
upon test harness initialization. They are accessible in the Create New/Update Environment wizard,
Step 5 User Code, User Code node, or by choosing Environment => User Code => Edit, and
opening the Harness Init section.
ENVIRO.USER_CODE_ONE_SHOT_INIT:
Harness initialization section
ENVIRO.END_USER_CODE_ONE_SHOT_INIT:
ENVIRO.USER_CODE_INITIALIZE:
ENVIRO.END_USER_CODE_INITIALIZE:
These commands delimit the test case initialization environment user code. This user code is
processed immediately before the harness calls the UUT. They are accessible in the Create
New/Update Environment wizard, Step 5 User Code, User Code node, or by choosing Environment
=> User Code => Edit, and opening the Test Case Init section.
ENVIRO.USER_CODE_INITIALIZE:
Test Case initialization section
ENVIRO.END_USER_CODE_INITIALIZE:
ENVIRO.USER_CODE_CAPTURE:
ENVIRO.END_USER_CODE_CAPTURE:
These commands delimit the test case termination environment user code. This user code is processed
immediately before the harness itself terminates. They are accessible in the Create New/Update
Environment wizard, Step 5 User Code, User Code node, or by choosing Environment => User Code
=> Edit, and opening the Test Case Term section.
ENVIRO.USER_CODE_CAPTURE:
Test case termination section
ENVIRO.END_USER_CODE_CAPTURE:
ENVIRO.USER_CODE_ONE_SHOT_TERM:
ENVIRO.END_USER_CODE_ONE_SHOT_TERM:
These commands delimit the test harness termination environment user code. This user code is
processed upon harness termination. They are accessible in the Create New/Update Environment
wizard, Step 5 User Code, User Code node, or by choosing Environment => User Code => Edit, and
opening the Harness Term section.
ENVIRO.USER_CODE_ONE_SHOT_TERM:
Harness termination section
ENVIRO.END_USER_CODE_ONE_SHOT_TERM:
ENVIRO.USER_CODE_STUB_PROCESSING:
ENVIRO.END_USER_CODE_STUB_PROCESSING:
These commands delimit the test stub processing environment user code. This user code is common to
all stubs, and is processed between Parameter Expected User Code and Parameter Input User Code.
They are accessible in the Create New/Update Environment wizard, Step 5 User Code, User Code
node, or by choosing Environment => User Code => Edit, and opening the Stub Processing
section.
ENVIRO.USER_CODE_STUB_PROCESSING:
Stub Processing section
ENVIRO.END_USER_CODE_STUB_PROCESSING:
ENVIRO.STUB_ENTRY_USER_CODE:
ENVIRO.END_STUB_ENTRY_USER_CODE:
These commands delimit the stub subprogram entry configuration user code. It is processed for every
stub upon entry, right after any Configure Stubs Beginning User Code and before any Expected
Parameter User Code of the stub. They are accessible in the Create New/Update Environment wizard,
Step 5 User Code, User Code node, or by choosing Environment => User Code => Edit, and
opening the Stub Entry section.
ENVIRO.STUB_ENTRY_USER_CODE:
Stub subprogram entry configuration code
ENVIRO.END_STUB_ENTRY_USER_CODE:
ENVIRO.STUB_EXIT_USER_CODE:
ENVIRO.END_STUB_EXIT_USER_CODE:
These commands delimit the stub subprogram exit configuration user code. It is processed for every
stub upon exit, right before any Configure Stubs Ending User Code and after any Input Parameter User
Code of the stub. They are accessible in the Create New/Update Environment wizard, Step 5 User
Code, User Code node, or by choosing Environment => User Code => Edit, and opening the Stub
Exit section.
ENVIRO.STUB_EXIT_USER_CODE:
Stub subprogram exit configuration code
ENVIRO.END_STUB_EXIT_USER_CODE:
ENVIRO.STUB_USER_CODE_FILE:
ENVIRO.END_STUB_ USER_CODE_FILE:
These commands delimit the beginning of stub and end of stub user code for Configure Stubs. There
are two sub-commands: one for the beginning of the stub
Repeat for each unit or subprogram that has stub user code to process at the beginning of the stub
and one for the end of stub. You can repeat these any number of times for different units or different
You can add or modify this user code by choosing Environment => Configure Stubs => Edit.
ENVIRO.STUB_USER_CODE_FILE:
BEGINNING_OF_STUB.unit.subprogram
Configure Stubs dialog, Beginning of Stub
END_BEGINNING_OF_STUB.unit.subprogram
Note: Repeat these sections for each unit or subprogram that has
stub user code to process at the beginning of the stub.
END_OF_STUB.unit.subprogram
Configure Stubs dialog, End of Stub
END_END_OF_STUB.unit.subprogram
ENVIRO.END_STUB_USER_CODE_FILE:
ENVIRO.STUB_DEPEND_USER_CODE_FILE:
ENVIRO.END_STUB_DEPEND_USER_CODE_FILE:
These commands delimit the stub dependencies user code for Configure Stubs. You can add or modify
this user code by choosing Environment => Configure Stubs => Edit, and opening the Stub
Dependencies cell.
For each stubbed unit for which you want to write stub dependency user code, you enclose that unit’s
name with BEGIN_UC and END_UC.
Note: Repeat the following for each unit that has a dependency
ENVIRO.STUB_DEPEND_USER_CODE_FILE:
BEGIN_UC:
unit
Configure Stubs dialog, Stub Dependencies column
END_UC:
ENVIRO.END_STUB_DEPEND_USER_CODE_FILE:
Example: Suppose a subprogram vcast_out is defined in a package named IO. In the following
example, that subprogram is called at the beginning of stub and end of stub for the unit GUI_STUB and
subprogram STUB_CONFIG_STUBS, as shown in the first portion of the example below. The second
portion shows the package IO “withed” in the Stub Dependency section.
ENVIRO.STUB_USER_CODE_FILE:
BEGINNING_OF_STUB.GUI_STUB.STUB_CONFIG_STUBS
IO.vcast_out("BEGIN CONFIG STUBS");
END_BEGINNING_OF_STUB.GUI_STUB.STUB_CONFIG_STUBS
END_OF_STUB.GUI_STUB.STUB_CONFIG_STUBS
IO.vcast_out("END CONFIG STUBS");
END_END_OF_STUB.GUI_STUB.STUB_CONFIG_STUBS
ENVIRO.END_STUB_USER_CODE_FILE:
ENVIRO.STUB_DEPEND_USER_CODE_FILE:
BEGIN_U\\\\DATASERVER
GUI_STUB
with IO;
END_U\\\\DATASERVER
ENVIRO.END_STUB_DEPEND_USER_CODE_FILE:
ENVIRO.ADDITIONAL_UNIT_BODIES
ENVIRO.END_ADDITIONAL_UNIT_BODIES
These two commands delimit the implementation portion of additional units that you want added to the
test harness, as though it were another source file (unit). For example, you may have a print utility, a
timing routine, or a data setup routine that is not part of your deliverable, but necessary for unit testing.
The code here is not parsed by VectorCAST during environment building. These tags are accessible in
the Create New/Update Environment wizard, Step 5 User Code, User Code node, or by choosing
Environment => User Code => Edit, and opening the Additional Unit Bodies section.
ENVIRO.ADDITIONAL_UNIT_BODIES:
Implementation for additional units
ENVIRO.END_ADDITIONAL_UNIT_BODIES:
ENVIRO.ADDITIONAL_UNIT_SPECS
ENVIRO.END_ADDITIONAL_UNIT_SPECS
These two commands delimit the specification portion of an additional code unit, for example, a class
header. These tags are accessible in the Create New/Update Environment wizard, Step 5 User Code,
User Code node, or by choosing Environment => User Code => Edit, and opening the Additional
Unit Specs section.
ENVIRO.ADDITIONAL_UNIT_SPECS:
Specifications for additional units
ENVIRO.END_ADDITIONAL_UNIT_SPECS:
ENVIRO.USER_CODE_TIMER_START:
ENVIRO.END_USER_CODE_TIMER_START:
Code to start timer after the test harness performs data setup, but just prior to calling the UUT. These
tags are accessible in the Create New/Update Environment wizard, Step 5 User Code, User Code
node, or by choosing Environment => User Code => Edit, and opening the UUT Timer Start
section.
ENVIRO.USER_CODE_TIMER_START:
Code to start timer
ENVIRO.END_USER_CODE_TIMER_START:
ENVIRO.USER_CODE_TIMER_STOP:
ENVIRO.END_USER_CODE_TIMER_STOP:
Code to stop timer immediately after returning from the call to the UUT. These tags are accessible in
the Create New/Update Environment wizard, Step 5 User Code, User Code node, or by choosing
Environment => User Code => Edit, and opening the UUT Timer Stop section.
ENVIRO.USER_CODE_TIMER_STOP:
Code to stop timer
ENVIRO.END_USER_CODE_TIMER_STOP:
ENVIRO.INDUSTRY_MODE
Code to set the Industry Mode. Industry Modes are set as follows for each Industry Mode type:
ENVIRO.END
Required. This command marks the end of the script data for a particular environment. It must be the
last line in the environment script. When this command is encountered, VectorCAST attempts to build
an environment using the values read since ENVIRO.NEW was read.
ENVIRO.STUB:DATABASE
ENVIRO.END
> All commands start with "TEST." to indicate that a test command follows.
> All commands that require data must use a colon (:) to indicate the start of the associated data.
> Comments may be inserted, and are indicated by inserting the comment delimiter "--" as the first
item on the line.
Script Commands
The following is a list of the valid script commands with a description of each command.
Required. Tell VectorCAST that the test case definition that follows defines a new test case, adds
values to an existing test case, replaces an existing test case, or removes an existing test case. You
can only use one (NEW, ADD, REPLACE or REMOVE) for each test case definition.
TEST.NEW is used to tell VectorCAST that the following commands are to start the construction of a
new test case.
TEST.ADD is used to add additional input or expected data values to an existing test case.
TEST.REPLACE is used to delete an existing test case with the same name and replace it, using the
data that follows in the test script. The script log informs you that the test case is being replaced.
Attributes for scalar data. These are specific to attributes contained in the test case that are unrelated to
input or expected values. For example, you may right-click on the input or expected value field of a
particular parameter and set the base for the value to be hex or octal. This attribute gets stored in the
test script so that the same attribute is used when it is re-imported into the same or a different
environment.
TEST.AUTOMATIC_FINALIZATION
Specify a <<COMPOUND>> or <<INIT>> test case as an Automatic Finalization test case. It sets a
flag indicating a <<COMPOUND>> or <<INIT>> test case should be run automatically after other test
cases. A single <<COMPOUND>> or <<INIT>> test case can be both Automatic Initialization and
Automatic Finalization. An Automatic Finalization test case can be executed on its own, but it cannot
be run before or after itself. There can only be one Automatic Finalization test per test environment.
When exporting a test script, the Automatic Finalization test cases are also exported.
TEST.AUTOMATIC_INITIALIZATION
TEST.BASIS_PATH: x of y
When basis path test cases are generated automatically (using Tools => Basis Path => Generate
Tests), and a test script is exported for these test cases, the TEST.BASIS_PATH command is used to
identify them as basis path test cases. The text “x of y” indicates that particular test case out of the total
number of basis path test cases generated for that subprogram.
TEST.COMPOUND_ONLY
This command is used to tell VectorCAST that the test case is to be executed only from within a
compound test case. During Batch Execute All, the test case is not executed unless it is part of a
compound.
TEST.CSV_NAME_COL:
This command is used in a CSV MAP test case. It specifies the number of the column containing the
test case name. Enter a value of "0" if you do not specify the test case name in the .csv file.
TEST.CSV_NOTES_COL:
This command is used in a CSV MAP test case. It specifies the number of the column containing test
case notes. Enter a value of "0" if you do not specify Notes in the .csv file.
TEST.END
This command is used to indicate the end of a test case and causes the previously entered test case
data to be stored into the VectorCAST environment.
This command is used to provide expected values for parameters and global objects.
See the section "Setting Input and Expected Values" on page 464 for a full description of the <value>
syntax.
TEST.EXPECTED_GLOBALS_USER_CODE: global_value
<enter user code here>
TEST.END_EXPECTED_GLOBALS_USER_CODE:
Together, these commands delimit the expected user code for global values. This user code is
evaluated when other global values are evaluated, so it is possible for this user code to be evaluated
when a test ends due to the event limit being reached.
TEST.EXPECTED_GLOBALS_USER_CODE:USER_GLOBALS_VCAST.<<GLOBAL>>.VECTORCAST_INT2
{{ <<USER_GLOBALS_VCAST.<<GLOBAL>>.VECTORCAST_INT2>> == ( 2 ) }}
TEST.END_EXPECTED_GLOBALS_USER_CODE:
TEST.EXPECTED_USER_CODE: <<testcase>>
<enter user code here>
TEST.END_EXPECTED_USER_CODE:
Together, these commands delimit the test case’s expected user code when “<<testcase>>” is used
(without the quotes), and is not associated with any single parameter.
TEST.EXPECTED_USER_CODE: unit.subprogram.parameter
<enter user code here>
TEST_END_EXPECTED_USER_CODE:
Together, these commands delimit a parameter’s expected user code in the UUT. Specify the
parameter to which the user code applies using the notation unit.subprogram.parameter.
TEST.FLOATING_POINT_TOLERANCE: number%
(or)
TEST.FLOATING_POINT_TOLERANCE: decimal number
Using one of these commands enables you to set a floating-point tolerance at the test case level. You
can set the floating-point tolerance at the environment level, the test case level, and the parameter
level. Each level overrides the previous one. Therefore, you can set an environment-level tolerance
(Tools => Options dialog, Execute tab), specify a test case tolerance for a particular test case, and
within that test case, specify a tolerance for one parameter.
In the test script, you can specify the floating-point tolerance as a percent, or as the decimal equivalent.
For example, to set the test case floating-point tolerance to 1.25%, use either of the following notations:
TEST.FLOATING_POINT_TOLERANCE: 1.25%
TEST.FLOATING_POINT_TOLERANCE: 0.0125
Upon test script import, the text "Test case floating point tolerance is: number%" is displayed in the log.
Note: For very small tolerances, the floating point representation of the value may not be exact
due to the resolution of floating point numbers; in this case, the floating point tolerance may
show more digits than the user expects. For example, a tolerance of 0.000000001 may actually
be displayed as 0.00000000099999.
TEST.FLOW:
<unit> . <subprogram>
TEST.END_FLOW:
This is the expected sequence of subprograms called in the Unit Under Test and stubbed units. For
each <unit>.<subprogram> that is called, there is a line in this section. For example, if the UUT is unit
A, the test harness calls unit A’s subprogram A() first, then unit B’s subprogram B(), then unit C’s
subprogram C(), then the UUT returns control to the test harness:
TEST.FLOW
A.A
B.B
C.C
A.A
TEST.END_FLOW
TEST.IMPORT_FAILED
VectorCAST adds this line to an exported test script file if the option “Strict test case importing” is on,
and there was an error when the script file was originally imported.
See also "Strict Test Case Importing" on page 280 for more information.
TEST.NAME: name
This command is used to provide a name for the test case. The name is case-sensitive, if you are using
it with TEST.ADD. This command is not required. If a TEST.NAME command is not provided,
VectorCAST will generate an automatic name as it does when creating test cases in the Test Case
Editor. If this command is used, the name provided must be unique. Otherwise, it will be ignored and an
automatic name will be assigned when the test script is processed.
TEST.NOTES:
<enter text here>
TEST.END_NOTES:
Together, these commands delimit a list of requirements or notes for the test case. Everything that
exists between these delimiters will be included in the requirements section of the test case. Note that
each of these commands must exist, by itself, on separate lines. Anything that follows these
commands on the same line will be ignored. If you have the Report option “Verbose management report”
turned on, then the first 89 characters in the Notes section are displayed in the Test Case Management
Report.
TEST.REQUIREMENT_KEY: requirement_key
When an environment has imported requirements through the Requirements Gateway, a requirement
can be associated with a testcase. Upon test script export, this command is used to identify the
requirement key associated with each test case. The requirement key is imported, and should not be
changed in the test script. If a test case has more than one requirement, then multiple commands are
used, with one key per line. (VectorCAST with Requirements Gateway license)
The TEST.SCRIPT_FEATURE line is automatically added by VectorCAST on test script export and
should not need to be modified by the user. These commands tell VectorCAST if any special features
are present in the environment which the test script importer should support.
TEST.VALUE:ADA_TESTCASE_VERSION2.FUNC_VECTOR_IMPLICIT_INTEGER.P(-10..10):3
However, if you turn on the option 'Disable direct array indexing' on the Tools => Options dialog,
Builder tab, then the ADA_DIRECT_ARRAY_INDEXING script feature is not written to the script and
the TEST.VALUE line is output in the script as:
TEST.VALUE:ADA_TESTCASE_VERSION2.FUNC_VECTOR_IMPLICIT_INTEGER.P(1..21):3
The command TEST.SLOT provides information on a slot in a compound test. It should be present
when the TEST.SUBPROGRAM line specifies <<COMPOUND>>.
l slot can be any number. The test cases in the compound are added in the order they are listed in
the TEST.SLOT command. slot is a quoted string. Example: “1”
l unit is a string identifying the unit under test. The name is case sensitive. unit is a quoted string.
Example: “file_io”. If the slot is a nested compound slot, unit is specified as <<COMPOUND>> in
quotes.
l subprogram name is a string identifying the subprogram in which this test case appears.
subprogram name is a quoted string. Example: “CreateFile”. If the slot is a nested compound slot,
subprogram name is specified as <<COMPOUND>> in quotes.
l testcase name is the name of the test case in the slot. testcase name is a quoted string. Example:
“CREATEFILE.001”
l number of iterations is an integer specifying how many times the test case should repeat. To “turn
off” a test case while keeping it in the compound, set the iterations to 0. Doing so causes the test
case to be skipped during execution of the compound, while remaining in the compound test case.
Note that setting iterations to 0 causes the test case to be skipped and not executed, and is not
the same as setting PRINT=FALSE, which still executes the test case but does not gather the
execution data. number of iterations is a quoted string. Example: “1”
l delay, in seconds indicates how many seconds (to a precision of tenths of a second) to delay after
executing the slot, before executing the next slot. delay is a quoted string. Example: "1.2" If not
included, the delay is assumed to be 0.
l print is either a TRUE or FALSE value and is used to prevent the reporting of data for a slot. The
default value is TRUE and if no value is specified, the value TRUE is assumed. The value FALSE
prevents the harness from reporting on executing data. The test is executed, but the test harness
does not gather any data for the execution report. Note that PRINT=FALSE is not the same as
setting iterations to 0 for the slot, where the test case will be skipped and not executed. If the
value is FALSE, the test case in the slot is required to have no expected values. Example:
"PRINT=FALSE".
For example, the following test script specifies a compound test case which has two slots with the test
case PLACEORDER.001 and .002 included in it. Note that the script for the test cases
PLACEORDER.001 and PLACEORDER.002 must be defined above the definition for the compound or
already exist in the environment at the time that the compound test script is encountered.
Note: Compound test case below doesn’t get a TEST.UNIT line itself.
-- COMPOUND TESTS
TEST.SUBPROGRAM:<<COMPOUND>>
TEST.NEW
TEST.NAME:<<COMPOUND>>.001
TEST.NOTES:
TEST.END_NOTES:
TEST.SLOT: "1", "MANAGER", "PLACE_ORDER", "1", "PLACEORDER.001"
TEST.SLOT: "2", "MANAGER", "PLACE_ORDER", "1", "PLACEORDER.002"
TEST.END
TEST.STUB_EXP_USER_CODE: unit.subprogram.parameter
<enter user code here>
TEST.END_STUB_EXP_USER_CODE:
TEST.STUB_VAL_USER_CODE: unit.subprogram.parameter
<enter user code here>
TEST.END_STUB_VAL_USER_CODE:
Required. This command is used to indicate a subprogram for which test cases will be built. The
subprogram name corresponds to any subprogram of the Unit Under Test specified in the TEST.UNIT
line. If several test cases are to be built for the same subprogram this command does not have to be
repeated until a different subprogram is desired. To create an <<INIT>> test case, use the subprogram
name <<INIT>>.
Required. This command identifies the unit that the test case is for. The name is not case sensitive.
This command can be used once to indicate that all of the following test case definitions are for a single
unit, until another TEST.UNIT line appears. This line is not required for <<INIT>> or
<<COMPOUND>> test cases.
This command is the most important of the test case scripting commands. It is used to provide the
individual input values for parameters and global objects.
See the section "Setting Input and Expected Values" on page 464 for a full description of the <value>
syntax.
TEST.VALUE_USER_CODE: <<testcase>>
<enter user code here>
TEST.END_VALUE_USER_CODE:
Together, these commands delimit the test case’s input user code, when “<<testcase>>” is used
(without the quotes), and is not associated with any single parameter.
TEST.VALUE_USER_CODE: unit.subprogram.parameter
<enter user code here>
TEST.END_VALUE_USER_CODE:
Together, these commands delimit a parameter’s input user code in the UUT. Specify the parameter to
which the user code applies using the notation unit.subprogram.parameter.
All test value commands have three fields. The first field is the test command and is always set to
“TEST.VALUE:” or “TEST.EXPECTED”. The second field is called the identifier field and the third field
is called the value field. Each of the three fields is separated by colons.
The identifier field consists of the unit name, a dot, the subprogram name, a dot, and the parameter
name. For example: manager.Place_Order.Seat.
Note: VectorCAST is not sensitive to spaces around the Value field. The colon marks the start
of the value and the end-of-line marker marks the end of the value.
Numeric Types
With numeric types, the value field is simply the number to be assigned. For example, the test value
commands to set values for the two numeric parameters of subprogram Place_Order would be as
follows:
TEST.VALUE:manager.Place_Order.Table: 3
TEST.VALUE:manager.Place_Order.Seat: 1
Note: When working with numeric parameters you may use based notation or exponential
notation as you would when building test cases in the GUI.
Structure Types
For structure types, the Identifier field is extended using the standard C syntax of referring to the fields
by their names with a '.' separating the field name from the rest of the identifier.
TEST.VALUE:manager.Place_Order.Order.Salad: CAESAR
TEST.VALUE:manager.Place_Order.Order.Entree: STEAK
TEST.VALUE:manager.Place_Order.Order.Dessert: CAKE
Enumeration Types
With enumeration types, the value field will be the enumeral that you are selecting. In the previous
example you may have noticed that the fields were all enumeration types. In this case the value field of
the commands are the enumerals.
Note that full named notation is neither required nor permitted when referring to the enumerals.
VectorCAST will determine the validity of the value from the context of the definition.
Note: Out of Range enumeral values are specified with test case scripting by entering a number
in place of the enumeral.
Boolean Types
Boolean types are similar to enumeration types, with the Value field limited to TRUE or FALSE.
l You must use the character delimiter if the character to be entered is itself the delimiter character
(apostrophe) or a blank character.
l You must use the string delimiter if the string contains blank characters or contains string
delimiters.
TEST.VALUE:database.Get_Table_Record.Data.Designator: d
TEST.VALUE:database.Get_Table_Record.Data.Designator: ' '
TEST.VALUE:database.Get_Table_Record.Data.Designator: '''
TEST.VALUE:database.Get_Table_Record.Data.Wait_Person: Tim
TEST.VALUE:database.Get_Table_Record.Data.Wait_Person: ""Me""
TEST.VALUE:database.Get_Table_Record.Data.Wait_Person: "J Smith"
Note: As you would expect, the case of any character or string value is maintained.
Access Types
When working with access types, the keyword "ALL" must be used as is consistent with standard Ada
syntax. For example, assume the following package specification is a unit under test or a stub.
package TEST is
type POINTER is access integer;
procedure TEST_IT (VALUE : POINTER);
end TEST;
TEST.VALUE:TEST.TEST_IT.VALUE.ALL: 4
Array Types
When working with Arrays, the same conventions must be used as are used when building array values
interactively. That is, the array index expression must be the Ada 'POS attribute of the array item that
you wish to select. The array index will always be enclosed in parenthesis. The following example
illustrates this:
TEST.VALUE:DATABASE.GET_TABLE_RECORD.DATA.ORDER(2).SALAD: GREEN
package TEST is
type INDEX is new integer range 1..100;
type BUFFER is array (INDEX range <>) of integer;
procedure TEST_IT (VALUE : BUFFER);
end TEST;
TEST.VALUE:TEST.TEST_IT.VALUE(1): 1
TEST.VALUE:TEST.TEST_IT.VALUE(2): 2
TEST.VALUE:TEST.TEST_IT.VALUE(3): 3
TEST.VALUE:TEST.TEST_IT.VALUE'first: 1
TEST.VALUE:TEST.TEST_IT.VALUE'last: 3
Record Types
For record types the Identifier field is extended using the standard Ada syntax of referring to the fields
by their names:
TEST.VALUE:MANAGER.PLACE_ORDER.ORDER.SALAD: CAESAR
TEST.VALUE:MANAGER.PLACE_ORDER.ORDER.ENTREE: STEAK
TEST.VALUE:MANAGER.PLACE_ORDER.ORDER.DESSERT: CAKE
TEST.VALUE:MANAGER.GET_CHECK_TOTAL.RETURN: 23.0
Global Objects
To set the value of global objects rather than subprogram parameters, the basic syntax is
UNIT.<<GLOBAL>>.OBJECT. All of the syntax related to the specific type of the object is the same as
it is for subprogram parameters. The following example uses the same types from the Basic Tutorial,
but we assume that there is a global object in unit MANAGER called ORDER_OBJECT.
TEST.VALUE:manager.<<GLOBAL>>.ORDER_OBJECT.ENTREE: STEAK
See the section "The Testcase Options Tab" on page 303for information on what each option does. To
set them to true or a value in the scripting language, the syntax is:
TEST.VALUE:<<OPTIONS>>.COMPARE_BEFORE_UUT:TRUE | FALSE
TEST.VALUE:<<OPTIONS>>.DATA_PARTITIONS:<value>
TEST.VALUE:<<OPTIONS>>.DISPLAY_FULL_STRING_DATA:TRUE | FALSE
TEST.VALUE:<<OPTIONS>>.DO_COMBINATION:TRUE | FALSE
TEST.VALUE:<<OPTIONS>>.EVENT_LIMIT:<value>
TEST.VALUE:<<OPTIONS>>.FLOAT_POINT_DIGITS_OF_PRECISION:<value>
TEST.VALUE:<<OPTIONS>>.FLOAT_POINT_TOLERANCE:<value>
TEST.VALUE:<<OPTIONS>>.GLOBAL_DATA_DISPLAY:EACH_EVENT | RANGE_ITERATION | SLOT_ITERATION |
TESTCASE
TEST.VALUE:<<OPTIONS>>.HEX_NOTATION_FOR_UNPRINTABLE_CHARS:TRUE | FALSE
TEST.VALUE:<<OPTIONS>>.MULTI_RETURN_SPANS_COMPOUND_ITERATIONS: TRUE | FALSE
TEST.VALUE:<<OPTIONS>>.MULTI_RETURN_SPANS_RANGE: TRUE | FALSE
TEST.VALUE:<<OPTIONS>>.MULTI_RETURN_SPANS_TESTCASES: TRUE | FALSE
TEST.VALUE:<<OPTIONS>>.REFERENCED_GLOBALS: TRUE | FALSE
To accommodate this type of testing, VectorCAST enables range expressions to be used in place of
individual scalar values for any parameter. This is accomplished with the following script syntax:
A simple example of using range expressions would be in the testing of trig functions.
package math is
type ANGLE is new float range 0.0..359.99;
function SINE (A : ANGLE) return float;
function ATAN (X, Y : ANGLE) return float;
Using the range testing capability you could build a single test case that would exercise the sine
function in the first quadrant every one tenth of a degree. The test script to accomplish this is shown
here:
TEST.SUBPROGRAM:SINE
TEST.NEW
TEST.VALUE: MATH.SINE.A:vary from: 0.0 to: 90.0 by: 0.1
TEST.EXPECTED: MATH.SINE.return:0.0..1.0
Note: The range values defined allow you to loop forward or backward through the specified
range as in the following examples:
Forward looping
Backward looping
When setting up a test case for range testing, you may vary different parameters simultaneously. This
means that each parameter can have a different range expressions within a single test case. This
capability enables you to vary multiple values simultaneously, or vary one value while holding others
constant
The following test script shows how two parameters can be varied simultaneously:
TEST.SUBPROGRAM:atan
TEST.NEW
TEST.VALUE:MATH.atan.X:vary from 1.0 to: 90.0 by: 1.0
TEST.VALUE:MATH.atan.Y:vary from 1.0 to: 90.0 by: 1.0
TEST.EXPECTED:MATH.ATAN.return:45.0
TEST.END
Note that in this example the expected result is used to ensure the value returned by the ATAN function
is always 45 (ATAN of 1).
The following script shows a test case where one parameter is held constant while another paramter is
varied:
TEST.SUBPROGRAM:atan
TEST.NEW
TEST.VALUE:MATH.atan.X:0
TEST.VALUE:MATH.atan.Y:vary from 1.0 to: 90.0 by: 1.0
TEST.EXPECTED:MATH.ATAN.return:0.0
TEST.END
Again in this example the expected result is used to ensure the value returned by the ATAN function is
always 0 (ATAN of 0).
Note: By default you may simultaneously vary up to 20 parameters at a time in a given test
case. If you need to vary more, edit the Maximum Varied Parameters option. It is located in the
Tools => Options dialog, Builder tab. For this option to take effect, you will need to rebuild the
current environment. The easiest way to do this is with Environment => Rebuild Environment. If
multiple values are varied in the same test case, you can choose to have the values vary in
parallel or combinatorially.
Because all of the actual values would show up in the test results from this test case, it would also be
possible to go back and verify the output value for any single data point.
List of Values
When entering a list of values in a test case script, simply use the syntax item1, item2, itemN as in the
following examples:
TEST.VALUE:MANAGER.PLACE.ORDER.SEAT:1,2,4
If a certain value should be repeated, rather than list it multiple times, just include the repeat count in
parentheses before the item.
TEST.VALUE:MANAGER.PLACE_ORDER.SEAT:1, (4)2, 4
In a list of expected values, if any value is acceptable, use the special notation <<ANY>>. Using
<<ANY>> indicates that any value that is compared to that spot in the list is acceptable. <<ANY>>
doesn’t make a test case pass or fail; it is used as a placeholder in a list of expected values.
TEST.VALUE:MANAGER.PLACE_ORDER.SEAT:1,(4)2,<<ANY>>
Note: For a list of values, if the number of values entered is less than the number of calls to the
particular subprogram, VectorCAST will use the last parameter value for the remaining calls.
Example:
TEST.VALUE:MANAGER.PLACE_ORDER.ORDER.BEVERAGE:WIiNnEe,NOo_ORDER,\
BEER,WINE,SODA,MIXED_DRINK,\
MIXED_DRINK,NO_ORDER
Note: The Script Reader will ignore all white space so you may indent and space the values for
maximum readability.
TEST.VALUE:TEST1.P(integer).VAL: 3
TEST.VALUE:TEST1.P(character).VAL:'d'
TEST.VALUE:TEST1.P(string).VAL:"this is a test"
TEST.VALUE:TEST1.P(boolean).VAL:TRUE
To test these subprograms in VectorCAST, you would create a test case for each subprogram and then
call the three test cases from a compound test case. The problem is there is no static way for
VectorCAST to pass the file pointer created by InitializeFile to the other two subprograms. With
parameter user code, you have the ability to set data dynamically. In this example, you want to pass the
file pointer returned from InitializeFile as an input into WriteToFile and CloseFile. With user code, you
use the name of the file pointer returned by InitializeFile as a parameter for the other two files. The
following test case script shows how to do that:
The highlighted lines show the scripting language used to set the parameter “FILE” to be the return value
from the call to InitializeFile. You can also put any other executable code between the TEST.VALUE_
USER_CODE and TEST.END_VALUE_USER_CODE delimiters.
Compiler Limitations
VectorCAST is subject to all limitations identified in the documentation provided with each of the
compilers supported by VectorCAST. The following sections identify these specific limitations and
apply to all platforms for each compiler supported by VectorCAST.
All Compilers
Note: A unit is not available when VectorCAST produces the status alert: "Source not Found:
{unit_name}"
To make use of this feature, store a copy of the source code that you want VectorCAST to use for a
particular unit into a file with the name of the unit suffixed with ".a" (e.g. for package CALENDAR use
filename calendar.a). This file must be stored in the VectorCAST distribution area on the following path:
Note: VectorCAST will only refer to this directory if it cannot find the source file for a unit through
the normal means. You CANNOT override the compiled version of a source file by inserting an
alternate version into this directory structure!
It is not necessary for the user supplied source file to be a complete specification of the unit, it is only
required that the source provided is syntactically correct. Therefore if you are only using one type from a
pre-defined package you can create a file with only that type definition.
This file will NOT be compiled, it is only used by the VectorCAST parser to allow users to pass
As an example, assume we are using the Alsys Ada World compiler under Linux and our code to be
tested uses the type CALENDAR.DAY_NUMBER. Since Alsys does not supply the source code with
for CALENDAR, you would be unable to manipulate a parameter or object of type DAY_NUMBER with
VectorCAST. To work around this, simply create a file called calendar.a and store it into
$VECTORCAST_DIR/source/ada_world
package CALENDAR is
type DAY_NUMBER is range 1..31;
end CALENDAR;
VADS / SunAda
Loader Commands
Any commands that are normally passed to the loader should be either embedded in the source file for
the unit under test using the pre-defined pragma "link_with" or should be inserted into the parent Ada
library "ada.lib" file as "WITHn" directives.
If you wish to set values of parameters and/or objects which use types from the UNSIGNED_TYPES
package you will need to modify the source file for this package. The source file can be found by setting
you working directory to any VADS Ada library and executing the VADS "which" command for unit
"unsigned_types". If you then edit this file (root privilege may be required) you will find that the file
consists of all comment lines. To allow VectorCAST to parse the types in this package you need to
uncomment the package declaration, the three types definitions (unsigned_integer, unsigned_short_
integer, and unsigned_tiny_integer), and the end package declaration. An example of the five
uncommented lines is shown below:
package unsigned_types is
-- 0..4294967295
-- 0..65535
-- 0..255
end unsigned_types;
After making these changes simply run VectorCAST normally and an environment will be built that
enables you to manipulate objects and parameters of these types.
For example, to define a boolean variable called debug and set its value to "true," set the option to be: "-
D debug boolean true"
AdaWorld / ActivAda
This enables VectorCAST to extract the source file from the Ada library.
avoided.
AdaWorld inserts a new line marker into source lines that are 80 characters or longer in the extracted
version. Even for comment lines! This may yield unpredictable results for the VectorCAST parser since
the resulting source file is not compilable Ada.
A work around is to use an AdaWorld start-up file that contains the following command:
global.terminal (width=255)
Note: For ActivAda, the location of the ADA.DEF file is passed to VectorCAST via the ADADEF
environment variable. Set the value of this environment variable to point at the ADA.DEF file to
be used.
To run VectorCAST with the AdaWorld debugger (AdaProbe), the unit under test must have been
compiled with the "debug=yes" option:
VectorCAST (host) requires the following parameters to be set up with default values via this method:
VectorCAST (TARGET) requires the following additional parameters to be set up with default values via
this method:
Some library units that are part of the Ada Run Time, (i.e. SYSTEM, TEXT_IO) cannot be accessed by
VectorCAST under AdaWorld because AdaWorld does not have source files for these entities.
ObjectAda
The parent library that is used to build a VectorCAST environment must have absolute path names for
all libraries For example your Ada library file should contain something similar to the following:
/home/first_subsystem/adalib
/home/second_subsystem/adalib
/home/base_types/adalib
Library units should be compiled from separate files. Library units should not be combined in the same
source file. For example package specifications and package bodies should be in separate source files.
Combining specifications and bodies in the same source file may result in unpredictable behavior.
If the parent library contains links to other libraries, these links should utilize absolute path names not
relative path names.
same file or compiling multiple unrelated units in the same file can lead to unpredictable behavior during
environment construction.
APEX
Starting VectorCAST
VectorCAST must be run from an Apex shell window in order for VectorCAST to have access to the
Apex Command Line Interface commands. The Apex shell window is accessed under the Apex Tools
menu. It is also possible to integrate VectorCAST into the Apex GUI; refer to your Apex documentation
for details on modifying the Apex Menu Files.
VectorCAST must be run from within an Apex sub-system for Summit, or from within a Rational
subsystem for ClearCase. Generally you will run VectorCAST from the parent directory that contains
the Apex View subdirectories; you cannot run from within a view directory (Summit).
GNAT
VectorCAST is compatible with GNAT 3.04 and higher. Versions of GNAT prior to 3.04 will not work
with VectorCAST.
File names
GNAT normally requires every unit to be in a single file, where the file name matches the unit name (e.g.
the specification for package fred_pkg will be in file fred_pkg.ads). When source code does not follow
this requirement, GNAT has two methods for resolving discrepancies. The easiest method to use is the
GNAT utility “gnatchop” – this utility will copy Ada code from a file into the correctly named file (e.g. if
file fred.ada contains the specification and body for fred_pkg, running “gnatchop” on fred.ada will yield
two files: fred_pkg.ads and fred_pkg.adb). Another method is to use the gnat.adc file, which enables
the user to map unit names to file names regardless of conventions (e.g. pragma SOURCE_FILE_
NAME ( FRED_PKG, SPEC_FILE_NAME => “fred.ads”); ).
/home/lib1
types.ads
/home/lib2
manager.ads
manager.adb
database.ads
database.adb
manager_driver.adb
To compile the “lib2” files that depend on types.ads you would enter the following command:
To “bind” and link the main procedure: manager_driver, you would enter the following command:
Using the environment variables, you can omit the “-I” and “-aO” options as follows:
VectorCAST-Specific Limitations
The following is a list of limitations imposed by VectorCAST and work-around solutions.
Subprogram Renames are not available for testing in the compilation unit where they are renamed.
Option: Test all subprograms in the context of the unit where they are defined.
All required external information must be provided for the unit under test to be run. That is, if the unit
solicits keyboard input during processing, this input must be provided by the tester. Similarly any
external input retrieved from any other hardware device should be simulated or actually provided to
ensure normal test function.
Options: None, tests will either hang or not run properly if expected input is not provided.
Options: Provide data values for arrays with more than 26 dimensions within the User Code
package.
Options: Provide a manually generated stub for these units and use the Re-Link command to
regenerate the executable test harness.
Code that contains an infinite loop and does not call a stubbed unit should be stubbed or modified to
have a finite number of loop iterations. Without doing this, control will never be passed back to the
operating system. The result of this is that VectorCAST will hang after each test case execution.
Options: Use the appropriate operating system command to kill the VectorCAST test harness
after each test execution.
Multiple library units within one source file may generate unpredictable results.
Options: If multiple units exist in a single file the "end" statements for the packages or
procedure must include the package /procedure name. This does not guarantee success, the
best option is to separate units into separate source files.
When building environments in which some of the parameters are unconstrained array types you may
get compile warnings, link errors, or execution errors if the maximum size of the unconstrained array
type is larger than the compilation system enables.
Parameters Defined with a 'class attribute are treated by ignoring the 'class attribute and allowing
manipulation of the parameter as if the type mark was the base type without the 'class.
Option: Use the User Code to set values for the parameter using classes derived from the
parent class.
The Unit Under Test can be a private package; VectorCAST automatically turns on whitebox testing. A
private package cannot be stubbed by VectorCAST. Private packages CAN be part of the execution
closure.
Option: None
Environment Variables
VCAST_ALTERNATE_REPORT
If this environment variable is defined, then a tab-separated report is generated after test execution, in
addition to the regular execution results. Use clicast reports alternate to extract the report to
a file or standard output.
VCAST_APEX_COPY_FILES
When building environments with Apex, copy all source files to local view rather than creating Linux
links.
VCAST_DISABLE_ADA_ARRAY_RANGE_CHECKING
Disables the ability to use index notation for arrays, leaving only positional notation. This environment
variable overrides the Builder option “Disable direct array indexing.”
VCAST_DISABLE_FILE_DELETE
VCAST_DISABLE_TEST_MAINTENANCE
If this environment variable is set to 1, then the VectorCAST Test Script Maintenance Utility is
disabled.
VCAST_GNAT_HIE_DASH_E
VCAST_GNAT_LD_OPTIONS
VCAST_IGNORE_COMPILE_ERRORS
Do not stop compiling when an error is encountered. For use if your compiler returns an exit code not
equal to zero even when successful.
VCAST_IGNORE_FILE_EXISTS_CHECK_FOR_COVERABLE
VCAST_IGNORE_TYPEMARKS
VCAST_LEGACY_CFG_FILE_INHERITANCE
Read *.CFG files from installation directory and user's home directory before reading from the local
directory. If the environment variable VCAST_LEGACY_CFG_FILE_INHERITANCE is set, any
options inherited from the installation directory or user's home directory are saved to the local *.CFG
file, creating a complete set of options from all sources in one place (the local *.CFG file). Once this is
done for each environment, the environment variable no longer needs to be set when running the
regression script.
VCAST_MAX_INSTRUMENTATION_LINE_LENGTH
Set the maximum length of a source line generated by the coverage instrumentation. When the value of
the environment variable is set to 80 or less, lines in the instrumented source code are wrapped at a
convenient location around 80 characters. If the value is greater than 80, lines are wrapped at that
number of characters.
VCAST_NEVER_STUB_LIST
A comma-separated list of unit names (without the file extension) that should not be stubbed even when
the STUB ALL option is selected. This environment variable can also be set to the full path to a file, with
one unit per line. This feature is useful when the never-stub list is long, or when sharing the list with
others.
VCAST_NO_STANDARD_PKG_USAGE
By default, the Multiunit Whitebox harness redefined TRUE and FALSE by using
VCAST_NO_USER_EXCEPTION_SUPPORT
VCAST_OLD_ACTIVADA
VCAST_PARENT_LIBRARY
Over-rides the setting for ENVIRO.PARENT_LIB in an environment script that was created for a
regression test. Useful for running a regression test with a different Parent Library than then one that
was used when the regression test was created.
VCAST_RAVEN_HEAP_SIZE
For RAVEN target, set size of heap VectorCAST creates for each pointer type.
VCAST_RAVEN_STDOUT
VCAST_RECURSION_DEPTH
Maximum number of levels to recurse when trying to generate basis path data.
VCAST_REGRESSION_COMMAND
An additional command that should be added to regression test shell script or batch file.
See also "To Post-Process the Regression Scripts " on page 133.
VCAST_RPTS_PRETTY_PRINT_HTML
VCAST_S68_LOCATION
VCAST_TIC_INDICATES_MACHINE_CODE
Tic mark (‘) at beginning of line indicates line contains machine code.
VCAST_TORNADO_OLD_SCRIPT
When running vxWorks, do not complain about missing symbols. In VectorCAST version 5.0b, this
environment variable was renamed VCAST_DONT_CHECK_FOR_VXWORKS_UNRESOLVED_
SYMBOLS.
VCAST_USE_REMOVE_IMPORTS
Use Apex command to remove imports from copied views; otherwise, the data files are modified
directly.
VCAST_WHITEBOX
Force whitebox environment building. This environment variable overrides the setting of the Whitebox
option in the Tools => Options dialog, Builder tab.
VCV_EDIT_LINE_NUMBER
This environment variable is set when opening files in an external editor. This variable is set to the line
number of the selected function. The external editor can then be set to a script command that uses the
environment variable VCV_EDIT_LINE_NUMBER to go to a specific line number in the external editor.
When VCV_EDIT_LINE_NUMBER is set to 0, the file opens and sets the cursor to the top.
VCV_ENVIRONMENT_DIR
This environment variable is set to the full path to the current environment directory whenever the
environment is opened.
This environment variable is available to all commands invoked by VectorCAST from the open
environment, including:
> C_PREPROCESS_CMD,
> C_COMPILE_CMD,
> C_LINK_CMD,
> VCAST_CMD_DURING_ENV_OPEN,
> VCAST_CMD_DURING_ENV_CLOSE,
> VCAST_PRE_EXECUTE_CMD, and
> C_EXECUTE_CMD.
VCV_ENVIRONMENT_NAME
This environment variable is set to the name of the environment while the environment is open.
This environment variable is available to all commands invoked by VectorCAST from the open
environment, including:
> C_PREPROCESS_CMD,
> C_COMPILE_CMD,
> C_LINK_CMD,
> VCAST_CMD_DURING_ENV_OPEN,
> VCAST_CMD_DURING_ENV_CLOSE,
> VCAST_PRE_EXECUTE_CMD, and
> C_EXECUTE_CMD.
VECTORCAST_DIR
VECTOR_LICENSE_FILE
clicast help
clicast -x tgt
Language
Builder
Execute
Report
Target
Coverage
Command Format
Linux systems:
$VECTORCAST_DIR/clicast -eenv -uunit -ssub -ttestcase command arguments
Windows systems:
%VECTORCAST_DIR%\clicast /e:env /u:unit /s:sub /t:testcase command arguments
where env is the name of the environment, unit is the name of a unit under test, sub is the name of a
subprogram, and testcase is the name of a testcase. If the environment has only one UUT, then -u
unit is optional.
Throughout this User Guide, the corresponding CLICAST command is presented after each feature or
option available in the VectorCAST application. The CLICAST command uses the following format:
> The vertical bar | stands for “or”, indicating that one of several choices is to be included in the
command.
> Square brackets around a parameter indicate that a parameter is optional. For example,
indicates that the environment name must be specified, but the unit name can optionally be
<output file>
[<output file>]
indicates that if the optional output filename is not present, standard output is used.
> Capital letters in a command name indicate the minimum that needs to be typed to uniquely
identify the command. For example,
Tool Options
ADA_DEBUG_CMD
Description: This option will over-ride the default debug command that VectorCAST uses to invoke the
debugger. For example, the default debug command for GNAT is: gdb, if you prefer to use gvd, you may
enter that command here.
ADA_EXECUTE_CMD
Description: This option must be set when 'Execution process uses stdin/stdout for file I/O' is True.
When using this execution method, you provide the path to your custom script, batch file, or other
mechanism that handles downloading to the target, test execution, and any other preparation or post-
processing. The script is passed a single argument with the name of the test harness executable
(UUT_INTE for the normal harness, or UUT_INST for the coverage-instrumented harness). Use this
option to override the default test execution method for your particular target. (This option will override
the 'Execute command' option).
ADACAST_IO_BODY
ADACAST_IO_SPEC
BINDER_OPTIONS
Description: GNAT Only. Command-line options to be used when invoking the binder.
COMPILATION_SYSTEM
Description: Ada compiler to be used to compile and link the test harness.
COMPILER_OPTIONS
COMREADER_BAUD
COMREADER_COMPORT
COMREADER_DATA_BITS
COMREADER_ENABLED
COMREADER_PARITY
COMREADER_STOP_BITS
COVERAGE_ANIMATION_PLAY_SPEED_RATIO
Description: By default, coverage animation proceeds at the rate of one line per second. To go faster,
increase the value. To go slower, decrease the value.
COVERAGE_IO_TYPE
Description: Type of coverage I/O that the instrumented harness uses to record coverage data.
> Real time coverage I/O gathers coverage data and writes it to the results file immediately.
> Buffered coverage I/O gathers coverage data and buffers it in memory until the end of the test
case, or on test harness termination if the option VCAST_DUMP_COVERAGE_AT_EXIT is set
to TRUE. For Cover environments, buffered I/O gathers coverage data and buffers it in memory
until VCAST_DUMP_COVERAGE_DATA() is called. If the option VCAST_DUMP_
COVERAGE_AT_EXIT is set to TRUE then the data is instead written to the results file when the
application terminates and no call to VCAST_DUMP_COVERAGE_DATA() is needed.
> Animation coverage I/O writes data to the results file in the order it was encountered, and does not
remove duplicates. Animation coverage I/O is required for Basis Path coverage, and supports
replaying the order of execution of the code in the Coverage Viewer.
COVERAGE_TARGET
COVERAGE_TYPE
EVENT_LIMIT
Description: Maximum number of events for the harness to process. The harness will abort execution
when this limit is exceeded. This is useful for testing code with infinite loops. If this limit is 0, no results
data will be captured (coverage data will still be captured).
EXECUTABLE_EXTENSION
EXECUTE ALL
Description: This command executes all test cases in the specified environment. Regardless of the
pass or fail status of the test execution, if a report file name is given, then additionally a Testcase
Management Report is generated with the given filename. The exit status from the 'execute all'
command is zero if all tests pass and non-zero otherwise.
FLOATING_POINT_TOLERANCE
Description: Percentage that a floating point value can vary from its expected value and still be
considered a match.
GLOBALS_DISPLAY
Description: When to display / check global values in execution results: at each event; at the end of
each range iteration; at the end of each slot iteration; at the end of each slot.
GNATCHOP_OPTIONS
GNATMAKE_OPTIONS
Description: GNAT Only. Switches used to control behavior of gnatmake (such as '-j8' to use multiple
processes).
GPR_OPTIONS
Description: GNAT Only. Switches used to control behavior of GPR file (such as '-Xname=value' for
GPR external variables).
LIBRARY_OPTIONS
Description: Command-line options to be used when creating the test Ada library.
LINES_PER_PAGE
LINKER_OPTIONS
MANAGE_ENVIRONMENT_VARIABLE
Description: Environment variables used when building, executing, importing, or executing python
scripts.
MANAGE_PARENT_LIB
MANAGE_ROOT_SOURCE_DIR
MAX_VARY_RANGE
Description: Maximum number of scalars that can be varied in one test case. Changes to this value will
take effect after the environment is rebuilt. Reducing this value to 0 will completely remove list and
range processing from the test harness, significantly reducing the size of the harness.
METRICS_TOTAL_LEVEL_DISPLAY
Description: This option allows you to specify which results to filter out.
NO_DELAY_STATEMENTS
Description: If this option is set, the harness will be built with no 'delay' statements (typically used for
allowing tests with tasks to run more smoothly).
NUMBER_OF_DATA_PARTITIONS
Description: Number of iterations to span an entire range of a scalar parameter. Used with Range
Values to allow specification of iterations instead of range delta.
PAGE_WIDTH
PREPARE_COMPILED_IN_DATA
Description: Path to your custom script, batch file, or other mechanism that will read the standard input
file generated by the tool and generate source code to populate the file input object in the test harness.
This script is executed every time the harness is run; it is immediately followed by a compile of the input
object and a link of the executable.
PREPROCESSOR_OPTIONS
Description: Command-line options to be used when invoking the Ada preprocessor (only available for
some compilers).
RANGE_CHECK
Description: Indicates whether range checking should be performed for input and expected test case
values. 'All', means that type ranges will be enforced, and out-of-range test data will not be accepted.
'Disable' allows out-of-range test data values to be used. 'None' disables all range processing. This is
useful when target communication is slow.
REPORT_OBJECTS
Description: When preparing execution reports, if this option is set, VectorCAST will display ALL global
objects. If the option is not set, then only global objects that are part of the test case (as input or
expected values) will be displayed.
REPORT_PARAMETERS
Description: When preparing execution reports, if this option is set, VectorCAST will display ALL
parameters for each test event. If the option is not set, then only parameters that are part of the test
case (as input or expected values) will be displayed.
SHOW_NOTES_IN_FULL_REPORT
Description: This option will add a section to the Full Report listing test notes and requirements.
STANDARD_ERROR
Description: If this option is set, any standard error from the test will be captured to a file for inclusion in
the test execution report.
STANDARD_OUTPUT
Description: If this option is set, any standard output from the test will be captured to a file for inclusion
in the test execution report.
STUB_DEPENDENCIES
Description: Default setting for Stub Dependencies option in the environment builder. ALL means stub
all dependencies. NONE means do not stub any dependencies.
TARGET_AE653_MAKE_FILE
Description: This option provides the command used to perform a 'make' for the vxWorks 653 target.
TARGET_APEX_LIBRARY
Description: For Apex Exec. This option tells VectorCAST which ada library to use. If this option is set
to Full, the full Ada run-time library will be used. If this option is set to MARK, the reduced run-time will
be used and the test harness will be built with the appropriate restrictions.
TARGET_BOARD_NAME
Description: Name of target board. For vxWorks this is the name of the target server, for other targets it
is the hostname or IP address of the target.
TARGET_BOOT_HOSTNAME
Description: For vxWorks. This is the name that the target knows the boot host as. If this option is not
set, VectorCAST assumes that the hostname of the machine it is running on, is the boot host.
TARGET_BUILD_FILE
Description: For Green Hills Compilers only. This is the name of the default build file. The '.bld'
extension should NOT be used (e.g. ppc, sim800). Builder flags can also be provided in this option (e.g.
:language=ada).
TARGET_COMMAND_OPTIONS
TARGET_COMMAND_VERB
TARGET_CPU
Description: Used for SingleStep debugger. The name of the 'CPU' to be passed to the 'debug'
command.
TARGET_INTEGRITY_ADDRESS_SPACE_MODE
Description: For Green Hills Integrity. This option tells VectorCAST how to link the application. If this
option is set to Kernel, the harness will be linked with the OS as a kernel space application. If the option
is set to Virtual, a virtual address space application will be created.
TARGET_INTEGRITY_LIBRARY
Description: For Green Hills Integrity. This option tells VectorCAST which ada library to use. If this
option is set to Full, the full Ada run-time library will be used. If this option is set to GMART, or
GSTART, the reduced run-time will be used and the test harness will be built with the appropriate
restrictions.
TARGET_IO_DIRECTORY
Description: Directory where Input/Output files for target test execution will be stored. This option
should be used when the target does not have read/write permission for the test environment directory.
TARGET_SIM_CMD
TARGET_SUPPORTS_TIC_IMAGE
Description: Set this option to True when the target run-time does not support the 'image by default, but
the run-time being used has been modified to support 'image.
TARGET_TFTP_DIR
Description: This is the directory that the executable files will be copied to so that the target can be
booted via tftp.
TARGET_USE_RTSERV_COMM
Description: If this option is set, and you are building a Kernel application, VectorCAST will copy the
executable files to the tftp directory, and then prompt you to manually boot the target board. This option
is assumed set if you are building a Virtual Address Space applicaton
TARGET_USE_TORNSERV
TARGET_VARIANT
Description: This option describes the compiler / target combination used to build an environment.
TARGET_XFER_METHOD
Description: Method of target data transfer. This option controls the value passed to the Aonix transfer
command when a test is run.
TEST_CASE_TIMEOUT
UNCON_ARRAY_SIZE
Description: Default size for unconstrained arrays where the index range is unknown or very large. May
be specified as [lower bound]:[upper bound] or just [upper bound].
VCAST_ADA_ACCESSOR_FUNCTIONS
Description: If this option is set, VectorCAST will access Ada options from accessor functions. If this
is set to false, VectorCAST will access Ada options from global objects.
VCAST_ADA_FILE_EXTENSIONS
VCAST_ADA_LANGUAGE_SPECIFICATION
VCAST_ADAOPTS
Description: Set this option to tell VectorCAST/Ada that you are using a version of the ObjectAda
compiler that is newer than version 7.2. If 'adaopts -l -u' reports an error, this flag needs to be set.
VCAST_ADD_DONT_STUB_UNITS
Description: This option will force the environment builder to add units specified as DONT_STUB in the
environment script into the environment database. This is especially useful to get custom coverage on
units that have otherwise been ignored.
VCAST_ALLOW_STUB_OF_STD_LIB
Description: Set this option if you want to enable stubbing of the standard Ada packages (VCAST_IO,
SYSTEM, etc).
VCAST_APEX_CLEARCASE
Description: This option tells VectorCAST that Apex is being used with the ClearCase CM tool rather
than the default Summit CM Tool. In ClearCase mode, the structure of the Apex views and sub-
systems is quite different. This option allows VectorCAST to accommodate those differences.
VCAST_APEX_INSTALLATION_DIR
Description: Comma-separated list of strings to search for in paths to imported subsystems that
identify standard ada libraries that VectorCAST will not parse/stub. Default (unspecified) value for this
option is 'base/ada'.
VCAST_APEX_MUTUAL_IMPORTS
Description: When this option is set, VectorCAST will process source files found in the mutual imports
of the parent view.
VCAST_APPEND_TO_TESTINSS
Description: This option tells VectorCAST to open the coverage data file for appending, rather than
always creating the data file.
VCAST_CMD_AFTER_INSTRUMENTATION
Description: This command will be run after VectorCAST has instrumented a file.
VCAST_CMD_DURING_ENV_CLOSE
Description: Command to be executed while closing a unit test environment, where <cmd> is an
executable application or script to be executed in the environment directory. The command is executed
just as the environment begins to close. If running a script, the interpreter should be specified.
Note that when using clicast, the user should be aware that every command taking the argument -e
<env> opens and closes the environment. Therefore, the options can be set immediately before calling
the clicast command of interest, and unset afterwards.
VCAST_CMD_DURING_ENV_OPEN
Description: Command to be executed while opening a unit test environment, where <cmd> is an
executable application or script to be executed in the environment directory. The command is executed
right before building the range data during environment open. If running a script, the interpreter should be
specified.
Note that when using clicast, the user should be aware that every command taking the argument -e
<env> opens and closes the environment. Therefore, the options can be set immediately before calling
the clicast command of interest, and unset afterwards.
VCAST_COMMAND_LINE_DEBUGGER
Description: This option causes VectorCAST to bring up a shell window to run the debugger.
VCAST_COMPACT
Description: This option causes VectorCAST to compress the execution results for scalar array
elements that are the same. If a five element array has all values = 0, this option will enable the report to
have one output line of (1..5) rather than 5 lines of one value per line.
VCAST_COMPILE_DUMB_STUBS
Description: When a dependent unit is stubbed, VectorCAST creates a 'dumb' stub for any unit 'with'ed
by the stubbed units specification. By setting this option, these 'dumb' bodies will be compiled and
linked into the harness, allowing the execution closure to be completed with less effort.
VCAST_COMPILE_GENERIC_STUBS
Description: If you set this option, VectorCAST/Ada will compile stubs that are created for generic
packages.
VCAST_CONFIGURABLE_ADA_IO
Description: VectorCAST will generate VCAST_STDIN.DAT containing all input data for harness, and
assume all output data will be written to standard output.
VCAST_CONVERT_OCTAL_HEX_STRINGS_TO_ASCII
Description: This option is used to determine whether or not to convert "unprintable" strings encoded in
octal or hexidecimal notation to ASCII encoding in the report output. It does not affect the actual string
comparisons in the test execution.
VCAST_COVER_SEPARATE_TYPES_FILES
VCAST_COVERAGE_DATABASE_CACHE_SIZE
Description: Increasing the maximum coverage database cache size may increase performance, but
take care not to set it to a value greater than the amount of available system memory. For a system with
2GB, a 1000 MB maximum cache is probably sufficient. Changes to this option take effect when re-
opening an environment.
VCAST_COVERAGE_DATABASE_SIMPLIFIED_LOCKING
Description: Certain file system configurations, especially some using NFS, do not have a properly
functioning POSIX locking implementation. Enabling this option makes the database layer use an
alternate, simpler locking method which should allow the database to function on any file system.
However, when enabled, only one user or instance of VectorCAST will be able to access the database
at a time.
VCAST_COVERAGE_FIELD_WIDTH
Description: The width (in characters) of the left margin of the coverage viewer for C/C++ files
instrumented prior to VectorCAST 2020 and all Ada files. Increase this number if you have a large
number of subprograms or a subprogram has a large number of statements or branches.
VCAST_CUSTOM_REPORT_FORMAT
VCAST_DEFAULT_SCRIPT_HEADER
Description: This option allows you to enter a path to a text file that contains a custom header for all
VCAST_DETECT_UNUSED_EXPECTED_UC
Description: When enabled, this option detects test-specific expected user code that has not been
executed during a test. Such code is most likely to be associated with a stub that has not been called.
When True and some testcase or parameter user code has not been executed, the test fails, and the
Execution Report displays the unused values, the unit, and whether that unit is stubbed or not-stubbed.
Disable this option if you have added parameter user code to a stub that you expect to never get called.
(You might add code to an uncalled stub if you want the test to show a failure if the stub ever gets
called.)
VCAST_DISABLE_AUTO_BASIS_PATH_GEN
Description: When this option is set, VectorCAST will not generate Basis Path information unless
specifically requested by the user.
VCAST_DISABLE_BOOLEAN_CAST
Description: Select this option to prevent VectorCAST from casting conditional expressions to Boolean
when instrumenting for branch or MC/DC coverage.
VCAST_DISABLE_DIRECT_ARRAY_INDEXING
Description: This option is used to allow VectorCAST to know how to display the subscripts of an array
in the Parameter Tree and in the test script. For example, if you have an array subscripted from -10 to
10, if this option is set, VectorCAST displays the subscripts from 1 to 21. If the option is not set
(default), VectorCAST displays the subscripts from -10 to 10.
VCAST_DISPLAY_EMPTY_COVERAGE
Description: Display annotated source code in reports even if no run-time coverage data exists. If this
option is disabled, the message "No Coverage Data Exists" will be displayed instead.
VCAST_DISPLAY_FUNCTION_COVERAGE
Description: Extra column that reports if a function was entered. This option is only used if the coverage
type is not Function.
VCAST_DO_COMBINATION
Description: When this option is on, and multiple parameters have a range of values, all combinations of
the range values will be used as input data. (e.g. Two parameters being varied from 1..5 will result in 25
calls to the UUT) If this option is grey, it means that the open environment does not support
combination testing. Rebuild the environment to use this feature.
VCAST_DONT_STUB_ANCESTORS
Description: If this option is set, VectorCAST will not prompt for stubbing ancestor packages of the Unit
Under Test. Also, VectorCAST will add any sub-programs defined in the ancestors to the sub-program
list of the Unit Under Test.
VCAST_EMPTY_TESTCASE_FAIL
Description: If this option is set, then empty test cases will be marked as failed.
VCAST_ENHANCED_FOR_LOOP_COVERAGE
Description: When this option is true, coverage for Ada 'for' statements will show both True if the loop
condition has ever evaluated as True, and False if it has ever evaluated as False. When this option is
not set, only the True condition is checked.
VCAST_ENVIRONMENT_FILES
Description: These files will be copied into the environment directory during an environment build or
rebuild.
VCAST_EXECUTE_WITH_STDOUT
Description: This option tells VectorCAST to compile input test data into the harness, and map
standard output to VCAST_STDOUT.DAT. The normal Execute Command is still used to execute the
test.
VCAST_EXPECTED_BEFORE_UUT_CALL
Description: If a stub is called before the first UUT call, allow comparisons of expected values during
the stub execution.
VCAST_EXTENDED_EXCEPTION_INFORMATION
Description: When an exception is propagated out of a UUT call, setting this flag to TRUE will cause
ADA.EXCEPTIONS.EXCEPTION_MESSAGE to be called.
VCAST_FIELD_DOT_NOTATION
Description: This option is used to tell VectorCAST's execution results and test case data listing
reports to use field dot notation whenever a field name is printed.
VCAST_FILE_ENCODING
Description: Indicates what file encoding is used for source files being processed by VectorCAST.
<encoding format> is one of the following:
VCAST_FILE_INDEX
Description: Select this option to enable VectorCAST to use a file indexing mode that significantly
reduces file I/O required for running coverage or unit tests.
VCAST_FILE_VERSION_COMMAND
Description: This command will be executed for each source file in the test environment, and a section
will be added to the Full report and Test Case Management report listing the name of each unit in the
test environment and the information generated by this command. If this command is empty, then the
File Version section of the report will not be created.
VCAST_FLOAT_PRECISION
Description: Controls the number of digits used when the test harness prints floating point Actual
Values for the Execution Report or Range Data. This option's setting is significant in the comparison of
Expected Values for floating point numbers as well as determining the min and max values.
In an Ada environment, floating point values are printed in scientific notation and the precision value
controls the number of digits to the right of the decimal, with a single digit to the left of the decimal. A
precision of 0 results in the maximum avaliable digits being printed.
VCAST_FORCE_REPORT_DATES
Description: This option will force all the dates and times in the report configuration table and the
management table to all be the same. This helps diff the reports much more easily.
VCAST_GEN_ALL_EXECUTION_REPORT_FORMATS
Description: This option is used to determine whether or not to generate the execution reports in all the
available formats (e.g. HTML or TEXT).
VCAST_GH_INT_FILE
Description: This is the custom integrate file passed to the Green Hills 'intex' command. This file should
follow the general format of the default file found in the VectorCAST installation directory, which means
it should contain a 'Filename' line with the text VCAST_FILE (to be replaced with the VectorCAST
executable name) and a 'Starit' line with the value 'true'.
VCAST_GH_INTEX_CMD
Description: When set, VectorCAST will invoke the Green Hills intex utility with this command
immediately after linking the test harness. Refer to the VCAST_GH_INT_FILE environment variable for
information on how to specify a custom integrate file. Example: intex -
kernel=C:\GHS\int408\sim800\kernel -bspdir=C:\GHS\int408\sim800 -
target=C:\GHS\int408\sim800\default.bsp vcast.int.
VCAST_GH_PP_COMMAND
Description: This option allows you to assign a preprocessor for VectorCAST to use on ADA files when
using the Green Hills compiler.
VCAST_GMART_FILE_IO
Description: If this option is set, VectorCAST will use File I/O to communicate with the target. If it is not
set, asynchronous monitor communication will be used.
VCAST_GNAT_INCLUDE_PATH
Description: GNAT Only. This value will override the ADA_INCLUDE_PATH environment variable
VCAST_GNAT_OBJECTS_PATH
Description: GNAT Only. This value will override the ADA_OBJECTS_PATH environment variable
used by GNAT to find source files.
VCAST_GPR_TEMPLATE
Description: This file will be used by VectorCAST to make a local GPR build structure to be used as the
parent for the VectorCAST environment.
VCAST_GPR_USES_NAMING
Description: The GNAT Project file uses 'package NAMING' to override GNAT's default file naming
conventions.
VCAST_IGNORE_INCOMPLETE_TESTS
Description: When building Basis Path or MC/DC test cases there are three outcomes for each test:
Complete, Partial, and Template. When this option is on, VectorCAST will discard the Partial and
Template tests, and only load the Complete tests.
VCAST_IMPORT_FOREIGN_OBJECTS
Description: When the harness needs to create an object at a specific memory location with the 'use at
<address>' construct, if this option is set to TRUE then a 'pragma import' command will be used to
control compiler initialization of the object.
VCAST_INSTRUMENT_ASSIGNMENTS
Description: Setting this option will cause assignment statements with Boolean operators or the
conditional operator '?' to be covered by branch and MC/DC coverage, unless the statement is
immediately preceded by the comment: VCAST_DONT_DO_MCDC. If this option is not set, MC/DC
will still be done on assignment statements preceded by: VCAST_DO_MCDC.
VCAST_INSTRUMENT_PARAMETERS
Description: Setting this option will cause parameters in function calls to be covered by branch and
MC/DC coverage.
VCAST_MAX_COVERED_SUBPROGRAMS
Description: If 'Use static memory allocation' is True and Coverage I/O is Buffered, VectorCAST must
allocate a specific amount of space to store coverage data each time a different subprogram call is
encountered. This number tells VectorCAST how many different subprogram calls for which it should
set aside memory.
VCAST_MAX_MCDC_ABORT_INST
Description: While instrumenting with MC/DC, if a conditional statement contains more conditions than
possible to instrument, then abort instrumenting that file. For Ada, the maximum is 26.
VCAST_MAX_MCDC_CONDITIONS
If an expression exceeds this limit, equivalence pairs are calculated as results are added to the
environment, and only table rows that contribute to a satisfied pair are displayed.
VCAST_MAX_MCDC_STATEMENTS
Description: If 'Use static memory allocation' is True, VectorCAST must allocate a specific amount of
space to store coverage data each time an MC/DC expression is encountered. This number tells
VectorCAST how many MC/DC expressions for which it should set aside memory.
VCAST_MAX_STRING_LENGTH
Description: This specifies the size of temporary character arrays that VectorCAST creates in the test
harness. Longer strings are needed specifically when test cases use long lists of values as stimulus
data. For example, a stimulus value for the sine function of: 10,20,30,40,50,60 ... If you are running in a
target environment with limited heap and stack resources, making this value smaller will reduce the
VectorCAST test harness use of heap and stack. Changes to this value will take effect after the
environment is recompiled.
VCAST_MAX_TABLE_SUBCONDITIONS
Description: When VectorCAST generates equivalence matrices for an expression, if the expression
has more than this many subconditions, no table information will be displayed. Pair information will still
be calculated.
VCAST_MAX_TARGET_FILES
Description: This option limits the total number of files that the test harness is allowed to open while
running a compound test using 'Execution process uses stdin/stdout for file IO' (VCAST_
CONFIGURABLE_ADA_IO is TRUE). This option is useful for executing compound tests on a target
with limited memory. If set too low, test execution for the test does not start. Environment must be
recompiled for changes to take effect.
VCAST_MULTIBYTE_CHARACTERS
Description: Enable this clicast-only option if source code files are encoded using Shift JIS.
VCAST_NEW_ARCHITECTURE
Description: Set this option to allow manipulation of private types from all units in the environment, not
just the Unit Under Test.
VCAST_NO_EXCEPTION_HANDLERS
Description: If this option is set, the harness will be built with no exception-catching capabilities.
VCAST_NO_RECURSIVE_CALLS
Description: By default, the VectorCAST test harness uses recursive calls to improve processing. If
the 'NO_RECURSION' restriction is activated in the code under test, the harness needs to be built
differently.
VCAST_NO_STANDARD_PKG_USAGE
Description: By default, the Multiunit Whitebox harness redefines TRUE and FALSE by using
STANDARD.BOOLEAN. This option disables that override, necessary when user source files use
'STANDARD' in a different context.
VCAST_NOTES_SECTION_TEMPLATE
Description: This option allows you to enter a path to a text file that contains a template for the Notes
section of the test case.
VCAST_OLD_STYLE_MANAGEMENT_REPORT
Description: When set, this option reverts the Testcase Management Report to the behavior found in
VectorCAST version 6.0 and prior, in which the main body of the report combined the Expecteds and
Control Flow in one fraction in the "Pass/Fail" column.
VCAST_POST_LINK_COMMAND
VCAST_PRAGMA_IMPORT
Description: This option tells VectorCAST to build stubs for subprograms in stubbed units that are
defined using 'pragma import'.
VCAST_PRE_EXECUTE_CMD
Description: The specified executable or batch file script is executed before the test harness is called
during test case execution. Useful for rebooting your target, for example. If a relative path to this
command is used, it should be relative to the environment directory.
VCAST_PRECOMPILE_SCRIPT
Description: This command is executed prior to full compilations of the test harness. It can be used to
make modifications to harness files before the initial compilation. Note that this command may also be
called for subsequent harness compilations, but it is not called before every compilation.
VCAST_PREPEND_TO_PATH_DIRS
Description: These directories will be prepended to the PATH environment variable value. This can be
used to put your compiler on the PATH. Multiple directories should be separated by ';' (Windows) or ':'
(Linux). Note that some compilers require other environment variables to be set as well in order for the
compiler to work properly.
VCAST_PSC_MAKE_FILE_BASE_DIRECTORY
Description: This option provides the path to the vxWorks 653 build directory containing the existing
partition build. This directory should contain a directory (with the same name as the Target Board
VCAST_RAVEN_FILE_IO
Description: Run Raven target using file I/O instead of standard output.
VCAST_RECOMPILE_SEPARATES
Description: This option tells VectorCAST that you want to recompile the body of the Unit Under Test
before linking, but after all stubs are compiled. This is useful when the Unit Under Test calls 'in-lined'
subprograms from stubbed units.
VCAST_REFERENCED_GLOBALS
Description: When this option is on, the VectorCAST test case editor will display only those global
objects and stubs that are referenced by the function under test.
VCAST_REPOSITORY
VCAST_RPTS_AUTOMATIC_BGCOLOR
Description: This color is used for text indicating automatic initialization and finalization tests.
VCAST_RPTS_BODY_BGCOLOR
VCAST_RPTS_COMPLEXITY_COLUMN_WIDTH
VCAST_RPTS_COVERAGE_RESULT_COLUMN_WIDTH
VCAST_RPTS_CUSTOM_CONFIG
Description: Custom report configuration. If this option points to an existing file, then the contents of
that file is used as the custom report configuration, otherwise the contents of the option is used.
VCAST_RPTS_CUSTOM_CSS
Description: Reports use the default cascading style sheets (*.css) located in $VECTORCAST_
DIR/python/vector/apps/ReportBuilder/css/, unless a custom style sheet is specified in
this option. Environment variables in the path to the custom style sheet are supported, using this syntax
$(ENV_VAR). When used, the contents of the custom style sheet are embedded in each report,
between the <style>...</style> tags in the header section after the default styles allowing for
CSS style overriding.
VCAST_RPTS_DATE_COLUMN_WIDTH
VCAST_RPTS_DEFAULT_FONT_COLOR
VCAST_RPTS_DEFAULT_FONT_FACE
VCAST_RPTS_DEFAULT_FONT_SIZE
VCAST_RPTS_EMPTY_DATA_STRING
Description: Text to show in blank cells in the Execution and Test Case Data report sections. The
default value is unset.
VCAST_RPTS_DELIMITER
Description: Character separator used in two CLICAST report commands: cover report cvs_metrics
and reports alternate. To enter a tab, use \t.
VCAST_RPTS_FAIL_COLOR
Description: This color is used for text indicating failing test cases or failing totals.
VCAST_RPTS_HEADER
Description: Prepend a text string to the title or add a new section to the Full Report, just below the title,
to include the contents of a file in TEXT or HTML format. Input can be a text string, or a file containing
several lines. If an HTML file is provided, the contents should provide a <div> section with a table. For
example:
<div class='report-block'>
<h2>Additional Data</h2>
<table class='table table-small'>
<tr><th>Tests created by</th><td>Fred Williams</td></tr>
<tr><th>Source revision</th><td>123abc789</td></tr>
</table>
</div>
VCAST_RPTS_HEADING_FONT_SIZE
VCAST_RPTS_NOTES_COLUMN_WIDTH
VCAST_RPTS_PASS_COLOR
Description: This color is used for text indicating passing test cases or passing totals.
VCAST_RPTS_PRETTY_PRINT_HTML
Description: This option enables whitespace indentation in HTML and XML report-related files. It is off
by default to allow better performance and reduced memory usage.
VCAST_RPTS_REQUIREMENTS_COLUMN_WIDTH
VCAST_RPTS_RESULT_COLUMN_WIDTH
VCAST_RPTS_SECTION_TITLE_BGCOLOR
VCAST_RPTS_SELF_CONTAINED
Description: When True, HTML reports are created as a single, self-contained file with an embedded
stylesheet and embedded images (for all built-in, non user-modified reports). Set the option to True
(default value) when planning to email reports. When False, HTML reports are created as multiple files,
storing stylesheets and images in the same directory as the report. Set the option to False when serving
reports from a webserver.
VCAST_RPTS_SHOW_VERSION
Description: Show the VectorCAST version in the configuration section of the reports.
VCAST_RPTS_SUBPROGRAM_COLUMN_WIDTH
VCAST_RPTS_TABLE_BORDER_SIZE
VCAST_RPTS_TABLE_CELL_PADDING
VCAST_RPTS_TABLE_DATA_BGCOLOR
VCAST_RPTS_TABLE_DATA_FAIL_BGCOLOR
Description: This background color is used for failing data cells in reports and tables.
VCAST_RPTS_TABLE_DATA_PARTIAL_BGCOLOR
Description: This background color is used for partially passing data cells in reports and tables, as well
as warning messages in reports.
VCAST_RPTS_TABLE_DATA_PASS_BGCOLOR
Description: This background color is used for passing data cells in reports and tables.
VCAST_RPTS_TABLE_DATA_TEXT_ALIGNMENT
Description: Text alignment (left, right, or center) for table data in HTML reports.
VCAST_RPTS_TABLE_HEADING_BGCOLOR
VCAST_RPTS_TESTCASE_COLUMN_WIDTH
VCAST_RPTS_UNIT_COLUMN_WIDTH
VCAST_RPTS_USER_REPORTS_DIR
VCAST_RPTS_WRAP_NOTES
Description: This option will cause the text in the notes section to wrap at the first space before 80
characters.
VCAST_SCRIPT_EXPAND_ARRAYS
Description: By default, when an array of scalars is written to a test script, identical values are
condensed to one script line. By setting this option, each array element will get its own script line.
VCAST_SCRIPT_IN_CREATION_ORDER
Description: By default, test scripts are exported in alphabetical order within unit and subprogram.
Setting this option will cause the script to be generated in the order that the test cases were created.
VCAST_SCRIPT_LOG_SHOW_ONLY_ERRORS
Description: In script logs for large test scripts, error messages can be overwhelmed by the number of
status messages. Setting this flag prevents normal status messages from being added to the log - only
error messages remain.
VCAST_SHOW_ONLY_DATA_WITH_EXPECTED_VALUES
Description: When this option is set, global and parameter data will only be displayed in the test
execution report if expected values are provided. This option will result in smaller execution reports and
faster test execution in cases where there are is a lot of global and/or parameter data input.
VCAST_SHOW_ONLY_EVENTS_WITH_EXPECTED_VALUES
Description: When this option is set, events will only be displayed in the test execution report if
expected values exist for that event. This option will result in smaller execution reports in cases where
there are is a lot of global and/or parameter data input.
VCAST_SHOW_STDOUT_CONSOLE
Description: When this option is set, a new console will be opened during test harness execution to
facilitate Standard I/O. If 'Redirect standard output' or 'Redirect standard error' is set, this option is
ignored. (Windows Only)
VCAST_SIMPLIFIED_CONDITION_COVERAGE
Description: This option suppresses the computation and reporting of Equivalence Pairs when MC/DC
Coverage is active. Since Equivalence Pairs are not computed, MC/DC tests cannot be generated.
This option is primarily used for Medical Device testing, and is defaulted to True when IEC-62304
(Medical) Class C Code Coverage is selected. The default is False for all other cases.
VCAST_SORT_METRICS_RPT_BY_DIR
VCAST_SPEC_FILE_EXTENSIONS
Description: Use this option to specify the ada spec file extensions.
VCAST_SPLIT_INSTRUMENTED_FILE
Description: Approximate number of lines in an source file. Used when source unit has separates, and
the compiler cannot handle a single file containing the unit and all its separates.
VCAST_SPLIT_LARGE_FILE
Description: When the Unit Under Test or a Custom Coverage unit contains 'separates', VectorCAST
will combine them into one file for parsing and instrumentation. Set this option if this combined file is too
large for the compiler. (Only works for Green Hills, ObjectAda, and ActivAda compilers.)
VCAST_STRICT_TEST_CASE_IMPORT
Description: Normally, if there are errors when importing a test script, VectorCAST will ignore any
object errors. When this option is selected, errors in the script will cause the testcase to not be allowed
to execute until the script is fixed or until the testcase is edited.
VCAST_SUPPORT_TAGGED_INHERITANCE
Description: When set, the tool will add functions defined in other units that contain a parent of a tagged
type defined in the current unit.
VCAST_SUPPRESS_COVERABLE_FUNCTIONS
VCAST_SUPPRESS_DEBUG_FLAG
Description: This option will force VectorCAST to compile source code without debug information.
VCAST_TEST_ANCESTORS
Description: When set, the tool will add functions defined in the ancestors of the UUT to the list of
functions that can be tested for the environment.
VCAST_TESTCASE_FAIL_ON_NO_EXP_RETURN
Description: If this option is set, then testcases that have no expected return value or expected user
code for the return value will be marked as failed. The default value is FALSE. Note that this option is
specific to the return value of the subprogram under test only.
VCAST_TESTCASE_FAIL_ON_NO_EXPECTED
Description: If this option is set, any test cases that have no expected results or expected user code
will be marked as failed. Its default value is FALSE.
VCAST_UNCONSTRAINED_STRING_SIZE
VCAST_UNCOVERED_LINE_INDICATOR
#$%&+
VCAST_UNEXPECTED_EXCEPTIONS_FAIL
Description: If this option is set, then if an exception is raised during the execution of a testcase and no
exception was expected, the testcase is marked as failed.
VCAST_UNEXPECTED_SIGNALS_FAIL
Description: If this option is set, then if a signal is raised during the execution of a testcase then
testcase is marked as failed.
VCAST_UNIT_TYPE
Description: Default setting for Unit type in Create New Environment Wizard. UUT allows stubbing
only for functions external to unit being tested; Stub By Function (SBF) allows stubbing of functions
within the unit being tested.
VCAST_USE_COMPOUND_FOR_BATCH
Description: This option allows you to run all of your test cases during a single test driver execution.
This is useful if there are many steps involved in running on your target platform. When this option is
set, and you choose to Batch Execute All, VectorCAST creates a temporary compound test case to
execute all tests. The temporary compound contains all Initialization test cases, followed by all simple
and Compound test cases.
VCAST_USE_GH_BLD_FILE
Description: Set this to an appropriate '.bld' file so that VectorCAST will compile and link using the
'build' command instead of compiling each file separately.
VCAST_USE_GNATMAKE
Description: This option forces VectorCAST to use 'gnatmake' to compile, bind and link the test
harness. If this option is not set, VectorCAST will use 'gcc' to compile the test harness files, 'gnatbind'
to bind, and 'gnatlink' to link.
VCAST_USE_GPRBUILD
Description: This option forces VectorCAST to use 'gprbuild' to compile/link environments built from
GPR files. This may slow the build process, as this command will compile objects in other languages
that are part of the build process.
VCAST_USE_RELATIVE_PATHS
Description: If set, the new environment dialog will use relative paths. The target I/O directory will also
use relative paths.
VCAST_USE_STATIC_MEMORY
Description: This option tells VectorCAST that only static data allocation can be used on the target
platform. This is used when collecting coverage data. If this is set to True, then use the options
VCAST_MAX_MCDC_STATEMENTS and VCAST_MAX_COVERED_SUBPROGRAMS to
configure the static allocation.
VCAST_USER_CODE_FILE
Description: Set this to a file containing environment user code (in scripting format) to get this code
automatically inserted into every environment built.
VCAST_USER_CONSTRAINTS
Description: List of array index types that have a large range and are used for unconstrained arrays.
VCAST_USING_GNAT_5
Description: Newer versions of GNAT use different syntax for the 'gnatchop' command. If 'gcc --
version' returns 3.4.4 or greater, this option should be set to TRUE.
VCAST_VERBOSE_MANAGEMENT_REPORT
Description: This option will copy the first line from the 'Test Notes' box in the test case editor to the
test case management report. One possible use for this is for test requirement numbers to be correlated
to test cases in the test case management report.
VCAST_VXWORKS_RTP_MODE
Description: Set this option to run VxWorks executables as a Real-Time process (via the 'rtpSp'
command) rather than as a standard process (via the 'ld' and 'sp' commands).
VCAST_WIDE_CHARACTER_SUPPORT
Description: Allow VectorCAST to parse Ada code containing wide characters by using ADA.WIDE_
TEXT_IO when reading code. When this option is off, VectorCAST reverts to using ADA.TEXT_IO.
VCAST_WRAP_TEXT_REPORT
Description: In some text reports, if text is too long for a table cell then this option will allow the text to
wrap in a new row.
WHITEBOX
Description: Default setting for Whitebox checkbox in environment builder. (See the Whitebox
Processing section of User's Guide for more details.)
Requirements Gateway 3.0 supports the following RGW operations from the command line:
Note: Command line operations for RGW must be run from a directory with a CCAST_.CFG file
that indicates where the repository is located.
When RGW begins, it loads the ADACAST_.CFG file located in the current working directory and looks
for the repository path in VCAST_REPOSITORY. If it cannot locate an entry for VCAST_
REPOSITORY, it returns this text: "VCAST_REPOSITORY not set in .CFG file. Please set it using
'clicast option'."
where:
<command> is the command to execute
<arguments> are the arguments for the command
RGW Commands
Configure Create_subsystem
Syntax clicast -lada RGw Configure Create_subsystem
<Subsystem Name> <Python Script Name>
- Polarion
- Jama
csv_gateway.py
jama_gateway.py
polarion_gateway.py
Upon execution, the script checks the CCAST_.CFG file for the path to
the repository. If it is unable to locate the RGW repository, it returns an
error.
Configure Get
Syntax clicast -lada RGw Configure Get <Settings Profile
Name> <Option Name>
Description This command returns any of the configuration options for a specific
subsystem. Use Configure Test to determine the names of options for
a particular subsystem.
Configure Interactive_setup
Syntax clicast -lada RGw Configure Interactive_setup
<Subsystem Name> <Script Mode>
Description Runs an interactive setup for the specified Subsystem. Leave Script
Mode blank, and the Python script will report all available modes. Valid
modes are Import and Export.
Configure Report
Syntax clicast -lada RGw Configure Report <Subsystem
Name>
Configure Set
Syntax clicast -lada RGw Configure Set <Subsystem Name>
<Option Name> <Option Value>
Description This command sets any of the configuration options for a particular
subsystem. Use Configure Test to determine the names of options that
need to be set for a particular subsystem.
Configure Test
Syntax clicast -lada RGw Configure Test <Subsystem Name>
<Mode>
Descriptio This command tests the RGW configuration for the subsystem identified by
n <Subsystem Name>. It first checks that all required options are set for the
particular subsystem, and then performs a connection test or document
existence check. It passes the <Mode> into the script, which then performs
the test.
Export
Syntax clicast -lada RGw Export <Subsystem Name>
Description This command exports Requirements Test Data from the RGW
Repository to an external Requirements System.
Import
Syntax clicast -lada RGw IMport <Subsystem Name>
Description This command imports Requirements into the RGW Repository from
an external Requirements system.
- Polarion
- Jama
Initialize
Syntax clicast -lada RGw INitialize
Requirement Add
Syntax clicast -lada RGw Requirement Add <Requirement
Group Name>, <Requirement Key>, <Requirement ID>,
<Requirement Title>, <Requirement Description>
Requirement Clear_all
Syntax clicast -lada RGw Requirement Clear_all
Description This command clears ALL requirements from the RGW Repository. If
any test cases are linked when trying to clear requirements from a
repository, they must be unlinked first.
Requirement Remove
Syntax clicast -lada RGw Requirement REMove <Requirement
Key>
Requirement Report
Syntax clicast lada RGw Requirement REPort <Query Type>
<Query ARg 1> <Query Arg2>
Description This command returns Requirements that match the supplied query.
Requirement Update
Syntax clicast -lada RGw Requirement Update <Requirement
Group Name> <Requirement Key> <Attribute Key>
<Attribute Value>
Description This command updates a single requirement attribute with a new value
for the Requirement specified by <Requirement Key>.
Testcase Link
Syntax clicast -lada -e <env> -u <unit> -s <sub> -t
<test> RGw Testcase Link <Requirement Key>
Description This command links a Requirement to the Test Case specified by the
Test Case identification flags: -e -u -s -t.
Testcase Migrate_links
Syntax clicast -lada -e <env> RGw Testcase Migrate_links
Description This command migrates all legacy test case requirement links from
the specified Environment into the Requirements Repository.
Testcase Report
Syntax clicast -lada -e <env> -u <unit> -s <sub> -t
<test> RGw Testcase Report
Description This command returns all Requirements linked to the Test Case
specified in the flags.
Testcase Unlink
Syntax clicast -lada -e <env> -u <unit> -s <sub> -t
<test> RGw Testcase Unlink <Requirement Key>
Description This command removes the link between a Requirement and a Test
Case specified by the Test Case identification flags: -e -u -s -t.
rgw_clean.py
Syntax rgw_clean.py [-h] [--no_backup] [--at_date AT_
DATE] DATATYPE REPOSITORY_DB_FILE
Location $VECTORCAST_DIR/python/vector/apps/RGWUtility
Description This is an RGW python script which cleans out the historical
Requirements and test execution data from an RGW Repository
database.
Positional arguments:
DATATYPE
Specify the data type to operate on. This can be one of:
requirement/req (the specified operation works on Requirements
only), or
testcase/tc (the specified operation works on Test Cases only).
REPOSITORY_DB_FILE
Specify the RGW Repository database file to operate on.
Optional arguments:
-h, --help
Show this help message and exit.
--no_backup, -nb
Disable Repository database backup. By default, backup is enabled.
The Repository database is backed up to a file named
requirements.n.db where "n" is an incrementing number starting
from 0.
Example This example removes all requirements data before the date "2017-05-
11 15:35:40":
$VECTORCAST_DIR/vpython.exe $VECTORCAST_
DIR/python/vector/apps/RGWUtility/rgw_clean.py --
at_date "2017-05-11 15:35:40" req requirements.db
This example removes all the test case data up to the current
date/time:
$VECTORCAST_DIR/vpython.exe $VECTORCAST_
DIR/python/vector/apps/RGWUtility/rgw_clean.py tc
requirements.db
rgw_report.py
Syntax rgw_report.py [-h][--from_date FROM_DATE] [--to_
date TO_DATE][--at_date AT_DATE][--action_type
Location $VECTORCAST_DIR/python/vector/apps/RGWUtility
Positional arguments:
REPORT_TYPE
Specify the report type. This can be one of:
events (an output of historical requirements or test case data
events), or
snapshot (an output of a snapshot of historical requirements or test
case data at any moment in time).
DATATYPE
Specify the data type to operate on. This can be one of:
requirement/req (the specified operation works on Requirements
only), or
testcase/tc (the specified operation works on Test Cases only).
REPOSITORY_DB_FILE
Specify the RGW Repository database file to operate on.
Optional arguments:
-h, --help
Show this help message and exit.
reset}
Restrict the specified operation to the specified action type, where:
all = all action types
import = requirement import
update = requirement update (re-import)
delete = requirement delete
export = test case data export
link = link test case to requirement
unlink = unlink test case from requirement
run = test case data execute result
reset = reset
If unspecified, then the default is 'all'.
Example This example queries all the Requirements import / update events in a
time range:
$VECTORCAST_DIR/vpython.exe $VECTORCAST_
DIR/python/vector/apps/RGWUtility/rgw_report.py -
fd "2017-05-11 15:33:20" -td "2017-05-11 15:38:00"
--output_file output.xml events req
requirements.db
additional information about exception background color for failing cells 299
occurrence 121 background color for partially passing cells 299
aggregate coverage report 334 background color for passing cells 298
All Values button 192, 196 background colors
allow stubs of Ada runtime 119 changing for coverage output 361
alphabetize parameter tree parameters 171 backgrounds
alphabetize test cases 57, 149 printing 75
alternate direct array indexing 120 basis path
animated control flow 347 edit script for 236
animation coverage type 352 viewing for subprogram 235, 342
animation play speed 357 basis path analysis 342
Any button basis path output
in List Values tab 174, 206 building test cases from 342
in Range Values tab 204 basis paths
in Scalar Values tab 201 do not automatically generate 236
Apex mutual imports 120 batch execute all 254
apply to array indices” 194, 202 batch file 129-131, 134, 483
apply to expanded array indices only” 194, 202 binder/linker log file 109
array constraints blank cells text 296
modifying 197 Branch coverage 319
array types breakpoints
entering multi-dimension indices 197 removing 351
546
Index: buffered coverage I/O – CLICAST
547
Index: clicast actuals to expected – circular references
548
Index: clicast actuals to expected – circular references
549
Index: clicast actuals to expected – circular references
VCAST_DISABLE_DIRECT_ARRAY_ 513
INDEXING 508
VCAST_GNAT_OBJECTS_PATH 112,
VCAST_DISPLAY_EMPTY_ 514
COVERAGE 287, 509
VCAST_GPR_TEMPLATE 514
VCAST_DISPLAY_FUNCTION_
VCAST_GPR_USES_NAMING 125, 514
COVERAGE 287, 509
VCAST_IGNORE_INCOMPLETE_
VCAST_DO_COMBINATION 284, 307,
TESTS 281, 285, 514
509
VCAST_IMPORT_FOREIGN_
VCAST_DONT_STUB_
OBJECTS 514
ANCESTORS 119, 509
VCAST_INSTRUMENT_
VCAST_EMPTY_TESTCASE_FAIL 509
ASSIGNMENTS 515
VCAST_ENHANCED_FOR_LOOP_
VCAST_INSTRUMENT_
COVERAGE 510
PARAMETERS 354, 515
VCAST_ENVIRONMENT_FILES 510
VCAST_MAX_COVERED_
VCAST_EXECUTE_WITH_ SUBPROGRAMS 356, 515
STDOUT 510
VCAST_MAX_MCDC_ABORT_
VCAST_EXPECTED_BEFORE_UUT_ INST 515
CALL 292, 307, 510
VCAST_MAX_MCDC_
VCAST_EXTENDED_EXCEPTION_ CONDITIONS 353, 516
INFORMATION 121, 510
VCAST_MAX_MCDC_
VCAST_FIELD_DOT_NOTATION 511 STATEMENTS 356, 516
550
Index: clicast actuals to expected – circular references
551
Index: clicast actuals to expected – circular references
VCAST_RPTS_TABLE_DATA_PASS_ VCAST_SUPPORT_TAGGED_
BGCOLOR 299, 526 INHERITANCE 530
VCAST_RPTS_TABLE_DATA_TEXT_ VCAST_SUPPRESS_COVERABLE_
ALIGNMENT 526 FUNCTIONS 530
VCAST_RPTS_TABLE_HEADING_ VCAST_SUPPRESS_DEBUG_
BGCOLOR 527 FLAG 121, 125, 530
VCAST_RPTS_TESTCASE_COLUMN_ VCAST_TARGET_DIRECT_ARRAY_
WIDTH 301, 527 INDEXING 121
552
Index: clicast actuals to expected – compound test cases
553
Index: configuration file – coverage
554
Index: coverage analysis editor – CSV Options panel
555
Index: CSV reports – data types
556
Index: database – ENVIRO.STUB_EXIT_USER_CODE
557
Index: ENVIRO.STUB_USER_CODE_FILE – Environment menu
558
Index: environment name – examples
559
Index: execute all command – files
execute all command 493 using when viewing text files 259
execute commands, CLICAST 254 fail if no expected return 284
execute option fail on unexpected exceptions 280
automatically clear test user code 280 fail on unexpected signals 284
fail if no expected return 284 File menu
executing compound test cases 251 Close 35, 38
executing test cases 19, 246 Close All 38
executing test cases with your debugger 255 Close Environment 127
execution status viewer 50 Delete Environment 138
exiting 22 Exit 22
exiting VectorCAST 22 New 92
expand all subprograms 331 new document 73
expanding arrays 189-191 Open 73, 127
expected results Print 75
range of values 469 Print Setup 74
unused” 250 Recent Environments 126-127
using ANY in list of values 470 Save 74
expected user code 404 Save All 74
expected user code comparisons Save As 74
width of 248 Set Working Directory 90, 127
expected values file version command 294
any (placeholder) 200-201, 209 files
defined 169, 173 .CFG 233
entering as a list 174 .vcast-qt 31, 57-58
entering as a range 173 AALINKER.LIS 109, 441
entering as a scalar 173 configuration 233
range of 199, 208 created by environment 107
set expected to actual 169, 258 environment_name.bat 129
export test scripts 210 environment_name.cvr 129
external browser environment_name.env 129, 138
setting up 56 environment_name.sh 129
using when viewing html files 259 environment_name.tst 129
external editor exporting scripts 210
scrolling to first line of function 54 GLOBALS.C 450
setting up 53 importing 385
using in user code 385 importing scripts 211
560
Index: filter – industry mode
561
Index: INIT test cases – MC/DC
562
Index: MDI window – messages
entering data directly into 170 Clicast failed - parameter inconsistency 446
<environment name> could not be Deleting files from environment build 442
opened 441 Determining basis paths 444
<environment> deleted successfully 444 Driver could not be invoked 441
<type mark> could not be found 442 Driver Program did not link 441
Abnormal Termination of Compile and Environment build but not compiled 441
Link 440
Environment build but not linked 441
after exporting coverage script 363
Environment build but run-time error
after importing coverage results 362 exists 442
Building Driver 439 Environment built but run time error
Building Harness for unit_name 439 exists 438
Building Min Mid Max test cases 445 Environment Built Successfully 439
Building test case script file 443 Environment data not expanded 445
Building test case script template 443 Environment Directory Creation Failed 440
Building the CLI script 443 Environment has not been compiled 445
Cannot Build Environment for Environment has unresolved link errors 445
Environment must be specified on command
unit name 441
line 446
563
Index: metrics report – multiunit whitebox
Getting size and range data 444 Subprogram must be specified on command
line 446
Harness has run-time errors 445
Subprogram name is invalid 446
Illegal numeric entry 442
Syntax Error in Unit
Index expression error 442
instrumentation data line <number> 446 unit name' 440
Instrumented Harness did not link 441 Test case must be specified on command
line 446
Internal Tool Error (Parser) - Write STR 440
Test case name is invalid 446
Invalid environment name 444
Test execution failed 442
Invalid test case name 443
Type not supported 443
Linking Environment 439
Updating aggregate coverage report 445
Linking Instrumented Harness 444
Updating coverage data 445
No change made to test data 442
Value out of range 442
No expected results exist 443
VECTORCAST_DIR Environment variable
No files specified 446
is not set 438
No results exist 443
metrics report 335
Null record 443
METRICS_TOTAL_LEVEL_DISPLAY 287
Parsing UUT body for whitebox
min-mid-max 153
conversion 439
modifications
Performing A Private Check-Out Of Source
Files 439 to source code and relinking 136, 143
Processing global objects 444 multi-dimensional arrays 197
Processing parameters 444 multi-unit whitebox
Processing unit name 439 using to manipulate private types 188
Processing UUT unit name 439 multiple value syntax 207
Processing whitebox data 439 multiunit whitebox 96, 120, 125
Rebuilding instrumented harness 445
564
Index: nested compound – options
565
Index: Options tab – options
environment user code in designated uut 123 real-time coverage I/O 352
566
Index: Options tab – raising exceptions from stubs
567
Index: range – reports
range removing
entering data as 173 breakpoints 351
range check 277 removing a custom tool 241
range data renaming
rebuilding 144 environment 127
range of values for expected results 469 renaming a test case 148
range of values for test creation 468 renaming test cases 149
range testing 205 Replace button
range values tab 204 in List tab 207
RANGE_CHECK 277 report complexity column width 301
ranges as expected results 205 report components 68
reading data from database layout template 70
user code example 396 report file 69
real-time coverage I/O 352 section file 70
rebuild environment 137 report coverage results column width 302
recompiling report customization 59
automatic 140 report date column width 301
create script 140 report format 299
instrumented code 140 report header 296, 299
run script 143 report notes column width 302
record types 178 report results column width 301
redirect standard output 279 report subprogram column width 300
redoing an edit action 76 report testcase column width 301
regression scripts report unit column width 300
creating 129 reports
integrating with ClearCaseTM 135 aggregate coverage report 334
post-processing automatically 133 controlling the test execution report 250
regression test 129 coverage summary 19
relinking creating 19
test harness 143 diagnostic 243
remember last working directory 90 execution history 19
Remove All button expand report objects 250
in List tab 207 expand report parameters 250
Remove button expanding information included 250
in List tab 207 full 263
metrics 335
568
Index: requirements – right-click menu
569
Index: run-time errors – smart stubs
570
Index: sort Metrics report by directory – tech support
571
Index: technical support – test cases
572
Index: Test Compile button – test requirements
573
Index: test script header template – toolbar
574
Index: Tools – user code
575
Index: User Code tab – VCAST_CUSTOM_REPORT_FORMAT
576
Index: VCAST_DETECT_UNUSED_EXPECTED_UC – VCAST_TESTCASE_FAIL_ON_NO_
VCAST_DETECT_UNUSED_EXPECTED_ VCAST_OLD_STYLE_MANAGEMENT_
UC 281 REPORT 293
VCAST_DISABLE_ADA_ARRAY_RANGE_ VCAST_PARENT_LIBRARY 448, 483
CHECKING 481 VCAST_RAVEN_HEAP_SIZE 483
VCAST_DISABLE_FILE_DELETE 481 VCAST_RAVEN_STDOUT 483
VCAST_DISABLE_TEST_MAINTENANCE 481 VCAST_RECURSION_DEPTH 483
VCAST_DISPLAY_EMPTY_COVERAGE 287 VCAST_REGRESSION_COMMAND 483
VCAST_DISPLAY_FUNCTION_ VCAST_RPTS_COMPLEXITY_COLUMN_
COVERAGE 287 WIDTH 301
VCAST_DO_COMBINATION 282, 305 VCAST_RPTS_COVERAGE_RESULT_
VCAST_DO_MCDC 320 COLUMN_WIDTH 302
VCAST_DONT_DO_MCDC 320 VCAST_RPTS_CUSTOM_CSS 299
VCAST_DONT_INSTRUMENT_END 323 VCAST_RPTS_DATE_COLUMN_WIDTH 301
VCAST_DONT_INSTRUMENT_START 323 VCAST_RPTS_EMPTY_DATA_STRING 296
VCAST_EXPECTED_BEFORE_UUT_ VCAST_RPTS_HEADER 296, 299
CALL 292, 307 VCAST_RPTS_NOTES_COLUMN_WIDTH 302
VCAST_FILE_VERSION_COMMAND 294 VCAST_RPTS_PRETTY_PRINT_HTML 483
VCAST_FLOAT_PRECISION 294, 303 VCAST_RPTS_RESULT_COLUMN_
VCAST_GNAT_HIE_DASH_E 481 WIDTH 301
VCAST_GNAT_LD_OPTIONS 482 VCAST_RPTS_SHOW_VERSION 293
VCAST_IGNORE_COMPILE_ERRORS 482 VCAST_RPTS_SUBPROGRAM_COLUMN_
VCAST_IGNORE_FILE_EXISTS_CHECK_ WIDTH 300
FOR_COVERABLE 482 VCAST_RPTS_TESTCASE_COLUMN_
VCAST_IGNORE_INCOMPLETE_TESTS 285 WIDTH 301
577
Index: VCAST_TESTCASE_FAIL_ON_NO_EXPECTED – wrap text in Notes
starting 21 opening 73
578